Soft Message System

Really early work on the soft message system.

The soft message system is a project that occupied my spare time from November 2009 to November 2010. The premise of the project was to some how in near real time convey the notion of a hug over long distances. I really believe that we have distance based communications capability way beyond what is currently seen in the main stream. There has been a real lack of innovation in distance communication, and this was my attempt to contribute something original. In the end, I was not able to finish, but if you'll humor me; you can learn about a radical project probably ahead of its time and too complex for an individual to market successfully.

So let me explain what the soft message system is. The soft message system is comprised of a soft USB device that contains several pressure sensors, a client application on a computer, and a central server. The computer application and device create messages that are transferred round the world to other people that are paired with your soft message system. This allows you to transfer audio messages with associated pressure data to anywhere in the world, where it is re-created in a game-like timed replay scenario.

Early software demonstrating challenge points, 3d model, and file transfer.

One of the major issues with real time communications is that the US has a terrible broadband infrastructure in rural areas. My family happens to be located in a rurual area with extremely limited broadband capabilities. The soft message system was engineered based on a per to per based system coupled with queuing.

In the initial stages I did a ton of research concerning NAT traversal and peer to peer data transfer. Basically I was researching how to allow seamless network communication without the need for users to manually configure their home Internet routers. Nearly seamless network configuration and utilization of peer to peer technologies has contributed greatly to Skype's success.

Slightly more advanced software in July 2010. Full 3d model of the physical object has been removed to increase performance. I think I am in the middle of a recording scenario.

I became extremely fascinated by the XMPP protocol and its associated extensions and based the soft message system on XMPP. You've probably never heard of XMPP, but it is a giant in modern real time communications (in 2011 anyway). Google voice, chat and Facebook's IM service are all based on XMPP. I chose the protocol because it's servers automatically setup many of the NAT traversal technologies needed for effective peer to peer communication.

The soft message system records a users interaction with a pillow-like soft pressure sensor along with an audio message. Those two communication mechanisims are fused and sent to remote clients. The remote clients enter a playback game like scenario where they must manipulate their soft message sensor in the same way the sending client did when recording the original message. If receivers are not able to mimic the received message, then they are not able to hear the entire audio message.

Originally I figured I could some how utilize FSR based pressure sensors and made a pre-tentioning rig for them out of a tennis ball and screws. These sensors were terrible for the kind of interaction I needed, so I was able to learn about stitching together custom soft pressure sensors that proved to be a lot more reliable.

Met Research Objectives

1. I was able to learn a great deal about writing server type multi-threaded software in Java.
2. I was able to learn about the WIX Windows Installer framework extensively.
3. I was able to learn about cross platform Java development including Linux and Mac OSX development and how to disguise cross platform applications as native apps.
4. I have become a total XMPP fan. I hope I can utilize the foundation of code I created for transferring files in real time utilizing the protocol in future products.
5. This wasn't a planned objective, but I learned how to stitch together my own custom soft pressure sensors thanks to Plusea at Instructables
6. Was able to learn a decent chunk about cryptography. Specifically concerning base 64 encoding and how to utilize HMACs for network communication verification in Flash and Java.

Failed Objectives

1. Finishing a prototype and partner pair to send to close friends and family.
2. This is unfortunately a really complex project that would require marketing of a large firm to succeed. I didn't realize this when I started development.
3. I failed at the user experience during the playback process. Quite often its way too hard to match the playback and hearing the sent message is extremely difficult.
4. I did a poor job at serialization/de-serialization of commands from the java server to the Flash client. I invented a protocol instead of using something like JSON, or AMF which I could have obtained from several open source projects.

I really hope I am able to utilize the foundations I have learned from this project. Recently I have used some of the code in my work projects, but it only encompasses a vast plethora of P2P based awesomeness. It's hard for me to describe how much I enjoy the limit pushing paradigms of multi-threaded and server programming.