Why Flash Will Never Be a Good Telepresence Medium With The Current Features

One of the small reasons I originally got into Flash programming was because I figured with such a large install base and great media capabilities; Flash may have been an area people were overlooking for telepresence and tele-operated control of robots. After becoming intimately familiar with the video capabilities of Flash, I believe it will never be suitable as a replacement for a tool like Skype.

I attest that I am capable of making this statement via the following. I work with Flash and other programming languages every day in creating interactive museum exhibits and have been for the past two years. I have come to my conclusion after spending a few years working with Red5 and recently making a video recording kiosk using Red5 and Flash. I am also independently working with Java on a command based socket server that deals with a lot of low level network transfer and media manipulation.

Now that I have explained what qualifies me to take this stance, I will get into the details where Flash is lacking.

Old and Incontrollable Video Encoders

Flash can currently only encode video using the On2 VP6 and Sorensen Screen? Im not even sure. I have never been able to figure out how to manually switch the encoders because there is such sparse documentation on it! These are ancient technology when put up against modern encoders and decoders like X264 and H264. The amount of network traffic and unpredictability of the flash player when using these encoders is unpredictable. You're just supposed to be happy with it automatically doing whatever it wants to. The Netstream, Netconnection, and Camera classes provide no control of the actual encoders. This kind of unpredictability is not acceptable from a professional programming standpoint when trying to meet clients expectations of rich media and responsive applications.

Its impossible to control the network traffic and internal buffer that the Flash Player generates when it is encoding video.

When doing network transport programming in Java and C++ you have the ability to pretty closely control the flow of data in and out of your applications by using buffers and sockets which all have a flush mechanism to allow you to immediately dump all data to the network. This is not the case with Flash Player and its encoders. They are abstracted into the Netstream and Netconnection classes and you are unable to obtain manual control of them. When you are publishing a live video stream from Flash to a media server like Flash Media Server or Red5 you will see insane and completely incontrollable fluctuations in the Netstream's bufferLength depending on the situations with the video you are encoding. For example, if there is not much motion, then the flash players buffer may stay relatively low, as soon as you move, the buffer may jump. Are you in a dark room? The buffer will probably stay pretty low. As soon as the light conditions change, look at the buffer jump into infinity. I understand the challenges of encoding quality video in near real time, but the lack of control over this process is unacceptable when trying to create good experiences for end users.

There is no way to control how fast the Flash Players buffer empties to a media server

In order to actually record flv files to the Flash Media Server or Red5, you have to wait until the Netstream buffer empties. Many people new to video recording in Flash will close their NetStreams as soon as they finish recording. They end up with broken flv files that skip all over mysteriously. The key is that you have to wait for all the recorded video and audio data to get to the media server. The way you do this is to monitor the Netstreams bufferLength until it reaches zero. Its too bad that the rate at which this happens is a complete mystery with the current Flash/Flex/Air API's providing no way to control this. Even running all on a local machine I have been unable to quickly transport tcp packets in the buffer from the Flash player to Flash Media Server or Several Versions of Red5 .7,.8. and .9.

No Thread Support

When programming in Actionscript there is no way to control threads that I am aware of. If you search around you will find articles about certain components being independently multi-threaded, but this doesn't give programmers the ability to launch other threads. When doing things like encoding video and audio, this can be a huge advantage.

When working with other programming languages like Java and its old JMF framework to record video and audio, its no fuss. Things just work. I can launch threads to quickly take care of tasks. I don't have to program around bad garbage collection or mysterious network transport classes. This may be due to the unique marriage that Flash programming has with user interface elements; but it just seems like you can do more with less in other languages when non-user interface tasks are considered.