Network Computing is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Streaming Media Servers: Page 3 of 15

 

We created a quality survey to rate the video clarity, color, text rendering and overall playback smoothness. Nine Network Computing editors and contributors took the test. To make sure the test was fair, we set up the clients next to each other, so the participants could see all the media players at the same time. Each video was embedded into a blank Web page with a black background, and we hid the video controllers.

We ran all four players simultaneously and let the participants view any video as many times as they wanted. Apple's and RealNetworks' logos appeared for a few seconds during buffering, but we blocked the screen with a poster board during that period to keep the test blind. The first 14 seconds of the video was a stationary image, which gave us enough time to start up the streams, remove the poster board blockers and get out of the way. Participants graded the streams on a scale from 1 to 10 at each bit rate. The raw score survey results are available online here.

As we expected, the average score increased along with the video bit rate. Dropping from 1 Mbps to 768 Kbps didn't result in a big drop in quality, but going down to 256 Kbps did. The average score across all vendors at 1 Mbps was 7.4 out of 10 points. This dropped to 6.9 for the 768-Kbps test but plunged to 4.5 for 256 Kbps.

We were surprised by Darwin's poor quality showing at high bit rates. The frame rate was noticeably lower than the other competitors and produced a jerky video. This was less apparent in the 256-Kbps stream, and there was no jerky motion when viewing the video on the streaming server's console. Apple has released an update to QuickTime that touted improved H.264 playback, but it came out after our tests. Darwin's saving grace in our quality survey was its superior sharpness, vibrant color and excellent text rendering. Other vendors' players produced a slightly fuzzier, softly out-of-focus look, especially at lower bit rates. Our testers rated Flash's picture quality the best. The moving images were smooth and crisp, though small text didn't encode particularly well and was somewhat hard to read.

Our video streams occasionally dropped or stopped for 10 to 30 seconds during playback. There was no discernable or reproducible pattern to these performance glitches, and in some test runs, no videos dropped at all. Since the dropped streams were so sporadic, we instructed our testers to ignore them when answering the quality survey, and we considered stream stability as a separate factor in the overall quality grade. Darwin Streaming Server scored the best in stream stability; it never lost or rebuffered a stream during any of our test runs. All the other products did, even on a 10-Mbps network with no packet loss and 10-ms round-trip latency. This occurred rarely, however, at 10 Mbps, and showed up more predominately in the simulated broadband-connection test. Even at 2 percent packet loss, which is significant, Darwin was playable with minimal artifacts and no rebuffering. Although the servers offer some control over buffering time, the end user chiefly controls this setting. Flash is the exception, as the programmer dictates the applet's features.