|
Video quality has always been a subjective matter. What one person likes, another may not. To counter this, we came up with a completely blind test for our quality. We created screen shots of the same scene from each player at different encoding rates: 56, 128, 256, 384 and 512 Kbps. We then cropped the shots to exclude the player and presented them to our volunteer judges. With each encoder, we tried several different combinations of quality for each bit rate and presented all the options to 24 judges, based in homes and offices all over the country. To achieve our final numbers for a given bandwidth/player combination, we calculated the top scores from each vendor, to give each one the best possible result.
We chose a cult movie classic, The Adventures of Buckaroo Banzai Across the Eighth Dimension, on DVD, as our source. This movie has enough action and static areas to put the encoders through their paces. Our encoding computers had Intel dual-processor motherboards with 800-MHz Pentium IIIs, running Windows 2000 Professional. Microsoft and RealNetworks recommended we use the ViewCast Corp. Osprey 500 video capture card, while Sorenson recommended the Winnov Videum card for its tests. Both cards performed well. The Osprey is much more expensive than the Videum but provides support for digital video and audio input in addition to the normal analog inputs. We had intended to use the Osprey card for all our tests but found that Sorenson's Broadcaster had problems with some, so we used the Videum for those tests.
For the Osprey card, we provided a FireWire (IEEE 1394) digital video signal from our DVD player through a signal converter. This allowed our video to maintain its digital form the entire way through the process. For the Videum card, we connected our DVD player via an S-Video connection and unbalanced stereo audio.
Our servers were set up on the same Pentium III-based dual-processor hardware, with the only difference being the operating system. Apple's and RealNetworks' servers were loaded on a computer running FreeBSD, while Microsoft's server was tested on a Windows 2000 Server box, as that is the only platform Media Services runs on. For clients, we loaded all three players on computers running Windows 98, Windows 2000, Macintosh OS 9.2 and OS X. Bandwidth usage was monitored by EtherPeek software from WildPackets.
For our tests on the products from Amnis and VBrick, we used the same DVD source connected directly to each vendor's box. We then routed the IP video through an Empirix PacketSphere to simulate network degradation. The video then arrived at a computer running Windows 2000, where it was displayed in the player. Because we were unable to take screen shots of the MPEG video, for quality tests we set up two identical monitors side by side and compared the video coming from both systems simultaneously.
In creating our report card, we could not do a strict apples-to-apples comparison. Video quality is most heavily weighted in both cases, but other factors deserve different amounts of attention. Thus, we considered quality 30 percent of the score in the hardware testing but 40 percent in the software testing.
|
RSS
