Because video quality is subjective and every use has different requirements, we decided to take a more logical path with our testing. We looked for obvious loss in quality, from dropped frames, pixilation or decoded pixels not being updated when they should be. Latency and dropped packets are the biggest problem for broadcast-quality video, so all our tests were designed to see how the competitors handled these situations.

We set up the tests in our Dallas lab over two simulated networks. We created an Ethernet network with an Empirix Hammer PacketSphere in the middle to test the Amnis, Cisco, GDC, Minerva and Optibase boxes. We also tested the GDC MAC 500 over an ATM network; to do this, we placed an Adtech AX4000 between the encoder and decoder on a DS-3 ATM network.

At the vendors' request, we used a DVD for our source material -- though we would have preferred an analog source to eliminate any confusion between the original encoding quality of the DVD and the encoding quality of the encoder being tested. We ran Chapter 29 of The Matrix as our source DVD because just about every pixel changes in every frame. That scene also has a wide range of audio with deep bass, which let us test the audio encoding. The video was fed via an S-Video connection to all the units except the Amnis NAC 3000, for which a composite was used. On the decode side, we fed the video back to a Sony studio monitor via an S-Video connection. Audio was routed to the AUX inputs of a JVC desktop stereo.

To destroy the network, we conducted four different tests with our Empirix Hammer PacketSphere. We tested packet loss of 1 percent to 5 percent, in 1 percent increments. We duplicated 5 percent, 10 percent and 20 percent of the packets on the network in an attempt to confuse the decoders; we succeeded there. Packet reordering occurred between 5 percent and 40 percent, in 5 percent increments. Latency deviation was the big test and was set up in a Gaussian distribution of the following ranges: 20 to 25 milliseconds, 20 to 30 ms, 20 to 40 ms, 20 to 80 ms, 20 to 160 ms, 20 to 320 ms and 20 to 500 ms. Only the Amnis NAC 4000 unit could handle deviation of more than 5 ms. Latency deviation can occur when traffic on the network is overloaded or when the network reroutes packets to their destinations. Although a good network may have a deviation of only 1 ms, we felt it was important to test in the worst-case scenario.