But, this year, there is a very big story hovering over the horizon--namely, the looming foreshadow of gigabit Ethernet and its influenc e on the future of ATM. This, however, is not a testing issue--except insofar as it tests the fortitude of those venture capitalists who put all that money into ATM companies.

The data published here reflects the cumulative results of the last two years of free public testing in the lab, combined with the results from the same tests run if the vendor has rented the lab (if the release of the data was OK'd by the vendor), plus any private tests run for companies like Strategic Networks Consulting Inc. (SNCI). Results have been removed in cases where products have been discontinued. Devices from vendors that have been bought out are listed under the new parent company's name. Test results from new versions of products replace any old results.

We did do a series of ATM switch tests in conjunction with SNCI, but the results of those tests are not included here because the meth odology of testing ATM switches and the format of the results are quite different due to the different nature of ATM switch archit ecture. The results of those tests can be found on the SNCI Web page (www.snci.com).

Another Barking Dog
This year, in addition to our switch and router test results, we include test results from almost a dozen products with ATM uplinks and they show very good performance. ATM at 155 Mbps seems to have finally arrived. (Just in time to be confronted with Ethernet at gigabit speeds.)

· Ethernet Switches. As far as switches go, the results are not all that different from last year's--perfection is a basic requirement. Just about all 10-Mbps Ethernet switches can run at wire speed for all of the ports they can support. I would expect this to continue to be the case, not because users need to be able to run at more than 14,000 packets per second per port (your network has died and gone to hell if you need this level of performance), but because if a device is not operating at wire speed, competitors will trash the "slower" device.

The 100-Mbps Ethernet story with swi tches is a little more varied. Not all the products operate at full wire speed, even if they are all vastly faster than any rational real-world network would be.

· Layer 3 Switches. We've lumped all the router-like products, even though some marketing people would rather call some of the products Layer 3 switches rather than routers.

· Routers. The router story has been made more interesting by the advent of Layer 3 switches. This new breed of devices, whether the latest technology or merely revised marketing, tends to offer very high performance for significantly reduced cost (if what they tell me is correct). In general, I try to stay away from analyzing costs for the devices we test because it can be very hard to arrange like-to-like comparisons. The router versus switch war seems to be quieting down, now that many vendors that used to make only switches are starting to add routing functionality--a good thing, in my mind.

The Future

The year ahead in the network technology world is going to be very interesting indeed. 155-Mbps ATM products seem to have finally gotten solid but so have 100-Mbps Ethernet switches. Because 100-Mbps Ethernet is just about the same speed as 155-Mbps ATM, when ATM is used as a backbone and you factor in the cell header overhead and frame packing inefficiency, the performance advantage of ATM is not all that clear. In addition, the ever-clearer understanding that ATM to the desktop is not likely to have any future because Ethernet dominates in this area means ATM's edge on the quality-of-service (QoS) front is irrelevant. That's because ATM will not be end-to-end, so ATM QoS controls cannot be end-to-end. Actually, this gloomy prospect for local backbone ATM technology may prompt an increase in the number of ATM devices we'll see in the lab over the next year as vendors make a last-gasp attempt to show their products are worth investing in.

Meanwhile, it looks like there will be a whole pile of gigabit Ethernet products showing up over the next few months. The advent of "simple" (whatever that means) gigabit Ethernet is going to have a big impact on the yet-to-be-proven 622-Mbps ATM and the yet-to-be-seen 1.2-Gbps ATM, especially when gigabit-speed Ethernet is starting out at half the price per interface port of the slower 622-Mbps ATM. All in all, the prospects for campus-level ATM don't look too good. WAN ATM still seems like a good bet, largely because of the level of investment the telcos have put into ATM technology.

So, it looks like our lab may need to invest more time and effort in the Ethernet arena than in the harder to test ATM arena. There are worse fates.

Scott Bradner can be reached at sob@harvard.edu.

To download an Adobe Acrobat .pdf format version of the complete Bradner Report Charts, including Latency Testing, click here.

Testing Throughput and Packet Loss Rate All tests used in the Harvard Network Device Test Lab are based on work by The Internet Engineering Task Force (IETF) Benchmarking Methodology Working Group (BMWG), which has published two RFCs (see www. ietf.org). RFC 1242 defines testing terminology; RFC 1944, testing methodology. All testing is done using special-purpose standalone testers to ensure the most reliable results. The scripts that drive the testers are available on the lab's Web site at ndtl.harvard.edu/ndtl. Because of space limitations, we are reporting here only the results of our tests with 64-byte packets, but all devic es were tested with a full range of packet sizes. We also performed latency tests, which can be seen online at www.NetworkComputing.com/815/815f1.html. Throughput The throughput test determines the highest rate at which a router or switch can receive, process and forward packets without loss. This value is important because a pause of up to a few seconds may occur when a packet is lost from a data stream: The application--realizing data was lost--must retransmit the missing data. The test is performed by having the tester send a 30-second burst of traffic through the device at half the rate theoretically possible for the given test conditions. The number of sent packets is then compared to the number received. If all were received, the data rate is increased and the trial is rerun. If all were not received, the rate is decreased and the trial is rerun. This process repeats until a rate is found at which all offered packets are forwarded, but where a rate of even one more packet per second (pps) would result in a packet being lost. The Ethernet throughput tests were performed using Netcom Systems' SmartBits or FORE Systems' PowerBits. Packet-Loss Rate The packet-loss rate test characterizes the performance of a router over the full range of possible traffic loads. The test begins with a 30-second trial of traffic sent at the maximum rate theoretically possible for a packet size and a media through the device under test. The number of packets received by the tester is subtracted from the number sent to yield the loss rate. The rate at which packets are forwarded also is measured. Due to space considerations, we've pared this information to include only the maximum forwarding rate observed during the packet-loss rate test. This forwarding rate is independent of the amount of packet loss. All raw data developed during this test, along with graphs of the results, are on the lab's Web site. The maximum theoretical data rate depends on the packet size and type of media in use . For exampl e, the maximum theoretical rate for the smallest legal packets on 10-Mbps Ethernet (each 64 bytes long) is 14,880 pps. The maximum theoretical rate for the largest legal Ethernet packets (1,518 bytes long) is 812 pps. The Ethernet packet-loss rate tests reported here were performed using SmartBits and PowerBits.

Updated July 31, 1997