Gigabit Test Setup at the Labs of Neal Nelson and Associates (1 (312) 755-1000)

In our test bed configuration, the first LAN adapter on each machine was reserved for task control and intermachine communication. We referred to this as the "gray" LAN; this LAN was physically isolated with its own hubs and switches. A second and third LAN adapter in each machine (the "black" and "green" adapters) was connected to the gigabit switches under test.

The gigabit switches were configured as two groups of 48 ports with a gigabit link connecting them. Half of the black LAN adapter cables were connected to the "left" group of 10/100 ports and the other half were connected to the "right" group. Similarly, half of the green LAN connections were connected to the left group and the other half to the right group.

All of the LAN adapters were configured to run full duplex at 100 Mbps. The theoretical maximum bandwidth of this configuration would be approximately 4.8 Gbps in each direction. Since the gigabit link should support 1 Gbps in each direction, it seemed likely that this configuration would stress the gigabit link enough for us to measure some differences between gigabit devices.

VLANs

The two virtual LAN configurations we used were selected to allow testing of both Layer 2 and Layer 3 traffic.

For Layer 2 tests, all black LAN cables were connected on the same class B subnet and all green adapter cables were assigned to a different class B subnet. For Layer 3 tests, every LAN adapter was given a unique class B subnet. During testing, traffic was controlled so that communication would always stay within a color group. In the dual-VLAN configuration--where all adapters were talking to the other nodes on the same subnet--no routing function was required; all addressing could be satisfied by Layer 2 of the TCP/IP protocol stack. On the other hand, in the 96-VLAN configuration, every communication between any two ports crossed from one subnet to another. Thus, routing activities were required at the TCP/IP Layer 3 level.

Traffic Characteristics

The vast majority TCP/IP traffic on the Internet is generated by telnet, FTP and Web applications.

Telnet sessions normally generate a large number of very small messages. These sessions usually are configured so that each typed character echoes back from the destination before it appears in the telnet window. Since most users type slowly, this means each typed character results in two very small messages, one in each direction. Each of these messages will be the minimum message size supported by TCP/IP and will contain a single character of user data.

FTP sessions frequently generate a large number of fairly large messages. Users normally use FTP to transfer files between machines, and during the file transfer phase, the ftp program will be sending data at the largest message size supported by the network (about 1,500 bytes for Ethernet TCP/IP).

Web sessions create an interesting mix of short and long messages. Typing and mouse activity is processed by the local browser client and does not reach the network. When a user clicks on a URL, a short message is sent over the Internet. In response to this message, an HTML page may be displayed that frequently will include some large image files. The common Web traffic sequence is a small request followed by several replies that include media-sized HTML pages with large images.

We included all three of these traffic types in Network Computing's test.

Test Scenarios

We created three test scenarios for these three traffic types.

The telnet test typed a sequence of 49 characters. After a character was typed, the sending computer would wait until that character was echoed back from the remote end before the next character would be typed. When the entire sequence of 49 characters had been successfully typed and echoed, the test would loop and repeat this process.

The FTP test executed a loop that performed a "put" of a 4-MB file followed by a "get" of a 4-MB file. All users on a given machine read from the same 4-MB file, so the file always remained in the computer's disk cache memory. For both the "put" and "get," the destination was specified as /dev/null. This ensured that the ftp transfers never did any physical disk I/O, which in turn prevented disk I/O from becoming a limiting factor in the test.

The Web test displayed a series of Web pages where each page contained a small amount of HTML text (less than 1,000 bytes) and three GIF images of approximately 200 KB each. A small set of images was used on a rotating basis to ensure that they would also reside in the disk cache during the test and prevent disk reads from limiting the traffic presented to the LAN.

As an additional traffic type, we created a "mix" in which approximately 40 percent of the active sessions performed the Web activity, 40 percent were telnet users and the remaining 20 percent were ftp tasks.

User Load Levels

One of the goals of the test was to increase the network traffic in steps and locate a point where the gigabit link became a bottleneck. Another goal was to determine a reasonable capacity for a high-speed data network using gigabit links. We elected to run a sequence of 12, 24, 48, 96, 192 and 384 active user sessions for each measurement type. We felt that these six points would include a portion where data was moving at "wire speed" and another portion where some component became the bottleneck. It seemed possible that we could approximately locate the transition point (the "knee in the curve") where degradation began.

Measurement Control

An additional factor that we felt we must control in this test was the possibility that we would inadvertently measure some factor other than the gigabit link speed. If the test-bed computer or the speed of the local interfaces on the gigabit boxes became a bottleneck before the gigabit link did, then we might accurately measure degradation but attribute it to the wrong component.

Our solution was to run identical workload sequences with three different combinations of source and destination data points. In one case, the source and destination pairs were always contained with a group. That is, data requests from the left-side ports would be directed to other left-side ports, and likewise for the righthand side. In a second case, every request passed through the gigabit link. That is, all requests from left-side ports were directed at right-side addresses and vice versa. Finally, a third case was tested where half of the users were confined to the local side, while the other half crossed the gigabit link to the opposite side.

All of the devices tested had very high bandwidth capacity within the chassis, so the tests where all traffic was confined to a side would establish a baseline measurement. These numbers would define a "no degradation" condition. The measurements where all traffic passed through the gigabit link could be compared to the first set, and the difference should be attributed to the gigabit link. The third set was a control set because those users passing between the sides should have adequate bandwidth, unless the local traffic with a side interfered in some way.