Web Load-Balancers: How We Tested
In an effort to quantify Web load-balancer performance, we created test environments similar to both a corporate intranet and the Internet. To accomplish this, we placed an ATM-based network between our client and server systems. We chose ATM for its ability to limit bandwidth during portions of our testing. To emulate Internet traffic, bandwidth was constrained to 45 Mbps, the speed of a T3 connection. For intranet testing, we bypassed our ATM network and used Fast Ethernet to access our servers.

Our ATM network consisted of two FORE Systems ES-3810 edge switches and one ASX-200BX ATM switch. On the server side, a 3Com SuperStack II Switch 3300 Fast Ethernet switch was attached to the ES-3810 via a Fast Ethernet port. Each server was connected to the SuperStack. On the client side, a Lucent Technologies P550 Cajun Fast Ethernet Switch was placed between the ES-3810 and the client cluster.

Our server farm consisted of eight 200-MHz Pentium Pro systems configured with 64 MB of RAM. An additional 300-MHz Pentium II was used to test Resonate Central Dispatch. This system served as a noncontent-bearing server and worked as a part of the load-balancing system, rather than a back-end server. Our client systems consisted of eight rack-mounted Cubix ERS-Fault Tolerant II units, each of which contained seven 200-MHz Pentium MMX PCs with Fast Ethernet NICs; we had a total of 56 client machines. Each client then ran multiple instances of the testing scripts in order to emulate a total of 100 client machines. To take advantage of the gigabit ports on the Alteon ACEswitch 180s, we inserted a gigabit blade into the Cajun P550 switch, which provided a gigabit connection to our server cluster. Servers were also connected to Alteon's ACEswitch 180 using a Cisco Catalyst 5000 with a gigabit blade.

To measure load-balancing performance, we ran tests in two different scenarios. Using the custom client scripts developed in-house, each client randomly retrieved Web content from a pool of URLs for 10 minutes. The first test was performed with Fast Ethernet bandwidth. The same scripts were then run again using the new network configuration limiting the bandwidth to T3 speeds (45 Mbps). Each load-balancer's redundancy/failover features were also tested during a separate test run. Numerous failures were simulated on the primary load-balancer, including removed network cables, power loss to the unit and removed failover cables.

Performance presented itself as a tricky issue; products were all over the map in terms of throughput and efficiency. It's important to clarify that for Internet use, many sites will be limited by the amount of access bandwidth. In evaluating the performance numbers, saturating the 45-Mbps bandwidth test was good for a passing grade, while acing our Fast Ethernet testing meant above-average grades. Server failover worked flawlessly for each product review. Unless otherwise specified we were satisfied with the failover mechanism in each product.