Upcoming Events

Cloud Connect
Santa Clara
Feb 13-16, 2012

Cloud Connect brings together the entire cloud eco-system to better understand the transformation we're experiencing and promises to be the defining event of the cloud computing industry. Learn about the latest cloud technologies and platforms from thought leaders in Cloud Connect’s comprehensive conference.

Register Now!

F E A T U R E

Cisco Cures the Chicago Blues

November 12, 2001
By Brian Eirich and Greg Shipley

Interpreting Our Results

While most firewall-testing results are based on simple throughput numbers, our testing for the HA portion of this review was focused on measuring stateful-failover capabilities. Our HA-testing criteria adhered to a few core principles: Sessions had to be TCP; they had to be real sessions, consisting of complete ramp-ups and tear-downs, and repeatable; and our session counts had to be high in volume (it should be noted, however, that our 200,000-high-session-count test might have been harsher than most Web environments). Readers should look at these results as a baseline for guidance, not as a definitive conclusion. For example, if your environment will never experience 200,000 concurrent connections, the high-session-count tests, while interesting, may be less relevant to your decision-making.

Real-world conditions vary from our tests in a number of ways. Most real-world Web environments are going to have Web pages of varying sizes. For our tests we chose to limit our page size to 1 Kb. This allowed us to create a large number of concurrent connections without exceeding 50-Mbps of throughput. Because we stayed away from using HTTP transactions with TCP persistence/keep-alives set, it is safe to assume that one of our Web transactions is equal to one TCP connection.

We performed four different tests: a 1,000-concurrent-connections test, a 50,000-concurrent-connections test, a 100,000-concurrent-connections test and a 200,000-concurrent-connections test. We ran each test at least three times to include a baseline, a simple failover test, and a failover test with four extra sessions injected: two SSH and two FTP. Those four extra sessions were watched alongside the TCP/HTTP sessions being monitored by the Caw Networks devices). All devices were completely reset before each test was performed.

Our results were generated from the Caw WebAvalanche and WebReflector reports, which give volumes of detailed information (see "Caw's WebReflector Makes Load Testing a Cakewalk"). However, we focused on some key components for our results chart to simplify the digestive effort. We have also provided a summary chart (above). In addition, it should be noted that during our tests some of the firewall products failed over unexpectedly while under load. Other firewalls simply crumbled under the heavy-connection tests. Obviously, we were unable to test the HA functionality of any firewall that was unstable during high-session rates. We noted this in the charts as well.