How We Tested Firewall Performance
We used Chariot testing software from Ganymede to blast HTTP traffic patterns through each firewall. We ran the Chariot Endpoint software on eight P6s. Four P6s were plugged into each of two Cisco Systems' 100-Mbps switches running at full-duplex. The firewall was connected between the switches. The traffic was symmetric with two clients and two servers emulated on each side. For our baseline, we connected both switches via a crossover cable.

We ran two Chariot consoles, one on each switch, on two additional machines. The consoles synchronize the machines, tell them what test scripts to run, and summarize the results. Two consoles were used because the Chariot software otherwise became confused when NAT (Network Address Translation) was turned on, as it apparently tried to reconcile the actual addresses in the IP header with the addresses of the machines it was monitoring.

We ran into problems during testing that illustrated the differences between proxy-style firewalls and stateful inspection-style firewalls. In fact, we were not able to run the test through the AXENT and CyberGuard HTTP proxies because the Chariot software embeds a 4-byte header in the data portion of every packet for timing and synchronization purposes. In this case, it appeared immediately before the get command. Since the HTTP protocol resides in the data portion, this header made the HTTP request nonstandard. A proxy firewall runs a process very similar to the process on a Web server of listening for incoming HTTP requests. The HTTP proxy does not know what to do with an invalid HTTP request, any more than a Web server would, so it discards the packet.

When we sent the same HTTP request with the imbedded header to a real Web server, it rejected the packet as well. This would make the proxy's efforts appear to be redundant. Vendors of proxy-based firewalls will claim that because they write their HTTP process with security as such a high priority, it is less likely that a harmful packet will even arrive at an actual Web server behind the firewall.

For some reason, the SecureZone firewall passed the request, even though it is a proxy-style firewall. All of the nonproxy firewalls also passed the request. If you are considering a proxy server, consider exactly what it protects your Web server from, and decide if it is worth decreased performance and increased complexity.

We wound up pushing the traffic through a generic proxy on the Raptor. A generic proxy doesn't actually run the traffic through a specific application. CyberGuard's firewall had the option of using "packet filtering" instead of proxying, which also sidesteps the HTTP process. The difference between the two is that Raptor's generic proxy will establish a separate TCP connection for every TCP session through the firewall, while packet filtering on the CyberGuard will pass the original TCP packet. We also ran the test through Raptor's telnet proxy as telnet does not care about syntax. For some reason, this performed better than the generic proxy.

We also tested the products with NAT. We configured statically mapped NAT mappings and forced two pairs of the Chariot Endpoints to use the translated addresses. This forced the IP headers to be rewritten for all of this traffic.

The Chariot software was very helpful for running the test and reporting on the results. It is unfortunate that it relies on a header in the beginning of the data portion of the packet. Though this is obviously necessary to maintain accuracy, it should not be terribly difficult to find a place to embed the header so that it still looks like valid HTTP syntax.