The stability tests subjected the firewalls to traffic from several protocol randomizer and mutation tools at a maximum of 350Mbps and 60,000 packets per second, NSS reported in "Network Firewall 2011 Comparative Test Results." Appliances that received failing grades crashed and no longer allowed traffic through. The Fortinet appliance rebooted after it failed; Fortinet supplied a patch upgrade, but the box crashed again in subsequent testing and needed to be manually restarted.
"These are conditions that should be tested for," says Rick Moy, NSS president and CEO. "They weren't very stable when exposed to attacks." (The report recommends that users running FortiOS 4.0 MR2 Patch 1, Build 279, upgrade.)
The firewall appliances tested were the CheckPoint Power-1 11065; Cisco ASA 5585-40; Fortinet Fortigate 3950B; Juniper SRX 5800; Palo Alto Networks PA-4020; and SonicWall NSA E8500. All the firewalls passed two other stability tests: blocking and alerting under a constant stream of attacks over an extended period of time and continuing to pass most or all legitimate traffic under those conditions.
Only the CheckPoint system thwarted the TCP Split Handshake Spoof, which targets a client machine, sending an extra crafted packet and instructing the target to initiate a session with the attacker, which gains what appears to be normal access as a trusted internal device.