Hoping to dispel our skepticism, we set out to test some of these devices to see how helpful they are. Over the course of a month we stocked our partner Neohapsis labs, in Chicago, with offerings from Captus Networks Corp., Foundry Networks, Mazu Networks, Radware, Reactive Network Solutions and Top Layer Networks. And, of course, we had enough support hardware to add five degrees to the ambient temperature.
Next we formulated a testing game plan, including the use of various common DoS attacks (see "How We Tested Anti-DoS Devices" and "DoS Dossier"). Common attacks were chosen to test the analysis and mitigation capabilities of the devices, which were graded based on how much attack traffic they filtered/stopped -- and on how much legitimate baseline traffic was also filtered/ stopped in the process. The premise is simple: The ultimate mitigation is unplugging your Web server or stopping all traffic to it. Yes, you stopped the attack. Yes, you may have saved your server from meltdown. But you're also losing all your potential customers. That's not good, which is why devices that were able to stop the most attack traffic while having the smallest impact on legitimate traffic were graded higher than devices that blocked a lot of legitimate traffic while filtering the attack.
Methods of Stopping DoS Attacks
Before proceeding, we need to clarify the difference between a DoS attack and a DDoS (distributed DoS) attack. In the latter, many hosts act in a cooperative manner to perform a joint DoS attack. Thus, a DDoS attack is just a large-scale DoS attack. Because DoS attacks are often centered on the principle of overuse of limited resources, having multiple computers involved potentially makes overrunning targets easier.
If the attack is coming from a single source or even a small number of sources, blocking those sources is a simple matter. However, stopping a DoS attack is not always simple. Often, it's quite the opposite, because in some types of attacks, spoofing or otherwise faking the source address is possible. In addition, the spoofed source address may be random. The end result is that traffic coming from a single attacking host appears to be coming from hundreds, thousands, even millions of different hosts. Blocking these individual random spoofed hosts is futile since the attack will continue using other spoofed source addresses.
So What to Do?
The first method of stopping a DoS attack is simply to drop all traffic related to the target hosts. This is a good tactic for a nonessential protocol, like ICMP (Internet Control Message Protocol), but dropping TCP or UDP (User Datagram Protocol) can impact legitimate traffic, such as HTTP or DNS. However, denying all traffic does keep the attack traffic from impacting the target; thus, in some cases (like a SYN flood), this is better than nothing.
If an attack is originating from one or small number of true hosts, as opposed to being randomly spoofed, a device that tracks source IP addresses will be able to home in on the specific offenders and drop all traffic from those hosts. This will effectively block the attack, earning a perfect mark for this review. However, tracking every unique source IP address is quite a processing feat, requiring large amounts of memory. Therefore, a few of the devices cut corners by dividing the Internet into smaller, more manageable chunks. While this lets the devices track the general origin of an attack, blocking chunks of the Internet -- particularly if they are big chunks or user-dense areas, like cable-modem segments or America Online user proxies -- hurts legitimate traffic. However, this still can be an effective form of attack mitigation, and therefore could earn a decent grade.
The next mitigation method is the floodgate approach. This means the device drops most traffic but occasionally lets small amounts of traffic pass for a short period. That traffic is likely to consist of both legitimate and attack traffic. In this situation, the packet flow is reduced to smaller bursts, keeping the target system from being overloaded while also attempting to accommodate some legitimate traffic. The floodgate approach is particularly useful against SYN floods but is also effective against a few UDP protocols, like DNS.
That brings us to the last mitigation tactic: traffic analysis with selective filtering. In this case the device actually data-mines traffic characteristics to come up with a common feature to distinguish (and thus filter) attack traffic. Of course, this is easier said than done and depends on some feature being common and unique to attack traffic; examples would be a static TTL (time to live), sequence number, source port or IP ID. Fortunately, many publicly available off-the-shelf DoS attack tools do produce such phenomenon, so this approach does seem to be effective for the time being.
We admit we were skeptical that a device could mitigate certain types of DoS attacks automatically, particularly random-source address attacks. We were not alone in our skepticism: Arbor Networks and Asta Networks designed their products around the principle that you can't sufficiently automitigate attacks, believing that you need to have user decision involved. After seeing these devices in action, however, we found that they help significantly against a large chunk of known attacks. A few nasty and well-designed DoS attacks may slip past, though (for more on Asta's and Arbor's products, see "The Nonmitigators").
As with any tool, learning what the device can and can't do is essential to using it correctly and realizing where your vulnerabilities lie. Knowing what attacks each of these products can handle will let you use them effectively in your fight against the hordes of kiddies that have nothing better to do than throw large quantities of packets around on the Internet.
And the Winner is ...
The devices we tested are not necessarily equals. We have switches, load-balancers, firewalls and traffic analyzers in the mix. Some are meant to watch over carrier-class backbones; others, to protect a small cluster of hosts. Some devices' sole purpose is to handle DoS attacks, while for others this protection is a bonus to support or complement other capabilities. Some devices feature full packet- and historical-analysis GUIs, while others barely tell you than an attack happened.
Not all the devices performed well in comparison with one another, but then not all environments are equal. Our goal is to help you put our results in context for your situation; we looked at what types of attacks each device is capable of mitigating and how it goes about doing so. This is not a "which one is the best for everything" bake-off.
To make things even more complex, the quantitative results ended in a three-way tie. Therefore, we picked the winner based on our subjective judgment of the devices. The Radware FireProof 2.2 with SynApps software reigned supreme for hands-off attack mitigation. The deciding factors were price and the features provided for that price. In Radware's case we had gigabit routing, switching and firewall load-balancing.
The FireProof is a versatile, self-contained device that would be easy to deploy at your network's border or internally. Its sister device, the Web Server Director, also includes SynApps but offers Web server-centric (rather than firewall-centric) features.
That's not to say the FireProof is the perfect choice. It didn't catch all the attacks, nor did it mitigate them all perfectly. That's why we have to give an honorable mention to Mazu's TrafficMaster. The TrafficMaster has a data-analysis engine that we feel, given a little more maturation time, will be able to better tackle attack traffic without impacting your legitimate users. In fact, by print time the TrafficMaster should be sporting a few additions and changes, which will make it a worthwhile consideration.
We also feel that, as DoS attacks get more sophisticated, traffic analysis is the only method of stopping an attack without blocking legitimate users. TrafficMaster is also one of two devices we tested (the other being Reactive's FloodGuard) that can be deployed at multiple network ingress and egress points, allowing for a cooperative mitigation effort to protect your entire network.
Those of you looking for a firewall should consider Captus' CaptIO, whose TLIDS (Traffic-Limiting Intrusion-Detection System) engine bears the beginning of traffic-analysis features, with more surely to come. Plus, the device's firewalling capabilities mean you can add DoS mitigation to your network without increasing complexity, by using the CaptIO as both a firewall and a DoS protection device.
