Upcoming Events

Cloud Connect
Santa Clara
Feb 13-16, 2012

Cloud Connect brings together the entire cloud eco-system to better understand the transformation we're experiencing and promises to be the defining event of the cloud computing industry. Learn about the latest cloud technologies and platforms from thought leaders in Cloud Connect’s comprehensive conference.

Register Now!


	F E A T U R E

Intrusion Detection, Take Two November 15, 1999 Behind the Numbers How We Tested Intrusion Detection Focusing our tests on sensitivity to hostile activity, reporting capabilities and packet reassembly, we gauged how fierce an attack these systems could handle. By Greg Shipley It's not easy to test intrusion-detection systems. We spent several months in our Chicago lab putting these packages through their paces, using an assortment of attack tools, utilities and load-generation mechanisms. Our goal was to measure how accurately the systems detected hostile activity, how well they conveyed such activity to the user and how far we had to push them before they'd fall over. We also conducted some substantial testing of the systems' ability to reassemble fragmented packets. Although the problem with fragmentation has not been well-publicized, it is an extremely important issue. In 1998, Secure Networks (since acquired by Network Associates) published a paper detailing the problems that modern-day intrusion-detection systems had with fragmentation. A later article in Phrack Magazine revisited the issue (see www.phrack.com). The authors proved that by simply fragmenting the packets that carried their attacks, they could sneak past every commercial IDS on the market. Unfortunately, most vendors have ignored this issue, allowing skillful intruders to sneak past many IDS signatures. Earlier this year, a tool called "fragrouter" was released by Anzen's Dug Song that allowed for the easy fragmentation of all hostile traffic. Vendors have finally noticed. We used fragrouter in our testing, and sure enough, only Network Security Wizards' Dragon, Network Flight Recorder's NFR Intrusion Detection Appliance and Network Ice Corp.'s BlackIce detected the fragmented attacks. Of the three, only Dragon kept reassembling at higher speeds (see the measurements of fragmentation reassembly thresholds in "Network IDS Failure Points" at below to the right). Our Test Setup We deployed the intrusion-detection sensors on 500-MHz Compaq Pentium III systems with 256 MB of RAM. On the attack front (see "Testing Environment," above), we used a vast assortment of exploit scripts and reconnaissance tools available in the security community (for more information on these tools, see Packet Storm Internet Security Solutions' Web site at packetstorm.securify.com). We combined scanning and exploiting techniques in series, just as a real intruder would. Our arsenal included: Sscan: This older, very chatty script probes and reports on a wide range of system services and version numbers. It also performs some CGI querying. Chaff: This Perl script tests basic vulnerabilities in FTP and Web servers. Designed to set off IDS alarms, Chaff was used extensively in our fragmentation testing. Nmap: No IDS testing would be complete without using Fyodor's souped-up port-scanner. Wacky-scan: This Perl script mutates nmap's scanning behavior. Most port-scanning is performed vertically; that is, one host is scanned for x ports, the next is scanned for x ports and so on. Wacky-scan performs "horizontal" scanning and introduces a greater amount of delay between recon packets. Whisker: This CGI vulnerability scanner includes some anti-IDS checking abilities. RFP's Whisker looks for a defined set of CGI programs known to have security flaws. In our tests, it was able to sneak past everything. The usual suspects: Here we threw together many well-known stand-alone exploits and denial-of-service attacks: LAND, teardrop, kod, Bind exploits, wu-ftpd exploits and imap/ pop exploits, among others. Our target servers were placed on a 100-Mbps shared Ethernet segment with two entry points: one from the Internet and one from a WAN connection. We deployed an assortment of BSD, IBM AIX, Linux, Windows 95/98 and NT and Solaris machines. We then added an assortment of load-generating machines using Coffee Computing's FileMetric 1.2, Ganymede Software's Chariot and RND Network's Webload. We had to be careful with load generation because of the issues surrounding fragmentation and TCP sequencing, which only some products track. If the IDS sees an attack or packet go by with the same TCP sequence, it will ignore that intruder, which would nullify any attempts at fair testing. Therefore, we couldn't rely on some of the more basic load-generation and replay tools. Our first round of testing had one goal: Make sure the intrusion-detection systems caught just what they claimed they would. On an idle network, these products perform almost flawlessly--but no one uses them on an idle network. Our second round introduced varying levels of traffic using Chariot. (See the measurements for start of inspection failure in "Network IDS Failure Points" to the right.) After about 40 Mbps, several products began dropping frames. For example, Cisco Systems' NetRanger stopped detecting our Winnuke attacks (only seven packets), but continued to pick up our ftp exploits. However, at 80 percent utilization (at approximately 12,000 packets per second) many started failing horribly--they stopped detecting almost everything. Our third round got flat-out nasty. We brought in fragrouter to validate vendors' packet reassembly claims, and things got ugly. Machines began rebooting. Axent Technologies' NetProwler tossed Dr. Watson at us religiously. CyberSafe Corp.'s Centrax was crashing at random intervals. Of the network-based systems, only the machines running Internet Security Systems' RealSecure, BlackIce and Dragon did not crash or reboot at some point during the testing. Greg Shipley is a Chicago-based consultant. Send your comments on this article to him at gshipley@neohapsis.com.
PAGE: 1 I 2 I 3 I 4 I 5 I 6 I 7 I 8 I 9 I 10 I 11 I 12 I 13 I NEXT PAGE

Video