Setting Up an Intrusion Detection System

An IDS helps analyze and respond to attacks from both inside and outside the network. Learn how to set one up for optimal protection.

July 2, 2004

7 Min Read
Network Computing logo

You also can specify what the IDS should do when it detects a break-in attempt. It can log the activity, send an alert to a console or pager, and send a command to firewalls or routers. The most common action is to log the event--doing so provides forensic data for analyzing successful exploits and updating firewall, router and server policies to prevent recurrences. In many cases, the IDS handles only the logs and alerts, while the firewalls, routers and servers handle intrusion prevention.

Some IDSs can access new signature files generated by the vendor or a user community. In most cases, however, you must update the IDS regularly about threatening or illegitimate network behavior. If you don't, the IDS can't pinpoint exploits that haven't yet been identified in a signature.

Location Is Everything

So where do you set up an IDS? That depends on where (from which network or network segment) you expect threats to originate. The most obvious location is at the network perimeter, just inside the firewall. That's a hotspot because traffic that doesn't get through the firewall is of no interest, and any logging system that captures unfiltered Internet activity is likely to fill up quickly. Positioning an IDS inside the firewall helps you understand attacks that originate outside your network. It may not, however, cover exploits that originate from inside your network targeting your hosts, depending on your network's topology.


Choosing the Optimal Setup

Click to Enlarge

Depending on how your network is organized, you might need multiple IDSs or sensors to cover all the bases. At the very least, an IDS at the core router or switch will see most traffic streams coming through the network operations center. Be sure an IDS at this location can examine packets traversing the network in both directions--it's easy to set up a device on a half-duplex link inadvertently and miss traffic critical to determining the nature of an attack.

Some IDSs coordinate input from multiple sensors into a single reporting console, which lets you receive notification of illicit traffic from anywhere within the network. However, multiple monitoring locations means more data to store, examine and act upon.

Automated tools for analyzing IDS logs are available, but most interpretation is done by an IT person who's trained in what to look for and knows your traffic patterns. He or she combs through the IDS log to see how a perpetrator got past your security systems.

A successful IDS deployment doesn't need heavy CPU horsepower. It does, however, need to be connected to the network properly and have enough storage to allow useful analysis of the data (see "Step by Step,").

You can install the IDS via a span port on a switch, for example, or via a network tap. Each method has its advantages and disadvantages.Many switches and routers have a span port that provides a single port into the network fabric for accessing traffic at the full speed of the port. The primary advantages are cost and simplicity because the span port comes packaged with the switches and routers.

The downside is that the span port tends to be one of the lower-speed ports of the device, and can provide only as many bits as can flow through a single port. A switch with a gigabit uplink and multiple 100-Mbps ports will have a 100-Mbps span port, for instance. When traffic is heavy, streams may not be available to the IDS because the total flow exceeds the capacity of the single-span port bandwidth.

Taps, meanwhile, let an IDS "see" the traffic on a network link, but not become part of the link. In the case of the switch mentioned above, a tap could be inserted into the gigabit link to provide access to the entire data stream, but not affect the network bandwidth. Additionally, devices attached to taps don't require network addresses, and a security device without a network address is less likely to be specifically targeted by an attack.

The growing sophistication of exploits makes it imperative for both sides of a network transaction to be monitored. In some cases, this means putting a tap on a full-duplex port or making sure that the span port is full duplex. Or it may mean a pair of taps or ports (one for each traffic direction).

You also need an interface for administration and control. For security, this port shouldn't be connected to the rest of the production network. If you haven't established a separate administration network--a network with no unsecured access from the production network or remote hosts--do so now.And beware that your log file will grow quickly, so consider a terabyte of storage for IDS log files so you can keep data on the history and genesis of a successful attack. The last thing you need is your log storage system filling up and copying over log files while you're under attack.

Care and Feeding of Your IDS

With the IDS attached and plenty of storage available, it's time to start building the library of signatures. Visit www.snort.org to see how a widely used open-source IDS handles signatures and alerts. The Snort community has developed various ways of managing signature files and analyzing the contents of log files.

The size and complexity of these files drives home just how significant an investment in human resources IDS requires. The exception is a network that changes very slowly and has a narrow range of vulnerabilities. Most organizations need a dedicated IDS person studying log files to understand which transactions represent normal traffic and which are actual attacks. Once your IDS person has nailed that down, he or she will need to examine your IDS logs daily and revise the signature files frequently.

Although the IDS typically shows you how to make changes to routers, firewalls and servers, sometimes it alerts other devices to shut down attacks in progress. Some of Check Point Software's firewalls, for example, take commands from the IDS, and many IDSs forward commands for shutting down connections or blocking traffic to or from specific IP addresses.There are now also intrusion-prevention systems that take action against attacks themselves. Some are closely tied to firewalls or routers, while others incorporate firewall functions. In either case, the core IPS system still depends on an up-to-date signature file for recognizing illicit behavior and reacting to it.

Curtis Franklin Jr. is a senior technology editor for Network Computing. He has been writing about the computer and network industries since 1985. Write to him at [email protected].

To Get the Most out of your IDS ...

1. Be prepared to invest in staff for both initial setup and continued care and feeding.

2. Stay current. Get patches for OSs, servers and clients up to date, and remediate vulnerabilities before the IDS deployment.3. Decide on location. Where you put your IDS determines whether you get data on attacks launched from outside your network or on those launched from both outside and within.

4. Get a handle on control. If you don't have a separate, secure administrative network, create one. Your production network shouldn't control the IDS and pass messages between infrastructure components.

5. Get more storage. IDS log files tend to be huge, and you'll want to retain data for more than a day or two before the log files are rewritten because of space constraints.

6. Once it's attached, be sure your IDS sees both sides of network transactions.

7. Be observant. Study your log files and learn which alerts are meaningful, so you can modify alert types, logged events, and so on, as needed.span class="black12">8. Manage your signature and log files. Update your IDS signature files as new attacks emerge, and add signatures that alert you to usage-violation behavior. Develop a system for discarding logs, saving them for long-term use and pinpointing tell-tale signs of programmers behaving badly.

"Little Boxes, Big Bite,"

"Inside NIP Hype,"

"NIP Attacks in the Bud,"

"Intrusion Detection: Bright Future or Dead End?""Intrusion Detection, or Intrusion Prevention?"

SUBSCRIBE TO OUR NEWSLETTER
Stay informed! Sign up to get expert advice and insight delivered direct to your inbox
More Insights