During our tests, we deployed a number of intrusion-prevention systems on our production network, taking some lumps in the process. Here are our notes from the school of hard knocks:
Disable all blocking until you understand exactly what will be blocked.
Err on the side of caution. If your IPS has the option to block just a single packet or stream, let
it do just that. If your IPS can shun an IP address
for a period of time, be careful that you don't set it to block for too long, potentially shunning future legitimate traffic. Remember, many client IP addresses are randomly assigned.
Don't shun IP addresses based on connectionless traffic, like UDP, ICMP or TCP traffic that is not part of an existing stream. You're asking for a denial of service.
Be sure you understand what constitutes legitimate traffic, and don't accept a vendor's claim that a signature has a low false-positive rate. Your traffic is unique to your network; only you can assess what constitutes a false positive.
Test signatures that are blocking candidates for false positives.
Tune your rule base to match the applications and services that will be protected by the NIP device. This will reduce the false-positive rate.
If you can't afford any blocked connections to a service, such as SMTP, don't use NIP. Harden your server instead.