Some of us were a bit skeptical of the open-source Nessus project's thoroughness until it discovered the greatest number of vulnerabilities. That's a hard fact to argue with, and we are now eating our words. Nessus identified every hole that we set up, with the exception of some Sendmail 8.7.1 buffer overflow (HP-UX) and the problems with our wu-ftpd 2.4.2 (academic) Beta 18 (Linux) deployment. Make no mistake, these are not small holes; either of them could be used to compromise your environment. However, Nessus Security Scanner still got the highest overall score simply because it did more things right than the other products.

Nessus Security Scanner's architecture is a little different from the other scanners we tested, as it uses a client/server model. This allows a central server to do all the scanning while results are monitored and reviewed on distributed administrative clients. The scanning engine is Unix-based, while the administrative consoles can be run under Windows or Unix X Windows. Nessus Security Scanner supports command-line interaction as well. During our testing, however, we ran both the console and the engine on a single machine running Linux. Another difference is that not only is Nessus Security Scanner open source, but the architecture for creating vulnerability checks is quite open as well. Nessus uses a scripting language called NASL (Nessus Attack Scripting Language), which lets mere mortals actually create vulnerability configuration checks. While creating checks in NASL is not a piece of cake, it's not rocket science either.

On the reporting front, Nessus Security Scanner tends to fall a little short, and the GUI could be a little better organized. For example, all found vulnerabilities in the GUI are indexed by port and system, which is a real pain in the butt when you're delegating remediation efforts.

Once we discovered how to export information to an HTML report, however, we found that the data was a lot easier to work with than when it was in the GUI.

One thing we particularly like is Nessus Security Scanner's "honesty" when it guesses about vulnerabilities and possibly presents inaccurate data. For example, if the product made an assumption about a particular service that might not be entirely accurate, it warned us of this assumption. This is much appreciated, especially after wading through pages and pages of false positives generated by products like CyberCop Scanner. If the people contributing to the Nessus project beef up the reporting mechanism with regard to product fixes and add some more vulnerability checks, Nessus Security Scanner could easily surpass its commercial competitors--products such as CyberCop Scanner and Internet Scanner--in more than just finding vulnerabilities.

Found 15 out of 17 vulnerabilities -- Nessus Security Scanner, www.nessus.org.

Axent Technologies NetRecon 3.0 + SU7

NetRecon's strengths lie in a solid interface and a strong back-end reporting system that provided us with a good amount of comprehensive fix information. While it didn't do as well as Nessus Security Scanner or ISS' Internet Scanner in discovering vulnerabilities, it was more thorough than the majority of the products tested. Like Nessus, Axent's offering should also be commended for reporting the assumptions that may need further investigation by hand rather than misleading users by stating findings in a definitive manner.

Axent touts that its product uses a "progressive scanning" methodology that employs "path analysis." In English, NetRecon was designed with the ability to leverage information found on one server and use it against another. For example, if NetRecon uncovers password information on Server A, it will attempt to use that information on Server B. While we admit that this is a cool concept, we'd rather see NetRecon identify some of the more common vulnerabilities than watch it run off trying to execute secondary exploitation techniques. For example, NetRecon missed two of the most common vulnerabilities exploited on the Internet today: the BIND (Berkeley Internet Name Domain) NXT bug (Linux) and the RDS/MDAC (Remote Data Services/Microsoft Data Access Components) vulnerabilities found in Microsoft's IIS (Internet Information Server). It also missed some other common but less critical vulnerabilities, such as SNMP community string guessing.

NetRecon exhibited some truly odd behavior when diagnosing one of our Linux machines, and we suspect the "path analysis" was at the root of this. What we initially thought was a DNS misconfiguration turned out to be NetRecon "using" information gained from the Samba process on one of our Linux machines.

We had a Samba installation on our Linux machine with a default configuration that identified itself as "localhost." Apparently, NetRecon grabbed the name "localhost" off the Linux machine's Samba service, performed a DNS lookup on it and then proceeded to scan itself. This resulted in NetRecon's conclusion that we had some mutant Linux and NT mix that exhibited both Linux and NT vulnerabilities. Obviously this was incorrect.

If Axent cleaned up some of NetRecon's glitches and expanded its vulnerability checks, it would be right up there as a strong vulnerability-assessment tool. Right now, however, it lags behind Nessus Security Scanner and ISS' Internet Scanner in overall usefulness.

Found 13 out of 18 vulnerabilities -- NetRecon 3.0 + SU7, Axent Technologies, (301) 258-5043, (800) 44-AXENT; fax (301) 670-3586; www.axent.com.