• Monitor compliance against a defined configuration across multiple administrative domains and OSs from a single management console;
• Query systems for configuration, user accounts, access controls, and patch and service pack levels;
• Provide multilevel reports on computer configurations, from detailed technical information to high-level roll-ups; and
• Optionally, fix discovered problems proactively.
With this model in mind we gathered seven security policy monitors--BindView Development Corp.'s bv-Control 7.2 and Policy Operation Center 4.2, Computer Associates International's eTrust Policy Compliance 7.4, Configuresoft's Enterprise Configuration Manager 4.0 and Security Update Manager 2.0, NetIQ's VigilEnt Security Manager 4.0, Pedestal Software's SecurityExpressions 3.0, PoliVec's Security Automation Suite (Builder 2.6, Scanner 3.5 and Enforcer 1.1), and Symantec's Enterprise Security Manager 5.5--in our Syracuse University Real-World Labs®. Xacta Corp. declined, saying its product was between versions, and Tivoli did not respond to our invitation.
We tested these products on our production and test servers and desktops, which run a mix of Microsoft Windows NT 4.0, Windows 2000 Pro, Server and Windows XP, Sun Microsystems Solaris 2.7 and 2.8, and Red Hat 7.3 and 8.0--in all, more than 100 machines in various states of configuration and patch levels.
What We Want
We looked to create compliance checks from our existing policy. Compliance checks can be as simple as testing for a registry key value or a Windows 2000 Group Policy Object setting or as complicated as checking the user/ group rights to directories and files across all platforms. All the products we tested let us create complicated compliance checks: For example, we could check the audit configuration on a subset of computers. What counted here was the ease of defining those checks--for skilled administrators, time is big money. Perks like context-sensitive drop-down selections, feature definitions, sample compliance checks and complex expression building are pluses. BindView and NetIQ nailed this area.
Once compliance checks are made, the generated reports must be informative and customized for your audiences. For example, executive-level reports don't need to contain technical details, and technical reports aren't enhanced by roll-ups. Nearly all the products we tested can export reports to various formats or databases--a huge plus. The products from Configuresoft and Symantec shine in reporting because of their overall readability, level of detail and ability to create reports with varying levels of granularity.
PoliVec's and NetIQ's security-policy-generation applications have security policy templates that can be customized and distributed to users for review and signatures. In addition, policy statements that are enforceable on desktops and servers, such as password requirements and group security settings, can be generated automatically into a template that is used to check compliance on target computers. BindView's Policy Operations Center creates policies but doesn't export compliance checks.
Although reports are important, ad hoc queries are key when you're figuring out what can be reported and for determining the state of some network features. Like policy building, defining queries should be relatively simple. The more targeted and complex the queries, the better the score. For example, when we asked for a list of users who hadn't logged in for 30 days, we wanted just those accounts, not a report with all accounts listing a date or an interval showing last login.
Rounding out our requirements were OS support and remediation. Many heterogeneous networks run only Windows and common Unix-based OSs, like Solaris and Linux, but if you're dealing with more exotic species like IBM's AIX, OS/390, AS/400 or VMS, you need a policy-monitoring system that can support all your platforms.
The value of remediation depends on the role security personnel play in your organization. All the products tested could make some changes to target configurations. But only Configuresoft's and Pedestal's products could push out Windows security patches. Service-pack installation was supported by Pedestal's SecurityExpresssions through a customized script. However, in many organizations, operations or desktop/server staff control the deployment of patches, service packs and configuration options--the authority to make changes or deploy software crosses boundaries, and those boundaries will have to be defined before using remediation features.
All the products that required agents offered remote-deployment and silent-installation packages, which we could distribute through a login script or via a desktop-management application. While the Windows desktop administrator in us favors the agentless monitoring programs offered by Pedestal, PoliVec and BindView, the downside is coordinating the domain or local logins for each target. Agents run as a system account and aren't prone to losing communications because a user account changes its password or the computer is off-network.
Finally, cost is a sensitive issue, so we weighted price heavily in our scoring. No matter how you slice it, we couldn't justify spending more than $200,000 on 1,000 licenses. Even $150,000 is a stretch.
Because there is no way to determine a reliable street price for these products, we asked for list price based on two scenarios and calculated grades based on this information. Of course, nobody pays list, so your cost will vary depending on your negotiating power. We scored on the assumption that discounts would be proportional. Note also that pricing for these products has a linear dimension because it's based on a per-system model.
Cost aside, we were pleased with all our entries. To one degree or another, each provided compliance monitoring and ad hoc query functionality. Reporting varied greatly across the board, however, as did OS and application support. BindView's solution captured our Editor's Choice by virtue of its granular policy and query definition and decent reporting, all at a reasonable price. Speaking of price, Pedestal really shone in this area and scored our Best Value award.
RSS
