To test policy monitoring applications, we used a mix of our production and test machines in a live environment. We installed agents, where applicable, remotely from a central point or through start-up scripts. We used two Microsoft Windows 2000 Active Directory Forests, three Windows NT 4.0 Domains, several Windows standalone production servers, five Windows XP desktops and more than 50 Windows 2000 Professional desktops. We also monitored our production Sun Solaris 2.7 and 2.8 servers, and Red Hat 7.3 and 8.0 computers. In all, we had more than 100 computers in the test. Most interesting was the sheer disarray of our patch status and configuration. While we don't exactly run an ultra-secure shop, we found, using all the products, that several key servers were woefully out of date--a problem we rectified.
We used the predefined policies that came with each product to run initial discoveries and reports. That gave us time to learn each application. Once we got comfortable, we tailored reports to our needs. We also created an informal policy based on ISO 17799 statements and modeled that policy within each product. As we patched systems and made configuration changes, we monitored systems in each application to see how changes were reflected in each product.