In addition to BMC's Knowledge Module information, we collected SNMP data from Hewlett-Packard Co.'s HP OpenView NNM, IBM's Tivoli NetView, and directly from SNMP-capable network devices. This additional data provided network and system performance and topology information and let us determine how well each MoM integrated with third-party data sources.

We based our grading on five factors: event management, usability, architecture and the pricing for two different scenarios (see the MoMs report card).

MoMs use event management to reduce, focus and make sense of the sea of reports that networked applications create. Event management encompasses simple tasks, such as filtering and "de-duplication" (simply tallying reports of repeated events); moderately complex ones, such as fault suppression (essentially ignoring downstream events); and truly sophisticated tasks that give these events meaning. Normalizing, or simplifying the event description, for example, lets an operator read the description and get a clue as to what has happened.

The most difficult task, meanwhile, is to correlate events to determine the problem's root cause. Of all the products, Smarts InCharge handled correlation best. This product, with its root-cause suggestions, provides a percentage of assuredness for each possibility and helps get to the real cause better than any other MoM we tested.

We considered the products' usability as well. In the simplest terms, MoMs must be intuitive enough to lessen learning difficulty but offer enough flexibility to get out of the way of an experienced operator by offering customizable, predictable navigation and tools.

Furthermore, these products must be portable--pushed to customers via Web consoles. In the usability category, we found Managed Objects' Formula's innovative GUI to be especially useful, though this factor didn't make up for some of the product's shortcomings.

The massive number of events created by networked devices and applications requires a robust architecture. Architectural considerations include how distributed and redundant the system is and, of particular importance for a MoM, the ability to integrate into third-party management products. All the products offer distributed processing and redundancy, letting them scale, but both Micromuse Netcool and BMC Patrol Enterprise Manager had distributed engines for filtering event streams close to the event source.

The issue of price is always important, especially with a product class of this magnitude. These products typically set an enterprise back by hundreds of thousands of dollars. That said, any product's ability to reach across the network and application management silos makes the cost reasonable if the application's downtime has enough dollar impact to justify these heavyweights.

MoM Pricing click to enlarge

We set up two scenarios--one a single site with 5,000 managed entities, and the other a huge organization with more than 100,000 entities contained in 500 branch offices and five regional offices. In calculating total cost of ownership, we included the manufacturer's suggested retail price, maintenance fees, professional services and training expenses (see MoM pricing chart).

Best price? Aprisma's Spectrum xsight for a single site: $66,000 plus $13,200 annual maintenance, with training included. Curiously, Aprisma rang up the highest price in our multisite scenario: $468,000 plus $93,600 annual maintenance and $1,925 per user for three days of training. Unlike the other vendors, Aprisma recommended deploying servers at each of the sites in the second scenario, whereas the competitors say they would locate their servers centrally.

Smarts InCharge won our Editor's Choice award, primarily because it handles correlation better than any other product we tested. Aprisma's Spectrum followed closely, thanks to its strong usability and correlation abilities. Aprisma's super-low pricing for our single-site scenario also makes the product worthy of a Best Value award, despite its high quote in the larger setting.

Micromuse Netcool/Omnibus and BMC Patrol Enterprise Manager have the best architecture for focusing lots of events, but neither has the superior event correlation abilities shown by the products from Smarts and Aprisma.

All these products are complex, so plan on reading the manual thrice. And have your wallet ready: You'll need professional services!