Vantage serves up end-to-end performance data very well. By combining its systems, network and client data collections, this product can correlate precisely how long a transaction takes to complete, and where any delays occur. When a transaction is fired off, for example, a client-side agent watches threads, CPU utilization and other elements of the client environment. A probe collects the data in transit and provides a packet trace of the transaction. Finally, VantageView displays these performance metrics together.
VantageView's canned client, network and server reports offer high-level views and thoroughly link to the lower-level data. Client-side reports, for example, show trended availability and latency, and link to the underlying transaction detail, which links to the specific thread analysis for each transaction.
Network reports include daily traffic totals with overlays of transaction data that show top protocols, servers and workstations by volume. You can compare one day's traffic to another by flipping back and forth between days (a side-by-side chart display would have made this comparison better). Canned server reports display server health via CPU and memory statistics.
VantageView includes a new component, Visualizer, that lets you graphically map live data to a bitmap. Compuware smartly lets you use Visio diagrams with this feature. You have to buy Visio, but you can view those diagrams, with status, within a browser. Sweet!
Among Vantage's other components, ClientVantage is notable for its extremely clear service-level reporting. ClientVantage reports look at transaction response time in unbelievable detail. For example, starting with a month's summary of data, a response time can be drilled into, showing the day in hour increments. You can drill even further into specific client-resource environments to determine the point at which the SLA (service-level agreement) was violated. No other product in the test had this level of detail.
But to determine what's normal and what's an exception, you must know where to look for the current data then create a report that compares it to past traffic.
Administering groups of ClientVantage, ServerVantage and Application Expert agents is infinitely flexible; Vantage can conform to any departmental or user grouping needed. This allows performance and fault agent functions to be configured and arranged to provide differing alerts, transactions and performance metrics.
On the other hand, creating robotic transactions using ClientVantage and Application Expert is likely to bloody your knuckles. Compuware's authoring tool, QARun, must run on a separate box, and, while it supports any type of transaction, you have to figure out how it works first. Not a good use of time. Also, managing the robotic scripts is not seamless enough for the state of this technology. For example, QARun creates transaction scripts in an Access database by default. Though the product supports MS SQL and Oracle, distributing the database (as our test required) requires considerably more work on either platform. ProactiveNet and Concord's eHealth Suite have more centralized, integrated support.
To define client-side transactions to the ClientVantage server, you must export the script manually to a mapped drive on the server, then collect data or insert checkpoints. That's just too clunky and error-prone for mass deployment.
Each Vantage component includes two to five days of on-site "implementation assurance" to ensure quick deployment. This service, performed by skilled Compuware consultants, consists of installation verification, configuration, initial deployment and training. However, even with all the support, Vantage is better suited to application-development testing than to alleviating network managers' real-world hassles.
Vantage 8, starts at $19,000, Compuware Corp., (800) COMPUWARE, (248) 737-7300. www.compuware.com
Argent Software The Argent Guardian 6.0a
As a systems management application that doesn't need agents on servers to gather data, The Argent Guardian is different from the other products. It has strong rule-based exception management and can monitor Microsoft Windows NT and Unix servers without an agent footprint. Because the product includes polling of SNMP MIBs without additional cost, it's also a remarkable value: tens of thousands of dollars less than NetQoS, the nearest competitor.
This product collects data via API calls on NT and via SSH or telnet on Unix--a handy feature because you don't have to manage agent distribution. But you still will need to beg for access if you don't control the servers because the process logs into each server when gathering the performance metrics.
This package comes with a huge selection of predefined data collection rules, grouped into categories such as Active Directory, Event Log, Performance SNMP, Log and SLA. Within these groups, there are subgroupings. For example, the SNMP category features rules that apply to Cisco, Compaq, Hewlett-Packard, Linux and many other platforms and devices. It's impressive to see 20 predefined Cisco rules that cover CPU, memory, Layer 2, Layer 3 and other performance metrics. An SLA rule measures availability; Argent says this is better than a ping because a busy system might answer a ping but be running applications that are too busy to respond. Good point.
The Argent Guardian also has a huge set of alarms, which can notify you via all the usual methods, such as beeper, pager or SNMP. Some unique inclusions are based on application and database metrics, such as SQL, Exchange, start and stop services and even system shutdowns. There's nothing like having power!
Argent Predictor, the product's trending engine, requires a defined set of data to collect. There are canned reports, but we wanted to trend our infrastructure and Web server to get a correlated view, as with all the other products, to avoid creating empty reports when a monitored device doesn't support the metric specified in the data-collection scenario.
There is a basic network report that gathers total bytes per interface and has a handy integrated MIB browser to allow for quick point-and-click selection of a specific SNMP OID to be monitored, but a shortage of some details limited the data's usefulness to hard core network management types.
Further, network devices cannot have their baseline created unless licensed. So, while we get real-time SNMP checks of network devices free, trending data costs extra.
Argent administration uses Microsoft MMC consoles and a giant Web publishing download called UBI (Universal Browser Interface). In 10 minutes, I had the MMC reports available, read-only, to my local neighborhood browser. It works, and it's easy. The product doesn't allow for portioning of data by user or application group, the way NetScout, Concord and Compuware's products do. Also, authentication relies on Windows 2000 or NT and at first denied us access to machines on untrusted domains. To avoid this problem, we created a user on the local domain that matched the user on the Argent server's domain.
Argent uses calendars. A main calendar defines day-to-day periods, such as work hours, after-hours, weekends, and holidays, but also lets you define and monitor special periods, such as the holiday shopping season. Simply define that season, then deploy the calendar. This sophisticated function is common on mainframe-scheduling applications, but rarely seen in client-server technology.
The product also supports regional agents--essentially, remote engines that check the health of the main monitoring engine and do some of the polling. The vendor recommends one regional agent per 100 devices. Whereas the other vendors in this test charge for additional distributed polling engines, Argent charges only for an additional license; the distributed functionality is included.
Argent's pricing was the lowest of all the products tested, in part due to the free SNMP polling. However, trending SNMP devices requires a license, and trending is necessary to determine what is normal and what is an exception. Still, Argent indicated that its $58,700 price for our scenario was accurate, regardless of the licenses required to capture historical SNMP data. As always with pricing, your mileage will vary, but this seems like the deal of the year.
The Argent Guardian 6.0a, $15,000, Argent Software, (860) 674-1700. www.argent.com
NetScout Systems nGenius Performance Manager 1.4
Low-level probe data and its high cost work against NetScout, but nGenius does have some benefits. The product has a superior reporting paradigm, and implementing its proprietary data sources doesn't require any cycles or installation from existing production servers, switches or routers. That's worth something.
And NetScout has high-level performance views covered. A browser-launched Java applet reports performance in an attractively designed newspaper paradigm. Reports are published as articles, and users are subscribed to newsstands that partition data into information. This eye candy does a good job of covering everyone from executives to operations staff to engineers with very little work.
The nGenius probes use a nice trick to track network services, such as DNS and SMTP. A virtual interface is defined, and well-known TCP and UDP ports are statically assigned. Then, when traffic matches those ports, it's mapped to a particular transaction. This release of nGenius cannot use dynamic ports, but NetScout says a future release of nGenius will be able to do so.
There is nothing wrong with nGenius' CLI (command-line interface) for configuring the probes, but we expected those data-collection instances to be reflected in the management application. As it is, each probe must be checked individually for the proper configuration.
We also learned that the Java engine doesn't yet support Windows XP, as none of the three versions of the Java Virtual Machine launched from within Microsoft Internet Explorer 6.0.2 or Netscape 7.0 would run. NetScout says XP is still to be certified as a base client OS.
nGenius Performance Manager 1.4, $50,000, NetScout Systems, (800) 357-7666, (978) 614-4000. www.netscout.com
NetQoS SuperAgent 3.0
It's all about focus for NetQoS's SuperAgent: This package requires you to focus on what you are going to monitor. SuperAgent also only monitors TCP applications, admittedly the primary application protocol, but not the only, so the focus may be too tight for some.
Out of the box, NetQos SuperAgent is a blank slate--a 1U Dell appliance whose installation amounts to configuring a IP address in Windows 2000 and determining where on the network to monitor. SuperAgent's high level comes from what you tell it to monitor. The biggest job is figuring out what traffic to capture.
This is in contrast to the other probe-like products that gather data off the wire and then try to make sense of it by categorizing the traffic. For example, once data sources were defined, the other products we tested began some default reporting that we could adjust. The SuperAgent approach requires more implementation work, but also reduces the noise level of unwanted statistics.
SuperAgent helps with configuration by listing a huge number of existing well-known and registered applications, monitoring any user-definable port and displaying TCP applications that it has seen for inclusion. This last point is useful, as it reduces the amount of work to define TCP ports that are to be reported on.
SuperAgent's Alarm display is nicely compact, simply divided into three categories: standard, custom and additional alarms. The standard alarms measure network round-trip time, throughput and loss rate. All are displayed in eight-hour, daily, weekly and monthly buckets. Each one is linked to a summary display for the time period sorted by type and those to the specific error conditions creating the alarm. It makes for a very spiffy display. The 11 custom alarms cover connection time, refused sessions, open sessions and timed-out sessions, among others. You create the additional alarms. For example, during implementation, we created packet-focused tracking fragments and discards. Links drilled down to reveal more detail and the specific cause of the alarm.
We upgraded SuperAgent during the test--an act that required a console connection to download the files and run the setup. Had we not been on the console of the machine, we would not have been able to complete the upgrade, as the remote control hung at the end of the process. NetQoS's upgrade read-me document says a reboot may be necessary if the appliance hangs, so you're out of luck if this happens when you're across town. After the reboot, everything worked fine.
SuperAgent 3.0, $29,500, NetQoS, (877) 835-9575, (512) 407-9443. www.netqos.com
Bruce Boardman is executive editor of Network Computing, testing and writing about network management and systems. He has 12 years' IT experience managing networks and distributed computing for a financial service provider. Write to him at Bruce Boardman at bboardman@nwc.com.
REPORTS
ANALYTICS
Follow 
