Out of the box, UpTime caught more error conditions than NSM+, but when carefully configured, the NetScout product matched UpTime in alerting and reporting. The Sync and Adtran products offer comparatively rudimentary capabilities but are a dramatic improvement over an unmanaged network and are substantially less expensive than the NetScout and Visual products. Like NSM+, Adtran IQ View required more configuration than UpTime but provided adequate alerting and reporting tools. As noted, performance problems with Sync's CSU/DSU probes kept alerts from properly registering to the NMS.

Identifying unusual customer-side network activity, like a new bandwidth-hogging application, is difficult using any of these platforms. We blasted a stream of IP packets over a not-well-known UDP (User Datagram Protocol) port. UpTime and NSM+ correctly reported the burst as a high-volume IP stream. Only NetScout let us monitor down to the IP port level, although the process of discovering this rogue application was time-consuming and cumbersome.

NetScout Systems NetScout Manager Plus 5.2 with NetScout WAN Probe
While Visual UpTime is targeted right at frame relay management, NetScout Manager Plus is a generalized mechanism for managing practically any network. UpTime is a tactical tool for frame relay only, but NetScout users make a strategic choice to instrument as much of the network as possible. By virtue of its use of RMON (Remote Monitoring)- and RMON2-based probes and management software, NSM+ is a more open, flexible and information-rich system than any other tested here.

The probe and software options available from NetScout are impressive, including almost any interface you'd need--Ethernet, token ring, FDDI, CDDI, 100BASE-TX/FX, Serial and switch ports, with Gigabit Ethernet in the works. You can even configure an NT-based PC as an RMON2 monitoring device with NetScout Agent for NT.

Delay measurements illustrate NSM+'s broad reach. All of the other systems attempt to measure latency exclusively between probes (dubbed "agents" in NetScout parlance). Only NetScout can measure delay to other systems--hosts, clients, routers and anything else that can respond to a ping. With NetScout's add-on Resource Monitoring application, alarm thresholds can be set for ping latency as well as loss. Oddly, we were not able to monitor both loss and delay to a given target--an omission the company promises to fix soon.

IP ping messages are also low priority and may get dropped or buffered by routers in favor of higher-priority traffic, resulting in incorrectly high latency or loss measurements. This is less of a problem when the traffic end points sit at the CSU/DSU, because traffic once submitted to the frame relay network is not prioritized within a given PVC. NetScout claims to adjust for this, but we were unable to verify it in our testing. In contrast, Visual uses frame correlation of actual user traffic to detect delay, though it could not be used to generate alerts in the version we received for review. Until Visual offers this capability, NetScout takes the lead in delay-measurement capabilities.

NSM+ is a broad suite of network monitoring and analysis tools, broken into four modes: Administration, Traffic, Protocol and Application. The Administration applications include agent configuration, definitions of which protocol and traffic statistics to collect, alarm thresholds and report definitions. Traffic applications deal with Layer 1 and Layer 2 information. Protocol tools work with network-layer data, while Application tools deal with upper layers.

It takes some effort to understand NetScout's terminology. For example, NetScout defines "domains" as sets of network protocols--such as TCP traffic for a certain set of agents. Those accustomed to Microsoft Corp.'s use of the term "domain" might expect it to be a collection of segments, but this isn't the case. NetScout's definition is more of a horizontal traffic layer across the entire network, instead of a vertical partition between networks as you might intuit.

Terminology aside, the suite of applications is incredibly powerful, offering a broad range of traffic discovery, analysis, fault management and reporting tools. The Trend Reporter application includes many predefined report templates in both graphical and textual formats. These include WAN usage, billing, round-trip delay history, network availability, conversations and protocol distribution.