FEATURE

RMON: Unfinished Business. Get the Picture?

by Peter Morrissey and Bruce Boardman

Getting an intelligent, quick, meaningful, graphical look at the network's status is the dream of every network manager. But no one RMON product does it all yet; the picture is still unfinished. So, hit the snooze bar and keep dreaming.

The broad-daylight, two-cups-of-coffee reality is that while all of these products are good at collecting statistics about how the network is doing, they aren't very good at interpretation. Daily netw ork engineer chores, like relating the percentage of errors and collisions to utilization, remain uninterpreted, and they generally are not even available on a single printout or display.

The doubling of network traffic, tracked through alarm thresholds-even if automatically, intelligently set-may hardly be the end of the world if, say, utilization has moved from 10 percent to 20 percent. So don't blow your event flag. While this traffic pattern may show a trend, you're left to draw that conclusion on your own. And this is exactly our point: All of the products collect the statistics, and some are beginning to simultaneously display the collected statistics, but for all of their cool, graphical network maps and snazzy graphs, they haven't a clue as to th e meaning.

That being said, however, some of the RMON products currently available show promise. Hewlett-Packard's NetMetrix and Armon/Bay Network's OnSite are starting to bring the dream into reality through the strength of their applic ations. HP is the best at collecting and formatting data, and Armon's OnSite is the best at automatically setting alarms, but neither can interpret what the baseline they are generating means. We also tested products from SolCom, Technically Elite and Triticom. All five were in our tests last year [see "Probing the Depths of RMON (Amen for Extensions)," page 98, February 1, 1995]. All of the products have shown improvement and some very bright spots, but the others lack the depth of the applications of HP and Armon.

Our attempts to include products from Axon/3Com and Frontier were completely frustrating. Both companies indicated an interest in participating. Axon even committed to sending a product, but in the end bowed out, indicating a lack of technical resources-whatever that may mean.

Hewlett-Packard NetMetrix

Keeping a large, routed network running smoothly is a challenge some of us have to deal with on a daily basis. HP's v4.51 NetMetrix package is designed for just such a task, and although it might be classified as an RMON product, its true standards-based applications are overshadowed by its hefty extensions. Indeed, its Internetwork Monitor and Load Monitor applications provide potent views of network traffic that are a harbinger of RMON2. Only Armon's product approached such functionality.

In spite of these impressive applications, we were disappointed at the low priority given to standards-based RMON apps. We weren't excited about being suddenly forced out of the slick GUI, into the Unix command line and vi editor to make some of the RMON applications work properly. We were also concerned about the probe's performance, since it was the only one unable to capture all the packets in our test.

NetMetrix is a powerful network management tool, and you should plan on investing in powerful hardware to run it. Although it is pricey, its fea tures are unmatched.

HP Carries the Load Fortunately, applications like the Load Monitor go a long way to compensate for some of the shortcomings. Its display is broken up into quadrants that provide a utilization graph, followed by top 10 source and destination bar graphs and pie charts indicating protocol breakdowns. You can substitute a graph showing conversation pairs for the source and destination graphs. Selecting the utilization graph updates the rest of the graphs to reflect what's happening at that point in time. For example, you could click on the highest peak in utilization and you'd see which devices were the top contributors, along with the protocols involved at the time.

You can also switch the order of the graphs. For instance, if you placed the conversation graph ahead of the trend graph, clicking on a particular conversation pair will display the network utilization over time for that particular conversation, followed by the protocol breakdown for that conversa tion. One disappointment here was that some of the higher-level protocols used in our network were represented in the "other" category on the graph. The emerging RMON2 standard will provide a way for network managers to add definitions for protocols.

The Internetwork Monitor application provides a different perspective. It attempts to integrate the traffic from multiple probes, and it can archive the data. HP uses a Mid Level Manager application running on the workstation to accomplish this. All of the probes involved have to be associated with this application, which aggregates the data and draws a network map of nodes represented by icons. Conversations are indicated by lines drawn between the different nodes on the map. The volume of traffic is represented by colors as well as line thickness. Clicking on one of the lines will reveal the average packets per second in each direction since the previous selection.

We had a number of difficulties interpreting the map at times. For example, it was not always clear which direction of the conversation was represented by each number. And it was not always very clear what role our routers played in the picture. It would have been even more difficult if we had not already been familiar with the topology of our network. It would be helpful if there was a way to customize the view so that it more accurately reflected our network topology. It did alert us to some MBONE traffic on our backbone that we were unaware of by showing a lot of traffic between one of our hosts and a device on the Internet.

For any local network with an HP probe attached, a matrix of local conversations is graphically represented by mapping all the devices on a ring, with a line between each device involved in a conversation. Each probe was also able to see conversations in and out of its network, as well as across its network in the case of a probe that would be on a backbone network. You can do some modeling by dragging a node from one network to another , which causes the network traffic to be updated accordingly. Unfortunately, we did not have enough probes to thoroughly test this feature. We also found that it was easy to lose track of the devices after moving them.

HP generally recommends that you have a probe resident on each network you want to monitor to take full advantage of the Internetwork Monitor application. This could get expensive on a large network, but it may be worthwhile for some. You can install the software versions of the probe on HP and Sun Unix hosts to help minimize the cost.

We found that if we moved a probe and forgot to tell the management software we were moving it beforehand, it became very confused and wouldn't recognize the probes in the new location. Running command line processes fixed the problem.

Getting Into the Packets HP's Packet Capture application sports a full complement of decodes. The adjustable decode windows are laid out in the typical summary/detail/hex format. Despite no t having color-coded decodes, it was the easiest to read. We were also impressed by the fact that HP Packet Capture would automatically stay on the same layer in the detail section when moving from one packet to another.

Another area where HP excelled in standard RMON support was its statistics and history applications. These groups make use of a graph that lets you view hundreds of samples in a crisp display. The color coding for each statistic can be customized, and you can click on any point in the graph for a time stamp. We did not like the fact that the only deltas available for these studies were five, 30 and 1,800 seconds, even though the RMON specification does give the user the option of choosing any interval.

We also found it confusing that the applications that displayed these deltas were respectively called hourly, weekly and monthly. It would help if the true interval was at least displayed in the title of the graph. Another problem was that if the probe was not already runn ing one of these studies, you had to run a command line process to start them up, reflecting a bias toward HP probes, which run the studies by default.

We were also annoyed that we could not run any generic RMON applications on a non-HP device until we'd manually edited the OID string. We were disappointed that the Host and Matrix applications would only allow you to look at one or a pair of MAC addresses at one time.

Switch support also needs some work. We were forced to define an icon for every interface on the switch, and we did not have any utilities available that would help us to easily understand the mapping between the physical ports on the device and the interface indexes (or IFIndexes) numbering, which you have to know to designate a particular port. All this information is easily available from simple MIB II queries, of which both Technically Elite and SolCom were able to take advantage.

Even though this product can run as a standalone application, you will not ge t any audio or visual clues when there is an alarm unless you run it with HP OpenView. HP claims that it is very well integrated with OpenView.

Armon (Bay Networks) OnSite

This RMON management/probe combination from Bay Networks is hot. We only hope that Bay's recent purchase of Armon doesn't relax the application developments that have, until now, made OnSite a winner. Since receiving our February 1995 RMON Editor's Choice award, Armon has added a graphical map of network conversations to OnSite, improved its probe performance and added a database and report writer. But even with all of this, its merely so-so switch interoperability cost it the crown this time.

Apple of My Eye Applications The Global statistics application showed us statistics for all the OnSite and third-party probes at one time. With the base product, all defined Token-Ring probes are also view able. If you add the nonstandard, but RMON-like, OnSite FDDI module, those Armon probes are also displayed.

In a large network with lots of probes, the global display can get very busy. OnSite addresses this issue by allowing for definable subsets. Each subset can be assigned to a user ID as an added measure of security and control over network management. We had no problem setting up multiple users and probe sets, which we then could apply to EyeNet.

EyeNet is like the Internetwork Monitor on HP's NetMetrix platform, showing network layer conversations across the network and across the Internet. Although Eyenet is easier to read than HP's screen, and is more tightly integrated (providing mouse clicks to all the other RMON and proprietary extensions and carrying forward capture filters based on the conversations chosen), OnSite was still not as detailed as NetMetrix. Where NetMetrix showed hosts or hid them within an icon, and wrote the statistics of the network layer conversations that it was mapping to a database for later playback, EyeNet only displayed the conversations, with no relationship to previous traffic patterns.

We were able to limit the number of conversations on both HP and Armon platforms using RMON TopN criteria, a feature foreshadowing RMON2.

NetReporter somewhat makes up for OnSite's lack of logging. It schedules and stores preset report statistics to an Informix database. Armon has received requests from users to provide a full-blown ad hoc report writer, which Armon is considering.

For example, total collisions was reported in a vacuum as a single statistic without any idea of what the network utilization was at the time. We had to create another report for the same period and compare the two. While this isn't difficult, not having a crystal ball to know when problems are going to arise means making educated guesses and refining the kinds of reports run, and the times to run them. Armon recognizes this as an area in which to improve.

We give Bay credit for OnSite's tight integration between applications; it simplified the initial launch window and had the largest number of hot links between applications. If it seems to make sense to link from statistics to capture, Host Matrix to capture, or Global statistics (including protocol layer statistics) to specific segment traffic, Armon has the link.

Making It Work The last time we rammed packets down the OnSite probe's throat, it choked at the higher packet rates, dropping packets. This time-bravissimo-the probe performed flawlessly.

One more nice but not new feature worth mentioning is that on multiport probes, you only need to set a single IP address, and the index mapping in the management station takes care of describing and pointing to other attached segments. This not only saves on IP addresses, but it can mean the u se of a separate network for management that doesn't interfere with, or isn't bothered by, production network traffic.

Whe n it came to setting up filters and capturing packets, we had little to complain about. The filter and channel setup is very flexible and easy to use, coming with a ton of predefined filters and channels you can modify. We also easily set up our own, but we were disappointed that manual offsets could not be set-a function OnSite was obviously doing behind the scenes.

On the face of it, the capture decode is adequate. A split screen showing a summary and varying levels of detail that featured indentation, color and the choice of verbose, ASCII, hex and EBCIDIC decode was pretty thorough. Compared to HP's NetMetrix and its monochromatic decode, it appears better at first glance.

A closer look showed us an RMON packet verbosely decoded down to the MIB2 OID, stopping at the RMON decode-that may be a small thing, but not what we expected from an RMON vendor. Highlighting specific summary line fields made filtering and searching easy. However, it didn't fix the detail screen at a chosen layer when we moved between packets: Nothing is more of a hassle than having to scroll up from the MAC layer to SNMP every time you choose a new packet.

OnSite failed to hit a home run with the Ethernet switches used in our testing. It struck out with the Cabletron ESXMIM, only got a provisional base hit with the 3Com LinkSwitch 1000 but connected on the Cabletron MicroMMAC Hub. Armon diagnosed the difficulties and is working on a fix.

The problem with the LinkSwitch 1000 stemmed from the GUI's ability to only support up to four IFIndexes. Since the 3Com named the BNC Ethernet port feeding the switch as IFIndex 1 and assigned the next two to SLIP ports, the first switched port showed up on IFIndex 4, the last available port in OnSite's GUI. Since switches are somewhat new, and Armon only makes a four-port probe, it hasn't had a reason to support more than four ports. We we re, however, able to initiate applications like Statistics and History using command line keywords that pointed at ports h igher than IFIndex 4.

The ESXMIM was more serious, failing every available group on every port. A trace revealed a rejection of the SNMP GET if the syntax for the initial IFIndex didn't include a trailing zero. There was some debate between the two vendors as to who was following the standard, but in the end OnSite was unable to talk to the switch.

Technically Elite MeterWare Win95/NT

Sometimes you have to take a step back to move forward. Late last year, Technically Elite, primarily a probe vendor, took over Network Application Technologies (NAT), an RMON software and probe vendor. Although it still sells NAT's probe, most of its focus appears to be on NAT's software and Technically Elite's probe. This makes sense, since Technically Elite won our Editor's Choice award in last year's RMON testing, where we looked at individual probes. Yet the NAT software didn't fare very well, which is why we see this as a step backward.

But since the merger, the management software has donned a Windows95 and NT user interface-evidence that Technically Elite is positioned to move in the right direction. Both vendors have demonstrated a strong commitment to the RMON standard, which makes us confident that they are serious about using the RMON2 standard as a springboard to the next level of network management.

If you are looking for a powerful, interoperable RMON tool that is easy to get up and running, we highly recommend MeterWare. If you are looking for an ideal network management tool, we suggest you watch to see how it develops. The fact that Technically Elite does not support any RMON2-like extensions hampered its ability to show any network information beyond the layer 2 data RMON was designed for. And, even some of the straight RMON-based apps were not as useful as those in the HP and Armon products.

Showing Off on RMON If you have a good understa nding of RMON, and enjoy working with it on an intimate level, you will appreciate the MeterWare software. T echnically Elite gives you standards-based RMON control that surpasses even that of Armon and HP, which tend to shield network managers from how RMON functions. It could be argued that the application's ability to troubleshoot the network is more important than how it accomplishes the task. This is obviously HP's philosophy.

Technically Elite goes to the other extreme. We consider ourselves to be fairly knowledgeable about RMON and still found it difficult at times to understand the procedures necessary to access the data we wanted. For the most part, we were glad to have the flexibility that this provides, but Technically Elite could make MeterWare a lot more useful by working on things like ease of use and more sophisticated displays.

When you choose one of the devices you have defined in MeterWare's network map, you are presented with a simple menu of choices named after each of the nine RMON Ethernet groups. Once you choose a group, you're taken to the applications that can be run ag ainst that group. The statistics and history groups, for example, are each associated with a number of applications consisting of graphs and tables that display the data. The graphs are simple and straightforward.

We found it difficult to read the time stamps that were displayed horizontally at the bottom of the bar graph, in spite of the fact that the display was limited to only about 40 deltas. It would have helped if there was a way to click on the graph to get another readout of the exact time stamp.

The other major difficulty was that we couldn't always display the relevant statistics together. For example, error counts-especially collision counts-are irrelevant if you can't at least display a count of good packets to get a rough idea of the ratios. We don't understand why Technically Elite and other vendors don't go even a step further and do the simple math necessary to show the rat ios.

Singling Out Traffic We were pleased to see that Technically Elite carried on the tabular display of the Matrix group from the original NAT software. Most vendors literally display this information in an X/Y matrix, which can be very difficult to read, especially when there are a lot of devices. MeterWare simply displays each combination of pairs in a table, alongside all relevant statistics. This makes it very easy to see at a glance which conversations are contributing the most traffic or errors.

With a right-click, you can easily change the statistic sorted. And, if you want to know the percentage of traffic contributed by each pair, you can easily load the table into a spreadsheet and perform the calculations there. In general, this is a good example of an RMON vendor presenting the data in a meaningful format. It would be nice if Technically Elite calculated the relative percentages for each pair.

Both the Host and Matrix applications also display a colored dot next to each address, which indicates whether an address has seen activity between sampling int ervals. This is a good way to monitor the status of the individual network devices.

By clicking on any one MAC address, you can be taken into the capture/filter application with a filter automatically set up for the highlighted address. Although the capture/filter application provided a lot of flexibility for setting up filters, we found it confusing to set up filters from scratch. The decodes also left a lot to be desired. In many cases, such as with Novell IPX, the decodes did not go above layer 3.

MeterWare contains some good surprises, like a report writer that lets you create reports by pointing and clicking on the MIB walker application. We also liked the fact that you could easily list off all the control table entries and their corresponding IFIndex entries. This made it easier to manage the Cabletron and 3Com switches we used for interoperability testing, since we could look inside the switche s for clues about physical port mapping and the corresponding RMON groups being used. This also let us monitor and control the resources used by the different groups in the probes when we did our performance testing.

SolCom LANMaster

We liked how easy SolCom's LANMaster was to use, and SolCom definitely has one of the best probes around, but its Windows interface does not scale well. LANMaster comes with and runs on CastleRock's element manager, a Windows 3.1 network management platform. SolCom also has a Unix version, but says it feels most network managers prefer the ease of the Windows version.

Easy Is in the Eye of the Beholder Finding out about RMON is one thing LANMaster does well, but doing something with it is another thing entirely. For this entire test we stressed applications that could go beyond the raw statistics and tell us what was going on without having to piece the puzzle together. Running over the CastleRock platform provides one of the best MIB walks we've ever used and really caters to the SNMP literate. For our testing this was perfect , but it doesn't help with applications that give the RMON-impaired a clue about what's going on with the network.

The Windows interface was easy to use, and, as long as we were looking at a single probe, the screen layout was really not a problem. However, as we added multiple probes, hubs and switches, the confusing number of screens open within Windows made it obvious that this was not a platform that was going to scale, much less help us make easy comparisons to what was going on with the network.

Furthermore, the first version of LANMaster we received hung and created Windows GPFs. An upgrade eliminated the GPFs, but the hanging, caused by a lack of interaction between the CastleRock management autodiscovery and the LANMaster application, continued to be a problem. SolCom is aware of this and says it has a fix in the works.

Applications SolCom has added its Automatic Data Gather, which compare s with HP and Armon's multiprobe applications that pull back only new history information and create a database of activity from multiple probes. LANMaster includes an Excel macro to create custom reports.

This pulling down of only new histories highlights SolCom's design goal for LANMaster. SolCom believes that the most important role RMON products play is that of sending alarms when thresholds are exceeded. For this, no fancy graphical screens are needed, and SolCom says that it avoids placing unneeded traffic on the network, which SolCom says can chew up as much as two percent of a 10-Mbps Ethernet. During our testing, we didn't see anything close to two percent.

Another application unique to LANMaster is the ability to recognize and flag new hosts on the network. Only active hosts are displayed, and the first and most recent transmission time is indicated.

We had great success testing SolCom's management software with Cabletron's MicroMMAC hub and the ESXMIM switch, but we couldn't get any information from 3Com's LinkSwitch 1000. We MIB walk ed through the entire sequence with SolCom, verifying each SNMP GET, but were unable to determine LANMaster's failure to pull back information from the LinkSwitch.

Triticom Software/Probe

Triticom's Software/Probe is an unpretentious, entry-level product. Fortunately, it also comes with an unassuming price. If you are looking for an inexpensive, simple piece of software to get some quick RMON statistics, look no further.

Master of the Obvious RMON Groups MasteRMON currently runs under Windows 3.1, and with it, defining an RMON device is a very simple process. You're even given the opportunity to enter the IFIndex you wish to observe, making it work adequately with network switches. You will, however, have to run an instance of MasteRMON software for every device, and every interface of every device that you want to monitor at any given time.

The Triticom MasteRMON software only supports the Statistics, Host and Matrix groups. The Statistics group uses trend gr aphs and dials to display quick summaries of the data. It logs statistics data in order to compensate for the lack of the History group. It also supports alarms, again by monitoring statistics data on the management station.

MasteRMON does come with its own baselining utility. After it has been monitoring Statistics data at predefined intervals, you can tell it to set alarm thresholds at a specified percentage above the peaks it has already observed.

MasteRMON was the only management software that didn't support the capture/filter groups. By press time, however, Triticom plans to have an add-on product that provides this function, along with other improvements. It will make use of its LANdecoder protocol analyzer software (tested in "Software Analyzers Bring Right Features, Right Price," July 1, 1995, page 116), which currently runs on a standalone PC. This feature alone could make MasteRMON worth its price. n

Peter Morrissey is a network systems programmer at Sy racuse University. He can be reached on the Internet at ppmorris@mailbox.syr.edu. Bruce Boardman can be reached at bboardman@nwc.com.

RMON-How We Tested Probe Performance

While our general testing focused on how much network information each solution provided, from RMON to RMON2-like extensions, the performance tests only exercised RMON functionality.

We measured each probe's performance by filtering for a MAC source address, while the network was under load and each probe was running RMON studies. The concurrently run studies included both a 30-second and 30-minute History, one Statistics, one Matrix and one HostTopN study. To this we added a 7,000 packet per second (pps) load using a 100-byte packet that yielded about 70 percent utilization. Using a Network General Sniffer, we transmitted 602 control packets containing the MAC source address for which the probes' packet capture filters were set with a 500 microsecond delay.

We ran the test using three sc enarios, running each three times, for a total of nine tests. The first scenario was without any traffic to validate our capture filter. The second used a unicast address in the background destination traffic, creating the same 70 percent utilization, but it wasn't as stressful as our third test, which used a broadcast in the destination address, forcing the probes to look at each packet.

We were delighted to see the Armon OnSite probe do so well, and were disappointed that HP LAN Probe III, while doing better than it did in our last round of tests, still couldn't keep up. It was no surprise that SolCom's LANrover Plus and Technically Elite's RMON Plus kept up, because they had done the same in our last test. Triticom's probe, RMONster, does not support the Capture and Filter groups and was not part of the test.

Testing RMON Hub And Switch Interoperability

We tested interoperability using a Cabletron MIMMac hub, a 3Com EtherLink 1000 switch and a Cabletron ESXMIM switch . None supported the Capture and Filter groups. The MIMMac hub supported the seven other RMON groups. The EtherLink 1000 and ESXMIM supported Statistics, History, Alarm and Events. The groups not supported by the switches-Host, TopN and Matrix-are noted as such in our chart, with the exception of the Triticom management station, which also didn't support the History group. The ESXMIN could have supported the HOST, TopN and Matrix with the addition of 2 MB of memory.

We didn't have any problems getting the management stations to use existing RMON studies. We tested by starting studies on one management console and then viewing and using the study from another. This worked fine. We also tried to delete the study while it was being used and found that this worked as we expected. If we deleted it from the owning management console, the deletion took place. If a deletion was attempted by a different management console, the deletion failed. In addition, the Armon OnSite manag er started up its own study when we deleted a study it was displaying.

We did, however, have a problem getting the OnSite software to work with the ESXMIM. This problem opened a debate between Cabletron and Armon. At press time, with the problem still unresolved, the bottom line is this: OnSite's failed to read RMON from the ESXMIN.

The other complete failure was with SolCom LANMaster's problem with the EtherLink 1000. Even though at press time the failure remains unexplained, an upgrade to the LANMaster software during testing considerably improved its ability to look at multiple interface probes like the

EtherLink 1000. We suspect the first switched port on the EtherLink 1000 starting at IFIndex 4 caused LANMaster to fail, even though a MIB walk of each SNMP GET showed nothing unusual being returned from the EtherLink 1000.

May 15, 1996