Network Computingwww.networkcomputing.com

Only a few hours into a recent spate of dDoS (distributed denial of service) attacks, a Network Computing editor received a news release announcing that Keynote Systems was the first to discover that Yahoo.com had crashed. Keynote's claim, which we couldn't disprove, brought with it a sobering message that network and Web administrators face every day: The Internet is full of pitfalls and malicious entities. Keeping on top of your Web site's availability and performance is tantamount to good e-business.

Our test setup could not accommodate software-based services that required modifications to our servers. In fact, for many Web sites, it would be unheard of to unnecessarily modify anything in a production environment. Platform incompatibilities and political friction among IT staffs will undoubtedly arise if Web site performance-monitoring services require modifications to your production Web site. Some companies, however, may welcome more intrusive monitoring solutions for the sake of more comprehensive monitoring statistics. We won't try to influence your position on this matter. However, we chose only services that are agnostic to a Web site's infrastructure.

Our tests concentrated on each service's ability to monitor and report on our Web site's performance. For additional comparative purposes, we configured each service to monitor the sites of InfoWorld and Network World as well. In the end, our analysis made us acutely aware of performance-crippling aspects of our Web site. Although each service indicated different specifics, the relative differences pinpointed between sites were consistent among the services. Frankly, we see the real value in those relative differences, rather than in the accuracy of the measurements. Notifications and alerts also were very important in selecting the best service. After all, identifying a problem is always the first step in remedying it.

The five vendors monitored our site from remote locations across the network. For our report on them, we focused on each service's monitoring, analysis, ease of use, notifications and overall perceived value.

In the end, Service Metrics' extremely flexible and easy-to-use reporting and monitoring features brought its SM-Web service to the top. Keynote's MyKeynote service has equally compelling features, but falls slightly behind in ease of use. The offerings from Evity, Freshwater and WebPartner are less flexible than those of the front-runners. Although Service Metrics and Keynote are clearly a head above the rest, we recommend you watch all five companies. Their network infrastructures and user interfaces change almost daily. In fact, the whole field is burgeoning. During the last stages of our tests, we were deluged with information about new service offerings that met our criteria, but missed our deadlines.

Our tests revealed substantial differences in the services' maturity and quality. All five services required nothing more than filling out short Web-based forms for each monitored URL and they all provided loads of information regarding our slow and bulky Web site. However, their tools for in-depth analysis varied widely. Service Metrics and Keynote provided the best analysis tools and reports. Although Evity SiteAngel 2000 presents a somewhat different paradigm--it monitors the speed and availability of the components of an Internet transaction--it was more useful and comprehensive than the services from WebPartner or Freshwater. Price matters, but value is critical. We measured price in terms of overall perceived value, weighing features and services abilities with the end cost.