Compuware has been in the performance-management business since before the category existed. PointForward is a logical extension of its many leading systems- and network-monitoring and usage products. However, we still felt like we were testing version 1.0.

Compuware's results resembled those of the other services, finding that a load of 200 to 300 users is the site's current capacity. And it found fault with ASP and back-end database transactions, as did Mercury, Keynote and Exodus. Both of these findings are clearly supported by Compuware's Application Advantage, which broke out client, network and server performance metrics. But the output from PointForward did not fully document the testing or the results, and provided by far the briefest set of reports.

Pointing the Finger

Compuware's reports come as HTML pages served by the secure PointForward site. This is handy for distributed viewing, but the pages required some browser reloads as we navigated. It was easier to download the results for offline viewing.

Compuware's Results Click here to enlarge

All the recommendations came from PointForward's analysis report, which suggested the minimization of ASP usage, based on Compuware's experience and its VantagePoint data, which shows elapsed time breakdowns for server, client and network for every page element. Presented as a Microsoft Excel spreadsheet, the report clearly pointed to ASP as taking the most time for each page.

The summary reports page was so concise that it contained absolutely no summary analysis of the test results. A number of graphs displayed each of the browsed paths over the elapsed time of the test. These graphs showed the overall response time for the entire set of pages making up a transaction. A tabled breakdown accompanied these graphed results, with page-by-page breakdowns but no analysis of what the graphs meant or how the Web site was performing.

The individual transaction graphs did not indicate load value. Rather, load value appeared at the bottom of a summary report that combined all the transactions and load over time. We could then index the load to time values and get a sense of each transaction's response during the test.

Summary data included information about the locations and the number of users executing each of the scripts. However, it would have been easier to correlate the data if that breakdown had been listed in one place, instead of attached to each separate graph.

Also frustrating, the graphs combined useful results with useless results. For example, the load for each test was used to create the scale of the Y-axis. This presentation skewed the scale and reduced the graphical correlation value of each. Finding the point at which the site transitioned from acceptable to unacceptable performance was a guessing game. By comparison, this important information was easy to find in Keynote's output.

While we found the interaction professional and the results accurate, the lack of adequate test annotation and almost nonexistent analysis prevents us from recommending PointForward until these shortcomings are addressed.

PointForward, performance monitoring starts at $200, Compuware Corp., (248) 737-7300, (800) 521-9353; fax (248) 737-7119. www.compuware.com