Exodus Performance Labs' StressPoint makes some of the same recommendations the other monitors make, but understanding how it got to those results is difficult. Not impossible -- just difficult.

Exodus performed more separate tests (six, to be exact) than any other vendor in our roundup. As with the other vendors, Exodus loaded the site with a random mix of browsing and searches touching on the most used portions. All the subsequent tests focused on separate sections of the site. The reports annotated most of the test but left out portions. We ended our testing with low confidence in the results.

StressPoint's top-level results were similar to Active Test's. Exodus showed a slightly more conservative 240-user load to Mercury's 260, before response time exceeded the magic eight-second mark.

Each of the graphs showed the load and test values over that load. Most of the graphs supported the summarized results of 240 users as the point at which eight seconds began to be violated and 320 users as the point where all pages were loading at rates longer than eight seconds (see screen, below).

We expected the first test, which did some general browsing and data lookup, to be the benchmarking load. That is to say, we thought it would represent a normal mix of users doing what normal users do concurrently on the site. However, the first test was plagued with errors at load levels as low as 120 users and never went beyond 180 users. The overall results of the first test were sketchy and without a performance graph, forcing us to piece together what happened using a server log chronicle.

Exodus Performance Labs' StressPoint Click here to enlarge

A Real Head-Scratcher

Exodus' summary indicated that it used this first test to determine the direction of subsequent tests. However, Exodus did not report what those determinations were. We were left scratching our heads as to how the site could be rated to 240 users when the first script loaded only to 120 users without error.

We assumed that the executive-summary recommendations were based on the idea that if any part of the site could withstand a particular load, that was the upper level of site performance--hence the 240-user recommendation.

The documentation is pretty good, and the methodology is OK, but the reporting is poorly compiled and far more confusing than it needs to be. If we were to look back at this testing in a year, we would have to decipher the results again or write our own report to explain the vendor's report.

StressPoint, $18,000, Exodus Performance Labs (formerly KeyLabs), (888) 2EXODUS; fax (801) 226-8205. www.exoduslabs.net