Fortunately, there's a better way. Help is available in the form of (get ready -- it's a mouthful) Web site performance load-testing services. Why should you employ such a service? Does your core business involve placing PCs around the world, developing software that will let each PC act like multiple Web clients and delivering expert analysis of the test results? No? Didn't think so.
Those are the tasks of Web site performance load-testing services. To see how well they do their jobs, we tested the testers: Atesto Technologies' Automated Load Test and Site Response Watch, Compuware Corp.'s PointForward, Exodus Performance Labs' StressPoint, Keynote Systems' KeyReadiness and Mercury Interactive Corp.'s Active Test.
As our sample Web site, we used CountryWatch, an international site that provides gated and free news, and statistics for every country in the world. CountryWatch typifies an e-commerce site, with gated secure access, back-end SQL databases, dynamic ASP (Active Server Page) generation and shopping-cart purchases linked to off-site credit-card purchase sites. Each vendor made its own recommendations for the site and drove its own process, just as if it were in an engagement with CountryWatch.
Getting these services to load a Web site to the point of breaking seems like an easy enough process. Pick a Web site, point some service vendors at it and wait for the test results to roll in. Well, it didn't quite work that way. When you own your site but have it hosted, two layers of production control must be satisfied: the development and content that the site owner maintains, and the infrastructure discipline that a good hosting facility enforces. Luckily, we had a good contact with our host, VeriCenter. Technically, VeriCenter is an AIP (application infrastructure provider). Its team of professional technical and project people guided us through some choppy waters while we lined up the five vendors and a live international Web site.
We, CountryWatch and VeriCenter were concerned about the impact this type of loading would have on the two organizations' shared infrastructure. But it turned out to be a nonissue: The loading didn't include any agents or special monitoring software behind the firewall, with the exception of Microsoft Windows NT perfmon and telnet monitoring statistics on the load-balancing and network infrastructure. Also, we ran all the tests in the evening, when the logs indicated lower site activity.
Our winner, Keynote's KeyReadiness service, edged out Mercury's Active Test by less than 1 percent on our report card. KeyReadiness provides a benchmark that finds a site's failure points and clearly identifies the current capacity. Active Test ran the most diagnostic tests, leaving no doubt as to what failed and why. You won't go wrong with either service.
Both Compuware and Exodus are clearly in the second tier. The prices of their products are similar to those of Mercury's and Keynote's offerings, but the methodology each employs is sometimes cloudy, and the results are harder to understand.
Atesto stood alone as the only vendor to take the self-service route. The company focuses on ease of use and very low cost, but you must analyze the results yourself.
RSS
