Network Computingwww.networkcomputing.com

Measuring server performance in a way that's applicable to the real world is no small chore, but we set out to accomplish just that.

It's all too easy to run all the I/O and processor benchmarks you want, only to spit out some arbitrary number that's irrelevant to anything other than that particular test. But in a real-world environment, especially where servers are involved, performance boils down to finding and eliminating bottlenecks, many of which are independent of applications. To measure our real-world environment, we turned to Bluecurve's Dynameasure 2.0.

Dynameasure is able to stress-test servers running SQL, Oracle, Exchange and IIS on both production and nonproduction servers. With the test agents installed on the client machines, motors are started on each of the agents, which serve to generate the simulated workload. Each client can have multiple motors; it should be noted, however, that one motor is not necessarily equivalent to one user. Because messaging is a mainstay enterprise application, we chose Bluecurve's messaging workload profile for our performance tests (it is available for download at www.bluecurve.com).

Bluecurve's standard messaging workload is representative of typical messaging scenarios experienced by end users: It contains a realistic mix of user transactions applied against a repository of messaging data, and measures important and commonly used components of Microsoft Exchange. The workload, which Bluecurve developed with the help of Intel and several large customers, does not pretend to be an ultimate profile that matches an ideal average user. Rather, it is meant to serve as a yardstick based on a broad mix of the most common actions.

Our Test Bed
Our client machines consisted of 100 400-MHz Pentium II machines, each with 64 MB of RAM. We worked with KeyLabs, an independent testing facility in Lindon, Utah, to set up our test environment. The clients ran Windows NT workstation with Service Pack 3, Outlook 97 and the Bluecurve test agent. The messaging server ran Microsoft Exchange Server 5.5 with SP 2. Our network infrastructure was built around an Extreme Networks' Summit48 switch with 100-Mbps links running to all the clients.

For each test run we began with five motors and ramped up to 200 motors over nine incremental steps. We conducted the tests in such a way so as to achieve optimal granularity. It took approximately eight hours for each server to build the messaging data set, and then three hours to run the actual benchmark. The results of particular interest are TPS (transactions per second), ART (average response time) and KBps (KB per second) (see "Quad-Processor Performance" above and to the right). We also looked at network, server and client utilization throughout the tests.

Test Results
For the first six steps in the TPS test, the results remained relatively close, with the Hewlett-Packard LH4 edging out its two competitors. From Step 7 onward, however, the LH4 dipped and leveled off. The server displayed similar results in our other tests. It's apparent that at Step 6, which reflects the load of 125 motors, HP's disk subsystem can no longer cope with the stress. Meanwhile, both IBM's Netfinity and Compaq's ProLiant servers held their own throughout with minimal performance hits. By Step 9, the ProLiant had edged out Netfinity.

The story is much the same for the KBps results. The LH4 took a bit of a dive at Step 6, with the Netfinity server following suit. The ProLiant overtakes them both and only at Step 7 does it begin to plateau.

The biggest news comes in the ART results. Again, the LH4's performance starts to degrade at Step 6, while the Netfinity server holds on until Step 7 before it begins to drop off. Here's where the ProLiant shows its might; it barely skipped a beat throughout this test.

The Big Picture
In their current configurations, the Compaq ProLiant 6500 can outperform the other two overall in BPS and TPS and shows superior scalability with stable ART. The IBM Netfinity showed great performance overall, but simply couldn't hang with the Compaq server at the end. The LH4 broke strong out of the gate, but took the biggest hit when the going got tough.

Send your comments on this article to Dave Fetters at dfetters@nwc.com.