Web Infrastructure: Relieve Web Performance Anxiety

A useful test for measuring end-to-end performance is called functionality verification. It confirms whether the Web page that was downloaded is the real thing or an error page, so you can detect when your Web server can't reach the database server. Knowing your server can handle 1 million connections is fine, but if 40 percent of those connections contain SQL error messages, you have a problem.

Use as much realistic traffic as possible in your Web site testing. Download a typical page or execute multiple database inserts and queries. Merely initiating a TCP "open" and "close" isn't enough, unless you're conducting a Layer 4 firewall test, for instance, where content doesn't matter much. But if you're testing a Layer 7-aware device, you need real data.

Simulate user behavior, too, to ensure the test is realistic. Walk through the site, performing multiple tasks and factor in some "think time." It takes longer than one second for a user to download a complex Web form, fill it out and submit it, so add a few seconds or a minute before submitting a form to ensure it's an accurate representation.

Choosing the right benchmarking tools for your tests can be challenging. They vary in functionality (see the table, below, for some examples) and price. In some cases, you'll need to buy both benchmarking hardware and software. You can also try shareware, freeware and cheapware benchmarking tools, but they may lack necessary features. A realistic test also includes multiple types of traffic and staggered start times for your user's scripts. Which protocols you test depends on what services your site offers. For a Web test, you may only need HTTP and HTTPS. But if you have a Layer 7 firewall and your network most often uses HTTP, FTP, DNS and SMTP, you'll need to test the firewall with all these types of traffic.

And it's best to test for sustainable traffic, because traffic ramps up over a period of time. This way the device being tested won't be overwhelmed, and you'll find the maximum capacity of the device before it fails. So start with 100 connections, then 500, then 1,000 and so forth. Traffic will arrive at the site in shifts--some will be handshakes, others data transfers and sessions terminating.No site gets hit with millions of simultaneous, legitimate connections arriving within a second of each other. But don't write off the possibility of sudden spikes. To test for spikes, sharply increase the rate of new connections.

Calculating the sustained rate your site can support depends on your test definitions. Are you testing for maximum TCP connections or for the maximum number of users? A single user may have multiple TCP sessions going at once, may wander around a Web site with just one TCP session at a time or have short periods of time with no TCP sessions open at all. You will need to figure out what users normally do on your site. Do they read long pages of text, or rapidly jump through pages?

Your Web server tests should gather a combination of sustainable TCP connections, gets per second and response time. The number of gets per second determines how fast a single user session will execute. Response time tells how long the actual transaction will take. These elements can be tricky to test. Consider the differences in HTTP 1.0 versus HTTP 1.1 protocols. The newer HTTP version supports "keepalives," so you only need one connection, whereas version 1.0 requires one get for every element on a page. Older browsers use 1.0, so keep that in mind when determining the maximum number of users your site will support. The best bet here is to test a mix of 1.0 and 1.1 HTTP traffic, and make sure the benchmark tool pulls down all the objects and not just the main page. Otherwise you won't get a valid benchmark.

Testing applications, meanwhile, is different from the simple Web test. Grabbing the list of top stories off www.techweb.com, for example, is merely pulling a page with a few objects. But testing Web applications on a site entails measuring multiple steps, including filling out forms.

And don't forget that users come to your public Web site from the Internet, not from your LAN. That may sound obvious, but it puts your test in perspective: Did your end-to-end tests blast away at 100 Mbps with 2 milliseconds latency? If so, that's not a realistic measure. Most users have Internet links slower than 2 Mbps; the majority of users in the United States still rely on dial-up services. Latency varies, but for modem users, expect a minimum 200-ms round-trip delay per packet. If there are many transactions between the client and server, latency can easily add several seconds. This places an extra burden on send and receive queues (though you can optimize them to improve performance).Most benchmarking tools can simulate low bandwidth. But if you want to inject latency or network problems like dropped packets somewhere beyond the end nodes, you'll need a specialty device such as the Shunra Software's Storm. Don't forget to stress-test your firewall, Internet router and any other device in between (see Step by Step).

Something Old, Something New

If you decide to upgrade your Web infrastructure or change the design altogether based on the results of your end-to-end benchmarking tests, beware of overbuilding your site. Say you have a Web server that usually gets about 1 million hits per month. One day your site gets a mention on nationwide television, and suddenly your traffic spikes to 1 million hits per hour for several hours. That doesn't justify restructuring your site to handle so much traffic: Buying more hardware and bandwidth than you really need is wasteful. You really can't plan for these extreme cases, so build only to your expected peak loads.

And before going live with your upgraded Web infrastructure, make sure you run the old and new Web sites simultaneously. It's best to launch the new site during your slowest periods, unannounced. This lets you iron out any lingering problems behind the scenes. Test the new site from somewhere else on the Internet, too, such as from a dial-up or DSL connection. Any upgrade, even one as minor as adding RAM, has the potential to break some piece of software, so test-drive your new Web site components thoroughly.

Michael J. DeMaria is an associate technology editor based at Network Computing's Syracuse University Real-World Labs®. Write to him at [email protected]. 1. Test the simple devices, such as switches and routers, that don't maintain state tables by blasting traffic through them. Test various packet sizes.2. Test the firewall. Perform throughput tests similar to Step #1. For session tests, use the techniques you would employ for testing Web servers. It's important to gauge sessions for stateful packet-filter firewalls, because they maintain a table of all active connections.

3. Test caching devices. Make sure you use varied content, including HTML, graphics and Java files, and not the same page again and again. Add a few cache misses, and don't forget that some dynamic pages can't be cached.

4. Test the Web servers and load balancers. Check the successful hits per second and download times. Turn on compression and see if more pages are served.

5. Test SSL encryption speeds, if necessary. Whether you're running SSL in software or off-loading it to an accelerator card, test the encrypted Web traffic. Even with off-loaded SSL, the server has more traffic floating around its PCI bus than it does without any encryption.

6. Test the disk-access systems. Pull site data from a mounted network drive and test disk performance where the log files are written.7. Test any authentication servers, including LDAP directories. Most LDAP directories work independent of the Web servers.

8. If your site contains mail forms, test if your e-mail server can handle the traffic by spamming it.

9. Test the entire infrastructure. This will be the most time-consuming test to set up--you'll be simultaneously hitting the Web server, access databases and authentication systems'and handling SSL.