Network Computing is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Performance Anxiety

Whenever a new technology hits the streets, the question of whether it can keep up with network traffic quickly follows and with good reason. If a product becomes a network bottle neck or fails to process everything it should be processing, the product has failed. That fear of failure, or perception of being slow, often drives vendors to make optimistic performance claims about their products and drives reviewers like myself put vendors products to the test.
This doesn't happen in a vacuum either, IT buyers, whether they are customers or readers want to know whether a product will keep up with traffic. Beware of performance claims about NAC products regardless of the source. Unlike network device testing, there is a whole lot more going on in NAC products that will affect performance and capacity.

In our next round of testing NAC products, I am purposely staying away from performance testing because there just isn't a reliable way to do it meaningfully. Sure, I could toss packets at the NAC devices all day, but meaningful traffic like you would see in a real network is some else entirely.

Designing a performance test, whether you are a vendor or a reviewer, is fraught with decisions and compromises. First and foremost is the realization that there are N+1 test cases in the world that may be interesting to test but there is also a finite amount of time and resources available. Naturally, not every thing gets tested. So a scenario has to be developed that accomplishes a limited set of goals. Sure, the test is contrived to a degree, like all simulated tests are, but a well balanced test tells you something about product performance.

When building a network test, the type of traffic that the device under test (DUT) processes has to be evaluated and created. For a layer 2 switch, it's often sufficient to properly formatted Ethernet frames and the payload doesn't really matter. As the DUT moves up the OSI layer through networking (IP), Transport (TCP/UDP) and beyond, the difficulty in created real traffic rises dramatically. Tools that were designed to test switching and routing will fail to create the protocol stacks to test layer 4 devices like a stateful packet filter firewall. Likewise, the tools that can test a layer 4 firewall often fail at testing an application firewall???a firewall that process, for example, HTTP transactions. Today, it's not hard to generate a large number of TCP sessions and run traffic over them. It is still hard to generate a large number of HTTP users interacting with a web application.

Work-arounds that probably don't
When we start talking about NAC, the performance issues are more difficult to test and maybe even more difficult to describe. Of course, the performance issues will vary depending on how the product sits in the network. Not only do the NAC products have to process packets, depending on how they work, they also have to process users and events. A large user population is hard to simulate realistically.

  • 1