Network Computing is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Vendorspeak Exposed!

Right off the bat, you'll notice that our fictional vendor hasn't disclosed who tested the IT Pro or who paid for that testing. Even independent testing labs can be biased in presenting results because they're getting paid to produce marketing collateral. Vendor-funded reports rarely show that company's product anywhere but on top. Beyond that bias, beware these other common traps vendors set.

1. Deceptive Eye Candy

"More than twice as many transactions per second."

Colorful performance comparison graphs can make bold statements about the capabilities of such networking products as firewalls, Web and application servers, load balancers and cryptographic accelerators, but they rarely are a true indicator of performance.

In our example, the chart clearly shows that Vendor B's Ethernet Thingy can perform more than twice as many transactions per second as its competition. Great, but how does the vendor define transaction? This is an important omission, as that definition varies not only from product to product, but also from test to test. An HTTP transaction could be defined as a single get request or a page that requires multiple get requests. The size of the data transferred is also an important piece of the equation. Many products are tuned to produce high-performance metrics with relatively small pieces of data (1 KB) but degrade dramatically as the size of data increases.

  • 1