Network Computingwww.networkcomputing.com

How about characterizing the most-efficient and least-efficient transactions (by criteria such as round-trips required and volume of data transferred) to show that the numbers are based on real cases? If all the numbers quoted are from the single most efficient interaction, they'll be meaningless to users deploying the software.

Now that we're starting to see a year 2000 statement on every vendor's Web page, I'd love to see an application networkability statement page become just as common. I'd like that warning (or ingredients label data) to be highly visible rather than hidden until the customer has already purchased the application. Why is it that all the data published on the details of these applications by the vendors is marked confidential?

And why do contacts for RDBMS (relational database management system) vendors always seem to preclude publishing the results of any comparative testing? For those of you not keeping score, that's Oracle's typical behavior. Eventually, your customer will ask you to provide some assurance of performance in realistic networking scenarios.

Application Scenario Publishing Give us some evidence to prove that the application has been tested in a variety of conditions. I'll give PeopleSoft some credit here. PeopleSoft has gone out of its way to publish information on behavior and performance in a common frame relay case for its version 7 release. Still, the document is marked proprietary. However, PeopleSoft hasn't really shown the limits of its technology, just a common scenario. Also, its results are primarily after the screen-format information has been delivered down to the desktop, and no results address the problem users might experience the morning after screen changes are made by IT and hundreds of users connect for the first time.

Are there any benchmarks? No. In the three-tier world, nothing like the TPC numbers for RDBMS access performance exists. Granted, all the applications are different and apples to apples will be hard to achieve. But maybe in modules like human resources, some common operations for comparison exist. Either way, we need some information, even if it's not in comparison to competitive products.

Needed: Application SLAs Most corporations want guarantees or at least SLAs (service-level agreements) on application performance. In a networked environment of any complexity, achieving any guarantee is at best an ongoing, constant and iterative process. I don't mean to be redundant, but again: Making applications perform up to service-level requirements for business users is difficult. IT is often required to offer the business a particular service level, usually including an average or maximum response time for typical transactions, overall uptime and availability goals, and application transaction load volume thresholds. If the business can't articulate or properly quantify these requirements (and IT can't measure or better enforce them in deployment), then IT will be blamed later for lack of adherence to new standards of performance.

I just wish that IT didn't have to do all the work itself to come up with enforceable SLAs for commonly used network applications. IT understands that tools to monitor adherence to those SLAs will cost extra, but they're required. It's acceptable if the monitoring tools are from third parties (in fact, it's probably better that way--it keeps vendors from cooking up their own numbers). But can't the main application vendor help a little bit in the planning stage so that customers have a better idea of what performance might be?

I understand this is yet more work for the vendor. Third-party labs can be hired to do such testing. Unfortunately, networking people do not purchase applications. Usually, they only find out about such purchases after contracts are signed, and then it's too late. There's no particular negotiation position with the vendor after the contracts have been signed. Users, if you know what's good for you, get networking issues into the negotiation for large packaged-software purchases. If you don't, you'll pay for it later.

I guess I should have put a warning at the beginning of this column: Buyer beware. Application performance is your problem, not the vendor's. But to me, that's a "duh."

What is less a "duh" warning is this: Vendor beware. Buyers will go elsewhere if you can't accurately explain what you think application performance will be like in multiple scenarios, or offer valuable guidelines (limits, baselines and architectural and functional data in addition to the currently common but barely adequate scenario results or calculations) for reasonable deployments. Vendors do need to help make their applications deploy successfully; as more of these deployments are over WANs with vastly different characteristics than LANs, users need more evidence of what will work and what won't.

I guess this lack of data does keep consultants and advisers like me in business. But if lack of explicit warnings and ingredient labels means the vendor's customer goes out of business, what's the point? We need truth and value in labeling software products for networkability fitness.

By the way, don't use your toaster while in the bathtub, and your mileage may vary. Duh.

Bruce Robertson is a program director with the META Group's Global Networking Strategies service. He can be reached at Bruce.Robertson@metagroup.com.