Network Computingwww.networkcomputing.com

That philosophy has served us well, and I think it's served you well, too. However, a trend has developed within the vendor community that's quite disturbing indeed. The vendors, particularly some of the larger players, are becoming gun-shy. They either refuse to participate in reviews or they have such strict rules about participation that the review has the potential to become meaningless.

Let me say that in general, smaller players are eager to participate in our reviews. Let's face it, when they're battling corporations hundreds or even thousands of times their size, tests like ours are the only level playing field they've got.

But let's take a quick look at some of the big players in the industry. Each has been a little quirky from time to time. Still, some are far quirkier than others.

The Big Guns Novell's products were among the first that we reviewed here at Network Computing, and while at times Novell has had trouble supporting us, it has shown up for virtually every NOS and groupware review we've ever done. The folks at Novell believe in their products and want them reviewed. Given the competition and market forces that have affected Novell in recent years, I give them a lot of credit for this.

Compaq, Intel and Microsoft all seem to be cut from the same cloth as Novell. Each has been eager to work with this magazine, particularly as new products have been introduced. None of them always wins, yet they always stand behind their products and work with us to make the tests as useful to you as possible.

IBM, as you might expect, can be a little trickier to deal with. Perhaps our biggest problem is finding the right people within IBM to talk to about a particular product. But to be fair, once we finally do find the right person, IBM is almost always willing to participate. IBM does have fairly strict rules about entering prerelease products into comparative reviews, but that's understandable.

The folks at Bay Networks, in the past, required that we send a precise test plan detailing everything we expected to do to the products. Supposedly, they'd run the same the tests in-house and decide whether or not to participate based on the results. It's an interesting approach, but as far as I know, Bay no longer uses it.

Computer Associates has, for a long time, had somewhat of a running battle with us over how we test enterprise management products. These products are huge and very complex, so in order to give them the attention they deserve, we review pieces of the systems throughout the year and then summarize it all at year's end. CA doesn't like that approach and won't participate in our reviews. For that company, it's all or nothing. So unfortunately, you'll get nothing from us on CA.

Cisco--now there's a weird one. Sometimes Cisco's in, sometimes Cisco's out. Sometimes the decision to participate is based on what products we want, sometimes it isn't. Most recently, Cisco refused to be in any remote-access reviews because some other magazine did a bad review on one of its products. That's a little like saying, "I ate a bad meal in a restaurant once, so now I won't eat in any restaurant, ever!" It's irrational, and anyone at Cisco worth his or her stock options knows it. We all know Cisco makes good products--why not release them for review to prove it?

In every review, we create a short list of vendors we think we need to include in order to cover the topic adequately. For routers and such, it's companies like Bay, Cisco and 3Com. For enterprise management, it's CA, Hewlett-Packard and Tivoli. You get the idea. When you read our reviews, take a look at those first few paragraphs and see who refused to participate in our reviews and why. It may speak volumes more than we ever could about the company's opinion of its own product.

Art Wittmann can be reached at awittmann@nwc.com.