Yes, participation in reviews--especially comparative reviews that pit vendors' products against those of their competitors--carries risks. "A bad review can cast a shadow over not just the product but the company as a whole," vendors tell us. "It's not worth the risk." The reality, though, is that our lab tests mirror the pounding their products will take in your real-world environment. Wouldn't a vendor rather find out about and address potentially fatal flaws before readers buy its products?
Another vendor roadblock we often hit: "What if you don't test the features we've worked so hard to improve?" We develop our test plans for each review months ahead of the publication date, and we share the gist of those test plans with vendors so they know what they're getting into--no surprises. We're also starting to share our test plans and other article information online, giving you an opportunity to participate in our planning (see blog.networkcomputing.com/webserv.html for information about our upcoming Web services platform review).
"But what if you discover a problem with the product that we didn't know about?" We don't show vendors our test results and reviews before an article is published, but we do give them a heads up if we find a problem they can readily solve. We're not out to get them--we're just trying to help readers make wise choices about buying and implementing technology. If a flaw is identified and fixed before readers have to ante up, then we've done those readers (and the vendors) a valuable service.
"OK, but what if you make a mistake?" Well, we're not perfect, and our tests are complex, so we'll acknowledge that's a risk--but it's a mitigated risk. We have a staff of full-time and freelance writers, all current or former IT professionals, who live, eat and breathe technology in our Real-World Labs®, and we have more than 12 years' experience successfully reviewing products and services for publication. We're not bragging (well, maybe just a little) when we say nobody does it better.