This week I've had a couple of vendors that are heavily promoting their customer case studies. In the last 10 years, I've been on all three sides of the customer case study game, and it's worth pointing out the harsh realities. In my view, customer case studies are a deeply flawed metric for product evaluation because the process around them is highly distorted, if not plain dodgy.
The three sides are the vendor, the reseller and the customer--each of which has its own reasons for case studies. Vendor motivations are rather obvious, but what is in the deal for customers and resellers?
Many companies are keen to get publicity of any kind. And it's common for vendors to offer a case study during the bid process as a benefit for those companies for selecting them over a competitor product. But when a customer is getting a benefit from publicity, you need to carefully consider its honesty. This especially applies to universities and retail businesses.
Customers that have endorsed a product often receive services or goods as thanks or consideration. This might include a test unit, free maintenance or free training. All these things help a customer to go public about its choice and increase its happiness level. You can be absolutely certain that the customer is receiving preferential treatment at many levels. The risk to the vendor that the customer would retract its endorsement is small, and that risk can be managed by providing close support and internal escalation. Of course, this hammer for solving vendor problems is another motivating factor for a smart customer.
Remember that some people will track down the people in the case study to ask questions and check the referral. Nothing is more damaging that a negative comment or observation from an endorsed customer in a case study.
It's also worth noting that vendor and reseller sales people get considerable recognition inside their organization and often use them as part of their career plan. Resellers like to boost their recognition by pretending they have unique value or special customers so that the vendor will bring them more leads.
What does the reader get? They get confirmation that the product works and that a least a few companies have deployed it, but you have to take that with a grain of salt when making a buying decision.
