Networking

12:55 PM
Greg Ferro
Greg Ferro
Commentary
Connect Directly
Twitter
LinkedIn
Google+
RSS
E-Mail
50%
50%
Repost This

The Dodgy World Of Vendors' Customer Case Studies

This week I've had a couple of vendors that are heavily promoting their customer case studies. In the last 10 years, I've been on all three sides of the customer case study game, and it's worth pointing out the harsh realities. In my view, customer case studies are a deeply flawed metric for product evaluation because the process around them is highly distorted, if not plain dodgy.

This week I've had a couple of vendors that are heavily promoting their customer case studies. In the last 10 years, I've been on all three sides of the customer case study game, and it's worth pointing out the harsh realities. In my view, customer case studies are a deeply flawed metric for product evaluation because the process around them is highly distorted, if not plain dodgy.

The three sides are the vendor, the reseller and the customer--each of which has its own reasons for case studies. Vendor motivations are rather obvious, but what is in the deal for customers and resellers?

Many companies are keen to get publicity of any kind. And it's common for vendors to offer a case study during the bid process as a benefit for those companies for selecting them over a competitor product. But when a customer is getting a benefit from publicity, you need to carefully consider its honesty. This especially applies to universities and retail businesses.

Customers that have endorsed a product often receive services or goods as thanks or consideration. This might include a test unit, free maintenance or free training. All these things help a customer to go public about its choice and increase its happiness level. You can be absolutely certain that the customer is receiving preferential treatment at many levels. The risk to the vendor that the customer would retract its endorsement is small, and that risk can be managed by providing close support and internal escalation. Of course, this hammer for solving vendor problems is another motivating factor for a smart customer.

Remember that some people will track down the people in the case study to ask questions and check the referral. Nothing is more damaging that a negative comment or observation from an endorsed customer in a case study.

It's also worth noting that vendor and reseller sales people get considerable recognition inside their organization and often use them as part of their career plan. Resellers like to boost their recognition by pretending they have unique value or special customers so that the vendor will bring them more leads.

What does the reader get? They get confirmation that the product works and that a least a few companies have deployed it, but you have to take that with a grain of salt when making a buying decision.

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
SGHill-NWCmod
50%
50%
SGHill-NWCmod,
User Rank: Apprentice
4/15/2012 | 7:37:23 AM
re: The Dodgy World Of Vendors' Customer Case Studies
As you say, vendor-prooffered case studies should certainly be treated with a bit of caution. As a journalist I usually avoid them in determining the final pros and cons of a given solution. But what a case study offers, as opposed to a simple review, is a sense of context that is hard to establish when researching the relative value of a technology in the abstract. A good case study will establish a relationship between that product and a real-world application in a production environment that would be difficult to conjecture in any other way. A mini proof of concept, (if you will), that may well mirror some of your challenges and give you insight.

It's useful to look at case studies in the same way as any other customer feedback, nice to hear but not likely to prevent one from the completing the due-diligence process. Do your best to filter out the marketing-speak and look for the information that's useful. Truth be told, only you completely understand the nature of your particular application, so ultimately you should take case studies - as well as blog posts and product reviews from well-meaning technologists - with a grain of salt.
Tyson S
50%
50%
Tyson S,
User Rank: Apprentice
4/12/2012 | 6:50:04 PM
re: The Dodgy World Of Vendors' Customer Case Studies
I agree that there are bad case studies out there that are little more than an endorsement. There are also good ones that provide specifics on how the customer uses the product or service. What types of problems do they use it for? How do they use the product or service to solve those problems, or what does it look like "operationalized"? How often do they use it? Who uses it?

If a case study answers these questions in a clear manner, then I wouldn't say it's dodgy at all. If, however, the case study glosses over these details, then who knows what's really going on.

You make a great point about being able to actually talk to the customer referenced in the case study!

In regard to motives, when I recruit customers for case studies, I do tell them that it will give them visibility within their company and among their industry peers. Nothing wrong with that! It can be a morale boost to the team to have their success recognized publicly.
More Blogs from Commentary
Infrastructure Challenge: Build Your Community
Network Computing provides the platform; help us make it your community.
Edge Devices Are The Brains Of The Network
In any type of network, the edge is where all the action takes place. Think of the edge as the brains of the network, while the core is just the dumb muscle.
Fight Software Piracy With SaaS
SaaS makes application deployment easy and effective. It could eliminate software piracy once and for all.
SDN: Waiting For The Trickle-Down Effect
Like server virtualization and 10 Gigabit Ethernet, SDN will eventually become a technology that small and midsized enterprises can use. But it's going to require some new packaging.
IT Certification Exam Success In 4 Steps
There are no shortcuts to obtaining passing scores, but focusing on key fundamentals of proper study and preparation will help you master the art of certification.
Hot Topics
6
IT Certification Exam Success In 4 Steps
Amy Arnold, CCNP/DP/Voice,  4/22/2014
6
Edge Devices Are The Brains Of The Network
Orhan Ergun, Network Architect,  4/23/2014
White Papers
Register for Network Computing Newsletters
Cartoon
Current Issue
2014 Private Cloud Survey
2014 Private Cloud Survey
Respondents are on a roll: 53% brought their private clouds from concept to production in less than one year, and 60% ­extend their clouds across multiple datacenters. But expertise is scarce, with 51% saying acquiring skilled employees is a roadblock.
Video
Slideshows
Twitter Feed