• 05/08/2014
    8:00 AM
    Robert Mullins
  • Robert Mullins
  • News
  • Connect Directly
  • Rating: 
    0 votes
    Vote up!
    Vote down!

IT Pros Mistrust Cloud Performance Claims

Survey shows companies think cloud provider performance problems are hidden behind hollow SLA guarantees.

A survey of companies that use the cloud to deliver business applications shows that nearly three out of four of them think their cloud providers are hiding performance problems from them.

The survey of 704 senior IT pros commissioned by Compuware, a provider of application performance management technology, shows that 73% of respondents believe that companies hide performance shortfalls with application delivery. Seventy-nine percent believe that cloud provider service level agreements (SLAs) are poorly written and don’t adequately explain what assurances providers actually make. Survey respondents view SLAs as too simplistic.

"Companies fear that infrastructure and platform-level problems are being hidden behind hollow guarantees, with little incentive for cloud providers to give any greater levels of transparency," a whitepaper based on the global survey states. "Vanity SLAs provide a false sense of security and do not capture the true customer and end user requirements that properly mitigate business risks with cloud infrastructure."

Simplistic SLAs, the report indicates, are those that emphasize "availability" of the cloud application without accounting for many variables in application delivery.

For instance, a company may be assured by its cloud provider that its website will be available year-round, but performance may suffer if there’s a rush of business for an online retailer around the holidays, at a floral site prior to Mother’s Day or on a health insurance site during open enrollment for coverage.

A site could easily find itself in a position where it has been available all year long but suddenly fails when it really counts, "yet the cloud provider has still met the minimum availability SLA," the report states.

The essence of the problem in such instances is the difference between "availability" and "performance," said Thomas Mendel, an analyst with Research in Action, which conducted the survey for Compuware.

"Most cloud providers will say they have 99.999 or whatever percent availability. That’s not the problem," Mendel told Network Computing. "Many of them will not give you a performance guarantee because there are very few companies who can actually guarantee performance over the long run."

Performance shortfalls could result in customers getting on the site, but their transaction not being completed, meaning Mom doesn’t get her flowers and the customer doesn’t get signed up for the insurance he or she seeks.

The whitepaper advises companies to insist on flexible SLAs that are based on their needs instead of the cloud providers’ reporting requirements. Additional metrics that should be disclosed include end-user response time, the error rate on transactions, and capacity planning.

In addition, Compuware’s survey reveals that cloud customers believe that platform-as-a-service (PaaS) and infrastructure-as-a-service (IaaS) providers are not always forthcoming about whether service problems are related to failures in the provider’s own infrastructure. The report advises companies to regard cloud providers as an extension of their own IT departments and hold them accountable for problems that impact the company’s bottom line.

However, that can be problematic, too, Mendel said, because while a customer can see into its own data center, it has little idea what’s going on in the cloud.

"With this end-to-end SLA, it is very difficult to figure out if something goes wrong, who’s to blame? And there’s no one who offers a good solution to bridge that gap," he says.

The survey showed that 62% of respondents said troubleshooting application performance is difficult because they lack the necessary visibility into the cloud environment. Compuware notes that some cloud providers are starting to offer premium services to improve that visibility, such as guaranteed IOPS, but warns that those kinds of options are just point solutions and may not provide the complete visibility customers really need.

The survey also highlighted another problem: Performance issues traced to adjacent tenants in the cloud environment, which the report describes as "noisy neighbors" that may be monopolizing resources in the cloud. Sixty percent of cloud customers surveyed said they worry about that.

There is hope for improvement. As cloud deployments get more complicated and customers demand more solid SLAs, the cloud market will be increasingly dominated by the larger cloud providers with the capabilities to deliver the necessary level of availability and performance, Mendel said.


Cloud performance

I'm not really surprised at the results of this survey, or the lack of success providers have in predicting cloud performance. It's kind of like when you buy a car -- do you really expect to get the MPG that's listed on the sticker? No. And the factors that play into enterprise cloud performace are much more complex. Then again, I'm sure it depends on the service, the price, and how business-critical it is. I'm interested to hear from readers about their expectations. 


A lack of visibility into cloud provider environments has been a long-standing issue for enterprises, particularly on the security front. The Cloud Security Alliance has tried to address this through its various initiatives.

I'm not sure I'm convinced that cloud performance will improve as the market is dominated by large cloud providers, as Mendel says.

Cloud Understanding
I'd estimate that 80% of cloud users do not understand what they purchased and the inherently different paradigm that is cloud computing. This survey is fundamentially flawed, and the problem is the set of misconceptions among the user group. It is not about speeds and feeds, it is about workloads and results.
Re: Cloud Understanding

Thanks for your comment Harry. Can you elaborate on how it's about workloads and results, not speeds and feeds?

Re: Cloud Understanding

Harry, I'm not sure if the problem you're pointing out is necessarily isolated to the cloud or just to technology and efficiency in general. Measuring proicessing speeds and uptime does not equate to business success --there's a lot of other dots that need to be connected in between, and that's very complicated and different for every organization. So the business needs to understand the technolgy and exactly how it is contributing before it can measure whether it is being sucessful or not.

Performance-based SLAs

Service providers use an availability-based SLA because that's a relatively easy thing
to measure and prove SLA compliance. What's much harder is a performance-based
SLA. Ideally, the performance metric would be directly related to application
transaction performance—what the end-user actually experiences. But we all know
how difficult (and often very costly) it is to obtain those reliable application
performance metrics. Another possibility is measuring the performance of the
underlying infrastructure (which in the end is what delivers these application services).
An Infrastructure Performance Management (IPM) platform like VirtualWisdom from
Virtual Instruments ( can accomplish that, at least from a storage transaction performance perspective. Because application transaction performance is so closely linked to infrastructure performance, monitoring and managing infrastructure
performance provides a mechanism for defining and monitoring performance-based
SLAs without having to instrument each and every application.  In addition, VirtualWisdom provides both the service provider and end-user with real-time and historical infrastructure performance metrics. This visibility and transparency will enable end-users to have more
trust in the service they're receiving and allow service providers to create
SLAs that align with not only application availability needs but also with
performance requirements.

Minor correction

The number of IT pros surveyed for this study was 740 not 704, Compuware tells us.