Data centers

09:06 AM
Connect Directly
LinkedIn
Google+
Twitter
RSS
E-Mail
50%
50%

Guide To IaaS Cloud Provider Performance

Choosing an Infrastructure-as-a-service cloud provider? Use these benchmarks to compare AWS, Google, Azure, Rackspace, and more.

I've been frustrated by a lack of comprehensive comparison benchmarks between Infrastructure-as-a-Service (IaaS) providers and within providers themselves. Most performance data shows only a small number of available "instance types" in a limited region, with no way to compare pricing or application performance.

As an Amazon Web Services customer, I know from my own experience that the c1.xlarge performs really well for many applications, but no one appears to have ever compared that to other cloud services.

So, I decided to run my own IaaS performance project and benchmarked every instance type across the major public IaaS provider regions on a consistent setup. My primary focus was to provide a broad, comprehensive view of most serious compute options available to IaaS customers.

Part 1: Test Methodology
First, I explained my goals for the benchmarks and how I used the tools UnixBench and SysBench’s MySQL to test different instance types in different data centers across different providers.

Part 2: Amazon Web Services
I launched more than 175 AWS VMs across instance types and compared performance by region and availability zone. Here you can read the detailed results.

Part 3: Amazon Web Services C3
After Amazon rolled out the C3 family of instances, I ran the benchmarking tests using C3 and updated my AWS charts accordingly.

Part 4: Google Compute Engine
I compared instance types across Google Compute Engine and against AWS, finding it a solid competitor to Amazon's offerings.

Part 5: Microsoft Azure Virtual Machines
I tested Windows Azure Virtual Machines and compared Azure to AWS and Google Compute Engine. Here you'll find the results.

Part 6: Rackspace Cloud Servers
I evaluated Rackspace Cloud Servers' compute resources and how they compare to AWS, Google Compute Engine, and Azure. See how they measure up.

Part 7: SoftLayer
In this part, I looked at SoftLayer, an IBM company, which offers bare-metal (not virtualized) pay-by-the-hour services.

Part 8: HP Public Cloud
I tested HP Public Cloud across regions and compared it to the other IaaS providers I've benchmarked. HP has recently focused on providing an OpenStack-based public cloud.

Do you have experience with these IaaS providers? Let us know how they fared in the comments.

 

Joe began his career by winning the 1996 Weird Software Contest with the Mutant Chicken Races and creating the first Windows-based iPod application. Over the past ten years, Joe transitioned from development to systems design and data analysis, creating the first BuildFax ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Threaded  |  Newest First  |  Oldest First
MarciaNWC
50%
50%
MarciaNWC,
User Rank: Strategist
7/23/2014 | 11:59:06 AM
Rackspace
I wonder how the uncertainty about Rackspace's future is affecting the IaaS landscape. The company announced in May that it was evaluating its options, but has yet to find a buyer.  
kbannan100
50%
50%
kbannan100,
User Rank: Apprentice
7/23/2014 | 2:20:29 PM
Useful for private cloud implementations?
I also wonder how applicable this strategy might be for those looking to provide similar services such as IT-as-a-Service or other IaaS offerings internally? I would think it is just as important to create benchmarks and/or measure internal services against external benchmarks. It's only then that you can assure users that you're giving them everything and then some when it comes to performance and offerings.

--KB

Me: http://bit.ly/1iMdSE5  

 
Jeff Jerome
50%
50%
Jeff Jerome,
User Rank: Apprentice
7/31/2014 | 11:24:45 PM
Re: Useful for private cloud implementations?
IT as a service - is getting very popular and helps keep certain level clients of cost down.  And at the same time these vendors can keep up with the multitude of technologies by dispersing across many people.
KenethB
50%
50%
KenethB,
User Rank: Apprentice
8/19/2014 | 7:42:30 PM
Alternative IaaS Providers
This is definitely a nice thing you put together, but I would be interested in seeing it expanded to see how a few more providers like Digital Ocean, Atlantic.net, and Linode compare against the players you mentioned.
Susan Fogarty
50%
50%
Susan Fogarty,
User Rank: Strategist
8/20/2014 | 9:22:46 AM
Re: Alternative IaaS Providers
KennethB, thanks for the feedback. We'll look into doing some expanded testing. Do you use any of the other providers you listed?
Cartoon
Slideshows
Audio Interviews
Archived Audio Interviews
Jeremy Schulman, founder of Schprockits, a network automation startup operating in stealth mode, joins us to explore whether networking professionals all need to learn programming in order to remain employed.
White Papers
Register for Network Computing Newsletters
Current Issue
2014 Private Cloud Survey
2014 Private Cloud Survey
Respondents are on a roll: 53% brought their private clouds from concept to production in less than one year, and 60% ­extend their clouds across multiple datacenters. But expertise is scarce, with 51% saying acquiring skilled employees is a roadblock.
Video
Twitter Feed