Joe Masters Emison

Upcoming Events

Where the Cloud Touches Down: Simplifying Data Center Infrastructure Management

Thursday, July 25, 2013
10:00 AM PT/1:00 PM ET

In most data centers, DCIM rests on a shaky foundation of manual record keeping and scattered documentation. OpManager replaces data center documentation with a single repository for data, QRCodes for asset tracking, accurate 3D mapping of asset locations, and a configuration management database (CMDB). In this webcast, sponsored by ManageEngine, you will see how a real-world datacenter mapping stored in racktables gets imported into OpManager, which then provides a 3D visualization of where assets actually are. You'll also see how the QR Code generator helps you make the link between real assets and the monitoring world, and how the layered CMDB provides a single point of view for all your configuration data.

Register Now!

A Network Computing Webinar:
SDN First Steps

Thursday, August 8, 2013
11:00 AM PT / 2:00 PM ET

This webinar will help attendees understand the overall concept of SDN and its benefits, describe the different conceptual approaches to SDN, and examine the various technologies, both proprietary and open source, that are emerging. It will also help users decide whether SDN makes sense in their environment, and outline the first steps IT can take for testing SDN technologies.

Register Now!

More Events »

Subscribe to Newsletter

  • Keep up with all of the latest news and analysis on the fast-moving IT industry with Network Computing newsletters.
Sign Up

See more from this blogger

IaaS Performance Benchmarks Part 3: AWS C3

Editor's note: This post was updated Monday with information from Amazon after initial publication.

This is the third part in a series of articles about creating my own IaaS performance benchmarking project. In the first part, I explained the methodology I'm using to test instance types across major IaaS providers. In the second part, I ran benchmarks across every instance type in every U.S. region run by Amazon Web Services (AWS), the most widely used IaaS provider. In this part, I look specifically at the new C3 family of instances that AWS launched at its re:Invent conference, and update my AWS charts accordingly.

More Insights


More >>

White Papers

More >>


More >>

Amazon Web Services has been around long enough to want to deprecate/retire some of its instance types, as seen both by the excellent M3 family (replacing the M1 instance types) and now the new C3 family (replacing the C1 instances).

While it appeared to me initially that Elastic Block Storage (EBS) backing was required for the new C3 instances, Amazon informed me that's not the case. In my testing, I used EBS-backed C3s.

Two important notes about these updated benchmarks. First, in order to get these results, you must run C3 instances on Hardware Virtual Machine (HVM) EBS volumes, not ordinary EBS volumes. AWS will let you run on regular EBS volumes, but there is absolutely no reason to do this, because it costs the same to run under HVM EBS volumes, and the performance is significantly better. (For more on this, take a look at my first set of AWS results and my comparison of EBS-backed M3 instances versus HVM EBS-backed M3 instances).

Second, I'm dropping the SysBench results from these and further benchmarks, because, after hearing significant feedback from benchmarking experts, I don’t think they’re adding anything more to these foot-deep, miles-wide analyses across IaaS vendors. Now onto the results.

[Read about a new initiative to develop a framework that provides secure connectivity from any device to a cloud application in "Cloud Security Alliance Launches Secure Network Effort."]

To test the C3 family, I launched 30 AWS VMs -- five new C3 instance types each in six different availability zones (AZs) in US-East and US-Oregon (you can't launch HVM EBS-backed instances in US-West). As I found with the other AWS instance types, the C3 instance types had consistent performance for each instance type.

My main takeaway: the C3 family is pretty fantastic, and really sets a new bar for affordable performance. Many people use the c1.medium and c1.xlarge instances for Web servers and databases, and if you do, switch to the C3 posthaste, as you get significantly better performance for essentially the same price.

View Larger

If you’re running single-threaded, general-purpose applications, you’re crazy if you’re not using the C3 or M3 families. The c3.large, in particular, is really fantastic, at 15 cents per hour on demand. And, in general, unless you can’t afford 15 cents/hour, it’s hard to make a case that you should be running any of the M1 or C1 instances any more.

View Larger

The one-thread-per-core UnixBench results encompass such a large range on AWS that it’s necessary to break them into two charts in order to understand the range across instance types. I have chosen to do this on price, as opposed to performance, but please note that there are instances on the cheaper chart that handily beat instances on the more expensive chart in these UnixBench scores.

View Larger

One interesting note is that the C3 series doesn’t quite scale linearly with respect to these UnixBench scores: The average score jumps from 1867 to 2966 to 4876 to 6924 to 8472, but the price doubles each time. These benchmarks are running on CentOS (we saw previously in this series that running on Ubuntu improved the scores significantly), and I’m using RightScale’s CentOS images to get parity across IaaS vendors, so take the specific scaling here with a grain of salt. However, I would strongly suggest that you test how well they scale for your application, because it may make sense to stick with the c3.large and scale horizontally.

All in all, the C3 family looks excellent in these benchmarks, and if you’re talking to other IaaS vendors that are showing you comparison benchmarks, you need to make sure that they’re doing a comparison against the C3s. Note, though, they almost certainly aren’t -- I haven’t seen a set of benchmarks touted by another IaaS vendor that ventured outside of the deprecated M1 family.

In the next few weeks, I’ll post my results from Google Compute Engine and Windows Azure, and comparing them to the best-performing AWS instance types. And again, please remember the aim of this benchmarking project: I’m trying to give all of us a high-level view on IaaS offerings. I would advise you to do further testing, based upon what you can afford.

Joe Masters Emison oversees the award-winning BuildFax cloud architecture and frequently speaks on cloud topics.

Related Reading

Network Computing encourages readers to engage in spirited, healthy debate, including taking us to task. However, Network Computing moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing/SPAM. Network Computing further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | Please read our commenting policy.
Vendor Comparisons
Network Computing’s Vendor Comparisons provide extensive details on products and services, including downloadable feature matrices. Our categories include:

Research and Reports

Network Computing: April 2013

TechWeb Careers