Cloud Infrastructure

11:49 AM
Connect Directly

IaaS Performance Benchmarks Part 6: Rackspace Cloud Servers

As part of my project comparing IaaS services, I tested Rackspace Cloud Servers and also compared Rackspace to other IaaS providers. Here are the results.

This is the sixth part in a series of articles about creating my own IaaS performance benchmarking project. In the first part, I explained my methodology for testing instance types across major IaaS providers. I've run benchmarks for Amazon Web Services, Google Compute Engine, and Microsoft Azure Virtual Machines. In this part, I look at Rackspace Cloud Servers, which have been around for a long time and been though a few revisions.

Rackspace focuses on support and its managed hosting options, and I believe it is the only major IaaS provider that offers a built-in plan for on-demand servers that includes managed hosting support. However, for the purposes of these benchmarks, I am only considering the features and prices of the compute resources without support services.

As of today, there are three U.S. regions available for Rackspace through RightScale (the cloud management service I’m using to run most of my benchmarks, as it gives me image parity): Chicago, Dallas, and Virginia. In each of those regions, I kicked off one of each “Performance” and “Standard” instance type that Rackspace offers, for a total of 48 VMs. Rackspace announced its Performance Cloud Servers in November, when it also renamed its "Next-Gen" instance types as Standard.

As with the previously-tested cloud providers, I ran on RightScale’s CentOS 6.4 images. Rackspace offers a somewhat-confusing discount program based on how long you’re willing to commit and whether you’ll pay up front. According to Rackspace, it is possible to get up to a 37% discount on its pricing with a three-year commitment and paying up front, so I have used a 37% discount as the “discounted pricing” amount.

Rackspace's Performance servers in theory should be competitive with Amazon’s newer instance types (e.g., M3, C3, I2) and Google Compute Engine. For example, the Performance instance types apparently run on Intel Xeon E5-2670 2.60GHz processors; AWS lists its M3 family as running on the E5-2670, the C3 family on the E5-2680, the I2 family on the E5-2670v2, and GCE says it uses 2.6GHz+ Xeon Sandy Bridge processors. However, on CentOS 6.4, I found that Rackspace's Performance instance types really didn’t perform significantly better than its Standard instance types.

Why? This description in a Rackspace developer blog post offers some clues: “So, terminology aside, today, when you boot any Linux image on Rackspace Cloud, that image uses standard paravirtualization (PV), for Windows and FreeBSD we use HVM. In the case of the new Performance Cloud Servers, we still default to PV images, and are currently testing PV on HVM images.”

But what exactly does this mean? Unfortunately, Rackspace’s own support portal has no useful information on PV vs. hardware-assisted virtualization (HVM), even though here, it means the difference between UnixBench scores that are competitive with AWS and GCE and scores that end up well below Azure. The short story is that you should see substantially higher UnixBench scores using HVM virtualization, at least on the processors being used. By not having HVM virtualization available yet for its Performance instance types for Linux, Rackspace is not offering a particularly competitive product today.

[Security concerns are driving organizations to consider private and hybrid clouds, according to a Rackspace survey last year. Read the report in "Private, Hybrid Cloud Interest Spurred By Security and Control."]

Based on conversations with several individuals at Rackspace, I believe (a) that Rackspace will have HVM support for Linux for its Performance instance types at some point in the first half of 2014 (note that this is my guess, not anything they said), and (b) that it will probably perform similarly to the benchmarks run on the beta version that Rackspace has made available, which gives numbers more than 3X better than what I got.

I will discuss my results by including both the “what if Rackspace supported HVM for Linux today” and the “what I actually got” perspectives. I should note that Rackspace has graciously given me access to a pre-release version of a CentOS image that can be launched with HVM on the Rackspace Cloud, which I plan to test before the conclusion of this IaaS benchmark series.

Rackspace Benchmark Results

Here are the main takeaways that I have from testing across Rackspace:

• For the first time in testing IaaS providers, I saw a regional difference: Rackspace Chicago delivered the three top single UnixBench scores (from the three most expensive instance types), and five of the top ten, but also five of the six worst scores (from the least expensive instance types). This requires further investigation to draw any significant conclusion, since I ran all these in a single day.

• The best price-per-performance for me was with the 512MB Standard, with the 1GB Performance a distant second. Of the more beefy instance types, the 8GB Performance stands out as having much better price-per-performance than the others (likely because it has eight vCPUs; you have to go to the 60GB Performance to get more than that).

• Performance in the single-threaded UnixBench results does appear to vary, at least a little bit, based on price, and the Performance types do better than the Standard types, even with everything running on PV virtualization. In particular, the 8GB, 15GB, 30GB, 60GB, 90GB, and 120GB Performance types all scored at least 544 (average) on the single-threaded UnixBench scores, whereas the rest of the instance types all scored 493 or less (average). However, under HVM virtualization, it appears that results over 1500 should be expected.

View Larger

View Larger

NEXT: Comparing Rackspace To Other IaaS Providers

Joe began his career by winning the 1996 Weird Software Contest with the Mutant Chicken Races and creating the first Windows-based iPod application. Over the past ten years, Joe transitioned from development to systems design and data analysis, creating the first BuildFax ... View Full Bio
1 of 2
Comment  | 
Print  | 
More Insights
Hot Topics
What To Do If Your Cloud Provider Is Acquired
Andrew Froehlich, President & Lead Network Architect, West Gate Networks,  8/29/2014
VMware NSX Banks On Security
Marcia Savage, Managing Editor, Network Computing,  8/28/2014
The Rise Of White-Box Storage
Jim O'Reilly, Consultant,  8/27/2014
White Papers
Register for Network Computing Newsletters
Current Issue
2014 Private Cloud Survey
2014 Private Cloud Survey
Respondents are on a roll: 53% brought their private clouds from concept to production in less than one year, and 60% ­extend their clouds across multiple datacenters. But expertise is scarce, with 51% saying acquiring skilled employees is a roadblock.
Twitter Feed