Cloud Infrastructure

11:49 AM
Connect Directly
Repost This

IaaS Performance Benchmarks Part 6: Rackspace Cloud Servers

As part of my project comparing IaaS services, I tested Rackspace Cloud Servers and also compared Rackspace to other IaaS providers. Here are the results.

This is the sixth part in a series of articles about creating my own IaaS performance benchmarking project. In the first part, I explained my methodology for testing instance types across major IaaS providers. I've run benchmarks for Amazon Web Services, Google Compute Engine, and Microsoft Azure Virtual Machines. In this part, I look at Rackspace Cloud Servers, which have been around for a long time and been though a few revisions.

Rackspace focuses on support and its managed hosting options, and I believe it is the only major IaaS provider that offers a built-in plan for on-demand servers that includes managed hosting support. However, for the purposes of these benchmarks, I am only considering the features and prices of the compute resources without support services.

As of today, there are three U.S. regions available for Rackspace through RightScale (the cloud management service I’m using to run most of my benchmarks, as it gives me image parity): Chicago, Dallas, and Virginia. In each of those regions, I kicked off one of each “Performance” and “Standard” instance type that Rackspace offers, for a total of 48 VMs. Rackspace announced its Performance Cloud Servers in November, when it also renamed its "Next-Gen" instance types as Standard.

As with the previously-tested cloud providers, I ran on RightScale’s CentOS 6.4 images. Rackspace offers a somewhat-confusing discount program based on how long you’re willing to commit and whether you’ll pay up front. According to Rackspace, it is possible to get up to a 37% discount on its pricing with a three-year commitment and paying up front, so I have used a 37% discount as the “discounted pricing” amount.

Rackspace's Performance servers in theory should be competitive with Amazon’s newer instance types (e.g., M3, C3, I2) and Google Compute Engine. For example, the Performance instance types apparently run on Intel Xeon E5-2670 2.60GHz processors; AWS lists its M3 family as running on the E5-2670, the C3 family on the E5-2680, the I2 family on the E5-2670v2, and GCE says it uses 2.6GHz+ Xeon Sandy Bridge processors. However, on CentOS 6.4, I found that Rackspace's Performance instance types really didn’t perform significantly better than its Standard instance types.

Why? This description in a Rackspace developer blog post offers some clues: “So, terminology aside, today, when you boot any Linux image on Rackspace Cloud, that image uses standard paravirtualization (PV), for Windows and FreeBSD we use HVM. In the case of the new Performance Cloud Servers, we still default to PV images, and are currently testing PV on HVM images.”

But what exactly does this mean? Unfortunately, Rackspace’s own support portal has no useful information on PV vs. hardware-assisted virtualization (HVM), even though here, it means the difference between UnixBench scores that are competitive with AWS and GCE and scores that end up well below Azure. The short story is that you should see substantially higher UnixBench scores using HVM virtualization, at least on the processors being used. By not having HVM virtualization available yet for its Performance instance types for Linux, Rackspace is not offering a particularly competitive product today.

[Security concerns are driving organizations to consider private and hybrid clouds, according to a Rackspace survey last year. Read the report in "Private, Hybrid Cloud Interest Spurred By Security and Control."]

Based on conversations with several individuals at Rackspace, I believe (a) that Rackspace will have HVM support for Linux for its Performance instance types at some point in the first half of 2014 (note that this is my guess, not anything they said), and (b) that it will probably perform similarly to the benchmarks run on the beta version that Rackspace has made available, which gives numbers more than 3X better than what I got.

I will discuss my results by including both the “what if Rackspace supported HVM for Linux today” and the “what I actually got” perspectives. I should note that Rackspace has graciously given me access to a pre-release version of a CentOS image that can be launched with HVM on the Rackspace Cloud, which I plan to test before the conclusion of this IaaS benchmark series.

Rackspace Benchmark Results

Here are the main takeaways that I have from testing across Rackspace:

• For the first time in testing IaaS providers, I saw a regional difference: Rackspace Chicago delivered the three top single UnixBench scores (from the three most expensive instance types), and five of the top ten, but also five of the six worst scores (from the least expensive instance types). This requires further investigation to draw any significant conclusion, since I ran all these in a single day.

• The best price-per-performance for me was with the 512MB Standard, with the 1GB Performance a distant second. Of the more beefy instance types, the 8GB Performance stands out as having much better price-per-performance than the others (likely because it has eight vCPUs; you have to go to the 60GB Performance to get more than that).

• Performance in the single-threaded UnixBench results does appear to vary, at least a little bit, based on price, and the Performance types do better than the Standard types, even with everything running on PV virtualization. In particular, the 8GB, 15GB, 30GB, 60GB, 90GB, and 120GB Performance types all scored at least 544 (average) on the single-threaded UnixBench scores, whereas the rest of the instance types all scored 493 or less (average). However, under HVM virtualization, it appears that results over 1500 should be expected.

View Larger

View Larger

NEXT: Comparing Rackspace To Other IaaS Providers

1 of 2
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Apprentice
3/12/2014 | 5:11:51 PM
re: IaaS Performance Benchmarks Part 6: Rackspace Cloud Servers
Mr Joe Please can you tell me which tools you use to ran this benchmark or how you ran I'll be very happy and appreciate.
Joe Masters Emison
Joe Masters Emison,
User Rank: Apprentice
1/3/2014 | 9:31:22 PM
re: IaaS Performance Benchmarks Part 6: Rackspace Cloud Servers
I think there are customers who want what they believe is the benefit of having expertise in the "managed" model, but I think that we are moving to a world where the best-run servers are orchestrated, not managed. This is similar to the history of (much) manufacturing--if you can automate things without human involvement, then you will generally have better (and likely cheaper) processes.

So then the question becomes whether the right vehicle to get an orchestrated architecture in place is "managed hosting", and I am dubious that it is. I think most of the sysadmins/experts who "manage" hosting don't have the proper DevOps perspective. You're really looking for a good cloud architect who doesn't want to manage servers, they want to manage an application infrastructure. Maybe Rackspace is providing that, but when they price their "managed hosting" as part of a server's hourly price, I just don't think they're signaling that they get modern cloud architectures.
User Rank: Strategist
1/3/2014 | 7:28:38 PM
re: IaaS Performance Benchmarks Part 6: Rackspace Cloud Servers
Hi Joe -- Do you think there are use cases where the managed model works better?
More Blogs from Commentary
Infrastructure Challenge: Build Your Community
Network Computing provides the platform; help us make it your community.
Edge Devices Are The Brains Of The Network
In any type of network, the edge is where all the action takes place. Think of the edge as the brains of the network, while the core is just the dumb muscle.
Fight Software Piracy With SaaS
SaaS makes application deployment easy and effective. It could eliminate software piracy once and for all.
SDN: Waiting For The Trickle-Down Effect
Like server virtualization and 10 Gigabit Ethernet, SDN will eventually become a technology that small and midsized enterprises can use. But it's going to require some new packaging.
IT Certification Exam Success In 4 Steps
There are no shortcuts to obtaining passing scores, but focusing on key fundamentals of proper study and preparation will help you master the art of certification.
Hot Topics
Fight Software Piracy With SaaS
Andrew Froehlich, President & Lead Network Architect, West Gate Networks,  4/22/2014
Edge Devices Are The Brains Of The Network
Orhan Ergun, Network Architect,  4/23/2014
White Papers
Register for Network Computing Newsletters
Current Issue
2014 Private Cloud Survey
2014 Private Cloud Survey
Respondents are on a roll: 53% brought their private clouds from concept to production in less than one year, and 60% ­extend their clouds across multiple datacenters. But expertise is scarce, with 51% saying acquiring skilled employees is a roadblock.
Twitter Feed