This is the seventh part in a series of articles about creating my own IaaS performance benchmarking project. In the first part, I explained my methodology for testing instance types across major IaaS providers. I've run benchmarks for Amazon Web Services, Google Compute Engine, Microsoft Azure Virtual Machines, and Rackspace Cloud Servers. In this part, I look at SoftLayer, which was acquired by IBM last year.
SoftLayer is unique among the IaaS providers I'm testing in that it has bare-metal (meaning not virtualized) pay-by-the-hour services that some industry observers think may give the company a significant edge.
As of today, there are six U.S. regions available for SoftLayer through RightScale (the cloud management service I’m using to run most of my benchmarks, as it gives me image parity): San Jose, Seattle, Washington, D.C., and three in Dallas (Dallas 1, Dallas 5, and Dallas 6). SoftLayer offers extremely configurable servers (far more so than the other major IaaS providers, which offer specific “instance types”), but RightScale has created five instance types for SoftLayer. I tested those, so I launched a total of 30 VMs through RightScale.
In addition, I also launched seven different bare-metal instances through SoftLayer’s own control panel since RightScale doesn't yet support SoftLayer's bare-metal servers. Consequently these aren’t as apples-to-apples as staying within RightScale, but I did run CentOS 6.4 on each of them, and I think the results are valid enough to help spur further investigation.
SoftLayer has what appears to be the simplest discounting system -- much better than AWS’s Reserved Instances or Rackspace’s commitment plus prepayment discounts. SoftLayer offers discounts from roughly 5% to 30% if you purchase servers by the month instead of by the hour.
As I’ll detail below, SoftLayer’s bare-metal performance looks pretty solid. However, launching bare-metal instances was a different experience than launching virtualized instances. Most of the virtualized instances came up in under 10 minutes, but the bare-metal instances took at least an hour, and in most cases, closer to five hours from order to operational. At least two of my bare-metal instances required human intervention; they were apparently launched running CentOS 6.3, but required upgrading to CentOS 6.4 before they could be made operational. (Never mind that the “order form” for them only allowed me to choose CentOS 6.0).
Also, I was unable to get any bare-metal instance with fewer than eight cores and 8GB of RAM. I ordered 2-core/2GB RAM instances on five separate occasions in different data centers and got 8-core/8GB machines every time. However, SoftLayer assured me that I would be charged the 2-core/2GB price for these, and I was -- only 50 cents/hour.
On one hand, SoftLayer’s bare-metal instances can be launched via the same API as its virtualized instances, and you can pay for them by the hour, both of which make them quite cloud-like. On the other hand, I’m not exactly sure what all is required of SoftLayer to launch or terminate bare-metal instances, but my impression is that some kind of human intervention is required for them. You’ll have to decide for yourself whether they meet your definition of cloud computing.
Here are the specifications of the instances I launched with SoftLayer:
SoftLayer Benchmark Results
Here are the main takeaways that I have from testing across SoftLayer:
• SoftLayer has the biggest differences in performance between regions that I’ve seen. The Washington, D.C. region was underwhelming to say the least, and Dallas 5 and 6 were significantly better than the rest. Washington, D.C. had the worst or second-worst score for each instance type, and Dallas 5 and 6 were in the top three regions for every instance type, except for Extra Small.
• Like other clouds, the best price-per-performance for single-core UnixBench is the cheapest one; here, it's the Extra Small. However, unlike other clouds, the bare-metal instances actually have similar price-per-performance to the Extra Small. (And I would suspect that the bare-metal instance types with fewer cores would do even better, but as stated above, when I tried to order one with two cores, I ended up with an instance with eight cores).
[IBM says it's devised a method for avoiding cloud performance problems by dynamically managing network bandwidth in cloud computing environments. Get the details in "IBM Invention Aims To Fix Cloud Bottlenecks."]
• If the bare-metal instances are “cloud” enough for you, there are limited use cases for running virtual instances with SoftLayer (e.g. cheap instances, ephemeral instances, lots of RAM), since the performance difference is dramatic. The bare-metal instances are so much better in terms of price-per-performance that your need for the virtual instances will be limited. That said, if you buy SoftLayer for the bare metal, it will be nice to have the option to have the virtualized instances within the same networks/data centers within minutes.
NEXT: Comparing Soft layer to Other IaaS ProvidersJoe began his career by winning the 1996 Weird Software Contest with the Mutant Chicken Races and creating the first Windows-based iPod application. Over the past ten years, Joe transitioned from development to systems design and data analysis, creating the first BuildFax ... View Full Bio