Primary Storage and The Cloud

A fellow Byte and Switch Blogger recently wrote about Emulex's new Cloud Adapter and I think correctly nailed it as essentially a write splitter. While I as hung up on whether or not they get to call this a Cloud Adapter, I do think it is one of the very first steps in how the cloud will be involved in the block I/O path.

As I wrote about a few entries ago, the first use case for primary storage in the cloud is well under way, essentially being used by software as a service providers for their internal storage. Actual interactive use of cloud storage is as an archive or as a slow file share for collaboration and of course as a backup destination.

Make no mistake however, these and the Emulex adapter are all just first steps, within a few years many small to medium sized data centers will be using the cloud as their primary storage platform. Before you race down to the comments section and let me know that I'm insane, let me explain.

The technology we need to implement primary storage in the cloud already exists and it has for decades; its called a cache. It's absolutely true that even with a few years of progress, internet bandwidth won't sustain the I/O rates that most applications need but we can cache that I/O locally and then move it to the cloud as time allows. Basically what would happen is that you would install a relatively small, disk cache locally that would store active blocks of data, copy those changing blocks to the cloud as bandwidth allowed and then as those blocks aged they would remove the copy off the local cache.

Big portions of this technology exist today. We have file level cacheing on WAN's, cloud based archives that deploy an appliance locally and migrate data to the cloud, block level data movement on storage arrays that can move old blocks of data to different tiers of storage based on access and we have the ever decreasing cost of solid state disk.One of the biggest areas that would need to be addressed is the individual management of this multi-tenant environment. As we discussed in an article "Virtual Domains" this technology needs to be in place before cloud based primary storage could be successful. Most businesses will want to define their own LUNS and manage their own storage, of course with the cloud outsourcing that could be easier as well.

The final part of the puzzle is putting this all together into a single solution from a single provider. OK that's the hard part, but I did give it three years to come together and clearly cloud storage providers have ample ways to provide services until the technology matures, point being they will be well established when the technology does become available.