EMC Lays Down The Virtual Storage Gauntlet

In a recent event focused on "virtual storage," EMC fearlessly stated that "this changes everything." Thus, the company audaciously claims to "boldly go where no one has gone before."  We industry analysts love that type of claim as it stirs up controversy, discussion and the need to write millions of words analyzing and dissecting its meaning, value and validity not only now, but also over the next few years. So, in a thumbnail sketch (relatively speaking) rather than a million words, let's begin to examine what is happening and what it means.

The virtual storage concept was announced by Pat Gelsinger, president of EMC's Information Infrastructure Products Group, to a gathering of industry analysts in person and on the phone. Mr. Gelsinger had a long and illustrious career at Intel (29 1/2 years) and moved to EMC six months ago for the opportunity to get involved in the introduction of disruptive technologies. Enter virtual storage.

Mr. Gelsinger, followed up later by Chuck Hollis, VP of Global Marketing and CTO, reviewed in their blogs some of the key concepts underlying EMC's vision, starting with the private cloud. EMC's view is that the private cloud is the seamless integration of the virtualized data center, the internal cloud created by organizations with large and mid-size data center and external cloud computing resources offered by service providers and others. The key characteristic of a private cloud is that IT is still held accountable for its results, meaning that they have to have control over IT resources and the service delivery that is associated with those resources.

EMC believes that the goal is to create progressively larger and larger pools of resources, enabling better optimization and flexibility. The term "federation" is used to describe that orchestrated pooling of resources, and breaks down into two types: federation of resources (computing, storage and networking) and federation of control (resource allocation, service delivery, security and compliance among them). Federation also aggregates and extends the view of resources across geographies.

Virtual Storage
In essence, virtual storage is the separation of information from physical storage media and location.  It still allows IT control over information no matter where it is in the world, and that means in both internal and external data centers as part of the private cloud. Also, to be clear, virtual storage is not storage virtualization, which is primarily about better managing the storage on one or more arrays in a single data center. Virtual storage is the separation of information (which is logical) from storage (which is physical). That does not mean that information does not have to reside on physical storage, but the application owner and the user does not have to be aware of which piece of physical media that the information resides on.By way of comparison, you could say that virtual storage is taking a path similar to server virtualization. Round one was about consolidation, i.e., putting multiple virtual machines (VMs) on one physical server to improve server utilization. In server virtualization, round two was about moving VMs from one physical server to another for load-balancing or other reasons. On the storage side, round two leads to new approaches to managing storage-on-demand, dynamic resizing and relocation of workloads and improved availability.

Now federation enables cooperation among pools of server and storage resources necessary to bring about IT as a service through dynamic application availability and data movement where even global distances are not a barrier.

That sounds wonderful, so why hasn't federation on a global basis happened before? In limited senses, it has, but up to now, the general and universal problem of actually how to do virtual storage on a global basis, rather than just think about it, had not been solved. There have been three challenges to coming up with a global solution: latency, bandwidth and consistency. As EMC points out, location matters in order to improve protection and availability of information as well as optimize costs and utilization. In other words, EMC's view is that it is necessary to exploit distance if we are going to get to the "this changes everything" stage. Yet those three challenges -- latency, bandwidth and consistency -- get in the way of a general solution.

Still, the answer to the challenge has been known for some time -- use caching. The problem in getting to the answer is something called "distributed cache coherency," which addresses the three challenges of latency, bandwidth and consistency. In fact, the problem has been around for a long time (for example, see a technical paper "Design and implementation of a distributed cache coherence protocol," published in April 1990 -- 20 years ago!). That indicates that the problem has been an intractable one

Gelsinger really didn't go into any detail on how EMC has solved this problem or what kind of intellectual property (IP) they have derived as a result. However, the company is confident that it has solved the distributed cache coherency problem. EMC is now publicly committed to being able to build practical storage federation using virtual storage on top of distributed cache coherence. Gelsinger used the meeting to make statements of direction rather than product announcements. However, without making a formal product announcement, he stated that an appliance-based product will be the first fruit of EMC's virtual storage concept. One might expect its debut in May at EMC World in Boston.My Take
Recall the famous query of Benjamin Franklin, "Of what use is a new born baby?" Well, with virtual storage, in the sense of EMC's first appliance-form instantiation, we have a little bit of a wait before its first delivery. Yes, that "baby" will have a lot of potential, but it is unfair at this early stage to have unrealistic expectations as to when that potential will be fulfilled. Virtual storage needs to be given the breathing room and time to mature.

That said, EMC's initial vision of virtual storage is block-based (which is fine), but it obviously needs to be connected to the file-based world. That connection is critical if virtual storage is to realize its full potential. As it stands now, virtual storage deals with data, not with information, and a fine-grained knowledge of information is required to enable a full range of policies to be applied. Mr. Gelsinger confirmed that will happen at some point in the future, and it does not need to be available day one as a great deal can be accomplished using block-based storage alone.

All that said, we believe EMC's virtual storage baby is likely to be beautiful and much can be done and accomplished before file and deep information knowledge is added in. Overall, EMC should be given a great deal of credit for directly addressing the critical issue of distributed cache coherency and being able to promulgate initial virtual storage solutions which moves us closer to the private cloud. Overall, it will be fun to explore all the implications, ramifications and competitive reactions as virtual storage grows from infancy to adulthood.