Virtual SANs: Scale Out's Next Iteration

Storage system design often seems as if it's stuck in the Big Iron era of mainframes, a consolidated world of enormous, expensive machines managed by a high priesthood of experts. Yet as computing has become virtualized and democratized, with distributed systems knitted into self-service clouds where every developer can create his own dedicated sandbox, mainframes have had to adapt.

Storage systems are following along, as scale-out designs using distributed object and file systems have become a popular technology for providing large, scalable storage pools over an Ethernet backbone. The concept was commercialized over a decade ago by pioneers like EqualLogic, Isilon and Spinnaker, which were acquired, by Dell, EMC and NetApp, respectively.

Yet useful as they are, and indeed I have long recommended scale-out systems as alternatives to traditional large, centralized storage frames, as both a more efficient and cost effective means of providing shared storage, they've lacked all-purpose adaptability because they can't provide block-level storage required for databases and DB-backed applications.

That's changing as the concepts underlying distributed, networked file systems collide with the virtual servers and cloud stacks. This fusion could ultimately threaten the hegemony of traditional SANs over mission-critical, transaction-oriented, back-office applications.

One of the first to marry a scale-out design with SAN versatility was Coraid, which emerged from a project in the Linux community to develop a native Ethernet storage protocol, ATA over Ethernet (AoE) that operated at Layer 2, thus eliminating the IP overhead plaguing iSCSI. But unlike FCoE, AoE, which as the name implies encapsulates the SATA command set within Ethernet frames, was made with simplicity in mind. It is much more efficient than FCoE, which shoehorns the Fibre Channel stack into a physical layer it wasn't originally designed for.

[ Join us at Interop Las Vegas for access to 125+ IT sessions and 300+ exhibiting companies. Register today! ]

Coraid, which has been shipping its EtherDrive storage nodes for several years, long struggled to gain much visibility outside niches in academia, government/military and hosting providers, but appears to be catching on as pan-virtualized, cloud infrastructure gradually seeps from early adopters to mainstream enterprise IT. In retrospect, it looks to be a case of a company (and technology) ahead of its time, coupled with plenty of competitor-incited FUD about a scary new storage protocol.

Coraid's scale-out virtual SAN comes in two pieces. The building blocks are its EtherDrive storage nodes. These are typical scale-out appliances featuring 16 to 36 drive bays sporting either mechanical disks or SDDs. They use AoE to present raw storage volumes as LUNs to any system running an AoE stack. (Drivers are available for Linux, Windows, OS X, Solaris, VMware and OpenBSD.) But in Coraid's first instantiation, each host was responsible for attaching to LUNs on different storage bricks; in other words, volumes couldn't natively span nodes. Instead, the host was responsible for setting up RAID stripes across multiple EtherDrives. It was still a nicely distributed system, in that volumes could withstand both multidisk and multinode failures because the storage nodes themselves were already RAID protected, but wasn't a fully virtualized SAN.

Coraid eliminated this shortcoming a couple of years ago with the VSX SAN virtualization appliance, a device that can create so-called macro LUNs that stripe across multiple nodes. It includes features de rigueur for any enterprise SAN, including synchronous mirroring, asynchronous remote replication, cloning, snapshots and thin provisioning. The final piece of the puzzle came when the company introduced management and automation software, EtherCloud, that simplifies the deployment to the point of allowing self-service storage provisioning for virtualized applications and can handle upwards of petabytes of pooled capacity.

Next page: Rethinking SANs