Purpose-Built Or Off-The-Shelf Hardware: A Tale Of Two Systems

Until recently, the conventional wisdom has been that building a high-performance storage system required custom silicon. EMC’s Symmetrix, HDS’s USP, BlueArc’s Titan and HP’s 3Par systems are all crammed full of custom chips. However, with AMD and Intel boosting the performance of x86 processors, some industry observers, including our own Stephen Foskett, have suggested that the days of custom hardware are numbered. Recent solid-state storage system announcements from Kaminario and Astute Networks demonstrate that this argument is far from over.

Kaminario’s K2 takes the off-the-shelf approach, turning a chassis full of blade servers, usually a Dell M1000e, into a fire-breathing solid-state storage system with just 120us of read latency in the all-DRAM version Kaminario’s been shipping for about a year. All-DRAM storage is cool, but a full K2-D system holds just 12 Tbytes of data and will set you back over a million dollars. While that works for the Wall Street guys, where an extra millisecond of latency could mean someone else gets to make $150,000 on a trade, those of us who live in the real world will be more interested in how the K2-H brings new meaning to the term "hybrid system," since it mixes in MLC-based Fusion-IO cards and DRAM where other hybridsmix flash and spinning disks. Using all-flash boosts latency up to a worst-case 260us, but also reduces the price to $30,000 per Tbyte for a pure flash system. Cost per gigabyte and latency vary with the DRAM-to-flash ratio, as you would expect.

The K2 systems are built using a RAIN (Redundant Array of Independent Nodes) model. Some of the blade servers are ioDirectors, which connect to the servers via Fibre Channel and to the data nodes via 10-Gbps Ethernet. When a server writes data to the K2 system, the ioDirector writes it to two data nodes for reliability. While the K2 uses Fusion-IO flash and DRAM for active LUNs, each data node also has more conventional disk drives. The system uses these drives both as a backup medium, spooling data off to spinning disk in the background, and to hold snapshot data, since even flash is too expensive for these functions.

In the custom hardware corner, wearing the black trunks, is Astute Network’s ViSX G3, with the patented DataPump Engine. Astute tells us that the DataPump Engine chip, which is, among other things, a 10-core RISC processor, is the key to its being able to crank 80,000 IOPs out of a box that consumes just 300 watts of power.

The iSCSI-based ViSX G3 is more a rack-mount SSD system than a complete storage array, relying on host operating system volume managers or, more specifically, vCenter’s storage management for snapshots, replication and the lot. Since VMware’s been adding more storage features to vSphere, this model could be attractive to VMware-centric environments, as it allows them to use the same storage management functions across their solid-state and spinning disk systems.

Old hands like this humble reporter are amazed by how vendors manage to turn off-the-shelf servers and disks, both solid state and spinning, into storage systems that would have required multiple custom chips in even the recent past. However, as Astute’s systems demonstrate, there are still places where custom hardware can cut the power consumption or cost. We as consumers get to benefit from this clash of technologies.