Virident Unlocks PCIe Flash From a Single Server

Posted by Howard Marks
February 22, 2013

PCIe SSDs and flash cards provide faster access to storage than any other non-volatile solution, but the downside to PCIe storage is that it's only accessible and manageable from the server the PCIe card is plugged into. This can be a real problem when that server crashes and your data is trapped inside. Virident's FlashMAX Fabric software addresses the problem by making PCIe flash act more like a SAN or as a distributed cache.

Over the past few years the folks at Fusion-IO have been promoting the concept of treating a PCIe flash card as something different than just an SSD with a faster interface. They've put their money where their mouths are, delivering APIs that allow their cards to be accessed in new ways.

Virident, a PCIe flash vendor that recently got a $40 million investment from Seagate, also talks about different approaches to flash access. With the announcement of Virident's FlashMAX Fabric, we can see the differences between Fusion-IO's and Virident's visions.

Fusion-io is trying to change the PCIe flash access paradigm by making people treat flash more like memory and less like storage. The company has come up with a few interesting APIs, including a key-value pair, a limited file system and a very interesting atomic dual-write API. All could be useful in simplifying applications and improving efficiency, but all are still using the PCIe flash card as local, non-volatile memory/storage, while leaving the sharing and failover issues to upper layers in the application stack.

Like Fusion-io, Virident is creating new ways to talk to the PCIe flash. While Fusion-IO keeps its functionality within a single server, Virident expands the influence of a PCIe flash card across multiple servers via software. Its new FlashMAX Fabric software directly addresses the resiliency, availability and multiple access limitation of PCIe storage. FlashMAX Fabric does this by interacting directly with the flash translation layer (FTL) on Virident's FlashMAX cards, bypassing the SSD API, which reduces both latency and the write amplification that can shorten the life of a flash memory device.

The FlashMAX Fabric has three components. The first is vHA, which synchronously replicates the data from flash volumes from one server into the flash card on another server. Synchronous replication will, by definition, always introduce some additional latency into a write. Because synchronous replication requires the data to be written to both data stores (in this case, the Virident FlashMAX cards), it's essential to minimize the network latency introduced by replication.

Virident uses RDMA (Remote Direct Memory Access) to allow the cache software on one server to write directly into the flash translation layer of the other. The initial version relies on Infiniband for RDMA access between machines. Virident promises to support RoCE (RDMA over Converged Ethernet)in the future to allow 10-Gbps Ethernet users to join the party.

The second component, vShare, lets servers access the memory on flash cards in other servers--again, via RDMA. With vShare, an organization could put a few terabytes of FlashMAX cards in a server, replicate it to another server via vHA, and then access the resulting pool of volumes from several other servers.

The third component, much like Dell's FluidCache or PernixData's recently announced product, uses flash cards in several servers to create a distributed, replicated cache. Servers with flash cards, and additional servers without, can use the cache to speed application performance. This architecture is a step forward for performance because it can cache writes, resiliency and workload migration support over the products currently on the market.

Virident's roadmap for FlashMAX Fabric will come in four versions. Version 1, planned for delivery soon, will provide two-node replication, access to a volume on a remote system's flash card and write-through caching. Version 2, due in the fourth quarter, will add support for multiple flash cards in each server. It will also include two-node storage pooling that allows both servers to access the same volume and write-back caching. Version 3 adds cluster management, allowing storage and cache pooling across more than two servers. Version 4 will be a clustered block storage pool with N+1 availability. FlashMAX Fabric will initially be released for Linux and vSphere environments, with Windows to follow.

I was frankly getting a little impatient with Virident, as it's been talking very generally about changing how people access PCIe flash for a long time without delivering on the promise. Now that I see the vision, I'm enjoying the view. The future of storage is software, and FlashMAX is a step in the path to the software defined storage nirvana we've been promised.

Howard Marks, chief scientist at DeepStorage.net, is a featured speaker at Interop Las Vegas this May, including the conference session "Deploying SSDs in the Data Center." Register here today.

Related Reading