Welcome To The Persistent Memory Era

Hewlett Packard Enterprise's announcement earlier this month that it will  offer "persistent memory" as an option on select ProLiant Gen 9 server models represents as big an opportunity for software-delivered storage as it does for in-memory computing. Now that customers can order servers complete with NVRAM from HPE, storage software vendors can even the playing field with their more hardware-oriented competitors.

The first rule of storage is never lose data and to enforce that rule we, in the International Society of Steely–Eyed Storage Guys, insist that a storage system not acknowledge a write request until the data has been written some place safe.

For the past 20 years, that first safe place was a little bit of DRAM on the RAID, or array, controller that was backed up by a battery to retain data if power to the controller failed. A typical dual-controller array would write new data to this non-volatile RAM (NVRAM) on both controllers before acknowledging.

Like almost everything else in the storage world, today’s NVRAM -- including the 8 GB NVDIMMs HPE’s selling -- takes advantage of that miracle medium of the age, NAND flash. The NVRAM device detects a power failure and using the energy provided by a capacitor, dumps the contents of the DRAM into flash.

HPE persistent memory crop.jpg

Using flash to backup DRAM means modern NVRAM can retain data indefinitely. Even if the batteries were in perfect condition, which they frequently weren’t, old-style NVRAM would lose its contents if power wasn’t restored in a day or less.

As we’ve made the transition to software-defined storage architectures, NVRAM has remained the secret ingredient for vendors delivering their solutions as appliances even if those appliances were industry-standard servers. The appliance vendors could qualify and integrate an OEM NVRAM solution by using one of the few motherboard/BIOS combinations that supported first-generation NVDIMMs from AgigA Tech, Viking or Micron. Alternatively, they could use PCIe NVRAM cards from Curtiss-Wright or Marvell.

Learn more from Howard when he leads a panel of industry luminaries in the session Tearing Down the Storage Silo: Managing Storage with Code at Interop Las Vegas. Don't miss out! Register now for Interop, May 2-6, and receive $200 off.

While flash is another form of non-volatile memory, even in an all-flash system, NVRAM serves several important functions. First, since DRAM is roughly 100 times faster than even a PCIe/NVMe flash device, the system can write data to NVRAM in both controllers and acknowledge the write hundreds of microseconds faster than it could write to flash. Using NVRAM as the write cache/log device also allows the DRAM, which has unlimited write endurance, to hold the hottest data blocks that are updated thousands of times a day, aggregating writes and thereby extending the life of the flash behind it.

Until HPE’s announcement, vendors that license their storage software without hardware -- including all hyper-converged infrastructure products -- didn’t have NVRAM available. They could either take the safe route and write data to flash before acknowledging, or take the risky path and acknowledge data when it’s in memory on multiple nodes.

Taking the safe route places a lot of stress on the SSDs, which in turn requires more expensive high-endurance SSDs, and limits the system’s write performance. That’s still better than the risky path; I’ve seen too many data center power failures to trust the risky path.

There is possible middle path: Using a local UPS to power the whole system long enough for the DRAM write buffer to be flushed. But I haven’t seen it implemented.

HPE’s new products show how far the standardization efforts for non-volatile memory DIMMs have come. Joint efforts between SNIA’s NVDIMM Committee and JEDEC, the organization that sets standards for DIMM modules and sockets, have standardized the flash-backed DRAM as NVDIMM-N. This provides a standard way for BIOSs and motherboards to recognize NVDIMMs and to control functions like retrieving the flash contents back to DRAM.

HPE’s taken full advantage of the new 12-volt power connection on DDR4 DIMM sockets to provide a centralized power source for the NVDIMMs. I’ve worked on a few early OEM systems that required individual capacitors for each NVDIMM and the wiring mess that creates is something I’ll be glad to get behind me.

Of course, things will get really interesting when I can buy ProLiant DL360s or DL380s and plug in not only non–volatile DRAM NVDIMM-Ns but bigger flash or 3D XPoint DIMMs that will hold not the 8 GB of each of HPE’s sticks, but terabytes of data at almost DRAM speed.

Even so, I’m looking forward to the first storage software that will use HPE’s persistent memory as a fast-write cache. It would seem a natural for HPE’s own StoreVirtual.

Learn more about storage and IT infrastructure at Interop Las Vegas. Don't miss out! Register now for Interop, May 2-6, and receive $200 off.