Special Coverage Series

Network Computing

Special Coverage Series


Putting Flash On The Memory Bus

Diablo Technologies' new Memory Channel Storage devices plug flash storage into the DIMM slots. If Diablo's performance claims prove true, expect to see some fast servers built around the technology.

For the past several years, PCIe SSDs have been the kings of the storage performance hill. Move the storage closer to the processor, the story goes, and you boost throughput and reduce latency. Figuring that if the PCIe bus is close to the processor, the memory DIMM slots are even closer; Diablo Technologies designed its Memory Channel Storage to connect there.

Memory Channel Storage devices, like Diablo’s reference design TERADIMM, plug 200 Gbytes to 400 Gbytes of flash into a standard DDR3 DIMM socket. Since the QPI memory controller is on the die of today’s Xeon processors, the DIMM sockets provide much higher bandwidth and lower latency that the PCIe slots.

More Insights

Webcasts

More >>

White Papers

More >>

Reports

More >>

While it would be cool to just put a dozen 400-Gbyte DIMMs in a server and have 4.8 Tbytes of directly addressable spaces, simply mapping flash into the processor’s address space has a few major drawbacks.

The first is that today’s OSes just aren’t set up to handle large fluctuations in memory latency. Sure, back when I was in college, the mainframe had fast core and a much larger, 4-Mbyte pool of much slower storage. When you submitted a job, you’d pay more for fast core. Windows and Linux were never designed for that kind of computer and don’t prioritize memory usage by location. Even worse, treating an inherently block-addressable system like flash as it if it's byte-addressable DRAM will create a huge amount of write amplification. This will wear out the flash because it has to write a whole page for every 1-byte update.

Rather than just mapping the flash into the processor’s address space, Diablo’s drivers for Windows, Linux and VMware ESXi present the flash as a block storage device, basically using the Memory Channel Storage device as a very fast SSD or directly plugged into the virtual memory paging mechanism. With the virtual memory driver, the flash appears to the processor to be memory, but data is paged in and out of the flash from DRAM in 4K pages. The OS knows which pages are swapped out to flash and can prioritize between the two memory pools.

Diablo won’t be out peddling flash DIMMs, but is selling the technology plus its controller ASIC to OEMs to build devices around it--similar to the way SATA flash controller vendors like LSI/SandForce and Marvell operate. The first flash DIMMs should come from Smart Storage Systems late this year and will use Smart’s Guardian Technology to extend the write endurance of MLC flash to 10 full-drive writes a day for five years.

[EMC recently bulked up its flash products division by acquiring a startup that creates shared storage pools across server clusters. Read the details in EMC Flash Division Expands with Scale IO Deal."]

We should also see the first servers that can run with the TERADIMM. In addition to the usual server vendor qualifications, they require a patch to the server’s UEFI BIOS, and that means an even more extensive testing period. The Diablo guys tell me they have one major OEM with the technology in the pipeline and a couple of others that are in early stages of the process.

If Diablo actually delivers on its claims--150K IOPS read and 85K writes per module--those servers are going to be darned quick. Like the latest enterprise SSDs from Intel and Virident, Diablo's technology also claims deterministic latencies under 50 seconds, all at the same cost per IOP as SATA SSDs.

Strictly speaking, the TERADIMM, and its actual for-sale derivatives from Smart and possibly others, aren’t the first DIMM form factor SSDs. That honor goes to Viking’s SATADIMM. While SATADIMMs plug into a DDR3 DIMM socket, they only use it for power; data is sent through a separate SATA connector, and the SATADIMM, other than its form factor, is a pretty vanilla SATA SSD.

The DIMM form factor is a natural for blade and high-density server environments where PCIe slots, if they exist at all, are a severely limited resource. I wrote about the tyranny of the mezzanine blades when I wrote about Dell’s VRTX. If the next generation of blades from your favorite vendor has the UEFI support, you can use flash DIMMs instead of the mezzanine SSD your server vendor’s blessed.

Diablo’s Memory Channel Storage, like NVMe, has the potential to disrupt the PCIe SSD market, bringing even higher levels of performance by getting closer to the processor.



Related Reading



Network Computing encourages readers to engage in spirited, healthy debate, including taking us to task. However, Network Computing moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing/SPAM. Network Computing further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | Please read our commenting policy.
 

Editor's Choice

Research: 2014 State of Server Technology

Research: 2014 State of Server Technology

Buying power and influence are rapidly shifting to service providers. Where does that leave enterprise IT? Not at the cutting edge, thatís for sure: Only 19% are increasing both the number and capability of servers, budgets are level or down for 60% and just 12% are using new micro technology.
Get full survey results now! »

Vendor Turf Wars

Vendor Turf Wars

The enterprise tech market used to be an orderly place, where vendors had clearly defined markets. No more. Driven both by increasing complexity and Wall Street demands for growth, big vendors are duking it out for primacy -- and refusing to work together for IT's benefit. Must we now pick a side, or is neutrality an option?
Get the Digital Issue »

WEBCAST: Software Defined Networking (SDN) First Steps

WEBCAST: Software Defined Networking (SDN) First Steps


Software defined networking encompasses several emerging technologies that bring programmable interfaces to data center networks and promise to make networks more observable and automated, as well as better suited to the specific needs of large virtualized data centers. Attend this webcast to learn the overall concept of SDN and its benefits, describe the different conceptual approaches to SDN, and examine the various technologies, both proprietary and open source, that are emerging.
Register Today »

Related Content

From Our Sponsor

How Data Center Infrastructure Management Software Improves Planning and Cuts Operational Cost

How Data Center Infrastructure Management Software Improves Planning and Cuts Operational Cost

Business executives are challenging their IT staffs to convert data centers from cost centers into producers of business value. Data centers can make a significant impact to the bottom line by enabling the business to respond more quickly to market demands. This paper demonstrates, through a series of examples, how data center infrastructure management software tools can simplify operational processes, cut costs, and speed up information delivery.

Impact of Hot and Cold Aisle Containment on Data Center Temperature and Efficiency

Impact of Hot and Cold Aisle Containment on Data Center Temperature and Efficiency

Both hot-air and cold-air containment can improve the predictability and efficiency of traditional data center cooling systems. While both approaches minimize the mixing of hot and cold air, there are practical differences in implementation and operation that have significant consequences on work environment conditions, PUE, and economizer mode hours. The choice of hot-aisle containment over cold-aisle containment can save 43% in annual cooling system energy cost, corresponding to a 15% reduction in annualized PUE. This paper examines both methodologies and highlights the reasons why hot-aisle containment emerges as the preferred best practice for new data centers.

Monitoring Physical Threats in the Data Center

Monitoring Physical Threats in the Data Center

Traditional methodologies for monitoring the data center environment are no longer sufficient. With technologies such as blade servers driving up cooling demands and regulations such as Sarbanes-Oxley driving up data security requirements, the physical environment in the data center must be watched more closely. While well understood protocols exist for monitoring physical devices such as UPS systems, computer room air conditioners, and fire suppression systems, there is a class of distributed monitoring points that is often ignored. This paper describes this class of threats, suggests approaches to deploying monitoring devices, and provides best practices in leveraging the collected data to reduce downtime.

Cooling Strategies for Ultra-High Density Racks and Blade Servers

Cooling Strategies for Ultra-High Density Racks and Blade Servers

Rack power of 10 kW per rack or more can result from the deployment of high density information technology equipment such as blade servers. This creates difficult cooling challenges in a data center environment where the industry average rack power consumption is under 2 kW. Five strategies for deploying ultra-high power racks are described, covering practical solutions for both new and existing data centers.

Power and Cooling Capacity Management for Data Centers

Power and Cooling Capacity Management for Data Centers

High density IT equipment stresses the power density capability of modern data centers. Installation and unmanaged proliferation of this equipment can lead to unexpected problems with power and cooling infrastructure including overheating, overloads, and loss of redundancy. The ability to measure and predict power and cooling capability at the rack enclosure level is required to ensure predictable performance and optimize use of the physical infrastructure resource. This paper describes the principles for achieving power and cooling capacity management.