Storage researchers are looking to innovations like 3D NAND, erasure coding, and utilizing DNA to provide greater data density, faster performance, and low cost.
We're all familiar with the data deluge, and the need to store and analyze exponential data growth is driving storage manufacturers to provide new technologies to keep that data flowing. According to IDC estimates, the amount of data stored in the world's computer systems is doubling every two years, and is likely to continue increasing at that rate into the next decade.
In order to deal with that data volume, manufacturers are looking for ways to increase storage density, or to store more information in less space.
Another key factor is Moore's Law. Back in 1965, Intel co-founder Gordon Moore observed that the number of components per integrated circuit was doubling every two years. This means that processors were getting about twice as fast every two years, and the law continues to hold true to this day.
However, as CPUs have gotten faster, storage hasn't always kept up. With storage becoming a bottleneck, manufacturers are also looking for ways to increase performance in order to keep pace with Moore's Law.
And, of course, manufacturers also need to keep prices low in order to satisfy their customers.
In order to meet the demands for greater density, faster performance, and low cost, researchers have proposed several innovative new technologies. We look at nine innovations that could drastically affect the storage industry in the near future. Some are already on the market, while others are still on their way to general availability.
3D NAND is a new type of flash storage that can fit three times as much capacity in the same amount of space as traditional (2D) NAND flash. In fact, with 3D NAND flash, manufacturers can fit 3.5 TB of data on a solid state drive (SSD) the size of a stick of gum.
Earlier types of NAND flash arranged memory cells in a two-dimensional grid pattern, but as the name suggests, 3D NAND also stacks memory cells in layers on top of each other, allowing for greater density, faster performance and lower costs per GB. In March 2015, Intel and Micron announced availability of 3D NAND products, which are expected to begin shipping later this year.
(Image: Micron Technology)
Although it sounds like a technique for deleting data, erasure coding is actually a method of data protection, and it offers an alternative to RAID technology. Similar to parity, it uses a mathematical function to determine if any data has been lost. If it has, then the array is able to recover the data, which has been spread across several disks in the array.
Erasure coding is actually a fairly old technology, but it hasn't been widely used because it adds a lot of complexity to the storage system. However, as the amount of data organizations store continues to grow exponentially, erasure coding is attracting interest as a way to reduce costs, particularly for cold or archived data.
In an attempt to build higher-capacity, energy-efficient hard drives, a few manufacturers have come up with a novel solution: filling the hard drive case with helium. Because helium is one-seventh as dense as air, the spinning platters in a helium drive encounter less resistance and experience almost no turbulence. As a result, manufacturers can put more platters into a drive and track data on those platters more precisely, greatly increasing drive density. It also decreases energy use, makes the drive quieter and improves reliability.
It took some time for helium drives to move from concept to reality because it was difficult to prevent the helium from leaking out of the drives. Now, HGST and Seagate both have helium drives currently on the market, and HGST reports that of the one million helium drives it shipped in the first year of production, none experienced seal failure in the field.
Shingled magnetic recording
For years, hard drive manufacturers increased the density of their drives in part by reducing the size of the tracks (the area where data is written on the drive) and the space between data tracks. Eventually, they reached a limit because they physically could not squeeze any more tracks onto a disk without risking data loss.
One solution to that problem is called shingled magnetic recording (SMR). In SMR devices, the part of the drive that reads data from the tracks is much smaller than the part that writes the data. That means that the tracks can be overlapped, like shingles, without impacting data integrity.
SMR drives are currently available for purchase, and the technology continues to improve.
Most SSDs connect to systems using interfaces designed for hard drives, specifically SAS or SATA. Although these interfaces work pretty well, they do impose some latency on the SSDs, which impacts performance. One option for overcoming this latency is Non-Volatile Memory Express (NVMe) over PCIe. However, while this approach improves performance, it isn't very scalable for situations where enterprises want to connect a lot of SSDs to their networks.
One proposed solution is NVMe over fabric. This approach eliminates both the latency and the scalability issues. The industry is currently working on defining NVMe over Fabric standards, and drives that take advantage of the technology should become available in the near future.
(Image: OpenFabrics Alliance)
In addition to working together on 3D NAND, Micron and Intel are also working on another innovative 3D technology that is an alternative to both flash and DRAM. 3D Xpoint (pronounced "three-dee cross-point") is still a ways from becoming available as a commercial product, but the two companies say it could be the biggest breakthrough in memory technology since the 1970s. It combines the functions of storage and memory in a single product that will be both inexpensive and fast. It's also non-volatile like flash, meaning that it retains data when the system is powered down. They say it could usher in a whole new style of system architecture and make big data processing much faster than has been possible in the past.
(Image: Micron Technology)
Heat-assisted magnetic recording (HAMR)
Another way to increase the capacity of hard drives is a technique called heat-assisted magnetic recording (HAMR). Because of the fundamental properties of magnetism, there's a limit to how small the tracks can be on a disk without the risk of data corruption. However, heating the disk allows the tracks to be quite a bit smaller, increasing the amount of data that can be stored.
With HAMR, a tiny laser heats up the part the part of the spinning disk that is being written, allowing the write tracks to be much smaller. At least that's the theory, as no manufacturers currently have HAMR drives on the market. However, they anticipate that HAMR drives will likely become available by 2018.
In current hard drives, information is stored in a continuous magnetic film. Bit-patterned media (BPM) is a proposed new kind of magnetic media for hard drives that would store data in tiny islands rather than in a continuous film. The benefit of this arrangement is that it would allow manufacturers to significantly increase hard drive density. BPM could also be combined with other emerging technologies like HAMR in order to achieve even greater densities.
Several papers have been published about the potential of BPM, and Toshiba and other manufacturers have conducted research into production. However, they are still working on overcoming the challenges of fabrication, and hard drives using this technology are not yet on the market.
By far the most futuristic of the technologies listed here, DNA storage is a concept that leverages advances in biotechnology to allow systems to store data in DNA molecules. The attractiveness of this option is that it could overcome limits of silicon to achieve very high density, storing up to 1 exabyte in a cubic millimeter of material. DNA is also very durable with a half-life of more than 500 years.
However, before this technology could become a reality, scientists need to make considerable advances in manipulating DNA. Researchers at the University of Washington are Microsoft are collaborating to overcome those challenges, but it will likely be many years before DNA storage could become a reality.