During my recent panel session with Howard Marks and Vanessa Alvarez at Interop, a question came up regarding the "big data" concept: How is it that storage capacity is so important to so many companies when performance has never been so critical? How can these two diverging requirements coexist in the same market, solutions, and devices?
The storage industry has long focused on exactly these two requirements: Devices must meet the performance demands of applications while accommodating an ever-increasing demand for capacity. Most advancements in the industry have focused on one of these areas, often enabled by a companion advancement in the other.
It is interesting to consider that RAID was originally developed with performance, rather than capacity, in mind. Combining multiple disks in a RAID set delivered the
input/output operations per second (IOPS) of a mainframe direct access storage device (DASD) at a fraction of the cost. This is the reason for the initial acronym, a redundant array of inexpensive disks.
But something happened along the way: Enterprise storage makers began building centralized storage devices based on RAID, and realized that the same technology could deliver massive capacity, as well. Soon, the race was joined to develop gigabyte-, terabyte- and petabyte-scale storage arrays.
It is challenging to deliver performance as well as capacity. Massive storage arrays have often suffered in terms of responsiveness, so much so that vendors rarely
fill a device with disks. Typical storage array configurations are usually an order of magnitude smaller than the theoretical maximum, and capacity is often
sacrificed in the name of performance.
