Storage

09:37 AM
George Crump
George Crump
Commentary
50%
50%

Big Data Changes Storage Needs

Growth of unstructured data forces IT managers to look for new ways to scale storage capabilities, with cloud storage a leading option.

In our last entry we talked about the trend of converging storage and compute infrastructures into a single platform. In this entry we will talk about another form of convergence--the convergence of cloud storage into the data center. Like everything else in cloud, implementation of cloud storage can take many forms and connecting it to a data center adds several more issues.

The big driver for a cloud storage system--whether private, public, or hybrid--is the unprecedented growth in unstructured data. Unstructured data is typically thought of as user data and made up of things like documents, spreadsheet, and presentations. User data has grown to encompass much larger data sets like audio and video. No longer can rich media simply be wiped from data center storage systems, since in many cases this data has a legitimate purpose and value to the organization.

The bigger challenge may be from machine-generated data, which is generated by devices or even other applications. Machines can generate data much faster than humans can and the need to store all this data, thanks to initiatives like big data analytics, is becoming more important. As we discussed in our recent article "The TCO Problem of Storage", conventional storage systems may not be able to deliver the economics required to keep pace with the rate of unstructured data growth.

The combination of increased user data plus machine-generated data is the key that will drive the trend toward many data centers having some form of cloud storage. These storage systems are going to need to be scalable, affordable, and utilize capacity as well as data protection efficiently. Most important, these systems may need to break from the traditions of the past and use object-based storage technology to be able to meet the demand to store billions and billions of bits of unstructured content.

The first form that this trend may take is essentially borrowing a page from the public cloud provider's playbook by creating storage systems that are built from commodity-class hardware that is clustered together and leverage an object storage model. There are several companies today that will sell you an object-based storage systems designed to provide this type of functionality, so you don't have to develop your own as some of the early cloud providers did.

The second form that this trend may take is leveraging public cloud storage providers to store the data. The problem, of course, with using the public cloud to store all of your data is that many organizations may not be able to tolerate the latency involved in transferring data to and from a public facility. And laws and regulations may prohibit that option for many industries.

There are several solutions that extend the public cloud into the private data center by using some form of caching appliance. This allows for active data to be stored locally while being replicated to the cloud. Potentially more interesting, as we discussed in a recent case study, some cloud providers are able to set up a cluster of their nodes in your data center. This becomes more than a cache--it quite literally extends a portion of the public cloud directly into your data center. This allows for high-performance local transfers while being a full member of the entire public cloud provider's infrastructure. It also means significantly less investment in staff to manage and maintain the storage.

In either case, the net result of this trend may break down the wall between public and private cloud storage. As data centers become larger, their capabilities to singularly deal with unstructured data may be untenable and, as a result, they will need to leverage a model where they have maximum scalability locally and unlimited scalability remotely.

Follow Storage Switzerland on Twitter

George Crump is lead analyst of Storage Switzerland, an IT analyst firm focused on the storage and virtualization segments. Storage Switzerland's disclosure statement.

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Ajherrera
50%
50%
Ajherrera,
User Rank: Apprentice
1/16/2012 | 9:50:08 PM
re: Big Data Changes Storage Needs
Great article George. One thing to add is that not all object storage solutions are created equal. Organizations should make sure that the solution they choose to power their cloud storage needs can scale up and out incrementally (by the drive) with no provisioning or downtime. The "service" aspect of cloud storage is key. When you provide storage as a service your employees, customers or subscribers will expect it to be available both from an accessibility and capacity perspective. This is one of our main focuses at caringo -

http://www.caringo.com/solutio...
tomleyden
50%
50%
tomleyden,
User Rank: Apprentice
1/16/2012 | 10:42:32 AM
re: Big Data Changes Storage Needs
Interesting post, George, this is very much in line with what I just wrote about Object Storage for Big Unstructured Data: http://www.amplidata.com/?p=10...

In short, I expect Object Storage (one uniform pool of massively scalable storage, accessible over REST) to become the new paradigm next to SAN (block, database-optimized) and NAS (file-based, unstructured data but with limited scalability).
Linear1
50%
50%
Linear1,
User Rank: Apprentice
1/16/2012 | 12:15:18 AM
re: Big Data Changes Storage Needs
When involved with data storage one may expect to be emcumbered to anlylize the indiference when they all belong to the same system why is there not a seperation of systems?Clearly the analog indifference is the same so why is this?The algo sequence is apparent @13over78138241,the polarized circumfrence should not matter,The indifference is statatory there by the circumfrence is benine.Check this out 8100 3over12887251 that is the circumfrence ,just saying.
jmyal
50%
50%
jmyal,
User Rank: Apprentice
1/13/2012 | 9:40:11 PM
re: Big Data Changes Storage Needs
Wow! This data storage thing is getting bigger and bigger! I think we may get many more companies like www.dropbox.com and www.mypdv.com who are going to come up and start offering various forms of storage. It's rapidly increasing I think!
Cartoon
Hot Topics
4
VMware Launches Hyper-Converged Infrastructure Appliance
Marcia Savage, Managing Editor, Network Computing,  8/25/2014
4
The Rise Of White-Box Storage
Jim O'Reilly, Consultant,  8/27/2014
3
Cloud Storage: How To Pick A Provider
Crystal Bedell, freelance writer,  8/26/2014
White Papers
Register for Network Computing Newsletters
Current Issue
Video
Slideshows
Twitter Feed