The Critical Need For Data Disposal

Did you know that 69% of all data stored by enterprises has no value? That is, it is data that serves no useful purpose--none, zilch--yet continues to cost IT a bundle in storage resources. This has important implications to IT and enterprises that we need to explore.

One of my many interests is data governance, risk management and compliance (GRC), a subject that doesn't get as much attention as it deserves. So I was pleased to be invited to speak on data retention at the recent Excellence in Governance, Risk Management, and Compliance Conference (EGRC 2013) in Portland, Maine. This cozy conference provided a number of insights on GRC topics as well as the opportunity to meet attendees from both private and public organizations.

Among the issues I addressed in my presentation was data retention management, including both data disposal and data preservation. But I'd like to focus here on the need for data disposal: why it's important to take your data mountain and reduce it to a manageable and useful data molehill.

Smoking Gun

A recent vendor briefing included the information in Figure 1, and although I have long suspected the truth of it, I had no credible source to make a quantitative rather than a suggestive argument against it:

Figure 1: Data Retention Requirements

Source: 2012 Compliance, Governance, and Oversight Council Summit

The chart shows that 1% of data in an enterprise has to be preserved for litigation hold, and 5% has to be managed to cover compliance requirements. Another 25% is reasonably determined to have current business value. That means a whopping 69% of all data--more than two-thirds--has no value whatsoever!

One might quibble with the figure (what is the underlying research, etc.), but let's apply a "reasonableness" litmus test: For the most part, businesses and their IT organizations mainly focus on what is happening now (current transactions, emails and analyses) and not on the process by which data accumulates.

IT acts as the custodian of data (and usually bears the burden of the cost of storing and managing it), but is not the "owner" of that information. Although a business unit may be the official owner, individual employees act as "stewards" for particular data sets. But what if an employee leaves, with his or her email, Word documents, and the like left as no-longer-used data debris? Who knows and who manages it? The answer is: probably no one.

Reasons For Tackling Data Disposal

Now I will submit a challenge: How important is it to get rid of that useless data?

Assume that 20% of an IT budget is spent on storage and that 70% of your data is of no value to your business. That means 14% (plus or minus, depending on individual enterprise differences) of the average IT storage budget is simply wasted. Calling all CIOs: Does that attract your attention? Now, realistically, even if by some magic all the useless data could be safely disposed of, there wouldn't necessarily be instant savings. Although a lot of disk space would be saved, could an array be sold? Hard to say, and the money you'd get would likely be a lot less than you paid for it (the used car depreciation problem).

Freeing up disk space means future storage purchases could be deferred, but that does not translate to immediate savings. But seeking savings that can eventually be redirected to more productive purposes, such as currently underfunded yet desperately needed IT innovation, is a good reason to tackle the problem.

This issue is a "life goes on" type of problem. That means that while you may be able to live with it for the time being, the continued exponential influx of new data will exacerbate the situation over time, making it increasingly difficult to address.

Moving business to the cloud doesn't fix the issue, but it may force businesses to pay more attention. One of the objectives of cloud computing is to provide IT-as-a-service, where users can select the services they want from a self-service catalog. However, this nirvana comes at a price. Resources allocated to and consumed by users means that chargebacks (or at least showbacks) have to be used. And guess what? Does a business "owner" of data want to pay about $10 for every $3 of data that has some useful value?

Next page: The Challenge Of Data Disposal

It would take a long time to discuss the challenges of data disposal. It is not only not easy, it can be nearly intractable, and no high-tech Alexander the Great is likely to be able to cut the Gordian Knot of storage. Plus, there are logical and technical challenges to address. A data governance function has to set and establish policies. A methodology has to be found to separate the data wheat from the data chaff (which is a major entanglement and comingling problem). The process likely can't be done manually, so software tools to aid in the automation of the process are likely required.

Realistically, a triage concept to get the most reduction with the least investment may have to be employed. Even in the best circumstances, you will likely have to accept the fact that you can't get it all. Moreover, you not only have to tackle the current data mountain, but you have to put processes and procedures in place to try to prevent it from happening again. Cleaning up after the data elephant is not easy, but organizations have to acquire the shovels and discipline to make it happen.

Mesabi Musings

News of continued exponential storage growth is a continuing circus, with big data being one of the most prominent acrobats. But while it is less glamorous to talk about what is left after the parade of data elephants has gone by, that cleanup process has to take place.

If storage is 20% of an IT budget and 70% of the current data mountain has no value, roughly 14% of the annual IT budget may simply provide no value whatsoever. Facing up to the problem is the first thing that enterprises need to do. Figuring out how to solve the problem will require time, mental discipline and hard work, but the prize would seem to be worth the effort. How to address the problem so everybody benefits is a subject for another day.