Database Archiving Heats Up

IT storage pros are about to get slammed from two sides -- on the one side from the growth in and demand for archiving structured data, particularly database data -- and on the other side from vendors trying to capitalize on the opportunities.

Unlike unstructured data, such as email, this side of the archiving market has received relatively little attention, although a number of vendors are cranking up their efforts in this space. HP, for example, snapped up Outerbay to bolster its database archiving story and Solix was recently named in Byte and Switch's top 10 startup list. (See HP Hops on OuterBay and Top 10 Startups to Watch.) Other vendors playing in this part of the market include Applimation and Princeton Softech. (See Users Pick Princeton Softech.)

With the database archiving market heating up, we spoke to IT managers and analysts to discover the benefits and pitfalls of the technology.

Be Aware of Your Archiving Options

At least one analyst told Byte and Switch that most users are blissfully unaware of the options available to them. "Unlike unstructured data, there are lots of things that you can do with structured data," says Brian Babineau, senior analyst at the Enterprise Strategy Group citing, particularly, the ability to take subsets of the data and run tests on them while the main database runs in production mode."Most people don't know that these features and functions are available to them in database archiving," says Babineau, adding that vendors such as Solix and HP offer this capability.

Other useful features include the ability to "scramble" sensitive data. This involves, for example, mixing up data so that Social Security numbers do not align with other personal records. "The reason why [most IT] users don't know it is there is because they are responsible for database admin or storage admin -- they may not be responsible for security," says Babineau.

Consider Virtualization

The Austin Independent School District in Texas is considering the best way to develop its archive, according to Shlomi Harif, the organization's network systems director. With a broad range of student and organizational data that needs to be kept for varying numbers of years, the District needs a flexible solution, he tells Byte and Switch.

The other challenge is finding a technology that can shift database data and handle the applications supporting it. "You not only have to keep the data, but you also need to keep the application at the level that it was at," says the exec.At the moment VMware appears to be the likeliest candidate. "It takes a snapshot of everything down to the OS and the [database software] drivers," says Harif. "All you have to do is ensure that you have a similar class of hardware [to archive it on], and that's easy."

A trickier issue is the lifespan of the software. "I think that VMware-type offerings need to be able to handle about three years of backwards compatibility," says Harif, adding that this timeframe more or less fits the rest of his IT strategy. "Every server that we buy is expected to have at least four years of production use."

Get Out Your Crystal Ball

Building your database archiving strategy around your current needs is simply asking for trouble. IDC, for example, warned in a recent report that many firms are building their database archives with their own custom-built tools, running the risk of being unable to access the data a few years from now.

At least one user, Andrew Gibbons, director of operations at photo sharing site SmugMug, agreed that custom-built tools can be problematic. "What we're doing at the moment is a very manual process," he says, adding that his firm has built software to take weekly copies of database data and archive it.The exec explains that an automated database archiving product would make his life much easier: "Moving a couple of hundred Gbytes around can be a big challenge."

The other issue here is that key staff could leave the organization, taking critical skills out the door with them. "If you forget what you have done and new people take over, you have got a problem," says David Hill, principal of the Mesabi Group.

The trick is ensuring that you have the skills in place to deal with both the original database application and any new SQL queries that need to run on it. "Users have to make sure that they plan carefully for what they are doing [in the future]," says Hill.

Consider the Impact of ILM

The emergence of technologies such as Information Lifecycle Management (ILM), which enables IT managers to automatically shift their data onto less expensive forms of storage, is also putting database archiving in the spotlight. (See EMC Earnings Credit ILM Uptick, IBMs ILM 30th Anniversary, and HP Uses Softech Archive in ILM.)IDC warns that archiving is barely addressed by many database users. At the moment, many users are simply leaving unused data in their databases indefinitely. This scenario is extremely problematic because it wastes storage capacity, and could prove expensive in the long term.

ILM, of course, poses its own set of challenges to users. IT managers at the recent Data Protection Summit, for example, warned that not even the big-name ILM vendors such as EMC and HP have a silver bullet that can handle all of their data management woes. (See Users Shatter Storage Myths.)

One user, Eliot Kagan, systems engineer at24 Hour Fitness, which owns 400 fitness clubs across the U.S., recently urged CIOs to take this issue into their own hands, setting careful values for data before they send it off into archives.

— James Rogers, Senior Editor Byte and Switch