Dealing With De-Dupe Doubts

Data de-duplication has become a hot technology, given the urgent need storage managers have to reduce the bulk of backed-up data. And users appear happy with it -- despite a range of questions regarding vendor claims.

A word on the market: Besides early implementers Avamar, Data Domain, and Diligent, other vendors like Falconstor and Sepaton are getting ready to include data de-duplication into products. Symantec has introduced it. Meanwhile, consolidation is underway as EMC spent $165 million to buy Avamar earlier this month, and ADIC (now Quantum) shelled out $63 million to buy Rocksoft in March. (See EMC Picks Up Avamar, ADIC in De-Dupe Deal, and Symantec Dips Into De-Dupe.)

Each of the key de-duplication players claims striking consolidation ratios -- figures that don't appear to be playing out in actuality. Despite this, the level of consolidation users are really getting is good enough to satisfy them for now.

"We have two boxes. I'm getting about 12:1 [consolidation] on one and 9:1e on the other," says network administrator Lance Jeffrey of Idaho Central Credit Union. He's using de-duplication gear from Data Domain, which claims its systems can achieve 20:1 to 50:1 ratios.

Jeffrey explains the disparity between the vendor's claim and what he's getting down to the amount of image files on his network, which aren't as easy to de-dupe as other kinds of files.Data Domain co-founder Brian Biles says his firm's 20:1 ratio is loosely based on an average figure, and he maintains that companies will get substantially greater compression in many instances. Without commenting on Jeffrey's case, he notes that variables in the level of consolidation have to do not only with the kind of data being consolidated but with a company's specific backup policies and rules.

For instance, if a firm backs up Word files daily, it's likely to get a higher consolidation ratio than if it backs up less often, since the amount of data backed up will be less. Also, typical data center backup that does not involve remote sites will produce a lower consolidation ratio than one that involves remote sites, where a lot of data is commonly stored across multiple locations.

Biles doesn't hesitate to say his gear will perform much better than 20:1 consolidation. For enterprise databases and Exchange that are backed up daily, users can realize 40:1 or 60:1 ratios, he insists.

Other vendors provide even bigger figures. Avamar, for instance, claims an average consolidation ratio of 300:1 for data that is backed up daily -- regardless of the data type.

But at least one Avamar customer, Jason Paige, information systems manager at financial firm Integral Capital Partners, says he's getting approximately 20:1. Nonetheless, he's over the moon about that. "It's really powerful for Exchange," he says, noting that before he used Avamar for his approximately 100-Gbyte Exchange store, he could not perform a message-level backup for Exchange in less than 24 hours. With Avamar, he is able to do file-level, database, and message-level backups all within about an hour.Besides not caring about whether their vendors are exaggerating, neither Paige nor Jeffrey have any complaints about the speed at which de-duplicated data can be accessed or restored either.

Bottom line? The world of data de-duplication is young enough that no one appears to care about product-selling hyperbole. Apparently, the end result is sufficiently dramatic to erase any bad feeling that may come from supplier braggadocio.

Still, caveat emptor. "Full volume backup data will always have the largest amount of de-duplication because every time you backup you are backing up much of the same data over and over again," says one industry analyst, who asked not to be named. "It is this data set that most of the vendors claim their ratios." In reality, he notes, most users get a 2:1 or 3:1 ratio on incremental backups.

Mary Jander, Site Editor, Byte and Switch