Data Deduplication Update Part One

Now that EMC's acquisition of Data Domain has settled in, it makes sense to take a look at the state of the deduplication market. It seems that much of the controversy in this space has subsided in recent months. Partly because two of the larger competitors now have the same business cards, but also because the systems and technologies are getting better. The discussion is moving beyond how does dedupe work and toward what can these systems really do for you? This is a needed step on the road to full adoption.

For the next few entries I am going to look at backup deduplication and in a future entry I'll look at archive and primary storage. Excluding for a moment cloud based backup strategies that include dedupe, which I discussed yesterday on Information Week, the key players that come to mind in the backup deduplication market are EMC (Data Domain and Avamar), FalconStor/Nexsan, Sepaton, Exagrid, IBM (Diligent) and Quantum. In addition, there are all the software players that now do deduplication in software; Commvault, SyncSort (actually block level incremental backups), Symantec and others.

That is a fairly robust list, so obviously while the venom has died down in the deduplication market clearly the perceived opportunity has not. With a list that long, and I am sure my inbox will fill with the companies I missed. How is a customer to decide?

I think the first step is to decide whether you want to extend your current backup application, if there is a weak point in your backup strategy (VMware data protection as an example) or do you want to replace it? Most of the software specific solutions are going to require that you replace your current backup solution. Backup software is one of those "out of the frying pan and into the fire" type of projects. What looks shiny and new on the outside can be an ordeal when it comes to transition. I caution against moving to a new software application if your sole justification is to have deduplication built in. If you are going to take an extension sort of approach, there is still a long list of choices available. The downside to the extension approach is that you do have to move all your data across a IP or Fibre network, and it is deduped at the appliance. The plus side of course is that you don't need to learn a new backup application.

If you are a Symantec NetBackup customer, you have access to their Open Storage Option (OST). As we write about in our article "Do you Adapt or Replace your Backup Application?," this is an important advantage NetBackup customers and deduplication companies, if they take full advantage of the API, should be strongly considering. OST allows for tighter integration to NetBackup and improved performance. OST and how the various vendors support it is now a key differentiator in the market. Other backup applications would do well to catch up with Symantec in this area. There is much more to cover; replication, power managed deduplication, scalability and the ability to leverage the backup appliance for more than just backups. All will be covered in upcoming entries.