Primary Storage Data Reduction Heats Up

Things have been heating up in the primary storage data reduction market as a management change at Storwize has made them a lot more aggressive, leading to a whole new round of tweets, white papers and the like arguing the merits of fast LZW type compression vs. the hot and sexy technology of data deduplication. While they've been fighting with Ocarina, EMC and NetApp for the enterprise Nexenta, GreenBytes and HiFn have brought primary storage deduplication to the masses.

Howard Marks

April 23, 2010

3 Min Read
Network Computing logo

Things have been heating up in the primary storage data reduction market as a management change at Storwize has made them a lot more aggressive, leading to a whole new round of tweets, white papers and the like arguing the merits of fast LZW type compression vs. the hot and sexy technology of data deduplication.  While they've been fighting with Ocarina, EMC and NetApp for the enterprise Nexenta, GreenBytes and HiFn have brought primary storage deduplication to the masses.

GreenBytes and Nexenta both base their systems on Sun/Oracle's Open Solaris and ZFS. GreenBytes delivers ready-to-use storage appliances complete with a management GUI and GreenBytes' own inline data deduplication. They recently announced a new entry-level system with eight 500GB small form-factor drives and 32GB of flash that speeds up ZFS and hash table lookups for the deduplication all for $10,000.

Last year Sun and Greenbytes had their attorneys bringing suits over GreenByte calling their file system ZFSplus, with features curiously appearing in one or the other projects and similar annoyances.  Greenbyte's CEO Bob Petrocelli recently blogged that "I am happy to report that the matter is now behind us and has been settled in a mutually beneficial fashion."

Nexenta is sticking to software putting a BASH/Linux CLI and of course management GUI on OpenSolaris to create NexentaStor which they then sell direct to users that want to roll their own unified storage systems and to OEMs like OnStor and PogoLinux who'll sell fully packaged NAS appliances.  A SuperMicro server configured with 4TB of storage and a 4TB NexentaStor license is in the same $10,000 neighborhood as the GreenBytes GB-1000.

I'm a big fan of how ZFS handles RAID like data protection, snapshots and replication, and especially how it can use flash for log and read caches. I am somewhat concerned about how Oracle will continue to offer, support and extend these open source technologies. I'm therefore glad to say that SpectraLogic is working on a port of ZFS to FreeBSD. If Oracle decides to kill OpenSolaris, the open source community can continue to enhance ZFS without the rest of the OpenSolaris baggage. Some have argued for a Linux ZFS port, but ZFS breaks some of Linux's rules and would have to be run with FUSE  which is a performance killer. BSD and Solaris are more alike under the covers than Solaris and Linux.HiFn's has just started shipping samples of their BitWackr, which combines a PCIe card with HiFn's compression, encryption and hash-generation chip, which has software for Windows Server or Linux to create a data volume that's deduped, compressed and optionally encrypted in real time. Since BitWackr is designed to be installed in systems that aren't dedicated to providing storage services performing hash generation and compression in hardware leaves more CPU available for other things.

It's important that a system using hashes for Deduplication has the ability to do hash-table lookups quickly, and BitWackr can use an SSD to store the hash tables. BitWackr installs as a disk/RAID controller so data is still stored in the OS's native file system, so it can support applications like Exchange that insist on running with NTFS.

HiFn's parent company Exar, who recently snapped up 10Gbps Ethernet NIC maker Neterion, is peddling BitWackr as an OEM product for around a kilobuck, so while you might not see it on the store shelves any time soon, it just might be included as part of some of your favorite products. I've received an evaluation kit, complete with SSD for hash tables, from HiFn and will be setting it up in the lab after Interop. Watch this space for an update. Disclosure: At the time of publication, I am working for EMC and NetApp, but I'm not working for any other mentioned vendors. 

About the Author(s)

Howard Marks

Network Computing Blogger

Howard Marks</strong>&nbsp;is founder and chief scientist at Deepstorage LLC, a storage consultancy and independent test lab based in Santa Fe, N.M. and concentrating on storage and data center networking. In more than 25 years of consulting, Marks has designed and implemented storage systems, networks, management systems and Internet strategies at organizations including American Express, J.P. Morgan, Borden Foods, U.S. Tobacco, BBDO Worldwide, Foxwoods Resort Casino and the State University of New York at Purchase. The testing at DeepStorage Labs is informed by that real world experience.</p><p>He has been a frequent contributor to <em>Network Computing</em>&nbsp;and&nbsp;<em>InformationWeek</em>&nbsp;since 1999 and a speaker at industry conferences including Comnet, PC Expo, Interop and Microsoft's TechEd since 1990. He is the author of&nbsp;<em>Networking Windows</em>&nbsp;and co-author of&nbsp;<em>Windows NT Unleashed</em>&nbsp;(Sams).</p><p>He is co-host, with Ray Lucchesi of the monthly Greybeards on Storage podcast where the voices of experience discuss the latest issues in the storage world with industry leaders.&nbsp; You can find the podcast at: http://www.deepstorage.net/NEW/GBoS

SUBSCRIBE TO OUR NEWSLETTER
Stay informed! Sign up to get expert advice and insight delivered direct to your inbox

You May Also Like


More Insights