Storage

00:01 AM
Howard Marks
Howard Marks
Commentary
Connect Directly
Twitter
RSS
E-Mail
50%
50%

The Case For Lossy Compression

A key tenant of the international society of steely eyed storage guys (ISSESG?) has always been to give back to users exactly the same bits they stored. This has lead some ISSESG members to look askance at technologies like compression and data deduplication that could, if things go wrong, violate that tenant.

A key tenant of the international society of steely eyed storage guys (ISSESG?) has always been to give back to users exactly the same bits they stored. This has lead some ISSESG members to look askance at technologies like compression and data deduplication that could, if things go wrong, violate that tenant.

The problem is that users add bigger and bigger image files in their Word documents and PowerPoint presentations. In a world where even a $100 digital camera delivers 10 megapixel images, files get bloated pretty quickly. I learned firsthand how quickly over-resolution can chew up disk space when I was consulting with a major advertising agency during the dot-com boom. Employees would build a PowerPoint deck for a client from the graphics files they used for print advertising. These images were generally 1200x1200dpi with 24 bits of color, so a single slide deck could easily take up 200MBytes. Adding insult to injury, they projected the slides at 1024x768, so all that resolution went to waste.  And because they e-mailed multiple drafts of these monster files, the company eventually had to buy a fibre channel SAN just to support their Exchange server.

So how can ISSESGs honor their commitments to users while also preserving disk space? The vendors Neuxpower and Balesio suggest lossy compression is the answer. Neuxpower's NXpower products find images in common Office files like Word and PowerPoint and use well-known lossy compression algorithms to transcode imbedded images down to resolutions and color depths that are a better fit to the ultimate use of the files.

The resulting files are smaller, but since the images are still embedded JPEGs and the like, users don't need to decompress them or even know they've been changed. They just take up less space and load faster. NXpower can run on workstations or file servers. The workstation version can intercept Outlook traffic and compress files before they're sent via email.

As you can imagine, smaller files would be a big advantage in environments with limited bandwidth, such as ships at sea that use satellite links. Neuxpower sent me a report from a study by the navies of the US, Canada, Australia and New Zealand that showed data reduction of 60 to 85 percent. Balesio's file minimizer uses similar technology. According to Neuxpower, Balesio was a partner before developing its own software.

Howard Marks is founder and chief scientist at Deepstorage LLC, a storage consultancy and independent test lab based in Santa Fe, N.M. and concentrating on storage and data center networking. In more than 25 years of consulting, Marks has designed and implemented storage ... View Full Bio
Comment  | 
Print  | 
More Insights
Cartoon
Slideshows
Audio Interviews
Archived Audio Interviews
Jeremy Schulman, founder of Schprockits, a network automation startup operating in stealth mode, joins us to explore whether networking professionals all need to learn programming in order to remain employed.
White Papers
Register for Network Computing Newsletters
Current Issue
Video
Twitter Feed