The Case For Lossy Compression

A key tenant of the international society of steely eyed storage guys (ISSESG?) has always been to give back to users exactly the same bits they stored. This has lead some ISSESG members to look askance at technologies like compression and data deduplication that could, if things go wrong, violate that tenant.

The problem is that users add bigger and bigger image files in their Word documents and PowerPoint presentations. In a world where even a $100 digital camera delivers 10 megapixel images, files get bloated pretty quickly. I learned firsthand how quickly over-resolution can chew up disk space when I was consulting with a major advertising agency during the dot-com boom. Employees would build a PowerPoint deck for a client from the graphics files they used for print advertising. These images were generally 1200x1200dpi with 24 bits of color, so a single slide deck could easily take up 200MBytes. Adding insult to injury, they projected the slides at 1024x768, so all that resolution went to waste.  And because they e-mailed multiple drafts of these monster files, the company eventually had to buy a fibre channel SAN just to support their Exchange server.

So how can ISSESGs honor their commitments to users while also preserving disk space? The vendors Neuxpower and Balesio suggest lossy compression is the answer. Neuxpower's NXpower products find images in common Office files like Word and PowerPoint and use well-known lossy compression algorithms to transcode imbedded images down to resolutions and color depths that are a better fit to the ultimate use of the files.

The resulting files are smaller, but since the images are still embedded JPEGs and the like, users don't need to decompress them or even know they've been changed. They just take up less space and load faster. NXpower can run on workstations or file servers. The workstation version can intercept Outlook traffic and compress files before they're sent via email.

As you can imagine, smaller files would be a big advantage in environments with limited bandwidth, such as ships at sea that use satellite links. Neuxpower sent me a report from a study by the navies of the US, Canada, Australia and New Zealand that showed data reduction of 60 to 85 percent. Balesio's file minimizer uses similar technology. According to Neuxpower, Balesio was a partner before developing its own software.