The Tape Format Rot Fear Factor

As I speak with users at the backup seminars I teach around the country, I often hear them say that they avoid using tape for long-term data retention, expressing their concern with something like, "They come out with a new format every few years, and then you can't read your tapes." While it is technically true that eventually you'll have to migrate data from old tapes to new media, you should probably be able to go longer between migrations with tape than with disk archives.

Now, don’t get me wrong--I’m not saying that tape format rot isn’t a concern, just that it’s an eminently controllable concern. I’ve played What kind of tape is it? with abox of tapes from the warehouse as much as anyone, but that problem comes from falling back on old backups because our organization doesn’t have a real data archiving strategy.

In addition, consolidation in the tape drive market has left the industry standardized on one midrange tape format, LTO Ultrium. The administrator running the data archives in the future will know LTO tapes even if only as the last tape format used before holographic data crystals replaced them. LTO drives can read tapes written on drives back two versions, so today’s current LTO-5 drive can read LTO-3 and LTO-4 tapes as well as its native LTO-5 format. Since LTO-3 drives are still being produced, a library with LTO-3 and LTO-5 drives can read any LTO tape written since LTO-1 was introduced back in 2000.

Some of the folks I talk to just don’t want to go through a tape format upgrade every few years. To those folks I say don’t. Just because LTO-6 tape drives come out, you don’t need to get them right away, I mean, these aren’t new iPhones. If LTO-5 gives you the capacity and performance you need, keep it. Wait until LTO-7 comes out.

In a well-managed archive, data should be migrated off old media before it’s stranded by retiring the drives that can read it, regardless of whether it’s stored on disk or tape. Tape vendors have been promising 20-year data retention on their products for years. Being a trusting soul, I recommend tapes be validated every two to three years, and that data be migrated to new tapes every 10 years.

There are significant economic pressures to migrate data from one generation of disk storage as quickly as every five years. The biggest pressure comes from the storage vendor significantly raising maintenance costs or discontinuing maintenance on systems by declaring them at end of life. Migrating to a new disk array also means a four to eight times increase in capacity per drive, and a corresponding reduction in power consumption.

Where disk array vendors practice 1950s-era planned obsolescence to get you to upgrade every five years, tape library vendors usually assume a longer service life for their products. As a result, it’s common to see users running 10 year old tape libraries, but rare to see 10-year-old disk arrays running in a data center.

Of course, vendor support policies vary. I’ve had vendors tell me that if I replace the LTO-1 drives in a library with new LTO-3 drives, the drives' warranty end date would be the library warranty end date, meaning there would be no warranty on drives put in a 7-year-old library. On the other hand, Spectra Logic has promised to provide service for libraries as long as it can get spare parts and not to raise its service rates beyond the rate of inflation.

Migrating data from one disk array to another is straightforward--just copy the files--but once you’re talking about tens of petabytes of data, nothing happens quickly. Transferring the contents of a typical midrange deduplicating target, like a DD690, DXi6700 or StoreOnce 4324 that holds 200 to 600 Tbytes of deduplicated data and can move data at 8 Tbytes per hour to a replacement, will take several days.

Transferring data from an old tape to a new one is a simple tape-copy command, but restacking data so the data from four LTO-5 tapes are consolidated on to a single LTO-7 is more complicated, as the archiving software has to be aware of the data’s new location. I’m looking forward to LTFS (LTO File System, a self-describing format for storing files on LTO-5 and later tapes) archive apps and libraries that can restack data automatically by policy.

Tape-format rot shouldn’t dissuade you from using tape for the long run. You’ll have to migrate data half as often as the disk folks if you implement every other LTO version. Next, we have to simplify the migration process.

Disclaimer: EMC and HP’s Storeonce group are clients of DeepStorage.net, Spectra Logic flew me to Boulder, bought me a few nice meals and drinks and worst of all made me think at their analyst’s day.