Network Computingwww.networkcomputing.com

HSM Migrates To NetWare Platform

Hierarchical Storage Management (HSM) promises near bottomless pits of storage for modern computer networks. It automatically migrates files between more expensive, higher-speed magnetic disk and less costly, but slower jukebox-based optical disc and magnetic tape units. Medium-speed, randomly erasable media, like rewriteable optical discs serve as a secondary level of HSM storage, holding files most likely to be needed by users. Lower-speed media that can't be erased randomly, like tape or write-once optical disc, support a tertiary level of HSM storage where little-used files reside.

NetWare networks are a perfect target for HSM systems, since they tend to support large numbers of users with seemingly insatiable appetites for file storage. We tested four NetWare HSM products: Alphatronix Inspire Migrator 1.20, Avail NetSpace 3.0, Cheyenne HSM 1.0 and Palindrome HSM 3.1A.

Cheyenne's HSM was nearly perfect. Avail's NetSpace came in slightly behind, flawed by its unwavering requirement for a separate dedicated server. Palindrome's HSM lacks transparent migration back from HSM storage, is difficult to use, and has weak documentation and a more difficult installation process. Alphatronix's Inspire Migrator has some serious HSM design limitations, specifically: a lack of tertiary HSM services; support for only one server and HSM hardware set per software package; ability to run only on and support NetWare 4.x servers; and defects in its integration of HSM and backup services.

We tested using a Hewlett-Packard 40T (32 slot) optical disc jukebox fo r secondary storage and an Exabyte EXB-210 tape library (10 slot) for tertiary storage for those products that supported it. We used a mix of NetWare 3.11, 3.12 and 4.01 servers.

Attempts to meaningfully test for return migration times were unsuccessful. The relatively slow speed limits of optical disc jukebox devices and tape autoloaders proved to be more of a factor than software design. The markedly different architectural designs made performance comparisons impractical. In general, we found return migration can be slow, taking as long as a minute with optical disc and five minutes with tape. Still, experience has proven that end users are more than willing to trade instant return migration for having to back up and delete files to assure adequate primary disk storage.

Cheyenne HSM v1.0

Each product is architected differently. Cheyenne HSM does an excellent job of supporting secondary and tertiary storage. In the most basic setup, secondary and tertiary storage hardware can be connected to a single existing and nondedicated network server. All HSM software runs on that server. It can target other servers on the network for HSM migration. You must use Cheyenne's own ArcServe backup software or a third-party system for standard backup. In addition, you can use the HSM tape system or a separate tape system to specifically back up files migrated to secondary or tertiary storage.

This basic setup is the easiest and least expensive. It requires no additional servers.

Cheyenne also supports the use of two separate, nondedicated HSM servers. Under such a design, seco ndary storage can be placed on one server and tertiary storage on the other. Backup is handled just as it is in the basic design.

This design is best for networks with large amounts of migrated data or for physically larger networks. Placing secondary and tertiary storage on different servers distributes demands for the server CPU. It also can put secondary (or tertiary) storage closer to a server more likely to use its files.

NetWare HSM products come with a complex array of programs, with many parameters to set. After installation, you must monitor the HSM process frequently and change parameters often. Cheyenne provides the best installation utility and its ongoing monitoring and maintenance program is very good.

Cheyenne's installation program uses a Windows-based front end to install quickly and easily both server and workstation files. The program also makes initial system configuration a breeze. Little boxes at the bottom of the main installation screen are underlined in red as each phase of the installation begins. Knowing where we were at any given moment made it easy to select installation and configuration options for primary, secondary and tertiary storage. Unlike other products, Cheyenne's installation program also automatically installed updated NetWare Loadable Modules (NLMs), such as CLIB.NLM, and placed copies of the old versions in a specially created subdirectory.

It is also easy to monitor HSM operations, set up HSM criteria (such as migration criteria) and schedules, and change hardware setups using Cheyenne's HSM Manager. You can also run many of these functions on a server using screens provided by the various Cheyenne HSM NLMs.Finally, Cheyenne nicely integrates HSM and backup services. You can set a parameter to assure that a file is not migrated until it is backed up by a standard backup system. It also includes its own backup system for files that have already been migrated. You can use regular HSM tape units to back up migrated files or install a separate unit. Cheyenne's ArcSe rve backup software will not return files that have been migrated. Instead, it backs up the placeholder file left on primary storage when a file is migrated. If you're using backup systems from other vendors, Cheyenne HSM allows return migration to be turned off automatically for a particular application or logged-in user, which forces the backup of the placeholder file only.

Avail NetSpace v3.0

NetSpace features an HSM design rivaled only by Cheyenne HSM and uses a dedicated Storage Server to connect to secondary and tertiary HSM hardware and run HSM software. This server requires at least a run-time version of NetWare. A second non-dedicated NetWare server becomes the Domain Server, which holds NetSpace administration programs and duplicate data files for the Storage Server. A single Avail HSM system can provide HSM services for multiple servers.

NetSpace is the only product to use hard disk as the first stage in HSM storage. This design, which can significantly improve HSM performance, is more prevalent in Unix-based HSM systems. The advantage is in migration performance. The disadvantage is in the cost of a dedicated HSM server and the secondary hard disks.

In addition to any HSM storage devices, you must connect a separate tape storage unit to the storage server. This device backs up files that are migrated to HSM storage. You must still use a third-party backup system for unmigrated primary storage files and for the placeholder files. You can run the latter backup device on the Domain Server, another NetWare server or on a workstation.

NetSpace integrates HSM and backup similarly to Cheyenne HSM. Although you can't use regular HSM tape devices, NetSpace's special tape unit assures that migrated files are well-protected. In addition, an NLM assures that files won't be migrated back to primary storage when opened for backup. Instead, just the placeholder files are backed up. To further reinforce Avail's concerns about file system integrity, NetSpace comes with a very good text file explaining the steps required to recover from a variety of disasters.

We found a minor security problem with NetSpace, however. The product stores the NetWare user ID and password it uses to access servers in fully readable ASCII format on the Storage Manager. Since the Storage Manager is used only by the HSM system and users do not need to log into it, this isn't a huge problem. Avail is looking into this situation.

Palindrome HSM v3.1A

Unlike the other products, Palindrome's HSM is tightly integrated with the company's The Network Archivist (TNA) backup software. You cannot run HSM without it. Where secondary storage is similar to that of other products, tertiary storage is essentially TNA backups.

Installing Palindrome HSM requires patience and a very good memory. You must install each piece-TNA backup system, support for optical disc and tape Autoloader devices, a queuing system for file backup and HSM migration, and HSM itself -- separately. There are separate manuals for each module. Sometimes key information about a specific module is placed in the manual for another module. This is not only a frustrating factor, it makes resolving problems a more difficult task.

Monitoring the system and changing configurations almost seems more complex than it is with Palindrome's management application, which runs in DOS. Sometimes you can complete an entire operation without leaving the DOS application -- for example, when configuring file migration rules using a nice pseudographic al directory tree for any server under HSM control. At other times -- when configuring hardware devices, for example -- the DOS application kicks off an NLM. You can view the screen for that NLM under RCONSOLE by pressing a function key, but getting back to the DOS application requires a full exit from RCONSOLE.

Generally, we found all of this easier than going to a server and running the same NLMs with command line switches. However, it would have been far better if the program had launched the NLMs and then updated an information screen about the NLM within the DOS application.

Palindrome integrates HSM and backup services in a unique way. Files cannot be migrated to HSM storage until they have been fully protected by TNA. Under TNA's Tower of Hanoi backup algorithm, files are fully protected when they have been saved to permanent archive tapes at least three times. Then they won't get backed up unless they change. Thus there's no chance that HSM and backup activities will impinge on the same file simultaneously or that a file will be migrated back when opened by the backup system.

Palindrome HSM is the only product that doesn't migrate files back to a user workstation automatically. Programs must be loaded on each user's workstation under DOS or Windows, and there are no such programs for other operating systems. The other three products let NLMs on the server do all of the migration work, which is so much better. Palindrome says they will make NLM-based return migration available in the second quarter of 1995.

Alphatronix Inspire Migrator v1.20

We found the product's server-only system maintenance tools more difficult to use than the workstation-based utilities the other products provide. Inspire Migrator's ability to handle only NetWare 4.x servers makes t he product unusable in too many of today's networks. Its one server per software/hardware package is also a serious limitation. On the bright side, Alphatronix reports that it soon will be providing HSM services to multiple NetWare 4.x servers from a single Inspire Migrator server.

Being a NetWare 4.x-only product, Inspire Migrator has no problem with most aspects of HSM-backup integration. As long as a 4.x-compatible backup system is used, it only backs up the placeholder file. You can set a parameter to prohibit migration before a file is backed up. Since the optical discs used for secondary storage are in standard DOS format, the migrated files on them can be backed up using a regular DOS or Windows-based backup system. Accessing the disks could prove a major pain, however, since Inspire Migrator provides no way to do this. You'll either have to use backup software from another vendor or remove discs from the optical jukebox for backup on a standalone DOS or Windows workstation with its own tape unit and separate optical jukebox or single optical drive.

Inspire Migrator saves data to optical disc in standard DOS format. This neat little feature, first developed by Alphatronix for its Unix-based HSM systems, provides an extra bit of security for users. If the HSM system is down for some reason, you can remove a disc from the optical jukebox and then access it in a standard workstation.

Barry Gerber can be reached at bgerber@nwc.com.

Integrating HSM And Backup

HSM systems must ensure full integration with tape backup systems. When a user attempts to use a migrated file, HSM returns the file to primary storage. However, when a tape backup system attempts to back up a file, return migration should not happen. If it did, primary storage could fill up rapidly, leading to a great deal of HSM out migration activity to clear space for incoming files. This, coupled with all the return migration caused by backup, could put a strain on the system and lead to unacceptab ly long backup periods.

There is a way to solve this problem. When a file is migrated to HSM storage, some sort of placeholder remains on primary storage disk. This placeholder is used when a user requests a directory of files and, in the case of NetWare, it also holds information on NetWare file rights.

A backup system should back up only the placeholder file for a migrated file. NetWare 4.x, which has built-in support for HSM, makes it easy for backup software to do this. All that is required are calls to NetWare by the backup software. NetWare 3.x doesn't include integrated HSM services and requires that the backup and/or HSM software assure placeholder-only backup when appropriate.

Given that the problem of placeholder-only backup has been solved, HSM-backup integration must also assure that files are indeed properly backed up. One very good way to do this is to prohibit migration until files are backed up to a standard backup system. This is the approach used by most HSM systems in two different ways.

In the simple approach, a file cannot be migrated until its archive bit is turned on, meaning that it has been backed up at least once. The problem with this approach is that some files will be backed up only once before being migrated, violating the basic principle that a file should be backed up more than one time. One way to ensure complete backup under these circumstances is to add a separate system to the HSM that backs up only migrated files. This is the solution offered by Avail and Cheyenne.

Another approach is to integrate very tightly migration and backup. Here a file must be fully backed up before being migrated. Palindrome uses this approach.

An HSM By Any Other Name

Avail's NetSpace and Conner Peripheral's HSM are almost identical. Avail created the original software. NetSpace supports a wider range of adapters, tape and optical disc units than the Conner product. Conner is no longer marketing its HSM package. At the time of this writing, Arcad a Software was in the process of modifying the base Avail HSM to create its own product. Arcada is selling Conner HSM until its own version is ready.