With storage-virtualization software, you can enable all the discrete JBODs connected to your SAN to be viewed as one virtual pool. This is not a new idea--mainframe systems used this approach to storage years ago--but it's just now starting to have significant impact on PC servers. Storage virtualization is an abstraction layer between the physical storage and the server operating system. Any storage connected to the network can be virtualized. This means that the storage in your SAN and on your varied NAS devices and discrete, attached-storage subsystems can be put in one storage pool.

One of the key ingredients for virtualization functionality moving forward is iSCSI or proprietary SCSI-over-Ethernet technologies (for an explanation, see November 12, 2001, "iSCSI: A New Era in Storage"). ISCSI is the encapsulation of standard SCSI commands into an IP packet, for transmission across an IP-based network. This kind of technology will allow the expansion of SANs into familiar technologies such as Ethernet. When coupled with iSCSI, the proposed 10 Gigabit Ethernet standard that will be coming of age this year poses a significant threat to standard Fibre Channel.

Every hard-disk subsystem has a range of storage that we think of as unusable. What we mean by this is that most system administrators will not allow a given individual storage subsystem to exceed between 90 percent and 97 percent capacity before adding more disk space. This figure varies depending on the intended use of the individual storage subsystem but holds true for most systems.

In fact, Gartner Dataquest reports that most companies have an excess of storage capacity, with an average utilization of the entire storage pool--both on the SAN network and in discrete storage components--of between 30 percent and 60 percent. This means that if you have an 800-GB storage array, it's possible to leave idle as much as 160 GB of storage on one array. Even at a 95 percent level, 40 GB of storage are not being used on an 800-GB array. If you spread this around to all your storage subsystems, you will see that there is a great deal of storage space just wasting away.

Virtualization solves that problem by taking all your discrete and SAN storage and making it one logical pool. Then, when Server X reaches 95 percent, you simply allocate more to the unit. The storage you allocate comes from the "pool" on the network. This also helps in the case of the overprovisioned server. You know the box we mean: It's the one on which you have 200 GB of available storage space for an application that takes 10 GB, but because of company "hardware-unification policies" you had to buy it anyway. That storage is now in the virtualization pool and is available for other servers to use. It transforms reallocation tasks from a time-consuming--and downtime-creating--experience to something that can be done from the comfort of one's own office.

The ultimate goal is to have your storage utilization for the entire SAN end up around the 85 percent mark.

Other advantages of virtualization involve automatic allocation, with e-mail notification for the administrator for systems that must have sufficient space. This kind of fine-tuning to allow for core-system reliability is one of the key benefits of virtualization software.

Although virtualization is still in its infancy, it eventually should let network administrators apply policies to the data, helping to curb the data explosion that has occurred because of the proliferation of cheap, plentiful storage.

Many companies have people--both inside and outside of the IS department--who simply say, "Why do I have to clean up my files? Just buy more storage." This can to lead to cost explosions on the network. Unused data, duplicate data and data that does not need to be on live storage can all be put out to long-term pasture or deleted.

Virtualization provides relief, letting administrators focus on cost-saving, high-level issues, not on the basic issues of moving spare storage from System A to System B. It will let the storage be used much more efficiently, all the way from low-level allocation to enabling time for high-level management of data through metadata statistics. This is considered one of the fastest growing SAN management-software segments. Tools for this particular market include Veritas Software's Volume Manager, with a more than 95 percent market share; followed by HP's Storage Allocator, DataCore Software Corp.'s SANsymphony and storage-virtualization features now embedded in Sun Microsystems' Solaris 8 operating system. Up and coming companies with new products include FalconStor and LeftHand Networks.

Data replication is not to be confused with the kind of replication that happens on the application level, such as what is done with DB2, Informix and Oracle database architectures or with e-mail architectures. This category deals with replication of the physical data. A good example would be the "dual data center" concept, where data from the live data center is replicated, usually on a delayed basis, to a secondary data center in a remote location. This system prevents data loss and system downtime should a disaster disable the primary data center.

Another piece of this market is the snapshot software that you see on many midrange and high-end NAS devices. This kind of software takes a snapshot of your data at a given time, then holds it on disk. You can, for example, make drastic changes to your databases and, if the changes don't work out, revert back using the image you took just before making the changes. These backup/restore operations happen much faster than restoration from tape and can be scheduled or performed on demand. The clear market leader among vendors of these products is EMC, with a 58.2 percent market share for its block-level array-based replication products.

Enterprise Storage Resource Management (ESRM)