Building a Storage Area Network

At our Network Computing Real-World Labs® in Green Bay, Wis., our primary testing SAN is a hybrid, with an Adaptec SANBlock-2 2-gigabit drive array, an Apple XServe Array and an Emulex Switch. We also have older Cisco and McData SAN switches, but we used just the Emulex switch to research this Workshop (not because it was better, but simply because it was readily available). Our two-way hyperthreaded servers with 3 GB of RAM use LSI Logic 2 Gigabit FC cards and run Windows 2000 SP 4.

Decisions, Decisions

One of the first tasks in designing a SAN is determining how much data to move from NAS and dedicated storage onto the SAN fabric. Moving just one application to a SAN isn't likely to provide you with an acceptable ROI unless that app is a bread-and- butter one. Exchange servers and databases are the best place to start when evaluating a SAN move.

Rather than require you drop the server, dismantle it, insert new drives and then work them into the system later, a SAN lets you add space and tell the system to ensure it's configured correctly. More SAN products now automatically portion out drives to needy logical volumes without your having to do anything other than click on a "looks good" button. And nearly all high-end SANs let you hot-swap drives to facilitate zero-downtime and disk space growth. These are benefits you're not likely getting from your dedicated storage, and they can make your users happier in the long run.If you don't want a SAN-in-a-box solution from Apple, Hewlett-Packard or Winchester Systems, for example, or a single-unit iSCSI solution from, say, Adaptec or EqualLogic, you'll need to buy the right individual components to make your SAN fly. These include drive enclosures (make certain you have drives to fill them), one or more SAN switches, host bus adapters, and software for management and backups. You'll also need Fibre Channel cabling, typically optical. Consider buying an FC tape backup unit to support the SAN. For some enterprises, backups are reason enough to go to a SAN--you can do high-speed disk-to-disk backups to the SAN, then stream the backups to tape at your leisure.

Be warned: FC still suffers from compatibility problems. Be sure all the different vendors' products you choose work with one another, or you may find yourself with FC SAN components that don't operate. The compatibility headaches are less common today, but vendors still often ask us for things like model number and firmware revision of the SAN pieces when they come in for testing, which is a bit disconcerting.

If the thought of building your SAN piece by piece and making certain it all works together is intimidating, buy a SAN-in-a-box kit instead, or ask your primary vendor to recommend parts for the rest of your SAN. Vendors know which components their products work with and will gladly save themselves customer-support time.

Backups are important with a SAN, and especially verifiable backups. There's no point in building a SAN if you can't restore it. The good news is you have options: true, near-real-time replication and larger tapes are available, both of which are important when you're putting a ton of data in one place and trusting that place to stay up.

Disk-to-disk-to-tape appliances are getting cheaper. And when you consider that most restores are "I just deleted a file I was working on yesterday, can you get it back"-type situations rather than massive downtime requiring a full restore, D2D2T is appealing. The last few days are on the disk, so you can restore yesterday's accidental deletion in seconds or minutes instead of tens of minutes or hours. You still have a tape to carry off-site, too.Consider how your SAN will be used, then choose a backup solution that fits. D2D2T solutions are generally more expensive than tape changers, so you'll have to decide if quick restores will be regular enough to get your ROI on the device.

Also consider how to slice up your disk space once your SAN components are all together. When the disks are in dedicated machines, you can just add disks as needed, but when you merge your space, you have to think about allocating disk space.

Logical volumes are divided up and assigned to servers, and while most solutions let you grow logical volumes, few let you shrink them. So start by modeling your existing dedicated storage for each system you're moving to the SAN, then grow the logical volumes as needed. Don't bother with ILM solutions that reduce disk space required by each machine or special calculations--each SAN, application and environment is different. The past is the best indicator of the future. If you know a specific application's storage needs are growing at 20 percent a year, then give it 120 percent of the space it had with your direct-attached storage system. You can always allocate more storage to that LUN (logical unit) in the future.

There's some trickery in sizing your SAN. Filling all the drive bays of a high-end solution with disks just big enough to support today's data doesn't guarantee future expandability. Don't let a vendor fill your box with small drives that don't grow with your SAN. Figure out how much space you'll need for five years, and size accordingly. Disk space is still cheap, but drive arrays for an FC SAN are not, nor is fitting them into your overall infrastructure. It's better to add drives than enclosures.The same goes for your switch. You need ports for each enclosure, each server and (possibly) a tape changer. Pick a switch that will support your needs for five years or more, or that has open slots so you can add ports over time (if it's a director-class device).

For backups, a multivendor fabric and virtual LUNs, you'll need management software that kicks butt. (You can split a "virtual disk" over several physical enclosures so you can manage disks from many sources--like several enclosures--as one.)

You can run the surprisingly useful tools each storage vendor includes, but they're typically too vendor-specific. Third-party tools such as Softek Storage Manager, Computer Associates/BrightStor SAN Manager and EMC/Legato's Disk Extender 2000 can help you manage different vendors' storage wares. It depends on what you need and how much you're willing to pay. This software will go a long way toward making your SAN painless.

Hook Me Up

Configuring a SAN isn't exactly plug and play, regardless of what the vendors tell you. You have to configure the switch, tell the host bus adapter cards where to find the storage and use a management tool to parse out the storage--all things that take a few tries if you've never done them before. Remember, FC is a networking protocol that requires the same stuff other networking protocols do--a way to uniquely identify a target and a path to return messages.SAN switches have grown up in many ways. They include VSAN (virtual SAN) features, which let you define things like "only these ports can see this target," similar to how VLANs trunk data. VSANs are useful for departments in your organization, such as HR, that have sensitive information on the SAN.

When you're first building your SAN, the notion of the fabric isn't that important. The fabric is the entire network of FC products, and in large installations, it is at least as complex as your LAN. If your fabric is small, there's little chance of errors causing downtime, so we won't delve into it any deeper here. Just remember that each device on the fabric has an ID as in an IP network, and duplicate names are as problematic on a SAN as on an IP network. FC's IDs are called World Wide Names, and though we don't recommend it, some vendors let you configure them yourself.

You'll have to configure permissions for your LAN. You can leave many products wide open, but it's best not to. Most switches now support either on-switch access control or some form of repository (ADS or RADIUS, for example) access control. Repository access control is easier--presumably, the users are already configured. Remember, mostly servers and applications will be accessing the SAN, so the users running your core business apps will need authentication.

Backups are backups, except that with a SAN, they're a bit more scary. A spoiled backup on your SAN could take down 20 servers and 40 applications instead of a single server with one or two applications on it. D2D2T helps: It's a lot quicker to verify a disk-based backup, and that's not even taking into consideration the massive restore-time improvements over tape-only solutions.

If you carefully evaluate your needs and seek the help of your SAN vendors, deploying your first SAN shouldn't be painful. Set aside a little extra time to configure it, though, even if the vendor calls it SAN-in-a-box. An off-the-shelf SAN won't suit your needs out of the box, so be ready to customize it. If you go with iSCSI, make certain you're working with an iSCSI SAN vendor that will be around for a while. This fast-growing market is due for some consolidation, so buy from a bigger vendor so you don't get left out in the cold.Whichever SAN solution you choose, storage- management software can help you manage and maintain uptime for your SAN. Figure out what features work for you, and buy the tool that fits best. But don't get more than you need--each software package you buy should provide unique functionality and will require maintenance.

Don MacVittie is a technology editor at Network Computing. Previously, he worked at WPS Resources as an application engineer. Write to him at [email protected].

Choosing and configuring your SAN can be simple if you cover all your bases.

1. Evaluate your current disk situation. Figure out how much disk space you're using, and how fast your data is growing.

2. Determine which applications (and therefore, how much disk space) will move to the SAN. Compound your annual growth for five years to figure out actual requirements.3. Decide on performance and feature requirements. Buy a SAN that can handle your load, but don't overbuy. A system that will support 10 million transactions per second is nice if you need it, but expensive if you don't.

4. Pick the packaging. Choose enclosures, host bus adapters and switches. Or buy a SAN-in-a-box.

5. Install and configure all the components. Each server on the SAN will require downtime for the host bus adapter to be installed, and the switches need IP connections. Then cable it all together.

6. Configure security and VLANs. Be sure you know how your switch handles access control, and set aside time to configure it.

7. Determine minimum LUN sizing. This is based on the projected space requirements you came up with (see Step 2, above).8. Build your logical volumes. Make them big enough to hold today's data, but don't go crazy--most systems let you add space later.

9. Choose a backup solution that fits your SAN. Look into FC backup appliances, and consider D2D2T appliances. If you have the extra space, replication isn't a bad idea either.