The highly scalable nature of a SAN makes this network of servers and storage devices interconnected with Fibre Channel hubs and switches a hot topic. With the e-commerce explosion, many companies are feeling the growing pains associated with data management. Providing sufficient disk space is only part of the equation. In fact, storage is so inexpensive that IT administrators give little thought to adding 50 GBs here and there when their servers run low on disk space. But as the server farms grow, the overhead associated with directly attached storage balloons out of control, causing administrators to manage data reactively. SANs let you manage virtually all your storage needs proactively while creating the high availability required by the server.

For e-commerce and companies with extensive ERP (enterprise resource planning) implementations, server farms have become the heartbeat of operations, and missing a beat could have fatal consequences. In a server-clustered environment, it makes sense to consolidate storage. To attain the high-availability benefits of a cluster, availability practices need to be carried to the storage subsystem. By sharing storage pools in a SAN, servers sharing data sets can fail over seamlessly. And, thanks to the high-speed Fibre Channel infrastructure between storage pools, you can reduce disaster recovery from several hours to less than a few minutes.

With anything but a smooth entry into the market, SAN vendors floundered as they attempted to define their own Fibre Channel standards. Luckily, the Fibre Channel Industry Association, the Storage Networking Industry Association and other organizations have pushed the technology and pooled resources to help develop a set of standards for management. However, these groups have yet to develop standards to address the lack of interoperability between switched fabric implementations. Compaq Computer Corp., EMC Corp., Hewlett-Packard Co., Sun Microsystems and other vendors have been quick to embrace the growing SAN market, offering complete solutions. Additionally, many vendors now offer a full line of products you can use to brew your own SAN.

Data Decisions

The first step in developing a SAN is to pinpoint the applications to be addressed. Whether you're designing a cluster of database and Web servers using a single data pool, a high-performance data-streaming network, a data-mining method or a unified backup system, you need to give careful consideration to the SAN infrastructure. Port densities, distance and bandwidth requirements, availability and segmentation all vary according to the application.

In a heterogeneous environment, it's important to examine the platforms that will participate in the SAN. Hardware and software support for SANs varies depending on the platforms. Once you have addressed these fundamental questions, you can begin constructing the SAN. Similar to a typical Ethernet infrastructure, a SAN comprises a few basic components: the Fibre Channel disk storage and tape libraries, fiber hubs and switches, HBAs (host bus adapters), and some form of SAN management.

To evaluate SAN functionality, we constructed in our Real-World Labs® at the University of Wisconsin-Madison a SAN consisting of six Intel servers running a combination of HBAs from Emulex Corp. and Qlogic Corp. For connectivity, we used a Vixel Corp. Vixel 2100 Zoning Managed Hub, as well as a Vixel 8100 Fabric Switch. Storage was provided by an nStor Technologies NexStor 18F fiber JBOD with 324 GB of disk storage. Our tape library consisted of an ADIC Scalar 218FC configured with two internal DLT drives.

In designing your SAN, one of the most important architectural hardware decisions is whether to go with an arbitrated loop or switched fabric. Arbitrated loop, with its shared bandwidth and round-robin data forwarding, was once the only choice. Relatively new is switched fabric, which dedicates full bandwidth on each port and allows simultaneous data transfers to a single node. Depending on your scaling and performance needs, a simple hub may suffice. In a larger storage environment, however, a fiber switch is a must. The hubs and switches in a SAN play a similar role to those in the Ethernet network.

In a workgroup or smaller departmental environment, a good foundation is a Fibre Channel hub in an arbitrated-loop configuration. Configured with four to 16 ports, hubs are attractive because they provide a high degree of interoperability for a low price. As per the Fibre Channel specifications, hubs support an aggregate bandwidth of 100 MB per second. Although a hub can support up to 127 devices, realistically you should restrict the total to just 30 devices. Because per-port costs of a switch are significantly higher than those of a hub (about $1,000 compared with $300), a hub is ideally suited to fan out the core switch ports to the connecting servers.

Hubs are limited in their scalability in part because of the way devices are added to the loop. To become aware of the other devices in the loop, each device must perform a LIP (loop initialization sequencer) when it is first attached. This action suspends the loop while the entire population on the loop acquires or verifies the current port addresses and is assigned an AL_PA (arbitrated loop physical address). Although the LIPs happen extremely fast, throughput-sensitive applications, such as video streaming and tape backups, are sensitive to hiccups. Also, if one device on the loop flakes out and floods the SAN with LIP calls, it could slow the SAN to a crawl or bring it down entirely. Many vendors have incorporated diagnostics to isolate the unresponsive port and drop it from the loop automatically.

For example, Vixel has produced a hybrid hub that lets its eight ports be segmented into up to four separate arbitrated loops, with up to 100 MB per second in each loop. We configured the devices on our SAN into three separate loops. To share devices in the different loops on the hub, we configured the fourth loop with devices from other ports in the other loops.

One of the benefits of using hubs is that interoperability between the different hub manufacturers is not a problem. The AL_PA process is transparent to the hubs. Therefore, negotiation between hubs is not required. Some of the major SAN hub and switch vendors include Ancor Communications, Brocade Communications Systems, McData Corp. and Vixel.