Review: iSCSI Modular SANs

We tested four dedicated iSCSI SANs in our Green Bay, Wis., Real-World Labs®: EMC's Clariion CX300i, EqualLogic's PS200E, LeftHand Networks' LeftHand SAN with Network Storage Module (NSM) 150 and MPC Computers' DataFrame 420. We also sent invitations to Aberdeen, which declined because its product is still in development, and Raid, which turned us down because of prior commitments. Relatively few vendors have committed to building dedicated iSCSI SANs. Instead, many are easing their way into this market by adding support for virtual, block-level iSCSI volumes to their NAS (network-attached storage) products. Although these solutions offer a level of convenience for environments that require a mix of file- and block-level storage (see "IP Storage Devices"), we chose to maintain our focus on SAN-specific products dedicated to providing block-level storage.

Three of those products--the EqualLogic, LeftHand Networks and MPC--are independent SAN modules that can be virtualized into a dynamically scalable storage environment and targeted in the sub-100-TB range. This building-block premise takes advantage of the power of multiple storage controllers and linear bandwidth increases through the clustering of multiple modules and their active Gigabit Ethernet ports.

The EMC, by contrast, uses the conventional SAN model: A centralized storage controller manages multiple dependent disk arrays. Although this architecture reduces the cost of additional disk arrays, the CX300i supports a maximum of 60 drives and tops out at 19.2 TB. Additional drive cabinets are linked with dual FC loops, but because the number of available Gigabit Ethernet ports stays the same even when you add drive enclosures, beware of the potential for bandwidth bottlenecking as the system hits its maximum capacity.

All four systems have features we consider basic to an iSCSI SAN environment: dynamic volume expansion, provisions for snapshots and replication, support for CHAP (Challenge Handshake Authentication Protocol), iSNS (Internet Storage Name Service) discovery and LUN (logical unit number) masking. In all cases, additional disk arrays can be added to the storage pool, managed from a centralized console and assigned without the interruption of production, though array performance will be reduced during reallocation. Every SAN we tested came fully loaded with drives in every slot, though each one lets you minimally configure it with three to five initial drives while offering a range of drive-size options for future growth. For more information, see our features chart.

In our real-world performance tests, the EqualLogic dominated the competition, with MPC and LeftHand Networks following at a distant second and third. For reasons EMC and Network Computing couldn't resolve at deadline, the CX300i had severe problems with Iometer read tests larger than 64 KB and produced questionable results from the matching write tests, even though smaller transfers worked as expected. Because all the other systems performed these tests under identical conditions and with repeatable results, we had no choice but to give EMC a low score in performance.

We compared the features, performance, management, scalability and price (both list and cost per terabyte) of all four systems. It was a relatively easy call to present our Editor's Choice award to EqualLogic. The PS200E Storage Array has noteworthy ease of use, and its performance ran circles around the rest. LeftHand Networks' NSM and MPC's DataFrame share common software, but the DataFrame has the lowest cost per terabyte, while LeftHand's product offers growth through smaller modules. EMC's Clariion has the widest matrix of configuration options and would be a great fit for environments with an installed base of EMC gear, but it tops the list in price as it unexplainably bottomed out in performance.

EqualLogic understands iSCSI. It offers a killer combination of storage-optimized controllers, efficient internal data architecture and a simplified control interface that makes storage management incredibly simple. Command-line die-hards who like to control their environment drive by drive may not embrace EqualLogic's simplified option set, but for everyone else, it's hard to dislike a system as fast and easy to use as the PS200E.

The PS200E is a completely independent, 3-U, 14-drive iSCSI SAN with the security of fully redundant, hot-swappable cooling, power supplies and controller/communications modules. Each controller contains a 64-bit RISC processor with 2 GB of battery-protected cache memory, dual serial console ports and three Gigabit Ethernet ports whose choice of copper or optical interfaces supports failover, bonding or an internal method of load-balancing across all three. The management software resides on a Compact Flash card mounted on each controller module, and the storage environment can be administered inline, locally or remotely over HTTP, SSH, SSL or telnet.Our test setup required an initial serial connection, through which we configured the Gigabit Ethernet ports and network addresses, as well as assigned names to the storage device and iSCSI "management group." Each of the two PS200Es we tested came loaded with 14 400-GB, 7,200-rpm SATA drives, for 5.6 TB of raw storage. Upon creating the storage group, we could choose either RAID 10 for optimum performance or RAID 50 for maximum storage.

EqualLogic's initial storage was easier to configure than any other test unit. Disk allocation is handled at group level automatically, and drives are assigned to arrays. Hot spares are specified based on the storage group's initial RAID configuration. Using this method, any new PS200Es added to the group automatically inherit the existing members' configuration. This sophisticated approach lets group members dynamically reprovision storage across disk resources, seamlessly load balance for efficiency, eliminate hot spots and even remove entire group members by automatically vacating the data to other members if space allows.

Base configuration for both PS200E units took only a few minutes. After assigning both modules to the management group, we logged in to the Web management interface to create volumes for our test servers. A simple three-step wizard helped allocate the volume, set iSCSI access and verify our choices to create the volume.The PS200E simply blew the competition away. The SAN's read/write statistics exceeded the other competitors' scores by as much as 2-to-1, and the 44,468 IOps score we recorded on our 512-byte read test was not only through the roof, but consistently repeatable.

The management interface provides excellent graphical information on group status and storage allocation, and a huge amount of statistical information on physical member status. In addition, the PS200E offers the snapshot and remote replication features you'd expect from the most sophisticated SAN environments, without any additional software or licensing costs. Remote management groups can be configured as replication partners, with one-way, two-way (reciprocal) or centralized replication, in which multiple groups can replicate to a specified group.

The PS200E offers data-center-class redundancy, impressive performance and high-end software capabilities that are hard to beat. Although each unit priced at the high end of the systems tested, our calculated price per terabyte came in at $9,250, with all software included. Only MPC's DataFrame came in with a lower cost per terabyte.

» PS200E (5.6TB), $51,800, EqualLogic, (888) 579-9762, (603) 579-9762. www.equallogic.com

The economical DataFrame 420 is MPC's first-generation iSCSI storage array. It came in as the least expensive setup, as well as the second-best performer, of our test group. Although the NSM 150 and this unit use LeftHand Networks' SAN/iQ software, MPC won over LeftHand in our dual-module performance tests with a combination of higher drive count and dual processors. The DataFrame 420 also supports SAN/iQ's automated LUN growth feature, and can support Fibre Channel using a PCI expansion port in the back of the module. This capability and SAN/iQ's ability to manage LUNs for iSCSI and FC simultaneously let the DataFrame 420 be dual-purposed.The 3U DataFrame 420 modules we tested came configured with 16 hot-swappable, 250-GB, 7,200-rpm SATA drives for 4 TB of raw storage. Each unit runs on dual 3-GHz Xeon processors, 4 GB of installed memory (12 GB maximum), dual power supplies and dual onboard Gigabit Ethernet ports that can be set for failover or bonding. The OS resides on redundant Compact Flash cards, and initial module configuration is done through a serial console interface.

Thanks to the SAN/iQ software, the DataFrame 420's configuration and setup were virtually identical to the NSM 150's. After configuring the IP addresses, we located the storage modules and built management groups, clusters, volumes and authentication groups for our test servers. The only difference we found was that the DataFrame 420's version of SAN/iQ doesn't offer volume lists. Although this made all the LUNs on the SAN visible to every iSCSI initiator, the SAN/iQ authentication model protected the LUNs from multiple accesses.

Unlike LeftHand's offering, MPC provides the SAN/iQ software package as a number of optional modules. Companies that want to use more than one DataFrame in a clustered group must buy a Scalability Pak to support virtualization across multiple modules. The base software also supports manual snapshots, but automated snapshots require the Configurable Snapshot Pak add-on. This arrangement seemed odd at first, but it keeps you from paying for more features than you need.The MPC DataFrame 420 offers a good combination of storage capacity and features, and in performance testing the DataFrame came in second, with read capabilities that trailed EqualLogic but exceeded LeftHand in both IOps and MBps. When we compiled the cost of setting up our two-module SAN, the DataFrame was the least expensive of the lot at only $6,500 per terabyte, including the one copy each of the optional Scalability and Snapshot expansion paks.

» DataFrame 420 (4TB), $21,998.00 (with basic SAN/iQ software), MPC Computers, (888) 224-4247, (208) 893-3434. www.mpccorp.com

With just 640 GB of raw storage per module, this SAN is the smallest, lowest priced SAN we tested. Such a configuration allows better incremental growth and options for creating redundancy at module level than the competition. The low entry price, however, becomes expensive for large-scale growth--more than $15,000 per terabyte, far higher than EqualLogic's or MPC's per-terabyte costs.

Each 1U storage module comes with four warm-swappable, 164-GB, 7,200-rpm SATA drives. The system contains a 2.8-GHz Xeon processor with 2 GB of RAM, one power supply, a nonvolatile RAM cache and dual Gigabit Ethernet ports that support failover or bonding. Initial configuration of the base IP setting is done using a serial console, and from that point all SAN control is handled by LeftHand's flagship product, the host-based SAN/iQ management software. SAN/iQ has a unique, automated LUN growth feature that offers great flexibility in storage allocation.

The SAN/iQ software provides a platform for managing and clustering modular SANs, regardless of fabric. When you load the Centralized Management Console, it searches the network to make any available storage devices accessible in the user interface. You can then log in to the individual modules to configure the RAID levels. The management software supports RAID 0, 1, 5, 10 and 50 across multiple modules.LeftHand suggests environments with multiple four-drive units can be configured as RAID 0 striped arrays at module level for greater performance and space. The vendor also recommends building volumes across multiple modules configured as a cluster, enabling two- or three-way volume replication, and using a whole module as a hot spare--an interesting option for some applications, but an unlikely scenario for the data center. For our dual-module tests, we created RAID 5 volumes and let the software split volumes across them.

Creating volumes and making them accessible is a multilayered process. Each storage module must be added to a management group, which is assigned a separate IP address for group management. Arrays are then created within that group and assigned to storage clusters. Next, volumes are created on the clusters and assigned to volume lists, which then must be assigned to authentication groups that control access to the volumes. This interface offers a great deal of granularity, but requires a fair amount of configuration time. The management interface does a good job of showing you the relationships within a complex storage environment, but it should provide more graphical information about the actual storage allocation status.

Through SAN/iQ, the NSM 150 offers another interesting feature: the ability to automatically grow volumes by establishing a target size with "soft" and "hard" thresholds. Rather than have you allocate a fixed amount of storage for each volume and end up with space that may never be used, this lets you start with smaller volumes that could increase without operator intervention to make more effective use of your actual storage space. Volumes support two- or three-way local replication, based on the number of modules available and the amount of space you want to dedicate to them. Remote replication is another option. The system also supports both local and remote automated snapshots, with hard and soft virtual allocation options similar to those of a standard volume.The NSM 150 offered midrange performance, but came in third with read speeds averaging 25 percent below those of MPC overall. This was to be expected, given that the NSM 150 is based on only four drives. The NSM 150 did better than the DataFrame 420 on the 8-KB and 64-KB write tests, edging out MPC by an average of 15 percent.

» NSM 150 (640 GB), $9,900 (with SAN/iQ software), LeftHand Networks, (866) 4-IPSANS, (303) 449-4100. www.lefthandnetworks.com

One of the largest Fibre Channel storage vendors, EMC is prepared to cash in on iSCSI's growing popularity. The vendor has introduced three dedicated iSCSI Clariion models, as well as fabric-switched iSCSI options for the larger, Symmetrix line of FC SANs. The iSCSI Clariion models differ from their FC counterparts only in their Gigabit Ethernet, rather than FC, network interface ports.

The CX300i we tested falls in the lower midrange of EMC's iSCSI offerings. Unlike the other models in this review, it starts with a base storage controller that expands through the use of up to three expansion-drive trays. Additional trays are connected by dedicated FC links, but all external access to the SAN is directed through the controllers' iSCSI ports.

The base CX300i processor chassis is a 3U array whose dual storage controller modules have dedicated storage processors, 2 GB of system cache and dual power supplies protected by an optional 1U battery module. Each storage processor has one Gigabit Ethernet management port, two Gigabit Ethernet iSCSI ports that operate in a failover mode, one serial port, one battery-status port and two FC ports for connecting additional drive modules. The CX300i was the only system that didn't use SATA drives; it came loaded with 15 146-GB, 10,000-rpm FC drives--2.19 TB of raw storage.Preparing the CX300i for use was the most involved of the SANs we tested, mostly because of the substantial number of control options. The Ethernet management ports handled the initial setup, while EMC's Navisphere Manager software took care of managing the SAN. The device needed additional agent software at server level during setup. The Navisphere management interface required a multilayered, manual process to build arrays, define storage groups, assign LUNs and set permissions. This level of complexity and granularity reminded us of LeftHand's SAN/iQ management approach. Because the CX300i is based on a dual-headed processing model, LUNs are assigned to one of the two available storage processors rather than being virtualized across multiple group members, though LUNs can be set to fail over or load balance across storage processors with additional software.

Unfortunately, the problems we encountered during testing were still unresolved at deadline. With our network set up exactly the same as our other vendor tests, the CX300i turned in very good performance in our NWC Custom and 8-KB read/write tests. But no matter what we tried, we couldn't get read tests of 64 KB and above to run correctly on the EMC system, even though we could read and write normally from the LUN in Windows. We spent hours working with EMC engineers to figure out the problem. Although we eliminated the network as the culprit, we have yet to determine whether the problems stemmed from the SAN's storage-system software, hardware or an unknown factor related to our test servers.

Under the circumstances, we had to score the CX300i accordingly. It's possible that the problem was specific to our test environment, but there wasn't enough time for us to resolve the issues with EMC before deadline.

Aside from the unresolved performance problem, the CX300i offers data-center-class construction and redundancy, and shares common application software with much of EMC's extensive line of FC SAN products. Optional software is required for snapshots and replication, and another package is available to provide enhanced failover, monitoring and performance-analysis capabilities.Unlike the other modular SANs, the CX300i is designed to support 365 GB to 19.2 TB of storage, and can be upgraded with a circuit board replacement to a larger CX500i without the need to migrate data. The unit can be customized via a large matrix of disk and software options, so the average cost per terabyte of a two-chassis, 30-disk SAN with software varies from $11,300 for a dual array of 15 FC and 15 ATA drives to $26,400 for an ultrahigh-performance, 30-FC drive/15,000-rpm array.

» EMC Clariion CX300i (2.2 TB), $43,700 (with Navisphere software), EMC, (800) 445-2588, (508) 435-1000. www.emc.com

Editor's Note: EMC has notified us that the problems we experienced with the Clariion CX300i during this review have been identified and resolved. Read more here.

Steven Hill owns and operates ToneCurve Technology, a digital imaging consulting company. Write to him at [email protected]. Post a comment or question on this story at www.nwc.com/go/ask.html.

R E V I E W

iSCI Modular SANs

Sorry,
your browser
is not Java
enabled

Welcome to NETWORK COMPUTING's Interactive Report Card, v2. To launch it, click on the Interactive Report Card ® icon above. The program components take a few moments to load.

Once launched, enter your own product feature weights and click the Recalc button. The Interactive Report Card ® will re-sort (and re-grade!) the products based on the new category weights

you entered.

Click here for more information about our Interactive Report Card ®.

In comparing iSCSI SANs, we considered several factors at both the hardware and software levels. Our pricing scores were based on per-module cost as well as estimated cost per terabyte, including the hardware and software required to build our two-module test SAN. To assess scalability, we examined expansion capabilities both within the array and between modules. We judged management based on ease of use, control capabilities and quality of the status information provided.

To test performance, we created a dedicated storage network with five white box Windows 2000 servers with 2.5-GHz Pentium processors and 512 MB of RAM. The units were connected to a Nortel Networks 5510-48 GbE Datacenter Switch using Intel Pro 1000/XT Server NICs and Microsoft's iSCSI Initiator 1.06. Each SAN was configured to use two independent storage controllers working in combination with one active GbE port per controller, with multiple ports assigned to failover. Jumbo frames and port bonding were kept disabled to minimize network-related inconsistencies.

All arrays were configured to use RAID 5 or RAID 50, and each server was assigned a 20-GB LUN (logical unit number) that was formatted as a basic-NTFS volume. Server loads were balanced across both available controllers to the best of each storage system's ability. We used Iometer 2004.07.30 (www.iometer.org) to create a 2-GB source file, which we copied to each LUN on our test servers. In Iometer, all tests were configured to allow four outstanding I/Os. They were each set to run for 1.5 minutes, with a ramp-up of 30 seconds.

For all linear read/write tests, we used one worker on each server and set the read/write distribution to 100 percent read for the read tests and 100 percent write for the write tests. Random/sequential distribution was set at 100 percent sequential.

Our IOps read test used a 512-byte transfer-request size. It was run with one worker for each server at 100 percent read and 100 percent sequential with four outstanding I/Os.Our NWC Custom Test ran using one worker for each server, and was set to transfer request sizes of 8 KB with 33 percent access distribution, 8 KB with 34 percent access distribution and 64 KB with 33 percent access distribution. In the first 8-KB segment, we set the read/write distribution to 100 percent read at 100 percent sequential. In the second 8-KB segment, we set read/write distribution to 67 percent read and 33 percent write at 100 percent random. In the 64-KB segment, we set read/write distribution to 100 percent read at 100 percent sequential.

All Storage Pipeline product reviews are conducted by current or former IT professionals in our Real-World Labs® or partner labs, according to our own test criteria. Vendor involvement is limited to assistance in configuration and troubleshooting. Storage Pipeline schedules reviews based solely on our editorial judgment of reader needs, and we conduct tests and publish results without vendor influence.

At last, there's a real choice in the SAN arena. Once the exclusive domain of Fibre Channel, SAN setups are now available with the less expensive, easier-to-use iSCSI interface. The result is storage that can connect easily in any Gigabit Ethernet environment.

Still, iSCSI SANs are facing the standard catch-22: Few vendors want to jump in with both feet until the technology proves to be a hit, and that won't happen until there's enough choice. As a result, many vendors are adding block-level iSCSI support to their NAS (network-attached storage) lines, but few have introduced dedicated iSCSI SANs.

We found just four vendors with iSCSI SAN modules to test: EMC, EqualLogic, LeftHand Networks and MPC Computers. The EqualLogic PS200E's combination of great features, winning performance and easy setup made it a shoe-in for Editor's Choice.0