Ethernet Technology Grows Up--And Out

Your SAN might be getting a speed boost soon. New 10 Gigabit Ethernet-based systems can reach throughput levels that previously were reserved for Fibre Channel architecture, possibly making Ethernet a more attractive option for high-end storage.

Big-name vendors are entering the race with a new wave of "enhanced Ethernet" networking and storage technologies, tapping lower-cost 10-Gbps Ethernet chipsets and hardware. And they're gunning for a SAN near you.

Before they arrive on your network, however, you'll need to figure out if these offerings--which include Ethernet extensions for the data center from Brocade, Cisco, and others, as well as Fibre Channel over Ethernet, or FCoE--provide an adequate ROI. What are the risks? And where does amped-up Ethernet leave iSCSI SANs, which tap TCP/IP and standard Ethernet to haul data at thousands of small and midsize organizations?

It's easy to see why iSCSI is popular with the masses. It runs on standard Ethernet networks, and IT personnel without Fibre Channel experience can attach iSCSI servers to block-based storage devices. If you know Ethernet and TCP/IP, you can build an iSCSI SAN and inexpensively attach servers to shared storage resources.

Deployments of iSCSI have experienced rapid growth in a wide variety of applications, including file servers, Microsoft Exchange and Microsoft SQL Server databases, and VMware ESX servers. A raft of iSCSI target systems are available from major vendors including Dell EqualLogic, EMC, Hitachi Data Systems, IBM, and NetApp.

Still, iSCSI hasn't joined Fibre Channel in legacy enterprise SANs and higher-performance apps for a couple of reasons. First, most iSCSI SANs run on Gigabit Ethernet today, versus 4-Gb or 8-Gb speeds for Fibre Channel. ISCSI uses as a transport TCP/IP, which introduces overhead from protocol processing and affects end-to-end latency and throughput compared with Fibre Channel. Second, the ecosystem of tools and tactics for deploying and managing Fibre Channel SANs is mature and pervasive. ISCSI SANs fall behind Fibre Channel in this area, which is a crucial consideration for storage admins.

Wide adoption of 10-Gbps Ethernet will boost iSCSI's popularity in enterprise storage, but even with better wire speed, protocol overhead and management issues remain.

Enter FCoE

DIG DEEPER

LAN And SAN Unite

New approaches are pushing the unified storage dream closer to reality.

Download this
InformationWeek Report

>> See all our Reports <<

Fibre Channel over Ethernet, now in development and before the T11 standards committee for ratification, is being promoted by major vendors, including Brocade, Cisco, EMC, Emulex, IBM, Intel, NetApp, QLogic, and Sun Microsystems. FCoE uses enhanced Ethernet as the physical network transport to deliver the standard Fibre Channel payload, skipping the overhead of TCP/IP, and is designed to look and act like native Fibre Channel to the upper software layers, including the operating system, applications, and management tools. Fibre Channel services such as discovery, worldwide name addressing, and zoning all work the same way in FCoE as in native Fibre Channel. However, because it doesn't use TCP/IP, FCoE traffic isn't routable across IP networks. FCoE is designed exclusively for low-latency, high-performance, Layer 2 data center networks.
Not Your Father's Ethernet
The FCoE protocol, like native Fibre Channel, requires the underlying physical transport to be lossless. As a result, vendors have developed extensions to the Ethernet standard specifically for lossless 10-Gbps Ethernet speeds and data center fabrics, which could carry all types of traffic. These extensions to the Ethernet standard are referred to collectively as "Converged Enhanced Ethernet," or CEE (by Brocade and others); or "Data Center Ethernet," or DCE (by Cisco). Cisco DCE extensions are a superset of the CEE extensions, and there's no consensus on what the final group of standards will look like. For the sake of simplicity, we'll use the term "enhanced Ethernet."

Enhanced Ethernet includes features that make it suitable as a universal I/O fabric for the data center, with the ability to transport storage payloads without the overhead of TCP/IP, and without packet loss.

FCoE requires enhanced Ethernet, which in turn requires 10-Gbps Ethernet-capable chipsets and hardware, including network adapters and switches. Cisco's DCE-capable Nexus 5000 switch is available for "top-of-rack" applications in 20- and 40-port models. The larger chassis-and-blade-based Nexus 7000 is designed primarily as a data center aggregation switch, but 10-Gb line cards with DCE extensions are scheduled to ship this year. Emulex, Intel, and QLogic are shipping DCE-capable 10-Gbps Ethernet NICs designed to work with Cisco's Nexus 5000 unified fabric. Brocade will likely ship FCoE-compatible products later this year.

Unfortunately, enhanced Ethernet requires new chipsets, so "standard" 10-Gbps Ethernet NICs and switches now shipping won't be compatible with extended 10-Gbps Ethernet; new hardware will be required.

When One Is Better Than Two
Enhanced Ethernet can carry multiple payload types, including TCP/IP and FCoE, so vendors are positioning it as the universal fabric for next-generation data centers, connecting servers with storage, IP networks, and other servers for clustering applications.

Data center architects and vendors dream of the perfect "unified" data center fabric, in which a single, high-performance fabric meets the needs of IP, storage, and interserver payloads. This is the promise of enhanced Ethernet and FCoE. A unified data center fabric provides some interesting business and technical benefits, including:

Less hardware and easier management. Only one pair of NIC cables is needed per server (for redundancy), instead of two NICs and two Fibre Channel host bus adapters. One set of switches is necessary instead of two, and there's only one data center to manage.

More flexibility and adaptability. A unified fabric is an enabler for next-generation virtualized data center architectures, where servers, storage, and other resources can be dynamically allocated to adapt to changing workloads and new applications, with fewer physical changes required. This plays nicely into the move to virtualize and automate the data center.

Lower power consumption. Fewer NICs, cables, and switches mean less wattage is needed. Reducing the number of components by one-half or more would provide substantial savings.

It's About Choices
At this point, iSCSI advocates will correctly point out that SANs built on standard Ethernet and iSCSI also represent a "unified" or "converged" data center architecture, albeit one that operates over TCP/IP with the associated advantages and limitations.

In reviewing the present state of Ethernet storage networks, and looking at what's coming next, several things are clear, while others fall into the "wait and see" category.

Deploying 10-Gbps Ethernet to the server is an easy decision, especially for connecting blade chassis and virtualized servers to the network. Because of its relative low cost and ease of implementation, iSCSI has earned a well-deserved role for entry- and midlevel storage network applications. The advent of lower-cost 10-Gbps Ethernet hardware will likely drive up the rate of adoption.

FCoE's biggest area of opportunity is higher-end storage systems, especially in enterprises, where iSCSI isn't prevalent. This is because FCoE promises to deliver performance similar to or better than Fibre Channel in all applications and integrate with the existing ecosystem of Fibre Channel SAN tools--and the skill sets of SAN administrators.

Many companies are interested in a high-performance unified fabric for the data center that retains Fibre Channel payloads and services but leverages the lower cost, lower power, and other benefits that come with a single fabric. The availability of lower-cost 10-Gbps Ethernet NICs and switches is sure to help drive deployments beyond just early-adopter companies. However, vendors still have to prove that FCoE and enhanced Ethernet can deliver on the promise of a unified fabric for the enterprise.

Vince Conroy is the CTO for FusionStorm; he leads the company's managed hosting and services operation. Write to him at Vince Conroy.