HLRS

An HPC center in Germany claims that sending Fibre Channel traffic at 10-Gbit/s over DWDM (dense wave division multiplexing) has actually saved money.

"Using DWDM was much cheaper," says Peter Haas, director of data and network management at the High Performance Computing Center Stuttgart (HLRS), which is affiliated with the University of Stuttgart and the University of Hohenheim less than 10 miles away.

Earlier this year, he and his group looked into ways of upgrading the speed of their research network, which shares data from clustered computers with scientific organizations worldwide. The two universities, about 7.5 miles apart, were already linked via a 2.5-Gbit/s private fiber connection. The optical links were generated by equipment from ADVA Optical Networking, which in turn connected with Fibre Channel directors from Cisco.

The group wanted its data to go faster than 2.5 Gbit/s. They considered adding a gateway module to the Cisco switches that would convert 2- and 4-Gbit/s Fibre Channel traffic to FCIP (Fibre Channel over Internet Protocol) for use on 10-Gbit/s Ethernet connections. But the cost of the gateways and the multiple, additional 4-Gbit/s links that would be required to achieve the high data rates was more than it would cost to upgrade to a full 10-Gbit/s on DWDM.

This was true, Haas maintains, even though the HLRS had to upgrade from Cisco MDS 9509 switches to 10-Gbit/s-capable MDS 9513 directors, which were still in beta test. Nonetheless, by April the configuration was up and running."We are now sharing scientific research and data around the world and have established grid computing architectures that allow us to share processing power across disparate sites for high-capacity scientific applications in a virtual setting. This network capability would not be possible without WDM as the foundation," Haas said in a prepared statement. (See ADVA Provides Solution.)

Haas says that more than speed was involved in making the 10-Gbit/s optical upgrade. Converting FC to TCP/IP and iSCSI would have introduced lots of extra code to the HLRS WAN connections, he insists, reducing performance.

How is it possible, a skeptic might ask, for a $100,000 director and ADVA's FSP 2000, which lists for roughly $40,000 for a 10-Gbit/s equipped pair of platforms, to be cheaper than a simple IP connection, particularly for sites that are so close together?

"You have consider how much data you have, the window you have in which to transport it, and the number of circuits you'd need to send it," says Todd Bundy, director of business development and alliances at ADVA.

For instance, it may seem cheaper to use an existing router (with gateways) and IP connections over leased lines to transport FC traffic -- unless it turns out that the storage traffic involves lots of backups during the day, which could slow down other applications on the network. At that point, the cost of adding more circuits could cover the cost of 10-Gbit/s DWDM.Also, if a company really needs 10-Gbit/s connectivity, DWDM can be substantially less expensive than 10-Gbit/s Sonet services, which typically are used only by ISPs and other carriers.

What's more, the cost of dark fiber has dropped to a point that makes it a reasonable alternative for corporate use -- a point that makers of optical gear, such as ADVA, Ciena, Nortel, and PacketLight, have long maintained. (See Insider: Enterprises Tap Private Fiber and Stretching SANs.)

There are many factors to consider, certainly. And on the downside, ADVA's Bundy acknowledges that for business continuity, distance can be a factor. Synchronous replication, for instance, typically can't take place on links greater than 200 kilometers, even though DWDM connections can traverse 1,000 or 2,000 miles for asynchronous backup.

At the very least, though, the news from Germany shows that for some enterprises, optical SAN extension may no longer be out of reach.

Mary Jander, Site Editor, Byte and Switch