Cisco Joins 20-Gig InfiniBand Party

Cisco's newly launched 20-Gbit/s InfiniBand switches come a bit behind the competition, but the vendor thinks the improved management tools it's throwing into the mix will bring InfiniBand into the data center. (See Cisco Doubles Up InfiniBand.)

Unveiled today, the Cisco SFS 7000D Double Data Rate (DDR) switches bump up the bandwidth of InfiniBand from 10 to 20 Gbit/s. Cisco says it will begin shipping 24-, 144-, and 288-port DDR switches in August.

Voltaire and SilverStorm Technologies are already shipping DDR switches. (See Voltaire Doubles Up, Voltaire Delivers 20 Gbit/s, and Livermore Clusters Voltaire.) There's a reason Cisco is behind. It got into InfiniBand by acquiring Topspin, which did not manufacture switches with more than 96 ports. (See Cisco Takes On Topspin.) So Cisco gets its larger switches through an OEM deal with SilverStorm.

Krish Ramakrishnan, VP of Cisco's server switching and virtualization division, says the increased bandwidth is only part of Cisco's strategy for moving InfiniBand beyond a high-performance computing (HPC) interconnect and into the data center.

Cisco modeled its InfiniBand switch management software after its Ethernet management applications. CiscoWorks LAN Management Solution (LMS) now supports InfiniBand, so administrators can manage Ethernet and InfiniBand through the same console.The theory is that while HPC users are willing to write their own scripts and use open-source for managing InfiniBand, data center administrators want management tools they are familiar with.

"We call this making InfiniBand data-center-ready. Our management tools were nowhere near what people need. People want enterprise-class management for InfiniBand rather than rag-tag tools," Ramakrishnan says, adding that InfiniBand had little or no support for important functions like network sniffers or port mirroring. But Cisco had all these tools on the Ethernet side and moved them over to the InfiniBand column. "I think in the long haul all this management will collapse under one umbrella."

At least one InfiniBand user is less than overwhelmed about the idea of better management. "It's just another interface to the network using a different protocol," says Saker Klippsten, head of engineering for special effects shop Zoic Studios. "You would think they would have done that from the start."

Zoic uses Cisco InfiniBand switches on an Isilon clustered file server system for rendering. (See Isilon Taps the Accelerator.) Klippsten says performance monitoring is a nice addition but uses open source software called CATCI to do that now.

"So far, we have not had to manage our InfiniBand switches to a large extent, only a few configuration settings," he says. "I'm an old-school, command-line type of guy, so software to manage the switches is not a big deal. What is a big deal is having the overall view of the network and how the switches are performing -- more of a bird's eye view on how data is moving around."But Klippsten is enthused about DDR. "The more the merrier," he says of the extra bandwidth.

Ramakrishnan sees DDR and better management going hand-in-hand to bring InfiniBand to the data center. He hopes the extra speed will prompt users to consider InfiniBand as a connection for server-to-storage rather than server-to-server. He also sees more data center applications requiring high bandwidth and low latency, such as Monte Carlo simulations and market data feeds.

Of course, Cisco certainly isn't looking to kill its Ethernet business. How does Ramakrishnan see Ethernet and InfiniBand shake out?

"People who want low latency at an acceptable price will most likely gravitate toward InfiniBand," he says. "Customers who want general purpose networking but would like to operate at 10-gig will deploy Ethernet. By default, Ethernet will be there and will make up the majority of the data center. But we're seeing a trend with customers deploying huge computer farms with different fabrics."

Dave Raffo, News Editor, Byte and SwitchOrganizations mentioned in this article: