Cisco's Nexus Targets Data Center's Future

In possibly its biggest product launch in four years, Cisco is unveiling its long-awaited follow-up to the Catalyst line of switches today, combining Ethernet and storage networking with an eye toward data center trends such as virtualization and faster switching speeds.

The ambitious new platform, named Nexus, is a stab at high-end Ethernet players such as Force10, Foundry, and Woven Systems. Cisco says it will outdo them, as Nexus will eventually reach 15 Tbit/s in switching density -- 7.5 Tbit/s each for the ingress and egress paths -- and 512 10-Gbit/s Ethernet ports per system.

Nexus will also incorporate storage switching, taking a bit of a swipe at Brocade and its Data Center Fabric strategy. (See Brocade Unveils Backbone Switch and Brocade Outlines Server/Storage Fabric.)

But the storage capabilities are yet to come: Cisco is basing its platform on Fibre Channel over Ethernet (FCOE), which won't be added to Nexus until that standard is complete. (See Storage Must Wait for Cisco's Data Center Vision .)

Figure 1: Nexus & FamilyThe Nexus 7000, flanked by its I/O modules

Perhaps more important, Nexus is aimed at the trend of virtualization in the data center, and it packs features to accommodate the kind of reliability that storage networking needs.

Nexus's arrival doesn't mean Catalyst and the storage-switching MDS line are doomed. Cisco says it has plans for all three in "Data Center 3.0," a phrase the company has been tossing around since August.

Big deal
Even though customers have to wait for FCOE and other important features -- a lossless version of Ethernet being standardized for data centers, for instance -- the impact of Cisco's new switch shouldn't be underestimated, analysts say.

"It's their biggest announcement since CRS-1," says Zeus Kerravala, an analyst with Yankee Group. (See Cisco Unveils the HFR.)

Nexus is big enough to at least temporarily stymie the competition, says analyst Nick Lippis of Lippis Enterprises."Cisco made a huge commitment here," he says. "Without Nexus, Force10 would have had a good IPO story this year. Now, investment bankers will pause."

The announcement comes one day before a Juniper press and analyst event in New York City, where that company is expected to unveil its own big-deal data center switch. (See Cisco, Juniper Ready Data Center Assault.)

When queried about this, Cisco adopts a nonchalant approach. "Interesting timing, isn't it?" says Deepak Munjal, marketing manager for Cisco's data center team. (A Cisco spokeswoman says the product had been planned for a January 29 release, but was pushed up by one day.)

Yet another OS

As with its CRS-1 router, Cisco is eager to show how big an effort Nexus has been. The platform took four years and $250 million to develop, requiring contributions from more than 500 engineers. Its innards include some technology from Cisco's MDS line of storage gear, and even some IP routing knowhow from Procket Networks, the core-routing startup Cisco acquired in 2004. (See Cisco to Pay $89M for Procket Assets.)

"No other company really has the resources and the intellectual property in the networking and storage spaces to match that," Lippis says of Cisco's effort.Nexus also sports a new operating system, something Cisco says the team insisted on from the beginning. Yes, that means having yet another operating system to support, in addition to the different trains of the Internetwork Operating System (IOS) out there, the SAN-OS that runs the MDS gear, and the modular IOS-XR.

"We realized we're making a tradeoff here, but the reality is, when you look at data center requirements like lossless Ethernet, all of those necessitated more of a new design," Munjal says.

It had to be done, Kerravala says. "All new OSs take time to mature, but that probably had to happen to bring the new functionality in."

NX-OS, as the operating system is called, does borrow from IOS and SAN-OS, and it's based on a modular architecture like Cisco's IOS XR.

To Page 2Density wars
The Nexus era starts with the Nexus 7000, due to reach general availability in the second quarter.

It's a 10-slot Ethernet switch able to hold a maximum of 256 10-Gbit/s Ethernet ports. An 18-slot version, coming "within six months," according to Munjal, would bring that figure to 512 ports, giving Cisco an apparent lead in 10-Gbit/s Ethernet density.

Table 1: Nexus's Numbers

The density crown is debatable. Woven's gear comes with multi-chassis capability that allows the creation of one gigantic mesh of Ethernet switching. Woven claims it's possible to build a monster multi-chassis switch with more than 4,000 10-Gbit/s Ethernet ports -- although it does take a lot of chassis to get that far, says Derek Granath, Woven's vice president of marketing.

Nexus will have interfaces for 40- and 100-Gbit/s Ethernet, once those standards emerge, Munjal says.The box will be outfitted with a switching fabric that handles both Ethernet and FCOE -- but that won't be available until after the FCOE standard is done. Munjal estimates a one-year wait there.

How about that 15 Tbit/s (really 7.5 Tbit/s) claim? Actually, Nexus starts out with more like 3.7 Tbit/s.

Here's the math: The Nexus 7000 holds five switching module cards, each capable of offering 46 Gbit/s of traffic to every line card. Each line card has access to all five switching modules, giving it access to 230 Gbit/s worth of switching capacity. Multiply that by eight line cards possible, then double it (because Cisco always does), and you get 3.7 Tbit/s.

Now, the bigger number comes into play with the 18-slot version. It's going to be able to deliver just less than 500 Tbit/s of switching to each of 16 line cards. The non-rounded-off numbers will multiply out to make Cisco's 15 Tbit/s mark.

That's all on paper, of course. A Nexus 7000 line card can't use all 230 Gbit/s worth of switching at once, Munjal notes. But he says future iterations will let customers "fully populate" the 230 Gbit/s, especially as 40- and 100-Gbit/s technologies become available.Getting graceful

Big numbers aside, Nexus's bigger contribution arguably lies in areas such as high availability, offering features that apparently forced Cisco's hand in creating NX-OS.

One example is a feature Cisco calls "graceful system operations." One of Ethernet's main problems is the time it takes for the network to recover when a node is shut down for administrative work. Protocols such as spanning tree redraw the network's map for the rest of the nodes, but the delay can extend beyond a few seconds -- unacceptable territory for applications like VOIP calls.

Nexus gets around the problem by letting the rest of the network correct itself first before shutting down. Nexus first resets its parameters to divert traffic away -- setting route times to infinity and blocking other nodes' spanning tree requests. This causes the network to start acting as if the Nexus is already offline while keeping the node available for traffic if necessary. Once the network is ready, the Nexus shuts down.

That won't help in cases where someone unplugs a crucial cable. And, of course, it assumes the network "converges" on a new Ethernet map; if it somehow doesn't, the Nexus has to get a hard shutdown. The point, though, is that operators can take down Nexus without having to worry about any delays elsewhere on the network.

"If you believe the data center is being increasingly virtualized, then the data center becomes network-centric where the network connects various pools of computing resources," analyst Kerravala says. "The network then needs to look and act like a virtual backplane, meaning it needs to be high performance and continuously available. So Nexus is a network device that enables better virtual computing."Catalyst keeps kicking
Amid the hoopla about Nexus, Cisco is being adamant that the box doesn't replace Catalyst or MDS.

It's easy to believe the company for now, because Cisco's plans don't call for Nexus to ever support intelligent applications such as firewalls or load balancing. Those have been be left to Catalyst.

Moreover, Nexus isn't intended for every type of data center. "You don't need Nexus functionality everywhere. Catalyst makes a great edge switch or smaller data center," Kerravala says.

MDS can breathe easy because Nexus won't support Fibre Channel. By sticking to FCOE instead, Nexus keeps an Ethernet identity. That makes it palatable to Cisco's resellers, which are accustomed to Ethernet. "The native Fibre Channel market today really is done by strategic partners," the likes of EMC or IBM, Munjal says.

Cisco is giving the venerable Catalyst some love today, partly to show the Nexus isn't obliterating it. Among Cisco's announcements today are some updates to the 6509 chassis, including a module for 16 ports of 10-Gbit/s Ethernet.Nexus starts as a core network offering for data centers and will eventually descend into the data center's access layer with blade-sized versions. But the platform is primarily aimed at the core. "It's not a box that's going to find its way into campus and metro environments," Munjal says.

Cisco expects the box to have some service provider play, too, but as an Ethernet box; it's really not meant for metro IP/MPLS networks. Nexus does carry full IP routing capabilities, but Munjal says that's meant to let the box connect to an IP network if necessary.

Have a comment on this story? Please click "Discuss" below. If you'd like to contact Byte and Switch's editors directly, send us a message.