Juniper's QFX5100 Switches Knit A Fabric

Juniper this week announced a new switch line that serves as the hardware platform for its MetaFabric architecture. Juniper claims the switches can play multiple roles: both a top of rack (leaf) switch and an aggregation (spine) switch, as well as campus switch.

There are three models in the QFX5100 family. The models are identical in functionality but vary in port density.

1. The QFX5100-48S has 48 10-GbE ports with six QSFP+ 40-GbE ports in a 1U form factor.

2. In a 2U form factor, the QFX5100-96S has 96 10-GbE ports, but only eight QSFP+ 40GbE ports. This is a trade-off with the 48S in terms of uplink oversubscription. While the 48S uplink oversubscription is 2:1 (480:240), the 96S is 3:1 (960:320).

3. The QFX5100-24Q has 24 fixed 40-GbE QSFP+ ports, with two expansion modules that can add four more QSFP+ ports each, all in 1U. QSFP+ allows a network operator to connect devices at 40 GbE, or to channelize the interface into four discrete 10-GbE interfaces when using a breakout cable.

The Juniper QFX5100 switches are based on the Broadcom Trident II chipset, like the recently announced Arista 7050X and Cisco Nexus 3100. Raw performance numbers are identical for the switches from these three vendors; the differences are in the software.

[New switches are only one component of Juniper's MetaFabric. Get the full picture in "Juniper Launches New Fabric Architecture, Switch Line."]

On the performance side, the QFX5100 switches are non-blocking, so they offer wire-speed forwarding. Juniper claims they get as low as 550ns port-to-port latency. While not the lowest on the market, this is respectable number, beyond adequate for most environments. By way of comparison, the Mellanox SX1036 features the lowest port-to-port latency I am aware of, at a reported 230ns. The SX1036 is aimed at organizations where fractions of a millisecond affect business operations.

The QFX5100 line has a Flexible Forwarding Table (FFT), which means the number of MAC or routing entries varies, depending on how the table resources are allocated by a network operator. It supports up to 288,000 MAC addresses in pure Layer 2 mode. In Layer 3 host mode, it supports up to 128,000 host entries. In Longest Prefix Match (LPM) mode, it supports up to 128,000 prefixes.

Juniper says the QFX5100 will be available during the end of 2013 and early 2014, depending on the model.

Software Highlights

I found several software features of the new switch line worth calling out. First, Topology-Independent In-Service Software Upgrade (TISSU) is a hitless upgrade feature. The "topology-independent" part of TISSU means an individual QFX5100 switch can upgrade to a new software image with no impact to traffic transiting the switch.

TISSU is achieved through software virtualization. Two instances of Junos run inside of virtual machines on top of Linux as an active/standby pair. One instance is upgraded and then failed over while the other instance is upgraded. Most industry ISSU schemes rely on upgrading one piece of redundant hardware at a time, reducing aggregate bandwidth by half and creating a temporary single point of failure during the upgrade process. TISSU is a positive step for data center operations.

Second, with the introduction of the MetaFabric architecture, Juniper has added Virtual Chassis Fabric, an amalgam of QFabric and Virtual Chassis ideas. QFabric was and is Juniper's flagship fabric product, targeted at large data centers that need an any-to-any, low-latency network at scale.

QFabric is limited in the sense that it can only be plumbed in a star topology: All access switches must be uplinked to a pair of special-purpose (that is, not Ethernet) interconnect switches sitting in the middle of the network. Switches participating in a Virtual Chassis act as single physical switch, with a single point of management and unified control plane. Virtual Chassis topology is more flexible than QFabric, as all switch ports are Ethernet; there is no unique interconnect uplink port. Virtual Chassis deployments to date have been limited to 10 switches.

With Virtual Chassis Fabric, up to 20 QFX5100s can be deployed in a leaf-spine topology. From two to four QFX5100s can act as spine switches; up to 18 can be leaf switches. Other Juniper switches can participate in Virtual Chassis Fabric, including the EX4300, QFX3500, and QFX3600. These products that have been in the Juniper lineup for a while now, offering customers some investment protection.

Notably, the QFX5100 can also be a pure QFabric node switch or a Virtual Chassis member. Juniper is making sure that the new switches will fit well into the designs their customers have already deployed, while at the same time giving them some interesting design options for the future.