Shortest Path Bridging Will Rock Your World

Whether you're planning to deploy FCoE in the data center or voice and video on your LAN, chances are you will have to redesign your network to be more efficient and robust than it is today. The way we design LANs today means we often over-provision because we build choke points through which all traffic has to pass even when the traffic is going to a neighbor. The distribution and core layers have to be over built just to overcome the choke points. It's wasteful and only addresses a symptom of a tree like design, such as inefficient paths. Short path bridging is going to change all that.

Take a look at your LAN architecture. It's a tree with a single path from one node to another. Ethernet is a dumb protocol. The switching logic goes something like this: Step 1: Determine which port has the destination MAC, Step 2: Spit the frame out that port.

Ethernet forwarding's benefit is that it's simple. The drawback is that you are limited in what you can do. If you have two 48Gb switches that can forward a full 96Gb between all ports on the switch, and host A on switch A wants to talk to host B on switch B, the frames have to pass through--you guessed it-- a 1Gbps uplink port. To get more inter-switch capacity, you need either a fatter uplink like a 10Gb port or you can aggregate multiple 1Gb ports together. Switch-to-switch isn't interesting but highlights the uplink problem.

If you have three or more Ethernet switches, you can't link them together in a mesh. Remember, Ethernet forwarding is dumb. Putting three or more switches into a mesh creates a loop, which means the switches keep tossing frames to each other like a hot potato. What you have to do is ensure that there is one path and only one path between switches. Anyone who has run cable will tell you that is easier said than done. I don't know any IT person who hasn't mistakenly created a network loop. The spanning tree protocol addresses that issue by ensuring that when there is a loop in the network, the network devices discover the loop and disable one link to remove the loop. Problem solved. A nice side effect is that we can now have redundant links between switches--intentional loops--for fail-over and spanning tree will fix it. The downside is you're paying for uplinks that aren't being used.

But we have only eliminated loops, and we haven't changed network design much. We still have to contend with choke points in a multi-tier architecture. Whether you have an access tier, distribution tier, core tier or some other number of tiers, the basic design is a tree with one path from any node to any other node. As you go up the network, you need to add capacity to handle the aggregation of all traffic.Obviously, if you just blindly build a tree with no thought to where your servers and clients reside and who talks to what most often, then you are begging for trouble. But even well-designed networks are constrained. Greg Ferro's Network Fabric:TRILL for Server and Network People. Welcome RBridges describes some other problems in common network designs. Adding a new server means having to figure out the best place to cable it in so that most clients will access it the quickest way. The tree is one reason why there were so many departmental servers deployed. But with companies consolidating servers back into a data center, the pressure is building yet again.

What we really want is to have traffic move between nodes across the shortest path through the network while retaining redundancy and capacity. Enter shortest-path bridging. The idea with shortest-path bridging is to use a well-understood routing protocol, Intermediate System to Intermediate System, IS-IS, (is it "is is" or "eye-S eye-S"?) to determine the shortest path between nodes automagically. There are two standards works in progress. The IETF's Transparent Interconnection of Lots of Links (Trill) and the IEEE 802.1aq Shortest Path Bridging are roughly similar in that they are both trying to solve the same problem. Both are using IS-IS, and both are working with VLANs. In both protocol designs, an overriding goal was to have it be largely automatic and transparent to upper layers like IP. I am not at a point yet where I have a bead on which protocol will become widely deployed. Let's hope the switch vendors converge on one and one only or fully support both.

Basically they work like this. Each switch advertises the nodes it knows about to all the other switches, and eventually all the switches in the network have the same picture of the network and therefore can forward frames to the next hop in the shortest path. IS-IS is different from other routing protocols where each switch or router node has only a partial view of the path between nodes. In the event of the topology change, a switch or link is added or removed, then all the switches have to converge all over again, a process that could take a few minutes depending on the number of switches and links involved. So you have to go learn IS-IS. I'd start now since link-state routing protocols are different from distance-vector routing protocols, and you will have to properly configure how IS-IS behaves.

There are several upsides with shortest-path bridging. The first is that you can easily form mesh networks that distribute load more evenly across your network topology since you have removed big choke points like core links for traffic that only needs to go from one distribution switch to another. You can stop wasting money on redundant links that sit idle until there is a failure. Given the price of high bandwidth ports, you definitely want to take advantage of everything you have. And you can design your network to better suit your needs rather than having to shoe horn into a tree. In essence, there is the potential to flatten your network removing one or more tiers while providing better performance between nodes.

The downside is that a shortest-path network is going to be far more dynamic than what you are used to. The IS-IS topology can quickly change based on events in your switch gear. That makes traffic management and troubleshooting difficult. In a tree, you can put network taps or span ports at choke points that feed  analyzers and see what is going on. When the paths through the network are dynamic, you don't know where the flows are going. Also, when the network changes, it will take time to converge.TRILL is well along the standardization path and is as good as done as is 802.1aq. When we will start to see them showing up in products (and whether this will be a software or hardware upgrade) is anyone's guess. Regardless of what you do with your network, you are going to want to use short-path bridging if you can. After coming back from Voicecon and hearing about the professed increased adoption of VoIP and video-conferencing, as well as other changes that are impacting the network, it's clear that the rigid network designs of today will be a hindrance tomorrow. Luckily, you don't have to make a wholesale switch, rather you can slowly migrate sections of you network.