Obviously, if you just blindly build a tree with no thought to where your servers and clients reside and who talks to what most often, then you are begging for trouble. But even well-designed networks are constrained. Greg Ferro's Network Fabric:TRILL for Server and Network People. Welcome RBridges describes some other problems in common network designs. Adding a new server means having to figure out the best place to cable it in so that most clients will access it the quickest way. The tree is one reason why there were so many departmental servers deployed. But with companies consolidating servers back into a data center, the pressure is building yet again.
What we really want is to have traffic move between nodes across the shortest path through the network while retaining redundancy and capacity. Enter shortest-path bridging. The idea with shortest-path bridging is to use a well-understood routing protocol, Intermediate System to Intermediate System, IS-IS, (is it "is is" or "eye-S eye-S"?) to determine the shortest path between nodes automagically. There are two standards works in progress. The IETF's Transparent Interconnection of Lots of Links (Trill) and the IEEE 802.1aq Shortest Path Bridging are roughly similar in that they are both trying to solve the same problem. Both are using IS-IS, and both are working with VLANs. In both protocol designs, an overriding goal was to have it be largely automatic and transparent to upper layers like IP. I am not at a point yet where I have a bead on which protocol will become widely deployed. Let's hope the switch vendors converge on one and one only or fully support both.
Basically they work like this. Each switch advertises the nodes it knows about to all the other switches, and eventually all the switches in the network have the same picture of the network and therefore can forward frames to the next hop in the shortest path. IS-IS is different from other routing protocols where each switch or router node has only a partial view of the path between nodes. In the event of the topology change, a switch or link is added or removed, then all the switches have to converge all over again, a process that could take a few minutes depending on the number of switches and links involved. So you have to go learn IS-IS. I'd start now since link-state routing protocols are different from distance-vector routing protocols, and you will have to properly configure how IS-IS behaves.
There are several upsides with shortest-path bridging. The first is that you can easily form mesh networks that distribute load more evenly across your network topology since you have removed big choke points like core links for traffic that only needs to go from one distribution switch to another. You can stop wasting money on redundant links that sit idle until there is a failure. Given the price of high bandwidth ports, you definitely want to take advantage of everything you have. And you can design your network to better suit your needs rather than having to shoe horn into a tree. In essence, there is the potential to flatten your network removing one or more tiers while providing better performance between nodes.
The downside is that a shortest-path network is going to be far more dynamic than what you are used to. The IS-IS topology can quickly change based on events in your switch gear. That makes traffic management and troubleshooting difficult. In a tree, you can put network taps or span ports at choke points that feed analyzers and see what is going on. When the paths through the network are dynamic, you don't know where the flows are going. Also, when the network changes, it will take time to converge.
