The CoreBuilder 3500 is a regular Ethernet switch, with four distinct queues and support for a variety of media types. The 3500 also supports 802.1p and 802.1Q in production mode at the time of this writing, which is rather unusual (although probably not so unusual by the time you read this). The device can support 802.1Q or legacy devices on any port, and either accept packets that already have been prioritized or prioritize packets upon entry by matching specific protocols to specific priority values. For example, if you want to set Notes traffic to a higher priority than RealAudio, simply enable a filter in the switch for those protocols, then define the specific priority values you want associated with them.

The 3Com NICs that we tested also support 802.1Q directly through the use of 3Com's DynamicAccess software. DynamicAccess implements prioritization much in the same way as the 3500 does: It analyzes the outbound network traffic and then sets the 802.1Q priority based on predefined mapping. DynamicAccess is currently available for Windows95 and Windows NT, and supports IP and IPX on Ethernet and token ring. Future versions will support Unix and NetWare, as well as FDDI adapters.

Testing consisted of a pair of 200-MHz Dell Computer Corp. clients sending a variety of UDP (User Datagram Protocol) and TCP packets to a 266-MHz Dell PowerEdge server. The clients were set to run at 100 Mbps, while the server was set to 10 Mbps, allowing us to create a significant amount of congestion on the egress port. We then modified the prioritization characteristics for each of the clients and measured the number of packets received from each.

In general, things worked as one would expect, with both clients sending roughly the same amount of data with prioritization disabled, and one client sending approximately four times as much data as the other when highly discrepant prioritization values were assigned. However, these results depended on a wide variety of variables.

At first we saw very different results from the UDP and TCP tests that we performed. While UDP performance on the prioritized client rose quite steadily, TCP performance did not appear to improve much.

Since TCP requires a significant amount of handshaking and acknowledgments, it is very dependent on both ends of the connection being able to communicate effectively. In our first tests, we had enabled prioritization only on the client ports, which allowed the traffic to be sent very quickly. However, we had not enabled prioritization on the server port, so the switch did not return the server's ACK responses to the client's data very quickly at all, instead placing the unmarked packets into the low-priority queue. Once we enabled prioritization on both ends of the connection, the TCP performance of the client rose dramatically.

Most e-mail and database systems use TCP, so having the ability to implement prioritization services at all of the end-point systems will play an important part in your ability to use prioritization services effectively with those applications. Even those applications that rely on UDP traffic from both end points (such as voice over IP) will find this to be an important design consideration.

Hardware Matters ... This same basic conundrum manifested itself in many scenarios. During another test, we placed the server on another switch that was not 802.1Q-compliant and connected the legacy switch to the 3500. In this situation, the two clients were connected to the 3500 (with different levels of prioritization), while the server was connected to a legacy switch.

Although this model worked well during the UDP testing (where the clients were simply jamming packets through the 3500 to the older switch), it failed miserably in our TCP tests, as the ACK responses were unable to make it back through to the 3500 in a timely manner when sent from the legacy switch. Once we enabled per-packet prioritization on the 3500's legacy port, TCP performance started to climb again.

This type of environment will be quite common during migration efforts, and was the initial topology suggested to us by 3Com. It is important to realize that this architecture has a very negative impact on TCP connections, where both ends of the connection have to be able to communicate in order for either side to send data quickly. If you need prioritization to work with TCP, you'll need the gear everywhere.

Another hardware-specific aspect that can dictate your ability to effectively prioritize traffic is the number of queues that a switch provides and how much control you have over the amount of bandwidth and processing time that you want to allocate to those queues. For example, the 3500 we used only offered four queues, although there are eight different levels of prioritization available with 802.1Q. Thus, we were forced to group multiple priority levels to each queue, which shot any chance of our having eight distinct levels of service. You should demand eight queues if you plan to take advantage of distinct levels of prioritization in a big way.

Furthermore, we found that we only had a few levels of queue-processing available to us: "high," "low" and "best effort," with the additional option of flagging each queue as drop-eligible. Only by setting the high-priority queue to "high" and the low-priority queue to "low" with frame-dropping enabled were we able to get a diverse spread on our throughput tests. We would have liked to be able to devote more granular levels of priority to each of the queues, rather than simply defining a single high-priority queue and three low-priority ones.

... But Pipe Matters More However, the most dramatic problem we encountered was with sustained testing. The longer we ran our prioritization tests, the more often we would encounter problems, with the low-priority node simply falling off the network. Since each topology has a maximum amount of time that a frame can be delayed, during periods of sustained congestion the high-priority traffic will eventually take up all the available queue space, resulting in the lower-priority traffic being dropped at ever-increasing rates.

With UDP, this goes unnoticed from the sender's perspective, as the packets are sent without worry. But from the destination's point of view, the effect is that the traffic has simply stopped. Conversely, with TCP things get quite a bit messier. First a packet gets dropped, and thus no ACK is generated. After a while, the client's TCP stack will try to resend and will fail again. Finally, the client just gives up.

This is exactly what happens with e-mail and database clients on a saturated wire, by the way, which is why it is important to give your mission-critical applications a sufficient priority level to maintain operations in periods of high congestion.

Along these same lines, it's important to realize that you can't compensate for this by bumping up the priority of everything. Having all traffic run at the highest priority is exactly the same as having everything run at low priority: Everything fights for limited resources, and everything suffers.

This simply illustrates the need for sufficient bandwidth in general. Prioritization only allows you to pick which frames should be sent through (assuming they can be), but does not give you any additional bandwidth.

A simple rule of thumb to remember is that the importance of prioritization is equal to the level of contention for bandwidth. If you never have any contention whatsoever, then you don't need any prioritization. But if your network is fully saturated, you absolutely need prioritization services in order to ensure that the really important data gets through.

Eric Hall is president of EHS Co., a network technology research and testing company in San Mateo, Calif. He can be reached at ehall@ehsco.com.