The Survivor's Guide to 2004: Infrastructure

If you took our advice and put off purchasing 10-Gigabit last year, you'll be rewarded this year. Although 10-Gigabit is still costly, prices have come down sharply. Last year, prices typically ranged from $50,000 to $100,000 per port. This year, we've seen prices from $25,000 to $50,000 per port--in some cases, even less. And, as we anticipated, vendors have released second-generation Ethernet cards that are much more likely to have wire-speed performance.

The investment is clearly a hefty one, but it's better to bite the bullet and spend the money than to suffer the consequences of a saturated backbone. A good rule of thumb is to look for 60-Mbps spikes at five-minute intervals, then monitor in more detail during those intervals. Although all network traffic is different, users start seeing performance degradation when network utilization hits 100 percent, even for one second.

Just make sure the card you buy can handle access-control lists and QoS (Quality of Service) at wire speed--if there's an ASIC (application-specific integration circuit) involved, it very likely can, but get a written assurance from your vendor. If you don't need the card just yet, bear in mind that the longer you wait, the less expensive it will be. Time is on your side.

Cheaper Copper

If you're planning to deploy 10-Gigabit within the data center, there's very good news: An even cheaper version of 10-Gigabit will be coming out this year. 10GBase-CX4, IEEE standard 802.3ak, is the new copper-based version. Dan Dove, chairman of the IEEE 802.3ak committee, said he expected the standard to be approved by press time.

Look for this technology to appear in vendor equipment in the first half of 2004. It will use twinax cabling with four pairs of conductors to provide copper connectivity for distances of up to 15 meters. Because expensive optics aren't used, the price will likely be a fraction of that of existing fiber-based 10-Gigabit ports. The cabling will be almost identical to Infiniband (a high-speed, external computer bus connection), but with tighter specs. Unlike most previous versions of Ethernet, the spec for the cable will come from the IEEE, not the TIA/EIA, because this cable won't involve structured wiring. It will be strictly for jumper cables, making it ideal for short runs within data centers and connections between stackable hubs.

As for 10-Gigabit over twisted pair, that's another story. We don't expect this technology to appear before 2006. A big issue with 10GBase-T is the type of twisted pair cabling it will require. The IEEE Group studying 10GBase-T appears to believe that running this 10-Gig over twisted pair for 100 meters over Category 7 cable won't be a problem. Unfortunately, Cat 7, which features individually shielded pairs, is expensive and has no installed base.

Cat 6 will likely support distances of around 55 meters, but the IEEE will be working for a way to get Cat 6 closer to 100 meters. It's not clear how well 10GBase-T will run over Cat 5e, and distance could be an issue. The shorter distances Cat 5e can handle will be suitable for little more than the data center. Installing Cat 6 instead of Cat 5e, therefore, will increase your ability to take advantage of 10-Gigabit over twisted pair.At the other end of your LAN--the edge--vendors will be pushing you even harder next year to upgrade to gigabit, but stand your ground. If you've been replacing the PCs that connect to your wiring-closet switches, you've probably noticed that many desktop computers have built-in gigabit support. Perhaps you're thinking your lowly 10- and 100-Mbps switches aren't doing them justice. But remember, business needs should drive the technology choices, not the other way around. If your applications are running fine on 10-Mbps switched connections, you mustn't let pressure from a PC or switch vendor drive your network infrastructure.Consider telecommuters with DSL or cable-modem connections of 1 Mbps or less, or workers on the road getting by with 56-Kbps modem connections. Not that we would wish 56K on anybody, and we're all for speed, but if applications run fine on 1 Mbps and get by on 56K, it's difficult to imagine how gigabit bandwidth will provide ROI (return on investment) of any kind. Truth is, the need for gigabit at the edge remains the exception, not the rule. Before your vendors convince you your network is obsolete, ask them to name one application that will run any faster with Gigabit Ethernet switches at the edge. Then have them explain how this technology will improve employees' productivity.

Of course, there are a few legitimate reasons to upgrade your network, particularly if you're thinking ahead. You may have old 10-Mbps switches that the vendor no longer supports. Or perhaps your hardware-maintenance costs would drop with new equipment. Maybe your apps really are taxing the existing 10 Mbps. If you're moving around large CAD/CAM drawings or videos, for example, gigabit connectivity may give you a boost, especially if you have a newer PC that can process more than 100 Mbps of bandwidth. Don't forget to factor in PC bottlenecks from bus, CPU and especially hard disk I/O.

If you're already running at 100 Mbps, you shouldn't think about upgrading to gigabit unless you have a clearly defined, demonstrated need. If you've already decided to upgrade your switches for other reasons we've discussed, and you have 10 Mbps, 100 Mbps will probably suffice. However, it's worth at least getting pricing on gigabit as well. Hewlett-Packard's ProCurve switches, for example, run about $30 per port for 100 Mbps and $100 per port for gigabit.When you consider that gigabit gives you 10 times the performance for about three times the cost, the value looks good. Of course, if you know you won't exceed 100 Mbps, it's a waste of resources. Frankly, it's hard for us to to imagine how most desktop business applications will benefit from exceeding 100 Mbps of bandwidth in the near future, but as soon as they do, gigabit will be the solution. And if you can negotiate a good price on it, you'll have peace of mind from knowing you'll be in good shape for a long time to come.

Making the leap to gigabit or 100 Mbps requires the right cabling. Category 5 is suitable for 100 Mbps and typically for gigabit as well, but you'll need to have it tested to verify the latter, as such support is not guaranteed. Cat 5e was designed to tighten up the Cat 5 specs to make sure gigabit will work.By now, we hope you're convinced that not everybody needs Gigabit Ethernet. That doesn't mean, however, that all new edge switches are off the bargaining table. VoIP (voice over IP) and its need for PoE (power over Ethernet), for example, provide some incentive, albeit specialized, to upgrade your edge switches.

If you're weak in the security department, remember that most switches now come with standardized, network-based authentication, or IEEE 802.1X. This feature can help you provide more security at the edge, preventing just anybody from plugging a laptop into a network behind your firewall. Most switches now have Layer 2 QoS via 802.1p traffic prioritization. Useful for VoIP phones that share an Ethernet connection with a desktop, this feature helps mitigate the need for gigabit technology at the edge (though gigabit will guarantee your voice traffic at the edge will never be stomped on, even without Layer 2 QoS).

Many new switches also come with IEEE 802.3af PoE. This feature provides 48 volts of power over the Ethernet connection and is handy for deploying many wireless access points that support the feature. PoE eliminates the need to schedule an electrician to power the access points, which can be in out-of-the-way locations. PoE can also be used for network-based cameras. The cost for PoE comes to about $100 per port for 100-Mbps connections, based on recent pricing of HP's ProCurve switches.

PoE is also important for centralizing VoIP phones' power source and eliminating the need for external power packs. Battery backup can even be done from the wiring closet. Whatever you do, though, make sure it follows the PoE standard, 802.3af, which was ratified in June of this year. And be careful: Some vendors are still selling prestandard proprietary versions. Make sure your PoE switch is guaranteed, in writing, to support 802.3af.If you do go this route, also make sure your cabling is Category 5 or better. Furthermore, you must provide for extra power requirements in your wiring closets, as your existing circuits may not have enough capacity. Save yourself some time and check with an electrician ahead of time. And don't forget to plan for the extra space for the UPSs you'll need as well.

As We've Said Before...

You may be looking for areas in which to scale back, but we can assure you, network redundancy is always worth the investment. Did you get that? Always, especially at the core. After all, if a core router or switch goes, the network is down.

Once you install redundant routers and switches at the core, try to keep the connections to the edge standards-based, using protocols like VRRP (Virtual Router Redundancy Protocol), which will make it possible to fail over from one router to another. For connections from the core out, the 802.3ad Link Aggregation standard for trunking works well. Using standards here is critical to have different vendors at the core and on the rest of the network.

You also need redundancy at the carrier level. That is, if you rely on the Internet to do business, you should have a backup ISP. Even the best carrier can have an outage. The average cost of network downtime is millions of dollars annually, according to a study by Infonetics Research. Furthermore, many small carriers have disappeared, and large carriers such as MCI, Global Crossing and Qwest have had serious difficulties. There are signs of stabilization; MCI is emerging from Chapter 11 with no debt, for example, and that could give the company a competitive edge. Nevertheless, there's still pressure on the carriers to reduce their staff, and such downsizing could disrupt service.

Also, make sure you get the most out of multiple ISPs. Ideally, you should be able to send your traffic through the best performing and least expensive ISP at any given time. Route-optimization products, such as those from Internap Network Services, RouteScience Technologies, F5 Networks and Radware, can help. We tested these products earlier this month (see "Route Optimizers: Mapping Out the Best Route") and found them capable of providing reliable backup, ISP performance and cost optimization on multiple ISP links.

Moving Up the Stack

Vendors are already pitching their IPv6 wares. Although some of these products might work, we believe it's way too early to invest time and money in IPv6 technology. The sky isn't falling, and we're not on the verge of running out of IP addresses. If your vendor rep tries to tell you otherwise, tell him or her to read Goeff Huston's "IPv4--How Long Have We Got?", to see the math and logic behind IPv4's continued viability.There are a number of potential benefits to IPv6, including security, QoS and address allocation. However, the technology is still immature. Besides the vendors, nobody except the Department of Defense and countries in the Asia-Pacific region is giving IPv6 serious consideration at this time. On the other hand, if IPv6 ever does take hold, there will be many islands that need to be connected.

Peter Morrissey is a full-time faculty member of Syracuse University's School of Information Studies, and a contributing editor and columnist for Network Computing. Write to him at [email protected].

Post a comment or question on this story.

APC: Helping to make networks more reliable by providing UPSs for everything from the data center to the wiring closet.

Extreme Networks: Its new Blackdiamond 10K switch sets new standards for security, management and uptime in a core switch/router.

Foundry Networks: Foundry provides high-performance, high-functionality network routers and switches, with a strong commitment to standards and low costs.Hewlett-Packard: Manufactures ProCurve Layer 2 and Layer 3 Switches, which are gaining market share due to low cost and great warranty without compromising features.

Internap: Provides access to multiple ISPs via their private NAPs. Internap's recent purchase of route optimization hardware vendor NetVmg and route optimization service vendor Sockeye gives it the whole package in terms of improving Internet performance.

RadWare: Provides DNS-based ISP load balancing that scales from the small office to high-speed WAN connections.

• Network Computing infrastructure white papers and research reports

• "Route Optimizers: Mapping Out the Best Route"• "Premium Network, Four Ways"

• "Life in the Really Fast Lane"

• "Saving Money With Tiered Access"

• "The New Face of Authentication"

• "TechGuide: Every Little Gigabit Helps"• "Keeping Time With Your Network"

• "Better Bandwidth Management"