FCoE Adapter Market Noise

A series of news items and blog posts have me worked up about the FCoE adapter market. First, Emulex and Qlogic managed to find evidence in the the latest Dell'Oro Group market research report that they were gaining market share. Frankly it was a bit Orwellian reading the battling press releases. Then my fellow NWC blogger Tom Trainer asked the engaging question "Are Converged Network Adapters Suffocating the Virtual Machine?" pointing out that some CNAs might consume more than their share of CPU resources.

Frankly the whole discussion of CNA CPU utilization reminds me of the FUD the Fibre Channel camp put out years ago about iSCSI.  They claimed software iSCSI initiators would work the poor CPUs in your servers so couldn't get decent disk performance without specialized iSCSI HBAs and TOE cards. Well, in a series of tests in Network Computing's Labs we found that the Microsoft iSCSI initiator worked just fine and that machines running the partial TCP offload included with Broadcom's latest (as of 2003) LOM chips actually outperformed the available iSCSI HBAs. Note that well over 80 percent of iSCSI servers use software initiators from Microsoft or VMware. Most users of iSCSI HBAs prefer them for simplified boot from SAN.

While I haven't seen the data Tom referred to in his blog, I'm not really all that concerned that an Emulex CNA accessing a RAM disk over FCoE could drive CPU utilization to 80 percent. After all, that was for 512 byte sequential access and the Emulex outperformed the Qlogic in the test by three times. Yes that's an unacceptable CPU utilization, but it's also an unattainable performance level. If 100,000 4K IOPS took even 30 percent of the server's CPU I'd be more concerned. Because I expect to use a tenth of the 800,000 IOPS that generated the high CPU utilization, I expect the CNA to eat 8 percent of my CPU, which is an acceptable level.

Even more significantly, by the time FCoE goes mainstream in 2011-2012 Emulex will have had time to fix their drivers, and/or come out with a next-generation CNA. After all, VMware ESX 4.0 reduced CPU utilization almost 50% for iSCSI compared to 3.5. While faster CPUs solved the iSCSI CPU problem it won't be the solution for FCoE. That's because faster CPUs just mean more VMs per host; that is if you don't run out of memory first, as I usually do.

The current Fibre Channel networking market is a classic oligopoly. Each of the two market segments, switched and HBAs, has three vendors that together have over 90 percent market share. In the switch market Brocade is the dominant player, having absorbed several former competitors. Cisco is a strong number two, while Qlogic brings up the rear selling almost exclusively into the SME market. For HBAs, Qlogic is the leader with Emulex holding the number-two position and Brocade selling a few HBAs here and there.  With a total of four vendors controlling the market, Fibre Channel products have remained expensive compared to the more competitive Ethernet market. Even worse, vendors are free to use proprietary extensions and hardware compatibility lists (HCLs) as tools for account control.

FCoE promises Fibre Channel management with Ethernet economics. To deliver on that promise we as customers have to resist the pressures from the entrenched Fibre Channel vendors to make FCoE a special case of FC where every switch, HBA/CNA/NIC and cable has to be on the HCL. Ethernet works because it's almost entirely interoperable. Yes, there are proprietary extensions, and sometimes good ones. If the market sees a proprietary extension, say EtherChannel bonding, a standard version (802.3ad) comes along soon enough.

Software FCoE initiators are hitting the market now. Today you can download either open FCoE for Linux and Starwind's FCoE initiator for Windows. NIC vendors from Broadcom and Neterion to Chelsio will soon have their own FCoE offloads. They can help make the FCoE market like the iSCSI market, where everyone assumes things will work and costs come down.