Data centers

08:06 AM
Greg Ferro
Greg Ferro
Commentary
Connect Directly
Twitter
LinkedIn
Google+
RSS
E-Mail
50%
50%

25 GbE: A Big Deal That Will Arrive

Why all the hubbub about 25 Gigabit Ethernet? It allows the biggest jump in data center Ethernet performance for the smallest jump in price. That's why.

Companies like Google, Microsoft, Arista, and Mellanox are pushing the development of a 25 Gigabit Ethernet standard for top-of-tack server networking. Some may question the need for this technology, but they will soon see the benefits.

As Intel x86 CPUs improve in I/O and speed, 10 Gigabit Ethernet is no longer fast enough. For server to top-of-rack network connections to keep up, you would need to double the number of switches in each rack and use 10 GbE NICs. The hidden costs make this impractical.

Why 25 Gigabit Ethernet?
The next generation of Intel "Grantley" CPUs are due to ship in the third quarter, and servers with these processors will be able to forward data faster than 10 Gbit/s. In the high-density server farms of cloud providers like Google and Microsoft, using multiple 10 GbE would require twice as many Ethernet switches with their associated space, power, and cooling costs.

The 100 GbE Ethernet standard consists of four lanes of 25 Gbit/s Ethernet on four fiber or copper pairs. Small form-factor and C form-factor pluggable transceiver (SFP/CFP) modules have four lasers, each transmitting at 25 Gbit/s. Each lane requires a serializer/deserializer (SerDes) chipset.

The proposed 25 GbE standard will use the same physical silicon from a single 25 Gbit/s lane. This simplifies the process with just minor changes for forward error correction and lane alignment, and it reduces the cost when compared to 40 GbE.

"We expect, in the first generation, 2.5 times the performance at 25 gigabit for the 1.5 times the price of 10G," Anshul Sadana, senior vice president of customer engineering at Arista Networks, told me. "And in the second generation, we expect pricing parity for 2.5 times the performance."

Getting 25 Gbit/s performance for the same price of 10 -- combined with reduced operating costs -- makes this a compelling proposal.

IEEE, consensus, and snails' pace
The 25 Gigabit Ethernet Consortium has formed to accelerate IEEE adoption of standards. The IEEE has a notoriously slow process for voting and consensus building. This has led to splinter organizations like the Ethernet Alliance and the Metro Ethernet Forum, groups that develop standards and technology at a more practical and useful pace before passing them to the IEEE for approval.

This often presents a confusing picture of standards development and undermines the market belief in the IEEE as a meaningful organization. But if influencing the IEEE 802.3 board is the only way to get things done, then clearly the institution is not working in the best interests of the market.

Consortium value
The inclusion of Microsoft and Google in the 25 GbE group is significant, because they are major customers. At the end of 2013, the research firm Dell'Oro estimated last year that 10% of server sales were white-box servers. Other reports have suggested the number could be as high as 30%. Arista is known to supply both Microsoft and Google with its technology, including Mellanox network adapters.

Brocade most recently joined the consortium's ranks. And 10 additional vendors are reportedly in the final stages of signing up.

Physical connectors and cables
The standard does not specify physical cabling, but Sadana confirmed that the manufacturers will decide what to support. Practitioners expect twin-axial cable to be preferred initially over fiber for its lower power and cost.

Right move at the right time
During the IEEE discussions in the mid-2000s, there was much debate about 25 GbE versus 40 GbE. At the time, the committee chose to focus on delivering standards for backbone connections between switches for long-haul wide-area networks. In 2014, the rise of cloud providers has changed the landscape for data center Ethernet. There's now a large market and increased demand for high-speed connectivity in the rack at a moderate cost.

The simplicity and use of the related components have the secondary benefit of increasing growth in 100 GbE. Increased production of VCSELs and other photonic components will drive down cost, allowing for greater adoption. Forward-thinking organizations like Google are sure to be considering this.

It's difficult to predict when 25 GbE will arrive, but it could happen in late 2015. However, don't expect to use 25 GbE in your enterprise data centers until 2016, when mainstream adoption is expected.

Greg has nearly 30 years of experience as an IT infrastructure engineer and has been focused on data networking for about 20, including 12 years as Cisco CCIE. He has worked in Asia and Europe as a network engineer and architect for a wide range of large and small firms in ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Threaded  |  Newest First  |  Oldest First
Jeff Jerome
50%
50%
Jeff Jerome,
User Rank: Apprentice
7/29/2014 | 9:44:40 AM
Need a big fat pipe coming in too
Great article interesting points and nice graphics.  As we look at what Google is doing with dropping fiber to the home and doing tests in certain geographies do you think that will migrate over to the pipes coming in too a data center?  Which could also be located in the same geography.
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Apprentice
7/29/2014 | 6:51:37 PM
A compelling argument for 25 Gbs
I had assumed the next step from 10 Gbs Etherenet would be to 40 Gbs, but the argument that the top of rack switch needs to move to 25 Gbs is a compelling one.
jgherbert
50%
50%
jgherbert,
User Rank: Ninja
7/31/2014 | 11:44:14 PM
Re: A compelling argument for 25 Gbs
*smiles* I'm suddenly reminded of the bad old ATM days when 10MBps Ethernet was common, and IBM really really really wanted to see 25Mbps ATM to the desktop.
Pablo Valerio
50%
50%
Pablo Valerio,
User Rank: Author
7/30/2014 | 6:48:02 AM
IEEE Approval
"The IEEE has a notoriously slow process for voting and consensus building. This has led to splinter organizations like the Ethernet Alliance and the Metro Ethernet Forum, groups that develop standards and technology at a more practical and useful pace before passing them to the IEEE for approval."

That is true, but the IEEE approval process also ensures compliance and compatibility. Left to their own devices many vendors and "alliances" will push more closed "standards" and form a cartel.
AbeG
50%
50%
AbeG,
User Rank: Black Belt
7/31/2014 | 11:07:56 PM
Digital Divide
I wonder if this will have a significant impact on ISPs and in turn, the debate on the digital divide.  As far as the US goes, there's still a tremendous amount of buildout necessary to bring faster speed to certain areas.  This affects not just consumers but companies as well.
Cartoon
Slideshows
Audio Interviews
Archived Audio Interviews
Jeremy Schulman, founder of Schprockits, a network automation startup operating in stealth mode, joins us to explore whether networking professionals all need to learn programming in order to remain employed.
White Papers
Register for Network Computing Newsletters
Current Issue
2014 Private Cloud Survey
2014 Private Cloud Survey
Respondents are on a roll: 53% brought their private clouds from concept to production in less than one year, and 60% ­extend their clouds across multiple datacenters. But expertise is scarce, with 51% saying acquiring skilled employees is a roadblock.
Video
Twitter Feed