Facebook Debuts Data Center Fabric Aggregator

At the Open Compute Project Summit in San Jose on Tuesday, Facebook engineers showcased their latest disaggregated networking design, taking the wraps off new data center hardware. Microsoft, meanwhile, announced an effort to disaggregate solid-state drives to make them more flexible for the cloud.

The Fabric Aggregator, built on Facebook's Wedge 100 gigabit top-of-rack switch and Open Switching System (FBOSS) software, is designed as a distributed network system to accommodate the social media giant's rapid growth. The company is planning to build its twelfth data center and is expanding one in Nebraska from two buildings to six.

"We had tremendous growth of east-west traffic," Sree Sankar, technical product manager at Facebook said, referring to the traffic flowing between buildings in a Facebook data center region. "We needed a change in the aggregation tier. We were already using the largest chassis switch."

The company needed a system that would provide power efficiency and have a flexible design, she said. Engineers used Wedge 100 and FBOSS as building blocks and developed a cabling assembly unit to emulate the backplane. The design provides operational efficiency, 60% better power efficiency, and higher port density. Sankar said Facebook was able to deploy it quickly in its data center regions in the past nine months. Engineers can easily scale Fabric Aggregator up or down according to data center demands.

"It redefines network capacity in our data centers," she said.

fabric aggregator.jpg

Facebook engineers wrote a detailed description of Fabric Aggregator in a blog post. They submitted the specifications for all the backplane options to the OCP, continuing their sharing tradition. Facebook's networking contributions to OCP include its Wedge switch and Edge Fabric traffic control system. The company has been a major proponent of network disaggregation, saying traditional proprietary network gear doesn't provide the flexibility and agility they need.

Seven years ago, Facebook spearheaded the creation of the Open Compute Project with a focus on open data center components such as racks and servers. The OCP now counts more than 4,000 engineers involved in its various projects and more than 370 specification and design packages, OCP CEO Rocky Bullock said in kicking off this week's OCP Summit, which drew some 3,000 attendees.  

Microsoft unveils Project Denali

While Facebook built on its disaggregated networking approach, Microsoft announced Project Denali, an effort to create new standards for flash storage to optimize it for the cloud through disaggregation.

Kushagra Vaid, general manager of Azure Infrastructure at Microsoft, said cloud providers are top consumers of flash storage, which amounts to billions of dollars in annual spending. SSDs, however, with their "monolithic architecture" aren't designed to be cloud friendly, he said.  

Any SSD innovation requires that the entire device be tested, and new functionality isn't provided in a consistent manner, he said. "At cloud scale, we want to drive every bit of efficiency," Vaid said. Microsoft engineers wanted to figure out a way to provide the same kind of flexibility and agility with SSDs as disaggregation brought to networking.

"Why can't we do the same thing with SSDs?" he said.

Project Denali "standardizes the SSD firmware interfaces by disaggregating the functionality for software-defined data layout and media management," Vaid wrote in a blog post.

Project Denali.png

"Project Denali is a standardization and evolution of Open Channel that defines the roles of SSD vs. that of the host in a standard interface. Media management, error correction, mapping of bad blocks and other functionality specific to the flash generation stays on the device while the host receives random writes, transmits streams of sequential writes, maintains the address map, and performs garbage collection. Denali allows for support of FPGAs or microcontrollers on the host side," he wrote.

Vaid said this disaggregation provides a lot of benefits. "The point of creating a standard is to give choice and provide flexibility… You can start to think at a bigger scale because of this disaggregation, and have each layer focus on what it does best."

Microsoft is working with several partners including CNEX Labs and Intel on Project Denali, which it plans to contribute to the OCP.

Hear more from Facebook and the Open Compute Project when they present live at the Network Transformation Summit at Interop ITX, April 30 and May 1 in Las Vegas. Register now!