Kill the Traditional PCIe Slot: A Modest Proposal

I've been inspired by the work of the Open Compute Project to propose my own change to traditional server form factors. The 2.5-inch PCIe slots now appearing on servers for hot-swap SSDs should replace current PCIe slots entirely. This would let me move all the connections, with the exception of power, to the front of the server.

Moving all the connections to the front of the server has several significant maintenance advantages, especially in large data centers.

First, it eliminates cable management arms. I'm sure someone has a data center full of 1u servers with properly installed cable management arms that allow them to pull a running server out of the rack without disconnecting any cables--but not in any data center I've ever worked in.

In my experience, there's the guy who read that power and Ethernet cables don't mix, so he declared that power cables shouldn't be in the management arm. Or someone was just lazy and used the 5-foot Ethernet cable, because running it through the arm would have required him to go hunt around for a 7-foot cable. Plus, cable management arms sag enough to get tangled on each other and block the airflow from the back of the servers into the hot aisle, causing hot air re-ingestion.

Second, it eliminates the need to make multiple trips to the back and the front of the server whenever you want to add a NIC or HBA. Consider the steps involved: pull the server out, open the server, install the card (which may require you to remove other stuff in the way), push the server back in, connect cables to new card, and then power up the server again. In a large data center with rows that are 20 or more racks long, and a hot aisle temp of 120 degrees F, going from front to back can take a while and be uncomfortable at best.

If we take a typical 1u server and replace its eight to ten 2.5-inch disk drive slots with slots that are the same size but include PCIe connections as well--like the ones on Dell's newest servers--we could use those slots for more than just the Micron PCIe SSDs that Dell currently sells for them. If Emulex, Qlogic and Intel made Fibre Channel HBAs and 10Gbps Ethernet CNAs, the vast majority of corporate servers could eliminate conventional PCIe slots all together.

The server vendors can move the LOM (LAN on Motherboard), IPMI/DRAC/iLO, VGA and USB ports to the front and we'd just connect power in the back. PCIe cards would now be in the fresh airflow from the cold aisle, rather than breathing air that's been heated by the disk drives and processors. The airflow would improve reliability.

The connector Dell uses supports SAS, SATA and four lanes of PCIe connection (as does the standardized version from the SSD Form Factor Working Group), so all eight slots could be used for disk drives or I/O cards. I expect most servers would have two disks for boot and one or two I/O cards.

My technical experts tell me that even though the PCIe SIG promotes the current versions of the bus as fully hot swappable, you'd probably have to reboot when installing a new card. That would still be several fewer steps than in today's world and would keep you from running back and forth between the hot and cold aisles, which my mother always told me would make me catch cold.

Sure, cabinet vendors and cable management vendors like Panduit will have to come up with clever new ways to dress the cables (though the server vendors could help by replacing the little ring handles with ones that swivel to hold cables), but we'd save on all the ball-bearing rails and cable management arms.

So I/O card and server vendors, what do you think? Is this idea crazy like a fox--or just plain crazy? And when can I buy a 2.5-inch CNA?