From Skepticism to Standing Room Only: Liquid Cooling's Breakout Year

At last November’s annual Supercomputing Conference, researchers, scientists, developers, and industry leaders gather for brainy discussions on topics like fault-tolerant algorithms, computational biology, and quantum computing. This year, however, a surprising subject found its way to panel presentations and exhibit booths: liquid cooling for data centers.

Computing giants like Lenovo and Dell demonstrated their latest powerful servers with state-of-the-art AI chips, complete with liquid cooling systems to keep them running smoothly. I’ve been in the data center industry for 15 years, and this was the first time liquid cooling has received this level of attention. It also demonstrated how far the conversation around liquid cooling has advanced, even from the beginning of 2023.

The first hint came at Data Center World in April, a conference that typically focuses on what’s important to people running facilities. I led the Content Advisory Board there in 2022 and 2023 — the first year included a feisty conversation between a tech analyst who wanted to have a session focused on liquid cooling and a data center manager who said liquid was never coming into his or his peers’ data centers (it was “just for the Metas of the world”). However, in 2023, that conversation started to evolve, and we agreed to host an educational workshop introducing liquid to many attendees.  

That workshop, it turns out, was a standing-room-only event filled with people eager to learn more. It was clear that most people were in the early stages of their journey, with some confusion around what “liquid” meant and a surprising amount of dogma for such an early-stage market.

Liquid cooling in data centers continued to pop up on conference schedules through the summer and fall. DCD Connect was held in June in California, where the issue of water scarcity was on everyone’s minds. Presenters gave stark examples of how much water data centers consume — up to 5 million gallons a day, as much as a town of 50,000 people. Or that in an air-cooled data center using chilled water, a single ChatGPT conversation uses 500ml of water, about the same as a 16-ounce plastic bottle. Those comparisons really drove home the importance of the fact that two-phase direct-to-chip liquid cooling solution consumes zero water.

A couple of months later, at the Open Compute Project Global Summit, the show floor could have been renamed Liquid Cooling Hall with booths everywhere you looked. That was my first surprise, given that OCP covers a wide range of open hardware and software topics. The session on liquid cooling for data centers was packed again and delivered an even bigger surprise: in just a few months, the questions had advanced from “What’s the benefit of liquid cooling?” to very specific and technical inquiries about coolants and dielectrics. It turns out that companies are trying to understand not only which liquid cooling technology will take off but also which dielectric inside of the liquid cooling technology will take off. Companies that build pumps and connectors can’t test for new liquids for every client, so they’re seeking consensus.

Another consensus seems to be building around direct-to-chip systems versus immersion systems. Now that early adopters are coming back to manufacturers with real-world feedback, and the discussion has moved beyond the theoretical and revealed issues that include dealing with leaks and trouble servicing water-based solutions. Anecdotally, one frustrated immersion vendor said they thought they were 2-3 years behind direct-to-chip on the adoption curve.

In September, real estate management firm JLL delivered a terrific market update at the Data Center Anti-Conference, a gathering that focuses on the entire lifecycle of the data center. Among the highlights from JLL were its findings about how the AI boom is leading to new infrastructure to accommodate those high-power-density server clusters. “Additional innovations will be needed to improve cooling and energy efficiency for AI uses given the sustainability goals of hyperscalers and colocation providers,” the report said.

This brings us back to Supercomputing to close out 2023, where leading high-powered computing companies like Supermicro and NVIDIA were requiring liquid cooling to keep their systems running. If industry conferences can be seen as a guide, it’s clear that 2023 was a year like no other for liquid cooling. If I had to guess how far and how fast the conversation has advanced in less than a year, I definitely would have been wrong. But in this case, I would have been happy to be wrong. And now I’m more eager than ever to see what happens next.

Liz Cruz is the Director of Product Marketing at Accelsius.

Related articles: