Nvidia’s next-generation DGX servers will have liquid cooling, per comments from Nvidia CEO Jensen Huang at the 2024 SIEPR Economic Summit. In between saying the upcoming system was “magnificent” and “beautiful,” Huang also mentioned the next DGX servers are “soon coming,” indicating that perhaps Nvidia’s next generation of graphics cards are on the horizon too.
Huang’s comments weren’t entirely clear, and he didn’t explicitly mention anything about DGX. Instead, during a monologue about the power of modern GPUs and AI processors, he said “one of our computers” and “the next one, soon coming, is liquid cooled.” Since Nvidia’s sole lineup of AI-focused computers is the DGX, it’s probably safe to say Huang is referring to DGX, or at least that’s the take of some IT industry analysts.
Servers, even those in large and cutting-edge data centers, still rely primarily on air cooling for CPUs and GPUs, including the current generation DGX. Even Nvidia’s high-end H100 and H200 graphics cards work well enough under air cooling, so the impetus to switch to liquid hasn’t been that great. However, as Nvidia’s upcoming Blackwell GPUs are said by Dell to consume up to 1,000 watts, liquid cooling may be required. Claims of Blackwell drawing 1,400 watts have also been thrown around.
2,000 watt processors might even be within the realm of possibility, as Intel researchers are attempting to design a liquid cooler that can handle that much heat. For years, processors have been getting hotter and more power-hungry, and to keep that trend going the industry will need to invest in cutting-edge cooling solutions.
All of this is in the interest of boosting performance, but not everyone is impressed by TDPs four digits long. Industry analyst Patrick Moorhead was particularly scathing about the idea of a liquid-cooled DGX, saying “So we’ve pulled nearly every lever so far to optimize performance at a kinda reasonable degree of heat/power… What’s next, liquid nitrogen?” Moorhead suggested hitting “the reset button” and implied integrating this technology just wouldn’t be sustainable.
Realistically, there probably has to be a point where power can no longer be increased to boost performance, but power consumption has steadily increased from generation to generation. A decade ago, 300 watts was considered pretty excessive, and now the highest-end CPUs and GPUs easily surpass that figure. Further increases to power don’t seem to be going away any time soon, as long as there’s no better way to boost performance reliably.