If the speed of evolution was not enough, new extreme density levels present serious challenges with cooling and also power. The higher the density, the more challenging it is to design a hybrid liquid- and air-cooled data center (both are needed) at 132 kW/rack versus 10 kW/rack due to physical space, potential for overheating and maintaining resiliency and efficiency.

Remember, slow migration up to 10 kW/rack happened over decades and the designs were deployed, tested and optimized. We don’t have this luxury now. We also won’t have multiple IT refresh cycles as the power and cooling needed for the next generation of GPUs will be much higher and without significant upgrades to the power and power distribution, as well as no additional liquid cooling with special racks, manifolds, cooling distribution units and chillers.

Data centers that are optimized to use the latest and greatest GPUs must be designed in anticipation of the power density needed—a year or two in advance. Again, these will be fresh designs without the opportunity to test and optimize and most data center operators don’t have large staffs of designers to create new designs for every new generation of Nvidia.

GPUs, as used by data center designers, were not challenged in a significant way over the years, and expertise wasn’t needed or developed. For organizations approaching this fast-changing landscape, I suggest the following: