AI chips are getting hotter – A microfluidics breakthrough goes straight to the silicon to cool up to three times better.

AI chips are getting hotter – A microfluidics breakthrough goes straight to the silicon to cool up to three times better.

AI is hot – literally.

The chips that datacenters use to run the latest AI breakthroughs generate much more heat than previous generations of silicon. Anybody whose phone or laptop has overheated knows that electronics don’t like to get hot. In the face of rising demand for AI and newer chip designs, the current cooling technology will put a ceiling on progress in just a few years.

To help address this problem, Microsoft has successfully tested a new cooling system that removed heat up to three times better than cold plates, an advanced cooling technology commonly used today. It uses microfluidics, an approach that brings liquid coolant directly inside the silicon – where the heat is. Tiny channels are etched directly on the back of the silicon chip, creating grooves that allow cooling liquid to flow directly onto the chip and more efficiently remove heat. The team also used AI to identify the unique heat signatures on a chip and direct the coolant with more precision. 

Researchers say microfluidics could boost efficiency and improve sustainability for next-generation AI chips. Most GPUs operating in today’s datacenters are currently cooled with cold plates, which are separated from the heat source by several layers that limit the amount of heat they can remove.

As each new generation of AI chips becomes more powerful, they generate more heat.

Sashi Majety, senior technical program manager for Cloud Operations and Innovation at Microsoft, said:

In as soon as five years, if you’re still relying heavily on traditional cold plate technology, you’re stuck,

Today, Microsoft announced that it has successfully developed an in-chip microfluidic cooling system that can effectively cool a server running core services for a simulated Teams meeting.

Judy Priest, corporate vice president and chief technical officer of Cloud Operations and Innovation at Microsoft, said:

Microfluidics would allow for more power-dense designs that will enable more features that customers care about and give better performance in a smaller amount of space,

Priest, said:

But we needed to prove the technology and the design worked, and then the very next thing I wanted to do was test reliability,

The company’s lab-scale tests showed microfluidics performed up to three times better than cold plates at removing heat, depending on workloads and configurations involved. Microfluidics also reduced the maximum temperature rise of the silicon inside a GPU by 65 percent, though this will vary by the type of chip. The team expects the advanced cooling technology would also improve power usage effectiveness, a key metric for measuring how energy efficient a datacenter is, and reduce operational costs.

Using AI to mimic nature

Husam Alissa, director of systems technology in Cloud Operations and Innovation at Microsoft, said:

Microfluidics is not a new concept, but getting it to work has been a challenge across the industry. Systems thinking is crucial when developing a technology like microfluidics.

You need to understand systems interactions across silicon, coolant, server and the datacenter to make the most of it,

Just getting the grooves right is hard. The microchannel dimensions are similar in size to human hair, meaning there’s no margin for error. As part of the prototyping effort, Microsoft collaborated with Swiss startup Corintis to use AI to help optimize a bio-inspired design to cool chips’ hot spots more efficiently than straight up-and-down channels, which they also tested. The bio-design resembles the veins in a leaf or a butterfly wing – nature has proven adept at finding the most efficient routes to distribute what’s needed. 

Microfluidics requires more than innovative channel design. It is a complex engineering challenge.

It required ensuring that the channels are deep enough to circulate adequate cooling liquid without clogging while not being so deep as to weaken the silicon such that it risks breaking. The team produced four design iterations in the past year alone.

READ the latest news shaping the AI Chips market at AI Chips News

AI chips are getting hotter – A microfluidics breakthrough goes straight to the silicon to cool up to three times better., source