*After posting this news, we have recieved additional information from Cray that states that the top plate is not being used for CPUs, but for the Slingshot interconnect. As a result, the system supports only 8 CPUs in 1U. The text below has been changed to reflect this new information.

The Supercomputing show was a hive of activity, with lots of whispers surrounding the next generation of x86 CPUs, such as AMD’s Rome platform and Intel’s Cascade Lake platform. One of the big announcements at the start of the show is that Cray has developed an AMD Rome based version of its Shasta supercomputer deployments.

Cray had a sizeable presence at the show, given its position as one of the most prominent supercomputer designers in the industry. At its booth, we saw one of the compute blades for the Shasta design:

It’s a bit difficult to see what is going on here, but there are what looks like eight processors in a 1U system. The processors are not facing in the traditional way for a server, in line with airflow, because the system is not air cooled.

The cooling plate and pipes connect to separate reservoirs that also supply additional cooling to the memory. Of course in this situation we’re not in a dense memory configuration, given the cooling required.

This display was being shown off in the CoolIT booth. CoolIT is a well-known specialist of liquid cooling for both the consumer and enterprise worlds, and they had a few interesting points on their implementation for Shasta.

Firstly, the eight CPU design with split memory cooling was obvious, but CoolIT stated that these processor cooling plates were designed to cool a CPU below and the custom Cray Slingshot interconnect above. We were told that with this design, the full cooling apparatus could go up to eight CPUs, several interconnect nodes, and up to 4kW.

The cooling loops are split between the front and the rear, recombining again near the front. This means that four cooling plates and memory will be pumping through in each loop. There is a small question about how the memory all fits in, physically, but this is certainly impressive.

CoolIT is claiming that its custom solution for Cray’s Shasta platform helps double the compute density over an aircooled system, offers 20% better rack-level efficiency, and up to 30% better facility-wide energy efficiency.

Shasta is set to be deployed at NERSC as part of its upcoming ‘Perlmutter’ supercomputer, slated for delivery in late 2020.

*This article was updated on Nov-20 with new information.

Related Reading

Comments Locked


View All Comments

  • rahvin - Tuesday, November 20, 2018 - link

    Water cooling loops have to be oversized for heat dissipation or you'll have a chance that you'll get a hot spot in the loop, it's just a safety factor like any other engineering.

    For example the configuration Cray has chosen here is going to supply the last CPU's in the loop with less water because the pipe are connected in a serial configuration and the water will choose the shortest path (unless they run this at extreme pressures). The extra cooling could be to cover that scenario. They could run this at lower wattage cooling but they'd need a manifold at the inlet to each server and individual lines to each cooling loop. The expense of that configuration, not to mention the lack of room for 8 lines of half inch tubing, is likely prohibitive when considered against just having excess cooling capacity to cover it.
  • iwod - Wednesday, November 21, 2018 - link

    Thx for the explanation!. Now just waiting for EPYC 2 to arrive.
  • SetiroN - Tuesday, November 20, 2018 - link

    Not to sound like an ass but... are those cellphone pictures? They're pretty bad.
  • PaoDeTech - Tuesday, November 20, 2018 - link

    How many blades do you need to run Crysis @60 fps?
  • Godaniabilo - Thursday, November 22, 2018 - link

    Hi, my name is Isabel. I like roleplay and storyplay. write me there http://badoos.ml
  • Mensentona - Friday, November 23, 2018 - link

    Hello everyone) I am a girl who is tired without attention and looking for a young man. Write me there http://badoos.ml

Log in

Don't have an account? Sign up now