1.5 Million Threads: AMD-Powered Archer 2 Supercomputer With 12,000 EPYC Rome CPUs

(Image credit: EPCC)

What would you do with 1.5 million threads of computational horsepower? The UK Research and Innovation (UKRI) center has an idea.

The Archer supercomputer system based in Edinburgh was first introduced in 2013 based on the Cray XC30 design and has since been in use by researchers in need of serious compute power. It was once among the top 20 supercomputers in the world, but that’s no longer the case and the time has come to replace it.

As such, the center announced that it has contracted with Cray to build the Archer 2 supercomputer, which will be based on AMD EPYC Rome processors.

The center will build the system out of 5,848 Shasta compute nodes (deep dive here). Each node will contain two second-gen AMD EPYC Rome processors with 64 cores apiece, with each clocked at 2.2GHz (likely the base frequency).

This adds up to 11,969 CPUs, and a grand total of 748,544 CPU cores and about 1.5 million threads (we rounded that number up simply because we’ve always wanted to).

(Image credit: Future)

Depending on how it is measured, averaged out over a handful of benchmarks, the new supercomputer will perform an average of 11 times faster than its predecessor.

Full technical specifications as posted on HPC Wire below:

  • Peak performance estimated at ~ 28 PFLOP/s
  • System Design:
  • 5,848 compute nodes, each with dual AMD Rome 64 core CPUs at 2.2GHz, for 748,544 cores in total and 1.57 PBytes of total system memory
  • 23x Shasta Mountain direct liquid cooled cabinets
  • 14.5 PBytes of Lustre work storage in 4 file systems
  • 1.1 PByte all-flash Lustre BurstBuffer file system
  • 1+1 PByte home file system in Disaster Recovery configuration using NetApp FAS8200
  • Cray next-generation Slingshot 100Gbps network in a diameter-three dragonfly topology, consisting of 46 compute groups, 1 I/O group and 1 Service group
  • Shasta River racks for management and post processing
  • Test and Development System (TDS) platform, to be installed in advance
  • Collaboration platform with 4 x compute nodes attached to 16 x Next Generation AMD GPUs
  • Software stack:
  • Cray Programming Environment including optimizing compilers and libraries for the AMD Rome CPU
  • Cray Linux Environment optimized for the AMD CPU blade based on SLES 15
  • Shasta Software Stack
  • SLURM work load manager
  • CrayPat as profiler
  • GDB4HPC as debugger

The center will install the Archer 2 supercomputer in the same room as the supercomputer it is replacing, meaning there will be a period of downtime. Archer will be shut down on February 18, 2020, and Archer 2 will be booted up 78 days later on May 6, if all goes according to schedule.

However, after bootup, the system will first go through a one-month stress testing period before the service will be open to researchers. Some researchers might be allowed to access the system during this testing phase, but the UKRI did say that access would be ‘limited.’

The new win builds on an ever-expanding list of EPYC Rome supercomputer wins, including the recent announcement of the exascale-class Frontier supercomputer.

Niels Broekhuijsen

Niels Broekhuijsen is a Contributing Writer for Tom's Hardware US. He reviews cases, water cooling and pc builds.

  • CLevere
    This story is partially made up as the links provided do not mention any of the specs listed. It is also a story from way back in March 2019 as that is when the Archer 2 was announced...
  • cryoburner
    CLevere said:
    This story is partially made up as the links provided do not mention any of the specs listed. It is also a story from way back in March 2019 as that is when the Archer 2 was announced...
    I'm not sure whether or not the article might have been edited, but there's a link just before the specs pointing to where they came from, which was an article posted just yesterday. And doing a quick search of the announcement in March, it looks like that was just a basic announcement that there would be a new supercomputer and what it was expected to cost, whereas this contains new details about its planned hardware specifications.
  • CLevere
    Hardware specifications that aren't listed in the article referenced...
  • CLevere
    OK, just checked and the link has been updated from when I read the article.
  • svan71
    Put me down for 2, my current desktop takes to long to open gmail.
  • hannibal
    Now we know where all zen chips fo :)
  • d3bug
    Yes yes, but can it run Crysis?
  • hannibal
    Yes it can... They did run crysis with treathripper without gpu ;)