Nvidia and Cineca today announced plans for the Leonardo supercomputer, which will be the "world's fastest AI supercomputer" when to goes online in 2021. Of course, things may change between now and the full deployment, but it's the fastest announced AI-focused supercomputer and the specs should put it near the top of any charts, regardless.
At the heart of Leonardo are "nearly 14,000" Nvidia A100 GPUs, with four GPUs per node. Each node will consist of a single Intel Sapphire Rapids CPU, which isn't slated to start shipping until next year. We do know Sapphire Rapids will launch in 2021, with DDR5 and PCIe 5 support, but we don't know core counts or clocks yet.
The key performance metric for the AI supercomputer will be FP16 training speed, and thanks to the 3rd generation Tensor cores in the A100, Leonardo has a peak theoretical performance over over 10 Exaflops. Of course, it's not just about Tensor cores, as 14,000 A100 GPUs should also prove potent in FP64 calculations.
Back of the napkin math suggests Leonardo could rank in the top three of the current Top500, since it should be over seven times as fast as Nvidia's Selene supercomputer. Still, network connectivity and other factors play a role.
Besides Leonardo, Cineca announced plans for three additional supercomputers, all using Nvidia A100 GPUs. MeluXina in Luxemborg will have 800 A100 GPUs, the IT4Innovations National Supercomputing Center in the Czech Republic will have a 560 A100 GPU based server, and in a bit of an ironic twist, the Vega supercomputer in Slovenia will have 240 A100 GPUs. (It's named after Slovenian mathematician Jurij Vega, not the AMD RX Vega or the Vega star system.)
French company Atos will be building three of the supercomputers, including Leonardo. HP Enterprise will assemble the fourth supercomputer.
