Chinese scientists have built what they call 'a brain-scale artificial intelligence model' based on one of its latest supercomputers featuring Sunway processors, reports SCMP. The scholars claim that the AI model with 174 trillion parameters could be used for a variety of applications starting from autonomous vehicles to scientific research.
The system used is the Sunway OceanLight (sometimes called OceanLite) is based on 96,000 nodes powered by Sunway's hybrid SW39010 390-core processors featuring almost 40 million cores in total (with CPUs evolved from the SW26010 processors). This machine was touted as one of the world's first exascale supercomputers, but since then the claim about its FP64 performance has been largely debunked when it turned out that the scientists lowered precision of calculations the machine performed while doing the work for the Gordon Bell trophy (which is sometimes called a 'Nobel Prize for supercomputing').
The Chinese team of researchers from the National Research Center of Parallel Computer Engineering and Technology (NRCPC) call their AI model bagualu — alchemist's pot — since it includes about 174 trillion parameters, which rivals the number of synapses in the human brain. Actual performance of OceanLight is now claimed to be 5.3 'AI' ExaFLOPS, even though earlier it was said to be 4.4 'mixed precision' ExaFLOPS, which once again shows that performance numbers of this systems disclosed by the NRCPC are not really accurate.
But even if OceanLight is slower than it is said to be, the work to create an AI training model with 174 trillion parameters is gargantuan. Scientists from the NRCPC told the South China Morning Post that to get a decent performance with its brain-scale AI model they had to introduce 'hardware-specific intra-node optimisations' as well as implement 'hybrid parallel strategies' at an unprecedented scale. Considering the fact that OceanLight features 96,000 nodes and nearly 40 million processing engines, optimizing hardware and software for this system is a big deal.
The announcement about the unprecedented AI model was made weeks after Oak Ridge National Laboratory officially unveiled its Frontier — the world's first supercomputer with a performance of 1.102 FP64 ExaFLOPS in the Linpack benchmark. It is noteworthy that NRCPC has not officially published performance results of its OceanLight in the Top 500 list of supercomputers.
