Microsoft Announces Massive AI Supercomputer, Claims Top 5 Ranking

Microsoft today announced it’s built a supercomputer hosted in its Azure cloud services, which it claims is among the top five supercomputers in the world. The announcement came at its Build 2020 developer conference, being hosted online due to COVID-19.

The supercomputer was built for OpenAI, a San Francisco-based artificial intelligence lab, and should be able to train large-scale AI models and make the framework for training them available to more developers.

Microsoft’s computer is a system with “more than 285,000 CPU cores, 10,000 GPUs and 400 gigabits per second of network connectivity for each GPU server.” That’s enough, the company says, to rank among the top five in the TOP500 supercomputers list of the most powerful in the world. It has not, however, actually submitted its supercomputer to the site.

“This is about being able to do a hundred exciting things in natural language processing at once and a hundred exciting things in computer vision, and when you start to see combinations of these perceptual domains, you’re going to have new applications that are hard to even imagine right now,” Microsoft chief technical officer Kevin Scott said in a release.

Microsoft's hope, per the company, is that the supercomputer’s resources, as well as training tools and AI models will be available to developers, customers and scientists through GitHub and Azure AI.

“We are seeing that larger-scale systems are an important component in training more powerful models,” OpenAI CEO Sam Altman said in Microsoft’s announcement. 

Additionally at Build, Microsoft said it would soon open source its Turing models for AI and “recipes for training them in Azure Machine Learning.” The company also debuted the next version of the DeepSpeed open source deep learning library for Pytorch, which should be more efficient than the previous version and reduces computing power for training.