Amazon Web Services has introduced its 3rd Generation Graviton processors for its AWS cloud datacenters. The new CPU promises up to twice the higher floating-point performance as well as up to three times higher machine learning performance compared to the 2nd Generation Graviton system-on-chip (SoC). The chip will be exclusively used in Amazon Web Services Datacenters.
AWS yet has to disclose all the details about its Graviton3 datacenter processor, but based on what we know today, the new SoC provides up to 25% higher compute performance (which probably means integer performance), up to double floating-point and superior cryptographic performance. The SoC also provides up to 50% greater memory bandwidth due to DDR5-4800 (first time for a cloud compute platform!) support compared to the Graviton2. In addition, the new Graviton3 also supports bfloat16 and offers up to 200% better performance in machine learning applications than its predecessor.
Amazon yet has to announce whether the new Graviton3 processor relies on Arm's off-the-shelf high-performance Neoverse cores (perhaps with a few modifications) or uses custom cores designed in house. One thing that Amazon does disclose is that the new cores support dedicated caches for every vCPU, and support for a new pointer authentication to improve security. In addition, the new SoCs feature always-on memory encryption.
In addition to being massively faster than its predecessor, the new Graviton3 processors also consumes up to 60% less power than the Graviton2, though Amazon does not disclose in which workloads. Obviously, some special-purpose accelerators or instructions can decrease power consumption and increase performance quite tangibly, but only for select workloads.
From what has been disclosed so far, Amazon clearly positions its Graviton3 primarily for performance-hungry workloads, including HPC, batch processing, electronic design automation (EDA), media encoding, scientific modelling, and CPU-based machine learning inferencing. AWS will preview the new SoCs with its new EC2 C7g instances.
One thing that is particularly noteworthy about Amazon's introduction of its Graviton3 processor is timing. The announcement of the Graviton3 SoC is made almost exactly two years after the formal launch of the Graviton2 CPU in early December 2019 and three years after the original Graviton in later November 2018. Such a precise two-year cadence signals how serious AWS is about its in-house CPUs and how it wants to keep closing the gap with x86-based processors from AMD and Intel.
