'GPUs still rule' asserts graphics guru Raja Koduri in response to custom AI silicon advocate

Raja Koduri, a GPU veteran who has designed graphics processors for AMD, Apple, ATI, S3 Graphics, and Intel, believes that GPUs will not be replaced by custom-built silicon for artificial intelligence (AI) and high-performance computing (HPC) any time soon. However, new architectures can still be designed on the heels of GPUs to better address these workloads, he believes. In fact, the custom silicon endeavors of AWS, Graphcore, Google, Microsoft, and Tesla probably prove his point about architectures.

"We have heard this statement since 2016, but GPUs still rule," Raja Koduri in response to an X post about the future of AI compute. "Why? I am still learning, but my observations so far: the 'purpose' of purpose-built silicon is not stable. AI is not as static as some people imagined and trivialize [like] 'it is just a bunch of matrix multiplies'."

Originally designed to process highly parallel graphics workloads (and power the best graphic cards), graphics processing units (GPUs) have evolved to accelerate artificial AI and HPC workloads. As a result, AI and HPC software workloads are optimized for GPUs to a large degree, because of Nvidia CUDA dominance. Nvidia's GPUs have become so versatile in adding support for new data formats that it has become inherently harder - even for custom silicon - to compete against them. AI and HPC GPUs are yet to take off for AMD and Intel.

Many purpose-built silicon solutions are lacking in important architectural support areas, often shifting the burden to software developers. This shift is somewhat problematic due to the scarcity of new system software talent and the overreliance on a small, aging pool of existing experts, opined Koduri.

"The system architecture — things like page tables, memory management, interrupt handling, debugging — of GPUs evolved over two decades and is a necessary evil to support production software stacks," Koduri wrote. "Many of the purpose-built silicon is deficient here and throw the burden onto 'software' people. There is not much new young system software talent coming into the workforce these days, so everyone competes for the same small pool of aging talent."

While Raja's comments make a lot of sense, he is a board-of-directors member of Tenstorrent, a producer of custom-built AI accelerators and HPC CPUs based on the RISC-V instruction set architecture, which makes his statements somewhat controversial. Yet, he expressed optimism for the development of new architectures that will address AI and HPC workloads. These future architectures would ideally emerge from the lessons learned so far, potentially offering a new purpose and overcoming the current limitations of both GPUs and purpose-built silicon. 

"But I am still optimistic that a new architecture with new purpose will evolve from the lessons learned so far," the renowned GPU specialist teased.