Chinese think tank advises data centers to stick with Nvidia AI GPUs — homegrown Chinese GPUs suffer from 'high costs' and 'complex engineering'

On Sunday, the Chinese-government-backed research institute and think tank CAICT issued a detailed report regarding China’s computational power. The report (via SCMP) warned data centers to keep on using Nvidia-based solutions. The institute cited 'high costs' and 'complex engineering’ as one of the few hurdles that data centers would face if they were to transfer their models to Chinese chips.

It should not be surprising that the US government has barred Nvidia from exporting its high-performance A100 and H100 GPUs to China. Another round of GPU bans came last year when the Biden Administration restricted exports of Nvidia’s A800 and H800 alternatives to China for obvious reasons. Furthermore, it appears that Nvidia has also stopped taking orders for its currently permitted HGX H20 GPU.

As the saying goes, "Necessity is the mother of invention,” and likewise, many domestic Chinese chip manufacturers have taken this issue into their own hands. However, despite the clear incentive any Chinese data center would have to opt for a local alternative, CAICT advises continuing with Nvidia.

The CAICT stated that "it involves complex engineering to transfer models trained on Nvidia GPUs to domestic solutions due to differences in hardware and software.”

Adding to the mix, new data centers in China face numerous problems, including the lack of GPU solutions, supply mismanagement, and rising computing power fragmentation, which essentially refers to the inefficient use of computer resources like a GPU. Data centers must merge solutions from different vendors and play a "mix-and-match” game, hoping to get their systems up and running. A large portion of this inefficiency is caused by incompatible software and varying hardware.

CAICT warned that "there are big discrepancies on hardware in IDCs, such as in GPUs, AI accelerators and network structures, which made it harder to manage and dispatch hardware resources to accommodate for differential computing needs of AI tasks, further impeding the use rate.”

While CAICT has suggested Chinese data centers avoid trouble by sticking with Nvidia, the large gap between supply and demand remains unaddressed. China's goal of self-sufficiency in the semiconductor market is still a "work in progress" as the country, despite investments worth tens of Billions of dollars, still imported $350 Billion of chips in 2023. Players like Intel and AMD are struggling to keep pace with Nvidia due to Team Green’s strong integration between hardware and software in the AI market, so homegrown Chinese GPUs might, at best, achieve parity in a few years.