Nvidia's H20 data center GPU is designed to comply with the latest U.S. export regulations from this month, but it's looking like it might be delayed, per a report from Reuters. Largely intended for use in China, the delay of H20 could pose a problem for Nvidia, as its product stack of Chinese-specific GPUs is somewhat thin without H20, and there are other companies in China that could fill the void.
The H20 datacenter GPU is currently Nvidia's fastest graphics card that the company can legally sell in China. Alongside two other compliant GPUs, it was announced earlier this month almost as soon as the newest American export regulations came into force.
The chart above shows the current U.S. regulations and details what is and isn't allowed. While performance density is largely not limited, performance (defined as TPP or Total Processing Performance) is severely restricted. Even with the lowest performance density, no GPUs shipped to China can have a TPP that's roughly on par with the A100. Below you can see how Nvidia's GPUs land on the chart (expand the tweet to see the full image).
AI compute restriction map (UBS)NVDA pic.twitter.com/opcrCF25aXNovember 11, 2023
This second chart (click for more detail) illustrates what actual GPUs are and aren't allowed. The A100 and A800 are banned, as are lower-end chips like the A30 and A40. Even the RTX 4090 just barely didn't make the cut and is also banned now. H20 brushes up against the restricted zone of GPU specifications, hitting what might be the sweet spot. With a rating of roughly 2,500 TPP, it is Nvidia's fastest GPU that can legally be sold in China.
It was expected that H20 would launch late this year with L20 and L2, two other compliant GPUs. However, Reuters reports that Nvidia has delayed H20 as late as March. The apparent cause for the delay is issues that server manufacturers are seeing with H20, which isn't too surprising given the tight deadline for H20's launch.
| GPU | HGX H20 | L20 PCle | L2 PCle |
|---|---|---|---|
| Architecture | GPU | Hopper | GH100 | Ada Lovelace | AD102 | Ada Lovelace | AD104 |
| Memory | 96 GB HBM3 | 48 GB GDDR6 w/ ECC | 24 GB GDDR6 w/ ECC |
| Total Processing Power (FP16/BF16) | 2,368 | 1,912 | 1,544 |
| Performance Density | 2.9 | 3.13 | 5.2 |
| Memory Bandwidth | 4.0 TB/s | 864 GB/s | 300 GB/s |
| INT8 I FP8 Tensor | 296 I 296 TFLOPS | 239 I 239 TFLOPS | 193 I 193 TFLOPS |
| BF16 I FP16 Tensor | 148 I 148 TFLOPS | 119.5 I 119.5 TFLOPS | 96.5 I 96.5 TFLOPS |
| TF32 Tensor | 74 TFLOPS | 59.8 TFLOPS | 48.3 TFLOPS |
| FP32 | 44 TFLOPS | 59.8 TFLOPS | 24.1 TFLOPS |
| FP64 | 1 TFLOPS | N/A | N/A |
| RT Core | N/A | Yes | Yes |
| MIG | Up to 7 MIG | N/A | N/A |
| L2 Cache | 60 MB | 96 MB | 36 MB |
| Media Engine | 7 NVDEC, 7 NVJPEG | 3 NVENC (+AV1), 3 NVDEC, 4 NVJPEG | 2 NVENC (AVI), 4 NVDEC, 4 NVJPEG |
| Power | 400 W | 275W | TBD |
| Form Factor | 8-way HGX | 2-slot FHFL | 1-slot LP |
| Interface | PCIe Gen5 x16: 128 GB/s | PCle Gen4 x16: 64 GB/s | PCle Gen4 x16: 64 GB/s |
| NVLink | 900 GB/s | - | - |
| Samples | November 2023 | November 2023 | November 2023 |
| Production | December 2023 | December 2023 | December 2023 |
On the surface, being unable to launch a product for four months might not be a backbreaker. If Chinese data centers need Nvidia GPUs now, they can just buy the L2 or the L20, which Reuters says are not delayed. However, Nvidia might not have all that much time to launch and sell H20 in China.
Last year, Nvidia's A800 and H800 were launched to comply with the first round of U.S. export bans. Those GPUs were then banned by regulations imposed in October of this year, meaning they were on the market for 11 months at most. If yet more regulations are written into law, then the H20 might also have a short lifespan in China, and that makes a four-month delay much more troublesome to Nvidia than it normally would.
Anxieties over Nvidia's ability to provide GPUs in the current environment are perhaps already leading to consequences. Just before Nvidia's latest made-for-China GPUs were announced, Baidu purchased Huawei ASICs instead. Baidu had previously bought thousands of Ampere and Hopper GPUs, and this about-face indicates it wasn't willing to wait for Nvidia's upcoming graphics chips.
In today's landscape of ever-changing U.S. regulations, flexibility and speed are required for American companies to compete in China. However, processors aren't the kind of product that have a quick turnaround time. In the case of H20, it seems Nvidia's attempts to get it out the door as soon as possible didn't pan out, and it could cost the American tech company both its performance edge as well as the confidence of Chinese customers.
