The rumored new lineup of artificial intelligence (AI) and high-performance computing (HPC) GPUs from Nvidia is perfectly aligned with the newest expanded export rules published by the U.S. Department of Commerce in mid-October, believes Patrick Moorhead, the head of Moor Insights and Strategy. He points out that, unlike some reports in the press, the company is not trying to evade the expanded U.S. sanctions on AI processors with its new data center GPUs. Meanwhile, the DoC recently explained which products cannot be shipped to China without a license, even if they are not designed for data centers, and the GeForce RTX 4090 is seemingly one of them.
"Yesterday, there were a flurry of articles written I thought suggested or were interpreted that Nvidia was trying to 'skirt' or 'pull a fast one' on the U.S. Government Export Control laws with a rumored line of new datacenter accelerator cards for China export," Moor wrote in a blog post. "I find this laughable. The downside for Nvidia would be immense. The company may be a fierce innovator and competitor, but they are not dumb."
The latest U.S. DoC export rules for data center AI and HPC processors cover GPUs and other AI accelerators shipped to China, Macau, Saudi Arabia, the United Arab Emirates, and Vietnam; they require vendors to apply for an export license if their products exceed specific performance and/or performance density levels. To make it easier for companies, the U.S. DoC recently held a public briefing, presenting a relatively simple chart that lets it quickly determine whether a processor can be shipped to China and other restricted countries.
The new rules can be somewhat convoluted: Here's a detailed look at what they allow and what they proscribe, and what it means for you.
Total Processing Performance
By performance, the new rules define the Total Processing Performance (TPP) score, essentially listed processing power multiplied by the length of operation (e.g., FLOPS or TOPS of 8/16/32/64 bits) without sparsity. The U.S. government does not want China to obtain processors — whether intended for data centers or client PCs — with a TPP score of 4800 without sparsity (in the case of matrix multiplication).
For example, Nvidia’s H100 has a listed FP16/BF16 performance of 989 TFLOPS with sparsity. Divide by two and multiply by 16 and you get its TPP score of 7,912, making it far too powerful for export to China.
This is why Nvidia’s GeForce RTX 4090/AD102 — one of the best graphics cards around — also falls into the category of export-licensable items. Its FP8 Tensor FLOPS performance (660 TFLOPS) hits a TPP score of 5,280. So, no, Nvidia and its partners cannot ship the GeForce RTX 4090 to China, effective November 16.
Performance Density
Another parameter the latest rules introduce is a Performance Density (PD) metric. This parameter is designed to avoid the loophole of acquiring numerous smaller data center AI chips, which, if combined, would be as powerful as restricted chips. PD is counted by dividing TPP by the die area measured in square millimeters. The die area includes built-in caches but excludes external memory devices like HBMs. This one is designed for minor high-density chips with a TPP score between 1600 and 4800.
For example, Nvidia’s L4/AD104 datacenter GPU has a TPP score of 1936 (242 FP8 TFLOPS * 8 = 1,936). Yet, its die size is 294 mm^2. Therefore, its performance density is 6.5, so the L4 cannot be shipped to China. Meanwhile, Nvidia’s GeForce RTX 4070 Ti — a non-datacenter product with a TPP score of 1936 — can be sent to China without restrictions.
The Interpretation
The exciting part here is the government's interpretation of whether a product is designed for data center use or not. In this case, the U.S. DoC plans to assess the destination of the particular product based on its characteristics instead of its branding. For example, a dual-slot GeForce RTX 4070 Ti with a blower or passive heatsink would be considered a data center board, no matter what it is formally called.
"Even if the manufacturer is not marketing the item for data center use, the item may still be designed for data center use based on the technical characteristics of the item," said Thea D. Rozman Kendler, assistant secretary of the U.S. Department of Commerce Bureau of Industry and Security.
Nvidia's (Alleged) China Data Center GPU Lineup
After the U.S. Department of Commerce published its new export rules for data center processors used for AI and HPC workloads in mid-October, they appeared so severe that almost no high-performance hardware could be sent to China and other countries. Nvidia, Intel, and AMD ship tons of AI and HPC hardware to Chinese customers, and losing those sales will cost them billions in revenue. This is why rumors started to spread that Nvidia was tricking the U.S. govt with its rumored lineup of data center products tailored specifically for the Chinese market.
| GPU | HGX H20 | L20 PCle | L2 PCle |
|---|---|---|---|
| Architecture | GPU | Hopper | GH100 | Ada Lovelace | AD102 | Ada Lovelace | AD104 |
| Memory | 96 GB HBM3 | 48 GB GDDR6 w/ ECC | 24 GB GDDR6 w/ ECC |
| Total Processing Power (FP16/BF16) | 2,368 | 1,912 | 1,544 |
| Performance Density | 2.9 | 3.13 | 5.2 |
| Memory Bandwidth | 4.0 TB/s | 864 GB/s | 300 GB/s |
| INT8 I FP8 Tensor | 296 I 296 TFLOPS | 239 I 239 TFLOPS | 193 I 193 TFLOPS |
| BF16 I FP16 Tensor | 148 I 148 TFLOPS | 119.5 I 119.5 TFLOPS | 96.5 I 96.5 TFLOPS |
| TF32 Tensor | 74 TFLOPS | 59.8 TFLOPS | 48.3 TFLOPS |
| FP32 | 44 TFLOPS | 59.8 TFLOPS | 24.1 TFLOPS |
| FP64 | 1 TFLOPS | N/A | N/A |
| RT Core | N/A | Yes | Yes |
| MIG | Up to 7 MIG | N/A | N/A |
| L2 Cache | 60 MB | 96 MB | 36 MB |
| Media Engine | 7 NVDEC, 7 NVJPEG | 3 NVENC (+AV1), 3 NVDEC, 4 NVJPEG | 2 NVENC (AVI), 4 NVDEC, 4 NVJPEG |
| Power | 400 W | 275W | TBD |
| Form Factor | 8-way HGX | 2-slot FHFL | 1-slot LP |
| Interface | PCIe Gen5 x16: 128 GB/s | PCle Gen4 x16: 64 GB/s | PCle Gen4 x16: 64 GB/s |
| NVLink | 900 GB/s | - | - |
| Samples | November 2023 | November 2023 | November 2023 |
| Production | December 2023 | December 2023 | December 2023 |
A close look at Nvidia's alleged data center product lineup for China reveals that the family is meticulously designed to avoid any possible violations of the latest U.S. export rules concerning AI and HPC GPUs. The new offerings are designed to fit into the green zone in the chart, thus complying with US sanctions against China while allowing Nvidia to recoup some of its lost $5 billion in sales in the increasingly restricted Chinese market.
