Nvidia deep dives Vera CPU for AI data centers — SPEC CPU 2026 benchmarks revealed, Olympus architecture specifics, and more

Throwing some shade at AMD. I see "Chiplet tax" on one of the slides.

But their comparisons (obviously) are to Turin rather than newly announced Venice.

The article said:

Vera is Nvidia’s first CPU with a core design created in-house, which is the Olympus core.

Only if you're talking about server CPUs. Before that, they made at least two generations of cores for tablet/embedded SoCs (i.e. Denver and Carmel). See: https://en.wikipedia.org/wiki/Project_Denver

The article said:

It uses a SOCAMM2 LPDDR5X memory system with capacity of up to 1.5 TB and speeds up to 9600 MT/s

Ah, but you missed an interesting detail. The way you reach 1.2 TB/s with 9600 MT/s memory is by scaling the memory datapath up to 1024 bits, which is the equivalent of 16x DDR5 DIMMs! Because SOCAMM2 each have a 128-bit data width, you only need 8 of them. I was staring at a photo of the Vera board and wondering how they possibly reached 1.2 TB/s, and that's when it clicked that they actually packed a 128-bit datapath in those little memory modules!

Looking at the block diagram of the core, I'm struck by a few things.
The L1i cache is the same size as Intel's Lion Cove, but surprisingly only 4-way. Intel used 8-way associativity for their I-cache for almost the past 20 years!
At 64 entries, its iTLB is also much smaller than Golden Cove (the last Intel core I could find data for), which has 256 entries.
However, it regains some ground by being fully-associative, whereas the iTLBs of Intel CPUs usually have only 8-way associativity.
The dTLB is more comparable to the size recent Intel CPUs use (112 vs. 128 entries in Lion Cove), but also fully-associative whereas Intel typically uses only 4-way associativity.
At 96k, the L1d is running 2x of Intel's. As with the L1i, its associativity is less (6-way instead of 12-way for Intel).
Its L2 cache is comparable to Raptor Cove's in size, but it's only 8-way associative whereas Raptor had 16-way associativity.
Its STLB is similar in size to the latest Intel core I have data on (3k vs. Golden Cove's 2k).

The article said:

Research on neural branch prediction dates back to the late 90s

AMD said it used a neural network-based branch predictor in the original Zen cores. Jim Keller briefly discuss it, in an interview. They apparently had an open contest for people to submit branch predictor algorithms and the neural one simply beat all of the others. If you search the patent database, you might be able to find some on it. That's probably the best chance of finding out how it actually works.

The article said:

At the last stage of the front end is that 10-wide decode, feeding more instructions into the execution engine per cycle than the 8-wide decode in AMD’s Zen 5 microarchitecture.

The decoder in Zen 5 is hard-partitioned between threads, limiting each to only 4 instructions per cycle. Even if there's only one thread running on the core (or if you disable SMT in BIOS), it still uses only a single 4-wide decoder.

But. We should consider that x86 instructions can encompass more functionality than an ARM64 instruction. So, maybe a typical ARM instruction stream needs like 5 instructions to do the same work as 4 x86 instructions. Just spitballing, here. It's not a huge difference, since most instructions are fairly simple and operating register-to-register, whereas x86 instructions get a lot of their density advantage by combining memory loads/stores with some arithmetic or logical operation, and supporting more sophisticated address arithmetic.

Looking only at decode-width is a little misleading, because CPUs like Zen 5 have much wider dispatch (8-way) from their micro-op cache. Since these are now micro-ops, they're more equivalent to ARM64 instructions, in terms of how much work each represents.

BTW, there are two other obvious points of comparison:
The Neoverse V2 cores, used in Nvidia's prior Grace CPUs, have a 6-wide decoder* (source: https://chipsandcheese.com/p/hot-chips-2023-arms-neoverse-v2 )
The Cortex-X925 cores, used in Nvidia's RTX Spark, have a 10-wide decoder (source: https://chipsandcheese.com/p/arms-cortex-x925-reaching-desktop )
* Note that Neoverse V2 still has a mOP cache with 8-wide dispatch. So, as with x86 P-cores, the decoder width is a little bit deceptive.

The article said:

Past the front end, the mid-core rename / allocation engine is built to keep instructions moving while waiting on dependencies.

You skipped a pretty big detail: no mOP cache!
Modern x86 P-cores and some 64-bit ARM cores had micro-op caches to avoid having to re-decode the same instructions. Once ARM dropped 32-bit compatibility, they started getting rid of those. Also, Intel either doesn't have them in their E-cores, or perhaps what they did was to put some of that into the I-cache.

So, it's interesting (but not all that surprising) that Nvidia followed ARM's approach of just skipping the mOP cache, entirely.

The article said:

For SIMD instructions, the execution engine includes a vector cluster for Arm’s Scalable Vector Extension (SVE), including six vector units that support 128-bit SVE instructions

It's darkly ironic that SVE's main selling point was to allow CPUs to scale all the way up to 2048 bits per vector, but all mainstream implementations (except for AWS Graviton 3) are just 128-bit.

The article said:

Traditional SMT time-slices execution, giving both threads access to all of the core resources and sharing them as instructions execute in parallel.

Not really. The way AMD describes SMT in Zen 5 is that certain competitively-shared resources have watermarks that limit how much a single thread is able to use, so that it doesn't starve out the other thread. However, when there's only one thread running on a core, more of those constraints go away and only the statically-partitioned resources (e.g. the decoder) remain exclusive.

My read on "Spatial Multithreading" is that it's Nvidia's marking machine trying to spin a weakness to make it sound more like a strength. This being Nvidia's first SMT implementation (AFAIK), it won't have the same sophistication as where Intel and AMD have gotten, over a couple decades of experience implementing and refining theirs.

The article said:

Nvidia’s second-generation Scalable Coherency Fabric (SCF). It underpins Nvidia’s approach of using a monolithic die as opposed to a chiplet-based design, distributing last level cache in a mesh across the die and avoiding the cross-CCD latency penalty with localized L3.

First, Intel has traditionally done the same thing. However, I need to catch up on what they've said about Sierra Forest.

Second, people tend to overestimate the impact of the cross-CCD thing. Unlike Intel CPUs, one AMD CCD won't write data to another's L3 slice. Each is private to that CCD, except for coherency. That said, when you've got multiple threads that are either exchanging or both modifying the same data, and they happen to be scheduled on different CCDs, that is when you feel the impact of the die-to-die communication.

Anyway, AMD's approach has clearly scaled better. I think that's why Sierra Forest looks like it moved in the direction of doing the same thing with segmenting its L3 cache.

The article said:

For I/O, Vera supports PCIe 6.4 with 88 lanes per CPU and bifurcation support down to x2.

I don't know what happened here, but the slide with the annotated die photo shows it supporting PCIe 6 with only 16 lanes.

P.S. As for the benchmarks, the initial Phoronix review was interesting, but now I'm just waiting for some independent analysis. From a CPU microarchitecture standpoint, what I really want to know is the single-threaded performance across the whole SPEC suite, and how the sub-scores compare with other CPUs. MT scaling is mildly interesting, but it really stacked that deck in its favor by having relatively few cores and huge amounts of memory bandwidth. So, I expect it scales well, but not well enough to outright beat Intel or AMD on anything that's not fundamentally bottlenecked by memory bandwidth.

usertests said:

But their comparisons (obviously) are to Turin rather than newly announced Venice.

Announced? Venice is launching in about a week! That's probably why Nvidia is making a bunch of noise about this now!

I like NVIDIAs marketing department scaling results per core. This is the height of aerobatics. Interesting how all would compare per core with the AMD 64core Turin 9575F, their king of single core benchmarks ? And how per core comparison will look for just the 64 core Venice? Huang will leave full of tears on leather jacket