Vera is Nvidia’s first CPU with a core design created in-house, which is the Olympus core. It’s built on Arm v9.2-A, but the design was created by Nvidia, unlike Grace, which leveraged a stock Arm design. Each Vera CPU has 88 Olympus cores on a monolithic die, breaking from the chiplet-based designs available from Nvidia’s x86 competitors.
Nvidia says Vera comes with a 1.5x increase in instructions per cycle (IPC) throughput compared to Grace, and 50% higher performance compared to x86 competitors (it seems that number is per-thread performance with a fully-loaded socket). Nvidia has a single 88-core design with Vera that supports spatial multithreading for 176 threads.
Cores / Threads |
88 / 176 |
L2 cache |
2 MB per core |
L3 cache |
164 MB per CPU |
Memory |
Up to 1.5 TB SOCAMM2 LPPDDR5X |
Memory speed |
Up to 9600 MT/s |
Memory bandwidth |
Up to 1.2 TB/s (aggregate), 14 GB/s (per core) |
PCIe |
88 PCIe 6.4 lanes (CPU only), 96 PCIe 6.4 lanes (Vera Rubin), bifurcation down to x2, CXL 3.1 |
Configurable TDP |
250W - 450W |
The CPU has a configurable TDP range of 250W to 450W. It uses a SOCAMM2 LPDDR5X memory system with capacity of up to 1.5 TB and speeds up to 9600 MT/s, and comes with 164 MB of L3 cache and 2 MB of L2 per core. Vera includes significantly less L3 than Intel’s highest-specced Xeon 6 and AMD’s Zen 5 chips. It actually has more L2 than L3 overall. This, presumably, is due to Nvidia’s fabric, which distributes the L3 in a mesh across the monolithic die.
Below, you can see a layout of the Olympus microarchitecture. Nvidia has disclosed some of the highlights of the architecture previously, such as the 10-wide instruction decode and neural branch predictor, but we now have a full view of the architecture courtesy of Nvidia’s Vera white paper.
The front end starts with Nvidia’s neural branch predictor that can run two branches per cycle “with zero penalties,” according to Nvidia. Research on neural branch prediction dates back to the late 90s, but Nvidia says it has a “novel” neural branch predictor, perhaps building on previous research such as BranchNet.
The BPU feeds into the Instruction Fetch Unit, which holds 64 KB of L1 instruction cache, and loads into a decode queue that supports 48 instructions (we’ll go into the memory/cache layout later). At the last stage of the front end is that 10-wide decode, feeding more instructions into the execution engine per cycle than the 8-wide decode in AMD’s Zen 5 microarchitecture.
Past the front end, the mid-core rename / allocation engine is built to keep instructions moving while waiting on dependencies. In addition to renaming and allocation, instructions work through value prediction, which can speculatively execute the instruction, and memory renaming, where the instruction can move forward while a load is happening if the data relationship can be determined.
Inside the execution engine, Nvidia includes eight simple Arithmetic Logic Units (ALUs), two complex ALUs, and four branch units for resolution. For SIMD instructions, the execution engine includes a vector cluster for Arm’s Scalable Vector Extension (SVE), including six vector units that support 128-bit SVE instructions at FP8 precision, along with two crypto-enabled vector units that can handle AES, SHA, and SM3, among other prominent algorithms. Keeping data moving through the engine are four load units and two store units.
The cores support spatial multithreading, giving a Vera CPU with 88 cores access to 176 threads. Traditional SMT time-slices execution, giving both threads access to all of the core resources and sharing them as instructions execute in parallel. With spatial multithreading, each thread of an Olympus core has access to dedicated resources, allowing one of the threads to execute with high-throughput while the other thread handles simple tasks, or to operate as two independent execution environments.
The execution resources are partitioned, explaining the wide decode front end. It’s not clear, however, if the SMT implementation can also opportunistically grab resources, particularly in the scenario Nvidia describes where one of the threads is maximizing throughput while the other handles smaller tasks.
There’s a lot going on in Vera between the 10-wide decode, neural branch predictor, and spatial multithreading, but perhaps the most significant architectural design point is Nvidia’s second-generation Scalable Coherency Fabric (SCF). It underpins Nvidia’s approach of using a monolithic die as opposed to a chiplet-based design, distributing last level cache in a mesh across the die and avoiding the cross-CCD latency penalty with localized L3.
The mesh distributes data through a series of Coherency Switch Nodes (CSNs) that serve as routing points between cores and the 164 MB of distributed L3. These routing points further connect the cores and L3 to the memory system, I/O, and NVLink C2C for cache-coherent communication between chips. Nvidia’s benchmarks comparing Vera to AMD’s Epyc 9755 show that AMD can achieve slightly lower core-to-core latencies within a cluster, but Vera maintains significantly better core-to-core latency across the die, as expected.
Nvidia says SCF inside Vera has 3.4 TB/s of bandwidth, enabling faster core-to-core communication, especially when traversing the die. However, Vera also supports Memory System Resource Partitioning and Monitoring (MPAM), allowing portions of L3 to be partitioned in multi-tenant environments.
Vera uses SOCAMM2 LPDDR5X, which is a relatively new advancement that Nvidia’s competitors haven’t had the chance to benefit from. With the use of SOCAMM2, LPDDR5X provides similar modularity and capacity as traditional RDIMMs, but at significantly lower power draw.
The memory can run at up to 9600 MT/s, with aggregate bandwidth of 1.2 TB/s and per-core bandwidth of 14 GB/s, doubling the bandwidth of Grace. The Vera board supports eight SOCAMM2, offering capacity ranging from 256 GB to 1.5 TB. Nvidia claims a “fully populated” memory subsystem consumes between 30W and 40W depending on capacity.
For I/O, Vera supports PCIe 6.4 with 88 lanes per CPU and bifurcation support down to x2. It also supports CXL 3.1.
Unlike Grace, Vera includes Arm’s Confidential Computing Architecture (CCA) and Realm Management Extension (RME), including Device Assignment and Coherent Device Assignment, offering a boon to multi-tenant environments where VM isolation is key. Nvidia also implements TDISP for coherent devices, allowing for encrypted communication between GPUs and PCIe devices.
