The GTC 2024 show floor was teeming with interesting tech demos as Nvidia's partners looked to capitalize on the glitz and glam of what will probably be the largest bevy of AI GPU announcements of the year. Astera Labs' demo of its working PCIe 6.0 connectivity with its new Aries retimers was among the most interesting — the company had a demo of a working PCIe 6.0 retimer that reduces the amount of energy required per bit to less than half that of the preceding generation.
The demo also gives us a compelling look at the nuts and bolts of the components required to assemble the world's fastest AI GPU clusters by increasing the reach of the PCIe 6.0 connection between next-gen GPUs and CPUs, like Nvidia's new Blackwell B100, B200, and GB200 silicon.
AI workloads require high levels of throughput between GPUs to process data at full speeds, which is why Nvidia has its custom NVLink and NVSwitch solutions to offer massive amounts of GPU-to-GPU communication, with up to 1.8 TB/s of bandwidth on GB200 systems like the GB200 NVL72. However, not everyone wants to go all-in with Nvidia, and a variety of partners still use PCIe-based GPUs.
Many of today's high-powered GPU clusters employ fast PCIe 5.0 x16 connections that deliver 128 GB/s of bidirectional bandwidth between the components to keep the GPUs as saturated as possible and utilize them to the fullest. The industry is now moving to the PCIe 6.0 interface to double bandwidth once again, reaching up to 256 GB/s of bandwidth on a x16 connection. This necessitates even faster supporting components to extend the reach of the interface, but performance-sapping bit error rates are becoming even more of a challenge with the new interface.
That's where Astera Labs' Aries Smart DSP retimers step in, with the small chips boosting the PCIe 6.2 signal in the above demo. We can see the Astera Labs demo systems, with the boards in the second image running an active PCIe 6.2 link between the host on the left and the load generator on the right. The Aries PCIe 6.0 retimer resides on the interposer board in the middle, ingesting and boosting the traffic through its two PCIe x16 connections (one on each side of the retimer chip).
The company also showed live power measurements of its chip, which it bills as the world's lowest-power retimer. While it was under load, the chip showed an 11W power draw. That's a considerable 15% reduction over the 13W consumed by PCIe 5.0 while delivering twice the bandwidth of the older interface. Together, that's a reduction of more than 2x for the amount of power required to move each bit. Part of the power reduction stems from moving to a newer TSMC node for the retimer silicon, but Astera isn't sharing which node it currently uses.
As you can see in the other images, Astera also had a bevy of PCIe 5.0 devices connected to its new retimer chips, all housed on a single system. This demo shows the interoperability of the retimer chips, but the company couldn't demo this system with working PCIe 6.0 GPUs, SSDs, NICs, and the like, as they remain under NDA.
The move from PCIe 4.0 to 5.0 doubled data transfer rates by increasing the bus frequency, which then imposed new restrictions like a roughly halved trace length. That meant devices had to be placed closer together to maintain signal integrity, along with requirements for thicker motherboards and more robust materials.
Retimers take an incoming PCIe signal, then clean and amplify it, thus increasing the reach of the connection to devices over longer distances. This is a must with multi-GPU deployments, where the various GPU slots can be quite distant from the CPU and chipsets providing PCIe connectivity. In fact, Astera Labs' current generation of PCIe 5.0 retimers are already a fixture in Nvidia's HGX systems (more on that below).
The new PCIe 6.0 interface doubles throughput once again, but the increase comes via a switch to PAM-4 encoding from NRZ instead of a frequency increase, so reach/distance requirements remain roughly the same. However, PAM-4 encoding is more susceptible to errors, so error rates have increased despite the addition of forward error correction (FEC). As such, while bandwidth has doubled, error rates have also increased, thus reducing the effective bandwidth.
Robust retimers help ensure signal integrity, and Astera Labs' customers can either design systems with longer trace lengths and accept the existing error rates, or they can use short trace lengths and reduce the error rate, thus boosting the effective bandwidth. This provides flexibility for their design points, and the company's COnnectivity System Management and Optimization Software (COSMO) allows customers to monitor the retimers in real-time during deployment, as pictured in the seventh image in the above album, providing easy options for Link, Fleet, and RAS management, optimizations, and diagnostics/troubleshooting.
Astera Labs has already had a big win as the company's previous-gen retimers power Nvidia's HGX GPU systems. Here, we can see an existing HGX H100 system with eight fire-breathing H100 GPUs in the OAM form factor placed on a universal baseboard (UBB, an OCP spec design).
This system now powers much of the world's AI infrastructure, so much so that it remains in a state of shortage, and we can see the eight rectangular Astera Labs PCIe 5.0 retimers on the left of the motherboard in the second image. Another eight retimers also reside on the right side of the motherboard, for a total of 16 chips per board, boosting the signals between the GPU and the CPUs that reside on a connected motherboard (not shown here). The connected motherboard (diagrams in the slide deck below) also houses nine retimers as well, so there is plenty of Astera silicon in every shipping Nvidia HGX system.
As you can imagine, Astera has already shipped a considerable amount of these chips to Nvidia. Though the company isn't speaking of any other design wins yet, we imagine that Nvidia's next-gen HGX Blackwell systems will also utilize Astera's silicon.
Astera Labs launched the Aries PCIe 6.0 retimers at the show, and above you can see the company's launch slide deck with plenty of additional details on the silicon. The Aries retimers will also be used extensively for CXL 3.1 deployments, an emerging type of interface that allows for a cache-coherent interconnect between CPUs and accelerators, like GPUs, smart I/O devices like DPUs, and various flavors of DDR4/DDR5 and persistent memories. The CXL interconnect allows the CPU to work on the same memory regions as the connected devices, thus improving performance and power efficiency while reducing software complexity and data movement.
The Aries retimers can be deployed in many different types of use cases, such as chip-to-chip, box-to-box, and rack-to-rack scenarios. The company claims its chips increase the reach between devices by over 3X compared to inexpensive interconnect options. The addition of support for the CXL 3.1 interconnect in addition to standard PCIe 6.2 connections significantly increases the number of use cases for the retimers, so we'll likely see plenty of Aries design wins over the coming years, not to mention the extension of its use on Nvidia's next-gen Blackwell platforms.
