Nvidia's homegrown memory design is almost standardized and ready for everyone to use — JEDEC says SOCAMM2 compact DRAM module for AI servers boasts higher speeds and broader compatibility
Nvidia-backed SOCAMM needed more refinement to be production-ready
(Image credit: Micron)
SOCAMM2 is on track to become the definitive version of the SOCAMM standard very soon. JEDEC has announced that SOCAMM2's design is nearing completion and will take advantage of the same LPDDR5X memory type as its predecessor. Two upgrades will be added to SOCAMM2: an SPD profile (what most might refer to as a "JEDEC profile") and improved LPDDR5X transfer rates of up to 9600 MT/s.
The new memory type will be incorporated into next-gen AI servers, says JEDEC. Although no official announcements have been made, Nvidia seems likely to take another shot at introducing SOCAMM2 alongside its Vera CPUs on its next-generation Rubin platform. Servers integrating the first SOCAMM memory modules were reportedly abandoned by Nvidia due to technical difficulties.
The multi-trillion-dollar AI giant reportedly couldn't get SOCAMM to operate within temperature limits, causing overheating on its Blackwell Ultra GB300 "Cordelia" platform. As such, Nvidia reconfigured "Cordelia" to traditional LPDDR memory for its release.
For reference, SOCAMM is a new memory design that is aimed squarely at the data center, particularly AI servers. SOCAMM is a more compact version of traditional DRAM DIMMs that purports to be more power-efficient.
For example, Micron's SOCAMM 128GB module allegedly consumes just a third of the power of an equivalent 128GB RDIMM. DRAM power consumption is very important in data center environments, where memory power consumption can often exceed that of the CPUs it's attached to.
The new form factor was inspired by CAMM2, and both SOCAMM and CAMM focus on improving memory capacity per unit area compared to traditional DIMMs. For example, the original SOCAMM modules from Micron measured just 14x90mm while featuring four 16-die LPDDR5 memory stacks.
Nvidia's original version was completely proprietary and was only compatible with its own hardware. It also only supported LPDDR5X transfer speeds of up to 8533 MT/s. SOCAMM2 supports even faster LPDDR5X and, in collaboration with JEDEC, should be available for hardware manufacturers other than Nvidia to adopt.
Stay On the Cutting Edge: Get the Tom's Hardware Newsletter
Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.
There's no official release date for SOCAMM2, but it is likely that the design will be finished in the next few months. This will give SK Hynix, Samsung, and Micron time to build production SOCAMM2 modules and, critically, give Nvidia time to test the modules for its Rubin deployments beginning next year.
SOCAMM is a more compact version of traditional DRAM DIMMs that purports to be more power-efficient.
And more importantly, it achieves that lower power at higher bandwidths, as it avoids all the signal integrity issues of DIMM slots and gets close to the performance of DRAM dies directly soldered to the host PCB.
The improvement in packaging volume over right-angle SODIMM slots is a minor factor at most.
The improvement in packaging volume over right-angle SODIMM slots is a minor factor at most.
What I think SOCAMM/2 does right that spoiled the prior CAMM designs is the fact that it's single-sided. Heat is becoming a significant issue for DDR5 scaling, hence why I think it was a bad decision for those other designs to put DRAM chips on the underside, where they'd be sandwiched against the motherboard.
