One of the advantages that servers based on Intel's 2nd and 3rd Generation Xeon Scalable processors have over AMD EPYC-powered machines is support for Optane Persistent Memory modules that can be used to expand main memory capacity relatively cheaply--something that is important for many applications. But Smart Modular has developed a card that can add Optane PMMs to any machine. Ironically, the card is Intel-based.
Apparently, demand for storage-class memory on existing AMD and Arm machines is significant enough to encourage Smart Modular to develop its special Kestral PCIe Optane add-in-card for these servers. The Kestral PCIe Optane AIC carries up to four 512GB Intel Optane Persistent Memory modules and can be plugged into any PCIe 3.0/4.0 x16 slot to add up to 2TB of Optane memory. A PCIe Gen4 x16 slot supports a peak bandwidth of 32GBps in each direction, which is higher when compared to one DDR4-3200 memory channel (25.6 GBps), but is lower when compared four DDR4 channels (102.4 GBps).
Intel's Xeon Scalable 'Cascade Lake' and 'Ice Lake-SP' processors communicate with Optane Persistent Memory modules that are plugged into regular DDR4 memory slots using the company's proprietary DDR-T protocol, which is not supported by competing CPUs. The protocol is required for Optane PMMs to provide a 64B cache line access granularity while taking into account the nature of these modules (which use a special Optane memory controller).
To make Intel's Persistent Memory modules work on its Kestral PCIe AIC, Smart Modular carries Intel's Stratix 10 DX FPGA, which fully supports the DDR-T protocol and works like a DDR4-to-DDR-T memory translator hub. Also, the AIC can use FPGA's four Arm Cortex-A53 cores to accelerate/enable certain workloads. The board comes with 2GB of DDR4 ECC memory and an 8GB eMMC storage module. The card consumes less than 75W in memory expansion use cases and less than 100W when used for memory acceleration. It comes in a full-height half-length (FHHL) form-factor and is equipped with a dual-slot passive cooler. The board can be used for field upgrades of existing servers to support new workloads.
While the card is compatible with virtually all existing servers (and even future machines with Nvidia Grace CPUs), perhaps the biggest challenge to make it work efficiently will be making software use it properly. Perhaps, for those customers who already use thousands of AMD EPYC-based machines and who need to repurpose them for different workloads that require expensive memory expansions, usage of Kestral with Optane Persistent Memory modules will make sense. Others may opt to expand their memory capacity using regular DIMMs, or just wait for next-generation servers with CXL protocol support.
"With the advancement of new interconnect standards such as CXL and OpenCAPI, Smart's new family of SMART Kestral AICs addresses the industry's need for a variety of new memory module form factors and interfaces for memory expansion and acceleration," said Mike Rubino, SMART Modular’s vice president of engineering.
Smart Modular's Kestral PCIe Optane AIC is now available for orders from the company. Pricing is unknown, but don't expect it to be low as we are dealing with a highly-specialized product based on a high-end FPGA.
