Skip to main content

Samsung's Memory-Semantic CXL SSD Brings a 20X Performance Uplift

Samsung
(Image credit: Samsung)

Samsung announced its Memory-Semantic SSD that combines flash and DRAM to provide a claimed 20X performance improvement compared to a traditional SSD. Modern solid-state drives with a PCIe Gen4 or PCIe Gen5 interface enable extremely high sequential read/write throughput and random access performance. But for many performance-hungry applications, this is not enough as they need storage devices explicitly tailored for their usage patterns. So, to meet the requirements of artificial intelligence (AI) and machine learning (ML) workloads, Samsung is turning to its new CXL-connected SSD. 

AI and ML applications usually process large datasets, but they do it in relatively small-sized data chunks, which is why traditional data center grade SSDs optimized for mixed workloads are usually not the best storage devices for those types of workloads.

To address such workloads, Samsung developed its special Memory-Semantic SSD that combines a huge built-in DRAM cache, traditional NAND storage, and a PCIe Gen5 interface with the CXL technology on top. Applications can write data to the DRAM cache at DRAM speeds and with low latency enabled by the CXL.mem protocol. Then, Samsung's proprietary controller (or controllers) of the drive transfers data to NAND memory. The result is a 20 times random read/write performance uplift compared to traditional SSDs. 

The CXL cache-coherent protocol promises to enable substantial performance uplift for applications that use various memory expansion modules and compute accelerators by enabling memory/cache coherency and dramatically reducing memory access latency by various compute devices. With its Memory-Semantic SSD, Samsung takes advantage of high-performance DRAM (like other Type 3/CXL.mem devices) and high-capacity NAND storage memory. To a large degree, Memory-Semantic SSD is a unique storage solution tailored to read and write small-sized data chunks at very high speeds. 

Samsung hasn't announced which version of the CXL protocol it supports, but the newly-announced CXL 3.0 revision will support mixed-media (like DRAM and storage) devices that can be connected to the fabric. However, these new Global Fabric Attached Memory (GFAM) devices are newly defined, so although Samsung's device meets the general description of this type of device, we likely won't see the expanded GFAM functionality until future generations of the SSD.

Samsung has not provided any details or specifications of its Memory-Semantic SSDs, so we do not know what kind of DRAM, 3D NAND, and controller the company uses. In fact, the company even did not disclose the actual performance numbers for the drive. 

Perhaps, that's because Memory-Semantic SSD is tailored for next-generation platforms (such as AMD's Genoa and Intel's Sapphire Rapids) that are quarters away, and Samsung cannot disclose relevant performance numbers as they are confidential for now. Another reason why Samsung does not publish even raw performance numbers or specifications is that its Memory-Semantic SSD is also quarters away. As a result, some of its specs may change by the time it becomes available.

Anton Shilov
Anton Shilov

Anton Shilov is a Freelance News Writer at Tom’s Hardware US. Over the past couple of decades, he has covered everything from CPUs and GPUs to supercomputers and from modern process technologies and latest fab tools to high-tech industry trends.

  • dehjomz
    I know this is a datacenter application, but for the consumer space, if/when CXL-enabled CPUs come to market, can the OS be installed to a CXL drive, and what, if anything, will a CXL-based drive do for OS load times, application load times, and game-level load times? Will CXL drives enable persistent memory, for near instant OS loads from cold boot? Is the proliferation of CXL why Intel cancelled Optane? This samsung drive seems like Optane on steroids.

    I have so many questions about what CXL will enable for the consumer market. Like as the PC consumer market begins its transition to DDR5 RAM, should I save my DDR4 RAM modules... will they be able to one day be used in a CXL.mem device for additional RAM/cache/storage?
    Reply
  • abufrejoval
    Well the promise of NV-DIMMs was that you wouldn't ever boot again unless you had to recover from a catastrophic failure. When memory is non-volatile you can just switch off and on and resume very much like you do on an Android.

    And then remember that servers aren't supposed to boot or be idle: cloud vendors invest much more in making sure that their ARM servers are always loaded than in copying these wonky turbo or power saving features x86 has from its desktop and notebook legacy.

    I don't think anyone on CXL comittees is thinking about consumer use cases, I doubt we'll see "CXL ready" stickers on any Intel notebooks or gamer kit.

    CXL is designed and hopefuly paid for by hyperscaler customers and/or deep government sponsored HPC pockets.

    Perhaps some consumer gaming or desktop kit may eventually speak CXL, but only once it's cheaper to reuse IP blocks from the cloud/HPC kit instead of maintaining PCIe-only libraries.

    Generally the industry does not want you to reuse any RAM kits (or any other components, for that matter): It's bad for business!

    But from former colleagues of mine, who produced HPC hardware, I also heard that DRAM really does age in HPC environments (probably in clouds, too) and would eventually become unreliable and fail, at which points all savings from recycling are quickly destroyed. Actually CPUs (and other logic chips) do, too, they just used to have higher margins built in.

    That's changing as the competition is heating up ...quite literally.

    And then the biggest challenge I see with CXL is that none of today's operating systems are near able to understand what it is or how to manage all the variety of devices thus connected. Remember that both Linux and VMS/WNT are DDOS, dictator disk operating systems with a bit of networking grafted on them and no notion of peers and on how to collaborate with them.

    Unix can't really tell a GPU from a printer or audio port. And believe me you won't like what happens on your screen if Linux were to try to schedule 10k GPU cores on an RTX 4000. And now just imagine it trying to manage the allocation of memory blocks between CPU and GPU cores spread around various motherboards and busses across the data center...

    I recommend you not hold your consumer breath for CXL.
    Reply
  • jkflipflop98
    None of that makes a shred of technical sense.
    Reply
  • InvalidError
    dehjomz said:
    I have so many questions about what CXL will enable for the consumer market.
    It won't do anything for the consumer market. CXL is little more than PCIe with a cache-coherency (and encryption in 2.0) layer on top. It makes no sense on single-socket consumer systems where the only system memory pool in the system is controlled by the only CPU in the system.

    CXL is primarily intended for stitching large-scale systems (multiple processors, accelerators, large-scale storage, etc. across multiple boards) together where something needs to keep tabs on what is being cached where to mitigate potential memory IO conflicts between all of the stuff that is on the bus competing for memory access across all of the different system-wide memory pools.

    Basically, CXL is "Big-Tin" stuff. Unnecessary overhead and complexity for consumer stuff.
    Reply
  • USAFRet
    dehjomz said:
    I know this is a datacenter application, but for the consumer space, if/when CXL-enabled CPUs come to market, can the OS be installed to a CXL drive, and what, if anything, will a CXL-based drive do for OS load times, application load times, and game-level load times? Will CXL drives enable persistent memory, for near instant OS loads from cold boot? Is the proliferation of CXL why Intel cancelled Optane? This samsung drive seems like Optane on steroids.

    I have so many questions about what CXL will enable for the consumer market. Like as the PC consumer market begins its transition to DDR5 RAM, should I save my DDR4 RAM modules... will they be able to one day be used in a CXL.mem device for additional RAM/cache/storage?
    We're already close to that point.

    Consider boot time

    HDD:
    15 sec of BIOS,
    15 sec of drive and Windows

    30 seconds total.

    SSD that is 5 times faster than the HDD
    15 sec of BIOS
    3 sec of drive and Windows

    18 secs total.

    Faster, but not magical.


    Take the spinning drive out of the equation with solid state, and we are way deep into diminishing returns.
    Raw drive speed is only a small part of the equation.
    Reply
  • InvalidError
    USAFRet said:
    Faster, but not magical.
    If you don't like boot times...

    Standby:
    0s BIOS
    0s boot
    ~1s for the monitor to detect sync regardless of SSD or HDD.

    I reboot my system maybe 4-5 times per year and that includes the 1H/2H Windows updates.
    Reply
  • USAFRet
    InvalidError said:
    I reboot my system maybe 4-5 times per year and that includes the 1H/2H Windows updates.
    That's about the same amount as I do.
    Major updates, and after a vacation.

    Other than that, it is ON and idling.
    Reply
  • Makaveli
    USAFRet said:
    That's about the same amount as I do.
    Major updates, and after a vacation.

    Other than that, it is ON and idling.

    Same but I also do a few more restarts due to GPU driver updates, AGESA Bios updates so my number is more in the 10-12 range for the year.
    Reply
  • escksu
    Despite the fancy names, this product is basically just having a DRAM cache/buffer pairing with NAND. Concept is not new and has been around for ages. HDDs have a small buffer that works the same way, just that its much smaller in size (~32-64MB). This is device level implrementation.

    For PC side, a portion of our main memory is being used as cache for data from SSD/HDD. Serves the same purpose.

    Btw, all these do not speed up boot time. When you power off the machine, the data from the cache is flushed to the storage. So, when you power up and boot to windows, the cache has to be filled from storge (bottleneck is storage here).
    Reply
  • escksu
    dehjomz said:
    I know this is a datacenter application, but for the consumer space, if/when CXL-enabled CPUs come to market, can the OS be installed to a CXL drive, and what, if anything, will a CXL-based drive do for OS load times, application load times, and game-level load times? Will CXL drives enable persistent memory, for near instant OS loads from cold boot? Is the proliferation of CXL why Intel cancelled Optane? This samsung drive seems like Optane on steroids.

    I have so many questions about what CXL will enable for the consumer market. Like as the PC consumer market begins its transition to DDR5 RAM, should I save my DDR4 RAM modules... will they be able to one day be used in a CXL.mem device for additional RAM/cache/storage?

    No, it does nothing for consumer market. Consumer PCs already have this speed all along. All PCs (be it windows/linux/MacOS)) are known to utilise part of the man memory for caching data from storage (be it HDD or SSD). So, using CXL SSD will do nothing to improve performance.

    Such SSDs are only mean for intensive servers by freeing up CPU and main memory for other tasks. In such environment, the CPU simply transfer data from main memory to CXL SSD DRAM cache (much faster than transferring to NAND). After that, the drive manages its own trnasfers to NAND.
    Reply