Meta fights soaring hardware costs by reusing old DDR4 server memory in new DDR5-only servers — custom CXL 2.0 chip marries legacy DDR4-2400 with cutting-edge DDR5-6400

72 32GB HPE DDR4-2666 ECC RDIMMs
(Image credit: cyberchief/Reddit)
18

The price of DDR5 memory is setting new highs these days as demand badly outstrips supply. In a bid to save money, Meta is recovering legacy DDR4 memory from used servers and is installing it into new machines using its in-house developed Vistara ASIC that enables it to connect old memory modules to its latest servers running AMD EPYC 'Turin' processors that only support DDR5 memory.

Interestingly, Meta is not the only company developing such a solution. Panmnesia, a startup from South Korea, has developed an off-the-shelf CXL controller and switch that enables servers to attach considerably larger memory pools without extending latency, which differentiates Panmnesia’s solution from competing CXL offerings.

Latest Videos FromTom's Hardware
Anton Shilov
Contributing Writer

Anton Shilov is a contributing writer at Tom’s Hardware. Over the past couple of decades, he has covered everything from CPUs and GPUs to supercomputers and from modern process technologies and latest fab tools to high-tech industry trends.

  • edzieba
    Somehow, RAMdisk returned.
    Reply
  • bit_user
    edzieba said:
    Somehow, RAMdisk returned.
    No, this is a true memory expansion. It does not appear to the system as a drive (NVMe or otherwise).

    That said, the approach of relying on page migration does presume certain access patterns and requires the direct-attached memory to be large enough to hold the working set (i.e. if you want to achieve good efficiency). I think those are very achievable hurdles.

    The article said:
    The software stack transparently exposes CXL memory as a separate NUMA node and enables Linux to migrate cold pages to the slower DDR4 tier (with 76 GB/s of bandwidth) and retain frequently accessed data in local DDR5 (with 614 GB/s of bandwidth).

    Back in the bad old days of DOS, there were memory-expander cards that I think worked in a fashion that more resembles VGA video memory addressing.
    https://en.wikipedia.org/wiki/Expanded_memory#Expansion_boards
    Here's an example:
    https://www.ebay.com/itm/307031071607
    Reply
  • derekullo
    bit_user said:
    No, this is a true memory expansion. It does not appear to the system as a drive (NVMe or otherwise).

    That said, the approach of relying on page migration does presume certain access patterns and requires the direct-attached memory to be large enough to hold the working set (i.e. if you want to achieve good efficiency). I think those are very achievable hurdles.


    Back in the bad old days of DOS, there were memory-expander cards that I think worked in a fashion that more resembles VGA video memory addressing.
    https://en.wikipedia.org/wiki/Expanded_memory#Expansion_boards
    Here's an example:
    https://www.ebay.com/itm/307031071607
    Somehow that Ebay listing threads the line between good condition and partly working!!!
    Reply
  • teeejay94
    Maybe stop allowing billionaires to buy up entire supply of products for their own personal gain
    Reply
  • usertests
    teeejay94 said:
    Maybe stop allowing billionaires to buy up entire supply of products for their own personal gain
    Stop being poor.
    Reply
  • bit_user
    teeejay94 said:
    Maybe stop allowing billionaires to buy up entire supply of products for their own personal gain
    That's not what's happening. It's actually lots of investors pouring money into the AI and datacenter companies, who are then buying up the memory. And if you have money invested in a retirement account, like a pension fund, IRA, or 401(k) fund, then you could be part of the problem (or, at least your money, as invested by the fund manager, might be contributing to the problem).

    One way to mitigate this problem of over-investment would be to increase interest rates. That's starting to seem likely, but it's too little, too late.
    Reply
  • timecop1818
    bit_user said:
    Here's an example:
    https://www.ebay.com/itm/307031071607
    man, that's funny. in today, that entire massive board can be replaced with a single chip SRAM and a simple CPLD to do the ISA stuff. literally two chips and a bit of power management and some bypass caps

    edit and probably some i/o buffers as unlikely anything modern runs at 5v
    Reply
  • Sp1r0s
    In any normal market, 300% and higher price increases would cause suppliers to increase production and also prompt new competitors to start building new production facilities at pace. They can sell something that used to be priced at $40-ish for $200 or more and make massive profits. Why aren't there new businesses taking advantage?

    This has been going on for about 8 months now and yet it seems the usual market reaction is not happening
    Reply
  • Setnev
    Sp1r0s said:
    In any normal market, 300% and higher price increases would cause suppliers to increase production and also prompt new competitors to start building new production facilities at pace. They can sell something that used to be priced at $40-ish for $200 or more and make massive profits. Why aren't there new businesses taking advantage?

    This has been going on for about 8 months now and yet it seems the usual market reaction is not happening
    Reply
  • Setnev
    Thats because there are only a handful of foundries and the process takes billions of dollars just to ramp up. What we need to do is stop overinvesting in cutting edge technologies and start investing in technologies that enhance what we currently have without making them obsolete so fast, just like this CXL chips adapters being developed. DDR4 already exists. We've proven older hardware can be retrofitted. Lets do this. This is the equivalent of an automaker spending billions of dollars to increase the thermal efficiency of an engine but only changing how it works inside and ignoring the outside expecting better results. New tech is reaching a point of diminished returns and at the consumer level, we're forced to pay up or shut up. Bring in the new era of chipmakers, innovators, and inventors that can turn our e-waste into more efficient technology.
    Reply