Intel Announces Optane DIMMs for Workstations, SSD 665P With 96-Layer QLC, Roadmap

Intel made a slew of announcements today, including workstation support for Optane DC Persistent Memory DIMMs, which will eventually lead to support on the mainstream client desktops, like gaming PCs. These DIMMs slot into the DDR4 interface, just like a stick of RAM, but use Optane memory instead of DRAM to offer up to 512GB of memory capacity per stick.

Intel also announced new SSD 665p manufactured with 96-Layer QLC flash and revealed a roadmap outlining its future Optane SSD products and new 144-Layer QLC NAND. The company also announced that it is working towards developing 5-bit-per-cell flash, otherwise known as penta-level cell (PLC), to offer a path to increased storage density and lower-priced storage.

Paul Alcorn
Editor-in-Chief

Paul Alcorn is the Editor-in-Chief for Tom's Hardware US. He also writes news and reviews on CPUs, storage, and enterprise hardware.

  • jimmysmitty
    One step closer. While not as fast as traditional memory these will be a big game changer for storage as they are vastly faster than even PCIe 4.0 x4.
    Reply
  • IntelUser2000
    PCIe 4.0 doesn't even compare. While the bandwidth is there, the latency is not. The Optane drives are at 10us. These DIMMs are an additional 20-40x faster in terms of latency.

    You could get the bandwidth to be at 500GB/s, but it'll still be unusable for system memory if the latencies are in the 100us range as with SSDs.
    Reply
  • jimmysmitty
    DavidC1 said:
    PCIe 4.0 doesn't even compare. While the bandwidth is there, the latency is not. The Optane drives are at 10us. These DIMMs are an additional 20-40x faster in terms of latency.

    You could get the bandwidth to be at 500GB/s, but it'll still be unusable for system memory if the latencies are in the 100us range as with SSDs.

    These are not Optane drives, they are Optane DIMMs. They are basically non-volatile memory. They are still in the ns for latency, although they are higher then regular DRAM. However most systems allow for mixed setups with some DRAM and some Optane DIMMs, at least the servers have. So it might be possible to have half of the DIMMs be faster DRAM and the others be larger size Optane DIMMs allowing for storage that has vastly higher bandwidth than NVMe and vastly faster latency than NVMe (average is 20ms)
    Reply
  • IntelUser2000
    jimmysmitty said:
    These are not Optane drives, they are Optane DIMMs.

    That's pretty much what I said.
    Reply
  • bit_user
    jimmysmitty said:
    They are still in the ns for latency, although they are higher then regular DRAM. However most systems allow for mixed setups with some DRAM and some Optane DIMMs, at least the servers have. So it might be possible to have half of the DIMMs be faster DRAM and the others be larger size Optane DIMMs allowing for storage that has vastly higher bandwidth than NVMe and vastly faster latency than NVMe (average is 20ms)
    I'm not sure where you got 20 ms for an average NVMe, but that ain't right. Maybe you meant 20 microseconds, but 20 ms is hard disk territory. 20 us is in the ballpark, for SSD reads.

    Here's a fairly recent measurement of SATA drive IOPS:
    Source: https://www.tomshardware.com/reviews/crucial-mx500-ssd-review-nand,5390-3.html
    So, if you just take the reciprocal of the QD1 IOPS, then you get 83 to 183 usec. However, that includes the time to read & transfer the whole 4k of data, as well as potentially traversing the entire OS I/O stack & probably also the filesystem driver. I don't know what the standard is for measuring read latency, but it might be less.

    Here's the same measurement of high-end NVMe drives, from the Intel 905P review:
    Source: https://www.tomshardware.com/reviews/intel-optane-ssd-905p,5600-2.html
    Again, the same math for the 905P indicates 14.8 usec for the fastest Optane NVMe and 58.1 to 98.5 usec for the NAND-based NVMe drives.


    As for how latencies and bandwidth compare between Optane DIMMs and DDR4, there's a handy table in this article:

    https://www.tomshardware.com/news/intel-optane-dimm-pricing-performance,39007.html
    According to that, the bandwidth of a single Optane DIMM is roughly in the same ballpark as some of the high-end PCIe 4.0 NVMe drives - not vastly higher. AFAIK, Intel CPUs only support up to two channels, meaning your max is only double that.

    And latency is about 100x lower, for Optane DIMMs vs NVMe. However, I'm certain the measurement methodology for the DIMMs is just like a memory access. They can't be going through the OS, any filesystem, or even the kernel. And it's not going to be 4k, either. If you would measure the 905P in the same way, maybe the difference could be brought to within 1 order of magnitude.
    Reply
  • TerryLaze
    jimmysmitty said:
    One step closer. While not as fast as traditional memory these will be a big game changer for storage as they are vastly faster than even PCIe 4.0 x4.
    That's not even the important part of it.
    The CPU will be aware that the memory addresses of the optane dim are storage so all the OS calls of gofer this file gofer that file will fall away,the CPU will spend less time (as in none at all) with senseless copying around files from storage to ram only to copy them back after the CPU worked on them.
    Reply
  • bit_user
    TerryLaze said:
    That's not even the important part of it.
    The CPU will be aware that the memory addresses of the optane dim are storage so all the OS calls of gofer this file gofer that file will fall away,the CPU will spend less time (as in none at all) with senseless copying around files from storage to ram only to copy them back after the CPU worked on them.
    That requires OS support, which will take time.

    https://www.phoronix.com/scan.php?page=news_item&px=Intel-PMEMFILE
    Then, if you want apps to use the data in-place, that will take yet more time (unless they're already written to use mmap()).
    Reply
  • TerryLaze
    bit_user said:
    That requires OS support, which will take time.

    https://www.phoronix.com/scan.php?page=news_item&px=Intel-PMEMFILE
    Then, if you want apps to use the data in-place, that will take yet more time (unless they're already written to use mmap()).
    The very article you link to states that intel tries to keep the kernel i.e OS out of it to get better performance.
    It's all about the CPU having direct access without having to go through OS calls.
    Apps don't have to be optimized for it either,at least the apps us common people care about,they will automatically load up- and do all of their I/O -faster.
    Very specialized very I/O heavy apps will have to be specially written to take full advantage but that's the worry of server/workstation IT.

    " Intel's new user-space file-system for persistent memory is designed for maximum performance and thus they want to get the kernel out of the way. "
    Reply
  • jimmysmitty
    TerryLaze said:
    The very article you link to states that intel tries to keep the kernel i.e OS out of it to get better performance.
    It's all about the CPU having direct access without having to go through OS calls.
    Apps don't have to be optimized for it either,at least the apps us common people care about,they will automatically load up- and do all of their I/O -faster.
    Very specialized very I/O heavy apps will have to be specially written to take full advantage but that's the worry of server/workstation IT.

    " Intel's new user-space file-system for persistent memory is designed for maximum performance and thus they want to get the kernel out of the way. "

    I am all for best performance but there is a downside to bypassing the kernal. If written wrong it could cause hard crashes. Its much like the old days of gaming where a game could crash a system while now it will typically crash the program and API instead.

    However I have some faith that since Intel is heavily developing these for HPCs and enterprise first they would work to help prevent that sort of issue. Can't have a poorly written application taking a server down.
    Reply
  • bit_user
    TerryLaze said:
    The very article you link to states that intel tries to keep the kernel i.e OS out of it to get better performance.
    It's all about the CPU having direct access without having to go through OS calls.
    The OS must necessarily be involved at some point, for security, if nothing else. I believe the goal is to remove the OS from the "fast path" (i.e. individual data-accesses).

    TerryLaze said:
    Apps don't have to be optimized for it either,at least the apps us common people care about,they will automatically load up- and do all of their I/O -faster.
    Apps currently make OS calls. So, either the user has to use a shim, like that article cites, or the OS needs to be modified to do effectively the same thing. So, I think we're looking at having some level of OS support, before this stuff is ready for prime time.
    Reply