Intel Optane Memory is SRT Cache...Again (Updated)

Update, 4/24/17, 09:25am PT: Intel announced that the Optane Memory M.2 2280 modules are now available. The company hasn't shared pricing information or from what retailers you'll be able to purchase the modules.

Original article: 3/27/17, 09:00am PT:

Intel's Smart Response Technology (SRT) was so successful that we haven't talked about it since it was first introduced in 2011. We say that with a heavy dose of sarcasm, because Intel plans to bring the technology back with a faster media. Cached I/O isn't anything new. For the last decade, Microsoft and Intel have given users several options, ranging from USB thumb drives to accelerate I/O to speedy purpose-built single-level cell (SLC) SSDs. With Optane, the technology might just catch on, but not the way Intel sees it.

Intel just announced two new products that bring Optane technology to the consumer desktop. Optane is loosely defined as the company's products built with 3D XPoint technology, a next generation non-volatile memory structure built from the ground up to reduce latency. The new Optane Memory products will ship in two capacities (16GB and 32GB) and give users access to a whole new performance tier--as long as you have the supporting technology in place, mainly a 200-series chipset.

Pricing for Optane Memory M.2 2280 modules start at just $44 (16GB) and peak at $75 (32GB). The operating system recognizes the new products as addressable storage, just like a regular hard disk drive or solid-state drive. Intel told us that support for the drives as cache starts with the latest 200-series chipset products that feature an additional four PCI Express lanes over the older 100-series chipset.

The magic happens when you enable a "modified" version of Smart Response Technology and build a cache array with the Optane Memory standing invisibly in front of an HDD or SSD. The Optane Memory becomes a cache device that accelerates I/O for data retained in its memory structure from previous I/O requests.

Despite spending time with Intel, we have a lot of unanswered questions about all of the Optane family of products. There are a few theories about the memory technology, but almost everyone agrees that 3D XPoint is some type of phase change memory. We would like to learn more about the controller and how it communicates with the memory. The controller is quite small; to date, the only SSD controllers we've seen of this size are two-channel DRAM-less models used in non-performance environments, like point of sale (POS) machines. We have a few theories about this but will hold our tongues until we've had a chance to dig deeper.

A Pyramid Scheme

Intel re-purposed the memory latency/price graphic to show where Optane Memory fits in different system configurations. The graphic shows an Intel Optane SSD (more on that in a bit) for high-end systems like what an "eSports Gamer" would utilize. The product announced today, Optane Memory, fits in mid-tier systems as a cache device for traditional SSDs using NAND flash technology and in lower priced systems as a cache device paired with a traditional hard disk drive.

Although the media is new, the technology and the practice were introduced in the Intel ecosystem in 2011. Smart Response Technology is a feature built into the Rapid Storage Technology driver. Intel's first attempt at "SSD Cache" used a 20GB SSD (model 311 "Larson Creek") featuring 34nm single-level cell (SLC) technology. This was later refreshed in 2012 with a 25nm SLC SSD using model number 313 branding (Hawley Creek).

Both products worked remarkably well and spawned additional products sold as "cache SSDs" from third-party vendors. Intel took steps to retain an advantage over competing solutions. In 2014, TRIM was removed from the SRT feature list, which left the SSD to handle all collection activities without the guidance from the operating system. This left lower-priced, MLC-based products that had less advanced controllers with a severe disadvantage. Intel also limited the cache size to just 64GB. If your SSD was larger than 64GB, the additional capacity could be used as a separate addressable volume with a drive letter. The limits imposed kept users from building a large cache to place in front of the hard disk drive cold storage tier.

Third Time's The Charm

Smart Response Technology (SRT) was introduced during a transitional time for consumer SSDs. Crucial had just released the first SATA 6Gbps SSD (the C300), and SandForce proved that in-house firmware led to increased sales. Users had a taste of flash and started to consume a lot of it. That drove the cost of the technology down. Older SSDs appeared on the secondary market for pennies on the dollar, and that started a boom that is now a thriving industry. For a few dollars, users could build a hybrid array with a little flash in front of a spinning disk; or, for a few dollars more, you could boot to a full-speed SSD. The market spoke, cache technology lost its I and was just on the way Out.

In contrast, Optane Memory reshuffles the board and starts off with an advantage over traditional NAND-based SSDs. Intel and Micron own 3D XPoint technology, and we don't expect to see the two competing against each other. Intel has stated that 3D XPoint will not be for sale to third-party customers, so you won't see, for example, an Adata Optane product on store shelves to drive costs down. Optane-based products are the new SLC, but this time no one plans to share the secret sauce to outside companies. If you want a 5x to 8x increase in low-queue depth performance, only two companies are in the kitchen.

Optane Memory comes during another transitional period for flash. The market has shifted to lower-cost, high-capacity products that use internal cache technology to mimic performance found in previous generations. Two-bit per cell NAND is nearly nonexistent in consumer SSDs, barring a few standout SSD manufacturers that entertain Toshiba executives. MLC will disappear within a year for all but very high-cost SSDs designed for prosumers.

Toshiba will finish the factories for 3D BiCS technology. Its partner, SanDisk, has already publicly stated that TLC is the new normal for consumer SSDs. If you thought TLC was bad, just wait for quad-level cell (QLC) flash that will certainly require an appetizer of Optane Memory to be palatable.

Intel has made several bold claims about Optane Memory technology. The new storage media reaches high speeds much faster than current NAND technology, which requires parallel reads and writes to achieve high performance. Optane Memory can surpass consumer SSD performance at high queue depths while operating at very low queue depths. This benefits users because most I/O comes at just queue depth 1 or 2. It's rare to reach queue depth 4 even while multitasking with several applications. 

Swing And A Miss: The Cost Of Failure

Cache technology brings a set of inherit risks. Stacking cache on top of cache amplifies the penalty of a cache miss. As an analogy: In baseball, you may go through several batters to get a hit. That takes time, and you could end up on the losing side of performance when an I/O request first strikes out the Optane cache, then the SLC layer cache, and finally has to reach out to the third layer. In this case, it's the TLC. The third batter gets a hit, but there are already two strikeouts on the scoreboard for the inning. Holding more data in the high-speed tier one cache helps to increase your chances of a hit. The more applications you use, the wider the strike zone becomes. Your core, daily use software is ready to swing for the fences, but the application you haven't run in six months has to be flown in from the Triple-A league. If your primary media is a hard disk drive, then you see a massive 20ms delay after swinging at fast balls for most of the day.

Cache Isn't King

Prosumers and working professionals that rely on performance consistency and use large data sets are not the core audience for Optane Memory. I'll be one of the first to take advantage of the new 3D XPoint technology in a RAID 0 array with three 32GB M.2 NVMe drives configured in a 96GB array to test what the technology is capable of.

Intel spoke softly about an Optane SSD, which will serve as a normal boot volume, and gave us permission to acknowledge its existence. So, we can confirm that it's going to be a Real Thing. Unfortunately, Intel didn't give us much more than that. We can speculate that the Optane SSD will have a relationship to the just-announced DC P4800X that's similar to the one between the DC P3700 (enterprise SSD) and the 750 Series (consumer NVMe SSD). Intel has reconfigured enterprise-class products in the past for those willing to spend the money for extreme performance. We expect an add-in card form factor but were also told that, "Optane becomes very interesting with U.2."

The message is just as clear as it is cryptic. The DC P4800X will see two larger capacity models around the same time Intel releases the technology in a U.2 form factor.

Chris Ramseyer
Chris Ramseyer is a Contributing Editor for Tom's Hardware US. He tests and reviews consumer storage.
  • jimmysmitty
    Everything has a target market and this is probably better for workstations rather than regular joes.

    However it is the start of something larger that so far only Intel is pushing towards, which is NVDIMMs. I can't wait to see large NVDIMMs so I can make an OS drive in the memory. Right now Kaby lake has memory bandwidth of about 50GB/s. That is 5x faster read speeds available than a x16 PCIe SSD Seagate announced last year (10GB/s).

    And that's on the mainstream platform. That would be instant on. I want that.

    Of course DIMM size and availability will take time, as well as affordability, but I am fine with Intel testing ways to help improve it.
  • manleysteele
    Call me pedantic, but, to me, it's not memory until it plugs into a memory slot. I already have 32 GBytes of memory in two memory slots with two more slots outstanding. The only reason I reserved two slots on this build was for a pair of 250 to 1000 GByte Optane memory modules that would make memory as storage viable. I could already have put two more DIMMS in at this size that will badly outpace this as cache (ramdisk).
  • nzalog
    If you're going to be pedantic at least be accurate. Memory is anything that can store data. If it's RAM, ROM, Disk or part of a human brain it's irrelevant
  • manleysteele
    We old guys know the difference between memory and storage.
  • Better use that money to buy more ram.
  • CRamseyer
    It is a beautiful new technology but we are still in the early phase. In the article we stated Optane is the next SLC. This is a very exciting moment in history but it will take time for Intel to give us what we really want.

    IMFT isn't the only company working on next generation memory technology, its just in the spotlight today and the first to mass production.

    Optane Memory AKA "Next Gen SSD Cache" is what we need moving forward with "low cost" TLC NAND. I think everyone will benefit from a $45 cache but we all really want a sizable boot volume that can hold a few games and other applications.

    It's pretty obvious that will come but the first iteration will come with an Extreme Edition price.
  • Pompompaihn
    Got a crazy idea that probably isn't the least bit functional, practical, or even grounded in reality and/or has already been done:

    Some form of PCI card that you can plug old generation RAM into to use as a dedicated RAM disk. Would be a good way of utilizing old DDR3 RAM or whatnot on new machines, and achieve basically what this is doing without needing to pay for NAND.

    *end idea*
  • CantBeStoped
    Can This cache a NVME M.2 drive? Would it even worth getting a Cache Drive for a M.2 SSD over 500GB?
  • zodiacfml
    I was thinking almost the same thing as the guy above with the comment on "RAM". Once I have been using systems with substantial RAM, HDD speed is not much of a problem to me anymore.

    I have an SSD in a laptop using now but the speed increase isn't dramatic except for installation of programs which feels a bit quicker.

    Booting is still slow but I don't shutdown my laptop anymore as I use it every day anyway.
  • RomeoReject
    I think the biggest reason this won't catch on is that it is already starting from a niche position - using a cache to increase certain tasks for the rest of the storage to handle is a small target to hit. But then they've gone from a small niche to a microscopic one by limiting this to one chipset that hasn't seen much adoption, and only keeping it to two manufacturers.

    I foresee this having the exact same long term impact as SRT did.