Apple Patent May Hint at the Future of Its Chips: A Multi-Level Hybrid Memory Subsystem

Apple has patented a hybrid memory subsystem that comprises at least two types of memory: a high-bandwidth low-density type of DRAM as well as a low-bandwidth high-density type of DRAM. The patent may provide a glimpse of how Apple sees the future of its system-on-chips. Naturally, the new patent will fuel speculation that we could see newer versions of Apple's M1 chip come with the new memory design, but this type of implementation could be used in several different types of chips.

Patents don't always manifest as products on the market, but given the feedback we've received from patent lawyer Kerry Creeron, it appears that Apple has gone to great pains, and expense, to patent this new technique in a wide array of jurisdictions around the world, implying that it could be used in Apple's products worldwide. 

New System Architecture Requires New Memory Architecture

Apple's switch to its M1 SoCs from Intel's CPUs and AMD's discrete GPUs is significant not only because the company is transitioning from x86 CPUs to Arm, and from Radeon CPUs to a PowerVR-derived architecture, but because it significantly changes the system architecture of its PCs. 

CPUs and discrete GPUs have their own memory subsystems, whereas SoCs usually rely on a unified memory architecture. UMA has numerous advantages over dedicated memory subsystems, but it also has one important disadvantage, too: in an UMA system, the CPU and GPU have to share memory capacity and bandwidth, something that may affect performance in certain scenarios.  

HBM2 and HBM2E types of memory provide very high bandwidth, but these memory devices cost a lot, consume a lot, and cannot be upgraded by the end-user. In contrast, building a high-capacity memory subsystem with enough bandwidth for a high-end GPU using conventional or GDDR-type memory isn't always possible, or feasible. In a bid to combine the best of both worlds, Apple has patented a hybrid memory subsystem combining HBM-like and DDR-like types of DRAM. 

Apple's patent entitled "memory system having combined high density, low bandwidth and low density, high bandwidth memories" describes various SoCs that use high-bandwidth cache DRAM as well as high-capacity main DRAM. The patent covers strictly system-on-chips and not PCs, so all DRAMs are supposed to be soldered down onto the substrate or onto the mainboard, just like the LPDDR4X chips that Apple uses on its M1 SoC. The architecture described in this very patent indicates that Apple doesn't envision usage of standard memory modules, at least with some of its hybrid memory subsystems. 

"With two types of DRAM forming the memory system, one of which may be optimized for bandwidth and the other of which may be optimized for capacity, the goals of bandwidth increase and capacity increase may both be realized, in some embodiments," a description by Apple reads. "[Some implementations of hybrid memory subsystems may] to implement energy efficiency improvements, which may provide a highly energy efficient memory solution that is also high performance and high bandwidth." 

Since Apple is a notebook-centric PC maker, it is not surprising that it is focusing on the energy efficiencies provided by hybrid memory subsystems. Meanwhile, it is obvious that the other advantages include more performance and capacity.  

The patent mostly covers various physical implementations of several hybrid memory subsystems comprised of cache DRAM and main DRAM interconnected using numerous technologies. For example, one layout includes through-silicon vias (TSVs) that pass through a stack of main DRAM memory chips, which allows installing a fairly significant amount of DRAM capacity while not using a lot of space. Meanwhile, the patent doesn't cover how operating systems or software could take advantage of a hybrid memory subsystem.

An Important Technology

Apple, just like any other high-tech company, files applications for hundreds of patents every year. Some of the patent applications become real products, but others do not. But Apple's patent application US10573368B2 was filed in 2016, and then the company filed appropriate applications in many jurisdictions all around the world, which implies that the architecture is expected to be used worldwide (i.e., in many Apple products).  

"This patent was filed in a wide array of jurisdictions, including the EP (European Patent Org.), U.S., China, and Japan," said Kerry Creeron, an attorney with the firm of Banner & Witcoff. "Such a wide filing strategy is expensive and typically only makes sense if IP protection is important." 

Another interesting thing to note about the patent is the name of one of the inventors; Sukalpa Biswa moved to Apple from PA Semi, so he must have a lot of experience with CPU design.

Not An Apple Exclusive

Apple's hybrid memory subsystems certainly look very impressive, but it's unlikely that this type of hybrid technology will be exclusive to Apple. 

Intel's modern Xeon Scalable processors can work both with conventional DDR4 SDRAM as well as Optane Memory (3D XPoint) modules, essentially supporting a hybrid memory subsystem. Intel's next-generation Xeon Scalable codenamed Sapphire Rapids processor will also support HBM, so its hybrid memory subsystem will likely look like the one Apple has patented.