Join for quick access. Enter your email below and we'll send confirmation plus sign you up to our newsletter.
By submitting your information, you confirm you are aged 16 or over,
have read our
Privacy Policy
and agree to the
Terms & Conditions. Geographical rules apply.
Stay On the Cutting Edge: Get the Tom's Hardware Newsletter
Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.
You are now subscribed
Your newsletter sign-up was successful
Join the club
Get full access to premium articles, exclusive features and a growing list of member rewards.
An account already exists for this email address, please log in.
New HBF spec outlines tech that can give GPUs terabytes of extra memory — Sandisk and SK hynix unveil spec with up to 16-Hi NAND stacks, 3 TB/s bandwidth, UCIe
Sandisk and SK hynix on Tuesday formally introduced the High Bandwidth Flash (HBF) specification, their jointly developed storage technology that promises to bring together the non-volatility of 3D NAND and the performance of High Bandwidth Memory (HBM), which will be handy for AI inference systems. The specification was released through the Open Compute Project (OCP), so it will be an open standard rather than a proprietary interface.
The initial specification defines HBF packages with capacities of up to 512GB using either 8-Hi or 16-Hi NAND die stacks, though these will not be standard 3D NAND stacks, but rather specialized devices with a fast interface. In fact, Sandisk once called them HBF core dies rather than 3D NAND die stacks.
Performance of HBF is divided into three bandwidth grades ranging from approximately 0.4 TB/s to 3.0 TB/s (though we are not sure whether this figure describes the full HBF subsystem or per-package bandwidth). Such a huge performance range implies that Sandisk and SK hynix expect HBF to have a multi-year roadmap featuring multiple implementations and generations of HBF. It is noteworthy that the most capable implementation of HBF (3 TB/s) is set to beat the memory bandwidth of a single HBM4 memory stack (2 TB/s), though it will be unlikely to beat HBM4 when it comes to latency.
Interestingly, SK hynix claims that HBF uses the Universal Chiplet Interconnect Express (UCIe) standard to simplify integration with heterogeneous computing platforms, whereas Sandisk claims that HBF is set to adopt the 'xPU-HBF' interface, which could be its definition of UCIe implemented by companies like Broadcom or Marvell.
In addition to capacity and performance targets, the specification establishes electrical and interface characteristics, packaging and reliability guidelines for stacked HBF devices, as well as software I/O requirements. For now, these specifications are not officially published by the OCP.
(Image credit: SanDisk)
Extracting 400 GB/s of bandwidth from a single 512GB HBF package is not a trivial task. To enable such a package, Sandisk once planned to use 16 HBF core dies that feature many, many arrays that can be accessed concurrently using dedicated read/write paths. Meanwhile, it is possible to reach over 400 GB/s of bandwidth per package using a single UCIe interface that runs at up to 64 GT/s and features 64 lanes. Yet, this means that the HBF base die will be a fairly complex piece of silicon.
Sandisk and SK hynix position HBF as a new memory tier for AI inference workloads by combining near-memory bandwidth with the higher capacity and non-volatility of NAND flash. The technology is aimed at workloads that require substantially larger memory pools close to compute than HBM alone can economically provide. For example, while the maximum capacity of an HBM4 stack is 64GB, an HBF stack can provide up to 512GB. Even at a lower bandwidth, such memory can be useful for inference workloads.
Stay On the Cutting Edge: Get the Tom's Hardware Newsletter
Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.
Arguably the biggest question about HBF is who is going to adopt the technology? Since Sandisk and SK hynix announced plans to collaborate on defining the HBF specification in 2025, only Google and Tenstorrent have expressed interest in participating in the HBF consortium. Meanwhile, AMD, Broadcom, Intel, Nvidia, Marvell, Micron, Qualcomm, Samsung, and Western Digital have so far expressed no interest in HBF.
Anton Shilov is a contributing writer at Tom’s Hardware. Over the past couple of decades, he has covered everything from CPUs and GPUs to supercomputers and from modern process technologies and latest fab tools to high-tech industry trends.
