According to a report, HBM4 memory was not expected to reach volume production any earlier than the end of Q1 2026, according TrendForce. However, an Nvidia spokesperson told Tom's Hardware Premium: "Our HBM4 memory partners remain on track for production shipments in the second half of this year as originally planned", disputing the report that production has slid as a result of a redesigned spec.

While Nvidia denies the claim that HBM4 production has slid, if the report holds, it may have stemmed from two converging factors: Nvidia’s decision to revise its memory requirements upward for its next-gen Rubin GPU platform, and the company’s short-term strategy to aggressively extend shipments of its current Blackwell architecture. All three major HBM suppliers were reportedly forced to redesign their HBM4 products to meet the new specifications, pushing mass manufacturing back by at least one quarter.

This shift keeps HBM3 and HBM3e as the prevailing standards across AI and high-performance GPU deployments through at least Q1 2026. Samsung may be first to qualify, given that it has reportedly passed Nvidia’s qualification tests, but SK hynix is still expected to maintain the majority share as the primary supplier to Nvidia. Micron, a more recent entrant in the HBM market, has already begun sampling 11 Gbps-class HBM4 parts, but is still building out volume readiness.

Aligning with internal cadence

The changes also realign Nvidia’s internal cadence, with the Rubin GPU line, which will use HBM4 exclusively, now set for volume availability in the second half of 2026. Rubin’s target specs are the main reason HBM4 was reportedly behind schedule, though Nvidia insists that the timeline remains on track.

According to TrendForce, Nvidia pushed for speeds higher than 11 Gbps per pin, which required all three vendors to retool their designs. Each HBM4 stack carries 2,048 data I/Os, so a 13 Gbps upgrade pushes aggregate per-stack bandwidth to over 2.6 TB/s. That level of throughput places new stress on base die logic and thermals.

SK hynix and Samsung began delivering engineering samples to Nvidia in late 2025, but with Nvidia allegedly demanding last-minute spec changes, those parts will now be insufficient for Rubin's requirements. Samsung is said to have a slight edge on qualification, due to its newer base die process and integration stack. Still, SK hynix is expected to retain the bulk of Nvidia’s business into 2026, given its existing allocation contracts.

This is not just about specs, however, but also Nvidia's broader control over memory ecosystems. Its sheer amount of purchasing power gives the company the leverage to shape JEDEC standards, dictate packaging needs, and pace supplier production cycles. NVIDIA accounts for over 60% of global HBM consumption in 2024, according to Morgan Stanley, and TSMC’s advanced packaging nodes — especially CoWoS — are already fully committed to Nvidia’s Blackwell and Grace Hopper-class parts. Moving to Rubin and HBM4 implies even greater substrate complexity, requiring further capacity expansion at both the foundry and substrate partner levels

Nvidia confirmed at CES that Rubin silicon is already in full production; however, system-level availability won’t follow until much later in the year, which may be due to memory and interconnect bottlenecks. Rubin will ship with up to 288 GB of HBM4 and will rely on revised versions of Nvidia’s NVLink interconnect, optimized for the increased bandwidth profile. Early Rubin configurations are expected to pair with Grace CPUs via a refreshed NVLink architecture, allowing up to 900 GB/s of coherent bandwidth per link.

HBM suppliers recalibrate for 2026 volumes

The delay offers both a challenge and a reprieve for vendors. The challenge lies in redesigning HBM4 dies to meet Nvidia’s updated timing and signal integrity requirements, but the reprieve comes in the form of extra runway — most HBM3 and HBM3e nodes are now sold out through late 2026, and the additional time allows vendors to optimize yields and scale packaging operations.

SK hynix will continue shipping the lion’s share of HBM volume throughout the quarter; it has a deep allocation pipeline with Nvidia and has committed the majority of its high-end DRAM lines to HBM production. Samsung, which initially expected HBM4 to arrive by 2025, has increased its cadence significantly and is now expected to reach high-volume HBM4 qualification sometime in Q2 2026. Micron is simultaneously ramping 11 Gbps-class HBM4 and sampling early HBM4E products with up to 16 dies and extended bandwidth ceilings.

JEDEC ratified the HBM4 standard in April 2025, specifying 2048-bit interfaces and per-pin speeds beginning at 6.4 Gbps, scaling up to over 12 Gbps. With Rubin and other high-performance AI accelerators now targeting 13 Gbps or higher, vendors are pushing the upper limits of thermal and power envelopes. Micron has said that it expects 64GB stacks to become common with HBM4E sometime after late 2027. Meanwhile, each Rubin GPU package on the NVL72 will have eight stacks of HBM4 memory delivering 288GB of capacity and 22 TB/s of bandwidth.

The delay also allows some equilibrium to form in packaging. TSMC’s CoWoS-L capacity has been under severe pressure due to Nvidia’s Blackwell and AMD’s MI300 ramp. By spacing out Rubin’s arrival, Nvidia is implicitly giving its suppliers time to expand interposer and bumping operations without triggering yield degradation or substrate shortages.

Implications for AMD, Intel, and downstream designs

Nvidia may exert outsized influence over HBM4 production, but it is not the only company exposed to delays. AMD has leaned heavily on HBM3 and HBM3e across the MI300 and MI350, with the upcoming MI400 designed around 432 GB of HBM4. If HBM4 volume production slips further, it would not just reshape Nvidia’s cadence but also place direct pressure on AMD’s MI400 rollout.

Intel’s Habana Gaudi line is still anchored on HBM2e, with 128 GB per accelerator in the Gaudi 3. The company is known to be planning a Gaudi 4-class device codenamed “Jaguar Shores”, likely for release in 2027, using HBM4E, so its current timeline remains unaffected by Nvidia’s spec shift. Intel’s packaging flows for AI silicon are distinct from Nvidia’s, and its later entry into HBM4 adoption may allow it to bypass early yield limitations.

The real downstream impact may surface in how HBM availability shapes product segmentation. Nvidia’s highest-end Blackwell and Rubin GPUs will continue to monopolize advanced memory stacks and interposer capacity, effectively limiting HBM4 to premium datacenter SKUs. There is currently no sign of HBM4 migrating into consumer GPUs or gaming cards, given the absence of any new GPU announcements at CES. Even as GDDR7 supply tightens, Nvidia has not shown any intent to merge AI and GeForce memory standards.

With Rubin silicon now in full production and mass memory availability locked to late Q1 or early Q2 2026, the HBM race continues — just a quarter later than planned.

Update 01/13/2026 6:51am PT: Article amended to add quote from Nvidia spokesperson.

High Bandwidth Memory (HBM) is the unsung hero behind the AI revolution. As the industry seeks to extract the most performance from frontier AI models, HBM powers the world's fastest GPUs and AI accelerators by keeping the intense computational engines fed with data at breakneck speed. This critical technology has rapidly matured over the past several years, and lately the pace of innovation has quickened, as industry behemoths like Nvidia and AMD look to facilitate the creation of more advanced artificial intelligence models.

All three major DRAM makers (Micron, Samsung, and SK hynix) have started volume production of 8-Hi HBM3E stacks, the newest form of the technology. And more robust and powerful forms of HBM memory are already in development.

Unfortunately, shortages of sophisticated HBM memory have hampered the supply of AI GPUs, which has manufacturers racing to add more production capacity to address any shortfalls. Meanwhile, the development of next-gen HBM products continues apace, hand-in-hand with new performance-enhancing technologies, which will power the next wave of AI accelerators.

Let's take a look at what's next on the HBM roadmaps for Micron, Samsung, and SK hynix based on official information and other sources.

Speeds and feeds

Memory bandwidth is a critical bottleneck in AI systems — AI models, particularly deep learning models, ingest tremendous amounts of data as they chew through workloads. But most forms of modern memory can't satiate AI's ravenous appetite for more data. That's where HBM steps in.

Traditional memory based on DDR, LPDDR, or GDDR uses 128–bit to 512-bit wide interfaces and employs high data transfer rates to provide bandwidth from 100 GB/s to 2 TB/s.

Unlike traditional memory, high-bandwidth memory (HBM) uses a very wide interface—1024 bits in the case of HBM2 and HBM3, or 2048 bits with HBM4—which multiplies bandwidth up to 4 TB/s—8 TB/s. Given the bandwidth-constrained nature of high-intensity parallel computation in GPUs and accelerators, this increased bandwidth translates directly to more performance.

However, the wide interface makes HBM difficult to produce, requiring multiple specialized DRAM devices interconnected using through-silicon vias (TSVs) stacked on top of a base die. HBM makers also vary the number of stacked memory dies to increase capacity, denoted by terminology such as 8-Hi for eight stacked dies, or 12-Hi for 12 stacked dies.

Due to its huge bandwidth, HBM is, and will continue to be, the de facto memory standard for AI systems, HPC ASICs, and GPUs.

12-Hi HBM3E is almost here

Today's highest-end AI accelerators — including Nvidia's H200 (141 GB), B200 (192 GB), and AMD's Instinct MI300X (192 GB) — use 24 GB 8-Hi HBM3E stacks based on 24 Gb DRAM devices. The next step for the industry is to adopt higher-capacity 36 GB 12-Hi HBM3E packages featuring 24 Gb memory dies. These will be used by Nvidia's upcoming B300-series and AMD's next-gen MI325X AI accelerators.

SK hynix has begun mass production of 36 GB 12-Hi HBM3E chips, whereas Micron has been sampling similar products since September, with mass production of the new packages understood to be imminent.

Samsung, on the other hand, was late to the party with its 8-Hi HBM3E certification, and its 12-Hi HBM3E dies also suffered from a slight delay. Samsung's delay is likely caused by sticking with its 1α fabrication technology, unlike Micron and SK hynix, which use a 1ß (5th Gen, 10nm-class) DRAM process to make their HBM3E DRAM ICs. By the time Nvidia's B300 enters mass production, Samsung will likely be able to compete with 12-Hi HBM3E 36 Gb offerings of its own.

HBM4: 2048-bit I/O and up to 16 layers

While manufacturers are still wrapping their hands around the upcoming HBM3E rollout, HBM4 and HBM4E are both already on the horizon.

The preliminary HBM4 specification (unveiled in July 2024) introduces a wider 2048-bit interface for HBM stacks. It also specifies 24 Gb and 32 Gb DRAM layers at up to 6.40 GT/s. The spec supports 4-Hi, 8-Hi, 12-Hi, and 16-Hi configurations, ensuring greater flexibility and potentially enabling even larger 64 Gb HBM4 packages.

Meanwhile, HBM4E may also boast high interface speeds of around 9 GT/s, as Rambus' HBM4 memory controller IP exceeds the announced capabilities of HBM4's JDEC-standard 6.40 GT/s speeds.

With HBM4E, memory makers will be able to customize base dies of packages (which SK hynix envisioned in early 2024) by adding additional functions, which could potentially extend to enhanced caches, custom interface protocols, and more.

All three leading memory manufacturers, Micron, Samsung, and SK hynix, have confirmed their intentions to produce HBM4 and HBM4E memory, but their overall rollout strategies may differ.

HBM4: 16-Hi stacks, but no 32 Gb devices on horizon

For now, none of the prominent DRAM producers have HBM4 or HBM4E stacks based on 32 Gb memory devices on their roadmaps. As such, all HBM4 and HBM4E products are expected to utilize smaller 24 Gb DRAM dies when they first roll out.

Micron is expected to keep using its proven 1ß (5th Gen, 10nm-class) process technology to make 24 Gb memory ICs for HBM4 stacks; however, Samsung plans to transition to 24 Gb DRAM dies made on its 1γ (6th Gen, 10nm-class) process with HBM4 and HBM4E. This will likely offer Samsung substantial performance, power efficiency, and cost advantages. SK hynix also intends to use 1ß for HBM4 DRAM ICs and may transition to a 1γ process for the HBM4E offering.

When it comes to layers, Micron lists 12-Hi and 16-Hi versions of its HBM4 and HBM4E offerings, whereas Samsung and SK hynix may go straight to 16-Hi HBM4 stacks. With the standard appearing to be 24 Gb DRAM devices and 12-Hi or 16-Hi stacks, HBM4 will increase per-package capacity to 48 Gb, a noticeable leap over HBM3E's 36Gb.

It's possible that by the time HBM4E finally arrives, the number of supported layers may exceed 16, with rumors that South Korean manufacturers could adopt a 20-layer design (which should be taken with a grain of salt). By then, DRAM makers may have also adopted a 32 Gb package for high-bandwidth offerings.

HBM4 & HBM4E production nodes

It should be no surprise that HBM4 and HBM4E memory stacks will rely on base dies produced by logic manufacturers using logic process technologies, thus boosting transfer speeds and signal integrity. TSMC and SK hynix were the first to disclose that they plan to use TSMC’s 12FFC+ and N5 base dies for HBM4. It's likely that Micron will also use TSMC's base dies (as the two companies are partners), though this has not been officially confirmed.

Samsung is expected to use its own Samsung Foundry nodes for HBM4 and HBM4E base dies. There is still uncertainty around the exact node it will employ, though it is reasonable to expect similar process technologies to TSMC's 12FFC+ and N5 processes.

HBM4 is coming in 2026 & HBM4E expected a year later

Samsung and SK hynix are rumored to introduce their first HBM4 offerings around Q3 2025, whereas Micron is projected to follow in Q4 2025. In both cases, 'introductions' likely mean the delivery of the first working samples to partners like AMD and Nvidia, not high-volume manufacturing.

Considering that the mass production of actual processors that support HBM4 is not expected until 2026, Micron's slight delay does not seem like a major issue.

Curiously, Micron began discussing HBM4E around a year before the planned mass production of HBM4 was set to commence. Typically, 'extended' variants of HBM specifications are introduced years after the original standard. According to Micron's official roadmap, HBM4E is set to arrive in late 2027.

HBM4E is likely to be used in the generation after the release of Nvidia's upcoming Rubin architecture and AMD's MI400 AI accelerator. Both are slated to support HBM4 in 2026.

If the industry requires customizable memory, HBM4E might land earlier than expected, but don't hold your breath — HBM development is notoriously challenging.

In a historic shake-up of the global memory market, SK hynix has overtaken Samsung Electronics to become the world’s top memory maker by revenue, marking a pivotal shift in an industry long dominated by its larger Korean rival. The moment comes not just as a financial milestone, but as a broader signal of how the memory business is being reshaped by the explosive demand for AI.

The news first emerged through multiple South Korean financial publications, with industry tracking firm Counterpoint and Omidia confirming SK hynix’s rise to the number one spot in Q2 2025. The company pulled in $9.66 billion in memory revenue, edging out Samsung’s $8.94 billion and securing a 36.2% market share compared to Samsung’s 33.5%. This marks the first time SK hynix has surpassed Samsung since its founding in 1983.

At the heart of this leap lies SK hynix’s early and aggressive move into high-bandwidth memory (HBM), the ultra-fast DRAM used in AI accelerators like Nvidia's B300. While Samsung was once seen as the leader in memory innovation, it now seems that SK Hynix has become Nvidia’s primary HBM supplier. The firm’s HBM3 and next-gen HBM3E modules are integral to the compute power behind large language models and generative AI workloads. Not only that, but Nvidia started with Samsung's GDDR7 in their mainstream RTX 50-series GPUs, only to bring in SK hynix's chips later.

SK hynix’s success is also visible in its operating profits. The company posted 2.6 trillion KRW (~$1.88 billion) in Q2 operating profit—its highest in two and a half years—despite lingering weakness in traditional DRAM and NAND markets. Analysts say this divergence highlights a fundamental realignment in the memory industry: AI-centric demand now drives profitability more than PC or smartphone shipments ever did.

Samsung’s semiconductor business is coming off a rough quarter, with profits plunging nearly 94% year-over-year. It’s the weakest the chip division has looked in six quarters, and the culprit isn’t hard to identify. Between U.S. export restrictions limiting the sale of advanced chips to China, and persistent inventory corrections, the company’s Device Solutions division recorded just 400 billion won ($287 million) in profit for Q2 2025—a steep fall from the 6.5 trillion won ($4.67 billion) it pulled in the same period last year. However, Samsung is betting big that artificial intelligence will flip that narrative by the end of the year.

The recovery, as Samsung sees it, lies in HBM3E—the latest generation of High Bandwidth Memory used to feed data to AI accelerators at blazing speeds. According to a ZDNet Korea report, the company is actively working to lower the production costs of HBM3E in an effort to court Nvidia, which has so far leaned heavily on SK hynix for its AI GPU supply chain.

Samsung’s strategy is simple: make HBM3E memory more affordable and available than anyone else and become indispensable to the future of AI computing. For context HBM3E (High Bandwidth Memory Gen 3E) is critical in modern AI accelerators, especially for large language model training where bandwidth and capacity bottlenecks limit performance. It’s the memory found on Nvidia’s top-end AI GPUs like the B300.

AMD unveiled its new MI350X and MI355X GPUs for AI workloads here at its Advancing AI 2025 event in San Jose, California, claiming the new accelerators offer a 3X performance boost over the prior-gen MI300X, positioning the company to improve its competitive footing against its market-leading rival, Nvidia. AMD claims it beats Nvidia in like-for-like inference benchmarks by up to 1.3X and leads by up to 1.13X in select training workloads. 

AMD also claims a 4X increase in "AI compute performance" compared to prior-generation AMD MI300X models and a 35X increase in inference performance, largely achieved by transitioning to the CDNA 4 architecture and utilizing a smaller, more advanced process node for the compute chiplets. AMD's production MI350 platforms began shipping last month. 

These two MI300 Series AI GPUs will power AMD rack-level solutions for the remainder of the year and into 2026 as the company builds to its MI400 rollout. 

The MI350X and MI355X share an identical underlying design, featuring up to 288GB of HBM3E memory, up to 8 TB/s of memory bandwidth, and new support for the FP4 and FP6 data types. However, the MI350X is geared for air-cooled solutions with a lower Total Board Power (TBP), while the MI355X pushes power consumption up a notch for liquid-cooled systems geared for the highest performance possible.

AMD will not release an APU version of this chip like it did with last generation's MI300A, which featured both CPU and GPU cores on a single die. In contrast, this generation will have GPU-only designs. 

AMD's MI355X comes with 1.6 times the HBM3E memory capacity of Nvidia's competing GB200 and B200 GPUs, but delivers the same 8TB/s of memory bandwidth. AMD claims a 2X advantage in peak FP64 / FP32 over Nvidia's chips, which isn't surprising given Nvidia's optimization focus on the more AI-friendly lower-precision formats. Notably, MI350's FP64 matrix performance has been halved compared to MI300X, though vector performance drops by roughly 4% gen-over-gen.

As we move down to lower-precision formats, such as FP16, FP8, and FP4, you can see that AMD generally matches or slightly exceeds the Nvidia comparables. One notable standout is FP6 performance, which runs at FP4 rates, which AMD sees as a differentiating feature. 

As we've also seen with Nvidia's competing chips, the new design and heightened performance also come with increased power consumption, which tops out at a 1,400W Total Board Power (TBP) for the liquid-cooled high-performance MI355X model. That's a marked increase from the MI300X's 750W and the MI325X's 1,000W thermal envelopes.

AMD says this increase in performance density allows its customers to cram more performance into a single rack, thereby decreasing the all-important performance-per-TCO (total cost of ownership) metric, which quantifies performance-per-dollar at the rack level.

The new chips feature numerous advancements in performance, but the fundamental design principles of merging 3D and 2.5D packaging technology remain unchanged, with the former being used to fuse the Accelerator Compute Dies (XCD) with the I/O Dies (IOD), while the latter is used to connect the IOD to each other and the 12-Hi HBM3E stacks. 

The chip features eight total XCD chiplets, each with 32 compute units (CU) enabled, for a total of 256 CU (AMD has four CU in reserve per XCD to improve yield; these are disabled as needed). The XCD's transition from 5nm with the prior generation to dies fabbed on TSMC's N3P process node for the MI350 series. The total chip weilds a whopping 185 billion transistors, a 21% increase in the transistor budget over the prior generation's 153 billion. 

Additionally, while the I/O Die (IOD) remains on the N6 process node, AMD has reduced the IOD from four tiles to two to simplify the design. This reorganization enabled AMD to double the Infinity Fabric bus width, improving bi-sectional bandwidth to up to 5.5 TB/s, while also reducing power consumption by lowering the bus frequency and voltage. This reduces uncore power requirements, allowing more power to be spent on compute. 

As with the MI300 series, the Infinity Cache (memory-side cache) sits in front of the HBM3E (32MB of cache per HBM stack).  

The completed processor connects to the host via a PCIe 5.0 x16 interface and presents itself as a single logical device to the host. The GPU communicates with other chips via seven Infinity Fabric links, providing a total of 1,075 GB/s of throughput. 

Both the MI350X and MI355X come in an OAM form factor and drop into the standardized UBB form factor servers (OCP spec), the same as prior-gen MI300X. AMD says this speeds time to deployment.

The chips communicate with one another via an all-to-all topology with eight accelerators per node communicating across 153.6 GB/s bi-directional Infinity Fabric links. Each node is powered by two of AMD's fifth-gen EPYC 'Turin' chips. 

AMD supports all forms of networking, but positions its new Pollara Ultra Ethernet Consortium-capable NICs (UEC) as an optimum scale-out solution, while the Ultra Accelerator Link (UAL) interconnect is employed for scale-up networking. 

AMD offers both Direct Liquid Cooling (DLC) and Air Cooled (AC) racks. The DLC racks feature 128 MI355X GPUs and 36TB of HBM3E, thanks to the increased density provided by the liquid cooling subsystem, which enables the use of smaller node form factors. The AC solutions top out at 64 GPUs and 18TB of HBM3E, utilizing larger nodes to dissipate the thermal load via air cooling. 

AMD has tremendously increased its focus on unleashing the power of rack-scale architectures, a glaring deficiency relative to Nvidia. AMD has undertaken a series of acquisitions and developed a strong and expanding roster of partner OEMs to further its goals. 

As one would expect, AMD shared some of its performance projections and benchmarks against not only its own prior-gen systems, but also against Nvidia comparables. As always, take vendor-provided benchmarks with a grain of salt. We have included the test notes below for your perusal.

AMD claims an eight-GPU MI355X setup ranges from 1.3X faster with four MI355X vs four DGX GB200 in Llama 3.1 405B, to 1.2X faster with eight MI355X against an eight-GPU B200 HGX config in inference in DeepSeek R1, and equivalent performance in Llama 3.1 405B (all tested at FP4). 

AMD also says MI355X is also competitive with Nvidia's B200 and GB200 in training workloads, though here it highlights either parity or a slight 1.13X performance advantage in a range of LLama models. 

AMD says the MI355X provides up to 4.2X more performance over the MI300X in AI Agent and Chatbot workloads, along with strong gains ranging from 2.6X to 3.8X in content generation, summarization, and conversational AI work. Other generational highlights include a 3X generational improvement in DeepSeek R1 and a 3.3X gain in Llama 4 Maverick.  

We will update this article as AMD shares more details during its keynote, which is occurring now. 

The Nvidia Blackwell Ultra B300 data center GPU was announced today during CEO Jensen Huang's keynote at GTC 2025 in San Jose, CA. Offering 50% more memory and FP4 compute than the existing B200 solution, it raises the stakes in the race to faster and more capable AI models yet again. Nvidia says it's "built for the age of reasoning," referencing more sophisticated AI LLMs like DeepSeek R1 that do more than just regurgitate previously digested information.

Naturally, Blackwell Ultra B300 isn't just about a single GPU. Along with the base B300 building block, there will be new B300 NVL16 server rack solutions, a GB300 DGX Station, and GB300 NV72L full rack solutions. Put eight NV72L racks together, and you get the full Blackwell Ultra DGX SuperPOD: 288 Grace CPUs, 576 Blackwell Utlra GPUs, 300TB of HBM3e memory, and 11.5 ExaFLOPS of FP4. These can be linked together in supercomputer solutions that Nvidia classifies as "AI factories."

While Nvidia says that Blackwell Ultra will have 1.5X more dense FP4 compute, what isn't clear is whether other compute have scaled similarly. We would expect that to be the case, but it's possible Nvidia has done more than simply enabling more SMs, boosting clocks, and increasing the capacity of the HBM3e stacks. Clocks may be slightly slower in FP8 or FP16 modes, for example. But here are the core specs that we have, with some inference of other data (indicated by question marks).

We asked for some clarification on the performance and details for Blackwell Ultra B300 and were told: "Blackwell Ultra GPUs (in GB300 and B300) are different chips than Blackwell GPUs (GB200 and B200). Blackwell Ultra GPUs are designed to meet the demand for test-time scaling inference with a 1.5X increase in the FP4 compute." Does that mean B300 is a physically larger chip to fit more tensor cores into the package? That seems to be the case, but we're awaiting further details.

What's clear is that the new B300 GPUs will offer significantly more computational throughput than the B200. Having 50% more on-package memory will enable even larger AI models with more parameters, and the accompanying compute will certainly help.

Nvidia gave some examples of the potential performance, though these were compared to Hopper, so that muddies the waters. We'd like to see comparisons between B200 and B300 in similar configurations — with the same number of GPUs, specifically. But that's not what we have.

By leveraging FP4 instructions, using B300 alongside its new Dynamo software library to help with serving reasoning models like DeepSeek, Nvidia says an NV72L rack can deliver 30X more inference performance than a similar Hopper configuration. That figure naturally derives from improvements to multiple areas of the product stack, so the faster NVLink, increased memory, added compute, and FP4 all factor into the equation.

In a related example, Blackwell Ultra can deliver up to 1,000 tokens/second with the DeepSeek R1-671B model, and it can do so faster. Hopper, meanwhile, only offers up to 100 tokens/second. So, there's a 10X increase in throughput, cutting the time to service a larger query from 1.5 minutes down to 10 seconds.

The B300 products should begin shipping before the end of the year, sometime in the second half of the year. Presumably, there won't be any packaging snafus this time, and things won't be delayed, though Nvidia does note that it made $11 billion in revenue from Blackwell B200/B100 last fiscal year. It's a safe bet to say it expects to dramatically increase that figure for the coming year.

SanDisk on Wednesday introduced an interesting new memory that could wed the capacity of 3D NAND and the extreme bandwidth enabled by high bandwidth memory (HBM). SanDisk's high-bandwidth flash (HBF) memory enables access to multiple high-capacity 3D NAND arrays in parallel, thus providing plenty of bandwidth and capacity. The company positions HBF as a solution for AI inference applications that require high bandwidth and capacity coupled with low power requirements. The first-generation HBF can enable up to 4TB of VRAM capacity on a GPU, and more capacity in future revisions. SanDisk also foresees this tech making its way to cellphones and other types of devices. The company hasn't announced a release date yet.

"We are calling it the HBF technology to augment HBM memory for AI inference workloads," said Alper Ilkbahar, memory technology chief at SanDisk. "We are going to match the bandwidth of HBM memory while delivering 8 to 16 times capacity at a similar cost point."

Conceptually, HBF is similar to HBM. It stacks multiple high-capacity, high-performance flash core dies interconnected using through-silicon vias (TSVs) on top of a logic die that can access flash arrays (or rather flash sub-arrays) in parallel. The underlying architecture of HBF is SanDisk's BICS 3D NAND using the CMOS directly bonded to Array (CBA) design that bonds a 3D NAND memory array on top of an I/O die made using logic process technology. That logic may be a key to enabling HBF.

"We challenged our engineers and said, what else could you do with this power of scaling," said Alper Ilkbahar. "The answer they came up with […] was moving to an architecture where we divide up this massive array into many, many arrays and access each of these arrays in parallel. When you do that, you get massive amounts of bandwidth.  Now, what can we build with this? We are going to build high bandwidth flash."

Traditional NAND die designs often treat the core NAND flash memory array as planes, pages, and blocks. A block is the smallest erasable area, and a page is the smallest writable area. HBF seems to break the die into 'many, many arrays' so they can be accessed concurrently. Each sub-array (with its own pages and blocks) presumably has its own dedicated read/write path. While this resembles how multi-plane NAND devices work, the HBF concept seems to go far beyond them.

For now, SanDisk says that its 1st-Gen HBF will use 16 HBF core dies. To enable such devices, SanDisk says that it has invented a proprietary stacking technology that features minimal warpage to enable stacking 16 HBF core dies, and a logic die that can simultaneously access data from multiple HBF core dies. The complexity of logic that can handle hundreds or thousands of concurrent data streams should be higher than that of a typical SSD controller.

Unfortunately, SanDisk does not disclose the actual performance numbers of its HBF products, so we can only wonder whether HBF matches the per-stack performance of the original HBM (~ 128 GB/s) or the shiny new HBM3E, which provides 1 TB/s per stack in the case of Nvidia's B200.

The only thing we know from a SanDisk-provided example is that eight HBF stacks feature 4 TB of NAND memory, so each stack can store 512 GB (21x more than one 8-Hi HBM3E stack that has a capacity of 24 GB). A 16-Hi 512 GB HBF stack means that each HBF core die is a 256 Gb 3D NAND device with some complex logic enabling die-level parallelism. Funneling hundreds of gigabytes of data per second from 16 3D NAND ICs is still quite a big deal, and we can only wonder how SanDisk can achieve that.

What we are sure about is that HBF will never match DRAM in per-bit latency, which is why SanDisk stresses that HBF products are aimed at read-intensive, high-throughput applications, such as big AI inference datasets. For many AI inference tasks, the critical factor is high throughput at a feasible cost rather than the ultra-low latency that HBM (or other types of DRAM) provides. So, while HBF may not replace HBM any time soon, it might occupy a spot on the market that requires high-capacity, high-bandwidth, NAND-like cost but not ultra-low latency. To simplify the transition from HBM, HBF has the same electrical interface with some protocol changes, though HBF is not drop-in compatible with HBM.

" We have tried to make it as close as possible mechanically and electrically to the HBM, but there are going to be minor protocol changes required that need to be enabled at the host devices," said Ilkbahar.

SanDisk didn't touch on write endurance. NAND has a finite lifespan that can only tolerate a certain number of writes. While SLC and pSLC technologies offer higher endurance than the TLC and QLC NAND used in consumer SSDs, this comes at the expense of capacity and adds cost. NAND is also typically written to at block granularity, whereas memory is addressable at the cache line level (i.e. typically 128KB for NAND blocks versus 32 bytes for a cache line). That's another key challenge.

SanDisk has a vision of how its HBF will evolve over three generations. Nonetheless, for now, SanDisk's HBF is largely a work in progress. SanDisk wants HBF to become an open standard with an open ecosystem, so it is forming a technical advisory board consisting of 'industry luminaries and partners.

Leading South Korean memory manufacturer SK hynix announced that it will showcase a suite of advanced memory solutions tailored for artificial intelligence (AI) applications at this year's Consumer Electronics Show (CES) in Las Vegas.

Building upon its 12-layer High Bandwidth Memory (HBM) technology, the company will display samples of its latest 16-layer HBM3E products, officially announced in November 2024. This advancement employs advanced MR-MUF processes to enhance thermal performance and mitigate chip warping, achieving industry-leading results.

With capacities of 48GB (3GB per individual die) per stack, the increased density will allow AI accelerators to utilize up to 384GB of HBM3E memory in an 8-stack configuration. The 16-layer HBM3E is designed to significantly boost AI learning by up to 18% and inference performance by up to 32% compared to the 12-layer version.

Nvidia's next-gen Rubin chips are slated for mass production later next year, thus the existence of HBM3E could be shortlived, as the new upcoming Nvidia chips will be based on HBM4. That shouldn't be a concern, though, as reports indicate that SK hynix achieved its tape-out phase in October 2024.

Addressing the escalating demand for high-capacity storage in AI data centers, SK hynix will also introduce new SSD solutions for enterprise users, including the 122TB 'D5-P5336' enterprise SSD, developed by its subsidiary Solidigm. This model is said to boast the highest capacity currently available in its category and is poised to set new standards in data storage solutions.

The memory and storage manufacturer will also talk about Compute Express Link (CXL) and Processing-In-Memory (PIM) technologies, which are said to be pivotal to the next generation of data center infrastructures. Modularized solutions like the CMM-Ax and AiMX will be featured, with the CMM-Ax being hailed as a groundbreaking solution that combines the scalability of CXL with computational capabilities, boosting performance and energy efficiency for next-generation server platforms.

With on-device AI becoming a popular trend, SK hynix also has plans to showcase 'LPCAMM2' and 'ZUFS4.0,' designed to enhance data processing speed and power efficiency in edge devices such as PCs and smartphones. These innovations aim to facilitate the integration of AI capabilities directly into consumer electronics, broadening the scope of AI applications.

The company announced last year that it was also working on a range of other products, including PCIe 6.0 SSDs, high-capacity QLC (Quad Level Cell) eSSDs made specifically for AI servers, and UFS 5.0 for mobile devices. SK hynix is also working on an LPCAMM2 module and soldered LPDDR5/6 memory using its 1cnm-node to power laptops and handheld consoles.

Today, Micron Technology has started constructing its multi-billion-dollar packaging facility for high-bandwidth memory (HBM) in Singapore. The company will invest $7 billion in the plant, as it expects demand for HBM3E, HBM4, and HBM4E memory to skyrocket in the coming years amid the AI boom. The facility is set to start operations in 2026.

Micron's packaging facility for high-bandwidth memory (HBM) is located next to Micron's existing fabs in Singapore that produce 3D NAND and DRAM. The new HBM assembly plant will commence production in 2026 and then plans to substantially increase its capacity in 2027. The facility will use advanced AI-driven automation to boost operational efficiency, though the company does not disclose where and how artificial intelligence will be used.

While Micron is leading the industry with premium HBM3E memory, when it comes to HBM market share, the company is still an underdog compared to Samsung and SK hynix. To some degree, this is conditioned by the fact that Micron does not have as vast DRAM manufacturing capacity as its rivals from South Korea (while HBM memory dies take up more capacity than conventional memory ICs). Still, to a certain degree, this can be attributed to the lack of vast HBM assembly capacity.

Micron is gradually increasing its HBM3E output at its existing facilities, hoping to grab a mid-20% HBM market share in mid-2025. However, with the new Singapore assembly facility coming online in 2026, the company hopes to get an even larger chunk of the market.

"As AI adoption proliferates across industries, the demand for advanced memory and storage solutions will continue to increase robustly," said Sanjay Mehrotra, president and CEO of Micron. "With the continued support of the Singapore government, our investment in this HBM advanced packaging facility strengthens our position to address the expanding AI opportunities ahead."

Although the new facility will be tailored for assembling HBM stacks, it can also be used to assemble 3D NAND packages since assembly technologies with through-silicon vias (TSVs) are generally similar.

The project will initially create around 1,400 jobs; the expansion could potentially increase that number to 3,000. These roles will include packaging development, assembly, and testing operations.

"This is Singapore’s first high-bandwidth memory advanced packaging facility, allowing us to contribute to global AI growth," said Png Cheong Boon, Singapore Economic Development Board Chairman. "It expands Singapore’s partnership with Micron and further strengthens the semiconductor ecosystem in Singapore."

At the SK AI Summit 2024, SK hynix CEO took the stage and revealed the industry's first 16-Hi HBM3E memory - beating both Samsung and Micron to the punch. With the development of HBM4 going strong, SK hynix prepared a 16-layer version of its HBM3E offerings to ensure "technological stability" and aims to offer samples as early as next year.

A few weeks ago, SK hynix unveiled a 12-Hi variant of its HBM3E memory - securing contracts from AMD (MI325X) and Nvidia (Blackwell Ultra). Raking in record profits last quarter, SK hynix is in full steam once again as the giant has just announced a 16-layer upgrade to its HBM3E lineup, boasting capacities of 48GB (3GB per individual die) per stack. This increase in density now allows AI accelerators to feature up to 384GB of HBM3E memory in an 8-stack configuration.

SK hynix claims an 18% improvement in training alongside a 32% boost in inference performance. Like its 12-Hi counterpart, the new 16-Hi HBM3E memory incorporates packaging technologies like MR-MUF which connects chips by melting the solder between them. SK hynix expects 16-Hi HBM3E samples to be ready by early 2025. However, this memory could be shortlived as Nvidia's next-gen Rubin chips are slated for mass production later next year and will be based on HBM4.

That's not all as the company is actively working on PCIe 6.0 SSDs, high-capacity QLC (Quad Level Cell) eSSDs aimed at AI servers, and UFS 5.0 for mobile devices. In addition, to power future laptops and even handhelds, SK hynix is developing an LPCAMM2 module and soldered LPDDR5/6 memory using its 1cnm-node. There isn't any mention of CAMM2 modules for desktops, so PC folk will need to wait - at least until CAMM2 adoption matures.

To overcome what SK hynix calls a "memory wall", the memory maker is developing solutions such as Processing Near Memory (PNM), Processing in Memory (PIM), and Computational Storage. Samsung has already demoed its version of PIM - wherein data is processed within the memory so that data doesn't have to move to an external processor.

HBM4 will double the channel width from 1024 bits to 2048 bits while supporting upwards of 16 vertically stacked DRAM dies (16-Hi) - each packing up to 4GB of memory. Those are some monumental upgrades, generation on generation, and should be ample to fulfill the high memory demands of upcoming AI GPUs.

Samsung's HBM4 tape-out is set to advance later this year. On the flip side, reports suggest that SK hynix already achieved its tape-out phase back in October. Following a traditional silicon development lifecycle, expect Nvidia and AMD to receive qualification samples by Q1/Q2 next year.

The SK AI Summit 2024 is being held at COEX Convention Center in Seoul November 4-5. The event is the largest AI symposium in Korea, the company claimed.

SK hynix has started mass production of its 12-Hi HBM3E memory stacks, ahead of its rivals. The new modules feature a 36GB capacity and set the stage for next-generation AI and HPC processors, such as AMD's Instinct MI325X which is due in the fourth quarter, and Nvidia's Blackwell Ultra which is expected to arrive in the second half of next year.   

SK hynix's 12-Hi 36GB HBM3E stacks pack twelve 3GB DRAM layers and feature a data transfer rate of 9.6 GT/s, thus providing a peak bandwidth of 1.22 TB/s per module. A memory subsystem featuring eight of the company's 12-Hi 36GB HBM3E stacks will thus offer a peak bandwidth of 9.83 TB/s. Real-world products will unlikely use these HBM3E memory devices at their full speed as developers tend to ensure ultimate reliability. We don't doubt that HBM3E memory subsystems will offer higher performance than their predecessors, though. 

Despite packing 50% more memory devices, the new 12-Hi HBM3E memory stacks feature the same z-height as their 8-Hi predecessors. To achieve this, SK hynix made DRAM devices 40% thinner. Also, to avoid structural issues that arise from using ultra-thin vertically stacked DRAMs interconnected using through silicon vias (TSVs), the manufacturer used its mass reflow molded underfill (MR-MUF) process that bonds the dies together all at once and fills the space between them with an improved underfill called liquid Epoxy Molding Compound. As a bonus, EMC also has better thermal conductivity. 

SK hynix is the first company to start mass production of 12-Hi HBM3E memory. While Samsung formally introduced its 12-Hi 36GB HBM3E stacks early this year, it has yet to start mass production of these products. Micron is sampling production-ready 12-Hi HBM3E devices, but it has yet to start high-volume production of these memory stacks. 

SK hynix plans to ship its 12-Hi 36GB HBM3E memory stacks by the end of the year, in time for AMD's Instinct MI325X accelerator for AI and HPC that will carry 244GB of HBM3E memory, and several quarters before Nvidia intends to start shipments of its Blackwell Ultra GPU for AI and HPC applications.  

"SK hynix has once again broken through technological limits demonstrating our industry leadership in AI memory," said Justin Kim, President (Head of AI Infra) at SK hynix. "We will continue our position as the No.1 global AI memory provider as we steadily prepare next-generation memory products to overcome the challenges of the AI era."

Micron previously announced plans to increase its high bandwidth memory (HBM) market share from the 'mid-single digits' today to mid-20% (i.e., ~25%) about a year from now. However, it did not share much detail about how it would ramp up the expansion of its HBM production capacity. Nikkei has now shed some light on the potential plans, and says that Micron will expand production capacity in Taiwan, boost its R&D operations in the U.S., and is even mulling making HBM3E in Malaysia. This is strictly unofficial information, so please take it with a grain of salt. 

HBM is a type of DRAM but it uses different memory devices than commodity memory modules. HBM DRAM dies are higher-capacity and feature a wider interface, which makes them a bit more difficult to produce. After production, each device has to be tested individually to ensure that it meets quality, performance, and power targets. Each HBM module stacks eight or twelve known good HBM dies on top of each other, connects them using through-silicon vias, and places the stack on a base logic die. To boost HBM3E output, Micron needs to increase production of HBM DRAM devices (which can come at the cost of producing fewer commodity DRAMs), ensure it has enough testing capacity for HBM DRAMs, as well as stacking and TSV capacity. 

Micron's largest HBM production site is in Taichung, Taiwan, where it produces DRAM dies, tests them, and then assembles HBM3E stacks. The company is currently adding capacity to its Taiwan operations, according to the source report, which does not detail which capacity is being added. This is certainly assuring (particularly for Nvidia), though Micron's HBM3E memory supply is sold out through most of 2025.

Additionally, Micron is looking at building HBM production capacity in Malaysia, where it already conducts commodity memory chip testing and assembly, Nikkei reports. Micron is unlikely to produce actual DRAM dies in Malaysia, as a memory fab takes years to build and equip. But testing HBM3E dies and maybe even assembling HBM3E in Malaysia is a possibility. Yet again, Micron has to confirm this itself.

Finally, Micron is reportedly also setting up additional test production lines for advanced HBM chips at its headquarters in Boise, Idaho, and expanding R&D facilities in the U.S., possibly to prep for next-generation HBM4 memory in late 2025 – 2026. 

Nvidia CEO Jensen Huang says Samsung’s advanced High Bandwidth Memory chips still aren’t ready for official certification. Nvidia’s sign-off is the last step before Samsung can begin supplying HBM3 and HBM3e components, essential to training Nvidia artificial intelligence (AI) platforms.

SK hynix is currently the primary supplier of HBM3 and HBM3e memory to Nvidia. These chips are important for the fast and efficient training of AI models, including ChatGPT and others. Nvidia is examining HBM chips produced by Samsung and Micron, but it hasn’t yet endorsed their usage. More engineering work is needed, Huang told reporters. However, it isn't entirely certain which engineers have the most work to do - those at Samsung or Nvidia (or a team involving both).

Recent reports suggested that Samsung’s latest HBM modules struggle with excessive heat and power consumption issues. Huang pointed out that the modules haven’t yet failed any qualification tests, but the HBM product isn’t quite ready for deployment. “We just have to do the engineering. It’s just not done,” Huang told reporters at a Tuesday briefing at Computex 2024.

Samsung, on the other hand, denied the accuracy of reports raising concerns over heat and power. According to Samsung, the testing of its most advanced HBM3e memory modules is progressing smoothly. It says its latest HBM products work fine with a wide range of processors, but it doesn’t specifically deny problems with Nvidia processors.

Samsung is still the overall largest producer of memory chips globally, even if it lags in HBM production capabilities. Samsung says it has begun mass production of its eight-layer HBM3e memory and will soon begin mass production of 12-layer modules. It expects to increase its supply of HBM by at least three times in 2024 compared with last year.

Asked directly about the alleged overheating and power consumption issues, Nvidia’s Huang also dismissed those reports. “There is no story there,” he said.

Korean-based SK hynix leads the pack in delivering HBM3 and HBM3e chips. The company’s production capacity for the chips is fully booked through next year, and SK hynix plans to spend $14.6 billion to build a new production complex to meet demand. 

Samsung’s investors have grown concerned that the electronics maker has yet to catch up with its smaller rival SK hynix. This may be one of the key factors that led to Samsung recently replacing the head of its semiconductor division.

Samsung has denied a media report claiming that its high-bandwidth memory (HBM) products failed Nvidia's quality tests due to issues such as excessive heat and power consumption. Samsung stated that it is testing HBM supply with its worldwide partners and emphasized ongoing cooperation to ensure quality and reliability, reports Business Korea. 

"[We are] smoothly conducting tests for HBM supply with various global partners," Samsung said in a statement published by Business Korea. "We are making efforts to improve quality and enhance reliability for all our products. We are rigorously testing the quality and performance of our HBM products to provide the best solutions to our customers."

Samsung recently started mass production of its fifth-generation HBM product — HBM3E devices in 24GB (8-Hi) and 36GB (12-Hi) capacity. The company reaffirmed its commitment to quality, though some analysts doubt its ability to quickly regain market share from SK hynix, according to Business Korea.

Now, while Samsung says that its latest HBM products — we presume that we are talking about HBM3E products — work fine with a wide range of processors, the company does not specifically deny whether they work just fine with all of Nvidia's processors. The details here are crucially important and the lack of them led us to speculate.

Although HBM memory is standardized by JEDEC and complies with all the requirements, solutions featuring HBM DRAM are highly custom solutions. Nvidia has multiple GPUs that work with HBM3E memory, including the Hopper architecture-based H200 as well as the Blackwell architecture-based B200, B100, and GB200. While all of them can use essentially the same HBM3E stacks, they may work with them differently and therefore have different requirements for their power consumption and heat dissipation. 

For example, Grace Blackwell GB200 uses the Blackwell B200 GPU with enhanced performance compared to 'ordinary' B200 Blackwell processors. As a result, Nvidia's Grace Blackwell platform has different requirements for memory power consumption and heat dissipation. In this case, Samsung's HBM3E memory may well suit H200, B200, and, say, AMD's upcoming Instinct MI350X (which is a rumored product, of course), but may not meet the requirements of Nvidia's GB200. Then again, we are tending to speculate here and we must remember that Samsung's HBM memory is claimed to work just fine with all of Nvidia's processors, despite the earlier media commentary.

Reuters reports that Samsung is facing challenges with its latest HBM3 and HBM3E memory stacks. They have failed Nvidia's tests due to excessive heat and power consumption issues. This setback is significant for Samsung as Nvidia dominates the global processors market for AI applications. But this could be an issue for Nvidia, which needs all the HBM supplies it can get to meet the demand for its processors based on the Hopper and Blackwell architectures.

Formally, Samsung's HBM3E memory stacks are the fastest in the industry, rated for data transfer rates of up to 9.8 GT/s/. Samsung officially started production of its 24 GB and 36 GB HBM3E in late April. However, the report from Reuters says that without elaboration, some of Samsung's HBM3 and HBM3E devices cannot pass Nvidia's validation for specific products.

Samsung's HBM3 and HBM3E stacks' failure to pass Nvidia's tests has been attributed to problems with excessive heat dissipation and power consumption, but again, there is no word which of Samsung's products overheat with which of Nvidia's GPUs. According to the report, Samsung has been working on these chips since last year but has yet to meet Nvidia's stringent requirements. It is a critical issue as Nvidia's approval is vital for Samsung's HBM business.

Despite these challenges, Samsung claims to optimize its products in collaboration with customers. The report says the company denies that the failures were due to heat and power issues and insists that testing is progressing as planned.

By contrast, competitors SK Hynix and Micron Technology have successfully supplied Nvidia with HBM3 and HBM3E modules. SK Hynix, in particular, has a technological edge due to its extensive investment in research and development of HBM memory in the last 10 – 12 years. At the same time, Samsung's struggles with meeting Nvidia's requirements mean better business opportunities for Micron and SK Hynix, whether these two companies can meet Nvidia's growing demands for HBM3E memory. Meanwhile, whether Samsung can supply its HBM3E devices to other companies, such as AMD, is unclear.

The market eagerly anticipates the rapid adoption of HBM3E memory, with shipments expected to concentrate in the latter half of the year. Therefore, Samsung has time to tweak its production process to solve the reported issues.

Samsung and SK hynix are finally retiring their respective DDR3 production lines for good, according to reports from IT Home. The two Korean memory manufacturers will reportedly stop supplying DDR3 memory to the market by the second half of this year. Both companies are making this change in response to growing demand for AI-optimized HBM3 memory, so Samsung and SK hynix are focusing on more lucrative markets.

It is crazy to think that DDR3 is still around, but DDR3 is still used today for niche applications that don't require bleeding edge DDR5 or even DDR4 memory. These devices mostly include cheaper/less complex embedded applications, including Wi-Fi routers and switches.

Given the profit margins, it's no surprise Samsung and SK hynix are canceling DDR3 production in favor of DDR5 and HBM3 memory types. SK hynix has confirmed that HBM3 memory demand in particular is spiraling out of control thanks to the AI boom. SK hynix's HBM supply is reportedly sold out for 2024 and most of 2025, leading to a 5–10 percent price hike next year for all HBM (HBM2E, HBM3, and HBM3E) memory types. Samsung has yet to publish its HBM bookings but we expect the company to be in the same predicament.

The intense HBM demand is expected to more than double HBM's market share by 2025, going from 2% in 2023 to 5% in 2024 and then to 10% in 2025. Even DDR5 memory is being impacted by HBM demand and will reportedly see a 20% price hike, as the top three memory manufacturers shift manufacturing priority to HBM.

The server and PC markets have long since moved away from DDR3, which first launched in 2007. DDR4 has been around since 2014, with DDR5 arriving in 2020. It's no surprise then that Samsung and SK hynix are finally ready to remove DDR3 from their production lines completely. While DDR3 may still be used in certain devices, demand for DDR5 and HBM is much higher and more profitable, so they understandably have top priority.

That's not to say that DDR3 production will completely cease, even though Samsung and SK hynix are leaving that sector. Micron and Nanya continue to produce limited quantities of DDR3 memory. The exit of Samsung and SK hynix from the DDR3 market could help elevate DDR3 pricing as production tapers off. Even that will only go so far, as there's not enough demand for older memory types to keep them around. To that end, DDR3 pricing is expected to rise continuously but slowly until it gets phased out completely.

The chief executive of SK hynix this week said that the company's supply of high bandwidth memory (HBM) has been sold out for 2024 and for most of 2025. The claims come on the heels of similar comments made by the CEO of Micron back in March, who said that the company's HBM production had been sold out throughout late 2025. Essentially, this means that demand for HBM exceeds supply.

SK hynix's announcement is more impactful than that of Micron due to the company's larger market presence. Currently holding between 46% to 49% of the HBM market, SK hynix's saturation contrasts starkly with Micron's smaller 4% to 6% market share, based on data from TrendForce. The combined fully booked production capacities of both SK hynix and Micron mean that over half of the industry’s total supply of HBM3 and HBM3E for the upcoming quarters is already sold out. While Samsung (the only remaining HBM producer) yet has to comment about its HBM bookings, it is likely that it faces similar demand and its high-bandwidth memory products are also sold out for quarters to come. That said, it is safe to say that HBM demand can barely meet its supply. 

This is not particularly surprising as escalating demand for advanced processors for AI training and inference is increasing demand for associated components, notably HBM memory and advanced packaging services. SK hynix, the company which first introduced high-bandwidth memory in 2014, is still the largest supplier of HBM stacks, to a large degree because it sells to Nvidia, the most successful supplier of GPUs for AI and HPC. Nvidia's H100, H200, and GH200 platforms rely heavily on SK hynix for its supply of HBM3 and HBM3E memory.

The surge in processor demand is directly linked to the expansion of AI adoption and processors like Nvidia's H100 and H200 needing fast and high-capacity memory, something traditional DDR5 and GDDR6 just cannot provide. As a result, HBM3 and HBM3E are witnessing unprecedented demand. Meanwhile, Nvidia is not alone with its demand for HBM as Amazon, AMD, Facebook, Google (Broadcom), Intel, and Microsoft are also ramping up production of their latest processors for AI and HPC.

In addition, the company said that it had begun sampling its new 12-Hi 36GB HBM3E stacks with customers, with plans to start mass shipments in the third quarter of this year. This development is part of SK hynix's strategy to maintain its leadership and meet the growing needs of its customers that need high-capacity HBM3E solutions.

We recently got the opportunity to visit one of Micron’s largest and most advanced DRAM manufacturing facilities. The visit impressed us on many levels: the scale of the fab, the advanced manufacturing technologies and techniques being used, and getting to look behind the scenes at the cutting-edge Taichung A3 facility. It was also a great pleasure to meet up with CVP and Head of Micron Taiwan Dr. Donghui Lu, who provided insight into the operations at the enormous ‘One Mega Taiwan’. Last but not least, we learned that Dr. Lu has enjoyed building PC DIY desktops since way back in 1996.

Arriving at Micron Taichung

Houli station, a small provincial stop, is the nearest rail transit link to Micron Taiwan. Emerging from the station you find several bicycle hire businesses outside for people wanting a leisurely ride around nearby parkland. However, around a five-minute taxi ride away is Micron’s colossal Fab 16 which is dedicated to making the world’s most advanced memory chips in Taiwan. Moreover, thousands of people work there (into five figures in the whole of Taiwan), and most need to be highly educated and highly trained for the tasks they do.

There are a lot of huge numbers to take in to weigh up this operation. To sum up the manufacturing facilities here, there are three memory chip production buildings: Taichung A1, A2, and A3. A1 and A2 came to Micron through acquisition, and A3 was the new Fab Micron constructed in 2021. Surrounding these fabs are complementary buildings for the vertically integrated parts of the business – things like MTB (Assembly and Test- backend), TCP (Taiwan Central Probe), and AATT (Advanced Assembly and Testing Taiwan). There are also manufacturing support industries like water, gas, and chemical processing and recycling plants dotted around the fabs.

Micron's newest fabs and buildings utilize sustainable resources and clever recycling. Taichung A3, for example, was constructed in a park with 600 trees. It also uses solar and rainwater resources. It is estimated that Micron's green credentials save 90 million KWh of electricity per year, and 75% of the water used is recycled and reused.

No one wants these fabs to stop due to adverse conditions like drought or power shortages. Thus Micron is aiming for all its Taiwan operations to achieve a 75% reduction in water consumption by 2030, net zero emissions by 2050 using renewable and controlling both direct and indirect emissions. Lastly, Micron Taiwan has already achieved a waste reuse, recycling, and recovery rate of 98.1% in FY23 thanks to its circular economy approach.

What happens at Micron Taichung?

DRAM chips for memory modules are produced in Micron facilities in Taiwan and Japan, with expansion plans announced in Boise, Idaho, and Syracuse, New York. The product portfolio using these memory chips currently ranges from ultra-high bandwidth products like HBM3E through graphics memory like GDDR6X, to LPDDR5 and DDR5 for computers. There are diverse form factors and target markets, too, spanning server, PC, mobile, auto, industrial, HMDs, and AIoT devices. You'll probably be most familiar with Micron DRAM-based products like DDR5 DIMMs and SODIMMs, but the latest LPCAMM2 mobile memory modules will be made by Micron, too.

Dr. Lu explained to us that, in brief, Micron takes in wafers and puts them through its FEOL (Front End of Line), MOL (Middle of Line), memory cell testing, BEOL (Back End of Line), and PROBE (product testing) phases before sending them off to subsequent assembly, test, and packaging steps. Interestingly, the Taichung and Taoyuan fabs work in a unified manner for this operational flow, which Micron dubs ‘One Mega Taiwan.’

The thousands of staff at Micron around the world are involved in activities as diverse as research, electronic design, engineering, product design, manufacturing, product sales, and marketing.

At the time of the visit, Dr. Lu and other Micron execs appeared to be excited by the impending announcement of new HBM3E chips. They even presented us with a commemorative HBM3E memory chip.

Micron executives we met during the visit were clearly energized regarding the prospect of the AI boom delivering an uplift to business. This starkly contrasts with the feeling during last year’s memory industry downturn, almost entirely due to the surge in interest in AI. While 2023 saw the industry take its foot off the accelerator, it takes 9 to 12 months to rebuild production momentum, and this lag is partly why we see the memory business as so cyclical.

The cyclical nature of the business doesn’t affect research, development, and technological refinement. Micron and the other big memory players are well aware that when the cycle turns, they can still benefit from having the best, most advanced memory technologies.

Ahead of SK hynix and Samsung?

Memory making is a highly competitive business, and the three big players are all keen to power the AI revolution. TrendForce notes that, with HMB3E, this will probably be the first time that HBM technology is embraced simultaneously by the power trio.

Micron says that its current memory chips are produced on its 1 alpha and 1 beta processes. 1 alpha process began mass production in Taiwan in 2021.  1 beta memory chips were developed in Japan, but mass production began in Taiwan in 2023. Demand for the current process is outstripping supply, and last year’s downturn is more than partly to blame. Micron uses 1 beta memory chips for its newly announced 24GB HBM3E, which Nvidia has chosen for its H200 GPU accelerators.

In 2025, Micron expects to start producing its 1 gamma memory chip, which was reaffirmed in its Fiscal Q2'2024 earnings call on March 21. 1 gamma will be first ramped in Taiwan and then in Japan. It is already testing this advanced memory, which is fabbed using state-of-the-art EUV machines. Understandably, we weren’t allowed to tour this area of the megafab.

Going forward, it isn’t only things like EUV that will yield benefits. DRAM fabrication, like NAND, will see increased verticality via the use of advanced packaging to boost key properties like performance and density.

What we saw at Micron Fab 16

The two key aspects of Micron’s Taichung operations that we were allowed to view were the Remote Operations Center (ROC) and the showroom at Taichung A3 which provides an expansive view across a fab clean room. Remember, this is where the process of creating the current state-of-the-art 1 beta memory chips on 300mm wafers begins.

Entering the ROC, visitors see a huge mission control area where staff monitors, troubleshoots, and optimizes a process with hundreds of discrete steps, spanning from supplied wafer to finished wafer (a wafer that still needs to be tested, prepared, cut, etc.).

Various data presentation techniques are leveraged in the ROC, with bar charts, color coding, and so on, to make it easily apparent what is happening (and what has recently happened) and highlight anything sub-optimal. Some charts also showed various semiconductor machine brands. The charts and colors made it easy to see an overview and then drill down into anything unusual or not working as expected.

Micron Taiwan is naturally an early adopter of AI in manufacturing, and this is something Dr. Lu is particularly proud of. As part of its smart manufacturing initiative, the firm uses AI and machine learning to accelerate yield ramping and strengthen real-time monitoring. Understanding and interpreting the data on the multitude of screens in the ROC has been improved and accelerated by AI.

Chinese New Year had just passed, but it was strange to see the CNY celebratory decorations in the ROC, as they weren’t red – they were all green. The ROC manager told us that Taiwan’s engineers are averse to red and love green, as a green light on a machine means that all is well. However, we didn’t spot any green packs of Guai Guai around the building.

Taichung A3 viewing

A3 is the newest fab in Taichung, which, unlike the others, has been designed from the ground up by Micron. During our visit to this area of the megafab, the tour started with information about how A3 is as automated as possible to cut down any chance of contamination.

The Taichung A3 clean room we saw from the showroom was full of advanced machinery and Overhead Hoist Transfer (OHT) robots with minimal human intrusions. We constantly saw robots zipping around, each typically holding a large transparent 'AutoPod' caddy. Also important were incredibly clean and constant streams of filtered air and water.  

In an earthquake-affected island like Taiwan, readers also won’t be surprised to hear that the machines in the clean room area were all securely attached to the elevated flooring. Dr. Lu told us that some of the processing machinery also featured vibration dampening to shrug off smaller tremors. 

Near the end of our visit, we passed through another huge installation at Micron Taichung – the food court. The facility has its own Starbucks, Burger King, 7-11, and various local food outlets for employees on this very high-security site to enjoy during breaks. It was honestly so big that it was difficult to see one end of the rest, food, and drink area from the other.

Executive profile: Dr. Donghui LuCVP of Front-End Manufacturing and Head of Micron Taiwan

Dr. Lu is the current Head of Micron Taiwan, responsible for manufacturing facilities in Taichung and Taoyuan and aligning the operations on the island to Micron’s global objectives and mission. He’s been at Micron for two years and was our host throughout the tour - from the introduction to Micron Taiwan, accompanying us through to the ROC, and guiding our tour of the Taichung A3 showroom. Dr. Lu has an interesting history in PC tech, and spent 20 years of his career at Intel.

Lu graduated from Tsinghua University in Beijing, China, but went to Ohio State University to further his materials science and engineering studies by completing a PhD. After receiving joint executive MBA degrees in the U.S., Lu worked at Intel for over 20 years. His Intel endeavors spanned technology development and transfer, manufacturing and ramp-up, and fab construction. Lu also played a critical leadership role in growing Intel’s memory business, transferring jointly developed 3D NAND and Optane technology nodes from Micron to Intel.

Dr. Lu is a PC enthusiast and happily recounted his earliest PC DIY memories. He told us that the first desktop PC he remembers building was back in 1996. The system ran Windows NT 3.51 but was updated to NT4 as soon as that came out. Multi-core PCs were not very common back then, but Lu remembers making a dual CPU system in 1997, as his wife wanted a powerful graphics workstation for her work. That dual CPU system used a Tyan motherboard, perhaps the Tyan Tahoe 2 ATX.

Working at Intel might sound like a dream job for PC DIYers. Interestingly, Dr. Lu remembers being allowed to pick up a new CPU for free every year – a nice little perk. For guidance, he would (and assures us he still does) check out Tom’s Hardware component reviews and guides. However, like all of us old-timers lament, he notes that overclocking isn’t the boon it used to be for PC DIYers and tinkerers.

Micron is currently an underdog in the market of high-bandwidth memory, but it looks like things are changing rapidly as the company said that its supply of HBM3E memory had been sold out for 2024 and allocated for most of 2025. For now, Micron has said that its HBM3E will show up in Nvidia's H200 GPU for artificial intelligence and high-performance computing, so it looks like Micron is poised to grab a sizeable HBM market share. 

"Our HBM is sold out for calendar 2024, and the overwhelming majority of our 2025 supply has already been allocated," said Sanjay Mehrotra, chief executive of Micron, in prepared remarks for the company's earnings call this week. "We continue to expect HBM bit share equivalent to our overall DRAM bit share sometime in calendar 2025." 

Micron's initial HBM3E stacks are 24 GB 8Hi modules featuring a data transfer rate of 9.2 GT/s and a peak memory bandwidth of over 1.2 TB/s per device. Six of these stacks will be used for Nvidia's H200 GPU for AI and HPC to enable 141 GB of high-bandwidth memory in total. Since Micron is the first company to start shipments of HBM3E commercially, it is going to sell a boatload of its HBM3E packages. 

"We are on track to generate several hundred million dollars of revenue from HBM in fiscal 2024 and expect HBM revenues to be accretive to our DRAM and overall gross margins starting in the fiscal third quarter," said Mehrotra.  

The head of Micron said that it had started sampling of its 12-Hi HBM3E cubes, which increase memory capacity by 50% and therefore enable AI training of larger language models. These 36 GB HBM3E cubes will be used for next-generation AI processors and their production will ramp up in 2025. 

Since the manufacturing of HBM involves production of specialty DRAMs, ramp-up of HBM will greatly affect Micron's ability to make DRAM ICs for mainstream applications. 

"The ramp of HBM production will constrain supply growth in non-HBM products," Mehrotra said. "Industrywide, HBM3E consumes approximately three times the wafer supply as DDR5 to produce a given number of bits in the same technology node."

China-based Wuhan Xinxin Semiconductor Manufacturing (XMC) is kicking off a project focused on developing and manufacturing high-bandwidth memory (HBM), as this type of DRAM is a crucial element for AI and HPC processors, reports DigiTimes. XMC is controlled by Yangtze Memory Technology Co. (YMTC), China's leading producer of 3D NAND, which is controlled by state-owned Tsinghua Unigroup, which means that China's government is behind the effort.

XMC, which produces logic, CIS, and NOR flash memory and is an integral part of YMTC's 3D NAND production, has reportedly issued invitations for bids to build assembly lines and develop sophisticated packaging technology for its HBM initiative. The project plans to employ 3D chip stacking technology and acquire 16 sets of equipment to reach a monthly production target of 3,000 wafers. Meanwhile, XMC is not even a member of the JEDEC standard-setting organization, which means that it formally cannot access or use HBM specifications. The good news, though, is that its owner, YMTC, is.

Yangtze Memory uses fabs originally built by XMC to make both 3D NAND memory cells using memory-oriented process technology and 3D NAND periphery logic (address decoding, page buffers, etc.) using a production node aimed at high-performance logic. This is how it manages to make 3D NAND chips with ultra-fast I/O ahead of all of its rivals.

Presumably, Tsinghua Unigroup decided it makes sense for XMC (or YMTC?) to enter the HBM business. This move signifies a strong effort by China's government to accelerate the domestic development of HBM technology and become self-sufficient in high-speed memory for its AI and HPC processors.

XMC is not the only company in China that is interested in producing HBM memory. For example, CXMT has been exploring HBM technology in general for some time (we first reported about it in August and then more recently in February).

DigiTimes claims that about 20 companies from the People's Republic, including material suppliers and packaging houses, are eyeing a slice of the HBM pie. Technology is complex, and competition is fierce, but there is much money to make, especially with high demand and rising prices.

Major Chinese packing companies like JECT, Tongfu Microelectronics, JCET, and SJ Semiconductor already have HBM packaging technology. JECT recently showed off its XDFOI high-density fan-out package solution, which is designed for HBM. Tongfu Microelectronics has reportedly teamed up with a significant Chinese DRAM maker (presumably CXMT) to work on HBM projects.

Micron announced today that it is starting volume production for its HBM3E memory, the company's latest memory for datacenter- and AI-class GPUs. In particular, Micron says its HBM3E is being used for Nvidia's upcoming H200, which is slated to launch in the second quarter of the year with six HBM3E memory chips. Additionally, Micron also said it would detail its upcoming 36GB HBM3E chips in March at Nvida's GTC conference.

Compared to regular HBM3, HBM3E boosts bandwidth from 1TB/s to up to 1.2TB/s, a modest but noticeable jump in performance. HBM3E also increases max capacity per chip to 36GB, though Micron's HBM3E for Nvidia's H200 will be 'just' 24GB per chip, on par with HBM3's maximum. These chips also support eight-high stacks, rather than the maximum 12 HBM3E can theoretically offer. Micron claims its particular HBM3E memory has 30% lower power consumption than its competitors, Samsung and Sk hynix, which also have their own implementations of HBM3E.

Micron didn't say whether it would be the only company providing Nvidia with HBM3E for H200, and it probably won't be. There is so much demand for Nvidia's AI GPUs that the company reportedly pre-purchased $1.3 billion worth of HBM3E memory from both Micron and Sk hynix, even though the latter's memory's bandwidth is only 1.15TB/s and according to Micron uses more power. For Nvidia however, being able to sell a slightly less performant product is more than worth it, given the demand.

Micron won't be able to claim it has the fastest HBM3E however, since Samsung's Shinebolt HBM3E memory is rated for 1.225TB/s. Still, Micron can at least claim it's the first to volume production, as far as we can tell. Neither Samsung nor Sk hynix have said they have achieved volume production, so presumably Micron is first in that respect.

AMD might also be a customer of Micron's HBM3E. The company's CTO Mark Papermaster has said AMD has "architected for HBM3E," implying that a version of its MI300 GPU or a next-generation product is set to use the same HBM3E found in H200. It's not clear whether it will be Micron's HBM3E in particular, however.

Micron also says it will be sharing details about its upcoming 36GB HBM3E chips at Nvidia's upcoming GTC conference in March. The company says its 36GB HBM3E chips will have more than 1.2TB/s of bandwidth, more power efficiency, and will bring the stacking limit to 12 chips. Sampling for these 36GB modules also begins in March.

HBM3E memory with a whopping 9.6 GT/s (9.6 gigatransfers, or billions of transfers a second, a typical measurement of memory bandwidth) data transfer rate over a 1024-bit interface has just hit mass production. But the demands of artificial intelligence (AI) and high-performance computing (HPC) industries are growing rapidly, so HBM4 memory with a 2048-bit interface is just about two years away. A vice president of SK Hynix recently said that his company is on track to mass produce HBM4 by 2026, reports Business Korea.

"With the advent of the AI computing era, generative AI is rapidly advancing," said Chun-hwan Kim, vice president of SK hynix, said at SEMICON Korea 2024. "The generative AI market is expected to grow at an annual rate of 35%." 

The rapid growth of the generative AI market calls for higher-performance processors, which in turn need higher memory bandwidth. As a result, HBM4 will be needed to radically increase DRAM throughput. SK Hynix hopes to start making next-generation HBM by 2026, which suggests late 2025. This somewhat corroborates Micron's plan to make HBM4 available in early 2026. 

With a 9.6 GT/s data transfer rate, a single HBM3E memory stack can offer a theoretical peak bandwidth of 1.2 TB/s, translating to a whopping 7.2 TB/s bandwidth for a memory subsystem consisting of six stacks. However, that bandwidth is theoretical. For example, Nvidia's H200 'only' offers up to 4.8 TB/s with H200, perhaps due to reliability and power concerns.

According to Micron, HBM4 will use a 2048-bit interface to increase theoretical peak memory bandwidth per stack to over 1.5 TB/s. To get there, HBM4 will need to feature a data transfer rate of around 6 GT/s, which will allow to keep the power consumption of next-generation DRAM in check. Meanwhile, a 2048-bit memory interface will require a very sophisticated routing on an interposer or just placing HBM4 stacks on top of a chip. In both cases, HBM4 will get more expensive than HBM3 and HBM3E.

SK Hynix's sentiment regarding HBM4 seems to be shared by Samsung, which says it is on track to produce HBM4 in 2026. Interestingly, Samsung is also developing customized HBM memory solutions for select clients.

"HBM4 is in development with a 2025 sampling and 2026 mass production timeline," said Jaejune Kim, Executive Vice President, Memory, at Samsung, at the latest earnings call with analysts and investors (via SeekingAlpha). "Demand for also customized HBM is growing, driven by generative AI and so we're also developing not only a standard product, but also a customized HBM optimized performance-wise for each customer by adding logic chips. Detailed specifications are being discussed with key customers."

ChangXin Memory Technologies (CXMT), China's leading domestic DRAM maker, plans to build high-bandwidth memory (HBM) used by AI and high-performance computing (HPC) processors, reports Nikkei. The company is currently procuring necessary equipment, so it isn't going to roll out its HBM products for another year or two, but the effort to compete against established players in the top league emphasizes how badly China wants self-sufficiency in memory. 

According to Nikkei's sources, CXMT has placed orders and obtained manufacturing and testing gear for HBM memory assembly and testing from suppliers in the U.S. and Japan. Notably, top U.S. equipment makers, such as Applied Materials and Lam Research, were granted export licenses by the U.S. authorities to export fab tools to the Chinese chip manufacturer in mid-2023, as reported by two Nikkei informants. 

HBM3 stacks eight or 12 memory devices with wide interfaces on top of each other, interconnects them using through silicon vias (TSVs), and then places them on a base logic die that uses a 1024-bit interface to connect to a host processor at an unmatched bandwidth. While the architecture seems simple enough in general, it is not by far, and making HBM is a complicated task. 

Production of HBM known good stack dies (KGSDs) is fundamentally different from making traditional memory devices as HBM makers have to produce memory devices and test them, produce a base die and test it, assemble the package, and connect all the ICs with TSVs, and then test the whole stack. It takes a lot of tools and expertise to produce HBM DRAM, but this type of memory beats everything on the market in terms of bandwidth and in terms of power efficiency. 

CXMT is already running one DRAM fab near Hefei, China, and is raising money to build the second one. The second Hefei plant is believed to adopt more sophisticated process technologies (albeit behind leading offerings from Micron, Samsung, and SK Hynix) and will likely also be used to build HBM DRAM devices and packages. Meanwhile, CXMT has yet to develop its own HBM production and packaging technologies. Furthermore, some of China's chipmakers are still developing technologies to integrate HBM with logic chips (such as TSMC's CoWoS). One of SMIC's executives said a few years ago that the company would have to develop advanced packaging nodes, so it is likely that, by now, the company has gathered enough know-how in this direction. 

"When your DRAM technology already lags behind global rivals, that puts your HBM technology at a disadvantage to be competitive in a fully commercial market," Brady Wang, a semiconductor analyst with Counterpoint, told Nikkei. "Not to mention that HBM production requires complex design and manufacturing expertise to materialize. […] It could be a steep climb." 

One thing to note here is that while current HBM3 and HBM3E with a 1024-bit interface use interposers to connect to host processors, many industry experts indicate that HBM4 with its 2048-bit interface will need to be placed directly on host processors, which will further complicate production, but will bring performance and power efficiency further. It is unclear whether CXMT is developing HBM3, HBM3E, or HBM4E products. Still, we would expect the firm to start from HBM3/HBM3E, which will not enable it to challenge Micron, Samsung, and SK Hynix in the next few years, but it will likely be good enough to fulfill some of China's domestic needs.  

CXMT's efforts to secure crucial equipment for HBM production highlight the company's plan to compete in the lucrative AI computing market. Meanwhile, financing the effort (part of CXMT's second fab project) by investors backed by local governments can be considered a part of China's broader strategy to mitigate the impact of U.S. export controls and reduce dependency on foreign technologies. 

If you sell more processors for artificial intelligence (AI) and high-performance computing (HPC) applications than anyone else in the industry and want it to stay this way, you need to ensure a steady supply of your products. This is exactly what Nvidia does: it not only pre-purchases TSMC's wafer capacity and packaging capacity, but it also secures the supply of HBM3 memory. Korean publication Chosun Biz reports that the company pre-purchased over $1.3 billion of HBM3 memory from Micron and SK Hynix.

According to the publication's conversations with industry insiders, Nvidia has made upfront payments ranging from 700 billion to 1 trillion Korean won (about $540 to $770 million) to Micron and SK Hynix. While there is no specific information about the designation of the payments, which could range between $1.080 billion and $1.54 billion, it is widely speculated within the industry that the purpose of the payments is to ensure a steady supply of HBM3E memory for its upcoming 2024 AI and HPC GPU releases.

Nvidia is ramping up two products featuring HBM3E memory: the H200 AI and HPC GPU featuring 141GB of HBM3E and the GH200 platform featuring a Grace CPU and an H200 GPU outfitted with 141GB of HBM3E memory. Both devices will be popular and require a tremendous amount of memory, so buying in advance makes a lot of sense for Nvidia. 

In fact, it is not uncommon for GPU makers to pre-purchase expensive memory products from their suppliers as it is easier to sell advanced GPUs with memory, particularly to smaller graphics card makers. In the case of AI and HPC GPUs used for computing, Nvidia tends to sell the finished PCIe cards and SXM modules rather than just the GPU die, so it makes sense for Nvidia to procure the HBM3E, too.

Keeping in mind that Nvidia's AI and HPC GPUs are sold out for quarters to come, the company needs to ensure a steady supply of memory for its H100, H200, GH200, and other products that use HBM3 or HBM3E.

It remains to be seen if Micron, SK Hynix, and Samsung will have enough capacity to supply HBM3 and HBM3E memory to other developers of AI and HPC solutions, such as AMD, AWS, and Intel. If they do not, Nvidia will be able to strengthen its position further in the growing AI hardware market in 2024.

Although many companies are developing accelerators for artificial intelligence (AI) workloads, Nvidia's CUDA platform is currently unrivaled regarding AI support. As a result, demand for Nvidia-based AI infrastructure is high. To address it, Amazon Web Services and Nvidia entered a strategic partnership under which AWS will offer Nvidia-based infrastructure for generative AI. The two companies will partner on several key projects.

"Today, we offer the widest range of Nvidia GPU solutions for workloads including graphics, gaming, high performance computing, machine learning, and now, generative AI," said Adam Selipsky, CEO at AWS. "We continue to innovate with Nvidia to make AWS the best place to run GPUs, combining next-gen Nvidia Grace Hopper Superchips with AWS's EFA powerful networking, EC2 UltraClusters' hyper-scale clustering, and Nitro's advanced virtualization capabilities." 

Project Ceiba is a cornerstone of this collaboration, aiming to create the world's fastest GPU-powered AI supercomputer hosted by AWS and available exclusively for Nvidia. This ambitious project will integrate 16,384 Nvidia GH200 Superchips (using the GH200 NVL32 solution packing 32 GH200 GPUs with 19.5 TB of unified memory) that are set to offer a staggering 65 'AI ExaFLOPS' of processing power. This supercomputer is for Nvidia's generative AI research and development projects. 

The Nvidia DGX Cloud hosted on AWS is another major component of the partnership. This AI-training-as-a-service platform is the first commercially available instance to incorporate the GH200 NVL32 machine with 19.5 TB of unified memory. The platform provides developers with the largest shared memory available in a single instance, significantly accelerating the training process for advanced generative AI and large language models, potentially exceeding 1 trillion parameters.

In addition, AWS will be the first to offer a cloud-based AI supercomputer based on Nvidia's GH200 Grace Hopper Superchips. This unique configuration will connect 32 Grace Hopper Superchips per instance using NVLink. It will scale up to thousands of GH200 Superchips (and 4.5 TB HBM3e memory) connected with Amazon's EFA networking and supported by advanced virtualization (AWS Nitro System) and hyper-scale clustering (Amazon EC2 UltraClusters).

The collaboration will also introduce new Nvidia-powered Amazon EC2 instances. The instances will feature H200 Tensor Core GPUs with up to 141 GB of HBM3e memory for large-scale generative AI and high-performance computing (HPC) workloads. Additionally, G6 and G6e instances, equipped with NvidiaL4 and L40S GPUs, respectively, are designed for a wide array of applications ranging from AI fine-tuning to 3D workflow development and leverage Nvidia Omniverse for creating AI-enabled 3D applications.

Finally, the collaboration will introduce Nvidia's advanced software to speed up generative AI development on AWS. This includes the NeMo LLM framework and NeMo Retriever for creating chatbots and summarization tools and BioNeMo for accelerating drug discovery processes. 

"Generative AI is transforming cloud workloads and putting accelerated computing at the foundation of diverse content generation," said Jensen Huang, founder and CEO of Nvidia. "Driven by a common mission to deliver cost-effective state-of-the-art generative AI to every customer, Nvidia and AWS are collaborating across the entire computing stack, spanning AI infrastructure, acceleration libraries, foundation models, to generative AI services."

Today, TrendForce shed some light on the future of high bandwidth memory (HBM) technology and HBM4, specifically, which is expected to debut in 2026. This upcoming technology promises to extend its interface to 2048 bits and substantially change the architecture. TrendForce claims that HBM4 will significantly depart from traditional standardized DRAM technologies to more customized solutions. 

HBM4 will be the first to use a 12nm logic process technology for its base die, which foundries, not DRAM makers, will now make. This development will be a collaborative effort between foundries and memory suppliers, which essentially means a symbiotic relationship to advance high-speed memory technology. The increased performance and enhanced feature set of HBM4 are tailored to meet the demands of future processors for artificial intelligence (AI) and high-performance computing (HPC) from key industry players such as AMD, Nvidia, and Intel. 

The shift towards customization in the HBM4 market is a crucial aspect identified by TrendForce. This deviates from the traditional, standardized approach of commodity DRAM, which signifies a major shift in the industry. Buyers are increasingly seeking custom specifications and are exploring innovative options, such as stacking HBM directly on top of the system-on-chips (SoCs).

This trend towards customization is expected to bring new design and pricing strategies to the HBM industry. As memory technology becomes more specialized and tailored to specific needs, it paves the way for a new HBM technology era characterized by innovation, specialization, and a departure from the one-size-fits-all approach today. This evolution in HBM4 and beyond indicates a dynamic and rapidly advancing landscape in high-speed memory technology.

Another standout feature of HBM4 is its transition from the current 12-layer (12Hi) stacks to more advanced 16-layer (16Hi) stacks, increasing memory module capacity. This transition, expected to be completed by 2027, will necessitate using new hybrid bonding techniques to increase layer count while maintaining the integrity of the memory stacks. 

But while HBM4 will revolutionize the memory market, it is still years away. As a result, HBM3E is going to have quite a long lifespan. 

Nvidia said that its Grace Hopper GH200 superchip featuring its own Arm-based CPU and Hopper-based GPU for artificial intelligence (AI) and high-performance computing (HPC) powers Jupiter, Europe's first ExaFLOPS supercomputer hosted at the Forschungszentrum Jülich facility in Germany. The machine can be used for both simulations and AI workloads, which sets it apart from the vast majority of supercomputer installed today. In addition, Nvidia said that its GH200 powers 40 supercomputers worldwide and their combine performance is around 200 'AI' ExaFLOPS.

The Jupiter supercomputer is powered by nearly 24,000 GH200 chips with 2.3 TB of HBM3E memory in total interconnected using Quantum-2 InfiniBand networking and cooled down using Eviden's BullSequana XH3000 liquid-cooling technology. The machine offers 1 ExaFLOPS of FP64 performance for simulations, including climate and weather modeling, material science, drug discovery, industrial engineering, and quantum computing. In addition, the machine can provide a whopping 90 ExaFLOPS of AI performance for training of large language models and similar workloads. All of this will be powered by Nvidia's software solutions like Earth-2, BioNeMo, Clara, cuQuantum, Modulus, and Omniverse.

Each node within the Jupiter supercomputer is a powerhouse by itself, featuring 288 Arm Neoverse cores and four H200 GPU for AI and HPC workloads and is capable of achieving a remarkable 16 PetaFLOPS of AI performance. The GH200 superchips are interconnected using Nvidia's NVLink connection, though Nvidia has not disclosed total bandwidth of this interconnection. 

"At the heart of Jupiter is NVIDIA’s accelerated computing platform, making it a groundbreaking system that will revolutionize scientific research," said Thomas Lippert, director of the Jülich Supercomputing Centre. "Jupiter combines exascale AI and exascale HPC with the world’s best AI software ecosystem to boost the training of foundational models to new heights."

While Jupiter is a remarkable system — at the end of the day, this is Europe's first ExaFLOPS machine — it is not the only GH200-based supercomputer. The University of Bristol, in particular, is building a supercomputer with more than 5,000 GH200 with 141GB HBM3E, aiming to be the most powerful in the UK. Nvidia's Grace Hopper GH 200 is going to be used in more than 40 supercomputers around the world built by companies like Dell, HPE, and Lenovo. Combined performance of these system will be about 200 AI ExaFLOPS.

HPE, for example, will be using Nvidia's Grace Hopper GH200 superchip in its HPE Cray EX2500 supercomputers. These machines can be scaled up massively, using thousands of GH200 chips, which means faster and more efficient AI model training.

Nvidia's GH200 will also be available from ASRock Rack, Asus, Gigabyte, and Ingrasys by the end of the year. Nvidia also plans to offer free access to the GH200 through its LaunchPad program, making this powerful tech available to more people.

The U.K. government on Wednesday unveiled the country's most powerful supercomputer to date, the Isambard-AI. The machine, which is set to be built by HPE, cost £225 million ($273.38 million) and uses over 5,000 of Nvidia's GH200 Grace Hopper Superchip units to deliver around 200 FP64 PetaFLOPS performance, which is enough to make it the world's fifth highest-performing supercomputer. The system will be installed at the National Composites Centre (NCC) located in the Bristol and Bath Science Park.

The Isambard-AI will rely on the HPE Cray EX architecture and will incorporate 5,448 NVIDIA GH200 Grace Hopper Superchips, which combine Nvidia's Arm-based 72-core Grace CPU with a Hopper-based GH100 AI and HPC GPU. But there is no word whether the machine will use GH200 with 96 GB HBM3 memory or GH200 with 141GB HBM3E memory.

The supercomputer will also feature nearly 25 petabytes of storage, using the Cray Clusterstor E1000 as well as HPE Slingshot 11 interconnect. To cool down Nvidia's GH200 Grace Hopper processors, Isambard-AI will use advanced direct liquid-cooling technology with a thermal re-use model that will heat local buildings. 

As far as performance is concerned, Isambard-AI is expected to achieve over 200 FP64 PetaFLOPS for high-performance computing that requires accurate calculations and simulations, and will also deliver over 21 ExaFLOPS for AI inference and training workloads that use lower precision. Performance of the supercomputer represents a tenfold improvement over the U.K.'s previous fastest supercomputer, according to Nvidia. 

"Isambard-AI represents a huge leap forward for AI computational power in the U.K.," said Simon McIntosh-Smith, a Bristol professor and director of the Isambard National Research Facility. "Today, Isambard-AI would rank within the top 10 fastest supercomputers in the world and, when in operation later in 2024, it will be one of the most powerful AI systems for open science anywhere." 

The creation of Isambard-AI aims to position the U.K. as a global leader in AI technology and research. In addition to spending £225 million ($273.38 million) on the machine, the U.K. government will spend another £75 million ($91.095 million) on the creation of the national Artificial Intelligence Research Resource (AIRR) center. 

"In building one of the world's fastest AI supercomputers, the UK is demonstrating the importance for nations to create their own infrastructure," said Ian Buck, vice president of Hyperscale and HPC at Nvidia. "Isambard-AI will provide researchers with the same state-of-the-art AI and HPC compute resources used by the world's leading AI pioneers, enabling the UK to introduce the next wave of AI and scientific breakthroughs."

We've heard about HBM4 memory several times over the past few months, and this week Samsung revealed that it expects HBM4 to be introduced by 2025. The new memory will feature a 2048-bit interface per stack, twice as wide as HBM3's 1024-bit.

"Looking ahead, HBM4 is expected to be introduced by 2025 with technologies optimized for high thermal properties in development, such as non-conductive film (NCF) assembly and hybrid copper bonding (HCB)," SangJoon Hwang, EVP and Head of DRAM Product and Technology Team at Samsung Electronics, wrote in a company blog post.

Although Samsung expects HBM4 to be introduced by 2025, its production will probably start in 2025–2026, as the industry will need to do quite a lot of preparing for the technology. In the meantime, Samsung will offer its customers its HBM3E memory stacks with a 9.8 GT/s data transfer rate that will offer bandwidth of 1.25 TB/s per stack.

Earlier this year Micron revealed that 'HBMNext' memory was going to emerge around 2026, providing per-stack capacities between 32GB and 64GB and peak bandwidth of 2 TB/s per stack or higher — a marked increase from HBM3E's 1.2 TB/s per stack. To build a 64GB stack, one will need a 16-Hi stack with 32GB memory devices. Although 16-Hi stacks are supported even by the HBM3 specification, nobody has announced such products so far and it looks like such dense stacks will only hit the market with HBM4.

To produce HBM4 memory stacks, including 16-Hi stacks, Samsung will need to polish off a couple of new technologies mentioned by SangJoon Hwang. One of these technologies is called NCF (non-conductive film) and is a polymer layer that protects TSVs at their solder points from insulation and mechanical shock. Another is HCB (hybrid copper bonding), which is a bonding technology that uses copper conductor and oxide film insulator instead of conventional solder to minimize distance between DRAM devices as well as enable smaller bumps required for a 2048-bit interface.

High-bandwidth memory (HBM) has come a long way in less than 10 years it has been on the market. It dramatically increased its data transfer rate, increased capacity by orders of magnitude, and gained a plethora of features. There is another major change incoming and this one is going to be drastic: next generation HBM4 memory stacks will feature a 2048-bit memory interface, according to a DigiTimes report citing Seoul Economy.

Increasing interface width from 1024-bit per stack to 2048-bit per stack will be the biggest change HBM memory technology has ever seen. Since 2015, all HBM stacks have featured a 1024-bit interface. But since the information comes from an unofficial source, it should be taken with a grain of salt.

It is unclear whether memory makers will be able to maintain a ~9 GT/s data transfer rates supported by HBM3E stacks for HBM4 stacks with a 2048-bit interface, but if they can, the increase in bus width will double peak bandwidth from 1.15 TB/s per stack to 2.30 TB/s per stack. It is also unclear how widening of a per-stack memory interface will affect the number of stacks that a processor and interposer will handle. 

Today's massive processors such as Nvidia's H100 support six 1024-bit wide HBM3/HBM3E known good stacked dies (KGSDs) using a massive 6144-bit wide interface, But if the interface of a single KGSD increases to 2048 bits, it remains to be seen whether processor developers will keep using the same number of HBM4 stacks, or reduce them.

There is also a concern that yields of KGSDs with a 2048-bit interface will decrease as it is harder to produce memory stacks with thousands of through silicon vias (TSVs), but the report says that Samsung and SK Hynix are confident that they will be able to achieve a '100%' yield with the new type of memory.

For now, memory stacks with a 2048-bit interface look pretty fantastic and we would consider this information with caution. Yet, there is no smoke without fire.

China is striving to develop its own high-bandwidth memory-like (HBM-like) for artificial intelligence and high-performance computing applications, according to a report from the South China Morning Post. ChangXin Memory Technologies (CXMT) is reportedly at the forefront of this initiative. 

HBM is the indisputable leader when it comes to performance as every HBM stack features a 1024-bit memory interface and a decent data transfer rate. Because of the wide interface and vertical stacking, production of HBM devices does not require the most advanced lithography. In fact, it is believed that global DRAM leaders use time-proven technologies for their HBM2E and HBM3 memory devices. 

What HBM does require is sophisticated packaging technologies — as connecting eight or twelve memory devices vertically using tiny through silicon vias (TSVs) is a complicated procedure. Yet, assembling an HBM-like known-good stacked die (KGSD) module is easier than producing a DRAM device on a 10nm-class process technology.

Despite technological limitations imposed by sanctions, industry insiders questioned by SCMP suggest that CXMT could still produce its own HBM-like memory. But it remains to be seen how wide the Chinese 'HBM' interface will be and how many DRAM devices per module it will be able to stack.

CXMT is China's leading domestic producer of DRAM, both in terms of technological prowess and in terms of manufacturing capabilities, so it is China's best bet for developing a proprietary type of memory designed to compete against industry-standard HBM in terms of bandwidth and capacity. Due to U.S. sanctions, CXMT and other Chinese chip manufacturers are constrained to use less advanced technologies for production, putting them at a competitive disadvantage globally.

AI and GPC processors are essential for various applications, including autonomous vehicles, artificial intelligence, and high-performance computing, so it is understandable why China wants to build its own HBM-like memory. Currently, companies like Biren can obtain HBM2E memory from leading DRAM suppliers, but if the U.S. government restricts access to this type of memory, China's only option will be to rely on its own technologies. As a result, development of HBM-like memory is part of a larger national agenda to become self-reliant in semiconductor technology.

SK Hynix said Monday that it finished the development of its first HBM3E memory modules and is now providing samples to its customers. The new memory stacks feature a data transfer rate of 9 GT/s, which exceeds the company's HBM3 stacks by a whopping 40%. 

SK Hynix intends to mass-produce its new HBM3E memory stacks in the first half of next year. However, the company never disclosed the capacity of the modules (as well as whether they use 12-Hi or 8-Hi architecture) or when exactly it is set to make them available. Market intelligence firm TrendForce recently said that SK Hynix is on track to make 24 GB HBM3E products in Q1 2024 and follow up with 36 GB HBM3E offerings in Q1 2025.

If the information from TrendForce is correct, SK Hynix's new HBM3E modules will arrive just in time when the market needs them. For example, Nvidia is set to start shipments of its Grace Hopper GH200 platform with 141 GB of HBM3E memory for artificial intelligence and high-performance computing applications in Q2 2024. While this does not mean that the Nvidia product is set to use SK Hynix's HBM3E, mass production of HBM3E in the first half of 2024 strengthens SK Hynix's standing as the leading supplier of HBM memory in terms of volume. 

However, it won't be able to take the performance crown. SK Hynix's new modules offer a 9 GT/s data transfer rate, a touch slower than Micron's 9.2 GT/s. While Micron's HBM3 Gen2 modules promise a bandwidth of up to 1.2 TB/s per stack, SK Hynix's peak at 1.15 TB/s. 

Although SK Hynix refrains from revealing the capacity of its HBM3E stacks, it says that they employ its Advanced Mass Reflow Molded Underfill (MR-RUF) technology. This approach shrinks the space between memory devices within an HBM stack, which speeds up heat dissipation by 10% and allows cramming a 12-Hi HBM configuration into the same z-height as an 8-Hi HBM module.

One of the intriguing things about the mass production of HBM3E memory in the first half of 2024 by Micron and SK Hynix is that this new standard still has not been formally published by JEDEC. Perhaps, the demand for higher-bandwidth memory from AI and HPC applications is so high that the companies are somewhat rushing mass production to meet it.

Memory makers are ramping up production capacity for high-bandwidth memory (HBM) due to rapidly surging orders from cloud service providers (CSPs) and developers of processors for artificial intelligence (AI) and high-performance computing (HPC), such as Nvidia, according to TrendForce. The market research firm estimates that annual bit shipments of HBM will grow 105% by 2024. 

TrendForce claims that leading DRAM producers like Micron, Samsung, and SK Hynix supplied enough HBM memory in 2022 to keep pricing predictable. Yet, a rapid increase in demand for AI servers in 2023 led to clients making early orders, pushing production limits. TrendForce, the market research firm now projects that aggressive supplier expansion will raise the HBM sufficiency ratio from -2.4% in 2022 to 0.6% in 2024.

In addition to surging demand for HBM in general, analysts from TrendForce also note ongoing transition of demand from HBM2e (used on Nvidia's H100 cards, AMD's Instinct MI250X, and Intel's Sapphire Rapids HBM and Ponte Vecchio products) to HBM3 (used on Nvidia's H100 SXM modules, AMD's upcoming Instinct MI300-series APUs and GPUs) The demand ratio for these is estimated at around 50% for HBM3 and 39% for HBM2e in 2023 and then HBM3 will represent 60% of shipped HBM in 2024. This burgeoning demand, coupled with a higher average selling price (ASP), is expected to significantly elevate HBM revenue in the coming year.

But to meet demand for HBM, makers of memory need to increase output of HBM. This is not particularly easy as in addition to making more memory devices, DRAM producers need to assemble these devices in 8-Hi or even 12-Hi stacks, which requires specialized equipment. To satisfy demand for HBM2, HBM2E, and HBM3 memory, DRAM makers need to procure additional tools to expand their HBM production lines and delivery and testing time for them is between 9 and 12 months, so tangible increase of HBM output is now expected sometimes in Q2 2024.

SK Hynix leads in HBM3 production with most of the output going to Nvidia, TrendForce claims, whereas Samsung mostly produces HBM2E for other processor developers and CSPs. Micron, which does not make HBM3, can only supply HBM2E (and Intel uses such memory on its Sapphire Rapids HBM processor, according to reports) while it is prepping to ramp up production of HBM3E (which the company calls HBM3 Gen2) in early 2024. Just yesterday Nvidia introduced the industry's first HBM3E-based AI and HPC platform and given that demand for Nvidia's products is overwhelming, it is likely that Micron will capitalize on the new GH200 Grace Hopper HBM3E-enabled platform.

Meanwhile, TrendForce anticipates that in 2023 ~ 2024 Samsung and SK Hynix will have nearly equal market shares, collectively comprising approximately 95%. By contrast, share of Micron is projected to be between 3% and 6%.

TrendForce observes a consistent decline of the average selling price for HBM products annually. To boost customer demand and respond to softening demand for previous-generation HBM types, suppliers are reducing HBM2e and HBM2 prices in 2023. Although 2024 pricing strategies remain undecided, there is potential for further price cuts for HBM2 and HBM2e due to increased HBM supply and suppliers' ambitions to expand market share, according to TrendForce. 

 Nonetheless, HBM3 prices are expected to remain stable, and with its higher average selling price relative to HBM2e and HBM2, it could drive the HBM revenue to an impressive $8.9 billion in 2024, marking a 127% YoY growth.

Nvidia on Tuesday unleashed a revamped version of its next-generation Grace Hopper Superchip platform with HBM3e memory for artificial intelligence and high-performance computing. The new version of the GH200 Grace Hopper features the same Grace CPU and GH100 Hopper compute GPU but comes with HBM3e memory boasting higher capacity and bandwidth. 

The new GH200 Grace Hopper Superchip is based on the 72-core Grace CPU outfitted with 480 GB of ECC LPDDR5X memory as well as the GH100 compute GPU that is paired with 141 GB of HBM3E memory that comes in six 24 GB stacks and uses a 6,144-bit memory interface. While Nvidia physically installs 144 GB of memory, only 141 GB is accessible for better yields.

Nvidia's current GH200 Grace Hopper Superchip comes with 96 GB of HBM3 memory, providing bandwidth of less than 4 TB/s. By contrast, the new model ups memory capacity by around 50% and increases bandwidth by over 25%. Such massive improvements enable the new platform to run larger AI models than the original version and provide tangible performance improvements (which will be particularly important for training).

According to Nvidia, Nvidia's GH200 Grace Hopper platform with HBM3 is currently in production and will be available commercially starting next month. By contrast, the GH200 Grace Hopper platform with HBM3e is now sampling and is expected to be available in the second quarter of 2024. Nvidia stressed that the new GH200 Grace Hopper uses the same Grace CPU and GH100 GPU silicon as the original version, so the company will not need to ramp up any new revisions or steppings.

Nvidia says that the original GH200 with HBM3 and the improved GH200 with HBM3E will co-exist on the market, which means that the latter will be sold at a premium given its higher performance enabled by the more advanced memory. 

"To meet surging demand for generative AI, data centers require accelerated computing platforms with specialized needs," said Jensen Huang, chief executive of Nvidia. "The new GH200 Grace Hopper Superchip platform delivers this with exceptional memory technology and bandwidth to improve throughput, the ability to connect GPUs to aggregate performance without compromise, and a server design that can be easily deployed across the entire data center."

Nvidia's next-generation Grace Hopper Superchip platform with HBM3e is fully compatible with Nvidia's MGX server specification and is, therefore drop-in compatible with existing server designs.

Nvidia is reportedly interested in evaluating SK Hynix's HBM3E samples, according to "industry sources" via a DigiTimes report. If the information is accurate, then Nvidia's next-generation compute GPU for artificial intelligence and high-performance computing applications could use HBM3E memory instead of HBM3.

According to the industry sources cited by Korea's Money Today and Seoul Economic Daily, Nvidia has requested samples of HBM3E from SK Hynix with a view to evaluating their impact on GPU performance.

SK Hynix's upcoming HBM3E memory will increase the data transfer rate from the current 6.40 GT/s to 8.0 GT/s. This enhancement will consequently elevate the per-stack bandwidth from 819.2 GB/s to a whopping 1 TB/s. However, there are uncertainties surrounding the HBM3E's compatibility with pre-existing HBM3 controllers and interfaces, as SK Hynix has not yet disclosed information on this aspect of the new technology. In any case, Nvidia and other developers of compute AI and HPC GPUs will need to evaluate technology.

SK Hynix seemingly intends to initiate sampling of its HBM3E memory in the latter half of 2023, with plans to begin large-scale production in late 2023 or 2024. SK Hynix plans to build HBM3E memory using its 1b nanometer fabrication process, which is the company's 5th generation 10nm-class node for DRAMs. This same fabrication process is currently used to produce DDR5-6400 DRAMs. The same technology will be used in production of LPDDR5T memory chips for high-performance, low-power applications. 

It remains to be seen which of Nvidia's compute GPUs will use HBM3E memory, though it is likely that the company will use the new type of memory for its next generation of processors due in 2024. Meanwhile, we do not know whether this will be a revamped Hopper GH100 compute GPU or something brand new.

SK Hynix currently controls over 50% of HBM memory market and is the only company to supply HBM3. It will also be exclusive maker of HBM3, at least initially.

Yole Development, a market research company, has projected a significant expansion of the HBM memory market as it has unique bandwidth advantage over other types of DRAM. The firm estimates that the market, valued at $705 million in 2023, will nearly double to reach a worth of $1.324 billion by the year 2027. 

SK Hynix has announced an enhanced version of its HBM3 memory that increases data transfer rate by 25% and therefore provides a sizeable performance boost for applications that uses this premium type of DRAM. Companies developing solutions for artificial intelligence (AI) and high-performance computing (HPC) will welcome HBM3E when it becomes available in 2024. 

SK Hynix's HBM3E memory will increase the data transfer rate from today's 6.40 GT/s to 8.0 GT/s, which will boost per-stack bandwidth from 819.2 GB/s to 1 TB/s. Compatibility of HBM3E with existing HBM3 controllers and interfaces remains unclear, as SK Hynix has yet to reveal details about this aspect of the technology.

SK Hynix plans to begin sampling its HBM3E memory in the second half of 2023 and start mass production in 2024. 

The company will use its 1b nanometer fabrication technology (the company's 5th Generation 10nm-class node for DRAMs) to produce its HBM3E memory. This production node is currently used to make DDR5-6400 DRAMs that are validated for Intel's next-generation Xeon Scalable platform. Eventually it will also be used to produce LPDDR5T memory chips for performance-demanding low-power applications. 

"Amid growing expectations that the memory market will start to recover from the second half, we believe our industry-leading DRAM technology, proven again through mass production of the 1bnm process this time, will help us improve earnings from the second half," said Jonghwan Kim, head of DRAM development at SK Hynix.

Assuming that SK Hynix's HBM3E memory development and mass production progresses as expected, SK Hynix will have a list of buyers eager to procure fast HBM3E memory for their next-generation solutions for AI and HPC. Meanwhile, SK Hynix, which is already the biggest HBM supplier according to TrendForce, will likely strengthen its positions if it becomes the first supplier of premium HBM3E memory.