Researchers in Korea and Japan have presented two separate memory-integration proposals that aim to increase HBM (High-Bandwidth Memory) capacity and bandwidth without trapping more heat inside ever-taller DRAM (Dynamic Random Access Memory) stacks, one of the most pressing challenges facing future AI accelerators. Presented at the 2026 IEEE/JSAP Symposium on VLSI Technology and Circuits held in June, the two approaches — V-Die from a Korean research collaboration and MOSAIC from a University of Tokyo-led group — both explore the same broad idea of standing DRAM memory dies on their edges instead of stacking the memory dies only upward like conventional HBM.

The Korean proposal, called Vertical-Die (V-Die), was presented by researchers at the Ulsan National Institute of Science and Technology (UNIST). The design rotates custom DRAM dies upright, drops through-silicon vias to free die area for more memory cells, gives each die its own bottom-edge I/O, and runs liquid-cooling channels between adjacent dies. In simulations against an HBM4 system at equal capacity, the V-Die system reportedly achieved 540 tokens per second on a GPT-3-sized workload, compared to 296 tokens per second for HBM4.

The Japanese project, MOSAIC, takes a similar “sideways stack” idea but focuses on the practical difficulty of connecting so many vertical dies to a GPU or package substrate. Presented by University of Tokyo researchers, the MOSAIC work uses orthogonal die stacking and a contactless die-to-die interface, in which data is transferred through tiny inductive coils rather than requiring every signal pad to land perfectly on a physical contact. The researchers say the prototype interface achieved up to 4 Gbps per channel, while the memory structure could double HBM4-class capacity in a DRAM-on-GPU configuration.

Both projects aim to solve the growing problem of AI chips being held back by memory. Modern accelerators can perform enormous amounts of computation, but large, powerful models depend on moving huge amounts of data between memory and compute. This is why HBM has become one of the defining technologies of modern AI hardware.

The technology addresses the memory wall by stacking multiple DRAM dies vertically on a base die and placing that stack very close to the processor. Nvidia's Blackwell Ultra B300, for instance, carries up to 288GB of HBM3E memory, without which much of the silicon would sit idle waiting for data. The dies are connected via through-silicon vias (TSVs) — tiny vertical channels etched through the silicon and filled with metal.

The stack then communicates with the GPU over an extremely wide interface, often routed through a silicon interposer or an advanced package. This is the core reason HBM can deliver terabytes per second of bandwidth: it uses a very wide, very short data path instead of sending memory traffic across a motherboard, as with conventional DIMMs (Dual In-line Memory Modules), physical sticks of RAM used in computers.

However, that same structure creates several problems. While taller stacks add more capacity, they also make it harder to remove heat. Heat generated in the lower dies and at the high-speed interface must pass through layers of silicon, bonding materials, underfill, and package structures before it reaches a heat spreader. Furthermore, TSVs consume die area that could otherwise be used for memory cells, and as bandwidth rises, more routing and I/O place additional pressure on both signal integrity and packaging costs.

HBM4, the latest generation of HBM, addresses a number of these challenges. Meanwhile, companies such as SK hynix, Samsung, and Micron are racing to improve speed, capacity, base-die performance, and thermal management. SK hynix has already shown iHBM, which embeds cooling elements into the HBM interface area, and Samsung has shown an HBM5 mockup with Heat Path Block cooling to more directly extract heat from the stack. However, they all retain the same upward stacking structure.

This convention is what V-Die and MOSAIC are challenging. By standing DRAM dies upright, the researchers expose far more silicon surface area to the cooling path. In theory, this turns the memory stack into something closer to a heat-sink fin array, where heat can move laterally and escape more directly instead of being trapped in the middle of a thick vertical pile. It also opens the door to new connection schemes along the bottom or side of each die, rather than forcing every die to communicate through TSVs running vertically through the stack.

For V-Die, the key shift is removing TSVs from the memory dies and replacing them with bottom-edge connections. Each DRAM die gets its own I/O along the bottom edge and connects directly to the substrate, with links reportedly spaced every 20 microns. The team says this layout gives four times as many connections as HBM4 and cuts memory read time by 37%, although some signals must travel farther across the package to reach the processor.

Cooling is the other half of the V-Die argument. The proposal places microfluidic cooling channels between adjacent upright DRAM dies, allowing coolant to dissipate heat closer to its source. According to the researchers, this could keep the stack around 45°C, far below the 80°C-plus range associated with dense HBM systems. In a simulated 16-die stack matched to H100-class hardware on a GPT-3-scale model, V-Die hit 540 tokens per second, compared to HBM4's 296, and cut first-token latency by 32%, or about 24 milliseconds.

MOSAIC, meanwhile, is focused on making the sideways stack manufacturable. Because the dies are assembled flat and then turned on edge, even a few microns of die-thickness variation across dozens of dies can add up to an alignment miss where the signal pads no longer land. The Japanese team’s answer is a contactless interface based on inductive coupling. One side of the memory die carries oblong coils, while a corresponding set of coils sits on the substrate or mating chip. Current in one coil induces a signal in the other, allowing data to cross the small gap without a direct metal-to-metal signal contact. This eliminates the need for precise overlapping, giving the package greater tolerance for assembly variation. Power, which requires fewer, larger connections than data, can still be supplied via physical contacts on the sides of the memory cube.

The VLSI MOSAIC prototype achieved up to 4 Gbps per channel and demonstrated TSV-free 3D integration for a memory-on-GPU layout. The team says the approach can enable twice the memory capacity of HBM4 without significantly increasing peak temperature. A related bump-MOSAIC hardware demonstration at ECTC used 100-micron-pitch microbumps, achieved stacking alignment within 6 microns as verified by X-ray CT, and showed a configuration with three times the thermal conductivity of conventional stacking while adding up to 30% more memory capacity.

While the results look promising, neither V-Die nor MOSAIC is close to replacing commercial HBM. Neither is close to shipping. V-Die is still a proposed architecture, with a prototype in the works to validate its thermal and electrical behavior; MOSAIC has proof-of-principle hardware, but the researchers have yet to show it scales to commercial DRAM capacity, yield, cost, and reliability.

Still, any viable solution to the multifaceted AI memory problem is a welcome development. SoftBank and Intel’s Z-Angle Memory (ZAM) and NEO Semiconductor’s 3D X-DRAM — both still in development — aim to solve the constraints of conventional memory. Meanwhile, the overall market is already feeling the squeeze on price and availability, even as memory makers divert capacity toward the more lucrative AI HBM and server products, driving consumer RAM prices even higher.

An Intel patent application published on July 2, 2026, surfaced by Underfox, has revealed the company's plans for a new high-bandwidth memory (HBM) architecture that aims to ease the packaging and cost bottleneck of today's interposer-based HBM. The patent application — filed back on December 26, 2024 — describes what Intel calls cross-batch memory (XBM), an "ultra-high-bandwidth memory with backend transistors" built with the goal of matching HBM4's footprint while swapping conventional DRAM and its ultra-wide interface for back-end-of-line (BEOL) transistors and serial Universal Chiplet Interconnect Express (UCIe) links.

Intel's proposed design is a memory stack that addresses the assembly costs that make conventional HBM expensive by dropping the costly silicon interposer and shrinking the package, while building in its own defect repair.

The filing lays out a stack of memory dies, each holding one-transistor one-capacitor (1T1C) DRAM fabricated in the back-end-of-line, wired together with through-silicon via (TSV) "gutters" and both-sided high-bandwidth interconnect (HBI) connections. Intel describes dies of roughly 1.5 gigabytes (GB) apiece — 768 "datablocks" arranged in a 32-by-24 grid, grouped into eight channels of eight sub-channels each — stacked eight high and scaling to 16. Data then leaves the stack over UCIe I/O bundles running at 32 gigatransfers per second (GT/s), funneled out through a base die.

To understand what Intel is changing, it helps to recall what standard high-bandwidth memory does. HBM stacks DRAM dies vertically on a base logic die, threads them together with TSVs, and communicates with the processor across a silicon interposer using an extremely wide parallel interface — on the order of 1,024 bits per stack. This width is how HBM delivers its bandwidth, but it is also what makes it expensive to package and hard to scale, as every one of those wires has to be routed through an interposer sitting between the memory and the compute die. As AI accelerators have outrun the rate at which memory can feed them, this "memory wall" has become the dominant constraint on performance, which is why nearly every large chipmaker is now attacking the interface and the stack rather than the logic.

XBM's first major change is structural. Conventional DRAM cells are built in the front-end-of-line, the base silicon layer where transistors are normally fabricated. XBM instead moves the 1T1C cell into the back-end-of-line, the metal-and-via stack above the transistor layer, using thin-film transistors. Building memory in the BEOL is what lets Intel pack the die into many small, independently addressable memory blocks, and it is the same backend-transistor direction Intel has pursued for placing memory directly over logic.

The second change is the interface. Rather than HBM's wide parallel PHY, XBM serializes data onto UCIe bundles at 32 GT/s, with the base die handling the serialize/deserialize step and routing all I/O to the compute die. Moving to a standard chiplet interconnect is what makes the design "chiplet-native" and, Intel argues, simpler and cheaper to package than an interposer-bound HBM stack. The tradeoff is that 32 GT/s is UCIe's current top data rate, so the interface is already running at the spec ceiling rather than leaving obvious headroom.

Intel also leans heavily on repairability. The base die carries dedicated spare channels, built-in self-repair (BISR), decode and debug logic, and four sub-channels of redundant memory arrays that act as fungible spares for defects in the dies above — post-assembly repair designed to claw back yield on a very tall stack.

A large portion of the patent application focuses not on the memory cell at all but on how to mount it. Intel details memory-on-package (MoP) and "reversed overhang" structures aimed at cutting the stack's Z-height — conventional MoP can add 300 to 350 micrometers (µm) — while removing the stiffener normally needed to control warpage and feeding DRAM power directly from the voltage regulator. This is the concrete basis for the "smaller, cheaper package" claim.

XBM should not be confused with ZAM (Z-Angle Memory), the architecture Intel is co-developing with SoftBank subsidiary SAIMEMORY and set to present at the VLSI Symposium 2026. ZAM's innovation is on the bonding side — a fusion-bonded, nine-layer stack of largely conventional DRAM with roughly 3-µm-thin silicon between tiers — and it reportedly targets around twice HBM4's bandwidth density, with commercialization aimed at 2029. XBM, by contrast, is an Intel-only filing that changes the DRAM transistor itself and the interface. Read together, they suggest Intel is running at least two parallel HBM alternatives, a fitting move for a company that began in 1968 as a memory maker.

The caveats on Intel’s proposed HBM architecture are the usual ones for a patent. The patent was filed 18 months ago, and there’s currently no product or roadmap, signaling potential intent rather than a shipping part. The UCIe interface is already at its rate ceiling, backend-transistor DRAM remains unproven at manufacturing scale, and the whole thing still has to justify itself against HBM4E and Intel's own ZAM timeline.

In a world where memory is no longer a commodity but a strategically valuable asset, customers are eager to sign long-term supply agreements (LTAs) with their suppliers to ensure a steady supply of 3D NAND and/or DRAM. Micron this week announced that it had signed 16 strategic customer agreements (SCAs), 14 of which are worth around $100 billion. Furthermore, the company expects to receive cash deposits and other commitments worth $22 billion, but has warned there is no foreseeable end in sight to the RAM crisis driving up PC component prices.

“14 of the 16 SCAs that we have signed have a cumulative revenue at minimum price per our contracts of approximately $100 billion over the remaining agreement term,” a statement by Micron reads. “Under the SCAs we have signed so far, we project to receive cash deposits and related financial commitments of $22 billion.”

Based on Micron’s claims, the company has about $100 billion of guaranteed baseline revenue already locked in under 14 of those 16 strategic customer agreements, assuming customers only buy the minimum committed volumes and only pay the minimum contract price. In reality, Micron can earn more if customers buy higher volumes or pay higher prices. Furthermore, Micron expects customers who signed these long-term SCAs to put up real money up front — or make equivalent binding financial commitments — as part of reserving future memory supply.

Micron claims it has signed strategic customer agreements with four 'very large customers' and three 'medium-sized customers,' which means that the contracts were inked with clients that previously did not commit to LTAs. The contracts are signed with a five-year term (except the automotive LTAs, which have a term of three years), from calendar 2026 to calendar 2030.

Micron claims that memory supply will be insufficient in 2027 and may improve gradually only in 2028. To that end, it is not surprising that its clients are willing to sign LTAs for 3D NAND and DRAM to ensure that they have enough memory for their products.

"With respect to supply, our customers are recognizing that supply shortages in memory and storage will take considerable time to improve," said Sanjay Mehrotra, chief executive of Micron, in prepared remarks. "Even as we expect industry supply to improve gradually in 2028, we currently do not have line of sight as to when memory supply will be able to catch up with increasing demand."

Normally, Micron and other memory producers inked LTAs with select clients only (read: with Apple, Nvidia). 16 LTAs is a lot for this kind of arrangement, and this looks like a business model shift for the company. It is noteworthy that the 16 signed contracts represent roughly 20% of Micron's DRAM volume and 33% of the company's NAND volume over the period through 2030. That said, Micron may sign more LTAs with more companies.

The so-called memory wall is a major performance limiter for many AI workloads, and high bandwidth memory (HBM) is not always a panacea since compute capability is growing faster than memory bandwidth. Qualcomm on Wednesday introduced its HBC near-memory compute architecture called high-bandwidth compute (HBC) that is designed to break the memory wall and enable the performance of certain AI workloads to scale linearly.

Qualcomm's approach to near-memory compute is pretty much straightforward: the company disaggregates the AI accelerator from the system-on-chip (SoC) and puts it under the LPDDR DRAM stack. The HBC accelerator connects to the LPDDR stack using through-silicon vias to provide maximum bandwidth and capacity without using expensive HBM memory and advanced packaging. Qualcomm does not disclose the actual bandwidth HBC provides, though the company claims that it offers 6X higher bandwidth-per-watt compared to HBM and over 200X capacity compared to on-chip SRAM.

"We have separated the AI accelerator from the XPU and placed the XPU directly beneath a DRAM stack," said Tony Pialis, Executive Vice President and General Manager of Data Center Business at Qualcomm. "This is very important because it gives us the performance advantages of SRAM with the density and capacity of stacked memory. In effect, the congestion associated with HBM is gone. The value to the industry is lower power consumption, less heat, and the elimination of the costly silicon interposer used by HBM solutions. We can also deploy multiple HBC stacks within a single compute device using standard packaging, which delivers a significant performance-per-cost advantage."

Putting DRAM on logic or next to logic is nothing new. All DRAM makers have experimented with near-memory compute architectures, but have failed to make them popular. More recently, GUC, a fabless ASIC design service company, proposed its DRAM-on-Logic (DoL) technology that places one to four DRAM layers on top of logic to get around 5 TB/s of memory bandwidth and offer higher performance than some HBM3E memory subsystems without using expensive advanced packaging and HBM3E stacks.

Since Qualcomm does not disclose actual performance numbers, it is hard to compare its HBC to GUC's offering. However, the biggest caveat about HBC is that Qualcomm does not tell us what the HBC accelerator actually does. In theory, it could be everything: a transformer-specific near-memory engine, a more general array of tensor cores, or some kind of preprocessing logic for AI inference or training.

Along with its HBC technology, Qualcomm also disclosed its HBC roadmap. While the company's AI200 accelerator, due later this year, will rely on LPDDR5X and offer 43 TB of RAM per rack, its successor AI250 will rely on the 1^st Generation HBC that will offer 18X bandwidth of AI200. The AI300 will use 2^nd Generation HBC that will provide 54X bandwidth of AI300.

SK hynix overtook Samsung Electronics on Monday to become South Korea's most valuable listed company, the first time Samsung has surrendered the top spot on the KOSPI index since November 2000, according to Reuters. SK hynix shares closed up 5.6% to lift its market capitalization to 2,080.4 trillion won ($1.35 trillion), edging past Samsung's 2,066.7 trillion won excluding preferred shares, after a rally of more than 340% this year built almost entirely on demand for the high-bandwidth memory it supplies to Nvidia and other AI chip buyers. The company held 61% of the global HBM market in 2025, against 17% for Samsung and 21% for Micron.

SK hynix is one of the few pure-play memory makers, while Samsung spans smartphones, displays, contract chipmaking, and home appliances, among many other markets. Investors value the focus on pure-play memory higher because HBM carries the industry's fattest margins and ties suppliers to specific AI accelerators, unlike commodity DRAM that buyers can swap between vendors.

SK hynix built its lead by continuing to invest in HBM through the 2023 downturn, when a memory price collapse pushed it to a 7.73 trillion won annual operating loss. Samsung, by contrast, reportedly hit yield and qualification delays on its HBM3E chips that slowed major Nvidia orders, the proximate reason for a 61% share against a 17% one.

Samsung's remaining stronghold is conventional DRAM, but even that margin is shrinking. Bank of America estimates put SK hynix's monthly DRAM output at roughly 589,000 wafers this year against Samsung's 691,000. SK hynix is projected to expand output by about 38% between 2025 and 2028, compared with 17.5% at Samsung, which would cut the production gap to under 10% by 2028 from around 23% in 2025. The capacity both companies are pouring into HBM is not going into the commodity chips behind the memory shortage they've warned could run past 2027, and SK hynix has pledged to double its memory wafer output within five years.

Samsung, meanwhile, disputes the ranking, telling Reuters that its market cap should include preferred shares, which would lift its value to 2,246.4 trillion won. The gap also reflects the HBM3 and HBM3E generations rather than what comes next. Nvidia CEO Jensen Huang confirmed earlier this month that Samsung, SK hynix, and Micron all passed HBM4 certification for the Vera Rubin platform, and Samsung shipped the industry's first 12-layer HBM4E samples on May 29th.

SK Group Chairman Chey Tae-won, who pushed through the original Hynix acquisition despite internal opposition, explained the strategy in a book published in January. "What I really wanted to accomplish when we acquired Hynix was to transform it from a commodity memory producer into a mainstream semiconductor company whose products are indispensable," Chey said.

A coalition of nine US trade associations has urged the Trump administration to take immediate action on what it describes as an emerging memory chip shortage driven by the explosive growth of AI data centers. In a June 3 letter sent to US Commerce Secretary Howard Lutnick and Treasury Secretary Scott Bessent shared with Tom's Hardware, the organizations — representing telecommunications providers, automakers, medical device manufacturers, and major retailers — warned that AI infrastructure deployments are consuming an outsized share of global memory production, creating supply constraints and price increases that could ripple across large segments of the US economy.

The coalition warned that the AI data center expansion, which has consumed an unprecedented share of global memory capacity, has led to a memory chip shortage that could lead to higher prices for consumer electronics, increased costs for broadband and telecommunications infrastructure, disruptions to automobile and medical device production, and delays affecting federal contractors attempting to fulfill government procurement obligations. The letter argues that these risks are emerging despite billions of dollars of US investment intended to strengthen domestic semiconductor supply chains.

The signatories acknowledged AI's importance but argue it shouldn't come at the expense of the rest of the economy. "While recent developments in AI offer the promise of generational technological advances and are important for US tech leadership, we must also ensure other key industries are not negatively impacted by this disruption in the marketplace," the coalition said.

The organizations are asking the administration to work directly with memory suppliers and major chip buyers to address the imbalance. Their recommendations include accelerating expansion of memory manufacturing capacity in the United States and allied nations, using trade agreements to strengthen supply-chain resilience, ensuring adequate memory supply for non-AI industries, leveraging CHIPS Act programs where possible, and reducing regulatory barriers that may slow capacity growth.

"We urge the Administration to work with memory chipmakers and chip buyers to assess steps that can be taken to address this imbalance in the memory market and protect against harm to consumers, workers, and businesses of all sizes," the letter states.

The warning arrives as memory manufacturers increasingly prioritize high-bandwidth memory (HBM), the specialized memory used in AI accelerators from companies such as Nvidia and AMD. Demand for HBM has surged over the past two years as hyperscalers race to deploy larger AI clusters, prompting memory suppliers to devote an increasing share of their production capacity to AI-oriented products.

Samsung and SK Hynix — which together with Micron control over 95% of global DRAM production — have been diverting wafer capacity toward high-margin HBM for AI accelerators, starving the commodity DRAM and NAND markets in the process. Both companies warned in April that significant shortages will continue through at least 2027. IDC, meanwhile, has already revised its 2026 PC market forecast downward by up to 9% as a direct consequence of memory scarcity and rising prices.

Industry analysts have repeatedly warned for months that AI demand is reshaping the economics of the memory market. While memory shortages have historically been cyclical, the coalition argues that AI infrastructure spending is creating a structural shift large enough to affect industries far removed from data centers. The letter marks the first coordinated, multi-industry push for federal intervention. Whether the administration will respond — and how — remains to be seen.

SK hynix announced iHBM today, a memory heat management technology designed to enhance AI system performance. The thermal packaging solution improves heat dissipation by integrating ICEs (integrated cooling elements) directly into the HBM package. SK hynix says the result is an over 30% reduction in thermal resistance, “ensuring stable operating characteristics even in high-temperature and high-load environments.”

The iHBM architecture embeds non-conductive silicon cooling elements directly into the Die-to-Die Physical Layer (D2D PHY), the critical, high-speed connection interface between the HBM base die and the AI processor, which is prone to high temperature spikes as a result of extreme data traffic. By placing cooling elements in this layer, SK hynix mitigates the severe thermal throttling that cripples AI system performance during heavy computational workloads.

The company believes that structurally preventing thermal throttling will enable next-generation memory layers (targeted for future generations like HBM5) to scale to higher stack heights and sustain maximum data transfer speeds under the heavy computational loads of AI data centers.

“iHBM is the optimal solution for minimizing heat generation developed by combining memory design capabilities and advanced packaging technology,” said SK hynix Vice President Lee Kang-wook. “We will proactively provide the value customers need in the AI environment and further solidify our leadership in AI memory.”

SK hynix plans to apply iHBM technology from next-generation products, such as HBM5, to meet the thermal management requirements of high-performance computing (HPC), AI data centers, and other ultra-high-density and ultra-high-bandwidth environments, thereby improving overall system stability and efficiency.

Heat management is one of the biggest challenges facing HBM (High-Bandwidth Memory) technology. Unlike conventional memory, HBM achieves massive bandwidth by vertically stacking multiple DRAM dies, dramatically shortening the distance data must travel and enabling far higher transfer speeds with better power efficiency.

To minimize latency and feed AI processors fast enough to avoid bottlenecks, HBM is placed extremely close to the GPU or AI accelerator on the same package, connected through a high-speed silicon interposer. However, this dense arrangement also creates severe thermal problems.

The Die-to-Die Physical Layer (D2D PHY) — the ultra-high-speed interface linking the processor and HBM stacks — continuously moves terabytes of data per second. As thousands of signaling lanes and billions of transistors switch at extremely high frequencies, switching losses, leakage current, and electrical resistance generate substantial heat.

The problem is compounded by the processor itself, which already produces enormous amounts of heat. With the HBM stacks packed tightly around the processor, heat accumulates rapidly in a very small area. When temperatures exceed safe limits, the system automatically reduces clock speeds and voltages through thermal throttling to prevent physical damage, lowering overall performance.

SK hynix's new iHBM approach attempts to tackle the problem at the structural level. Unlike conventional HBM cooling designs that primarily dissipate heat indirectly through the core die and surrounding package structures, the company's iHBM architecture instead places Integrated Cooling Elements (ICEs) directly around the D2D PHY region — the exact zone where thermal concentration is most severe. This approach creates a dedicated dissipation path at the source, reducing overall thermal resistance by 30% and allowing the chip to maintain stable operation under the high-temperature, high-pressure conditions that dense AI workloads demand.

SK hynix says the technology can be manufactured at scale using its existing Wafer Level Packaging (WLP) process, which is built on its Mass Reflow Molded Underfill (MR-MUF) packaging technology already used in commercial HBM products. The design is also architecturally compatible with existing System-in-Package configurations, meaning customers can integrate the new thermal capability without major redesigns.

At the North American Technology Symposium 2026, TSMC revealed its updated CoWoS packaging roadmap with major enhancements. Within chipmaking, the reticle limit is the largest size that a chip can be printed within a single step of the manufacturing process. TSMC's previous CoWoS-based system-in-packages (SiPs) roadmaps topped out at a 9.5-reticle size.

Now the company expects to produce 14-reticle and over 14-reticle-sized System-in-Packages (SiPs) with up to 24 HBM5E stacks by 2029. Such high integration is designed to meet the insatiable demand that AI accelerators have for both compute and memory bandwidth, and signals that packaging, not lithography, acts as a primary driver for semiconductor technologies.

"AI compute scaling is driven by the combination of advanced logic, SoIC 3D stacking, and CoWoS technologies," a statement by TSMC reads.

Bigger, hotter and hungrier

It is common for contemporary process technologies to scale slowly in transistor density, while full-node scaling enables 15% to 20% higher transistor density every three years. Intra-node improvements yield diminishing returns in density, but continue to provide performance improvements and greater power efficiency. This may not be a big problem for consumer product-makers, but it greatly affects the developers of AI and HPC applications, who must improve their solutions every year or two to remain competitive.

For those customers, TSMC has begun mass production of 5.5-reticle-sized CoWoS SiPs, supporting up to 12 HBM3E/HBM4 stacks and has achieved yields over 98%, according to the company.

In 2027, TSMC's CoWoS roadmap outlines a 9.5-reticle-sized interposer that supports 12 HBM5 stacks, which is expected to require a 120 mm by 150 mm substrate. In 2028, the company expects to produce a 14-reticle-sized interposer capable of carrying 20 3D-stacked compute chiplets and 20 HBM5 modules. By 2029, TSMC expects to produce interposers over 14 reticle sizes, with up to 24 HBM5E stacks. One standard reticle measures 26 mm by 33 mm (858 mm²), so a 14-reticle-sized interposer measures 12,020 mm², or the size of a small plate, and slightly larger than a CD.

An SiP that uses a 14-reticle-sized interposer and measures 12,020 mm² will consume an enormous amount of power, will require an exotic cooling solution (think exotic cold plates like those developed by Frore Systems, immersion cooling, or a combination of both), and will require a massive substrate, which will occupy a significant share of a server motherboard's real estate. The dimensions of the SiP alone will redefine how AI servers are built, whereas power consumption and cooling requirements are poised to open doors to a host of new technologies.

48x more compute transistors, 34x more bandwidth by 2029

Such gargantuan multi-chiplet processors show that advanced packaging is now the de facto scaling engine for the industry. In fact, TSMC's lateral CoWoS and vertical SoIC technologies enable faster growth of transistor budgets than traditional Moore's Law scaling. In addition, such SiPs also offer more memory bandwidth.

Based on TSMC's expectations, its customers will be able to put (at least) as many as 24 3D-stacked compute chiplets on one 14 reticle-sized CoWoS interposer by 2029, when A14 will be in mass production. When combined with scaling enabled by the latest process technologies (4x from N7 to A14), an ultra-high-end SiP from 2029 with 24 3D-stacked A14-based chiplets will be able to carry 48x more compute transistors than a high-end SiP with two N7-based chiplets from 2024, according to TSMC. Granted, we've rarely seen frontier dual-chiplet N7-based SiPs in 2024, even a cautious Nvidia opted to use 4NP instead.

There is a catch regarding 3D-stacked compute transistors, though. The bottom die may overheat, whereas the top die must get enough power to reach its full potential. To that end, many designs use the bottom die for cache (e.g., AMD's Zen 5-based CPUs with 3D V-Cache), not for compute.

Nonetheless, even a 24x increase in the number of compute transistors per high-end SiP in five years is a breakthrough that could not be achieved by Moore's law alone. However, such integration comes at a price. In the 2030s, cutting-edge SiPs with 24 3D-stacked compute chiplets and 24 HBM5E modules will likely cost an order of magnitude more than a high-end SiP from the mid-2020s.

In addition to compute capability, large CoWoS interposers also enable considerably higher memory bandwidth simply because they can carry more HBM stacks. It is not that simple, though. Total memory bandwidth scales dramatically, driven by the combination of wider HBM4 and HBM5 interfaces, more advanced HBM base dies built on TSMC’s N3P process, and ongoing CoWoS improvements that enable faster interconnect speeds. As a result, a high-performance SiP integrating 24 HBM5E stacks in 2029 is expected to deliver up to 34x higher bandwidth when compared to a reference SiP with eight HBM3 stacks in 2024, according to TSMC.

"HBM bandwidth scaling comes from multiple factors," said TSMC. "First, there is the memory itself — progressing from HBM3 to HBM4, with higher I/O counts. In addition, we are leveraging more advanced logic technologies for the base die, which allows us to push data rates well beyond 10 Gb/s per pin, something that was unheard of in traditional DRAM. At the same time, our CoWoS technology enables integration of more HBM stacks within a single package. […] All of these factors together — higher data rates, more I/O, and more stacks — contribute to the overall bandwidth scaling."

Slower transistor scaling

One of the things that strikes the eye about the current and upcoming process technologies due later this decade is the slow scaling of transistor density. While A14 is set to increase per-chip transistor density by 20% compared to N2 technology in 2028, its optical-shrink successor (A13) is only poised to provide a 6% higher density a year later.

Fortunately, TSMC continues to aggressively develop its CoWoS packaging technology, which promises to enable developers of system-in-packages to put 24 3D-stacked compute chiplets and 24 HBM5E modules onto one massive 14 reticle-sized interposer before the end of the decade. This will increase compute transistor count and memory bandwidth per SiP by 48x and 34x, respectively, compared to high-end data center SiPs in 2024, according to TSMC.

However, this level of integration will likely come at a high cost. System-in-packages with up to 24 3D-stacked compute chiplets and 24 HBM5E stacks in the 2030s will probably cost an order of magnitude more than high-end SiPs from the mid-2020s.

Micron’s planned $24 billion NAND flash expansion in Singapore will require 400 to 500 power transformers, which is more than double the 100 to 150 units a standard wafer fab typically needs, according to industry sources as reported by DigiTimes. The scale exceeds the annual output capacity of any single Taiwanese transformer manufacturer, turning heavy electrical equipment into a bottleneck for AI-driven semiconductor buildouts.

This level of demand from Micron reflects the power intensity of modern memory fabs tied to AI. HBM production for AI servers has driven every major memory maker into simultaneous expansion, and the electrical infrastructure required to support those fabs is now outpacing the supply chain built to serve it.

Micron's Singapore project, where production is targeted for late 2028, is one piece of a broader global buildout. The company has acquired PSMC's Miaoli Tongluo fab in Taiwan for $1.8 billion, with that facility slated for 2026, while new plants in Idaho and New York are underway, and a Hiroshima facility is expected to begin operations in the second half of 2026.

Samsung Electronics and SK hynix have also announced their own capacity expansions, all driven by the same demand curve: AI server deployments consuming HBM at volumes that existing production lines cannot satisfy. We’re now seeing a synchronized wave of fab construction across three continents as a result, with each project competing for the same pool of heavy electrical equipment and raw materials.

The toll this is taking is already visible in pricing and availability, with major heavy electrical equipment suppliers Fortune Electric and Allis Electric both having implemented price increases of 20% to 30%, driven by the surge in orders and rising costs of copper and other raw materials. Meanwhile, some transformer manufacturers have declined to quote on large-scale semiconductor projects entirely, citing an inability to meet the tight timelines and volume requirements. Industry sources say no single maker can absorb the scale of orders now flowing from the AI and semiconductor sectors.

International transformer brands, despite commanding higher prices, are gaining ground because their larger overseas factories can push out more units. Domestic Taiwanese manufacturers have responded by collaborating with secondary suppliers, dividing specifications and capacity across multiple firms to meet individual customer demands.

Transformers are also shared infrastructure, and, beyond fabs, the same equipment is needed for AI data center construction, utility-scale energy storage, and grid expansion projects. A supply chain that was already stretched before the AI buildout wave is now absorbing orders measured in the hundreds of units per project.

Unfortunately, delayed transformer deliveries will likely translate into delayed fabs, which in turn will push back the timelines for memory production that AI buyers are counting on. Data center operators planning new facilities are in the same queue, competing with semiconductor companies for equipment that takes months to manufacture and deliver.

SK Group chairman Chey Tae-won told reporters at Nvidia's GTC conference in San Jose on Monday that the global memory chip shortage is likely to persist for another four to five years, with industry-wide wafer supply lagging demand by more than 20%, Bloomberg reported. Chey, whose conglomerate controls SK Hynix, said leading memory makers are expanding capacity but are unlikely to fully meet demand until around 2030 because securing additional wafers takes at least four to five years, according to The Korea Times.

Chey warned that excessive focus on high-bandwidth memory could lead to shortages in conventional DRAM, potentially affecting smartphones and PCs. SK Hynix, Samsung, and Micron have shifted production in recent years toward HBM for AI accelerators, and the resulting shortfall in conventional DRAM output has driven steep price increases across consumer electronics.

SK Hynix holds roughly 57% of the global HBM market and 32% of overall DRAM, and the company is currently building a $13 billion HBM packaging and testing facility at its Cheongju complex in South Korea, with construction scheduled to begin next month and completion targeted for the end of 2027.

Samsung, meanwhile, is expanding DRAM capacity at its Pyeongtaek campus, with its P5 facility expected online by 2028. Micron is also planning a $9.6 billion HBM facility in Hiroshima, but initial output is not expected until 2028 either. Nearly all new capex is going toward HBM lines, where margins are highest.

Chey said SK Hynix is preparing measures to help stabilize DRAM prices, and that CEO Kwak Noh-jung is expected to announce a plan soon. He didn’t elaborate on what those measures would involve, though.

Gartner on February 26 projected global PC shipments will fall 10.4% and smartphone shipments 8.4% in 2026 compared to 2025 levels, driven by what the firm estimates will be a 130% surge in combined DRAM and SSD prices by the end of the year.

This, Gartner says, will lead to price increases of 17% among PCs year-over-year, leading to PC lifetimes extending by 15% for business users and 20% for consumers by the end of 2026, with the entry-level market facing the worst of it.

The artificial intelligence supercycle is reshaping the semiconductor and electronics industries, as the scale of the AI infrastructure buildout strains the entire supply chain. While developers of AI accelerators like Nvidia are cashing in on the AI boom, it's memory makers that will earn the most cash, according to estimates from TrendForce. Arguably, this is a result of the different business models and expansion strategies memory makers use compared to foundries, in addition to the behavior of the commodity market.

Demand outstrips supply

The company projects that while global foundry revenue is expected to total $218.7 billion, 3D NAND and DRAM revenue will reach $551.6 billion, which means that the total market for memory is more than twice as large as contract chip production. TrendForce attributes this to structural market changes caused by AI buildouts. The latter creates elevated demand for specific types of memory, creating shortages of all types of memory, and therefore affecting prices across the industry. As a consequence, while the AI industry does not need low-capacity commodity memory devices, they also become substantially more expensive amid tight supply. This creates the perfect conditions for memory makers to capitalize upon.

Indeed, the spot price of a 16 Gb DDR5 chip at DRAMeXchange was $38 on average, with a daily high of $53 and a daily low of $25. By contrast, the very same chip used to cost $4.75 on average just one year ago ($3.70 session low, $6.60 session high). Similar changes occurred to the prices of 3D NAND memory in recent quarters.

Fundamental differences

Just like some other analyst firms, TrendForce calls the AI megatrend a 'supercycle,' indicating its overwhelming ubiquity, which affects multiple industries, and its potential length.

There were two periods in the last few decades when revenue of memory makers grew significantly year-over-year for two years in a row: in 2017 – 2018, when hyperscalers built their vast data centers (+62% in 2017 and +27% in 2018), and in 2020 – 2021, when people increased purchases of PCs amid the COVID-19 pandemic. In both cases, memory makers increased capacity to meet demand and maintain their market share, which caused sharp drops in revenue back in 2019 and 2022.

The foundry industry — which is much more capital-intensive than the 3D NAND or DRAM industries — uses fabs that are harder and longer to build, and only suffered a year-over-year revenue decline in 2023.

The situation today is vastly different. On the one hand, leading developers of frontier AI models need the most powerful clusters to train their models, therefore creating demand for leading-edge hardware with expensive HBM3E memory and plenty of storage. On the other hand, these companies and their clients need more powerful inference systems to use those models. Therefore, demand for CPUs, AI accelerators, memory, and storage does not decline over time. Meanwhile, buyers like cloud service providers (CSPs) tend to be less sensitive to price increases, which is why 3D NAND and DRAM suppliers are expected to raise average selling prices more aggressively than in the past cycles.

Foundry vs. Commodity

3D NAND and DRAM are commodities, so their prices behave like prices of commodities, almost immediately reacting to tightening supply, increasing demand, or sentiment among buyers. While large PC makers purchase their memory at prices agreed upon every six months, a significant portion of memory is sold on the spot market.

This dynamic is reflected in TrendForce's projections that show memory revenue growth accelerating after the downturn of 2022 – 2023, including an expected 80% increase in 2024, followed by 46% growth in 2025, and a projected 134% surge in 2026.

By contrast, foundries tend to operate under long-term agreements that smooth price fluctuations, which prevents sharp swings that characterize memory markets. Even during periods of strong demand, foundry pricing adjustments occur gradually, which means slower revenue growth compared to memory vendors.

TrendForce models that following a 19% year-over-year revenue increase in 2024, the foundry market grew 25% in 2025 and will grow another 25% this year.

As a result, boosted by the AI supercycle and not constrained by long-term agreements, memory vendors will earn more than two times more this year alone compared to producers of logic, which have to adhere to their long-term contracts.

The biggest question

With HBM4 memory devices using four times more silicon than typical DRAM ICs, it is obvious that memory makers cannot meet all the demand that exists because of insufficient capacity, which results in price adjustments. However, the biggest question is how significantly current commodity 3D NAND and DRAM prices are influenced by insufficient supply, and how significantly they are influenced by typical commodity memory market behavior that dictates that customers buy more memory when it is getting more expensive, as it may get even more expensive in the future?

China’s two largest memory manufacturers, CXMT and YMTC, are said to each be embarking on an unprecedented expansion spree as, amid a global memory shortage, they see a golden opportunity to close the gap with the big-three incumbents Samsung, SK hynix, and Micron.

The two Chinese memory giants want to significantly expand their respective DRAM and NAND production over the next two years, according to reporting by Nikkei Asia, as AI infrastructure continues to squeeze the memory sector dry. The expansions, which are understood to center on new fabs in Shanghai and Wuhan, come as the big three continue to prioritize HBM output for AI accelerators over conventional DRAM.

New fabs targeted for 2027

CXMT is preparing a major DRAM facility in Shanghai that’s expected to be two-to-three times larger than its existing Hefei headquarters, with equipment and installation beginning later this year and volume production targeted for 2027. YMTC, traditionally a NAND flash supplier, is constructing a third fab in Wuhan that will also come online around 2027. Roughly half of that facility’s planned output is expected to be dedicated to DRAM rather than NAND — a significant deviation from the norm for YMTC.

“The company’s plants in Hefei and Beijing are already running at full capacity,” a source told Nikkei Asia, adding that CXMT sees very high demand from local companies to expand capacity as soon as possible. Samsung and SK hynix have both warned customers that memory supply tightness is likely to persist into 2027 as they continue allocating capacity to AI.

YMTC pivots to HBM

Commodity DDR5 for PCs and servers is constrained primarily because suppliers have redirected investment and wafer starts toward HBM, which carries substantially higher margins and is essential for AI accelerators. SK hynix currently dominates this space, supplying around 60% of global output, with Samsung and Micron splitting most of the remainder. All three vendors are expanding HBM capacity, but doing so comes at the expense of conventional DRAM, which has rocketed in price, creating a massive demand-side gap among consumers calling for more supply.

CXMT and YMTC clearly want to fill this gap. While the former remains a few generations behind the leading edge, it has demonstrated working DDR5 designs — mass production is still delayed — and has reportedly delivered HBM samples to domestic AI customers, including Huawei. Chinese sources cited by Nikkei suggest that CXMT intends to add dedicated HBM3 production lines as part of its Shanghai fab, with initial volumes aimed at domestic AI accelerators.

Meanwhile, YMTC is leveraging its packaging expertise to move into HBM. Rather than competing head-to-head with CXMT on DRAM process technology, YMTC is expected to focus on advanced assembly and HBM integration, working with local partners to produce memory stacks suitable for AI workloads. 50% of the company’s new plant’s capacity will produce DRAM, according to Nikkei. “They [YMTC] started to develop their own DRAM more than two years ago… now it’s only a matter of time for them to produce quality DRAM and HBM going forward,” said one of YMTC’s suppliers

A rock and a hard place

If successful, these expansion efforts will give China a vertically integrated domestic HBM supply chain, spanning DRAM wafer fabrication, stacking, and final assembly, without relying on outsiders.

That is exactly the outcome that the U.S. and its allies have attempted to slow down through export controls that restrict Chinese access to advanced manufacturing equipment. Currently, rules limit tool sales for sub-18nm DRAM processes and 128-layer or more 3D NAND, and explicitly target advanced packaging technologies relevant to HBM.

Despite these restrictions, Chinese memory makers have continued to make incremental progress by relying on older-generation tools and domestic equipment vendors. CXMT’s recent and unexpected demonstrations of high-speed DDR5 and LPDDR5X parts seriously highlight just how much headroom for progress still exists even in the absence of advanced tooling like EUV.

While it’s likely that Chinese HBM will trail the latest HBM3E and HBM4 designs from the likes of Samsung and SK, both in terms of bandwidth and density, it may be more than sufficient for domestic AI deployments. This creates an uncomfortable tradeoff for Western legislators and policymakers because any further tightening of export controls will just accelerate China’s push towards full self-sufficiency, while relaxing them would ease global shortages but undermine the intent of the restrictions.

Micron had already effectively lost access to the Chinese market following Beijing’s restrictions on its products in 2023, leaving Samsung and SK hynix as the primary foreign suppliers. Both Samsung and SK hynix operate large fabs in China that are now frozen in time due to U.S. rules, and both are investing heavily in next-gen memory outside China to stay ahead in HBM. As Chinese DRAM output rises, it’s entirely conceivable that both might look to exit the Chinese market entirely due to lack of viability, particularly if domestic progress emboldens Beijing to impose even tighter restrictions.

Expansion efforts by Chinese firms do not currently threaten the big three in high-end HBM, but they make waves in the DRAM market. Additional Chinese capacity aimed at domestic consumption reduces China’s dependence on imports and weakens the pricing power of global suppliers over time. In a market that’s already prone to relentless boom-bust cycles, any additional capacity could amplify volatility once, or if, AI demand stabilizes.

ChangXin Memory Technologies (CXMT), the leader of the Chinese DRAM industry, is on track to build third-gen high bandwidth memory (HBM3), according to Chosun Biz. Moreover, China-based Naura Technology, Maxwell, and U-Preseason are developing domestic tools specifically designed to assemble HBM memory stacks. If true, this means that in several years, China will have fully localized the production of HBM, which is crucial in the race to produce high-end AI accelerators, with Chinese companies under pressure to utilize homegrown silicon.

Built in China

Having started mass production of DDR5 and LPDDR5 memory, CXMT, China's largest and most advanced DRAM maker, plans to begin mass production of HBM3 this year. In parallel with preparing for volume production, CXMT has reportedly already supplied HBM samples to Chinese AI hardware developers, including Huawei, which is a surprise as Huawei's Ascend 950-series AI accelerators are set to rely on proprietary HiBL 1.0 and HiZQ 2.0 types of memory, not HBM3. Nonetheless, sampling of CXMT's HBM3 stacks indicates that the technology has reached a level of maturity suitable for customer evaluation and early integration.

For CXMT, HBM3 is a strategically important product for many reasons. First up, HBM is significantly more expensive than the 'commodity' DRAM that CXMT has produced so far, so entering a new market will affect CXMT's bottom line. Secondly, the availability of high-performance HBM3 is strategically important for China's AI sector. Thirdly, producing HBM3 stacks — that consist of eight or more large DRAMs with a wide I/O — will prove that CXMT is a world-class memory maker on par with Micron, Samsung, and SK hynix. Last but not least, making large HBM3 DRAM devices — which are three or four times larger than commodity DDR5 or LPDDR5 — consumes a lot of capacity. If CXMT manages to produce HBM3 in high volumes, it will need to demonstrate that it has enough capacity to address all segments of the market.

CXMT is not the only company in China to be working on HBM projects. Around two years ago Wuhan Xinxin Semiconductor Manufacturing (XMC), a foundry controlled by Yangtze Memory Technology Co. (YMTC), China's leading maker of 3D NAND, initiated a project to develop its own HBM packaging technologies using hybrid bonding and other IP from YMTC, and bought the appropriate equipment to reach a monthly production capacity of around 3,000 wafers.

XMC is also still working on its through-silicon via (TSV) process technology used in HBM, according to the report, but precise details, such as the exact development stages, remain unclear. Once XMC is done with development, it can offer integration services to DRAM makers or other interested parties.

Still, while a significant number of HBM characteristics — including performance efficiency and thermals — depend on packaging and hybrid-bond quality, XMC/YMTC will still need special HBM DRAM chips, which they will only be able to get from DRAM suppliers. We are not aware whether Chinese entities can access HBM DRAM ICs from foreign companies. If they cannot, then XMC/YMTC will have to assemble HBM modules based on memory devices made in China by companies like CXMT.

Domestic tools for HBM assembly

Building enough HBM for AI accelerators locally is one of the hardest challenges for the Chinese semiconductor industry, as the sector is in pursuit of two goals simultaneously. One of them is to reduce reliance on chips made abroad, so Chinese chipmakers must increase their output at all costs. Another is to reduce the country's reliance on export-controlled chipmaking tools, such as ASML's lithography machines. Therefore, China will need to develop competitive alternatives (or upgrade existing equipment) to be competitive with global rivals.

The U.S. export controls already limit multiple types of lithography and deposition tools to Chinese entities, with severely limited access to the advanced manufacturing tools required for the development and production of HBM. Consequently, China's HBM ambitions increasingly hinge on the rapid localization of etching, TSV, bonding, and packaging equipment rather than on memory design alone.

Given the limited amount of time that China may have before export limitations take place, local chipmakers must use what they have at hand while also preparing to use domestically developed tools. Chinese companies are working hard to develop and deploy their own tools required for HBM assembly; however, there is currently no indication of how ready the aforementioned tools might be for mass production.

Naura Technology, which is a leading supplier of etching equipment and currently commands 30% of the etching tools market in China, has expanded its portfolio specifically for HBM manufacturing by adapting and developing tools across multiple process steps, according to Chosun.

The company reportedly now offers not only etching systems, but also deposition systems for forming thin films on wafer surfaces, and cleaning tools optimized for HBM. These tools can be used to etch TSV structures in HBM DRAM dies, deposit liner and dielectric films required for TSV formation, and prepare wafers for subsequent bonding and other packaging steps. However, it's unclear whether these have been fully qualified and inserted into the mass production flows by companies like CXMT.

Maxwell Semiconductor has also reportedly developed next-generation hybrid bonding equipment designed to electrically and mechanically join multiple DRAM dies into a single HBM stack. Hybrid bonding is a key enabler for advanced HBM packaging as it enables ultra-fine-pitch copper-to-copper interconnects between dies and is therefore responsible for interconnect density and signal integrity.

U-Precision is another China-based WFE maker that offers equipment for HBM packaging, but it is unclear what exactly the company has to offer and whether it supplies its tools for HBM commercially.

Chinese HBM3 production likely by 2027

Based on media reports from China, there are multiple DRAM makers, chip packaging houses, and material suppliers working on developing and building HBM memory in the country. There are also wafer fab equipment makers that are either working on tools for HBM assembly itself or are already shipping them.

Keeping in mind the rapid progress that China's semiconductor industry has made in the last 20 years, we have little doubt that companies like CXMT and XMC/YMTC can establish volume production of HBM3 memory in 2026 ~ 2027.

However, even if CXMT and its peers can produce suitable DRAM dies, their HBM supply could be limited by back-end throughput and yield, specifically stacking/assembly, bonding, warpage, and slow overall package yield learning curves, as HBM3 is a new technology for CXMT. If Chinese memory makers start to use all-new equipment from local suppliers, their yield ramp could be even slower, especially if manufacturers lack proper inspection and testing tools.

The biggest challenge with ramping HBM is not production of large DRAM dies, stacking them on the base die, or even interconnecting them; it is the combination of these steps, where a single misstep ruins the whole package, and therefore the economy behind it. For now, China has no choice but to continue working on its HBM supply, but it may take years before it manages to saturate its own market with HBM, as this type of memory is inherently more difficult to produce than commodity DRAM like DDR5 or LPDDR5.

SK hynix, the world's biggest supplier of high-bandwidth memory (HBM), this week approved a ₩19 trillion ($12.896 billion) investment to build P&T7, a new advanced packaging and test facility in South Korea, dedicated solely to HBM. The plant will likely be the biggest HBM assembly and test facility in the world, but it will almost certainly not be the last HBM packaging and test facility of the same scale and cost going forward, considering the booming demand for memory, which has caused DRAM pricing to skyrocket.

Being the world's largest supplier of any type of product means you need to stay ahead of the whole industry, in terms of technology and production capacity, and the devil is in the details.

A massive facility

The company plans to construct what it calls P&T7 (Packaging & Testing 7) facility at the Cheongju Technopolis Industrial Complex, on a site measuring approximately 70,000 pyeong (approx. 231,405 square meters or 2,490,822 square feet). Construction is scheduled to begin in April 2026, and completion is targeted for the end of 2027, which is when the building will be finished, and is when SK hynix will begin installing equipment. Due to the equipping phase, expect the plant to come online toward the end of the decade on time for HBM4E, HBM5, and HBM5E types of memory.

The facility's dimensions are massive, resulting in a campus-scale site, rather than a back-end factory plot. For packaging and testing, the Fab P&T7 plot is unusually large; this may reflect the importance of HBM (and other exotic types of memory) assembly for the AI industry specifically, and the memory industry as a whole.

To put the scale into context, Intel's Ocotillo Campus near Chandler, Arizona, spans over 362,727 square meters, but it houses multiple front-end fab buildings, such as Fab 12, Fab 22, Fab 32, Fab 52, and Fab 62. Both Fab 52 and Fab 62 are expected to be capable of processing up to 40,000 wafer starts per month each when fully ramped, which makes them bigger than typical logic fabs run by TSMC.

While HBM packaging is a back-end activity, it still requires complex techniques like etching, lithography, hybrid bonding, and many other steps and tools used in logic production. This explains why HBM testing and packaging facilities are larger than typical back-end facilities.

Nonetheless, HBM packaging is inherently simpler than producing logic, so the scale of SK hynix's P&T7 plant is enormous, even by HBM standards. Its dimensions and investments dwarf SK hynix's HBM testing and assembly plant in West Lafayette, Indiana, which will cost the memory maker $4 billion and will span 39,948 square meters. Yet, keeping in mind that SK hynix expects the compound annual growth rate (CAGR) of HBM to be 33% from 2025 to 2030, it needs all the front-end DRAM and back-end packaging facilities it can build.

A strategic location

A key element of the project is its operational linkage with Fab M15X, a major SK hynix semiconductor manufacturing facility currently located in Cheongju that is currently being equipped with fab tools. The company expects the combined operation of M15X and P&T7 to create a tightly coupled manufacturing ecosystem capable of building HBM dies (which are three to four times larger than commodity DDR5). Therefore, SK hynix can test and package HBM in close proximity, essentially creating a vertically integrated manufacturing facility for HBM or other exotic types of memory that use multi-chip packaging.

The adjacency of Fab M15X and P&T7 is intended to elevate Cheongju into a new core base for SK hynix's AI memory operations, which will optimize production and packaging and can potentially enable the company to respond to market demand for HBM faster than it can today.

As a bonus, the close proximity of the front-end memory fab and the advanced packaging and testing facility will shorten feedback loops between engineers at both facilities, which might positively affect yields and/or performance. In fact, as the industry transits to more sophisticated types of memory for AI accelerators — such as HBM4 with a 2,048-bit interface — the bigger and more sophisticated assembly and testing plants for high-bandwidth memory located close to front-end DRAM fabs make sense.

From a strategic planning perspective, SK hynix says it had evaluated multiple domestic and international locations before selecting Cheongju for P&T7. The decision reflects multiple objectives. Firstly, the vast majority of SK hynix's memory is produced in South Korea, so it is reasonable to build its most advanced testing and packaging facility for AI DRAM nearby. Secondly, P&T7 working in proximity with M15X will reinforce the overall competitiveness of South Korea's semiconductor industry. Thirdly, a large project also supports regional development, a policy promoted by the government.

SK hynix notes that the site selection decision was not purely cost-driven; the company evaluated proximity to existing fabs, infrastructure readiness, and long-term supply-chain resilience. Cheongju was ultimately chosen as the location that balanced industrial efficiency best.

Although SK hynix never mentions it, there is one more factor that likely affected its decision to build its biggest HBM packaging facility in South Korea. Because the vast majority of AI accelerators are made in the region, not only in South Korea, but in Taiwan by TSMC.

Since SK hynix and TSMC share a lot of customers, the two companies work closely with one another. However, due to geopolitical and regional competitive reasons, SK hynix isn't likely to build a major facility in Taiwan, making South Korea the next-best option.

Technological importance

Building an extremely big HBM testing and packaging facility is important for SK hynix, not only from strategic and logistical points of view, but also from a technology development point of view.

Advanced packaging is as critical to HBM as the DRAM front-end fab because of HBM's inherent characteristics — extreme bandwidth, power efficiency, and density. These elements are enabled by the DRAM itself and packaging technology. A single DRAM die does not expose a 1,024-bit or 2,048-bit interface: it's enabled by stacking 8 – 16 dies, each featuring a 128-bit I/O, interconnected with Through Silicon Vias (TSVs), and routed through base dies and Redistribution layers (RDLs). To make everything work properly and according to specification, packaging must ensure bonding accuracy, optimal TSV resistance, and RDL integrity.

Power efficiency and thermals are also largely determined by packaging, as they define things like TSV and RDL parasitics, micro-bump or hybrid-bond quality, and interposer losses. Some packaging technologies have increased I/O voltage requirements and higher switching losses, which force manufacturers to reduce data transfer rates compared to offerings from their rivals.

Heat-extraction is also a big deal, and it entirely depends on packaging process technology (underfill and mold materials, as well as package-level thermal paths), as the better heat dissipation is between memory dies, the easier it is to cool the whole stack. SK hynix leads the industry with its mass reflow-molded underfill (MR-MUF).

Yield economics also increase the importance of packaging technology: even if a front-end DRAM fab achieves a 99% good die yield, if yield loss during packaging is significant (due to TSV defects, bond misalignment, and/or RDL defects), it hits margins across both the expensive DRAM fab and relatively inexpensive back-end packaging facility. Reliability qualification — burn-in, thermal cycling, and long-duration stress testing — is also performed at the package level and cannot be handled by generic back-end lines.

As HBM scales toward more dies per stack, wider interfaces, tighter pitches, and hybrid bonding, packaging complexity and costs of packaging and testing facilities will rise to logic-fab-class levels, which is probably what we are dealing with here.

Eventually, advanced packaging facilities — which already have costs similar to logic fabs from 2010 to 2015 — will become considerably more expensive, and DRAM makers will have to be willing to spend $20 billion or more on such facilities, as this will be the only way to enhance the bandwidth, efficiency, yields, and reliability of HBM memory going forward.

Packaging is a crucial process

Although HBM packaging is a back-end process, its reliance on lithography, etching, hybrid bonding, and other steps common in front-end fabs explains the cost of modern packaging facilities like logic fabs from the previous decade. Furthermore, increasingly complex next-generation types of HBM, starting from HBM4 and onwards, will encourage memory makers to build facilities like P&T7 costing tens of billions of dollars and located adjacent to DRAM fabs.

According to a report, HBM4 memory was not expected to reach volume production any earlier than the end of Q1 2026, according TrendForce. However, an Nvidia spokesperson told Tom's Hardware Premium: "Our HBM4 memory partners remain on track for production shipments in the second half of this year as originally planned", disputing the report that production has slid as a result of a redesigned spec.

While Nvidia denies the claim that HBM4 production has slid, if the report holds, it may have stemmed from two converging factors: Nvidia’s decision to revise its memory requirements upward for its next-gen Rubin GPU platform, and the company’s short-term strategy to aggressively extend shipments of its current Blackwell architecture. All three major HBM suppliers were reportedly forced to redesign their HBM4 products to meet the new specifications, pushing mass manufacturing back by at least one quarter.

This shift keeps HBM3 and HBM3e as the prevailing standards across AI and high-performance GPU deployments through at least Q1 2026. Samsung may be first to qualify, given that it has reportedly passed Nvidia’s qualification tests, but SK hynix is still expected to maintain the majority share as the primary supplier to Nvidia. Micron, a more recent entrant in the HBM market, has already begun sampling 11 Gbps-class HBM4 parts, but is still building out volume readiness.

Aligning with internal cadence

The changes also realign Nvidia’s internal cadence, with the Rubin GPU line, which will use HBM4 exclusively, now set for volume availability in the second half of 2026. Rubin’s target specs are the main reason HBM4 was reportedly behind schedule, though Nvidia insists that the timeline remains on track.

According to TrendForce, Nvidia pushed for speeds higher than 11 Gbps per pin, which required all three vendors to retool their designs. Each HBM4 stack carries 2,048 data I/Os, so a 13 Gbps upgrade pushes aggregate per-stack bandwidth to over 2.6 TB/s. That level of throughput places new stress on base die logic and thermals.

SK hynix and Samsung began delivering engineering samples to Nvidia in late 2025, but with Nvidia allegedly demanding last-minute spec changes, those parts will now be insufficient for Rubin's requirements. Samsung is said to have a slight edge on qualification, due to its newer base die process and integration stack. Still, SK hynix is expected to retain the bulk of Nvidia’s business into 2026, given its existing allocation contracts.

This is not just about specs, however, but also Nvidia's broader control over memory ecosystems. Its sheer amount of purchasing power gives the company the leverage to shape JEDEC standards, dictate packaging needs, and pace supplier production cycles. NVIDIA accounts for over 60% of global HBM consumption in 2024, according to Morgan Stanley, and TSMC’s advanced packaging nodes — especially CoWoS — are already fully committed to Nvidia’s Blackwell and Grace Hopper-class parts. Moving to Rubin and HBM4 implies even greater substrate complexity, requiring further capacity expansion at both the foundry and substrate partner levels

Nvidia confirmed at CES that Rubin silicon is already in full production; however, system-level availability won’t follow until much later in the year, which may be due to memory and interconnect bottlenecks. Rubin will ship with up to 288 GB of HBM4 and will rely on revised versions of Nvidia’s NVLink interconnect, optimized for the increased bandwidth profile. Early Rubin configurations are expected to pair with Grace CPUs via a refreshed NVLink architecture, allowing up to 900 GB/s of coherent bandwidth per link.

HBM suppliers recalibrate for 2026 volumes

The delay offers both a challenge and a reprieve for vendors. The challenge lies in redesigning HBM4 dies to meet Nvidia’s updated timing and signal integrity requirements, but the reprieve comes in the form of extra runway — most HBM3 and HBM3e nodes are now sold out through late 2026, and the additional time allows vendors to optimize yields and scale packaging operations.

SK hynix will continue shipping the lion’s share of HBM volume throughout the quarter; it has a deep allocation pipeline with Nvidia and has committed the majority of its high-end DRAM lines to HBM production. Samsung, which initially expected HBM4 to arrive by 2025, has increased its cadence significantly and is now expected to reach high-volume HBM4 qualification sometime in Q2 2026. Micron is simultaneously ramping 11 Gbps-class HBM4 and sampling early HBM4E products with up to 16 dies and extended bandwidth ceilings.

JEDEC ratified the HBM4 standard in April 2025, specifying 2048-bit interfaces and per-pin speeds beginning at 6.4 Gbps, scaling up to over 12 Gbps. With Rubin and other high-performance AI accelerators now targeting 13 Gbps or higher, vendors are pushing the upper limits of thermal and power envelopes. Micron has said that it expects 64GB stacks to become common with HBM4E sometime after late 2027. Meanwhile, each Rubin GPU package on the NVL72 will have eight stacks of HBM4 memory delivering 288GB of capacity and 22 TB/s of bandwidth.

The delay also allows some equilibrium to form in packaging. TSMC’s CoWoS-L capacity has been under severe pressure due to Nvidia’s Blackwell and AMD’s MI300 ramp. By spacing out Rubin’s arrival, Nvidia is implicitly giving its suppliers time to expand interposer and bumping operations without triggering yield degradation or substrate shortages.

Implications for AMD, Intel, and downstream designs

Nvidia may exert outsized influence over HBM4 production, but it is not the only company exposed to delays. AMD has leaned heavily on HBM3 and HBM3e across the MI300 and MI350, with the upcoming MI400 designed around 432 GB of HBM4. If HBM4 volume production slips further, it would not just reshape Nvidia’s cadence but also place direct pressure on AMD’s MI400 rollout.

Intel’s Habana Gaudi line is still anchored on HBM2e, with 128 GB per accelerator in the Gaudi 3. The company is known to be planning a Gaudi 4-class device codenamed “Jaguar Shores”, likely for release in 2027, using HBM4E, so its current timeline remains unaffected by Nvidia’s spec shift. Intel’s packaging flows for AI silicon are distinct from Nvidia’s, and its later entry into HBM4 adoption may allow it to bypass early yield limitations.

The real downstream impact may surface in how HBM availability shapes product segmentation. Nvidia’s highest-end Blackwell and Rubin GPUs will continue to monopolize advanced memory stacks and interposer capacity, effectively limiting HBM4 to premium datacenter SKUs. There is currently no sign of HBM4 migrating into consumer GPUs or gaming cards, given the absence of any new GPU announcements at CES. Even as GDDR7 supply tightens, Nvidia has not shown any intent to merge AI and GeForce memory standards.

With Rubin silicon now in full production and mass memory availability locked to late Q1 or early Q2 2026, the HBM race continues — just a quarter later than planned.

Update 01/13/2026 6:51am PT: Article amended to add quote from Nvidia spokesperson.

Micron this week announced that it would formally break ground on its fab site in Onondaga County, New York, on January 16. When fully built, the new site will house four fab modules and will cost around $100 billion in total. All four fab phases are scheduled to be completed by 2041, according to the company. The site will be an instrumental part of Micron's plan to build 40% of its DRAM output in the U.S. by the 2040s, up from virtually 0% today.

Five years of reviews and approvals

Micron's fab complex near Clay, New York, will be the largest semiconductor production facility in the state, as well as one of the largest fab complexes in the U.S., which will also outpace the company's site in Idaho when fully operational. In addition, $100 billion represents the largest private investment in the state ever, and one of the most expensive semiconductor manufacturing operations in America.

The groundbreaking ceremony to be held next Friday follows rigorous environmental review and necessary permit approvals, which has taken Micron around five years, something that delayed the whole project by around half a decade. Surprisingly, the groundbreaking ceremony precedes ground preparation, which is a rather unique situation. Normally, groundbreaking follows ground preparation, not preceding it. Yet, ground preparation will start more or less in-line with the schedule that the company outlined in 2025.

Micron's Environmental Impact Statement (EIS) filing, published in November 2025, indicates that Fab 1 site preparation was set to start in late 2025, but was obviously delayed by several weeks. The construction of the first facility is now set to begin in late 2026, and will run through Q2 2028. Given that equipping a fab typically takes nine to 24 months (depending on how you count), Micron expects the fab to start operations as early as Q1 2029. However, it will still take some time before the fab is fully equipped and ramped, so expect tangible DRAM output from Micron's New York Fab 1 by 2030, around five years later than originally planned.

The same documents schedule Fab 2 to break ground in the second half of 2028, Fab 3 in the second half of 2033, and Fab 4 in the first half of 2039. As a result, Micron's fab site near Clay, New York, will reach its full build-out and volume production by 2045, again, roughly five years behind the original timeline.

A long road ahead

Micron's new campus in New York will manufacture DRAM using advanced process technologies and is ultimately planned to feature four fabs with four cleanrooms of 600,000 square feet (55,700 m²) each, constructed in four major build-out phases.

To put the 600,000 square feet number into context, GlobalFoundries Fab 8, located in Malta, New York, has a cleanroom space of approximately 460,000 square feet (though this is set to expand). The initial fab will cost around $20 billion, with successors expected to cost more, as Micron adopts more Low-NA EUV tools (priced around $235 million per unit) and eventually High-NA EUV tools (circa $400 million per unit) for its process technologies.

The New York complex will complement Micron's new site near Boise, Idaho. Last year, construction of the first new fab in Idaho (ID1) was completed, and the facility is now projected to begin wafer output in the second half of 2027. The company is also hard at work in its efforts to speed up the completion of its fab ID2, which will be built adjacent to ID1, benefiting from shared infrastructure and R&D co-location. It is projected that ID2 will start mass production of DRAM ahead of New York's Fab 1, though no further details are known.

Micron's wafer fabrication facilities near Boise, Idaho, will be adjacent to the company's R&D center, so will adopt the latest process technologies and will benefit from short feedback loops between the fab's process-integration teams and R&D personnel nearby. Such close collaboration will likely enable faster time-to-yield, which will improve Micron's profitability.

The close collaboration between process-integration and R&D teams will be particularly instrumental given Micron's current focus on high-capacity enterprise-grade DDR5 for servers and high-bandwidth memory (HBM) for AI accelerators. Both types of products tend to consume large DRAM dies, so rapid yield improvement is particularly important for these kinds of applications, as larger DRAM dies are expensive to make.

The grand plan

Right now, Micron has five wafer production facilities globally: two 3D NAND fabs in Singapore, a DRAM fab in Japan near Hiroshima, and two DRAM fabs in Taiwan (near Taichung and near Taoyuan). In addition, the company has an HBM packaging facility in Singapore, plans to build another fab module in Japan, as well as an HBM assembly plant in the U.S.

Given that the HBM packaging facility in Virginia is expected to come online after Micron ramps up production of HBM memory devices in Idaho, expect it to begin assembling HBM5 or HBM6 stacks towards the end of the decade.

Micron's grand designs include building 40% of its DRAM output in the U.S. by sometime in the 2040s. To achieve this, Micron plans to invest around $150 billion in six fab modules in America through 2045, as well as spending an additional $50 billion on R&D over the same period. However, it is unclear whether $50 billion will be spent exclusively in the U.S. (which will be a major boost for the American semiconductor industry) or whether it will be spread across Micron's R&D facilities across the world.

In any case, Micron's groundbreaking ceremony in New York next week will mark a key landmark in the company's expansion in the U.S. as well as a major milestone in the broad return of DRAM production to America.

During a CES 2026 Q&A in Las Vegas, Nvidia CEO Jensen Huang was thrown a bit of a curveball. With SRAM-heavy accelerators, cheaper memory, and open weight AI models gaining traction, could Nvidia eventually ease its dependence on expensive HBM and the margins that come along with it?

During an Analyst Q&A at CES 2026, captured by SemiAnalysis / Ray Wang, what Huang responded with was not a roadmap away from HBM, nor an endorsement of leaner, cheaper inference hardware. Instead, he laid out his view of AI workloads as inherently unstable, constantly reshaped by new model architectures, new modalities, and new deployment patterns. Against that backdrop, he suggested, efficiency gains achieved by tuning hardware for a single problem tend to be short-lived.

Why SRAM looks attractive

Let’s take a step back for a moment and consider what Huang is getting at here. The industry, by and large, is actively searching for ways to make AI cheaper. SRAM accelerators, GDDR inference, and open weight models are all being pitched as pressure valves on Nvidia’s most expensive components, and Huang’s remarks are a reminder that while these ideas work in isolation, they collide with reality once they’re exposed to production-scale AI systems.

Huang did not dispute the performance advantages of SRAM-centric designs. In fact, he was explicit about their speed. "For some workloads, it could be insanely fast," he said, noting that SRAM access avoids the latency penalties of even the fastest external memory. "SRAM’s a lot faster than going off to even HBM memories."

This is why SRAM-heavy accelerators look so compelling in benchmarks and controlled demos. Designs that favor on-chip SRAM can deliver high throughput in constrained scenarios, but they run up against capacity limits in production AI workloads because SRAM cannot match the bandwidth-density balance provided by HBM, which is why most modern AI accelerators continue to pair compute with high-bandwidth DRAM packages.

But, Huang repeatedly returned to scale and variation as the breaking point. SRAM capacity simply does not grow fast enough to accommodate modern models once they leave the lab. Even within a single deployment, models can exceed on-chip memory as they add context length, routing logic, or additional modalities.

The moment a model spills beyond SRAM, the efficiency advantage collapses. At that point, the system either stalls or requires external memory, at which point the specialized design loses its edge. Huang’s argument was grounded in how production AI systems evolve after deployment. "If I keep everything on SRAM, then of course I don’t need HBM memory," he said, adding that "...the problem is the size of my model that I can keep inside these SRAMs is like 100 times smaller."

Workloads that refuse to stay still

Some of Huang’s more revealing comments came when he described how modern AI workloads behave in the wild. "Workloads are changing shape all the time," he said. "Sometimes you have MOEs. (Mixture of Experts). Sometimes you have multimodality stuff. Sometimes you’ve got diffusion models. Sometimes you have autoregressive models. Sometimes you have SSMs. (Sequential Server Management)"

Each of those architectures stresses hardware differently. Some are memory-bound, while others push interconnect bandwidth. Some demand low latency, while others tolerate batching and delay. "These models are all slightly different in shape and size," Huang summed it up bluntly. More importantly, those pressures shift dynamically. "Sometimes they move the pressure on the NVLink. Sometimes they move the pressure on HBM memory. Sometimes they move the pressure on all three," he said.

This is the core argument supporting Nvidia’s case for flexibility. A platform optimized narrowly for one memory pattern or execution model risks leaving expensive silicon idle when the workload changes. In shared data centers, where utilization across weeks and months determines whether it’s economically viable, that is a serious liability.

"You might be able to take one particular workload and push it to the extreme," Huang said. "But that 10% of the workload, or even 5% of the workload, if it’s not being used, then all of a sudden that part of the data center could have been used for something else." In other words, Huang is arguing that peak efficiency on a single task matters less than consistent usefulness across many.

Open models still run into memory limits

The original question also touched on open AI models and whether they might reduce Nvidia’s leverage over the AI stack. The suggestion was that open models, combined with SRAM-heavy designs and cheaper memory, could reduce reliance on Nvidia’s most expensive GPUs and improve margins across the stack.

While Huang has praised open models publicly and Nvidia has released its own open weights and datasets, his CES remarks made clear that openness does not eliminate infrastructure constraints. Training and serving competitive models still require enormous compute and memory resources, regardless of licensing. Open weights do not eliminate the need for large memory pools, fast interconnects, or flexible execution engines; they just change who owns the model.

This is important because many open models are evolving rapidly and, as they incorporate larger context windows, more experts, and multimodal inputs, their memory footprints will grow. Huang’s emphasis on flexibility applies here as well; supporting open models at scale does not reduce the importance of HBM or general-purpose GPUs. In many cases, it increases it.

The implication is that open source AI and alternative memory strategies are not existential threats to Nvidia’s platform. They are additional variables that increase workload diversity. That diversity, in Nvidia’s view, strengthens the case for hardware that can adapt rather than specialize.

Why Nvidia keeps choosing HBM

Ultimately, Huang’s CES comments amount to a clear statement of priorities. Nvidia is willing to accept higher bill of materials costs, reliance on scarce HBM, and complex system designs because they preserve optionality. That optionality protects customers from being locked into a narrow performance envelope and protects Nvidia from sudden shifts in model architecture that could devalue a more rigid accelerator lineup.

This stance also helps explain why Nvidia is less aggressive than some rivals in pushing single-purpose inference chips or extreme SRAM-heavy designs. Those approaches can win benchmarks and attract attention, but they assume a level of workload predictability that the current AI ecosystem no longer offers.

Huang’s argument is not that specialized hardware has no place. Rather, it is that in shared data centers, flexibility remains the dominant economic factor. As long as AI research continues to explore new architectures and hybrid pipelines, that logic is unlikely to change.

For now, Huang seems confident that customers will continue to pay for that flexibility, even as they complain about the cost of HBM and the price of GPUs. His remarks suggest the company sees no contradiction there. That view may be challenged if AI models stabilize or fragment into predictable tiers, but, right now, Huang made it clear that Nvidia does not believe that moment has arrived yet.

Micron has secured another major vote of confidence from the Taiwanese government, winning approval for an additional NT$4.7 billion (approximately $149 million) in subsidies to expand HBM research and development in Taiwan. Combined with an earlier grant awarded in 2021, the total public support now approaches NT$10 billion, or roughly $318 million, making Micron the largest single recipient of Taiwan’s flagship industrial R&D subsidies to date.

The funding, approved by Taiwan’s Ministry of Economic Affairs under its A+ Corporate Innovation and R&D Enhancement program, supports a three-year project running from November 2025 through October 2028. Micron’s total budget for the effort is NT$11.75 billion, with the company covering close to 60% of the cost itself.

The goal is to develop and industrialize leading-edge, high-performance, and HBM technologies in Taiwan at a time when HBM has become one of the most strategically constrained components in the global semiconductor supply chain, driven by surging demand from AI accelerators, data center GPUs, and high-performance computing systems.

HBM as strategic infrastructure

HBM has shifted from a niche technology used in a handful of HPC accelerators into a foundational element of modern AI hardware. Unlike conventional DDR or GDDR memory, HBM stacks multiple DRAM dies vertically and connects them through an ultra-wide interface, delivering orders of magnitude more bandwidth per watt.

Current HBM3E stacks already provide several terabytes per second of bandwidth, feeding GPUs such as Nvidia’s H200 and AMD’s MI300X without becoming a performance bottleneck. Next-gen HBM4 pushes this further by doubling the interface width to 2048 bits and supporting taller stacks with higher-density dies. In doing so, HBM4-class memory enables larger models, faster training, and more efficient inference without relying solely on brute-force compute scaling.

This, unfortunately, has also turned HBM into a choke point. AI accelerators cannot ship without it, and the ability to produce advanced HBM at scale now directly limits how many high-end GPUs can reach the market. As a result, memory suppliers have found themselves graduating from being interchangeable commodity vendors to fully-fledged semiconductor industry behemoths whose roadmaps influence the entire AI hardware ecosystem.

That explains why governments are increasingly willing to subsidize HBM development, and Taiwan’s Ministry of Economic Affairs has been explicit that memory is the missing pillar in its semiconductor ecosystem. The island already dominates advanced logic manufacturing and chip design, but has historically relied on foreign suppliers for cutting-edge memory technology. Supporting Micron’s HBM R&D is a way to anchor that capability locally.

Micron’s position

Micron is the smallest of the three major HBM suppliers by volume, behind SK hynix and Samsung, but it has rapidly closed the tech gap over the past two product generations. Micron’s HBM3E has already been qualified by major accelerator vendors, and the company has publicly stated that its HBM capacity for 2026 is fully booked. The company recently made headlines when it axed its Crucial consumer business to enable the company to focus more on producing HBM and storage devices to feed the AI beast.

SK hynix, meanwhile, currently holds a dominant share of the HBM market and has reportedly committed much of its near-term output to Nvidia, while Samsung is investing aggressively to regain momentum, pushing 12-layer HBM3E and preparing for HBM4 transitions. In that environment, Micron’s ability to accelerate development, improve yields, and bring new stacks to volume production earlier than rivals could materially affect market share.

Taiwan also offers Micron more than just money. The subsidy requires R&D to be conducted locally and encourages collaboration with Taiwanese companies, particularly in equipment, materials, and advanced packaging. That’s key because HBM development is not just about DRAM cell design but also involves through-silicon vias, wafer bonding, thermal management, and increasingly complex interposer and advanced packaging technologies.

Taiwan’s semiconductor supply chain is exceptionally strong in these areas. Locating HBM R&D there shortens feedback loops between design, process development, and manufacturing equipment vendors, which can translate into faster iteration cycles and more predictable ramp-ups.

Implications for Taiwan’s semiconductor industry

From Taiwan’s perspective, this deal is about long-term positioning, with the government estimating that Micron’s DRAM and HBM investments could generate more than NT$800 billion in domestic output value and create over 20,000 direct and indirect jobs. Those numbers are ambitious, but they underscore the scale of economic activity tied to advanced memory manufacturing.

More importantly, HBM development reinforces Taiwan’s role as a system-level semiconductor hub rather than a single-node manufacturing center. As AI hardware becomes more tightly integrated across logic, memory, and packaging, geographic clusters that can support all three gain an advantage. Taiwan already hosts the world’s most advanced logic fabs and a dense network of OSAT and materials suppliers. Adding leading-edge memory R&D to that mix strengthens the entire ecosystem.

There are also the obvious geopolitical elements. Memory supply has become a major concern for both the United States and its allies, particularly as AI hardware increasingly underpins economic and military capabilities. Micron is the only major U.S.-headquartered DRAM manufacturer, so supporting its advanced R&D footprint in Taiwan aligns with broader efforts to diversify and secure critical semiconductor supply chains without concentrating all leading-edge development in a single country.

Ultimately, this subsidy reinforces the broader industry consensus that the HBM shortage is not a transient problem that’s going to resolve itself or be resolved through incremental capacity additions. There’s a serious, structural deficit driven by sustained AI demand and increasing technical complexity — moving from HBM3E to HBM4 brings all new architectures and closer integration with advanced packaging — and the only way to solve it is through solid, calculated R&D; government-backed funding can meaningfully influence how quickly the necessary transitions occur.

By offsetting some of the risk and cost, Taiwan is effectively saying that it thinks HBM will remain a bottleneck well into the second half of the decade, and that it’s a bottleneck worth solving.

From memory shortages to rising GPU prices, 2025 seemed like a year of significant scarcity in the supply chain for all things semiconductors. But what does the future hold for this super-tight market in the years to come?

One school of thought suggests that in a couple of years, the story goes, today’s hyperscaler accelerators will spill out into the secondary market in a crypto-style deluge. Cheap ex-A100s and B200s – which could be considered AI factory cast-offs –will suddenly become available for everyone else looking to buy.

Data center hardware is often assumed to have a finite and sometimes short lifecycle, with depreciation schedules and refresh cycles that push older hardware into uselessness after a few years. But another group suggests AI compute doesn’t behave like a consumer GPU market, and the ‘three years and it’s done’ assumption is shakier than many people want to admit. As Stacy Rasgon, managing director and senior analyst at Bernstein, said in an interview with Tom’s Hardware Premium, the idea that “they disintegrate after three years, and they're no good, is bullshit.”

Some believe the present tightness in the market isn’t just a temporary crunch but is more a structural condition of the new post-AI norm in the market, with a closed loop where state-of-the-art hardware circulates between a handful of cloud and AI giants.

So what’s the reality? Ben Bajarin, an analyst at Creative Strategies, describes the current moment as a “gigacycle” rather than another chip boom. In his modelling, global semiconductor revenues climb from roughly $650 billion in 2024 to more than $1 trillion by the end of the decade. “There’s some catch-up necessary, but there’s also the fact that the semiconductor industry remains relatively conservative, because they are typically cyclical,” Bajarin said in an interview with Tom’s Hardware Premium. “So everybody’s very concerned about overcapacity.”

That conservatism matters because chipmaking capacity takes time, effort, and a lot of money to stand up and bring online. It’s for that reason that we’re likely to see tightness in the market remaining for a little while yet: demand is spiking, yes, but companies aren’t that keen to stand up their supply until they can absolutely guarantee a return. “They don’t want to be stuck with foundry capacity or supply capacity that they can’t use seven or eight years from now,” Bajarin said.

Looking at the numbers

According to Bajarin’s analysis, AI chips represented less than 0.2% of wafer starts in 2024, yet already generated roughly 20% of semiconductor revenue – a huge concentration on a single space, which helps explain why the shortages feel different from the pandemic-era GPU crunch. In 2020 and 2021, consumer demand surged, and supply chains seized up, but the underlying products were still relatively mass market in manufacturing terms. But today’s AI accelerators require leading-edge logic, exotic memory stacks, and advanced packaging.

It’s possible to make more of them, but not quickly, and not without knock-on effects. “If you look at the forecasts for wafer capacity or substrate capacity, nobody's scaling up,” cautions Bajarin.

Rasgon told us that while not everything is tight, the “really tight” parts of the system are concentrated in memory. Rasgon pointed to Micron, one of the three global DRAM giants, which has said memory tightness could persist beyond 2026, driven in large part by AI demand and High Bandwidth Memory (HBM). It’s notable that Micron recently closed down its consumer-facing business, Crucial, to focus on the more lucrative products it can sell – and markets it can sell into.

HBM is a different manufacturing and packaging challenge that can hoover up production capacity. HBM production consumes far more wafer resources than standard DRAM, according to Rasgon – so much so that producing a gigabyte of HBM can take “three or four times as many wafers” as producing a gigabyte of DDR5, which means shifting capacity into HBM effectively reduces the total number of DRAM bits the industry can supply.

Memory makers prioritising HBM for accelerators doesn’t just affect hyperscalers. It has a knock-on effect on PCs, servers, and other devices when standard DRAM is tighter and pricier than it would otherwise be, which is why companies have been pushing up prices for consumer hardware in recent weeks and months. Hyperscalers can often swallow higher component costs because they monetise the compute directly, whether through internal workloads or rented-out inference. Everyone else tends to feel the squeeze more immediately: OEMs and system builders face higher bill-of-materials costs and retail pricing changes for the worse if you’re an end customer.

Bajarin believes HBM will be one of the defining constraints of the remainder of the decade, projecting it to grow fourfold to more than $100 billion by 2030, while noting that HBM3E can require about three times the wafer supply per gigabyte compared with DDR5. But he’s not alone in thinking that: Micron has even talked about being unable to meet all demand from key customers, suggesting it can supply only around half to two-thirds of expected demand, even while raising capex and considering new projects.

Where’s the bottleneck?

There are a number of reasons for the current tightness, but even if the market had infinite wafers and infinite memory, it could still run into a chokepoint: advanced packaging.

The industry has been ramping up its CoWoS (chip-on-wafer-on-substrate) capacity aggressively, but it has also been unusually open about how hard it is to get ahead of demand. In early 2025, Nvidia CEO Jensen Huang said overall advanced packaging capacity had quadrupled in under two years but was still a bottleneck for the firm.

It’s not just Nvidia that is reckoning with the challenge. TrendForce, which tracks the space closely, has projected TSMC’s CoWoS capacity rising to around 75,000 wafers per month in 2025 and reaching roughly 120,000 to 130,000 wafers per month by the end of 2026. Such growth is a big leap – but it’s also unlikely to loosen current capacity constraints.

Bajarin highlighted the reason why in his analysis: CapEx by the top four cloud providers — Amazon, Google, Microsoft, and Meta — doubled to roughly $600 billion annually in just two years. Rasgon noted that some companies can wind up supply-constrained for reasons that have nothing to do with leading-edge demand being “off the charts.” In Intel’s case, he argued, it’s partly about where demand is versus where capacity has been cut. “They were actually scrapping tools in that older generation and selling them off for pennies on the dollar,” he said.

‘Forecasts’ versus ‘guesses’

Although it may seem obvious that demand will continue to grow because of the way the big tech companies are splashing the cash, it can be difficult to accurately forecast future demand because of the way the chip market works.

Rasgon said semiconductor companies sit “at the back of the supply chain,” which limits their ability to see end demand clearly, and encourages behaviour that makes the signal noisier. It’s a vicious circle exacerbated when supply is particularly tight and lead times stretch because customers start hoarding the chips they have and double-ordering new options, because they’re trying to secure parts from anywhere they can.

That can make demand look artificially huge until lead times ease and cancellations begin. Suppliers want to avoid being caught by sizing their production for a demand that doesn’t materialize. “Forecasting in semiconductors in general is an unsolved problem,” Rasgon said. “My general belief is that most, or frankly all, semiconductor management’s actual visibility of what is going on with demand is precisely zero.”

Bajarin points out that the industry works “very methodically” because it remembers boom-bust cycles, especially in memory. “We’re just going to have to live in a foreseeable cycle of supply tightness because of these dynamics that are historically true of the semiconductor industry,” he said.

When the market unwinds itself, and supply normalizes is, as is befitting a market that struggles with forecasting, impossible to tell. “As long as we're in this cycle where we're really building out a fundamental new infrastructure around AI, it's going to remain supply-constrained for the foreseeable future, if not through this entire cycle, just because of prior boom-bust cycles within the semiconductor industry,” said Bajarin.

But even if the industry could magically print accelerators, it still needs somewhere to run them. Data centers take time to build, to connect to power, and to cool at scale. “Even if we make all these GPUs, we can’t really house them because we don’t have the gigawatts,” said Bajarin. However, with the planning of small modular reactors and the expansion of electricity grids, there’s hope that the need can be met – eventually.

Chip cycles and recycling

Another potential unblocker of the tight market is the opportunity to reuse older generation GPUs as they enter the traded market, thanks to newer generations of chips constantly being cycled through big tech companies who want the cutting edge for inference and training.

Already, you can find small volumes of older data center GPUs on the market, including listings for Nvidia A100-class hardware through resellers and brokers. But the odd older generation of chips existing is a world away from a crypto-style glut of former AI accelerators appearing on the market. While AI firms want all the cutting-edge chips they can get their hands on, they’re not necessarily disposing of their older stock, either. A top-end accelerator isn’t a consumer graphics card that becomes obsolete in a couple of years. It’s capital equipment. And AI companies are learning to sweat it.

OpenAI CFO Sarah Friar underlined this in November, admitting that OpenAI still uses Nvidia’s Ampere chips – released in 2020 – for inference on its consumer-facing models. Training might use bleeding-edge tech, but inference can profitably run on older generations for a long time. If OpenAI is thinking that way, so too will other companies in the space. “Absolutely, the older stuff is still being used,” says Rasgon. “And in fact, not only is it being used, it's being used very, very profitably.”

For now, the clearest takeaway is that the current tightness isn’t only about making more chips. It’s about whether the industry can build enough of everything around them – including the buildings, cooling, and grid connections to run them – fast enough to match demand that’s still accelerating. “We're going to remain in a relative supply constraint across all of these vectors until either we've built the entire thing out and we have enough compute,” Bajarin said, “or it's a bubble, and it crashes.”

SK hynix is bringing its HBM ambitions to U.S. soil with a $3.9 billion plan to build its first domestic manufacturing facility — a 2.5D advanced packaging plant in West Lafayette, Indiana. The site, developed in partnership with Purdue University, is aimed at producing turnkey HBM modules for AI accelerators by 2028.

In bringing manufacturing to the U.S., SK hynix aims to vertically integrate its HBM supply chain, which currently relies on outside firms, to handle the delicate process of mounting HBM stacks to logic dies via interposers. That assembly work is increasingly a bottleneck for high-end GPUs, especially as demand for AI silicon continues to outstrip available packaging capacity. By owning both the memory and the interconnects, the company is clearly trying to target the kind of leverage that TSMC holds over the likes of Nvidia.

The Indiana facility, which is partially funded by $458 million in CHIPS Act grants and loans, is SK hynix’s first U.S. production site and part of a broader effort to bring critical semiconductor infrastructure closer to U.S. customers. It comes alongside the company’s new R&D outpost near Seattle, built in part to deepen ties with Nvidia and other domestic hyperscalers. When complete, the plant will package HBM chips with silicon interposers and integrate them with partner dies into a single, thermally optimized 2.5D module ready for use in AI servers or supercomputing clusters.

According to filings, SK hynix plans to operate a full mass-production line at the site, supported by a dedicated talent pipeline from Purdue. That puts it in direct competition with TSMC’s CoWoS platform, which has been the de facto standard for high-end HBM packaging since Nvidia’s Pascal era. And with TSMC’s CoWoS capacity effectively sold out through 2027, customers are already searching for alternatives.

Turnkey HBM

The biggest problem with HBM is that it’s a packaging challenge. HBM stacks multiple memory dies vertically using through-silicon vias (TSVs), all of which must be mounted on a large interposer next to a host processor. That assembly must account for thermal expansion, routing complexity, and thousands of microbumps, resulting in a tightly coupled chiplet module with massive I/O bandwidth and low power draw, ideal for AI training or HPC workloads.

Until now, HBM suppliers like SK hynix and Samsung have typically sold raw memory stacks, leaving GPU vendors to rely on foundry partners for packaging. Nvidia’s H100 and AMD’s MI300X, for instance, use HBM2e and HBM3 mounted via TSMC’s CoWoS process. But with demand for accelerators reaching historic highs — and HBM4 promising even more aggressive stack designs — the need for in-house packaging has become a priority.

SK hynix’s stated goal is to deliver a “turnkey” solution: HBM stacks already integrated with silicon interposers and, potentially, host dies from customers. That would allow hyperscalers or chip designers to skip TSMC entirely for final assembly, receiving ready-to-mount modules instead. It’s a fundamental shift in how HBM enters the supply chain, positioning SK hynix as a full-stack supplier rather than a component vendor.

This type of setup already has some precedent. TSMC has steadily expanded its role from foundry to integrator over the past decade, using its packaging platforms (CoWoS, InFO, SoIC) to create customer lock-in beyond wafer fabrication. SK hynix has obviously drawn inspiration from that playbook, starting from memory and working outward. It also places pressure on Samsung, which is reportedly evaluating its own U.S. packaging line to support future Tesla and AMD accelerator deployments.

The timing of all this is no accident. HBM demand is projected to be worth tens of billions by 2030, driven by AI model scaling and architectural shifts toward high-bandwidth chiplets. Nvidia’s Rubin platform — expected to launch late 2026 — will reportedly use HBM4E stacks with bandwidth over 1.2TB/s per module. That’s not achievable with standard memory interfaces or traditional DRAM. If SK hynix can offer a high-yield, pre-packaged solution at scale, it could become indispensable to Nvidia, AMD, and a whole host of other companies.

A national security priority

Advanced packaging has become a national security and industrial policy priority, and Washington is funding it accordingly. The Commerce Department’s CHIPS for America program set aside about $3 billion for the National Advanced Packaging Manufacturing Program to expand U.S. advanced packaging R&D and manufacturing capacity under the Biden administration in 2023, which remains under the incumbent Trump administration.

While TSMC and Intel have U.S. fabs in Arizona and Ohio, neither currently offers high-volume 2.5D packaging on U.S. soil. SK hynix's Indiana site will be the first to do so, and potentially the only one with dedicated capacity for third-party AI customers.

See, packaging isn't just about putting chips together; it determines yield, thermals, and even chip performance. Interposer routing, power distribution, and heat dissipation can make or break an accelerator, so by integrating packaging with memory production, SK hynix gains tighter control over thermal budgets and mechanical tolerances, reducing failure rates in multi-die modules. And by keeping the entire process in the U.S., it aligns with growing customer pressure to onshore AI supply chains.

TSMC is responding in kind, expediting its own Arizona-based packaging line with a target completion window of late 2027. But TSMC’s U.S. capacity is largely reserved for captive customers and its own Arizona wafer output. That leaves a vacuum for customers seeking independent packaging partners. SK hynix could fill that role, especially if its turnkey model proves more predictable than the fragmented flow of wafers and memory stacks across the Pacific.

This isn’t just about fabs versus packages, but rather, where the AI supply chain converges. The raw silicon may still come from Korea or Taiwan, but the modules that power large language models, scientific simulations, and autonomous workloads will increasingly be built where the compute is deployed. And if SK hynix succeeds, it won’t be the last memory maker to cross the packaging threshold.

Micron has already announced plans to co-locate HBM packaging at its Virginia site. Meanwhile, Samsung is in talks with U.S. partners, and Intel is integrating Foveros with its own internal chiplets. But with a turnkey HBM solution provided via a 2.5D packaging plant in the U.S., SK hynix could easily take the lead.

SK hynix is expanding its U.S. presence with a new office in the Seattle metropolitan area, placing the world’s leading HBM supplier within minutes of Nvidia, Amazon, and Microsoft.

According to industry sources cited by DigiTimes, the South Korean company has leased approximately 5,500 square feet at City Center Bellevue, just east of Seattle. While modest in size, the location and timing make the expansion far more consequential than a routine regional office opening.

SK hynix is sitting squarely in the middle of the ongoing AI hardware boom, supplying the majority of the HBM used in Nvidia’s data center accelerators and increasingly serving hyperscale customers building their own AI silicon. Establishing a physical foothold in the Pacific Northwest puts the company closer to the customers driving its fastest-growing and most important business.

Moving closer to the center of AI development

SK hynix has spent the past two years transforming itself from a cyclical commodity DRAM vendor into a leading supplier for AI infrastructure. That's visible in High Bandwidth Memory (HBM), where SK hynix was first to mass-produce HBM3 and has remained ahead in yield and volume as customers transitioned from HBM2E. In August 2025, the company overtook Samsung in global DRAM revenue for the first time, a change largely attributed to HBM shipments for Nvidia’s H100 and H200 AI accelerators.

Seattle and its surrounding suburbs have become one of the densest concentrations of the AI industry ecosystem outside of Silicon Valley. While Nvidia maintains a significant engineering presence in the region, Amazon’s AWS Skills Center is based nearby, and Microsoft’s Azure silicon and systems groups are spread across Redmond and Bellevue.

HBM is not a plug-and-play component. Memory stacks are co-designed with GPUs and AI accelerators, and performance, power, and reliability targets are refined through repeated cycles of joint validation. Physical proximity will allow faster iteration when issues arise, whether related to signal integrity, thermals, or packaging tolerances.

Amazon’s recent launch of its Trainium3 AI accelerator, which integrates 144GB of HBM3E, shows how quickly hyperscalers are increasing their reliance on stacked memory. Microsoft and Google are following similar paths with custom accelerators. Each of those programs depends on close coordination between the silicon designer and the memory supplier; a Seattle-area office gives SK hynix a seat at that table.

Broader U.S. localization

The Bellevue office also fits into a wider U.S. expansion that extends beyond customer support. In 2024, SK hynix announced plans for a $4 billion advanced packaging facility in Indiana, its first manufacturing investment in the United States. That site is intended to handle advanced HBM packaging and testing, with production targeted for 2028. While the Indiana project focuses on manufacturing, it appears the Seattle office will be oriented toward R&D, applications engineering, and customer engagement.

Taken together, these moves suggest SK hynix is deliberately localizing more of its AI-related operations in the U.S., where the bulk of demand is being generated. Advanced packaging has become as important as wafer fabrication for AI accelerators, and the ability to package memory near customers reduces both logistical complexity and geopolitical risk. It also aligns with U.S. industrial policy aimed at securing domestic supply chains for critical technologies.

There has been periodic speculation that SK hynix could eventually build a full DRAM fab in North America, though the company has not committed publicly to such a plan. Even without a fab, expanding engineering and packaging capabilities in the U.S. strengthens SK hynix’s position with American customers at a time when memory supply is extremely tight, thereby making long-term capacity planning a competitive differentiator.

The race to HBM4

SK hynix’s Seattle expansion also reflects intensifying competition in the HBM market. Samsung remains a formidable rival with deep manufacturing resources and its own U.S. investments, including advanced packaging capacity tied to its Texas operations. Micron, the sole U.S.-based DRAM manufacturer, is pursuing high-end server and automotive markets and sampling its own HBM4 designs, though its near-term capacity expansion is more constrained.

Meanwhile, SK hynix has already completed development of HBM4 and is understood to have delivered samples to Nvidia. Early engagement is particularly important with HBM4 because the transition involves higher stack counts, tighter power budgets, and more complex thermal challenges. Winning those designs early could easily lock in multi-year supply relationships.

Ultimately, with its Seattle expansion, SK hynix is telling Samsung and Micron that it intends to be embedded in the AI ecosystems of its largest customers rather than a distant component supplier. Being physically close to Nvidia and Amazon’s engineering teams during the transition to HBM4 will greatly improve the company’s odds of maintaining its lead, while also supporting adjacent efforts such as joint work with Nvidia on AI-optimized SSDs.

As we detailed in October, the memory and storage markets were expected to face significant supply constraints. In the months following, DDR5 prices have skyrocketed. Kits that had sold comfortably below $100 months earlier were surging into the hundreds by early December. CyberPowerPC, one of the largest system builders in North America, warned in November that contract DRAM prices had jumped 500% since early October, while another report indicated that they had risen 171.8% year over year.

The disruption is being driven not by demand from gamers or device makers, but by artificial intelligence. Specifically, by the way hyperscalers are soaking up wafer starts and advanced packaging lines for high-bandwidth memory (HBM) used in AI accelerators.

The consequence is that anyone or anything that’s not part of that ecosystem — PC builders, laptop OEMs, and even phone makers — is fighting over the scraps of a shrinking pool of commodity DRAM. And with new fabs still years from coming online, the shortage is expected to last well into the second half of the decade.

RAM kits up 2-4x in a quarter

The most visible impact can be seen with retailers. A Corsair Vengeance DDR5-6000 32GB (2x16GB) cost $134.99 in September before reaching more than $420 in early December. G.Skill, TeamGroup, and Kingston all adjusted channel pricing by double digits across Q3, citing tightening availability.

That translated to a whopping 2-4x price swing for enthusiasts buying RAM, depending on capacity and bin. The G. Skill Trident Z5 Neo RGB DDR5-6000, our tried-and-tested best RAM for gaming, is now only available via third-party sellers through Amazon, with 64GB kits attracting prices beyond $500, with one seller listing them for a whopping $881.87 as of December 18.

Suppliers and distributors have sharply tightened allocation of DDR5 memory, with some channel partners reporting severely limited quotes and rollovers on orders as capacity is diverted to AI-driven demand. Reports from Taiwan and broader market tracking show memory modules selling out or being bundled to secure placement, indicating that mainstream buyers are being deprioritized.

That squeeze quickly filtered into GPUs, where memory is a major cost driver. AMD board partners raised card prices by around $10 per 8GB of VRAM starting in November. A rumor suggests AMD could increase Radeon RX graphics card prices with 8 GB models up by about $20 and 16 GB models up by about $40 in response to climbing GDDR6 spot prices and memory costs. SSD pricing has also reversed direction. 2TB Gen 4 NVMe drives that had been available for $80 in the summer were back at $130 by November. Contract pricing on NAND rose 60% in November, and module-level spot prices followed.

Framework was forced to increase its pricing on the DDR5 memory configurable in Framework Laptop DIY Edition orders by 50% in response to “substantially higher costs” they are facing from suppliers and distributors. Meanwhile, Dell and HP have both flagged component pricing, with HP’s CEO stating that memory costs in particular were affecting margin on consumer systems and Dell COO Jeff Clarke saying he’s “never seen memory-chip costs rise this fast.” Raspberry Pi also raised prices on its 4GB and 8GB boards, citing supply constraints on LPDDR4X.

Starving the market

All this is not being caused by insufficient demand for DDR5, but by wafer and packaging capacity being redirected to high-margin, high-volume AI parts. High Bandwidth Memory (HBM) is the main pressure point.

HBM differs from conventional DRAM in both structure and cost. Instead of planar dies mounted on PCBs, HBM stacks multiple DRAM dies vertically, linked with through-silicon vias (TSVs), and mounts them on an interposer alongside compute logic. These stacks offer enormous bandwidth and proximity advantages for AI accelerators, but they are incredibly expensive in terms of materials, tooling requirements, and especially wafer area.

Each gigabyte of HBM consumes roughly three times the wafer capacity of DDR5. That reflects both yield loss from stacking and the fact that many DRAM dies in HBM stacks are smaller or binned lower than equivalent RAM sticks. The TSV process and wafer thinning introduce additional steps that lengthen production cycles, leading to a catch-22 situation that, even when yields are strong, means the vertical integration of HBM requires advanced packaging lines that remain globally scarce.

SK hynix, the largest supplier of HBM to Nvidia, has told investors that its advanced packaging lines are at capacity through 2026. Micron, which supplies HBM3E to Nvidia and other U.S. clients, is in a similar position. Samsung has HBM capacity reserved through its Foundry and Memory business lines for tier-1 cloud clients. These lines are not interchangeable with conventional DRAM; tools, masks, and equipment for HBM production occupy space that would otherwise produce DDR5 or LPDDR5.

With wafer starts flat and packaging lines locked, every wafer pushed into HBM removes capacity from commodity DRAM and NAND. And the volume committed to AI is enormous.

The Stargate effect

In July, OpenAI and Microsoft finalized plans for Project Stargate, a multi-site hyperscale AI infrastructure program, with Samsung and SK hynix together committing to up to 900,000 DRAM wafer starts per month to support the buildout “at an accelerated capacity,” according to OpenAI.

That deal alone represents roughly 35-40% of global DRAM wafer capacity. The wafers will be used not just for HBM stacks, but for LPDDR and ECC DRAM used in adjacent server memory. Nvidia has its own multi-year agreements in place, reportedly accounting for the majority of SK hynix's HBM output through 2026. These allocations are inflexible, with contracts fixed, volumes tiered, and, in many cases, wafers fronted at favorable prices in exchange for capacity guarantees.

Naturally, memory vendors are reaping the rewards. Micron posted a record $11.3 billion quarter in Q4 2025, driven by HBM and enterprise DRAM margins. It subsequently announced plans to exit the Crucial consumer brand by early 2026. Executives stated that winding down Crucial would free up wafer supply for strategic accounts. “The AI-driven growth in the data center has led to a surge in demand for memory and storage. Micron has made the difficult decision to exit the Crucial consumer business... to improve supply and support for our larger, strategic customers in faster-growing segments,” said Sumit Sadana, EVP and Chief Business Officer.

Meanwhile, Samsung has raised its memory chip prices by up to 60% since September, driven almost entirely by the high demand for building new AI-focused data centers.

No new fabs until 2027

Memory makers are responding to surging demand by building new fabs, but the leadtime on greenfield facilities is long, and — of course — virtually all capex is going to go to HBM lines first, because that’s where the money is. Among the most notable investments is Micron’s $9.6 billion Hiroshima HBM facility, announced in partnership with the Japanese government. Construction of this fab is expected to begin around May 2026, with its first output expected in 2028.

Samsung is also committing billions in investments to new DRAM capacity in Pyeongtaek and Taylor, Texas. These sites will include HBM packaging and DRAM wafer lines, but company executives have cautioned that HBM and high-margin enterprise DRAM will receive priority through 2027. The company recently accelerated Phase 4 of its Pyeongtaek expansion in a renewed effort to reclaim leadership in the AI memory space. As for SK hynix, it’s expanding in Icheon — where it was the first memory maker to assemble ASML’s High-NA EUV lithography system — and Cheongju, with new DRAM output slated for 2026-2027.

Unfortunately, there’s no sign of the DRAM supply shortfall easing up before 2027. TrendForce has cautioned time and time again that capacity growth is limited relative to demand as AI and server requirements absorb a disproportionate share of wafer starts and production resources, and its pricing outlook shows DRAM contract rates continuing to rise through 2026 with constrained supply conditions. A leaked SK hynix internal analysis forecasts PC DRAM supply trailing demand until at least late 2028, and general industry consensus is that meaningful relief for DDR5 and LPDDR5 supply — at prices suitable for consumer SKUs — will not come until 2028 or 2029 at the earliest.

The situation is similar for NAND, with wafer investment having lagged since the 2022 price collapse. Most new NAND lines being built now are intended for enterprise SSDs or embedded memory for AI accelerators; client SSDs will see higher costs and tighter supply through 2026.

Consumers are now an afterthought

The effects of this reallocation are already locked into the next generation of consumer hardware, because memory decisions are made years in advance. Laptop platforms shipping in 2026 and 2027 are being finalized now, at a moment when DRAM and NAND supply are both constrained and volatile. That will have consequences that go beyond pricing.

Memory configurations that require large, predictable allocations of DRAM are riskier in a market where suppliers are prioritizing AI contracts and spot pricing can move sharply month to month. These conditions favor fewer SKUs and longer reuse of validated configurations rather than aggressive spec increases. Meanwhile, DDR4, which was expected to fade out quickly after DDR5 adoption ramped, is now likely to persist far longer in entry-level and midrange systems simply because its supply chain is more stable.

GPUs are in a similar bind. GDDR6 and GDDR6X, while not interchangeable with HBM, compete for wafer capacity and backend resources at the same suppliers. That makes large VRAM increases expensive at exactly the moment when software requirements are rising. This won’t necessarily mean fewer GPUs, but it could lead to slower movement at the top end of memory configurations, with vendors prioritizing yield and availability over aggressive capacity scaling.

NAND is under comparable pressure as investment in new client-focused capacity slowed down after oversupply and subsequent pricing collapse in 2022. What capacity is coming online is disproportionately aimed at enterprise SSDs and embedded storage tied to accelerators, where margins are higher, and contracts are longer. That leaves consumer SSDs exposed to price swings and supply tightening, particularly at higher capacities, even as PCIe generations continue to advance.

The problem is that none of these pressures are transient shocks that can clear in a quarter or two. The fabs that would meaningfully expand DRAM and NAND supply are not scheduled to come online until 2027 at the earliest, and even then, priority will remain with HBM and enterprise products. Consumer markets are now an afterthought among memory makers and, given the current state of AI, it’s difficult to see that changing.

Cambricon Technologies is preparing one of the most aggressive production ramp-ups attempted by a Chinese AI chipmaker. According to figures reported by Bloomberg, the company aims to deliver around 500,000 AI accelerators in 2026, including as many as 300,000 units of its Siyuan 590 and 690 processors.

These numbers, which would more than triple Cambricon’s 2025 output, come at a time when Chinese companies are reworking their hardware roadmaps in response to U.S. export controls and general geopolitical uncertainty. It also brings the limits of China’s current semiconductor manufacturing capabilities to the surface, which will play as large a role in market demand in determining what Cambricon can successfully ship.

The Siyuan line has become the company’s flagship portfolio for training and inference. Cambricon already counts ByteDance as its largest customer and is expected to expand engagements with Alibaba as these firms scale their domestic AI clusters.

That shift is being shaped partly by domestic policy, with Chinese officials having spent much of the last few months urging major buyers to reduce their reliance on Nvidia hardware. That has created an opening for Cambricon and fellow domestic suppliers.

Foundry constraints

Having ambition is all well and good, but keep in mind that Cambricon doesn’t manufacture its own silicon. It depends on SMIC’s N+2 (7nm) processes for the Siyuan 590 and 690, the company’s most advanced production line that’s built entirely around DUV lithography rather than the EUV systems used by TSMC and Samsung. While N+2 can produce some pretty complex chips, it does so with a significant overhead in multi-patterning tests that raise cost and constrain performance.

This becomes apparent when you look at the numbers; Bloomberg reported yield rates of roughly 20% for Cambricon’s largest dies. That translates to four out of five chips coming off the wafer not meeting the targeted criteria, such as the chip having a defect or not meeting frequency or voltage targets (among other criteria). While some of those die could potentially be harvested for lower-power or lesser products, this yield level falls below most industry benchmarks. For instance, TSMC’s 2nm process has shown yields beyond 60% in test runs. Cambricon is therefore attempting to scale volume on a node that is both older and materially less efficient, so even if SMIC allocates more capacity, the proportion of chips that survive the manufacturing process will define the effective output.

Competition for those wafers adds another layer. Huawei, whose Ascend accelerators now form the backbone of many in-country training clusters, also relies on SMIC for advanced production. Demand for smartphone SoCs built on the same family of nodes has risen since Huawei’s return to 5G handsets. Any increase in Cambricon’s allocation requires SMIC to rebalance commitments across several strategically important customers, so the half-million-unit target for 2026 hints at strong confidence inside Cambricon that it can secure those slots. Ultimately, the foundry’s capacity and yields will determine what any ramp-up looks like in practice.

Memory supply is another challenge

Even if Cambricon clears the manufacturing hurdle, system integration presents its own bottleneck because AI accelerators require large pools of HBM to keep their compute units saturated. HBM3 and HBM3E are dominated by South Korean suppliers, and despite heavy investment, China has yet to produce a competitive domestic alternative. Huawei’s Ascend 910C, for example, still uses HBM stacks from SK Hynix and Samsung.

This is important because memory availability can limit the number of accelerators that can be deployed. Chinese cloud operators may secure sufficient chip volume but still face delays if they cannot pair those dies with matching HBM modules. With global demand rising across hyperscale data centers — which has arguably led to Micron disbanding its consumer business — neither capacity nor long-term supply for HBM is certain. Cambricon’s customers will need to lock in memory procurement early. Without that, it’s entirely possible that some fraction of the company’s planned 2026 output could sit idle until the right modules arrive.

Then there’s packaging. Multi-chiplet integration and high-speed interconnect routing all depend on advanced packaging lines. China’s capabilities have grown, but CoWoS-class technologies used by Nvidia, AMD, and TSMC remain out of reach for China. Cambricon must rely on what is available domestically, which dictates choices around chip partitioning and memory placement and influences both performance and scalability of the final accelerator boards.

A domestic ecosystem takes shape

Changing procurement patterns among Chinese tech giants are as consequential as the silicon itself. ByteDance already accounts for more than half of Cambricon’s orders, and Alibaba is expected to follow as its cloud arm builds out new AI capacity. This transition is already evident in Cambricon’s finances, with the company reporting a fourteen-fold increase in revenue in the September quarter and returning to profitability after several loss-making years.

Meanwhile, Huawei’s Ascend line continues to scale, and Hygon, MetaX, and Moore Threads are each pursuing different slices of the AI compute market. Still, Cambricon’s roadmap and its position inside China’s largest internet companies give it a measure of influence over how rapidly China’s home-grown ecosystem matures. If the Siyuan 690 achieves its planned performance targets and SMIC can nudge yields upward, Cambricon could offer a credible alternative to older Nvidia architectures for a growing set of workloads inside China.

None of this collapses the stark performance gap between China and the global high-end. Nvidia’s leading chips remain far ahead on raw throughput, memory bandwidth, and software tooling. However, that gap is weighted against availability and compliance for Chinese buyers who are facing pressure to avoid Nvidia silicon and the uncertainty around future supply. Cambricon’s 2026 goals reflect that, highlighting a strong demand from a captive market and an equally strong dependence on domestic manufacturing.

If Cambricon succeeds in approaching its goal of 500,000 units, it’ll prove that SMIC’s N+2 process can sustain far larger volumes of AI silicon than we currently think it can. If it falls short due to yield, however, it’ll put the structural limits of China’s domestic production on show for all to see as the country pushes hard for autonomy in advanced silicon. Either outcome will offer more clarity into where the country stands in building a full-stack AI chip pipeline.

Although the performance of high-bandwidth memory (HBM) has increased by an order of magnitude since its inception around a decade ago, many elements have remained fundamentally unchanged between HBM1 and HBM3E. But as the demands for bandwidth-hungry applications evolve, the technology must also change to accommodate them.

In new information revealed at TSMC's European OIP forum in late November, HBM4 and HBM4E will offer four major changes. HBM4 will receive a 2,048-bit interface and base dies produced using advanced logic technologies. Meanwhile, HBM4E will be able to utilize customizable base dies, which can be controlled with custom interfaces. These are dramatic shifts, which will have a big impact sooner than you might think.

HBM4, HBM4E, and C-HBM4E are on track to hit the market in 2026 and 2027, boasting the aforementioned 2048-bit standard interface with data transfer rates of up to 12.8 GT/s. Additionally, the customizable base dies will be able to use advanced logic technologies up to 3nm-class. This offers higher area efficiency, which TSMC claims represents a 2.5 times increase in performance.

HBM4: The next big thing

HBM4 — whose specification was officially published earlier this year — is the standard that sets the stage for a number of upcoming innovations in the AI and HPC memory market.

Each HBM4 memory stack features a 2,048-bit interface that officially supports data transfer rates of up to 8 GT/s, though controllers from controller specialists like Rambus and HBM4 stacks from leading DRAM vendors already support speeds of 10 GT/s or higher, since implementers want to have some reserve for additional peace of mind.

A stack with a 2,048-bit interface operating at 12 GT/s can deliver bandwidth of 2 TB/s, so an AI accelerator with eight HBM4 stacks will have access to potential bandwidth of 16 TB/s. And 12 GT/s could be just the beginning. Note that Cadence is already offering an HBM4E physical interface (PHY) with 12.8 GT/s support.

Internally, HBM4 doubles concurrency to 32 independent channels per stack (each split into two pseudo-channels), which reduces bank conflicts and raises efficiency throughput under highly parallel access patterns.

HBM4 stacks also support 24 Gb and 32 Gb DRAM devices and offer configurations for 4-Hi, 8-Hi, 12-Hi, and 16-Hi stacks, thus enabling capacities of up to 64 GB, which allows to build accelerators for next-generation AI models with trillions of parameters. Micron expects 64 GB stacks to become common with HBM4E sometime after late 2027, which aligns with Nvidia plans to equip its Rubin Ultra GPU with 1 TB of HBM4E memory.

The electrical specification of HBM4 broadens operating voltages with vendor-specific VDDQ options between 0.679V and 0.963V and VDDC of 0.97 V or 1.07 V, which enables DRAM makers to bin their offerings for efficiency or frequency while maintaining compatibility with the specification. On the security side of things, HBM4 supports directed refresh management (DRFM) to mitigate row-hammer attacks.

Because HBM4 expands its interface to 2,048 bits, it is supposed to have double the I/O contacts compared to previous-generation HBM stacks. Since it was close to impossible to produce a base die with proper routing using DRAM process technologies, memory makers like Micron, Samsung, and SK hynix collaborated with TSMC early on to ensure compatibility with CoWoS packaging technologies and to produce HBM4 base dies using 12FFC or N5 fabrication technologies.

Back then, it was thought that 12FFC would be used for 'regular' HBM4 base dies, which would be integrated with their host processors using advanced 2.5D process technologies, whereas N5 base dies would be used for HBM4 memory, which would then be integrated using direct bonding on logic chips.

At the European OIP 2025 forum, neither TSMC nor its partners mentioned N5-based HBM4 base dies for integration using hybrid bonding or similar technologies, which likely means that the project is not exactly a priority for now.

Potentially, the integration of HBM4 memory stacks on top of a high-performance processor creates significant thermal density, which would make it difficult to cool. It's also possible that hot compute chips can damage hot DRAM devices, and vice versa, but this is merely speculation.

There may also be hybrid-bonded SoIC-X 3D integrations, with stacked HBM4 on top of compute chiplets in development, but their developers do not want to share results just yet.

In any case, HBM4 base dies made by TSMC on its low-power 12FFC or N5 process technologies, as well as custom C-HBM4E base dies produced on TSMC's N3P node use lower voltages (0.8V – 0.75V vs 1.1V in case of HBM3E), and are up to two times more power efficient than base dies of HBM3E memory manufactured using DRAM technologies, according to TSMC.

On the other hand, since HBM4 requires a more sophisticated controller and a larger, more complex PHY compared to HBM3E (15mm^2 vs 11mm^2, according to GUC). HBM4's memory subsystems will be more power hungry than HBM3E subsystems, too. However, due to considerably higher bandwidth, enabled by HBM4, they will be considerably more power and area-efficient than predecessors.

As for IP readiness, GUC has taped out its HBM4 PHY IP on N3P in March 2025. This will be validated with HBM4 memory samples in Q1 2026, when the company will be formally able to claim that it has a silicon-proven and validated HBM4 memory solution. Additionally, the IP will be compatible with all types of CoWoS packaging (-S, -R, -L) and can address a variety of applications. HBM4 memory controllers are available from a range of companies, including Rambus. EDA developers like Cadence, Siemens EDA, and Synopsys.

HBM4E: 2.5X higher bandwidth than HBM3E

With the introduction of HBM4's 2,048-bit memory interface, JEDEC members had to slash maximum data transfer rates to 8 GT/s from around 9.4 GT/s supported by HBM3E, which still enables a dramatic bandwidth increase. However, HBM4E is set to push electrical and signaling limits higher by supporting per-pin data rates to 12 GT/s (by refining PHY for better signal margin and jitter control at higher frequencies) and extending total stack bandwidth to around 3 TB/s, while keeping the 2,048-bit interface and 32-channel architecture. As a result, the bandwidth offered by HBM4E stacks will be 2.5X higher compared to HBM3E, and even when area and PHY power are taken into account, HBM4E will be 1.7X more power efficient and 1.8X area efficient, according to GUC.

Standard HBM4E solutions will likely be able to use HBM4 base dies, though some memory makers may migrate to base dies using the N5 or N3P process technologies for higher performance and efficiency.

C-HBM4E: The first custom type of memory for AI and HPC

Virtually all leading DRAM makers have introduced proprietary DRAM solutions with certain levels of customization over the past decade, but none of them have gained traction. Starting with HBM4E, HBM memory will get a separate branch of customized solutions, which is set to feature unique capabilities and proprietary interfaces.

On a high level, C-HBM4E is an HBM4E memory stack with a custom base die. The stack retains standard HBM4E memory devices, which comply with clock and electrical requirements set by JEDEC. However, the base die can now be customized in several different ways, thus shifting emphasis from raw bandwidth to the integration of custom logic directly into memory devices, which can be achieved using several methods.

The easiest way — described by Rambus — is to retain the standard HBM4E interface, alongside built-in custom logic and/or caches on the base die, to add features or performance. As long as the HBM4E protocol with supporting firmware and software stacks is compliant, this may increase the performance of memory subsystems beyond increasing transfer rates or widening I/O.

A more complex method— envisioned by TSMC and Rambus — is to place the HBM4E memory controller and a custom die-to-die interface directly into the logic base die. A large part of the industry's focus is on reducing the number of traces required between the processor and the HBM base die, and a custom D2D interface will do just that. By shrinking the interface width, each memory stack consumes fewer I/O pins, which enables a single SoC to attach a greater number of HBM stacks without increasing package size or complexity.

A custom die made using TSMC's N3P technology would allow packing in an HBM4E memory controller, a custom D2D PHY, and potentially some additional logic. For example, KAIST envisions integration of near memory compute (NMC) processors, which will make at least some C-HBM4E solutions system-on-chips (SoCs) with basic processing capabilities.

If near-memory compute logic is indeed integrated into C-HBM4E, the software stack must evolve to become topology-aware and memory-aware, rather than treating a C-HBM4E stack as 'just' memory. Without changes to toolchains, drivers, and runtimes, near-memory compute becomes invisible silicon—present in hardware, but unused by software.

Runtime systems and compilers will need explicit knowledge of bank structure, channel placement, and in-memory execution units so that workloads can be scheduled where data physically resides, instead of being moved across the fabric. In addition to this, programming models will also need extensions to work with in-memory compute, or multi-tier memory systems in general. Finally, operating systems must support heterogeneous memory domains with non-uniform latency and asymmetric coherence, while profilers must observe and optimize execution occurring inside memory devices.

A vision beyond

If the figures published by TSMC and GUC are to be believed, then HBM's raw performance is set to increase by around 2.5 times within the next few years, thanks to HBM4E. This development opens the doors to memory subsystems with a 1 TB capacity and a whopping bandwidth of 48 TB/s. If custom compute logic inside base dies of HBM gets adopted by the industry, this might be the biggest shift in how computers work in decades.

Micron is preparing a major expansion of its Hiroshima operations to build a dedicated high-bandwidth memory (HBM) facility, according to a report by Nikkei Asia. The report says the company intends to invest 1.5 trillion yen — US$9.6 billion — in a new plant on its existing site, with construction scheduled to begin in May next year. Shipments would follow around 2028. Japan’s Ministry of Economy, Trade and Industry (METI) is expected to contribute up to 500 billion yen in subsidies to support the project, though neither Micron nor METI has confirmed the report.

HBM has arguably become the most constrained component in the AI supply chain, giving the project some real weight in terms of government subsidies. SK hynix currently leads the market and has committed most of its HBM, DRAM and NAND output to Nvidia through 2026. Samsung is working to catch up with its 12-layer HBM3E stacks, while Micron has pushed hard to grow its own presence through HBM3E supply deals with Nvidia and AMD. TrendForce data shows Micron moving toward roughly a quarter of the HBM market with 20% of shipments as production increases; a dedicated Hiroshima expansion could shift that balance further once it comes online.

Japan has been aggressive in courting this kind of investment, offering substantial government incentives to foreign chipmakers as part of a wider effort to rebuild domestic semiconductor capacity. TSMC’s Kumamoto fabs and the state-backed Rapidus project are already part of this strategy.

Micron itself has been a significant beneficiary. Last year, Micron announced plans to introduce EUV-based DRAM production on the same Hiroshima campus, investing 500 billion yen of its own cash and supported by nearly 200 billion yen in subsidies. The first LPDDR5X memory devices produced on its 1γ process at this facility then began sampling in May of this year.

The scale of the planned plant aligns with expectations for the next generation of AI accelerators. Nvidia and AMD are both shifting towards HBM4 and HBM4E, both of which require tighter process control and higher layer counts. Capacity has been thin throughout the current GPU cycle, with long lead times and allocation limits driven by the mismatch between demand and available wafer starts. If Micron’s new plant reaches volume production in 2028, it will arrive just as those next-generation GPUs are.

For Micron, Hiroshima offers political and financial stability at a moment when both geopolitics and the market cause increasing uncertainty, and Tokyo’s willingness to fund a third of the project removes some of the risk from a multiyear build. The long runway to 2028 also gives Micron a clear path to expand its role

Chinese memory maker ChangXin Memory Technologies (CXMT) is reportedly preparing to go public in the first quarter of 2026, with the company signing up China International Capital Corporation and CSC Financial, two state-owned banks, back in July to start the process. According to the South China Morning Post, CXMT aims to raise 20 to 40 billion Yuan (around US$2.8 to US$5.6 billion) from the offering. However, these details are subject to change, especially as the plan hasn’t been officially announced yet.

The planned IPO is coming at a time when the AI boom is straining the global memory and storage supply chains, with shortages and price increases expected to last a decade. Aside from that, the company is well-placed to replace Micron in China, especially as the latter is reportedly leaving the Chinese market two years after it was banned from ‘critical information infrastructure’.

The news about the public offering leaks as Chinese semiconductor stock prices are soaring, with CSI’s China semiconductor index benchmark jumping 49% from January. And with the ongoing trade spat between Beijing and Washington causing companies like Nvidia to lose local market share, many local investors are keen on placing bets on chipmakers that will make it big as the country targets semiconductor self-sufficiency. The United States has also banned the exports of HBM to China in December 2024, meaning Chinese AI chip manufacturers have no choice but to rely on domestic manufacturers for their supply. At the moment, CXMT is one of just three companies building HBM chips for domestic AI chip manufacturers, making it crucial for China’s progress.

CXMT has reportedly spent $6 to $7 billion on capital expenditure during 2023 and 2024, with the company expecting to spend around 5% more in 2025. This includes an HBM back-end packaging fab in Shanghai that’s planned to start commercial operations by late 2026. But despite all these investments, the Chinese chipmaker is still a few years behind its more established competitors. Micron has already started shipping HBM4 samples to some of its clients in June of this year, while South Korean chipmaker SK hynix is said to have already completed development and is preparing for mass production of next-generation memory modules.

Analysts say that CXMT plans to begin mass production of its own HBM3 in 2026, suggesting that it’s four years behind its rivals. Despite that, this is still a big step for the Chinese semiconductor industry, especially given that CXMT was only founded in 2016 with support from the Chinese government, whereas SK hynix, Samsung, and Micron, the current biggest memory manufacturers, have been in operation for several decades already.

Kioxia has developed a prototype of the 5TB high-bandwidth flash memory module with a bandwidth of 64 GB/s. It's essentially NAND-based memory for GPUs. Compared to HBM, High Bandwidth Flash (HBF) adapts the concept to NAND flash, offering 8-16x the capacity of DRAM-based HBM. By combining speed with persistent storage, HBF enables large AI datasets to be accessed efficiently while using less power. One of these HBF modules, which Kioxia has pushed to 64 GB/s, is what allows this capability.

When you hear “flash storage,” you usually think in terms of capacity first, speed second. Even the fastest PCIe 5.0 SSDs today—14 GB/s class drives like Samsung’s 9100 Pro—are dwarfed by the bandwidth demands of modern GPUs and CPUs. Kioxia’s new prototype turns that expectation on its head: a single flash module delivering 5 TB of capacity and 64 GB/s of sustained bandwidth over PCIe 6.0. To put that into perspective, that’s over 4x faster than the fastest PCIe 5.0 drives shipping today, and within striking distance of HBM2E’s per-stack throughput.

The key is how the system scales; instead of one central controller trying to manage an entire bank of NAND—which quickly becomes a bottleneck as more dies and channels are added—Kioxia gives each module its own controller. That controller sits right next to its NAND and links to others in a daisy-chain layout. This reduces crosstalk and eliminates the complexity of wide parallel buses, which become increasingly challenging to manage as speeds increase. Instead, data is passed along in series, with each link pushing 128 Gbps using PAM4 signaling.

PAM4 (Pulse Amplitude Modulation with four levels) doubles the data rate per symbol compared to traditional NRZ signaling, but it’s also more sensitive to noise and bit errors. To maintain signal integrity, Kioxia relies on equalization, error correction, and stronger pre-emphasis—similar to what PCIe 6.0 itself requires.

This helps explain the move to PCIe 6.0 as the host interface, as x16 lanes of PCIe 6.0 can theoretically handle around 128 GB/s bidirectional. Kioxia’s 64 GB/s target sits just under half that limit, leaving enough headroom for error correction and overhead without maxing out the bus.

As you might be expecting, latency is the main tradeoff. HBM memory works in hundreds of nanoseconds, almost like an extension of GPU registers. NAND flash—even with advanced controllers—still accesses data in tens of microseconds, which is magnitudes slower. Kioxia counters this with aggressive prefetching and controller-level caching, so sequential workloads are less affected. It doesn’t make NAND as fast as DRAM, but it narrows the gap enough that for streaming datasets, AI checkpoints, or large graph analytics, bandwidth matters more than raw latency.

Power is another crucial factor here, as Kioxia claims under 40W per module, which appears impressive when compared to traditional Gen5 SSDs that can draw up to 15W for ~14 GB/s. On a GB/s per Watt basis, this module is dramatically more efficient. That matters because in a hyperscale rack, a few hundred drives can easily consume multiple kilowatts. AI datacenters—already ballooning in power budgets thanks to H100 clusters—need every watt saved at the storage layer.

These modules also open up new system design options. With daisy-chained controllers, adding more modules doesn’t consume additional bandwidth, so performance scales linearly with capacity. A complete set of 16 could reach 80 TB of flash and over 1 TB/s of throughput—numbers once limited to parallel file systems or DRAM scratchpads. This makes it possible to treat storage as near-memory, sitting directly on the PCIe fabric alongside accelerators, rather than being stuck in back-end I/O.

Look at the numbers involved in AI cloud investment and data center buildout, and the stats are astonishing. The Magnificent 7 tech companies – the biggest tech giants in the world – have collectively invested more than $100 billion in data centers and other infrastructure in the last three months alone. The majority of that comes from four of the seven: Microsoft, Meta, Amazon, and Alphabet.

That spending is having an outsized effect on the economy. Jens Nordvig, the founder of Exante Data, believes that total spending on AI could account for 2% of U.S. GDP this year, based on projections and planned projects.

The same is true in China, where provinces and private companies alike are throwing more and more cash at AI buildouts. The scale of that spending is such that Chinese president Xi Jinping has stepped in, warning officials to be more cautious with their cash for fear of overspending. Not everyone is listening. Gartner, a consultancy firm, believes the world will spend nearly half a trillion dollars on data centers this year, up 42% from last year. McKinsey, another consultancy, believes that more than $5 trillion will be invested by 2030, so great is the demand.

Good for investors, but is it good for capex?

The kinds of eye-watering sums involved are good news for tech investors, shareholders in those Magnificent 7 firms, and plenty of others. The people leading those companies are making it clear they think it’s necessary. “It’s essential infrastructure,” said Jensen Huang, in Nvidia’s Q1 earnings call in May. “We’re clearly in the beginning of the buildout of this infrastructure.” But the massive interest in data centers is having other knock-on effects beyond making big tech companies even bigger. It’s reshaping how we think about the sectors and components that make those data centers work.

“The central problem today in AI is compute power, and the energy required is getting out of hand,” says Subramanian Iyer, distinguished professor at the Henry Samueli School of Engineering at UCLA, in an interview with Tom's Hardware Premium. Lots has been written about the energy impact of these large data centers, with some companies even starting to consider small modular reactor technology that would power them using nuclear. “That tells you how serious the power problem is,” Iyer says.

Google, for example, raised its 2025 capital expenditure budget to $85 billion from $75 billion because of investments in servers and data center construction, with further acceleration expected in 2026. Google’s monthly token processing also doubled from 480 trillion in May to over 980 trillion. (A little over a year earlier, the number of tokens Google processed was just 1% of that.) All of those tokens need processing. And that processing happens on hardware. Jefferies estimates that Google’s 980 trillion token compute is close to 200 million H100s operating 24 hours a day, seven days a week.

It all adds up to significant expenditure. Moore’s law isn’t completely dead, argues Iyer. But it’s changing. “Transistors are still scaling, but they're no longer getting cheaper,” he says. “In fact, they're getting more expensive.”

Data centers are changing

What data centers are used for is changing. Unlike their traditional predecessors, they now rely heavily on advanced GPUs, specialized networking, and high-powered cooling, meaning their bill of materials (BOM) has bloated. Estimates put the cost of a fully-equipped AI data center at around $10 million, with power and cooling systems and servers and IT equipment accounting for roughly a third each, with other key categories including network (15%) and storage (10%).

All of those are being squeezed by inflation and surging hardware requirements. But that’s only for smaller enterprise-focused setups: the hyperscale facilities of the type that Donald Trump and other countries around the world are looking at run into the billions of dollars per campus.

The underlying cost of components is also steadily rising. Average material costs increased by 3% and labor by 4% for key data center hardware over the past year, with concrete and copper cable among the biggest risers, according to Turner & Townsend. The smaller but still essential elements like power delivery, printed circuit boards, and advanced packaging are also rising in price thanks to chronic bottlenecks, especially for the high-end AI chips that require stacking and new thermal approaches.

Semiconductors used to drop reliably with each new process node, but that’s no longer the case as manufacturing them becomes more complex, and increased demand globally squeezes supply. TSMC is likely to raise the price of advanced nodes by over 15% in 2025, according to reports, passing on costs to buyers. It all means that every new data center costs more money than it used to.

When they were launched in 2020, Nvidia's then-top-tier DGX A100 servers cost $199,000. Prior reporting from Tom’s Hardware suggests analysts believe the GB200 server racks will cost $3 million. There's an argument that the price hike is down to rising manufacturing costs, with those fabs turning into gigaprojects, like TSMC's $65 billion Arizona complex. In part, the cost of these large-scale efforts is so great because the hardware behind them can be comparatively wasteful. “If you spend a megawatt of power for a data center,” says Iyer, “the actual work you’re getting is only about a third of that. The rest of it is pretty much all overhead.”

Those giant fab complexes cost as much money to equip as they do to build. Buyers are absorbing the cost of EUV machines to make the 2nm and 3nm chips populating data centers, which might have dozens of them – and that’s before considering the less advanced, but not significantly less expensive, tools for wafer etching, deposition, and inspection. A single high-end lithography EUV tool from ASML can reportedly cost $400 million alone.

Big tech’s intense AI buildout has forced even the world’s leading chip manufacturers, like TSMC, to invest at an unprecedented scale. Their Arizona cluster, which encompasses three advanced fabs, shows at what scale companies are operating. Elsewhere, Nvidia expects that up to $1 trillion will be spent globally upgrading data centers for AI workloads by 2028, further underlining the scale of the transformation.

Bigger tasks, bigger bills

One reason for the bigger bill is that the purpose – and the amount of work those data centers are being asked to do – has changed and increased. But the cost is also because the hardware requirements for those cloud servers and data centers have altered. Big tech capex keeps climbing because AI workloads now demand the bleeding-edge node – a shift in recent years that has been enacted by the rise of generative AI.

Silicon destined for servers once was able to lag the chips put into smartphones by a process generation or two, but is now “is par à pursue [on par with] with the bleeding edge,” said CJ Muse, an analyst specializing in semiconductors for Cantor Fitzgerland in an interview with Tom's Hardware Premium. That forces data center operators onto the most expensive wafers to cram in as many transistors, and as much compute per watt, as possible. All that comes with a hefty price tag. “A bleeding-edge 2nm fab at TSMC, for every 1,000 wafer starts at about $425 million, and so that adds up pretty quickly,” says Muse.

The race to be at the bleeding edge creates a domino effect. State-of-the-art processors are pointless if starved of data, making high-bandwidth memory (HBM) vital. But now memory is facing its own pressures on supply and cost. “From now on, the HBM segment should face a test of how HBM suppliers can manage supply and protect prices as their technology gap narrows and real competition begins,” said Jongwook Lee, a team leader at Samsung Securities, in a research report.

Lee and his colleagues foresee a future where the HBM market could split into ‘new’ product segments like HBM4, the higher-bandwidth, more luxe standard of memory, which would continue to enjoy a premium, and ‘old’ product segments, which would require discounts to remain competitive.

HBM, DRAM, and other factors further push prices

HBM manufacturing is vastly more complicated and supply-constrained than standard DRAM. With only Samsung, SK Hynix, and Micron as the three major suppliers, HBM can be especially vulnerable to supply disruptions or geopolitical shocks. Demand regularly exceeds supply, and lead times for HBM often top half a year, especially with advanced packaging capacity being booked years in advance for longstanding customers like Nvidia and AMD. It all means intense technical and economic headwinds in HBM, and the advanced packaging ecosystems they depend on, weigh heavily on the speed, cost, and security of the world’s AI data center buildout.

Even global competition for wafer fabrication equipment (WFE) is heating up. Chinese imports grew 14% year-over-year in June 2025, according to Jefferies, breaking a previous downward trend. June was the first month of positive growth in 2025, led by a surge in demand for specific machinery, including etching and deposition tools, which saw growth of 65% and 28% respectively.

Analysts at Jefferies believe that the unexpected growth was from China’s DRAM sector, and particularly CXMT, a major producer that has, to date, dodged being on the US sanctions list of entities not allowed to import chip tech into China. The US-China tech rivalry has led to stringent export controls and sanction lists that continue to constrain Chinese chipmakers from accessing critical semiconductor manufacturing technology to alleviate some of the supply pressures. That’s unlikely to change as Donald Trump continues to pursue an America first strategy for this – but could backfire if Trump pushes his hand too far. China dominates the processing of rare earth elements like neodymium, critical for high-performance components used in data center hardware. Sourcing rare earths, essential for AI chips and data center hardware, could become trickier if any one party chooses to weaponize access to them as part of trade negotiations. The political and regulatory headwinds are increasing cost pressures and investment risks, shaping the competitive landscape in unpredictable ways.

Nvidia's stranglehold, and how companies are fighting back

The problem every tech company faces is that they’re overly reliant on Nvidia at present. As a result, big cloud providers are weighing up whether to develop their own custom ASICs. Broadcom alone expects AI-specific custom silicon and networking sales to reach 42% of its revenue by 2026, according to Muse.

Major hyperscalers like Google, Amazon, and Meta are all actively rolling out custom ASIC chips, creating substantial opportunities for both established vendors and new entrants. Broadcom is booming: analysts say the firm’s custom ASIC and networking revenue for AI is expected to be around $18 billion by 2026, much of it driven by custom chips for hyperscale inference and high-bandwidth AI networking. The demand isn’t just coming from chips for inference. Networking ASICs, interconnect switches, and edge/IoT devices are all seeing surging demand.

Yet Muse points out that building successful custom chips is hard. “Google had three different teams building the TPU, and one was successful, the other two were not,” he says. The answer to that is for companies to try and develop their own ASIC strategy while also recognizing they need to go into the market and buy more GPUs.

That in turn is pushing up prices, in large part because companies that once kept themselves to themselves are not competing with one another. “I think the interesting change statement is that Meta, Amazon, Google and Microsoft all had fairly defined swimming lanes,” says Muse.

“Obviously there’s competition in offering cloud services, but their business models didn’t really overlap, and they all were all doing extraordinarily well,” he explains. That’s since changed. “Now they’re all competing head-to-head, and so there are going to be clear winners and losers.” That head-to-head competition is driving what Muse calls “this mad race and massive investments”.

The outcome will not only determine the next leader in tech, but could also redraw the global map of technological power for a generation.

Amidst all the theatrics of the ongoing China-U.S. semiconductor wars, SK Hynix—a South Korean giant also affected by tariffs—expects the global market for High Bandwidth Memory (HBM) chips used in artificial intelligence to grow by around 30% a year until 2030, driven by accelerating AI adoption and a shift toward more customized designs. The forecast, shared with Reuters, points to what the company sees as a long-term structural expansion in a sector traditionally treated like a commodity.

HBM is already one of the most sought-after components in AI datacenters, stacking memory dies vertically alongside a “base” logic die to improve performance and efficiency. SK Hynix, which commands the largest share of the HBM market, says demand is “firm and strong,” with capital spending by hyperscalers such as Amazon, Microsoft, and Google likely to be revised upward over time. The company estimates the market for custom HBM alone could be worth tens of billions of dollars by 2030.

Customization is becoming a key differentiator. While large customers—including GPU leaders—already receive bespoke HBM tuned for power or performance needs, SK Hynix expects more clients to move away from one-size-fits-all products. That shift, along with advances in packaging and the upcoming HBM4 generation, is making it harder for buyers to swap between rival offerings, supporting margins in a space once dominated by price competition.

Sandisk and SK hynix have just signed a memorandum of understanding to collaborate on what could become a pivotal advancement in AI memory infrastructure. Announced in a press release, this agreement aims to standardize “High Bandwidth Flash” (HBF), a NAND flash-based memory technology built into HBM-like packages. This marks the first serious industry push to fuse flash and DRAM-like bandwidth into a single stack, potentially transforming how AI models access and process data at scale.

Unlike traditional HBM, which relies exclusively on DRAM, HBF substitutes parts of the memory stack with NAND flash—trading raw latency for significantly higher capacity and non-volatility. The approach allows HBF to deliver up to 8–16x the capacity of DRAM-based HBM, at comparable costs, while still targeting similar bandwidth levels. Unlike DRAM, which requires constant power to retain data, NAND is non-volatile, enabling persistent storage with reduced energy demands.

That distinction is critical as AI inference scales out to more energy-constrained deployments. Hyperscalers (companies with massive cloud infrastructure for rent) are now pushing inference to the edge, and cooling budgets in AI data centers are already hitting practical limits.

A couple of years ago, a widely-discussed research paper titled “LLM in a Flash” proposed an architecture where large language models could be efficiently run by using SSDs as an additional memory tier, offloading some of the memory pressure from DRAM. Therefore, by marrying NAND flash’s high capacity with interface designs inspired by HBM’s bandwidth capabilities, Sandisk and SK hynix are effectively proposing a new memory class — one that could support large model inference without incurring the thermal and cost overheads of traditional HBM stacks.

The move is also aligned with broader industry shifts. Samsung, for instance, recently unveiled its own flash-backed AI storage tier dubbed “PBSSD” and is actively working on next-gen HBM4 DRAM expected to include logic die integration and potentially hybrid stacks. Meanwhile, Nvidia's roadmap through its Rubin and Vera GPUs continues to rely heavily on HBM, and integrating flash may offer a path to scale memory without linearly scaling cost and power. You can see our deep-dive into HBM roadmaps for Samsung, Micron, SK hynix to learn more.

Sandisk’s HBF prototype, shown at the Flash Memory Summit 2025, was developed using its proprietary BiCS NAND and CBA wafer bonding technologies. The company received the “Most Innovative Technology” award at the event, and announced the formation of a Technical Advisory Board to guide HBF’s development and ecosystem strategy. The board includes figures from both inside and outside Sandisk, underscoring the company's intent to establish HBF as a cross-industry standard, not just a proprietary product.

High Bandwidth Memory (HBM) is the unsung hero behind the AI revolution. As the industry seeks to extract the most performance from frontier AI models, HBM powers the world's fastest GPUs and AI accelerators by keeping the intense computational engines fed with data at breakneck speed. This critical technology has rapidly matured over the past several years, and lately the pace of innovation has quickened, as industry behemoths like Nvidia and AMD look to facilitate the creation of more advanced artificial intelligence models.

All three major DRAM makers (Micron, Samsung, and SK hynix) have started volume production of 8-Hi HBM3E stacks, the newest form of the technology. And more robust and powerful forms of HBM memory are already in development.

Unfortunately, shortages of sophisticated HBM memory have hampered the supply of AI GPUs, which has manufacturers racing to add more production capacity to address any shortfalls. Meanwhile, the development of next-gen HBM products continues apace, hand-in-hand with new performance-enhancing technologies, which will power the next wave of AI accelerators.

Let's take a look at what's next on the HBM roadmaps for Micron, Samsung, and SK hynix based on official information and other sources.

Speeds and feeds

Memory bandwidth is a critical bottleneck in AI systems — AI models, particularly deep learning models, ingest tremendous amounts of data as they chew through workloads. But most forms of modern memory can't satiate AI's ravenous appetite for more data. That's where HBM steps in.

Traditional memory based on DDR, LPDDR, or GDDR uses 128–bit to 512-bit wide interfaces and employs high data transfer rates to provide bandwidth from 100 GB/s to 2 TB/s.

Unlike traditional memory, high-bandwidth memory (HBM) uses a very wide interface—1024 bits in the case of HBM2 and HBM3, or 2048 bits with HBM4—which multiplies bandwidth up to 4 TB/s—8 TB/s. Given the bandwidth-constrained nature of high-intensity parallel computation in GPUs and accelerators, this increased bandwidth translates directly to more performance.

However, the wide interface makes HBM difficult to produce, requiring multiple specialized DRAM devices interconnected using through-silicon vias (TSVs) stacked on top of a base die. HBM makers also vary the number of stacked memory dies to increase capacity, denoted by terminology such as 8-Hi for eight stacked dies, or 12-Hi for 12 stacked dies.

Due to its huge bandwidth, HBM is, and will continue to be, the de facto memory standard for AI systems, HPC ASICs, and GPUs.

12-Hi HBM3E is almost here

Today's highest-end AI accelerators — including Nvidia's H200 (141 GB), B200 (192 GB), and AMD's Instinct MI300X (192 GB) — use 24 GB 8-Hi HBM3E stacks based on 24 Gb DRAM devices. The next step for the industry is to adopt higher-capacity 36 GB 12-Hi HBM3E packages featuring 24 Gb memory dies. These will be used by Nvidia's upcoming B300-series and AMD's next-gen MI325X AI accelerators.

SK hynix has begun mass production of 36 GB 12-Hi HBM3E chips, whereas Micron has been sampling similar products since September, with mass production of the new packages understood to be imminent.

Samsung, on the other hand, was late to the party with its 8-Hi HBM3E certification, and its 12-Hi HBM3E dies also suffered from a slight delay. Samsung's delay is likely caused by sticking with its 1α fabrication technology, unlike Micron and SK hynix, which use a 1ß (5th Gen, 10nm-class) DRAM process to make their HBM3E DRAM ICs. By the time Nvidia's B300 enters mass production, Samsung will likely be able to compete with 12-Hi HBM3E 36 Gb offerings of its own.

HBM4: 2048-bit I/O and up to 16 layers

While manufacturers are still wrapping their hands around the upcoming HBM3E rollout, HBM4 and HBM4E are both already on the horizon.

The preliminary HBM4 specification (unveiled in July 2024) introduces a wider 2048-bit interface for HBM stacks. It also specifies 24 Gb and 32 Gb DRAM layers at up to 6.40 GT/s. The spec supports 4-Hi, 8-Hi, 12-Hi, and 16-Hi configurations, ensuring greater flexibility and potentially enabling even larger 64 Gb HBM4 packages.

Meanwhile, HBM4E may also boast high interface speeds of around 9 GT/s, as Rambus' HBM4 memory controller IP exceeds the announced capabilities of HBM4's JDEC-standard 6.40 GT/s speeds.

With HBM4E, memory makers will be able to customize base dies of packages (which SK hynix envisioned in early 2024) by adding additional functions, which could potentially extend to enhanced caches, custom interface protocols, and more.

All three leading memory manufacturers, Micron, Samsung, and SK hynix, have confirmed their intentions to produce HBM4 and HBM4E memory, but their overall rollout strategies may differ.

HBM4: 16-Hi stacks, but no 32 Gb devices on horizon

For now, none of the prominent DRAM producers have HBM4 or HBM4E stacks based on 32 Gb memory devices on their roadmaps. As such, all HBM4 and HBM4E products are expected to utilize smaller 24 Gb DRAM dies when they first roll out.

Micron is expected to keep using its proven 1ß (5th Gen, 10nm-class) process technology to make 24 Gb memory ICs for HBM4 stacks; however, Samsung plans to transition to 24 Gb DRAM dies made on its 1γ (6th Gen, 10nm-class) process with HBM4 and HBM4E. This will likely offer Samsung substantial performance, power efficiency, and cost advantages. SK hynix also intends to use 1ß for HBM4 DRAM ICs and may transition to a 1γ process for the HBM4E offering.

When it comes to layers, Micron lists 12-Hi and 16-Hi versions of its HBM4 and HBM4E offerings, whereas Samsung and SK hynix may go straight to 16-Hi HBM4 stacks. With the standard appearing to be 24 Gb DRAM devices and 12-Hi or 16-Hi stacks, HBM4 will increase per-package capacity to 48 Gb, a noticeable leap over HBM3E's 36Gb.

It's possible that by the time HBM4E finally arrives, the number of supported layers may exceed 16, with rumors that South Korean manufacturers could adopt a 20-layer design (which should be taken with a grain of salt). By then, DRAM makers may have also adopted a 32 Gb package for high-bandwidth offerings.

HBM4 & HBM4E production nodes

It should be no surprise that HBM4 and HBM4E memory stacks will rely on base dies produced by logic manufacturers using logic process technologies, thus boosting transfer speeds and signal integrity. TSMC and SK hynix were the first to disclose that they plan to use TSMC’s 12FFC+ and N5 base dies for HBM4. It's likely that Micron will also use TSMC's base dies (as the two companies are partners), though this has not been officially confirmed.

Samsung is expected to use its own Samsung Foundry nodes for HBM4 and HBM4E base dies. There is still uncertainty around the exact node it will employ, though it is reasonable to expect similar process technologies to TSMC's 12FFC+ and N5 processes.

HBM4 is coming in 2026 & HBM4E expected a year later

Samsung and SK hynix are rumored to introduce their first HBM4 offerings around Q3 2025, whereas Micron is projected to follow in Q4 2025. In both cases, 'introductions' likely mean the delivery of the first working samples to partners like AMD and Nvidia, not high-volume manufacturing.

Considering that the mass production of actual processors that support HBM4 is not expected until 2026, Micron's slight delay does not seem like a major issue.

Curiously, Micron began discussing HBM4E around a year before the planned mass production of HBM4 was set to commence. Typically, 'extended' variants of HBM specifications are introduced years after the original standard. According to Micron's official roadmap, HBM4E is set to arrive in late 2027.

HBM4E is likely to be used in the generation after the release of Nvidia's upcoming Rubin architecture and AMD's MI400 AI accelerator. Both are slated to support HBM4 in 2026.

If the industry requires customizable memory, HBM4E might land earlier than expected, but don't hold your breath — HBM development is notoriously challenging.

Micron this month announced a renewed U.S. buildout strategy that expands investment plans to $150 million, plans to build an HBM packaging facility in Virginia, and invest some $50 billion in R&D. Micron's first new U.S. fab in years will begin operations in the second half of 2027, the company revealed this month.

Following the enactment of the CHIPS and Science Act in August 2022, Micron unveiled major plans to build new fabs worth over $115 billion in Idaho and New York with the aim of building 40% of its DRAM products in the U.S. over the next decade. Under the new plan, Micron projects investing $200 billion in U.S.-based memory production and R&D over the next 20+ years, with support from the U.S. government.

The effort includes $150 billion for manufacturing and $50 billion for R&D, with the goal of creating around 90,000 direct and indirect jobs. The new plan envisions two leading-edge DRAM fabs in Idaho, a site with four fabs in New York, and an HBM packaging facility in Virginia. Let's take a closer look at Micron's plans.

A $200 billion plan

The first part of the original plan involved building one of the world's largest and most advanced DRAM production facilities near Boise, Idaho, which is now known as Fab ID1. Once fully outfitted with production equipment, the cleanroom area of ID1 will span 600,000 square feet (~55,700 square meters). This is roughly double the cleanroom capacity of GlobalFoundries’ Fab 8 and on par with the large-scale fabs operated by competitors Samsung and SK Hynix in South Korea.

Micron's first new fab in Idaho (ID1) reached a key construction milestone in June 2025 and is expected to begin wafer output in the second half of calendar 2027, with customer qualifications following thereafter, Micron announced this week. The second Idaho fab (ID2) will be built adjacent to ID1, benefiting from shared infrastructure and R&D co-location. Micron expects ID2 to enter production before the New York fab, although the company didn't elaborate on the exact timing.

However, while Micron broke ground on its new fab in Idaho and ID1 is expected to begin operations in a couple of years, the company is still struggling to start construction near Clay, New York. Micron plans to begin groundwork for its New York fab by the end of 2025, after completion of federal and state environmental reviews. Micron's New York plan is even more ambitious than the Idaho plan, as it involves four fab phases with cleanroom areas of around 600,000 square feet (approximately 55,700 square meters). While no specific production timeline has been announced for this site, it is clear that the site is a part of Micron's strategic long-term effort to establish a robust domestic manufacturing footprint in support of both commercial and national computing needs.

In addition to building brand-new fabs, Micron is set to expand its facility in Manassas, Virginia. Currently, this plant manufactures memory chips for automotive, aerospace, defense, and industrial applications. After the upgrade, the fab will gain capacity as well as advanced packaging capabilities to assemble HBM memory stacks in the U.S. However, Micron will only add HBM capability to its Virginia plant after it ramps up production of enough DRAM wafers in the U.S. at its fabs in Boise, Idaho. That said, expect Micron to build HBM5 or HBM6 in the U.S.

"As part of this $200 billion investment plan, Micron plans to […] [bring] advanced packaging capabilities to the U.S. to support our long-term HBM growth plans after we have established sufficient DRAM wafer scale in our U.S. operations," said Sanjay Mehrotra, chief executive and chairman of Micron.

"[…] Our first Idaho fab, ID1, achieved another key construction milestone in June. We expect first DRAM wafer output at ID1 to begin in the second half of calendar 2027, with customer qualifications to follow. The second Idaho fab, ID2, will benefit from manufacturing economies of scale with ID1, and add to R&D co-location benefits with greater efficiencies and faster time to market. To meet anticipated demand, ID2 will begin production before the first New York fab. We expect to begin ground preparation in New York later this year following the completion of state and federal environmental reviews," he concluded.

A great unknown

Although Micron clearly states that it plans to invest some $150 billion in its manufacturing capacities in America, it only disclosed the completion schedule for its Fab ID1, which is about to be completed, and the groundwork for the first New York facility. No schedules for other projects have been mentioned, except that Micron intends to invest $150 billion in its U.S. operations over the course of 20 years or more.

But perhaps the biggest unknown in Micron's announcement is its $50 billion R&D project. The company spent between 10% and 20% of its revenue on R&D in 2022 – 2024 (from $3.1 billion to $3.43 billion). Last year, the company invested $3.43 billion (14% of revenue) on research and development, so $50 billion is Micron's R&D budget for around 14 years. Yet, the company conducts R&D operations not only in the U.S. but also in Japan, Taiwan, and Singapore. To that end, it is unclear whether $50 billion means increase of R&D operations in the U.S. at the expense of other sites, or additional spending of $50 billion dollars on R&D in the U.S. on top of regular expenditures over the course of the next 20+ years (less than $2.5 billion per year), which would be a significant addition.

"Micron's U.S. memory manufacturing and R&D plans underscore our commitment to driving innovation and strengthening the domestic semiconductor industry," said Mehrotra. "This approximately $200 billion investment will reinforce America’s technological leadership, create tens of thousands of American jobs across the semiconductor ecosystem and secure a domestic supply of semiconductors — critical to economic and national security."

Micron expects all its U.S. projects to qualify for the Advanced Manufacturing Investment Credit (AMIC) and has secured support at the local, state, and federal levels. This includes up to $6.4 billion in CHIPS Act funding for two fabs in Idaho, two in New York, and the Virginia site upgrade.

Summary

KAIST, a leading Korean national research institute, has released a 371-page paper that details the evolution of high-bandwidth memory (HBM) technologies through 2038, showing increases in bandwidth, capacity, I/O width, and thermals. The roadmap spans from HBM4 to HBM8, with developments in packaging, 3D stacking, memory-centric architectures with embedded NAND storage, and even machine learning-based methods to keep power consumption in check. 

Keep in mind that the document is about the hypothetical evolution of HBM tech given the current direction of the industry and research, not an actual roadmap of a commercial company. 

HBM capacity per stack will increase from 288 GB to 348 GB for HBM4, to 5,120 GB to 6144 GB for HBM8. Also, power requirements will scale with performance, rising from 75W per stack with HBM4 to 180W with HBM8. 

Between 2026 and 2038, memory bandwidth is projected to grow from 2 TB/s to 64 TB/s, while data transfer rates are set to rise from 8 GT/s to 32 GT/s. The I/O width per HBM package is also set to increase from the 1,024-bit interface of today's HBM3E to 2,048 bits with HBM4 and then all the way to 16,384 bits for HBM4. 

We already know pretty much everything about HBM4 and we know that HBM4E will add customizability to base dies to make HBM4E more tailored for particular applications (AI, HPC, networking, etc.). 

Expect such capabilities to remain in HBM5, which will also deploy stacked decoupling capacitors and 3D cache. With a new memory standard comes increased performance, so HBM5, expected to arrive in 2029, will retain HBM4's data rate but is projected to double the I/O count to 4,096, thereby raising bandwidth to 4 TB/s and per-stack capacity to 80 GB. 

Per stack power is expected to grow to 100 W, which will require more advanced cooling methods. Interestingly, KAIST expects HBM5 to continue using microbump technology (MR-MUF), although the industry is reportedly already looking at direct bonding with HBM4. In addition, HBM5 will also integrate L3 cache, LPDDR, and CXL interfaces on the base die, alongside thermal monitoring. KAIST also expects AI tools to start playing a role in optimizing physical layout and jitter reduction with the HBM5 generation.

HBM6 is projected to take over in 2032, increasing transfer speed to 16 GT/s and per-stack bandwidth to 8 TB/s. Capacity per stack is expected to reach 120 GB, and power climbs to 120W. Researchers at KAIST believe that HBM6 will adopt direct bonding without bumps, along with hybrid interposers combining silicon and glass. Architectural changes include multi-tower memory stacks, internal network switching, and extensive through-silicon via (TSV) distribution. AI design tools expand in scope, incorporating generative methods for signal and power modeling.

HBM7 and HBM8 will push things further, with HBM8 reaching 32 GT/s and 64 TB/s per stack. Capacities are projected to expand to 240 GB. Packaging is believed to adopt full 3D stacking and double-sided interposers with embedded fluid channels.

While HBM7 and HBM8 will still formally belong to the family of high-bandwidth memory solutions, their architectures are expected to dramatically differ from what we know as HBM today. While HBM5 will add L3 cache and interfaces for LPDDR memory, these generations are projected to incorporate NAND interfaces, enabling data movement from storage to HBM with minimal CPU, GPU, or ASIC involvement. That will come at the cost of power consumption, which is expected to be 180W per stack. AI agents will manage real-time co-optimization of thermal, power, and signal paths, according to KAIST.

Micron has confirmed its plans to raise memory prices, citing strong demand for DRAM and NAND flash in the years ahead. The company's latest announcement suggests that prices will continue to rise through 2025 and 2026 as supply constraints and growing demand from AI, data centers, and consumer electronics drive up costs.

The price hike comes as the memory market rebounds from a period of oversupply and declining revenues. Over the past year, DRAM and NAND flash prices have steadily recovered due to production cuts by major suppliers and increasing demand for high-performance computing and AI workloads. With Micron confirming its intention to raise prices, other memory manufacturers such as Samsung and SK Hynix are expected to follow suit, further solidifying the upward pricing trend.

In a statement to its channel partners, Micron cited "un-forecasted demand across various business segments" as a key factor behind the price increase. The company highlighted the increasing demand for AI-related applications and the necessity of maintaining a competitive product portfolio as drivers for these adjustments. Micron also encouraged partners to provide long-term forecasts to help ensure supply stability in the coming years.

One of the key drivers behind the price increase is the growing demand for high-bandwidth memory (HBM), which is crucial for AI accelerators and next-generation GPUs. As companies like Nvidia, AMD, and Intel push for more advanced AI solutions, the need for faster and more efficient memory solutions has surged. Micron and its competitors are ramping up production of HBM to meet this demand, but supply remains tight, contributing to rising costs.

The company recently announced an investment of $7 billion in a new HBM assembly facility in Singapore, aiming to meet the increased demands driven by AI advancements. The plant is scheduled to commence operations in 2026 and should enhance Micron's capacity to produce HBM3E, HBM4, and HBM4E memory, positioning the company to capture a larger share of the growing HBM market.

Additionally, the broader consumer electronics market is expected to see a resurgence, with PC and smartphone manufacturers increasing orders for DRAM and NAND flash. As device makers prepare for new product launches in late 2025 and 2026, memory demand is projected to remain strong, further justifying Micron's price adjustments.

With Micron setting the stage for a new pricing trend, all eyes are now on how competitors and customers will respond. If demand remains strong, the industry could see sustained price increases, impacting everything from gaming PCs to enterprise data centers in the years ahead.

Micron introduced its 16Gb DDR5 devices made on its new 1γ (1-gamma) fabrication process that uses EUV lithography, a first for Micron, today on March 25. The new IC not only delivers higher performance than its predecessor, but it also consumes less power and is poised to be cheaper to make. The company also said that its 1γ manufacturing technology (6th Generation 10nm-class node) will eventually be adopted for other DRAM products.

DDR5 at 9200 MT/s

Micron's lead 1γ product is the company's 16Gb (2GB) DDR5 IC that is rated for a 9200 MT/s data transfer rate at an industry-standard voltage of 1.1V. Compared to its predecessor — a 16Gb DDR5 IC made on 1β fabrication process — the new device consumes 20% less power and features a 30% higher bit density, which may translate into a comparable decrease in production cost once the new chips achieve yields comparable to that of 1β 16Gb DRAM devices. 

While Micron rates its latest 16Gb DDR5 ICs at 9200 MT/s, this speed bin is significantly higher than anything in the latest edition of the DDR5 specification. The company stresses that the chip can run at JEDEC-compliant speed grades just fine, and the higher speed bin will enable some future proofing and compatibility with next-generation CPUs. Micron also suggests that CUDIMMs or CXL-based memory modules could leverage higher-than-JEDEC speeds. DIMMs for enthusiasts will also likely adopt the new DRAMs for their post-10,000 MT/s modules.

Micron is currently sampling its 16Gb DDR5 ICs made on 1γ technology and products on their base (i.e., chips and modules) with laptop and server manufacturers and expects their qualifications to be completed in one or two quarters. That means we should see Micron's latest memory devices in retail products starting mid-2025. The company expects all types of memory modules — for desktops, laptops, and servers — to adopt its new memory chips.

Considering the fact that Micron's 1γ-based DRAMs offer a combination of valuable qualities for all market segments — enhanced performance for desktops as well as lower power consumption for notebooks and servers — we indeed expect the firm's latest 16 Gb DDR5 ICs to become quite popular when they hit the market.

Over time, Micron will use its 1γ fabrication technology to make other types of memory products, including GDDR7, LPDDR5X (at up to 9600 MT/s), and data center-grade products, so the node will become a workhorse for the company.

1γ manufacturing technology

Micron's 1γ manufacturing process is the company's first technology to adopt extreme ultraviolet lithography (EUV), something that other leading memory makers adopted years ago. It's been a while in coming and looks to offer significant benefits relative to existing product lines.

Micron did not disclose how many EUV layers the new production node uses, but we can speculate that the company uses EUV for critical layers that would otherwise require the usage of multi-patterning, which lengthens production cycles and can affect yields. Micron does say that 1γ uses EUV in conjunction with multi-patterning DUV techniques. Also, Micron's 1γ DRAM process technology adopts next-generation high-K metal gate technology and an all-new back-end-of-line (BEOL) circuitry.

" In addition to EUV adoption in 1γ, we have introduced our next generation high-K metal gate CMOS and advanced back-end-of-line processes, which together enable the 9200 MT/s [data transfer rate], a 15% performance improvement over 1β DRAM […] while reducing power by about 20% over 1β," said Shigeru Shiratake, senior vice president of DRAM Technology Development at Micron.

For now, Micron produces its 1γ DRAMs at its fabs in Japan, where the company's first EUV tool was installed in 2024. As the company ramps up production of 1γ memory, it will add more EUV systems to its fabs in Japan and Taiwan.

SanDisk on Wednesday introduced an interesting new memory that could wed the capacity of 3D NAND and the extreme bandwidth enabled by high bandwidth memory (HBM). SanDisk's high-bandwidth flash (HBF) memory enables access to multiple high-capacity 3D NAND arrays in parallel, thus providing plenty of bandwidth and capacity. The company positions HBF as a solution for AI inference applications that require high bandwidth and capacity coupled with low power requirements. The first-generation HBF can enable up to 4TB of VRAM capacity on a GPU, and more capacity in future revisions. SanDisk also foresees this tech making its way to cellphones and other types of devices. The company hasn't announced a release date yet.

"We are calling it the HBF technology to augment HBM memory for AI inference workloads," said Alper Ilkbahar, memory technology chief at SanDisk. "We are going to match the bandwidth of HBM memory while delivering 8 to 16 times capacity at a similar cost point."

Conceptually, HBF is similar to HBM. It stacks multiple high-capacity, high-performance flash core dies interconnected using through-silicon vias (TSVs) on top of a logic die that can access flash arrays (or rather flash sub-arrays) in parallel. The underlying architecture of HBF is SanDisk's BICS 3D NAND using the CMOS directly bonded to Array (CBA) design that bonds a 3D NAND memory array on top of an I/O die made using logic process technology. That logic may be a key to enabling HBF.

"We challenged our engineers and said, what else could you do with this power of scaling," said Alper Ilkbahar. "The answer they came up with […] was moving to an architecture where we divide up this massive array into many, many arrays and access each of these arrays in parallel. When you do that, you get massive amounts of bandwidth.  Now, what can we build with this? We are going to build high bandwidth flash."

Traditional NAND die designs often treat the core NAND flash memory array as planes, pages, and blocks. A block is the smallest erasable area, and a page is the smallest writable area. HBF seems to break the die into 'many, many arrays' so they can be accessed concurrently. Each sub-array (with its own pages and blocks) presumably has its own dedicated read/write path. While this resembles how multi-plane NAND devices work, the HBF concept seems to go far beyond them.

For now, SanDisk says that its 1st-Gen HBF will use 16 HBF core dies. To enable such devices, SanDisk says that it has invented a proprietary stacking technology that features minimal warpage to enable stacking 16 HBF core dies, and a logic die that can simultaneously access data from multiple HBF core dies. The complexity of logic that can handle hundreds or thousands of concurrent data streams should be higher than that of a typical SSD controller.

Unfortunately, SanDisk does not disclose the actual performance numbers of its HBF products, so we can only wonder whether HBF matches the per-stack performance of the original HBM (~ 128 GB/s) or the shiny new HBM3E, which provides 1 TB/s per stack in the case of Nvidia's B200.

The only thing we know from a SanDisk-provided example is that eight HBF stacks feature 4 TB of NAND memory, so each stack can store 512 GB (21x more than one 8-Hi HBM3E stack that has a capacity of 24 GB). A 16-Hi 512 GB HBF stack means that each HBF core die is a 256 Gb 3D NAND device with some complex logic enabling die-level parallelism. Funneling hundreds of gigabytes of data per second from 16 3D NAND ICs is still quite a big deal, and we can only wonder how SanDisk can achieve that.

What we are sure about is that HBF will never match DRAM in per-bit latency, which is why SanDisk stresses that HBF products are aimed at read-intensive, high-throughput applications, such as big AI inference datasets. For many AI inference tasks, the critical factor is high throughput at a feasible cost rather than the ultra-low latency that HBM (or other types of DRAM) provides. So, while HBF may not replace HBM any time soon, it might occupy a spot on the market that requires high-capacity, high-bandwidth, NAND-like cost but not ultra-low latency. To simplify the transition from HBM, HBF has the same electrical interface with some protocol changes, though HBF is not drop-in compatible with HBM.

" We have tried to make it as close as possible mechanically and electrically to the HBM, but there are going to be minor protocol changes required that need to be enabled at the host devices," said Ilkbahar.

SanDisk didn't touch on write endurance. NAND has a finite lifespan that can only tolerate a certain number of writes. While SLC and pSLC technologies offer higher endurance than the TLC and QLC NAND used in consumer SSDs, this comes at the expense of capacity and adds cost. NAND is also typically written to at block granularity, whereas memory is addressable at the cache line level (i.e. typically 128KB for NAND blocks versus 32 bytes for a cache line). That's another key challenge.

SanDisk has a vision of how its HBF will evolve over three generations. Nonetheless, for now, SanDisk's HBF is largely a work in progress. SanDisk wants HBF to become an open standard with an open ecosystem, so it is forming a technical advisory board consisting of 'industry luminaries and partners.

Chinese memory makers are slowly but surely adopting the production of high-bandwidth memory (HBM) for AI and HPC processors. This week, Nikkei reported that a third China-based producer, Tongfu Microelectronics, began sampling its HBM products with select customers. Such action indicates that the ecosystem required to make this type of memory is developing. Interestingly, AMD is a major client and shareholder of Tongfu.

Truth to be told, Tongfu Microelectronics is not exactly a DRAM maker. The company is the world's third-largest outsourced semiconductor assembly and test (OSAT) service provider, and its most notable customer is AMD through their TF-AMD joint venture. Participation of a major OSAT in the Chinese HBM race makes its contribution even more intriguing. For now, Tongfu is sampling HBM2 memory packages with select customers, Nikkei claims, citing "multiple" sources.

HBM memory uses specially designed DRAM dies stacked on top of a base die and interconnected through silicon vias (TSVs). Tongfu Microelectronics is not a maker of memory or logic, so it sources DRAM dies, and base dies from third parties and then does another hard part: assembly and testing of these components into HBM2 stacks that can be used with various processors. It is unclear whether Tongfu actually offers HBM2 integration services, but the service is not listed on its website. However, Nikkei claims that it is a supplier of Huawei, which has AI processors with HBM (which does not mean that Tongfu offers Huawei appropriate services).

The history of Tongfu Microelectronics is also interesting. The year 2015 was not exactly AMD's best year as the company was nearly facing bankruptcy, so in late 2015 it agreed to form a joint venture with Nantong Fujitsu Microelectronics (NFME), contributing its assembly and test (ATMP) facilities in Suzhou (China) and Penang (Malaysia) in exchange for $371 million cash and an equity stake in the newly formed entity called AMD's Assembly, Test, Mark, and Packaging (ATMP). Eventually, NFME was integrated into Tongfu Microelectronics through corporate restructuring, and now the latter manages the joint venture TF-AMD together with AMD.

ATMP and then TF-AMD inherited AMD's advanced packaging IP, though it is unclear whether this includes vertically-stacked packaging in general and TSVs. Still, all AMD client CPUs are packaged in China by Tongfu.

Tongfu is not the only HBM assembler in China. ChangXin Memory Technologies (CXMT), China's most advanced DRAM maker, has been producing HBM2 for some time. In addition, Wuhan Xinxin also began to ramp up HBM2 production in March 2024.

SK hynix this week posted record earnings, operating profits, and net income for 2024. The company managed to more than double its revenue and return to profitability mainly due to increased demand for premium types of dynamic random access memory (DRAM) such as HBM3/HBM3E as well as enterprise-grade SSDs.

SK hynix earned ₩66.193 trillion in revenue ($46.054 billion), ₩23.467 trillion in operating profit ($16.327 billion), and ₩19.797 trillion ($13.776 billion) net profit for the whole of 2024. That compares to ₩32.765 trillion in revenue ($22.8 billion) and a net loss of ₩9.138 trillion ($6.36 billion) in 2023. The company's operating margin was 35% in 2024, up from -24% in 2023. Although the company's revenue climb slowed in the fourth quarter, which is in line with seasonality and dropping 3D NAND prices, SK hynix operating margin still increased during the quarter as it ramped up production of premium HBM3E memory.

The company attributes its success to increased sales of AI memory products, including HBM3 and HBM3E for the latest accelerators like Nvidia's H100, H200, and B100/B200 as well as enterprise-grade solid-state drives that are used for AI training and inference systems. High-end AI servers tend to also contain a lot of memory, so while SK hynix did not explicitly mention high-density server-grade DDR5 chips and modules, they clearly played a role in its great results.

"With significantly increased portion of high value-added products, SK hynix has built fundamental to achieve sustainable revenues and profits even in times of market correction." said Kim Woohyun, Vice President and Chief Financial Officer (CFO) at SK hynix. "While maintaining the profitability-first commitment, the company will make flexible investment decisions in line with market situation."

Looking ahead, SK hynix expects continued growth in AI-driven markets, fueled by global investments in AI servers for training and inference. While the consumer market — client PCs and smartphones — is expected to slow down, AI-equipped devices will drive growth in the second half of the year.

To meet this demand, SK hynix will increase HBM3E supply, develop HBM4, and transition to advanced processes for DDR5 and LPDDR5 production. For 3D NAND flash memory, the company will prioritize profitability and flexible sales strategies.

Leading South Korean memory manufacturer SK hynix announced that it will showcase a suite of advanced memory solutions tailored for artificial intelligence (AI) applications at this year's Consumer Electronics Show (CES) in Las Vegas.

Building upon its 12-layer High Bandwidth Memory (HBM) technology, the company will display samples of its latest 16-layer HBM3E products, officially announced in November 2024. This advancement employs advanced MR-MUF processes to enhance thermal performance and mitigate chip warping, achieving industry-leading results.

With capacities of 48GB (3GB per individual die) per stack, the increased density will allow AI accelerators to utilize up to 384GB of HBM3E memory in an 8-stack configuration. The 16-layer HBM3E is designed to significantly boost AI learning by up to 18% and inference performance by up to 32% compared to the 12-layer version.

Nvidia's next-gen Rubin chips are slated for mass production later next year, thus the existence of HBM3E could be shortlived, as the new upcoming Nvidia chips will be based on HBM4. That shouldn't be a concern, though, as reports indicate that SK hynix achieved its tape-out phase in October 2024.

Addressing the escalating demand for high-capacity storage in AI data centers, SK hynix will also introduce new SSD solutions for enterprise users, including the 122TB 'D5-P5336' enterprise SSD, developed by its subsidiary Solidigm. This model is said to boast the highest capacity currently available in its category and is poised to set new standards in data storage solutions.

The memory and storage manufacturer will also talk about Compute Express Link (CXL) and Processing-In-Memory (PIM) technologies, which are said to be pivotal to the next generation of data center infrastructures. Modularized solutions like the CMM-Ax and AiMX will be featured, with the CMM-Ax being hailed as a groundbreaking solution that combines the scalability of CXL with computational capabilities, boosting performance and energy efficiency for next-generation server platforms.

With on-device AI becoming a popular trend, SK hynix also has plans to showcase 'LPCAMM2' and 'ZUFS4.0,' designed to enhance data processing speed and power efficiency in edge devices such as PCs and smartphones. These innovations aim to facilitate the integration of AI capabilities directly into consumer electronics, broadening the scope of AI applications.

The company announced last year that it was also working on a range of other products, including PCIe 6.0 SSDs, high-capacity QLC (Quad Level Cell) eSSDs made specifically for AI servers, and UFS 5.0 for mobile devices. SK hynix is also working on an LPCAMM2 module and soldered LPDDR5/6 memory using its 1cnm-node to power laptops and handheld consoles.

Today, Micron Technology has started constructing its multi-billion-dollar packaging facility for high-bandwidth memory (HBM) in Singapore. The company will invest $7 billion in the plant, as it expects demand for HBM3E, HBM4, and HBM4E memory to skyrocket in the coming years amid the AI boom. The facility is set to start operations in 2026.

Micron's packaging facility for high-bandwidth memory (HBM) is located next to Micron's existing fabs in Singapore that produce 3D NAND and DRAM. The new HBM assembly plant will commence production in 2026 and then plans to substantially increase its capacity in 2027. The facility will use advanced AI-driven automation to boost operational efficiency, though the company does not disclose where and how artificial intelligence will be used.

While Micron is leading the industry with premium HBM3E memory, when it comes to HBM market share, the company is still an underdog compared to Samsung and SK hynix. To some degree, this is conditioned by the fact that Micron does not have as vast DRAM manufacturing capacity as its rivals from South Korea (while HBM memory dies take up more capacity than conventional memory ICs). Still, to a certain degree, this can be attributed to the lack of vast HBM assembly capacity.

Micron is gradually increasing its HBM3E output at its existing facilities, hoping to grab a mid-20% HBM market share in mid-2025. However, with the new Singapore assembly facility coming online in 2026, the company hopes to get an even larger chunk of the market.

"As AI adoption proliferates across industries, the demand for advanced memory and storage solutions will continue to increase robustly," said Sanjay Mehrotra, president and CEO of Micron. "With the continued support of the Singapore government, our investment in this HBM advanced packaging facility strengthens our position to address the expanding AI opportunities ahead."

Although the new facility will be tailored for assembling HBM stacks, it can also be used to assemble 3D NAND packages since assembly technologies with through-silicon vias (TSVs) are generally similar.

The project will initially create around 1,400 jobs; the expansion could potentially increase that number to 3,000. These roles will include packaging development, assembly, and testing operations.

"This is Singapore’s first high-bandwidth memory advanced packaging facility, allowing us to contribute to global AI growth," said Png Cheong Boon, Singapore Economic Development Board Chairman. "It expands Singapore’s partnership with Micron and further strengthens the semiconductor ecosystem in Singapore."

The US government may introduce a new series of sanctions affecting at least 200 domestic chip makers in China, per a report from Reuters. These new regulations came to light in an e-mail by the US Chamber of Commerce to its members - the largest lobbying group in the US - also hinting at a forthcoming ban-hammer on the export of High Bandwidth Memory (HBM).

For the past few years, the US and China have waged a chip war, and recent policies have limited the export of equipment manufactured on US soil. These policies have not only thrown a wrench in China's ambitions to rise to the highest ranks in the semiconductor world but also the finances of global players like Nvidia, who are barred from exporting high-performance GPUs to China. Nonetheless, China is ambitiously working hard to achieve a sort of chip-autarky, but that comes with its own set of caveats, as it'd not only have to design its chips but also manufacture them.

The update sheds light on the Biden administration's recent efforts to impose stricter regulations on chip manufacturers in China. The latest swarm of sanctions reportedly targets roughly 200 Chinese firms, and US companies are prohibited from exporting select technologies or products to them. Reuters suggests that the US Department of Commerce aims to push these new regulations before the Thanksgiving break - or November 28. Neither the Department of Commerce nor the Chamber of Commerce responded to Reuters' request for comments.

Moreover, another wave of sanctions is set to follow in December targeting the export of HBM (High Bandwidth Memory), primarily to choke China's advance in the AI domain. The impacts of these restrictions are materializing given that Huawei's Kirin SoCs and Ascend AI accelerators will reportedly remain stuck at 7nm technology until 2026 as SMIC fails to procure cutting-edge Extreme Ultraviolet (EUV) machines from ASML necessary for their manufacture.

Globally, a handful of giants such as Intel and Samsung have their own chip design and manufacturing departments. Even then, Intel's latest processors are outsourced to Taiwan's TSMC - see Arrow Lake and Lunar Lake, for example. This means China must design its chips internally, which is a very complex task in itself, but then has to fabricate chips based on these designs on homegrown wafers from firms like SMIC. Post 7nm technology, overcoming the EUV barrier is compulsory if China is to achieve sub-5nm chip manufacturing capabilities and compete globally.

Nvidia has asked SK hynix to move up its delivery timeline for next-generation HBM4 memory chips by six months, according to SK Group Chairman Chey Tae-won, reports Reuters.

Initially, SK hynix planned to ship its HBM4 chips to customers in the latter half of 2025. Following Nvidia CEO Jensen Huang's request, the timeline was shortened, though the exact new schedule was not specified. Nvidia is currently working on its next-generation GPUs for AI and HPC that will use HBM4 memory (presumably codenamed Rubin). So the company must lay its hands on next-generation high-bandwidth memory sooner rather than later.

SK hynix continues to solidify its lead in the HBM market, propelled by increased demand from the AI industry. The company supplied Nvidia with 8-Hi and 12-Hi HBM3E for the company's current-generation products and, looking ahead, SK hynix plans to launch 12-layer HBM4 next year and aims to roll out a 16-layer version by 2026, aligning with anticipated industry needs.

Initially, SK Hynix was leaning towards using 1b DRAM technology for its HBM4 layers, but Samsung's choice of the more advanced 1c production technology has reportedly prompted SK hynix to reevaluate its approach.

The upcoming HBM4 standard will introduce memory layers of 24Gb and 32Gb, along with stacking options of 4-high, 8-high, 12-high, and 16-high TSV stacks. The exact configurations of initial HBM4 modules are still uncertain, Samsung and SK hynix plan to begin mass production of 12-high HBM4 stacks in the latter half of 2025. Speed bins of these modules will vary, depending on numerous factors, but JEDEC's preliminary standards set speeds of up to 6.4 GT/s.

To manufacture base dies for its HBM4 modules, SK hynix is partnering with TSMC. At the European Technology Symposium 2024, TSMC disclosed that it would produce these base dies using its 12FFC+ (12nm-class) and N5 (5nm-class) process technologies. The N5 process will enable higher logic density and finer interconnect pitches, which will allow memory to be directly integrated into CPUs and GPUs. Alternatively, the 12FFC+ process will provide a more cost-effective solution by using silicon interposers to connect memory with host processors, striking a balance between performance and affordability.