SK hynix is already working on technologies that will push HBM (High Bandwidth Memory) beyond current configurations. At Hot Chips 2026, the company showcased solutions involving hybrid bonding, new heat dissipation methods, more TSV connections, and future 3D designs where memory could be placed directly over AI accelerators. The manufacturer also included Intel’s EMIB among the 2.5D packaging technologies considered for integrating HBM.
The key points for the future of HBM in 30 seconds
- SK hynix is exploring hybrid bonding to overcome 16-layer stacks and reduce connection distances.
- HBM4 surpasses 2 TB/s per stack and doubles its I/O lines to 2,048 bits.
- The company is developing I-HBM, a solution to extract heat from particularly hot spots.
- It also considers Intel’s EMIB alongside various CoWoS variants for 2.5D packaging.
- In the longer term, SK hynix is studying a 3D integration with HBM placed directly over the accelerators.
This presentation helps to explain why memory has become one of the most complex components in AI accelerators. Improving GPU performance is no longer just about adding more compute units. It’s essential to supply data quickly, which requires increasing capacity, bandwidth, and connections simultaneously.
HBM has advanced this goal through the vertical stacking of DRAM chips. These layers communicate via TSV (Through-Silicon Via), vertical connections passing through the silicon. The memory stack and processor remain separate chips, typically placed side-by-side on an interposer.
However, this architecture is beginning to face physical constraints that grow harder to resolve with each generation.
HBM4 takes packaging to another level of complexity
SK hynix uses HBM3E to demonstrate how far this evolution has progressed. Compared to a reference GDDR6 configuration with 24 GB and 768 GB/s, four HBM3E stacks can reach up to 144 GB and around 4 TB/s, occupying about half the space, according to the company’s data.
HBM4 raises the demands once again.
The latest generation from SK hynix features a 2,048-bit interface, compared to the 1,024 bits previously used, and can exceed 2 TB/s bandwidth per stack. The documentation includes DRAM chips up to 24 Gb, with 12-layer configurations in production and 16-layer designs in qualification.
A stack can also include more than 20,000 TSVs and 16,148 microbumps at the base, underscoring how performance now relies on advanced packaging technologies.
More connections mean more space is needed.
Even as manufacturers gradually reduce TSV size and spacing, increasing their number raises the area requirement. A more challenging issue for AI accelerators is heat.
SK hynix estimates that a roughly doubling of bandwidth every two generations could generate about 2.2 times more thermal load on existing process and packaging technologies.
This is a particularly delicate limitation since HBM stacks are located very close to GPUs that can consume hundreds of watts.
Hybrid bonding to overcome 16 layers
One of SK hynix’s solutions is hybrid bonding, a technology that allows connecting different layers with much smaller joints than traditional microbumps.
Currently, two main methods are employed in HBM packaging: TC+NCF (Thermo-Compression + Non-Conductive Film) and MR-MUF (Mass Reflow Molded Underfill).
Each has advantages and trade-offs. TC+NCF offers better resistance to chip deformation but has higher thermal resistance and lower productivity. MR-MUF enhances productivity and thermal transfer but presents challenges related to deformation and gaps between chips.
SK hynix uses an evolution of MR-MUF in its 16-layer HBM3E designs, but considers hybrid bonding as a necessary technology to continue increasing stack height.
Data from the manufacturer suggests connections with pitches below 18 microns. This technology would also allow for roughly 24% thicker memory chips, and according to their measurements, achieve up to 35% lower thermal resistance even with more layers.
This improvement is significant because reducing chip thickness to pack in more layers complicates manufacturing and handling.
The industry could thus move from current 12- and 16-layer configurations toward future stacks of 16 to 20 layers, although SK hynix has not provided a specific commercial timeline for these designs in this presentation.
I-HBM aims to extract heat directly from hot spots
The Korean manufacturer is also developing a solution called I-HBM to address hot spots within the memory stack itself.
The idea involves integrating a component with high thermal conductivity and electrical insulation into the PHY interface between chips. This creates a dedicated route to vent heat away from the most concentrated areas.
SK hynix claims this design can reduce thermal resistance by more than 30%.
This concept aligns with other ongoing industry research to manage memory thermally at the chip package level, rather than relying solely on cooling systems placed over GPUs and memory modules.
Additionally, the company considers distributing TSVs dedicated to power delivery across different chip zones. The goal is to improve the PDN (Power Delivery Network), which supplies power to various components.
The recurring challenge remains: higher bandwidth involves more connections and greater energy consumption to keep everything operational.
Intel’s EMIB considered alongside CoWoS
Another detail from the presentation is the mention of EMIB (Embedded Multi-die Interconnect Bridge) from Intel as one of the technologies SK hynix considers for 2.5D HBM solutions.
EMIB uses small silicon bridges embedded in the substrate to connect neighboring chips, avoiding the need for a large silicon interposer in certain configurations.
The technology is listed alongside CoWoS-S, CoWoS-R, and CoWoS-L, different variants of TSMC’s advanced package system.
Including EMIB is technically relevant, but does not automatically confirm a commercial agreement between Intel and SK hynix for future HBM manufacturing. The data pertains to integration and packaging methods.
There are also speculations about potential corporate moves between the two companies regarding memory, but the provided documentation does not confirm any definitive project.
The next step could be stacking memory directly on the accelerator
The most intriguing part of the presentation looks beyond current 2.5D packaging.
SK hynix envisions 3D architectures where HBM could be placed directly over the accelerators.
Currently, HBM is already three-dimensional in that its DRAM chips are stacked vertically. Still, the entire stack is typically placed adjacent to the GPU or XPU.
The next move would be to vertically integrate memory and processor, creating a more direct connection.
Reducing the physical distance would enable shorter connections and potentially increase bandwidth while reducing the energy used for data transport.
This is precisely the direction in which different semiconductor manufacturers are starting to explore, driven by AI model growth. Larger model weights and extended cache hierarchies demand more capacity, while compute units need faster data access.
The challenge is that placing memory on top of high-power processors complicates heat dissipation. It also raises issues related to power delivery, manufacturing yield, defect repair, and packaging costs.
Therefore, SK hynix’s presentation should be viewed as a forward-looking technological vision, not an immediate announcement of 3D HBM products.
The evolution of HBM4 and future generations will increasingly depend on advances in packaging, vertical interconnections, power supply, and especially cooling—not just on improving DRAM itself. These factors will determine how much bandwidth can continue to grow for AI accelerators.
Frequently Asked Questions
What bandwidth can SK hynix’s HBM4 achieve?
SK hynix’s data indicates HBM4 can deliver over 2 TB/s per stack, utilizing a 2,048-bit interface. The actual performance depends on specific configurations.
What is hybrid bonding in HBM used for?
It enables creating smaller connections between memory layers and can improve thermal behavior. SK hynix considers it as one of the technologies to surpass current stacks of up to 16 layers.
What is I-HBM?
It’s a technology under study by SK hynix designed to create a dedicated heat venting path near hot spots within the memory interface. The company reports it can reduce thermal resistance by over 30%.
Will SK hynix use Intel’s EMIB?
SK hynix showed EMIB as part of the 2.5D packaging options considered, alongside various CoWoS variants. However, the documentation does not confirm a specific commercial product based solely on EMIB.
Source: wccftech
