Samsung Electronics is reinforcing its cooling technologies for HBM memory as packages built for artificial intelligence pack more memory and logic into less space. The company is working on thermal structures inside the package itself, while TSMC increases its advanced CoWoS packaging capacity to keep up with the growth of AI accelerators.
Samsung and HBM cooling in 30 seconds
- Samsung has developed the Heat Path Block (HPB) to pull heat out of especially hot zones of HBM.
- The critical point is the D2D PHY interface, which connects the memory to the AI processor.
- Samsung is validating HPB on HBM4E and plans to carry it over to future generations such as HBM5.
- TSMC is expanding CoWoS for packages with more logic and HBM memory.
- Higher density is turning cooling into a question of package design itself, not just the system.
The race to boost the performance of artificial intelligence accelerators is creating a parallel problem: it’s getting harder and harder to pull the heat generated inside advanced packages. HBM (High Bandwidth Memory) stacks several layers of DRAM next to AI processors and lets enormous amounts of data move, but higher speed and density also raise thermal power.
Samsung has decided to tackle one of the most problematic spots from inside the memory itself. Its Heat Path Block (HPB) technology creates a dedicated thermal path to move heat away from the physical communication layer between memory and processor.
The company showed this architecture at COMPUTEX 2026 alongside an HBM4E design and an HBM5 prototype. Samsung explained that the D2D PHY (Die-to-Die Physical Layer) zone of the base die is one of the main areas of heat generation.
Heat concentrates at the link between HBM and GPU
The problem has to do with how HBM is built. Memory chips are stacked vertically and connected through high-speed interconnects. In AI systems, that memory sits next to GPUs, accelerators, or ASICs inside an advanced package.
As bandwidth increases, so does the activity of the interfaces that connect HBM to the processor. That zone can become a thermal hotspot that’s hard to cool using conventional solutions placed on top of the package.
Samsung aims to solve part of the problem by routing heat through a dedicated path. HPB is designed to reduce thermal resistance around the D2D PHY zone and help heat reach a structure capable of dissipating it.
The company already showed samples and demonstrations of this technology in 2026. In its COMPUTEX presentation, it said it was validating HPB in HBM4E and planned to use it in future HBM5 generations.
Samsung is also developing HBM4E with a base die built on its 4-nanometer process. The company has said this generation can reach up to 14 Gbps per pin, with room to scale to 16 Gbps and surpass 4 TB/s of bandwidth per stack in certain future configurations.
The speed increase isn’t an isolated detail. When more bandwidth is packed into a package with limited physical space, thermal design ends up being shaped by the semiconductor’s own architecture.
Samsung and SK hynix look for solutions inside HBM itself
Samsung isn’t the only company tackling this problem. SK hynix has presented its own roadmap for the next generations of HBM through iHBM, an architecture that builds cooling elements directly into the area around the physical interface.
Both companies are targeting the same problem with different architectures. Samsung uses HPB as a thermal path within the HBM design, while SK hynix has presented cooling elements built into the D2D area.
The competition is especially relevant because HBM5 is being designed for AI systems that will demand more bandwidth and higher memory density. As the number of layers and operating speed increase, pulling heat out of the internal zones gets harder.
Samsung is also exploring deeper changes to memory architecture. In 2026 it presented concepts such as zHBM, a three-dimensional memory proposal that aims to bring memory even closer to the processor through advanced bonding technologies.
The company says its zHBM architecture could cut thermal resistance by more than half compared with HBM5 under the conditions envisioned for that concept. It’s a future technology, not a feature available in current HBM4 products.
TSMC increases the size of CoWoS packages
The other side of the equation is in advanced packaging. TSMC uses CoWoS (Chip on Wafer on Substrate) to integrate processors and HBM memory into large packages.
The growth of AI models is pushing accelerator makers to pack more memory and more logic into the same package. That increases the package’s area and also the amount of heat that has to be managed.
TSMC has developed several CoWoS variants to keep up with this evolution. CoWoS-L, for instance, combines a redistribution-based interposer with silicon interconnects and targets larger high-performance computing products.
The company has already said its next-generation CoWoS technologies are designed to integrate more logic and HBM memory. In its corporate documentation, TSMC notes that CoWoS-L entered volume production and that its larger variants are aimed at products with bigger interposers, as it keeps expanding its CoWoS packaging capacity to meet AI chip demand.
The capacity increase is necessary because CoWoS has become one of the most important points in the AI accelerator manufacturing chain. A cutting-edge GPU can have an advanced design and enough HBM, but it still needs a packaging process capable of bringing all those components together into a single product.
Cooling is now part of chip design itself
For years, server cooling could be analyzed mainly from the system’s point of view: heatsinks, fans, liquid cooling, or water loops.
With large AI packages, part of the problem has shifted inside the package itself.
The processor, the HBM memory, the interposer, and the interconnects form a set in which thermal decisions are tied to electrical and mechanical ones. A zone that generates too much heat can limit operating frequency or require more complex external cooling systems.
That’s why Samsung is working on structures like HPB while TSMC keeps expanding CoWoS. They’re different answers to the same shift: AI accelerators are growing faster than the ability to dissipate the heat from their packages.
The problem will grow even bigger if future generations simultaneously increase the number of HBM layers, the bandwidth, and the processor’s power. In that scenario, improving only the server’s cooling system won’t be enough. Part of the solution will have to be built into the semiconductor and the package itself.
For Samsung, this also opens an opportunity to differentiate its future HBM generations. The company controls memory, logic manufacturing, and packaging technologies, which lets it work on several parts of the system in a coordinated way.
TSMC, for its part, is reinforcing the advanced packaging used by many of the largest AI accelerators. The expansion of CoWoS and the evolution of its 3DFabric technologies respond to demand that no longer depends solely on making smaller transistors.
The next battle in AI chips is therefore being fought in a much smaller space: inside the package. More memory and more compute capacity require better connections, more integration surface, and new ways to pull out heat before it limits performance.
Frequently Asked Questions
What is Samsung’s Heat Path Block?
The Heat Path Block (HPB) is a thermal technology developed by Samsung to create a dedicated path for moving heat out of HBM structures. The company is validating it in HBM4E and plans to use it in future generations such as HBM5.
Why does HBM generate so much heat?
Higher bandwidth and faster communication increase activity and power consumption in the interfaces that connect memory to the processor. The D2D PHY zone is one of the points Samsung identifies as especially problematic.
What does TSMC have to do with HBM?
TSMC integrates HBM memory and processors through advanced packaging technologies such as CoWoS. The growing size and complexity of AI accelerators is pushing the company to expand and evolve these technologies.
Is Samsung already using HPB in HBM5?
Samsung has shown prototypes and said it’s validating HPB in HBM4E with plans to adopt it in future HBM5 generations. This validation shouldn’t be confused with mass production of HBM5.
