Two research teams from South Korea and Japan have proposed novel ways to integrate memory for artificial intelligence accelerators. Their proposals, called V-Die and MOSAIC, position DRAM memory chips vertically, supported on one edge, instead of stacking them horizontally as high-bandwidth memory (HBM) does.
The key points of V-Die and MOSAIC in 20 seconds
- Both projects place DDRAM chips along their edges to increase capacity and better dissipate heat.
- V-Die combines bottom connections with liquid cooling between chips.
- MOSAIC transmits data via inductive coupling without physical contact.
- Both are still in experimental phases and must demonstrate cost, reliability, and industrial performance.
The research was presented in June at the IEEE/JSAP Symposium on VLSI Technology and Circuits 2026. Although originating from independent teams using different connection methods, they share a common idea: the continual increase in stack height of HBM modules begins to pose challenging problems for conventional architectures.
HBM brings several dynamic random-access memory (DRAM) chips close to the processor and communicates via a very wide interface. This proximity allows for faster data movement with less energy compared to memory modules located farther from the GPU.
This system is suitable for training and running AI models, which need continuous access to parameters, activations, and data. However, each additional layer increases complexity. The heat generated inside must pass through silicon, bonding materials, and other layers before reaching the cooling system.
Silicon vias, known as TSVs (Through-Silicon Vias), also consume part of the chip. These metallic connections pass vertically through memory layers, transporting data and power but reducing space for the DRAM cells themselves. A taller stack can offer greater capacity but also demands more connections and complicates heat dissipation.
V-Die eliminates through-chip connections
V-Die’s proposal comes from a collaboration involving researchers from the Ulsan National Institute of Science and Technology (UNIST), Korea Advanced Institute of Science and Technology (KAIST), and Hanbat National University.
The design rotates conventional DRAM chips by 90 degrees and places them on a substrate next to the processor. Instead of TSVs, each chip uses its own I/O pins on the bottom edge. Connections to the substrate are distributed approximately every 20 micrometers.
Removing vertical vias leaves more surface area for data storage and prevents all chips from relying on a common matrix of through-stack connections. According to team estimates, this arrangement could enable four times more links than HBM4 and cut memory read times by 37%. These results are based on modeling, not on measurements from commercial systems.
Another key difference is cooling. V-Die maintains gaps between the vertical chips to allow channels for liquid coolant circulation. The fluid flows close to heat-generating surfaces, rather than relying solely on heat rising through the structure.
Simulations suggest operating temperatures around 45°C, compared to maximum values over 80°C in dense HBM configurations. However, this comparison depends on packaging, power consumption, and cooling design, so it cannot directly apply to any specific accelerator.
The team also simulated a setup with 16 chips, comparable to hardware based on NVIDIA’s H100 and similar to a GPT-3 load. V-Die achieved 540 tokens per second, versus 296 tokens per second for the HBM4 configuration of similar memory capacity.
The time to generate the first token was reduced by 32%, roughly 24 milliseconds in the evaluated scenario. This indicates that a wider interface might accelerate inference, but further validation with a physical prototype is needed. Researchers are working on such a device to assess its electrical and thermal behavior.
MOSAIC uses microscopic wireless connections
The Japanese MOSAIC project, developed by researchers from the University of Tokyo along with the University of Tohoku and the RIKEN institute, addresses a different practical challenge.
When numerous chips are placed on their edges, slight variations in thickness can shift connection points. Errors of a few micrometers per piece accumulate, potentially preventing pads from aligning correctly with substrate contacts.
MOSAIC bypasses part of this challenge through inductive coupling. The system employs small facing coils to transmit information across a microscopic gap, without direct metallic connections for each signal. Current in one coil creates a magnetic field that induces a corresponding signal in the other.
This connection tolerates misalignments better because the coils do not require super-precise overlap like direct contact. Power supply would still use conventional, larger, and fewer connections on the sides of the package.
The prototype presented at the symposium reached up to 4 Gbps per channel. The researchers argue that a memory arrangement directly atop a GPU could double the capacity of a system comparable to HBM4 without TSVs. The work was selected as a candidate for the best student paper award at the conference.
A configuration with 98 chips could provide 294 GB of memory. The vertical layout exposes large silicon surfaces, facilitating heat dissipation, although MOSAIC does not include the liquid channels proposed by V-Die.
The team also developed a MOSAIC variant with physical micro-connections. That related work achieved alignment under six micrometers and demonstrated thermal conductivity up to three times higher than a conventional stack, with capacity increases up to 30%. These results are from a different demonstration of the inductive interface, not a single prototype, and should not be conflated.
Two promising proposals, but still far from manufacturing
V-Die and MOSAIC address the same fundamental challenge from different angles. The South Korean project emphasizes bandwidth, connection density, and liquid cooling. The Japanese effort aims to overcome the difficulty of assembling large vertical chip stacks without requiring perfect alignment of all contacts.
Neither is ready to replace commercial HBM. V-Die remains largely in simulation, needing to demonstrate real-world effectiveness of its cooling channels, bottom connections, and signal routing in a packaged system.
MOSAIC has experimental hardware but must prove that inductive coupling can scale to thousands of channels without excessive space or power consumption. Key concerns include manufacturability, cost, repairability of defective links, and long-term reliability during continuous operation.
Memory industry development continues along conventional HBM lines, with more layers, new base chips, and improvements in materials. These academic proposals do not invalidate that progress but show that increasing height might no longer be the sole way to boost capacity and bandwidth.
Lateral placement offers more surface area for heat extraction and prompts reconsideration of how memory and processors connect. Its future will depend less on modeling figures and more on the ability to mass-produce high-yield, cost-effective units.
Frequently Asked Questions
What is the difference between V-Die and HBM memory?
HBM stacks multiple DRAM chips horizontally, communicating through vias that pass through silicon. V-Die arranges chips on their edges, eliminating these vias, and incorporates bottom connections and liquid cooling channels.
What is MOSAIC in AI memory?
MOSAIC is an architecture that arranges DRAM chips perpendicularly to the GPU, using an experimental interface with small coils to transmit data via inductive coupling without direct metallic contacts.
Does V-Die truly achieve 540 tokens per second?
This figure comes from a simulation with 16 chips, comparable hardware to an H100, and a load similar to GPT-3. It has not yet been confirmed in a commercial accelerator or a complete prototype.
When could these memories reach the market?
No commercial release date has been announced. Both technologies need to pass manufacturing, cost, power, performance, cooling, and reliability tests before competing with HBM.
