The race to accelerate artificial intelligence is no longer solely dependent on building more powerful GPUs. d-Matrix has detailed Raptor, its upcoming architecture for inference, which will use stacked DRAM directly integrated with logic to achieve up to 100 TB/s bandwidth with 32 GB of capacity, according to the specifications presented by the company. The proposal aims to reduce one of the main costs of current models: constantly moving enormous amounts of data between memory and computation units.
The keys to Raptor and 3D DRAM memory in 20 seconds
- d-Matrix proposes 32 GB of 3D DRAM and around 100 TB/s bandwidth.
- The company measures transfer energy consumption close to 0.37 pJ/bit with its technology.
- The logic is placed directly on the DRAM via 3D integration.
- The goal is to approach SRAM bandwidth with much higher capacity.
- Raptor is specifically designed for AI model inference.
This technology does not appear out of nowhere. d-Matrix introduced its 3DIMC concept in 2025 and later manufactured Pavehawk, an experimental chip with which it claims to have validated stacking DRAM memory within its power and performance targets. Raptor will be the commercial product aiming to bring this architecture to data centers.
The approach is particularly interesting because it addresses a growing problem alongside the models themselves: accelerators need to store increasing weights and maintain KV (Key-Value) caches that grow with longer contexts and higher user concurrency.
One of the slides shown by d-Matrix illustrates the problem with Kimi K3. Assuming 8-bit weights, the company estimates a model size of about 2,800 GB. For its KV cache example, a context of one million tokens requires 14.6 GB per user; with 64 concurrent users, that amount would be around 935 GB.
The message is simple: memory is becoming one of the main physical constraints in AI inference.
Between SRAM and HBM, a third option emerges
Current architectures must navigate a difficult compromise.
SRAM is extremely fast and can be integrated very close to the calculation units. For example, d-Matrix’s current Corsair accelerator uses an architecture based on SRAM precisely to achieve low latency during inference.
The company estimates the bandwidth of a couple of Corsair cards at around 300 TB/s, with an approximate latency of 1 nanosecond.
The problem is capacity.
A conventional 6T SRAM cell takes up much more space than a DRAM cell, and its density is insufficient to store the enormous models emerging today. d-Matrix places the practical limit of its SRAM solution around 4 GB per card pair.
HBM (High Bandwidth Memory) partially solves this problem.
By stacking multiple DRAM chips and connecting them via interposers, manufacturers can offer hundreds of gigabytes alongside GPUs and accelerators. NVIDIA, AMD, and nearly the entire AI acceleration industry depend on this architecture.
But HBM also has physical limits.
The number of stacks around the processor is limited by available space in the package, and increasing bandwidth involves more and faster connections.
d-Matrix estimates that configurations based on HBM4 will reach around 20 TB/s in next-generation systems, well below the bandwidth SRAM can provide.
Their solution is to change the problem’s geometry.
Instead of placing memory around the processor, Raptor places the logic directly on top of the DRAM.
100 TB/s without relying on a traditional HBM interface
The 3DIMC (3D In-Memory Compute) architecture uses extremely short vertical connections between memory and compute units.
d-Matrix explains that this allows eliminating much of the physical interfaces needed when data must move between separate chips.
Their previous documentation already anticipated roughly 20 TB/s per stack, about ten times the bandwidth per stack of what the company uses as a reference for HBM4, with a power consumption between 0.3 and 0.4 pJ/bit.
Raptor expands on this concept.
The presented configuration combines a logic die manufactured using TSMC’s N4 process with custom DRAM, connected by face-to-face links with approximately 36-micron spacing.
The company proposes a 32 GB configuration capable of delivering around 100 TB/s bandwidth.
The energy figure is equally notable: 0.37 picojoules per bit.
In comparison, d-Matrix references an HBM4 setup of about 192 GB and 18–20 TB/s, with a consumption of several picojoules per bit.
Cautions are necessary with comparisons. These are different architectures with varying capacities and metrics provided by the manufacturer itself. Additionally, Raptor is still in development, and no commercial platform has been deployed to verify these figures in independent applications.
d-Matrix does have experimental silicon. The company confirms that Pavehawk has validated their 3D DRAM technology, with tests across different voltages and temperatures recording approximately 0.4 pJ/bit even in the most challenging scenarios studied.
The big saving comes from moving less data
The energy cost associated with memory is becoming a significant issue for AI data centers.
It’s not enough to measure how much energy a matrix multiplication consumes. You also need to account for the energy required to fetch data from memory, transport it to the processor, and return the results.
Physically greater distances between memory and compute generally increase these costs.
d-Matrix calculates that a hypothetical HBM architecture capable of reaching 100 TB/s with a consumption of 2.4 pJ/bit would need about 1.92 kW solely to move data from memory.
The 3D architecture dramatically reduces this distance.
DRAM banks can connect directly with tensor engines placed on top. Raptor incorporates 256 tensor engines per chiplet, according to the technical specifications.
The company also aims to physically align memory banks with these engines to further reduce internal movement.
This approach explains why d-Matrix talks about memory-centric computing. The goal isn’t just to make faster DRAM but to redesign memory, compute, and interconnects together.
3D stacking creates new challenges too
Stacking logic and memory one above the other removes some limitations but introduces others.
One major concern is temperature.
Calculation circuits generate much more heat than conventional DRAM. If the processor were placed below, that heat would have to pass through memory layers before reaching the cooling system.
d-Matrix has chosen to place logic on top, facilitating contact with a cold substrate.
However, the DRAM must operate at higher-than-normal temperatures. The architecture is designed to sustain a junction temperature up to about 105°C.
This requires adjusting the memory refreshing process.
Raptor’s DRAM refresh cycles occur approximately every 4 milliseconds—about eight times more frequently, according to technical documentation. The company estimates this mechanism consumes about 1.37% of the available bandwidth.
Manufacturing also presents challenges.
A 3D architecture is economically viable only if memory defects do not force discarding entire sets. d-Matrix incorporates spare banks and error correction mechanisms to increase fault tolerance.
These include a Reed-Solomon T=2 scheme with CRC, designed to correct errors within transferred blocks.
Raptor is aimed at inference phase
d-Matrix does not seek to compete directly with GPUs across all workloads.
Its architecture is specifically tailored for low-latency generative inference, a scenario where the relationship between calculation capacity and memory bandwidth differs from training.
Token generation can be heavily memory-limited. Model weights must be repeatedly read as the KV cache associated with each user’s context grows.
Larger models, longer contexts, and more concurrent users increase pressure on the memory subsystem.
That’s why d-Matrix is also developing heterogeneous architectures where GPUs and specialized accelerators perform different parts of the process. The company suggests that combining Corsair with GPUs in disaggregated pipelines can significantly enhance inference performance, though results depend on the model and configuration.
Raptor represents the next phase.
The company is developing the accelerator with tech partners like Alchip, while Andes Technology will supply RISC-V AX46MPV cores for orchestration and vector computation. d-Matrix has already confirmed that Raptor will be its first commercial product based on 3DIMC.
The current challenge is to determine how much of the experimental silicon’s performance can be translated into production.
Achieving 100 TB/s with 32 GB of memory and about 0.37 pJ/bit would be a rare combination of bandwidth and efficiency. But Raptor must also demonstrate sustained performance, reliability, manufacturing costs, chip yield, and large-scale production capability.
The industry is actively seeking solutions to this challenge. The next generation of AI infrastructure will require not just more FLOPS, but the ability to feed those FLOPS with increasing data while avoiding turning memory into the system’s energy and performance bottleneck.
Frequently Asked Questions
What is d-Matrix’s 3D DRAM memory?
It’s a architecture that stacks logic and DRAM physically to create very short vertical connections. d-Matrix calls its technology 3DIMC and intends to use it commercially in its Raptor accelerator.
What bandwidth does Raptor promise?
The configuration announced by d-Matrix targets approximately 100 TB/s with 32 GB of 3D DRAM. These are manufacturer specifications for an architecture still in development.
Will 3D DRAM replace HBM?
Not necessarily. HBM offers high capacity and a broad industrial ecosystem. d-Matrix positions its 3D DRAM as an alternative, especially suited for inference workloads where bandwidth and data movement power are critical.
How much energy does Raptor’s memory consume?
d-Matrix reports approximately 0.37 pJ/bit, with earlier tests on Pavehawk showing values near 0.4 pJ/bit. The total performance of the commercial platform will be confirmed once Raptor is available.
via: wccftech
