d-Matrix used Hot Chips 2026 to share new details on Raptor, its next-generation accelerator for AI inference, built around an unusual architecture: stacking the compute die directly on top of a custom-built DRAM. The promised result is up to 100 TB/s of bandwidth per card with 32 GB of memory, a figure aimed squarely at one of generative inference’s biggest current bottlenecks: moving data to the compute units fast enough without blowing up power consumption.
Raptor and its 3D memory in 30 seconds
- Raptor stacks a 4nm compute die on DRAM using face-to-face bonding with a 36-micron pitch.
- The architecture reaches up to 100 TB/s of bandwidth per card with 32 GB of memory.
- d-Matrix measures 0.37 pJ/bit of energy use across the vertical interface.
- A paper presented at ISCA calculates up to 4.71x more throughput than an HBM-based variant.
- The design targets generative inference specifically, not a general replacement for training GPUs.
The pitch matters because much of the AI accelerator debate over the past few years has focused on boosting HBM (High Bandwidth Memory)’s capacity and speed. d-Matrix is proposing something different: bringing memory even closer to compute until the two effectively become a single three-dimensional structure.
The company had already shown its 3DIMC technology in 2025 through the experimental Pavehawk chip. Raptor will be the first commercial implementation of that architecture and will succeed Corsair, the inference accelerator d-Matrix currently has in production.
Raptor’s real target isn’t just computing faster
Token-by-token generation in a language model has an awkward property for hardware designers.
For much of the decode phase, an accelerator’s math units can have plenty of spare compute capacity, but they need a constant stream of weights, caches, and other data from memory.
The bottleneck then shifts from arithmetic operations to memory bandwidth.
The academic paper Early Silicon of Raptor: The First 3D-DRAM Accelerator for Generative Inference, presented at the 2026 International Symposium on Computer Architecture (ISCA), describes generative inference precisely as a workload that is predominantly memory-bound.
SRAM offers enormous bandwidth, but packing in large capacities is extremely expensive in silicon area. HBM significantly boosts capacity while keeping bandwidth high, though it still has to move information between separate chips through the package.
Raptor tries to shrink that distance even further.
Its compute die sits directly on top of a DRAM die developed specifically for the accelerator. The two are joined face to face through connections with a 36-micron pitch, creating a very high-density vertical interface.
The physical difference may look small, but moving a bit a few micrometers uses considerably less energy than doing it over longer links.
According to Sudeep Bhoja, d-Matrix’s co-founder and CTO, silicon the company has tested has measured around 0.37 picojoules per bit (pJ/bit) for that vertical movement.
d-Matrix had previously reported figures close to 0.4 pJ/bit during Pavehawk testing, so the new number is consistent with the progress the company had already been describing.
100 TB/s with just 32 GB: a deliberate choice
The headline spec is bandwidth.
Each Raptor card has 32 GB of memory and can hit up to 100 TB/s.
That figure far exceeds current HBM implementations’ bandwidth, but there’s an important nuance: Raptor isn’t trying to compete on huge memory capacity per card.
Its bet is elsewhere.
| Feature | Raptor 3D-DRAM | Typical HBM design |
|---|---|---|
| Memory-compute integration | DRAM stacked directly on logic | HBM placed beside the accelerator |
| Capacity per card | 32 GB | Can be considerably larger |
| Bandwidth | Up to 100 TB/s | Several TB/s depending on implementation |
| Data movement distance | Very short, via vertical interface | Longer, through the package |
| Main goal | Low-latency inference | Training and general inference |
| Advantage sought | Throughput and data-movement efficiency | High capacity and bandwidth |
Comparing the 100 TB/s figure directly against an HBM GPU on its own can lead to the wrong conclusion.
Raptor is built expressly for a specific work pattern. It’s not trying to be a general-purpose accelerator that can host every model and run any AI operation the way a conventional GPU does.
d-Matrix designs its architecture around generative inference with small batches, low latency, and a high number of simultaneous users.
The relatively limited 32 GB capacity also forces large models to be spread across multiple cards.
That’s where software, memory organization, and the system’s internal network come into play.
The 4.71x performance figure needs context
The academic paper tied to Raptor compares the 3D-DRAM architecture against hypothetical versions of the same accelerator using HBM and SRAM.
The researchers evaluated models including Llama 3.1 70B, DeepSeek-V3, Kimi K2, GPT-OSS, Whisper, and Canary.
According to the published results, the 3D-DRAM configuration delivers 4.71x more throughput than the HBM-based variant and 2.44x more than the SRAM-based one.
Those are striking numbers, but they don’t mean Raptor will run every model 4.7x faster than a commercial HBM GPU.
The comparison tries to isolate the effect of the memory architecture while holding other design elements constant under specific conditions. It’s an academic evaluation of the proposal, not an independent benchmark against specific commercial products from NVIDIA, AMD, or any other maker.
Some of the results also come from architectural modeling backed by early silicon characterization.
That’s precisely where the value lies: Raptor is no longer just a simulation. The team has fabricated and measured chips that let it verify essential parameters like power consumption, thermal behavior, and the reliability of the 3D interface.
Moving data costs more and more energy
One of the reasons the architecture is interesting comes down to energy consumption.
An accelerator’s power budget doesn’t go solely toward matrix multiplications.
Data also has to be transported.
d-Matrix compares its vertical interface’s roughly 0.37 pJ/bit with about 2.4 pJ/bit for moving information to the base die in an HBM4 implementation.
These aren’t directly interchangeable figures for calculating both systems’ total power consumption. The paths, interfaces, and functions being compared are different.
But they illustrate the physical principle the company is chasing: the shorter the distance between memory and compute, the less energy it takes to move each bit.
In inference that has to read enormous amounts of data over and over, that difference can add up quickly.
d-Matrix argues its approach cuts both energy and latency compared with architectures where memory and compute sit further apart. The company had already used Pavehawk to validate that hypothesis before carrying it over to Raptor.
A DRAM built specifically to sit under the accelerator
Raptor’s other distinctive feature is that d-Matrix isn’t simply placing an off-the-shelf memory chip underneath its processor.
The DRAM die is designed specifically for this architecture.
The company hasn’t publicly identified the manufacturer of that memory, at least not in the technical information shared at Hot Chips.
It has confirmed that the compute die uses a 4nm TSMC process.
The integration raises issues a conventional HBM-based system doesn’t have to solve the same way.
One is temperature.
Placing active logic directly on top of DRAM means the memory can run at higher-than-usual temperatures. That affects how long cells retain data and requires adjusting refresh policies.
The academic paper describes temperature-aware refresh mechanisms, along with redundancy and interleaved error-correction codes (ECC) to maintain reliability.
Another challenge is the sheer number of memory banks that can operate simultaneously.
Raptor uses a technique called stream-blocking to spread data streams, including generative models’ KV cache, across configurable 3D-DRAM channels and keep parallelism intact.
It also incorporates a variant of Data Bus Inversion adapted to the vertical interface to reduce the number of electrical transitions and, with them, power consumption.
These are less flashy details than the 100 TB/s headline, but they determine whether the technology can actually work outside the lab.
KV cache fits this design especially well
Memory has become an even bigger problem as model context windows keep growing.
During inference, a model keeps a KV cache holding information generated while processing previous tokens.
The larger the context and the more users served at once, the more memory that structure takes up.
But it also needs to be accessed quickly.
Raptor is designed with exactly these flows in mind. The ISCA paper describes how the KV cache can be distributed across memory channels without losing parallelism between banks.
The architecture aims to achieve two goals that often conflict: having a reasonable amount of DRAM while bringing it close enough to compute that it can operate with bandwidth closer to that of SRAM.
That combination explains the phrase d-Matrix uses to describe its proposal: DRAM-like capacity with SRAM-like locality and bandwidth.
Raptor still needs a network once a model outgrows one card
The 32 GB per card inevitably raises another question.
Models like Llama 3.1 70B, DeepSeek-V3, or Kimi K2 can need far more memory than that, even after applying various quantization techniques.
Raptor has to split them across multiple accelerators.
That means the full system’s performance also depends on how the cards communicate with each other.
The study argues that the 3D-DRAM architecture is less sensitive to network latency and bandwidth than the alternatives it analyzed. Part of the explanation lies in the huge internal bandwidth: local operations can complete very quickly, and the design tries to reduce unnecessary external data movement.
But the network doesn’t disappear.
In deployments with large models, accelerators still need to be coordinated, activations transported, and work divided.
d-Matrix has spent a long time building its platform around chiplets and high-speed interconnects. Its current Corsair architecture already uses a modular design specifically to scale beyond a single chip.
RISC-V will be inside Raptor too
Raptor will also include RISC-V-based cores.
d-Matrix announced in November 2025 that it had selected Andes Technology’s AndesCore AX46MPV as the orchestration and vector-compute CPU for its next architecture.
Those cores will handle load distribution, memory coordination, and runtime control, in addition to offloading certain operations such as activation functions.
The use of RISC-V is secondary to the memory innovation, but it shows just how far Raptor departs from a conventional GPU.
The system combines its own inference logic, 3D DRAM, RISC-V cores, and a chiplet architecture built from the ground up around data movement.
That’s also why it wouldn’t be accurate to think of it as simply a “faster GPU.”
The inference battle is shifting toward memory
Raptor arrives as the industry experiments with several answers to the same problem.
HBM keeps evolving toward HBM4 and later generations.
High Bandwidth Flash proposes adding a huge-capacity layer between HBM and SSDs for certain workloads.
CPU makers are adding matrix extensions.
And companies like d-Matrix are trying to physically bring compute and memory together to shrink the cost of moving information as much as possible.
These strategies can coexist because they solve different problems.
For training giant models, GPUs and accelerators with large amounts of HBM still offer a combination of capacity, bandwidth, interconnect, and software that’s hard to replace.
For interactive inference, where generating each token quickly for millions of users can matter more than having huge theoretical FP4 throughput, the balance shifts.
That’s where d-Matrix wants to position Raptor.
The company still has to prove in commercial systems that its early-silicon results and performance models hold up as the architecture scales, runs real-world applications, and competes with upcoming GPU generations.
But the work presented at ISCA and Hot Chips carries an important signal: improving AI performance no longer depends solely on building faster math units.
The cost of transporting the data those units need matters more and more.
Raptor takes that idea to an extreme physical solution: placing memory directly beneath compute and connecting the two through thousands of microscopic links.
If that architecture can carry its 100 TB/s from technical demos to commercially competitive servers, d-Matrix will have turned one of generative inference’s biggest problems, the distance between memory and compute, into its accelerator’s defining feature.
Frequently asked questions
What is d-Matrix Raptor?
Raptor is d-Matrix’s next-generation accelerator for AI inference. It uses a 3DIMC architecture that stacks compute logic directly on DRAM to cut the latency and energy spent moving data.
How much bandwidth does Raptor offer?
The architecture d-Matrix presented reaches up to 100 TB/s of bandwidth per card, with 32 GB of 3D-DRAM memory.
Is Raptor faster than an HBM GPU?
The study presented at ISCA calculates 4.71x more throughput compared with an equivalent HBM-based variant for the workloads evaluated. That figure shouldn’t be read as a universal improvement over any commercial GPU, since it depends on the architecture and scenarios analyzed.
What manufacturing process does Raptor use?
d-Matrix has stated that the compute die uses a 4nm TSMC process. The company hasn’t publicly identified the manufacturer of the custom DRAM in the information available so far.
