High Bandwidth Flash (HBF) wants to fill a space that has so far been largely empty in AI accelerators: a memory with far more capacity than HBM and considerably more bandwidth than conventional NAND storage. However, an analysis presented by OXMIQ Labs at Hot Chips 2026 shows that combination has real limits. HBF can be very attractive when the problem is fitting a huge model close to the processor, but it loses much of its edge when the workload needs to move that data continuously at high speed.
High Bandwidth Flash in 20 seconds
- HBF uses stacked NAND to bring hundreds of gigabytes of capacity closer to AI accelerators.
- The specification covers up to 512 GB per stack and roughly 3 TB/s at its most advanced tier.
- It can offer 8 to 16 times the capacity of HBM at comparable cost, according to its backers.
- OXMIQ warns that HBM remains better when bandwidth determines performance.
- Mixture of Experts models and very long contexts stand out among the most promising use cases.
The technology is still in its early days. SanDisk and SK hynix published the first technical HBF specification through the Open Compute Project (OCP) in early August, barely six months after formally kicking off the standardization work. Google and Tenstorrent have also joined the group and taken part in validating the technology.
HBF’s premise is simple to grasp: AI needs ever more memory, and HBM is extremely fast but also expensive and limited in capacity. NAND offers vastly more capacity at lower cost, though it has traditionally been too slow to sit directly next to a GPU.
HBF is trying to build something in between.
Between HBM and an SSD, but with a different architecture
High Bandwidth Flash uses 3D NAND memory organized into stacks and connected through high-bandwidth interfaces.
The first specification covers three performance tiers.
Grade 1 starts from a 256 GB stack and delivers roughly 384 GB/s. Grade 2 raises capacity to 512 GB and bandwidth to roughly 1.536 TB/s.
Grade 3 keeps 512 GB but reaches roughly 3.072 TB/s, using UCIe 2.0 at 32 GT/s.
| Technology | Typical capacity | Bandwidth | Strong point |
|---|---|---|---|
| HBM | Tens of GB per stack | Very high | Continuously feeding GPU/AI |
| HBF Grade 1 | 256 GB | 384 GB/s | Capacity |
| HBF Grade 2 | 512 GB | 1.536 TB/s | Capacity/bandwidth balance |
| HBF Grade 3 | 512 GB | 3.072 TB/s | Getting closer to HBM performance |
| NVMe SSD | Several TB | Much lower | Capacity and persistence |
The numbers make it clear why HBF is generating interest.
An accelerator could pack several terabytes of local memory without having to fit an equivalent amount of HBM.
SanDisk argues HBF can deliver 8 to 16 times more capacity than HBM at a comparable cost. That’s a company estimate and will depend on how commercial products evolve, but it captures the pitch well: HBF is competing mainly on capacity, not trying to become a cheap HBM.
And that distinction is essential.
14 times more memory doesn’t mean processing 14 times faster
OXMIQ Labs used Hot Chips to analyze what happens when HBF meets a realistic AI workload.
One of its models studies a rack of 72 accelerators running Kimi K2, a Mixture of Experts (MoE) model with roughly a trillion parameters.
Keeping a similar cost and power budget, the HBM-only configuration delivers around 20.7 TB of capacity and 1,584 TB/s of aggregate bandwidth.
Swapping HBM for HBF pushes capacity up to roughly 294.9 TB, about 14 times more.
The catch is that aggregate bandwidth drops to around 922 TB/s.
The gap perfectly illustrates the trade-off.
If the main problem is getting a model to fit in memory, HBF can provide a huge advantage.
If the model already fits and the goal is producing as many tokens per second as possible, HBM can end up being more efficient.
In OXMIQ’s example, HBF’s huge capacity would let each accelerator host a full instance of the model, running up to 72 instances per rack.
The HBM configuration would need eight accelerators just to host each full instance, cutting the number of independent instances that fit in the same rack.
But as the number of simultaneous users grows and each GPU needs constant access to large amounts of information, bandwidth starts to dominate performance.
In that scenario, HBM regains the advantage.
That’s why cost per gigabyte doesn’t necessarily determine cost per token.
MoE models are one of the most interesting use cases
High Bandwidth Flash can find an especially good fit in Mixture of Experts models.
These models contain multiple specialized parameter groups, called experts, but don’t use all of them to generate every token.
The system selects only certain experts depending on the information it’s processing.
That creates a very interesting property from a memory standpoint: the model can hold an enormous number of parameters that need to be available, even though a large share of them stays inactive at any given moment.
OXMIQ used as an example a model with roughly 1.56 TB of weights, of which about 1.45 TB, or 93%, would correspond to MoE experts.
Storing that entire set in HBM would be costly.
HBF would let the accelerator keep a much larger share of those parameters nearby, reserving HBM for data that needs frequent access.
The architecture could look conceptually like a hierarchy:
HBM for hot data, HBF for large, less-used sets, and SSD or remote storage for even colder information.
Unlike a conventional SSD, HBF would sit much closer to the processor and offer far higher bandwidth.
Less GPU-to-GPU traffic can offset slower memory
There’s another potential advantage.
Large MoE models typically spread their experts across multiple accelerators.
When a token needs an expert hosted on another GPU, systems have to exchange information over the interconnect network.
At scale, this all-to-all communication can consume a considerable share of available bandwidth and system energy.
With several terabytes of HBF near each accelerator, it would be possible to keep a much higher proportion of experts local.
That could reduce the need to spread them across so many GPUs and cut down on inter-accelerator traffic.
In that case, HBF would be trading one thing for another: less memory bandwidth in exchange for more local capacity and less dependence on the network.
But it won’t always work that way, either.
As batch size grows and many different requests arrive at once, the system can end up hitting a much wider variety of experts.
The part of the model considered “cold” then stops being cold.
If data has to keep shuttling from HBF to HBM, NAND’s lower speed starts to hurt performance.
Giant contexts offer another opportunity
The second especially interesting scenario is inference with very long contexts.
Language models keep a structure known as the KV cache to hold information from already-processed tokens and avoid recomputing certain operations.
When context reaches hundreds of thousands or even millions of tokens, that cache can consume enormous amounts of memory.
But some sparse-attention architectures don’t need to look at all of that content at every generation step.
A large KV cache could then live in HBF, with only the blocks needed at any given moment moving to HBM.
The idea is the same as before.
HBF works best when an application needs a huge amount of information nearby, but only touches a small fraction of it at any moment.
If it needs to read virtually everything constantly, HBM is a much better fit.
NAND also brings limits HBM doesn’t have
There’s also a physical difference that can’t be fixed just by adding more bandwidth.
HBF still runs on NAND Flash memory.
That means its operations don’t behave the same way DRAM’s do.
The specification calls for block reads and larger writes, and getting maximum performance requires relatively large transfers. OXMIQ also notes that data movement would happen via DMA rather than being integrated directly into the conventional CPU or GPU cache hierarchy.
There’s also the question of endurance.
NAND supports a limited number of write cycles. Systems would need to manage how cells are used to avoid premature wear.
That makes HBF especially well-suited to data that’s written rarely and then read repeatedly, like a model’s weights.
For workloads with constant writes, it can be much less appealing.
The biggest challenge may end up being software
Building the chips is only part of the challenge.
Today’s inference frameworks are built fundamentally around DRAM and HBM.
A hybrid system would have to constantly decide what information stays in HBM, what data can move to HBF, and when it needs to come back.
It would also need to plan ahead.
Waiting until a GPU needs a piece of data to start copying it from slower memory can stall processing. The software would need to do prefetching and move information before it’s needed.
OXMIQ specifically notes that platforms like vLLM would need dedicated support for HBF, including memory allocators, placement policies, prefetch mechanisms, and NAND wear monitoring.
Accelerator makers would also need to get involved.
AMD, NVIDIA, and other designers would need to provide the hardware mechanisms, drivers, and runtimes capable of efficiently moving information between HBM and HBF.
The technology can be defined on paper, but building a software ecosystem able to take advantage of it will likely be considerably harder.
HBF isn’t trying to kill HBM
HBF’s first open specification also helps correct an interpretation that has followed the technology since its earliest presentations.
SanDisk began talking publicly about High Bandwidth Flash in 2025 as a response to AI systems’ growing memory-capacity problem.
Since then the proposal has evolved.
SanDisk and SK hynix formalized a working group within the Open Compute Project in February 2026 and published the first technical specification in August. Google and Tenstorrent are already taking part in the process.
The architecture now taking shape makes it increasingly unlikely that HBF will fully replace HBM.
It makes more sense to think of it as a new layer within accelerators’ memory hierarchy.
HBM would keep handling the data that needs maximum bandwidth.
HBF would store hundreds of gigabytes or several terabytes that need to stay close but don’t require constant access.
And SSDs would keep providing even larger capacities at lower cost and higher latency.
The ultimate usefulness will depend on the models.
A dense model run with large batches may get little benefit from HBF. A massive Mixture of Experts model with many inactive parameters could be a much better candidate.
That’s why the conclusion OXMIQ presented at Hot Chips fits especially well: HBF is a specialized tool, not a universal fix for AI’s memory problem.
If the industry manages to bring hardware, drivers, and inference frameworks together, High Bandwidth Flash could carve out an important space between HBM and SSDs.
But its success likely won’t hinge on proving it can replace HBM.
It will hinge on finding the workloads where having terabytes next to the accelerator matters more than moving every byte at the highest possible speed.
Frequently asked questions
What is High Bandwidth Flash, or HBF?
HBF is a NAND Flash-based technology designed to provide far more capacity and bandwidth than conventional flash storage while sitting close to AI accelerators.
Can HBF replace HBM?
Not for every workload. HBM remains better suited when performance depends on constantly accessing large amounts of data, while HBF is more attractive when capacity is the main problem.
How much capacity can HBF offer?
The first specification covers stacks of up to 512 GB. Combining several stacks would let accelerators pack several terabytes of memory close to the processor.
When will we see HBF in commercial systems?
The open specification was published in August 2026 and the technology is still under development. SanDisk had previously targeted the first samples for 2026, but commercial adoption will also depend on support from accelerator makers and the software ecosystem.
