SanDisk is attempting to create a new memory category for artificial intelligence with High Bandwidth Flash (HBF), a technology aimed at bringing large amounts of NAND closer to accelerators without bearing the costs and capacity limitations of HBM memory. Its proposal gained visibility after the Investor Day held on August 13, although some comparisons made by the company have sparked technical criticism: HBF can offer substantial capacity, but its characteristics are very different from those of DRAM used in HBM.
Key Points of SanDisk’s HBF in 20 Seconds
- HBF uses stacked NAND memory to provide hundreds of gigabytes near AI accelerators.
- SanDisk and SK hynix are working on standardizing it within the Open Compute Project.
- The main goal is to store model weights during inference; not to replace all of HBM.
- NAND offers high capacity but has higher latency and write limitations.
- Early HBF samples are already part of SanDisk’s roadmap.
The idea addresses a problem increasingly influencing AI server design: more FLOPS are not always needed. For many inference workloads, the key shortage is sufficient memory to keep ever-larger models close to the accelerator.
High Bandwidth Memory (HBM) effectively solves bandwidth issues but is costly, with capacity limited by the number and size of stacks that can be integrated around the processor.
HBF aims to occupy a different space.
From 192 GB of HBM to several terabytes of flash beside the accelerator
SanDisk’s proposal, in simplified terms, involves applying some of the ideas used to build HBM to NAND memory.
HBF stacks numerous flash dies and leverages high parallelism to drastically increase bandwidth compared to a conventional SSD. SanDisk also uses technologies like CBA (CMOS directly Bonded to Array) to bring control logic closer to the NAND array.
The fundamental difference remains in the memory cells.
HBM uses DRAM. HBF uses NAND.
DRAM has much less capacity per unit area and requires power to maintain data, but provides much lower latency. NAND is slower, but much denser, cheaper, and also non-volatile.
This enables architectures with hundreds of gigabytes or even several terabytes available around an accelerator.
SanDisk already showed a particularly striking comparison during its 2025 Investor Day: a configuration with eight HBM stacks summed up to 192 GB, whereas eight HBF devices reached 4,096 GB, i.e., 4 TB.
The company later detailed that a 16-die HBF stack can provide 512 GB. Internal simulations also suggest that HBF can approach the performance of an HBM system with unlimited capacity when used for reading pre-trained weights of Llama 3.1 405B.
An important clarification: these are simulations conducted by SanDisk for a specific use case, not results from commercial products.
The potential advantage is straightforward to understand.
A model requiring several hundred gigabytes of weights could be kept much closer to the processor without filling the system with expensive HBM stacks or adding accelerators solely for more memory.
HBF makes sense for weights; KV cache presents a different challenge
This introduces a nuance that some comparisons between HBF and HBM may overlook.
During language model inference, different types of information are stored in memory, each with distinct access patterns.
On one hand are the model weights, the learned parameters from training. During inference, these are primarily read.
This scenario favors NAND.
On the other hand is the KV cache (key-value cache) generated during inference. It retains information about previously processed tokens to avoid recalculating it continually, and it grows with the context and requests.
Its behavior involves writes.
Here, HBF has an inherent disadvantage compared to HBM: NAND has a finite write endurance and significantly higher write latency than DRAM.
The high parallelism of HBF can greatly increase total read bandwidth but does not change the physical characteristics of each NAND cell.
Therefore, it makes more sense to see HBF as a new level within the memory hierarchy, rather than a universal replacement for HBM.
SK hynix itself has described HBF as a new layer between HBM and SSD, while SanDisk specifically targets inference needs.
A future architecture might, for example, keep extremely fast and frequently accessed data in HBM, while using HBF to house enormous amounts of weights.
This separation could be especially beneficial for Mixture of Experts (MoE) models, where only a subset of parameters is active for each token.
Bandwidth needs context
Another topic of discussion has been SanDisk’s comparisons between HBF and HBM.
The company has used reference scenarios with 192 GB of HBM, while HBM evolution continues to increase capacity and bandwidth per accelerator.
Comparing a future technology with a specific HBM configuration can give an incomplete picture if the generation used isn’t clearly specified.
The same applies to the numerical precision of models.
Using BF16 (Brain Floating Point 16) makes it simple to illustrate the memory size of parameters, but modern inference systems increasingly use lower-precision formats like FP8, INT8, or FP4, depending on the model and hardware.
Reducing precision also reduces the memory needed to store weights.
A 400-billion-parameter model would need about 800 GB with each parameter occupying two bytes in BF16. Using four bits per parameter, the theoretical size drops to approximately 200 GB, before considering scaling, metadata, and other overheads.
While this doesn’t eliminate capacity issues HBF aims to solve, it significantly reduces the scale of the problem.
It’s also important to distinguish between bandwidth and latency.
SanDisk and SK hynix are working on HBF specifications supporting configurations up to 512 GB per device and bandwidths reaching several TB/sec. The recent proposal estimates bandwidths around 0.4 to 3 TB/sec, depending on configuration.
That’s an enormous figure for flash memory.
But achieving several TB/sec by adding thousands of NAND operations in parallel does not turn NAND cells into DRAM. Latency-sensitive applications can behave very differently even if the total bandwidth appears comparable.
SanDisk and SK hynix aim to standardize HBF
HBF is not just an internal SanDisk project.
In August 2025, SanDisk and SK hynix signed an agreement to collaborate on developing their specifications. In February 2026, they advanced further by creating a working group within the Open Compute Project (OCP) to develop standardization.
This is significant because a new memory technology requires more than just chips.
Processors, accelerators, controllers, encapsulations, firmware, operating systems, and AI frameworks all need to understand data locations and decide when to move data.
HBF could effectively introduce another level in the AI server memory hierarchy:
GPU → HBM → HBF → SSD → Distributed Storage.
Software would then need to determine which data should reside in the fastest, most expensive memory and which can stay in higher-capacity levels.
SanDisk initially announced HBF samples for late 2026, with inference devices integrating the technology by early 2027. The company also emphasizes plans for capacities 8 to 16 times higher than HBM at comparable costs, though these projections are from the manufacturer and will need validation with commercial hardware.
The true appeal of HBF lies in transforming inference economics
The most interesting aspect of HBF likely isn’t whether it can beat HBM on technical specs.
They are memories designed for different problems.
HBM will remain especially valuable where latency, frequent writes, and extreme bandwidth are crucial. NAND can provide much larger capacity at a lower cost, with different access characteristics.
HBF seeks to exploit this trade-off.
If future accelerators can keep terabytes of weights close enough to the GPU, it could be possible to run certain large models with fewer dedicated memory accelerators.
This change would have significant economic implications for inference.
However, many uncertainties remain, including real-world latency under production loads, power consumption, cooling, sustained performance, write endurance, error management, and especially how real systems will orchestrate data between HBM and HBF.
Investor presentations tend to condense these issues into capacity and bandwidth figures. The memory architecture of an AI server is far more complex.
HBF doesn’t need to be faster than HBM to succeed. It needs to ensure that storing enormous models near the accelerator is sufficiently fast and much cheaper.
Frequently Asked Questions
What is HBF?
HBF stands for High Bandwidth Flash. It’s a NAND-based technology that uses stacking and high parallelism to provide much more capacity and bandwidth than conventional flash storage.
Will HBF replace HBM in AI GPUs?
Not necessarily. HBF is especially interesting for storing large quantities of weights during inference, while HBM maintains advantages in latency and frequent write operations.
What capacity can HBF offer?
SanDisk proposes devices with 512 GB using 16-die stacks, enabling several terabytes when multiple devices are combined around an accelerator.
When will HBF be available?
SanDisk plans to have initial samples by late 2026 and inference devices integrating HBF by early 2027. Commercial adoption will depend on standard development, hardware manufacturers, and software ecosystems.
