AMD has acquired Taalas, a small Canadian company experimenting with a radically different idea from current GPUs: integrating AI model weights directly into silicon. Its HC1 demonstrator achieved approximately 16,960 tokens per second running Llama 3.1 8B, a figure difficult to attain with conventional accelerators and which helps explain AMD’s interest in the company.
The key points of AMD’s purchase of Taalas in 20 seconds
- AMD announced the acquisition of Taalas on August 6, without revealing the deal’s financial details.
- Their HC1 demonstrator runs Llama 3.1 8B at about 16,960 tokens per second, according to Taalas.
- The company incorporates the model weights directly into the silicon to reduce data movement.
- The trade-off is reduced flexibility: significantly changing the model requires manufacturing new hardware.
- AMD plans to combine this specialized technology with its Instinct accelerators.
AMD hasn’t just bought another company that designs accelerators. Taalas offers a different approach to one of the major economic challenges in language model inference: continuously moving huge amounts of data between memory and compute units.
The operation also hints at a possible evolution of AI-focused data centers. Instead of using the same GPU for all inference phases, stable models might eventually run on hardware specifically built for them.
16,960 tokens per second with a 6-nanometer chip
Taalas introduced HC1 in February 2026 as a demonstration of its architecture. The chip is manufactured using a 6-nanometer process from TSMC, far from the most advanced nodes used by some of the latest AI accelerators.
Its most striking feature is something else.
HC1 was built specifically to run Llama 3.1 8B, the model with roughly 8 billion parameters presented by Meta in 2024.
Taalas reported a maximum performance close to 17,000 tokens per second per user. Follow-up tests on the public demo recorded speeds around 15,000 tokens per second, providing further evidence that the system operates within that order of magnitude.
The difference compared to other accelerators is enormous.
When Cerebras launched its inference service for Llama 3.1 8B, it reported speeds exceeding 1,800 tokens per second. Later documentation placed typical performance around 2,200 tokens/sec.
Taalas presented HC1 as approximately 8.5 times faster than Cerebras and 48 times faster than certain NVIDIA configurations used in their comparison.
It’s important to keep these figures as manufacturer comparisons. Tokens per second alone do not constitute a universal benchmark: accuracy, context length, batching, latency, model, configuration, and number of users all influence performance.
The truly interesting aspect is what lies behind the numbers.
The model stops traveling from memory
In a conventional accelerator, there is a physical separation between computation and memory.
Modern GPUs try to mitigate this limitation by using enormous amounts of HBM (High Bandwidth Memory). NVIDIA, AMD, and other manufacturers have increased capacity and bandwidth generation after generation.
But during certain inference phases, especially during sequential token generation, moving weights can become one of the main bottlenecks.
Taalas significantly alleviates this problem: the weights are physically embedded in the chip.
Its architecture uses a region called mask-ROM recall fabric. Information that needs to remain modifiable uses SRAM, including elements like KV caches and LoRA adaptations.
In simplified terms, the processor no longer constantly queries external memory for the weights it needs because much of that information is part of its physical structure.
The benefit is huge in both performance and power consumption.
The cost is reduced flexibility.
A GPU can run Llama today, Qwen tomorrow, and a completely different model the day after just by loading different weights.
The Taalas chip does not.
Major changes to the model require another silicon fabrication. Although Taalas has explained that their methodology enables some modifications through metallic layer adjustments, there remains a physical manufacturing process.
This introduces a variable normally associated with software into the semiconductor economic cycle: the model’s lifespan.
AMD aims to combine Taalas with Instinct, not replace its GPUs
This point helps clarify the purchase.
AMD has not announced that Taalas will replace Instinct. Quite the opposite: they aim to incorporate its technology into their accelerator strategy and develop system-level solutions together with AMD Instinct.
This opens a particularly interesting possibility: disaggregating different inference stages.
Initial prompt processing and subsequent token generation have different computational characteristics. It may not always be economically optimal to run both on the exact same hardware.
Future infrastructure could use programmable accelerators like Instinct for flexible phases and specialized chips for repeatedly executing the most stable parts of the process.
The infrastructure decision would then shift from simply:
“How many GPUs does this model need?”
To also include another question:
“Which parts of the workload should run on each type of accelerator?”
This evolution is also seen in other specialized inference architectures. Cerebras and Groq have demonstrated that there’s room outside conventional GPUs when low latency and high generation speeds are priority.
Economic risk: how long will the model be viable?
However, there’s a less technological and potentially more important issue.
A server with a GPU retains some economic flexibility. If a model becomes obsolete, the accelerator can be repurposed.
An ASIC built around a specific model has a different situation.
If the model disappears from production before amortizing the hardware its reusability drops dramatically.
This issue can even affect infrastructure financing. A general-purpose GPU has a secondary market and can be used for various workloads. Valuing a chip whose utility depends on a specific model architecture in a few years is much more complicated.
That’s likely why Taalas makes more sense initially where models change little and inference volume is huge.
Fraud detection, classification, industrial vision, certain recommendation systems, or highly stabilized enterprise models may fit that profile better than a generative service that updates models every few weeks.
The purchase of Taalas raises a question that goes far beyond 16,960 tokens per second.
In recent years, industry has aimed to develop increasingly programmable accelerators capable of running virtually any model. Taalas is exploring the opposite end: sacrificing some flexibility to eliminate costs that arise specifically from maintaining it.
AMD seems to believe both approaches can coexist.
And if that strategy succeeds, designing an AI inference platform may no longer mean choosing a single GPU family. The workload would be distributed among different accelerators depending on how much computation, memory, flexibility, and speed each phase requires.
Frequently Asked Questions
What speed does the Taalas HC1 chip achieve?
Taalas has announced approximately 16,960 tokens per second running Llama 3.1 8B. This is from the manufacturer, though tests on the public demo have produced results in the same range.
Why can Taalas generate tokens so quickly?
Its architecture embeds the model weights directly into the silicon, reducing data movement between external memory and compute units, which is a bottleneck in certain inference processes.
Can HC1 run any AI model?
No. HC1 is built specifically around Llama 3.1 8B. The architecture supports certain changes and adapters like LoRA, but major modifications to the model require new hardware.
Will AMD replace its Instinct GPUs with Taalas chips?
That’s not the plan. AMD states it intends to integrate Taalas technology into system solutions together with its Instinct GPUs, pointing toward more specialized inference architectures.
