OpenAI Faces Jalapeño Against NVIDIA Blackwell and Wins in Inference per Watt

OpenAI has published the initial results of Jalapeño, its first custom ASIC accelerator designed for AI inference. The figures show a significant advantage over NVIDIA’s Blackwell GB200 and GB300 systems across all three tested workloads. In GPT-OSS 120B, it reaches 85,448 mixed tokens per second per kilowatt, 1.9 times the performance of the comparison system, and achieves 1,459 tokens per second per user at its lowest latency point. These measurements are based on the SemiAnalysis InferenceX benchmark, though they do not yet confirm Jalapeño’s speed superiority over Blackwell for all AI workloads.

The key points about OpenAI Jalapeño in 20 seconds

• Jalapeño delivers between 1.5x and 1.9x more performance per watt across the three published models.
• It reaches 1,459 tokens/sec per user with GPT-OSS 120B.
• The comparisons involve NVIDIA GB200 and GB300 depending on the model.
• OpenAI has also tested DeepSeek R1 670B and Kimi K2.5 1T.
• Its internal deployment is expected to begin before the end of 2026.

The significance of this announcement lies both in the results themselves and in who is producing them. For years, OpenAI has been one of the major buyers of NVIDIA accelerators and intends to continue deploying them. Jalapeño introduces a second approach: designing silicon specifically tailored to inference needs, integrating accelerator hardware, memory, interconnects, software, and system architecture.

The published results also come from models with very different characteristics: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 with one trillion parameters. SemiAnalysis notes that Jalapeño outperformed evaluated NVIDIA, AMD, and Google accelerators in its tests against several open large models. While this broadens the scope of comparison, it should be distinguished from the specific NVIDIA results that OpenAI has presented in their charts.

Up to 1,459 tokens per second per user

One of the most striking figures is the maximum decoding speed per user.

Using GPT-OSS 120B, Jalapeño achieves 1,459 tokens/sec per user, compared to 535 tokens/sec for the reference GB200 system. That’s roughly 2.7 times faster.

The gap widens with DeepSeek R1: 700 versus 169 tokens/sec, about 4.1 times. With Kimi K2.5, it manages 694 tokens/sec compared to 182 from GB300, roughly 3.8 times.

This metric is particularly relevant for virtual assistants and AI agents, where the time to generate each response directly impacts the overall duration of multi-step tasks.

OpenAI emphasizes that one of Jalapeño’s goals was to avoid the typical trade-off between throughput and latency. A system might handle many requests simultaneously via batching, but that does not necessarily mean providing the fastest response to each user.

Jalapeño attempts to maintain both efficiency and low latency within a single architecture.

This is also reflected in the end-to-end latency measurements. OpenAI reports 1.03 seconds versus 1.80 seconds with GPT-OSS, 1.65 versus 5.99 seconds with DeepSeek R1, and 1.56 versus 5.31 seconds with Kimi K2.5.

The crucial battle: tokens per kilowatt

Absolute speed alone is only part of the story for data centers. Available electricity has become a key constraint in deploying large-scale AI infrastructure.

As a result, OpenAI has used performance per unit of power as one of its primary metrics.

The results are as follows:

The company normalized the comparison using each accelerator’s published nominal power: 700 W for Jalapeño, 1,200 W for GB200, and 1,400 W for GB300. OpenAI adds that its chip maintained steady performance in tests at a power consumption of 550 W or less.

This is an important nuance. The 85,448 tokens/sec/kW figure does not mean a single Jalapeño produces that speed continuously. The metric normalizes system performance relative to power to enable efficiency comparisons.

It’s also worth noting that these figures do not prove Jalapeño is 90% more powerful than Blackwell in general. Instead, they show that in GPT-OSS 120B and under the specific InferenceX methodology, Jalapeño achieves approximately 1.9 times the mixed throughput per kilowatt of the GB200 used in the comparison.

InferenceX offers results across multiple configurations and illustrates how performance, cost, and efficiency can vary depending on model size, precision, sequence length, and interactivity level.

Deciphering the 104x figure

Another astonishing result is 104.3 times.

However, it would be incorrect to interpret this as Jalapeño being 104 times faster than a NVIDIA GB300.

OpenAI conducted a different test. They fixed the maximum user speed achieved by NVIDIA’s platform for both systems and then calculated how much throughput per kilowatt each architecture could sustain at that point.

Using GPT-OSS at approximately 535 tokens/sec per user, Jalapeño achieved 22,935 mixed tokens/sec/kW compared to 427 from GB200: 53.7x more.

With DeepSeek R1, around 169 tokens/sec per user, the comparison is 12,258 versus 118, a 104.3x difference.

And for Kimi K2.5, at about 182 tokens/sec per user, the ratio is 6,744 versus 120, or 56.1 times.

This test highlights the cost of pushing an architecture toward very low latency. The Blackwell system’s total capacity significantly drops when required to sustain that speed per user simultaneously.

That’s where OpenAI aims to differentiate their ASIC.

Language model inference isn’t just about performing the same operations repeatedly. During prefill, the accelerator processes the prompt and involves high computation demand. During decode, tokens are generated sequentially, and memory bandwidth becomes much more critical.

Jalapeño is designed to reduce data movements between these phases. OpenAI explains it can explicitly place the model state, including the KV cache, and keep it local while assigning computational, memory, and network resources as needed for each stage.

This is a key difference from architectures that separate prefill and decode across different resources. It does not mean that such separation is inherently worse: NVIDIA and modern inference engines are already achieving highly competitive results using those techniques. InferenceX showcases configurations with GB200 and GB300 employing disaggregated inference to balance performance and interactivity.

OpenAI’s push for greater control over its infrastructure

Jalapeño does not represent a break from NVIDIA’s hardware.

OpenAI explicitly states it will continue deploying extensive NVIDIA and partner accelerators for both training and inference. Their ASIC should currently be viewed as another piece of an infrastructure that demands increasing capacity.

The difference is that OpenAI is beginning to control a part of the silicon upon which its services run.

SemiAnalysis reports that development was carried out in collaboration with Broadcom, starting in mid-2024. For a company that consumes data center-scale accelerators, improving tokens per megawatt can have significant economic implications—even without fully replacing GPUs.

Another notable aspect is that AI was used to develop and program the accelerator itself.

OpenAI states they moved from initial design to tapeout within nine months, leveraging their models to explore implementations, shorten design and verification cycles, and optimize arithmetic circuits. They later used Codex with GPT-Astra to adapt three open-weight models to Jalapeño in about two months.

In certain attention blocks and Mixture of Experts layers of GPT-OSS, AI-generated implementations have been shown to run between 1.5 and 1.8 times faster than those previously crafted by humans. OpenAI cautions that this comparison applies strictly to those blocks, not the entire model’s performance.

Looking ahead, the next test will be much broader than just benchmarking. OpenAI plans to begin integrating Jalapeño into their infrastructure before the end of 2026, as they continue qualifying it for production and developing their software stack. They are already working on a second-generation chip and have begun defining a third.

Until the hardware operates at scale in production, a crucial missing piece remains: understanding how these figures translate into availability, total cost of ownership, and sustained performance under real workload conditions.

Initial results already show something less debatable: OpenAI’s first-generation ASIC can compete with Blackwell in inference and, in published tests, surpass NVIDIA systems in latency and performance per watt. For a company whose token consumption continues to grow, that efficiency may become more important than just winning a FLOPS race.

Frequently Asked Questions

Is Jalapeño faster than NVIDIA Blackwell?

In the published tests with GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T, Jalapeño outperforms the reference systems GB200 or GB300 in both performance per watt and maximum decoding speed. However, these results do not automatically imply that this advantage applies to all models or workloads.

How many tokens per second does OpenAI Jalapeño achieve?

Its maximum published per-user throughput is 1,459 tokens/sec with GPT-OSS 120B. It also achieves 700 tokens/sec with DeepSeek R1 and 694 tokens/sec with Kimi K2.5.

Will Jalapeño replace NVIDIA GPUs at OpenAI?

That’s not the current plan. OpenAI states it will continue deploying NVIDIA and other vendor accelerators for both training and inference as it gradually incorporates its own silicon.

When will OpenAI start using Jalapeño?

The company expects to begin deploying it within their infrastructure before the end of 2026. Jalapeño is the first in a family, with a second generation already in development and a third planned.