In recent years, nearly all discussions about artificial intelligence have revolved around GPUs. It makes sense: training, inference, multimodal models, Mixture of Experts, and large contexts depend on increasingly powerful accelerators. But NVIDIA is pushing an idea worth paying close attention to: in the era of AI agents, the CPU is once again on the critical path.
NVIDIA’s argument with NVIDIA Vera is straightforward. An agent doesn’t just generate text. It reasons, calls tools, executes code, queries databases, analyzes results, runs tests, moves data, verifies outputs, and decides on the next step. At each of these phases, the GPU may be waiting for the CPU to finish the surrounding tasks. And in an AI data center, a GPU waiting equals money paused.
NVIDIA defines Vera as a new category of CPU for “maximum single-threaded performance at scale,” designed for AI agents and factories. It’s not just about having many cores. The thesis is that each core must be fast even when the entire chip is loaded, with sufficient memory bandwidth per core and predictable latencies.
The agent loop changes how to measure a CPU
Traditional workloads of many data center CPUs have been characterized by parallel loads, web services, virtualization, databases, microservices, and per-core efficiency. Cloud providers pushed for more cores per socket and better cost per capacity unit for years. That approach made sense for many uses but doesn’t always maximize the speed of each individual step.
Agent-based AI introduces another pattern. An agent works in a loop: the model decides, the CPU executes an action, the result returns to the model, and the cycle repeats. If a step takes longer, if a sandbox startup is slow, if an SQL query is delayed, or if a tool processes data with high latency, the entire agent slows down.
More cores help run more agents in parallel but do not speed up the sequential step within a specific agent on their own. That’s why NVIDIA emphasizes sustained single-threaded performance: it’s not enough to have high overall throughput; each individual action that enables the next model call must be completed quickly.
| Agent phase | Typical CPU role |
|---|---|
| Tool calling | Executing external or internal tools |
| Code | Launching scripts, tests, compilation, or sandboxes |
| Data | Filtering, transforming, querying, and moving information |
| Verification | Comparing results, validating outputs, reviewing errors |
| KV-cache and service | Managing memory, caches, and inference support |
| Orchestration | Coordinating steps, processes, and auxiliary calls |
This idea has a clear implication for infrastructure: an agent’s performance depends not only on the model or GPU but on the entire system supporting the loop.
What NVIDIA proposes with Vera
Vera uses 88 Olympus cores, designed by NVIDIA, which the company claims deliver 50% more instructions per cycle than Grace. The chip combines these cores with up to 1.2 TB/s of LPDDR5X memory bandwidth, less than 40 watts of memory power, and a monolithic compute die with 3.4 TB/s bandwidth core-to-core. According to NVIDIA, this design allows all cores to access full memory performance without internal bottlenecks.
This approach contrasts with recent data center CPU evolutions where chiplet designs enabled more cores and better costs but could also introduce memory access or inter-block communication penalties. NVIDIA calls this, with a very marketing-ish but quite illustrative phrase, the “chiplet tax.”
The company claims Vera achieves 1.8 times higher sustained performance per core than x86 on workloads that represent agent-like execution. They also cite Perplexity tests on real programming workflows—cloning repositories and running their test suites in sandboxes—where Vera completed about 1.5 times faster than x86, and ran concurrent sandboxes up to 1.9 times faster.
These figures should be viewed as what they are: data provided by NVIDIA and its partners. They do not replace independent testing in real client environments with their own models, tools, latencies, storage systems, and usage patterns. But they point toward an intriguing direction: agentic AI will force a reevaluation of data center metrics.
AI factories need more than just GPUs
In an AI factory, the most expensive asset is usually the GPU. That’s why any bottleneck reducing its utilization directly impacts task cost, response time, and infrastructure profitability. If an agent spends too many seconds waiting for the CPU to execute code, query data, or process results, the return on accelerators diminishes.
This ties into a broader trend. AI workloads are no longer just “prompt in, answer out.” Increasingly, applications combine models, agents, tools, search, databases, code, sandboxes, workflows, and security systems. Inference now becomes a part of a much longer chain.
NVIDIA also reports results in data workloads: Starburst claimed analysis of large-scale SQL is three times faster, and Redpanda achieved up to six times lower latency in real-time streaming compared to leading x86 servers—according to data shared by NVIDIA.
This explains why Vera isn’t just presented as a CPU to complement GPUs but as a component to unify multiple functions within the AI factory: agents, data, sandboxes, inference services, and reinforcement learning. Moreover, it will support the NVIDIA Vera Rubin platform and is linked to BlueField-4 STX, reinforcing NVIDIA’s intent to control more layers of the AI data center architecture.
Strategic message: NVIDIA aims for a bigger share of the server
Vera also carries a competitive message. NVIDIA doesn’t want to limit itself to selling GPUs. It aims to define the entire AI factory architecture: GPU, CPU, network, interconnect, DPU, software, libraries, orchestration, and tools. That integration can boost efficiency and reduce bottlenecks but also increases reliance on a single provider in already concentrated infrastructures.
For large customers, the decision will be both technical and economic. Is it worth adopting a more integrated platform if it improves per-agent performance, reduces latency, and better utilizes GPUs? In many cases, yes. But cost, availability, maturity, software compatibility, operational models, power consumption, cooling, and negotiation capacity also matter.
The announcement of the upcoming Rosa CPU, with a Rigel core based on Arm v9.2, shows that Vera is not an isolated experiment but part of a roadmap for proprietary CPUs tailored for the era of agentic AI. NVIDIA claims Rigel will deliver more performance per core than Olympus while maintaining the same silicon area, with improvements in instruction delivery, L2 cache, and memory management.
CPUs are no longer secondary
The most interesting aspect of Vera isn’t just the chip but the shift in focus. AI infrastructure is becoming more heterogeneous and increasingly dependent on the balance between components. A powerful GPU can be limited by storage, network, memory, CPU, scheduling, data, or software. The agent’s speed will be the speed of the entire system.
This forces infrastructure teams to ask new questions: It’s not enough to look at TFLOPS, GPU memory, or the number of accelerators per rack. They need to measure how long an agent takes to complete a real task, how many cycles it needs, what part of the time is spent on the model versus tools, how much latency sandboxes add, how long queries take, what data moves where, and where the loop stalls.
NVIDIA Vera embodies a clear response to this transition: CPUs with many cores, yes, but designed so each core remains fast under load. If the age of agents turns every second of waiting into a cost, then sustained single-threaded performance ceases to be just an architectural detail and becomes an economic variable.
GPUs will still be front and center. But in agent-based AI, increasingly, critical work happens between model calls. That’s where NVIDIA aims to position Vera.
Frequently Asked Questions
What is NVIDIA Vera?
It’s an NVIDIA CPU for AI data centers, designed for high per-core performance, large memory bandwidth, and agent-like workloads with many sequential steps.
Why does monothreaded performance matter in AI agents?
Because many agent steps depend on the previous result. If a tool, test, or query takes longer, the entire loop slows down.
Does Vera replace GPUs?
No. It’s meant to work alongside GPUs and accelerate CPU tasks surrounding the model: tools, code, data, verification, and orchestration.
What makes Vera different from traditional data center CPUs?
NVIDIA highlights its combination of 88 Olympus cores, high per-core performance, 1.2 TB/s of LPDDR5X memory bandwidth, and a 3.4 TB/s internal interconnect.
Are these performance figures independent?
They come from NVIDIA and partners like Perplexity, Starburst, or Redpanda. Each organization should validate them with their own real workloads.
via: blogs.nvidia
