Red Hat has released Red Hat AI 3.5, an update to its artificial intelligence platform aimed at moving projects from the pilot stage into production environments with controls that look more like enterprise infrastructure. The release adds pre-deployment evaluation for models and agents, new observability metrics, multi-tenancy on shared GPUs, and tools for deploying RAG applications and agents with more operational control.
Red Hat AI 3.5 in 20 seconds
- EvalHub arrives with security evaluation and auditable report generation.
- Red Hat adds inference metrics, token consumption, and GPU utilization tracking.
- The platform strengthens multi-tenancy with isolation via OpenShift Virtualization.
- AutoRAG and agent templates aim to speed up enterprise projects built on private data.
- Red Hat AI 3.5 is now generally available.
The update reflects a fairly clear shift in the enterprise AI market. Many organizations have already moved past the phase of experimenting with models and now need to solve less glamorous but harder problems: who’s consuming each GPU, how much each workload costs, how a model gets updated without breaking a service, what level of isolation exists between teams, and how to prove a system passed certain evaluations before going into production.
Red Hat wants to cover exactly that operational layer on top of OpenShift and its AI platform.
EvalHub and Garak bring security evaluation before deployment
One of the main additions is EvalHub, now generally available in Red Hat AI 3.5.
The tool is designed to evaluate models, RAG systems, and agents before deployment. According to Red Hat, it can automate security testing and generate reports intended for internal compliance and audit processes.
The company is also folding in scores from Garak, a tool for evaluating vulnerabilities and language model behavior, as part of its validated model catalog.
Red Hat says it has added more than 20 new models with performance testing and metrics covering security, personally identifiable information exposure, and toxicity risk.
Vendors represented include Google, NVIDIA, and Alibaba Cloud, with models such as Gemma 4, Nemotron 3, and Qwen.
It’s worth clarifying what this validation actually means. A favorable score on a given test suite doesn’t automatically make a model safe for any application, nor does it by itself amount to compliance with a specific regulation.
The value lies in having reproducible, comparable evidence that can be folded into an organization’s approval process.
Red Hat also flags some models as validated for tool calling, a capability especially relevant for agents capable of taking action on external systems.
| Area | What’s new in Red Hat AI 3.5 |
|---|---|
| Evaluation | EvalHub reaches general availability |
| Model security | Garak scores and risk testing |
| Agents | Models validated for tool calling |
| Compliance | Auditable evaluation reports |
| Updates | Controlled rollout of new versions |
Shared GPUs become a platform problem
Red Hat AI 3.5 also pays considerable attention to managing GPUs in shared environments.
When an experimental project uses a handful of dedicated GPUs, splitting resources is relatively simple. The problem changes when dozens of teams share accelerators and some workloads need low latency while others can run in the background.
The new version adds fair-share GPU scheduling, designed to distribute resources across different tenants, along with priority mechanisms for inference serving.
The system can apply admission control and route requests based on priority. The goal is to prevent a secondary workload from hogging all the capacity and affecting an application that needs to respond in real time.
Red Hat is also adding official support for running its AI platform on OpenShift hosted control planes deployed through OpenShift Virtualization.
This approach lets each tenant get its own control plane while consolidating the underlying physical hardware.
AI workloads can run inside virtual machines on servers with shared GPUs, adding a further layer of isolation between customers or business units.
It’s not full physical isolation, but it can be useful for internal service providers, private clouds, or corporate platforms where several teams use the same infrastructure.
More metrics to know who’s consuming tokens and GPUs
Observability is another area getting reinforced.
Red Hat AI 3.5 adds dashboards for monitoring inference health, model performance, and GPU utilization levels.
It also adds per-user token consumption metrics.
That data is especially practical for platform and FinOps teams. When AI moves from a limited pilot to hundreds or thousands of users, knowing how much each team consumes can be necessary to split costs, spot anomalous workloads, or justify capacity expansions.
Red Hat calls this showback, not necessarily chargeback.
The first shows each unit how much it consumes even without billing it internally. The second means actually passing that cost on to the relevant department.
The platform also adds visual agent tracing through MLflow, making it possible to follow the execution of multi-step flows involving tools and model calls.
Observability for agents is more complicated than for a traditional API, because a single user request can trigger several calls, searches, tool executions, and intermediate decisions.
AutoRAG tries to reduce the work of connecting models to enterprise data
Red Hat is also expanding its tools for RAG applications, short for Retrieval-Augmented Generation.
AutoRAG aims to automate part of the process of connecting internal repositories to AI applications.
Version 3.5 adds support for multilingual documents, conversational testing, and contextual retrieval, along with pgvector compatibility.
The company includes a visual interface for designing and evaluating the flow before taking it to production.
The goal is to cut down on the manual work needed to decide how documents get chunked, how embeddings are generated, which retrieval method is used, and how response quality gets evaluated.
Red Hat is also adding AutoML and a feature called Inference-Time Scaling, which dynamically adjusts the compute used based on the difficulty of a query.
The idea is to avoid assigning the same level of resources to a simple request and to one that requires more reasoning.
That doesn’t necessarily mean the system knows in advance the optimal cost of every question. It’s a management layer that tries to adapt inference spending to the detected complexity.
Agents arrive with templates and access controls
Red Hat AI 3.5 adds preconfigured templates for agents built around common tasks such as code review, document processing, and research.
These templates are part of AI Hub and provide a foundation with frameworks, tools, and deployment configurations.
The company wants agents to be born inside the platform’s security and operational controls, rather than having them bolted on afterward.
General availability also arrives for the Responses API and built-in RAG, with an open interface for multi-turn agent conversations.
Red Hat is also adding integration with NVIDIA NeMo Guardrails to intercept certain malicious or unauthorized tool calls.
That matters a lot when an agent can modify files, query databases, or take action on corporate systems.
A chatbot that answers poorly can produce incorrect information. An agent with access to tools can also carry out an incorrect action.
The risk surface changes.
More options for offloading memory outside the GPU
Memory management also gets improvements.
Red Hat is declaring CPU offloading generally available, letting part of the information move from GPU memory to the system’s main memory.
Secondary storage also enters as a developer preview through storage offloading.
These techniques can make it possible to work with longer conversations or larger models without directly increasing the amount of available HBM.
The cost is in the transfers.
Moving information between GPU, RAM, and storage is much slower than keeping it permanently inside the GPU’s high-speed memory, so the outcome depends on each specific workload.
This fits a broader trend in inference: treating memory as a multi-tier hierarchy instead of assuming everything must always live in HBM.
llm-d expands distributed inference beyond OpenShift
Another addition is the expansion of llm-d, Red Hat’s technology for distributed inference.
Red Hat AI 3.5 extends its reach to third-party Kubernetes services.
The company is announcing general availability on CoreWeave Kubernetes Service and Microsoft Azure, while Amazon EKS enters as a technology preview.
That makes it possible to use a more consistent model-serving layer even when the underlying Kubernetes infrastructure isn’t OpenShift.
Early support also shows up for vLLM Omni, designed to serve text, audio, and image generation from a common layer.
In this case, the feature is still in early access, so it shouldn’t be treated as a fully mature production capability yet.
Red Hat wants AI managed like any other critical infrastructure
The direction of Red Hat AI 3.5 is fairly clear.
The company is spending less energy competing over who has the most powerful model and more on solving the layer that forms around the model once an organization wants to use it continuously.
Evaluations, isolation, GPU usage, token tracking, progressive rollouts, RAG, agents, and traceability are all part of that problem.
Not every announced feature has the same level of maturity. Some are generally available, while others remain in developer preview, technology preview, or early access.
That distinction will matter to any company weighing whether to run Red Hat AI 3.5 in production.
The release is now generally available and is also part of Red Hat AI Factory with NVIDIA.
More than an update centered on new models, Red Hat AI 3.5 is an attempt to turn artificial intelligence into something platform teams can manage with controls similar to the ones they already use for applications, clusters, and other critical services.
Frequently asked questions
What is EvalHub in Red Hat AI 3.5?
EvalHub is a generally available tool for evaluating models, RAG systems, and agents before deployment. It can automate security testing and generate reports intended for internal compliance and audit processes.
How does Red Hat AI 3.5 handle shared GPUs?
It adds fair-share GPU scheduling and priority-based request routing for inference, plus hosted control planes via OpenShift Virtualization so each tenant gets isolation on shared GPU hardware.
What does AutoRAG do?
AutoRAG automates part of the work of connecting internal repositories to AI applications, with support for multilingual documents, conversational testing, contextual retrieval, and pgvector compatibility.
Is Red Hat AI 3.5 generally available?
Yes, the release is generally available and also part of Red Hat AI Factory with NVIDIA, although some individual features remain in developer preview, technology preview, or early access.
What’s the difference between showback and chargeback in Red Hat AI 3.5?
Showback shows each team how much it consumes in tokens and GPU usage without necessarily billing it internally, while chargeback means passing that cost directly on to the corresponding department.
