Red Hat has released Red Hat AI 3.5, a major update that adds safety, observability, and cost-management tools designed to help organizations run artificial intelligence systems as trusted enterprise infrastructure rather than experimental pilots. The release reflects a broader industry shift as companies move from testing generative AI models to deploying them under the same governance and operational controls as other production systems.

What's Driving the Move From AI Pilots to Production?

For the past few years, many enterprises have treated AI as a testing ground, running small proof-of-concept projects to understand what's possible. But that phase is ending. Companies are now asking harder questions: How do we run this at scale? How do we control costs? How do we ensure safety and compliance? Red Hat's latest release addresses these concerns head-on, signaling that the industry has moved past the "let's try AI" phase and into the "how do we operate AI responsibly" phase.

Joe Fernandes, Vice President and General Manager of Red Hat's AI Business Unit, captured this shift in the company's launch remarks, stating: "The conversation has moved from getting AI into production to running it at scale as trusted enterprise infrastructure, which requires safety evidence, governed agents, cost attribution and multi-tenancy." He added that Red Hat AI 3.5 delivers "the operational controls, verifiable trust and agentic foundations IT leaders need to run AI as a safe, controlled and accountable enterprise AI architecture across the hybrid cloud".

Joe Fernandes, Vice President and General Manager of Red Hat's AI Business Unit

"The conversation has moved from getting AI into production to running it at scale as trusted enterprise infrastructure, which requires safety evidence, governed agents, cost attribution and multi-tenancy," said Joe Fernandes, Vice President and General Manager, AI Business Unit at Red Hat.

Joe Fernandes, Vice President and General Manager, AI Business Unit, Red Hat

How Does Red Hat AI 3.5 Help Organizations Manage AI Costs and Resources?

One of the most practical challenges enterprises face is managing expensive computing hardware, particularly graphics processing units (GPUs), which power AI inference. Red Hat AI 3.5 introduces several features to address this constraint:

• Inference-Time Scaling: Adjusts compute use dynamically based on the complexity of a query, allowing organizations to manage AI infrastructure costs more closely when running reasoning-heavy workloads.
• Fair-Share Scheduling: Divides GPU resources across multiple teams or tenants, ensuring that no single group monopolizes expensive hardware.
• Priority-Aware Serving: Routes requests by importance, protecting time-sensitive workloads while allowing background jobs to use spare capacity.
• Storage Offloading: Handles longer conversations and larger documents without requiring additional GPU hardware, reducing the need for expensive upgrades.
• CPU Offloading: Now generally available, allowing certain processing tasks to run on less expensive central processing units (CPUs) instead of GPUs.

These tools directly address a core pain point for enterprises: the cost of running AI systems at scale. By making GPU allocation more efficient and intelligent, organizations can serve more users and workloads without proportionally increasing their hardware spending.

What Safety and Compliance Features Are Included?

Beyond cost management, Red Hat AI 3.5 emphasizes safety and compliance, recognizing that enterprises need to verify their AI systems are trustworthy before deploying them widely. The release includes several new tools for this purpose:

• EvalHub (General Availability): Automates benchmarking and compliance reporting for custom models, retrieval-augmented generation (RAG) systems, and AI agents. This tool helps organizations document that their AI systems meet safety and performance standards.
• Garak Benchmark Scores: Models in Red Hat's catalogue now include standardized safety scores, along with indicators for risks such as toxicity and exposure of personally identifiable information (PII).
• NeMo Guardrails: Built-in safeguards that intercept malicious tool calls, preventing AI agents from executing harmful commands.
• Real-Time Observability Dashboards: Show inference health, GPU use, and model performance in real time, while non-administrator users can view token consumption on a per-user basis for cost attribution.

These features address a critical need: enterprises want evidence that their AI systems are safe and compliant before they roll them out to production. By automating benchmarking and providing transparent risk indicators, Red Hat makes it easier for IT leaders to make informed deployment decisions.

How Are AI Agents Becoming More Practical for Enterprises?

AI agents, which are AI systems that can take actions and interact with tools and external systems, are becoming increasingly important for enterprise automation. Red Hat AI 3.5 expands support for agents with several new capabilities:

• Responses API (General Availability): Provides a standard interface for multi-turn agent interactions, making it easier for developers to build conversational AI systems that can handle complex workflows.
• AutoRAG: Links enterprise data repositories to agent applications, with added support for multilingual documents, conversational testing, and contextual retrieval.
• Pre-Built Agent Templates: Reference implementations for code review, document processing, and research workflows, giving teams a head start on common enterprise use cases.
• Tool-Calling Validation: More than 20 new models have been added to Red Hat's catalogue, with some tagged as validated for tool-calling, which is essential for agents that need to interact with software tools and external systems.

These additions reflect growing enterprise interest in automating tasks such as document handling, research, and software review. By providing templates, standard interfaces, and validated models, Red Hat is making it easier for organizations to build and deploy agents without starting from scratch.

What Does Multi-Tenancy Support Mean for Service Providers?

For internal platform teams and service providers that want to offer AI as a shared service, Red Hat AI 3.5 introduces stronger isolation and control options. The software now officially supports hosted control planes on OpenShift Virtualization, a technology that allows each tenant to get a dedicated cluster control plane while sharing the underlying hardware. AI workloads can also run in virtual machines on shared GPU-enabled infrastructure.

This capability is significant because it allows service providers to offer AI services to multiple customers while maintaining strong separation between their workloads and retaining central oversight of upgrades and infrastructure operations. It's a key enabler for the emerging market of AI-as-a-service offerings.

How Is Red Hat Expanding Kubernetes Support?

Red Hat is also extending distributed inference beyond its own OpenShift platform to third-party Kubernetes services, which are containerized computing environments used by many enterprises. Support is now generally available on CoreWeave CKS and Microsoft Azure, while Amazon EKS remains in technology preview. Additionally, a separate developer preview brings the Kubeflow Spark Operator into the workbench environment, allowing distributed data processing alongside model serving on the same platform.

This expansion signals that Red Hat is positioning its AI tools as platform-agnostic, allowing enterprises to run AI workloads wherever they've already invested in infrastructure, rather than forcing them to adopt a single vendor's ecosystem.

Why Does This Release Matter for the Broader AI Industry?

Red Hat AI 3.5 is significant because it reflects a maturation of the AI market. The early hype around generative AI focused on capabilities and performance, but enterprises are now asking practical questions about safety, cost, compliance, and operational control. Red Hat's focus on these areas suggests that the industry is moving beyond the "wow, AI can do amazing things" phase and into the "how do we run AI reliably and cost-effectively at scale" phase.

The emphasis on inference-time scaling is particularly noteworthy. Rather than simply throwing more computing power at AI problems, enterprises are looking for smarter ways to allocate resources based on the actual complexity of each task. This approach could help make AI systems more economically sustainable for organizations of all sizes, not just those with unlimited budgets for computing hardware.