Red Hat is betting that the future of enterprise AI belongs to companies that can deploy and manage existing models efficiently, not those building models from scratch. The company unveiled Red Hat AI 3.4, an updated platform designed to help organizations operationalize artificial intelligence at scale across hybrid cloud environments, with particular emphasis on inference workloads and autonomous agent management.

Why Is Inference Becoming More Important Than Model Training?

For years, the AI conversation centered on building larger, more powerful models. But Red Hat's latest platform signals a fundamental shift in how enterprises think about AI deployment. The company argues that inference, the process of running a trained model to generate predictions or responses, will become the dominant enterprise AI workload, not model training.

"What's really going to drive inference demand exponentially is AI agents. We provide a platform where customers can deploy and manage their AI agents across a hybrid infrastructure environment," said Joe Fernandes.

Joe Fernandes, Vice President and General Manager of Red Hat AI

This shift reflects a practical reality: most enterprises lack the resources and expertise to train foundational models from scratch. Instead, they want to take existing models from companies like OpenAI or Meta and connect them to their own proprietary data. Red Hat's platform is designed around this workflow, treating inference as the primary workload rather than an afterthought.

How to Deploy and Manage AI Agents Across Enterprise Infrastructure

• Model Governance: Red Hat's new model-as-a-service capability lets administrators expose internally approved AI models through controlled interfaces, track usage patterns, and apply policy controls to prevent unauthorized access or misuse.
• Speculative Decoding Optimization: The platform now supports speculative decoding techniques in the vLLM inference server, which can accelerate text generation up to three times while reducing inference costs, improving performance without sacrificing output quality.
• Confidential Computing: Red Hat added support for confidential containers running on NVIDIA confidential computing infrastructure, protecting AI workloads and agents even if other systems on the same hardware are compromised.
• Agent Observability and Tracing: New tracing capabilities track inference calls and tool usage, while support for Model Context Protocol gateways and catalogs enables better visibility into agent behavior across hybrid environments.
• Prompt Management and Evaluation: The platform treats prompts as managed enterprise assets and includes an evaluation hub designed to assess model and agent quality, safety, and accuracy using MLflow for experiment tracking.

"Pretraining models from scratch is limited to a few very large organizations. We find enterprise customers are more focused on consuming those models and then basically connecting them to their own data," explained Joe Fernandes.

Joe Fernandes, Vice President and General Manager of Red Hat AI

Red Hat also deepened its collaboration with NVIDIA, adding support for NVIDIA's Blackwell architecture and the upcoming Vera Rubin platform. The partnership includes participation in NVIDIA's OpenShell project for AI agent sandboxing and secure execution, as well as support for confidential containers within Red Hat OpenShift sandboxed containers.

What Infrastructure Changes Support Production AI Deployments?

Beyond the AI platform itself, Red Hat announced several infrastructure innovations designed to support enterprise AI at scale. Fedora Hummingbird Linux is a new rolling-release, image-based Linux distribution optimized for AI-driven development environments, with rapid upstream updates and minimal security vulnerabilities. The distribution removes registration barriers and supports anonymous, automated deployment workflows for AI agents.

The company also introduced Red Hat Hardened Images, minimal container images designed to support zero-CVE (Common Vulnerabilities and Exposures) security strategies. These images contain only the components required for applications to run and include software bills of materials and cryptographic verification, providing a security-first foundation for AI workloads.

Red Hat Enterprise Linux Long-Life Add-On extends support for specific releases indefinitely through annual renewals, addressing industries like aerospace, healthcare, and telecommunications that operate infrastructure with multi-decade lifecycles. This offering reflects the reality that enterprises need both rapid innovation cycles and long-term stability.

On the developer side, Red Hat announced general availability of Red Hat Desktop, a commercially supported version of the Podman Desktop open-source tool that simplifies container and Kubernetes management on local machines. With more than 4 million downloads, Podman Desktop has become the industry standard for working with Linux containers on developer laptops.

As enterprises move from AI experimentation to production deployments, the bottleneck is shifting from model quality to operational efficiency, governance, and safety. Red Hat's AI 3.4 platform reflects this reality, offering tools that help organizations maximize the value of existing AI models while maintaining control, visibility, and security across hybrid cloud environments. The focus on inference, agent management, and infrastructure stability positions enterprises to operationalize AI at scale without discarding existing infrastructure investments.