The conventional wisdom that enterprises should standardize on a single AI agent SDK (software development kit) for Kubernetes clusters is fundamentally flawed. Instead, organizations running on-premise infrastructure or in private virtual clouds need to evaluate frameworks based on two critical capabilities: the ability to run in containers they control and support for self-hosted models like those served through Ollama or vLLM.

What Makes an Agent SDK Enterprise-Ready?

Most tutorials on building AI agents gloss over the real operational requirements that matter in production environments. When platform and engineering leaders evaluate agent SDKs, they're not just looking for the fastest way to prototype. They need frameworks that can run in containerized environments without vendor lock-in and integrate with locally-hosted language models (LLMs), which are AI systems trained to understand and generate human language.

The distinction is crucial. Many popular agent SDKs treat self-hosted models as an afterthought or don't support them at all. This creates a hidden dependency on cloud-based APIs, which can introduce latency, data residency concerns, and ongoing costs that don't align with on-premise deployment strategies.

Which Frameworks Actually Support Local Models?

Six agent SDKs currently dominate enterprise conversations, and they sit on a spectrum of model flexibility. The frameworks with the broadest support for self-hosted deployments include:

• CrewAI: Offers the fastest time to production with a role-based agent collaboration model, MIT-licensed open-source code, and seamless integration with Ollama or self-hosted vLLM endpoints without significant configuration overhead.
• Google's Agent Development Kit (ADK): Ships in Python, TypeScript, Go, Java, and Kotlin with documented support for Ollama, vLLM, and other models through LiteLLM, a library that abstracts multiple AI providers into a single interface.
• Microsoft's Semantic Kernel (merged with AutoGen): Provides both loose, reasoning-driven orchestration and deterministic business-logic workflows, with first-party connectors for Ollama and support for open-weight models like Qwen and Mistral.
• LangGraph: Designed for complex, long-running workflows with durable checkpointed state that can pause, resume, and rewind, though it requires operational infrastructure like PostgreSQL and Redis.
• OpenAI Agents SDK: Supports self-hosting through OpenAI-compatible endpoints and extensions like LiteLLM, though OpenAI remains the default and best-supported path.

One notable exception is Anthropic's framework, which runs exclusively on Claude and offers no supported path to swap in alternative models, even self-hosted ones. While it can route traffic through Amazon Bedrock or Google's Gemini Enterprise Agent Platform for compliance purposes, those are all channels for hosting Claude, not alternative model vendors.

How to Evaluate Agent SDKs for Your Kubernetes Cluster

• Model Flexibility: Verify the framework explicitly supports Ollama, vLLM, or other self-hosted model servers. Check whether the documentation includes examples of local model integration, not just cloud API examples.
• Operational Weight: Assess infrastructure dependencies. Some frameworks like LangGraph require PostgreSQL for state management and Redis for streaming, while others like CrewAI can run with minimal additional services in a Kubernetes environment.
• Language and Licensing: Confirm the SDK is available in your team's primary language (Python, TypeScript, Go, Java, or.NET) and uses permissive open-source licenses like MIT or Apache 2.0 that allow commercial self-hosting.
• Control Granularity: Determine whether you need fine-grained control over each agent step or can work with higher-level abstractions. Role-based frameworks like CrewAI prioritize speed, while graph-based frameworks like LangGraph offer more explicit control.
• Kubernetes Integration: Look for native support through Helm charts or custom resource definitions (CRDs) that make the framework feel native to your cluster, rather than treating it as a generic containerized application.

The honest answer to the original question, "which agent SDK should we standardize on," is that the framing itself is problematic. Rather than forcing a single choice, enterprises benefit from understanding the trade-offs and selecting frameworks based on specific use cases within their organization.

Why One Size Doesn't Fit All

CrewAI excels at rapid prototyping and multi-agent collaboration with the lowest learning curve, making it ideal for teams moving quickly from concept to production. However, when you need fine-grained control over workflow steps or must handle complex state management across long-running processes, LangGraph's explicit graph model and checkpoint capabilities become more valuable, despite the operational overhead.

Google's ADK shines for organizations already invested in Google Cloud infrastructure and needing hierarchical agent delegation with native agent-to-agent (A2A) communication protocols. Microsoft's Semantic Kernel is the clear choice for.NET shops and enterprises that need both reasoning-driven and deterministic workflow orchestration in a single framework.

The critical insight for platform leaders is that model flexibility and self-hosting capability should be non-negotiable requirements, not optional features. As enterprises increasingly deploy AI workloads on-premise for latency, compliance, or data sovereignty reasons, the ability to run against locally-hosted models through Ollama or similar tools becomes a strategic differentiator. Frameworks that treat self-hosted models as a second-class citizen will eventually create technical debt and vendor dependencies that undermine the original on-premise deployment strategy.

" }