Self-hosted AI models are spreading rapidly across enterprises, but the shift is creating a security crisis that most organizations don't yet see. When companies move artificial intelligence (AI) workloads from cloud providers onto their own servers and laptops, they gain control over costs and data, but they also inherit every security responsibility that the cloud provider previously handled. The result is a landscape of unmonitored, unpatched, and often unauthenticated AI systems running inside corporate networks, frequently invisible to security teams.

The economics driving this shift are undeniable. Open-weight models, which are AI models whose trained parameters are publicly available for download and self-hosting, now process nearly a third of all tokens routed through Vercel's production gateway, yet account for less than 4 percent of total spending. Chinese open-weight models like DeepSeek and Qwen have become particularly attractive, running 60 to 90 percent cheaper than leading US cloud APIs. For enterprises managing large-scale AI workloads, the cost difference is transformative. But the security implications are being overlooked.

Why Are Enterprises Moving AI Workloads Locally?

Companies are adopting self-hosted AI for four primary reasons. First, they want to keep sensitive prompts and data inside their own networks rather than sending them to external cloud providers. A semiconductor company's design specifications, a pharmaceutical firm's trial data, or a bank's customer files all represent intellectual property that flows through AI agents as context. When that data transits someone else's infrastructure, it falls under someone else's terms of service and retention policies.

Second, the token economics of agentic AI have become unmanageable. When AI agents loop through multiple steps, calling tools and reasoning repeatedly, cloud API costs become unpredictable and grow with exactly the behavior companies want to encourage. Open-weight models running on owned hardware eliminate this variable cost problem.

Third, regulatory constraints make cloud AI impossible for certain workloads. Clinical data, export-controlled designs, classified programs, and financial records carry legal restrictions on where they can be processed. For these use cases, self-hosting is not optional; it is the only path forward.

Fourth, enterprises want direct control over the model itself, including the ability to fine-tune it, audit its behavior, and modify its guardrails. This level of ownership is impossible with a rented API.

What Security Risks Come With Local AI Deployment?

The security problem emerges because self-hosted AI is being deployed as software infrastructure, often outside the visibility of security teams. A developer can install Ollama, a popular tool for running local AI models, pull a model, and connect it to internal code or documents in minutes. A team can deploy vLLM, another inference runtime, on a cloud GPU instance without waiting for approval from a central AI platform. These deployments happen fast, and they happen quietly.

The exposure is measurable. Cisco Talos scanned Shodan, a search engine for internet-connected devices, for just ten minutes and found 1,139 publicly exposed Ollama endpoints. Of those, 214 answered model queries with no authentication credentials at all, and 36.6 percent were located in the United States. These are only the deployments visible from the public internet; the ones inside corporate networks are harder to count and far more numerous.

The inference runtimes themselves are production attack surfaces. Ollama, vLLM, and llama.cpp are fast-moving infrastructure projects that now sit beside valuable data and powerful compute. Ollama experienced a critical path traversal vulnerability (CVE-2024-37032) that allowed arbitrary file writes and remote code execution, particularly in exposed Docker installations running with elevated privileges. vLLM has had multiple critical vulnerabilities, including one where a malicious model could execute code during initialization by exploiting an assert statement that disappears under Python optimized mode. Another vLLM advisory showed that revision pins were not consistently applied to all artifacts a model loads, meaning a deployment could look reviewed and pinned while its runtime behavior still depended on mutable, unaudited components.

How Does Local AI Become a Gateway for Attackers?

The real risk compounds when a local model is connected to agents, tools, credentials, and data sources. Consider a local coding agent that can read source code, access cloud credentials, search internal documentation, invoke tool-calling protocols, delegate work to sub-agents, and run shell commands. Each capability is defensible on its own. The exposure lives in the combination. A model can receive untrusted content from logs, documents, repositories, or web results, then carry that context through a delegation path into a privileged action. The final shell command is local. The delegation happens in process. No gateway request ever exposes the full chain.

This reachability problem runs through the entire AI application stack, and it is why local AI security cannot stop at model inventory or network inspection. Traditional security tools each have structural boundaries where they stop seeing the threat:

• AI Gateways: They inspect traffic routed through them, but local inference never leaves the host, so there is no request to inspect.
• Data Loss Prevention (DLP) and Cloud Access Security Brokers (CASB): They monitor data moving to sanctioned services, but in-process context assembly and local file reads happen invisibly.
• Distributed Tracing: It produces rich telemetry where the application is instrumented, but coverage ends where instrumentation ends, and shadow deployments are never instrumented.
• Endpoint Detection and Response (EDR) and Security Information and Event Management (SIEM): They identify processes and network connections, but they cannot prove which model loaded or what it can reach at runtime.

Steps to Secure Self-Hosted AI Deployments

Organizations deploying self-hosted AI need to move beyond traditional security boundaries and implement controls specifically designed for local inference. The challenge is that configuration files and inventory systems tell you what should be running, not what actually loaded at runtime. Security teams need independent evidence of what the system is executing.

• Runtime Intelligence: Deploy tools that analyze user-space memory to report what an application actually loaded and executed, rather than relying on manifests or config files. This reveals shadow AI infrastructure within minutes of deployment.
• Model Supply Chain Auditing: Understand that loading a model can mean loading executable software from external repositories, including configuration files, tokenizers, processors, custom Python modules, and secondary weights. Verify that revision pins are consistently applied to every artifact a model loads, not just the primary weights.
• Inference Runtime Patching: Treat inference servers as production services that require the same patching discipline as any other production infrastructure. Monitor security advisories for Ollama, vLLM, llama.cpp, and other runtimes in active use.
• Agent Reachability Mapping: Document which models are connected to which agents, tools, credentials, and data sources. Map the full delegation path from untrusted input to privileged action, understanding that the risk lives in the combination of capabilities, not in any single component.

The fundamental challenge is that self-hosted AI security requires visibility into layers that traditional enterprise security tools were never designed to see. A developer installing Ollama on a laptop, a team deploying vLLM on a GPU cluster, or a desktop agent talking to a local model server all represent the same pattern: powerful AI infrastructure deployed outside the visibility of the security team, often without authentication, frequently connected to sensitive data, and running on endpoints that may not be patched.

What Does This Mean for Enterprise AI Strategy?

The shift toward self-hosted AI is not slowing down. The economics are too compelling, and the regulatory drivers are too strong. Microsoft CEO Satya Nadella recently published a post titled "The Reverse Information Paradox," arguing that enterprises using cloud AI "pay for intelligence twice": once in cash, and again in the proprietary knowledge they must reveal to make the intelligence useful. His prescription is to run models inside secure environments of the enterprise's own, keeping data, traces, evaluations, and adapted weights inside the corporate perimeter.

But moving the model inside the perimeter is only half the solution. The tools, telemetry, and logging around the model must stay inside too. Organizations that deploy self-hosted AI without addressing the security blind spots are trading one set of risks for another. The cloud provider's security boundary is replaced by the enterprise's own responsibility to secure every layer beneath the prompt, including the guardrails, the inference runtime, the agents, the tools, and the credentials wired to them.

For enterprises serious about self-hosted AI, the path forward requires treating local models as production infrastructure, not as developer toys. That means inventory, patching, authentication, network segmentation, and runtime visibility. It means understanding the model supply chain as an execution path, not just a file download. And it means accepting that the security team owns every layer beneath the prompt, because the enterprise now owns every layer beneath the prompt.