Self-hosted AI platforms like Ollama are increasingly becoming targets for cyberattacks, prompting security researchers to develop new tools and practices to protect local language model deployments. As more users and enterprises shift away from cloud-based AI services toward running large language models (LLMs) on their own hardware, the security implications of this trend are coming into sharper focus. A newly released open-source security scanner has identified vulnerabilities in popular self-hosted AI tools, signaling that the convenience of local AI comes with real risks that many users may not yet understand.

What Security Risks Come With Self-Hosted AI Models?

Tencent's Zhuque Lab has released AI-Infra-Guard, an open-source security scanner designed specifically to identify vulnerabilities in self-hosted AI systems. The tool fingerprints running services such as Ollama, vLLM, and ComfyUI, checking them against a database of more than 1,600 known vulnerabilities (CVEs, or common vulnerabilities and exposures). Beyond simple vulnerability scanning, the tool also inspects MCP (Model Context Protocol) servers and agent skills across 14 categories of risk and runs jailbreak evaluations against target models to test whether they can be tricked into producing harmful outputs.

The emergence of this security scanner reflects a broader shift in how organizations are deploying AI. While self-hosted models offer privacy benefits and eliminate dependency on cloud providers, they introduce new attack surfaces that traditional security teams may not be equipped to defend. Unlike cloud-based AI services, which are managed and patrolled by large security teams, locally hosted models run on infrastructure that individual users or smaller teams are responsible for securing.

How to Secure Your Self-Hosted AI Deployment

• Regular Vulnerability Scanning: Use tools like AI-Infra-Guard to regularly scan your Ollama, vLLM, or other self-hosted AI services against known CVEs and check for misconfigurations that could expose your models to attack.
• Jailbreak Testing: Periodically evaluate whether your deployed models can be manipulated into producing harmful content through adversarial prompts, and update your model versions or add safety guardrails if vulnerabilities are discovered.
• MCP Server Inspection: If you are using Model Context Protocol servers or integrating agent skills into your AI system, audit them across multiple risk categories to ensure third-party components do not introduce security gaps.

The security scanner developed by Tencent's team addresses a critical gap in the self-hosted AI ecosystem. Until now, users running Ollama or similar platforms had limited visibility into whether their deployments were vulnerable to known exploits. The tool's ability to check against 1,600 CVEs and evaluate jailbreak resistance provides a concrete way for organizations to assess their security posture before an attacker finds a weakness.

Why Are Self-Hosted AI Platforms Becoming Targets?

The shift toward self-hosted AI is driven by legitimate concerns about privacy, cost, and vendor lock-in. Users and enterprises want to run models locally without sending sensitive data to third-party cloud providers. However, this trend has also created a new attack surface. Self-hosted AI services often run on consumer or enterprise hardware that may not have the same level of security hardening as cloud infrastructure. Additionally, many users deploying Ollama or similar tools may not have dedicated security teams monitoring for threats in real time.

The existence of AI-Infra-Guard suggests that security researchers and vendors are beginning to recognize this risk. By providing an open-source tool that can fingerprint and audit self-hosted AI systems, Tencent's Zhuque Lab is helping to raise awareness about the need for security practices in the self-hosted AI space. This is particularly important as AI agents become more autonomous and capable of taking actions on behalf of users, increasing the potential impact of a compromised system.

The broader cybersecurity community is also grappling with how to secure AI systems in general. According to recent threat intelligence, AI agents are being given a larger role in cyberattacks, automating parts of attacks with less human involvement, including vulnerability scanning and credential harvesting. This arms race between defensive and offensive AI capabilities underscores why security tools tailored to self-hosted AI deployments are becoming essential.

For users who have chosen Ollama or other self-hosted platforms for their AI needs, the message is clear: convenience and privacy benefits must be balanced against the responsibility to maintain security. Running a local language model is no longer a set-it-and-forget-it proposition. It requires ongoing monitoring, regular updates, and the use of specialized security tools to detect and prevent exploitation. As the self-hosted AI ecosystem matures, security practices will likely become as important as performance and ease of use in determining which platforms users choose to adopt.