Researchers found that seven of nine image-editing tools tested on Hugging Face produced sexualized alterations of photographs, revealing a critical gap in how enterprises evaluate AI models from public repositories. The July 28 findings highlight a fundamental mismatch in AI governance: approving a familiar platform is not the same as reviewing the individual models, applications, and safeguards an organization actually plans to deploy.

What Did the Deepfake Testing Reveal?

The European nonprofit AI Forensics tested nine image-editing Spaces, which are interactive applications hosted on Hugging Face, that it described as among the platform's leading tools. Seven generated topless alterations while retaining the subject's facial identity and positioning. The researchers used a direct prompt and did not attempt to bypass safety controls, suggesting the tools lacked adequate built-in safeguards.

The research team also created decoy Spaces that recorded requests without producing images. Over seven days, they received more than 1,000 requests, with 73% classified as sexual in nature. Of those sexual requests, 83% sought to undress or sexualize a person in an uploaded image. The researchers noted that 95% of the apparent targets were women, while 6.7% of the requests appeared to involve children.

This is not an isolated problem. A 2025 ACM study identified almost 35,000 deepfake model variants across Hugging Face and Civitai, with most hosted on Civitai. In a subset of more than 2,000 models, researchers found that 96% targeted women and that many signaled an intended sexual use.

Why Is This a Vendor Risk Problem for Enterprises?

The core issue is structural. Hugging Face did not develop all the tools examined in the research, and the platform did not answer questions about its moderation and safety systems. While Hugging Face's content policy prohibits sexual content created without explicit consent, sexual content involving minors, and content used for harassment or bullying, enforcement appears inconsistent.

The problem extends beyond Hugging Face itself. A public AI platform, model publisher, application developer, and inference provider may all be different parties. When an enterprise approves Hugging Face as a vendor, it does not automatically vet the hundreds of thousands of individual models and applications hosted there. This creates a governance blind spot where responsibility for safety becomes unclear.

How Should Enterprises Manage AI Model Risk?

• Verify Provenance and Ownership: Record the publisher, repository, license, version, commit or cryptographic hash, dependencies, and inference provider. Cisco's open-source Model Provenance Kit is one example of tooling designed to trace model origins and modifications.
• Test the Deployed Configuration: Check whether the exact model and application have been tested for sexualized edits involving identifiable people and children. Confirm whether safeguards inspect prompts, outputs, or both, and whether users can disable them.
• Control Downloads and Updates: Require approval for models pulled from public hubs, especially when they include custom code, adapters, or revised weights. Contracts should require notice before a supplier changes its underlying model or inference provider.
• Assign Incident Responsibility: Agreements should identify who handles abuse reports, takedowns, evidence preservation, and notifications. Reviews should also cover dependencies below the primary supplier, as the LiteLLM-linked supply chain breach demonstrated in April.

The National Telecommunications and Information Administration (NTIA) noted in a July 2024 report that widely available model weights could lower barriers to producing nonconsensual intimate imagery and child sexual abuse material. However, the agency did not recommend broad restrictions on open models.

Public repositories remain useful sources of models and developer tools, but platform approval cannot replace model-level review. Before production use, enterprises should pin the approved version, document its origin, test its capabilities, and assign responsibility for failures. Open weights reduce a repository's technical control once a model is downloaded, making this due diligence even more critical.

The Hugging Face deepfake findings underscore a broader lesson: in the era of open-source AI, brand trust is necessary but not sufficient. Enterprises must treat each model as a distinct procurement decision, complete with its own security review, testing, and accountability structure.