DeepSeek V3 achieves a lower hallucination rate than GPT-4.1 on some benchmarks, but new research shows this advantage comes with a hidden cost: the model is more likely to omit useful information entirely rather than provide inaccurate answers. A peer-reviewed clinical study published in April 2026 found that while DeepSeek V3 never fabricated medical guidelines, it also rarely supplied the correct ones, raising questions about what "safer" AI actually means in practice.

What Does the Data Actually Show?

The comparison between these two models reveals a fundamental challenge in measuring AI reliability: different benchmarks tell very different stories. On Vectara's HHEM-2.3 hallucination leaderboard, which evaluates how often models introduce unsupported information when summarizing documents, GPT-4.1 achieved a 5.6% hallucination rate compared to DeepSeek V3's 6.1%. That's a narrow 0.5 percentage point gap across more than 7,700 articles spanning news, medicine, law, business, technology, science, sports, and education.

But the clinical study painted a starkly different picture. Researchers generated nearly 14,000 outputs from both models using simulated cases involving high cholesterol and type-2 diabetes, then checked whether outputs included real clinical guidelines, omitted relevant ones, or cited guidelines that couldn't be verified. The results exposed a critical trade-off:

• GPT-4.1 Guideline Accuracy (High Cholesterol): Included valid guidelines in 46.34% of outputs but hallucinated guidelines in 7.31% of cases
• DeepSeek V3 Guideline Accuracy (High Cholesterol): Never hallucinated guidelines but only included valid ones in 2.92% of outputs
• GPT-4.1 Guideline Accuracy (Type-2 Diabetes): Included valid guidelines in 69.21% of outputs but hallucinated in 8.48% of cases
• DeepSeek V3 Guideline Accuracy (Type-2 Diabetes): Never hallucinated but only included valid guidelines in 1.84% of outputs

Why Does a "Safe" Model Sometimes Give Worse Answers?

The clinical study authors framed their findings around three distinct risks: hallucination, omission, and inconsistency. The key insight is that a model can fail in multiple ways beyond simply inventing false information. DeepSeek V3's zero hallucination rate didn't reflect superior medical knowledge; it reflected a tendency to withhold information rather than provide it.

Think of it like asking two research assistants to identify a relevant government regulation. Assistant A finds the right rule most of the time but occasionally invents one. Assistant B never invents anything but rarely finds the actual rule either. If you measure only fabricated citations, Assistant B looks perfect. But if your task requires finding the relevant rule, Assistant B may still be far less useful. That's essentially what happened in the clinical evaluation.

"A model that says less can achieve fewer hallucinations without necessarily providing more accurate or useful information," the researchers noted in their analysis.

BMJ Health & Care Informatics Study Authors

How Should You Interpret Hallucination Benchmarks?

Hallucination comparisons are easy to misread because the term itself describes only one failure mode. A model can also refuse to answer, omit important facts, give incomplete answers, cite no source at all, use a correct source incorrectly, provide factually accurate statements that don't answer the question, or reach the right conclusion for the wrong reason. Averaging different types of failures into a single "hallucination score" can reward overly cautious models that simply avoid engaging with difficult questions.

The Vectara leaderboard comparison between GPT-4.1 and DeepSeek V3 represents one of the cleaner direct comparisons available because both models are evaluated using the same framework. However, the researchers emphasized that this finding should not automatically extend to open-ended questions, medicine, coding, legal research, web search, or citation generation. Each domain has different failure modes and different consequences for getting things wrong.

It's also worth noting that older articles about DeepSeek V3 often cite a 3.9% hallucination rate from Vectara. That number came from an earlier HHEM evaluation used when Vectara compared DeepSeek V3 with DeepSeek R1 in January 2025. The current leaderboard uses HHEM-2.3 and a newer dataset, where DeepSeek V3 is listed at 6.1%, so direct comparisons across different time periods can be misleading.

What This Means for Users Choosing Between Models

The clinical study did not prove that GPT-4.1 is medically superior overall. The researchers were specifically evaluating whether clinical guidelines were correctly identified and cited, not measuring overall diagnostic reasoning quality, medical knowledge accuracy, treatment quality, or whether every clinical recommendation was correct. The study's scope was deliberately narrow, and its findings should not be overgeneralized.

What the research does show is that "safer" AI requires careful definition. A model with a lower hallucination rate on one benchmark might be less useful on another task. When evaluating AI tools for your specific use case, consider not just how often they fabricate information, but also how often they provide complete, accurate, and relevant answers to your actual questions. The most reliable model is the one that balances avoiding false information with actually delivering the information you need.