Billions of dollars are being spent on AI for drug discovery, yet few new therapies have emerged from these efforts because the biological data used to train AI models is unreliable, poorly documented, and often contaminated. The root problem isn't the AI itself; it's that researchers fundamentally distrust the datasets their models learn from, creating a reproducibility crisis that undermines the entire enterprise.

Why Don't Scientists Trust Their Own Data?

Nearly three in four biomedical researchers believe the field is gripped by a reproducibility crisis. This skepticism isn't paranoia. Consider a concrete example: a 2021 analysis found that two commonly used cell lines, HEp-2 and INT 407, were actually contaminated with HeLa cells. These false cell lines appeared in nearly 10,000 published articles. Assuming an average of five citations per article, more than $4.9 billion may have been spent supporting research based on these two unauthenticated cell lines, with costs potentially reaching $14.8 billion under a more inclusive estimate.

The problem extends beyond contamination. Cell line genomes are often unstable, and when researchers passage cells (grow new generations from existing ones), mutations and genomic rearrangements occur. Researchers attempting to replicate studies using their own copies of the same cell lines may not realize that the digital sequence data can diverge significantly from the actual cell line tested.

How Does Missing Metadata Make Things Worse?

The data quality problem is amplified by incomplete documentation. A 2021 analysis published in Clinical Infectious Diseases found that more than a quarter of foodborne microbiological samples in public sequence databases were missing key metadata attributes. Without standardized, complete metadata, researchers cannot reliably compare datasets across studies, assess methods, or verify that two experiments actually tested the same thing.

This matters enormously in drug development. R&D costs now exceed $3.5 billion per novel drug, reflecting a five-decade decline in pharmaceutical R&D efficiency. When AI models are trained on datasets with inconsistent or missing metadata, they risk encoding and propagating the very inconsistencies researchers are trying to overcome.

"AI models are only as reliable as the biological data they are trained on, making authenticated, standardized datasets the foundation of AI-driven discovery," stated Patrick Boyle, PhD, Interim Chief Scientific Officer at ATCC.

Patrick Boyle, PhD, Interim Chief Scientific Officer at ATCC

What Makes Protein Structure Prediction Different?

There is one notable exception to this data crisis: protein structure prediction models like AlphaFold have succeeded despite the broader reproducibility problem. The reason is the Protein Data Bank (PDB), which is not only highly organized but contains highly reliable data. Most other types of training data in biology, particularly genomics and molecular data, are difficult to replicate without access to identical physical starting materials, equipment, and methods.

Even if researchers adhered to new standards for data collection, it could take decades at current rates to populate new databases with datasets approaching the quality of the PDB. This timeline mismatch creates an urgent problem: AI models for drug discovery cannot wait decades for better data infrastructure.

Steps to Build Trustworthy Biological Datasets for AI

• Authenticate Physical Materials: Every dataset must be anchored to authenticated biological materials, not just digital sequences. Researchers should demand that biobanks and curated repositories improve interoperability so that the same cell line or microbial strain produces consistent results across different organizations.
• Standardize Metadata Requirements: Datasets must include complete, standardized metadata that answers three fundamental questions: Where did this data come from? How was it generated and validated? Can it be traced back to a known, authenticated biological source? These cannot be mere checkboxes but must serve as conditions under which data can be trusted.
• Build Interconnected Data Infrastructure: Organizations that develop and maintain authenticated biological reference collections should form an interconnected ecosystem critical to establishing practical standards for material and data sharing. This makes it easier to collaborate across organizations and ensures that multiple datasets published on the same biological material can be reliably compared.

The infrastructure currently built to serve information and data to scientists doesn't provide what AI models actually need: well-curated, labeled data that is consistent across studies. Raw data, the software pipelines used to generate it, and the detailed methods needed to reproduce it are often buried in supplemental materials, if included at all.

Could AI Actually Fix Biology's Data Problem?

Rather than slowing down AI model development, experts argue that AI itself could be leveraged to plan and generate high-quality datasets. The rise of AI represents the single biggest opportunity to fix biology's reproducibility problem by improving data practices to make better models possible.

Imagine being able to start work at two different contract research organizations or cloud labs with the same cell line without shipping cells to each provider, or downloading a protocol for a favorite microbial strain and having it work on the first try. If multiple datasets are published on the same cell line, researchers should trust that they actually describe the same biological material.

The National Institutes of Health (NIH) has taken notice of this crisis, launching an initiative to elevate replication and reproducibility as foundational to "gold standard science." Until biological data infrastructure improves, however, the productivity gap between AI investment and actual drug discovery will persist. The bottleneck isn't computing power or algorithmic innovation; it's the trustworthiness of the data flowing into these systems.