Generative AI in Biology Risks Creating Nonexistent Discoveries

The hallucination problem in biological AI

Generative AI systems are expanding beyond text and images into biological research, where they design proteins, simulate cells, and generate synthetic data to fill experimental gaps. But these same systems can hallucinate—producing plausible-looking outputs that don't reflect underlying biology—and the consequences extend far beyond a chatbot's factual error.

Computational biologist Thomas Burger of Grenoble Alpes University examines this risk across ten potential applications of generative AI in biological research, according to findings first reported by ScienceAlert. The central concern: AI could convince researchers that a biological effect exists when it does not, or obscure genuine effects beneath distorted data.

Why it matters

Biological discoveries drive drug development, disease treatment, and our fundamental understanding of life. If AI-generated fabrications enter the evidence base as accepted findings, researchers could pursue ineffective treatments, overlook working therapies, or build entire research programs on nonexistent mechanisms. The risk scales with how directly AI outputs are treated as evidence rather than hypotheses to test.

Risk levels depend on how outputs are used

Burger identifies a critical distinction between AI applications based on how their outputs function in the research process.

Lower-risk applications include screening potential drugs or proteins. Here, AI rapidly evaluates candidates and selects a subset for laboratory testing. Mistakes waste time and resources—a promising candidate might be discarded, or an ineffective one investigated—but nothing becomes accepted as discovery without experimental validation.

The danger escalates when AI-generated synthetic data begins replacing actual experimental measurements. Researchers might use synthetic biological data to fill missing measurements, protect patient privacy, create control groups, or reduce animal testing. But if AI inserts features that were never present in the original biology, scientists could mistake fabricated patterns for genuine discoveries.

The corruption problem

"Most of the time, the problem is not about comparing a hallucination and a genuine biological discovery side by side," Burger told ScienceAlert. The issue is subtler: real data becoming corrupted by hallucinations during complex computational workflows that transform raw signals into biological descriptions.

Omics experiments generate massive datasets measuring genes, proteins, and other molecules. As AI processes this genuine data, it could alter signals in ways difficult to detect. The distortion affects researchers' conclusions without creating an obviously fabricated finding. Unless investigators deeply understand how the AI processed their data, they may not notice the corruption.

AlphaFold 3 provides a real-world example. Developers reported in Nature that the model could generate "hallucinated structures" in disordered protein regions, though low confidence scores can alert researchers to the problem.

Verification remains essential

AI errors don't always fabricate nonexistent effects. Models might add so much distortion that researchers overlook genuine effects, potentially missing evidence that a treatment works.

Burger acknowledges that hallucinations could theoretically lead to real discoveries through serendipity, similar to unexpected findings from laboratory errors. What matters is treating AI outputs appropriately: as ideas requiring experimental validation, not as observations that constitute evidence themselves.

No AI-proposed result qualifies as discovery until independently verified through real experiments.

These findings were detailed in an opinion article published in Patterns and first reported by ScienceAlert.