AI Bioweapon Risk Divides Experts as Models Gain Capability

Diverging views on AI-enabled biological threats

Artificial intelligence executives are publicly warning that their technology could help terrorists develop biological weapons, but biosecurity researchers remain divided on how seriously to take the threat.

Anthropic CEO Dario Amodei wrote earlier this year that while AI-assisted bioweapons use may not be imminent, "added up across millions of people and a few years of time, I think there is a serious risk of a major attack." In June, Amodei joined OpenAI CEO Sam Altman and dozens of other tech leaders in an open letter warning that AI's growing capabilities could "meaningfully erode" the knowledge barriers that have historically prevented bad actors from obtaining biological weapons.

Yet some researchers who directly test AI models see far lower risk. Allison Berke, a bioengineer at RAND who evaluates whether models can provide step-by-step pathogen recreation instructions, said her probability of AI causing a biological crisis "is particularly low among people who work on this."

What AI can actually do today

Stanford professor of medicine David Relman, an infectious disease and biosecurity expert on the Bulletin's Science and Security Board, tested an unrestricted AI model without public safety guardrails. For technical biological questions, he found the output "helpful, but not revolutionary" — mainly useful to people already familiar with the relevant biology.

The attack-planning capabilities proved more concerning. When Relman asked for a plan that would maximize chances of success, the model provided detailed advice on evading law enforcement detection and creating misdirection. "I didn't ask the model to do all these things," Relman said. "I said I want a clever, thoughtful plan that would maximize the chances of success. This is what it did."

Public models include guardrails designed to refuse dangerous requests, but these protections can be circumvented through "jailbreaking" techniques. "There is a world of experience and literature on how to jailbreak these models," Relman said. "It's absolutely doable."

Technical and historical barriers remain

Berke's skepticism stems partly from technical limitations in biological automation. Cloud labs where robots execute scientific instructions are "not yet at the stage where they could fully autonomously make a transmissible virus," she said. The situation might change if inexpensive benchtop synthesizers became widely available, truly "democratizing availability."

Historical precedent also suggests bioweapons may not appeal to terrorists. Between 1970 and 2019, the Global Terrorism Database recorded over 200,000 terrorist attacks; just a few dozen involved biological agents. Even the Japanese cult Aum Shinrikyo, which attempted multiple bioweapons attacks with botulinum toxin and anthrax, caused no casualties with biological agents before turning to chemical weapons for their deadly 1995 Tokyo subway attack.

The 189 countries party to the Biological Weapons Convention have renounced these weapons partly because they suffer serious military drawbacks: environmental conditions affect attacks, contagious agents risk blowback, lengthy incubation periods limit utility, and pathogens may persist and mutate unpredictably.

Why it matters

The disagreement over AI bioweapon risk reflects a broader challenge in evaluating emerging technology threats. OpenAI now classifies biological risk as "high capability" under its evaluation framework, meaning models could "uplift their knowledge and potentially produce a novice level threat." The company provides pre-release models to the Department of Commerce's Center for AI Standards and Innovation for testing, though some officials want mandatory rather than voluntary evaluation.

Relman argues that relying on historical patterns is dangerous given both improving AI technology and what he sees as degrading social norms. "Where we will find ourselves a year from now is likely to be, without intervention, a much more risky circumstance," he said, advocating for regulations restricting certain models' data access and requiring user vetting in some cases.

These details were first reported by the Bulletin of the Atomic Scientists.