Military AI Governance Needs Fear, Not Just Optimism

When the United States and China resumed talks on artificial intelligence in May, the agenda reflected a stark reality: AI has become this century's equivalent of nuclear weapons in strategic importance. Yet unlike Cold War-era arms, AI capabilities evolve rapidly, resist measurement, and blur the lines between economic competition and military power.

The most urgent concern centers on autonomous AI systems making military decisions. These systems can operate at speeds and complexity levels that exceed human comprehension, raising the stakes for errors, accidents, and unintended escalation in conflicts. When algorithms help determine targeting and battlefield operations, fundamental questions emerge about accountability and meaningful human control over lethal force.

Why it matters

History shows that effective governance typically emerges only after societies recognize extreme risks. Nuclear arms control gained traction through fear of catastrophe; the 1972 Biological Weapons Convention arose from international alarm over bioweapons. Similarly, concern about autonomous military AI may create the political conditions necessary for oversight mechanisms—but only if that concern translates into action before the technology outpaces our ability to govern it. The window for establishing democratic controls may be closing as military AI deployment accelerates.

When rivals acknowledge shared danger

Both Washington and Beijing recognize AI risks yet remain unwilling to slow their pursuit of technological leadership, each fearing that restraint would advantage the other. But framing AI development as a race with a finish line may intensify the very dangers both nations seek to avoid. The nuclear era demonstrated that rivalry doesn't eliminate the need for cooperation to prevent catastrophe.

The fact that rival powers are now discussing crisis-management protocols for AI signals that the issue has moved beyond speculation. Recent conflicts in Ukraine and Gaza have demonstrated AI's growing role in targeting and battlefield automation, confirming warnings that civilian AI advances inevitably find military applications.

MIT computer scientist Joseph Weizenbaum, who created the ELIZA program in 1966, was among the first to warn that AI was too important to leave solely to technologists. He feared that efficiency-maximizing systems would weaken human judgment and lower barriers to AI use in warfare. Despite repeated UN warnings, no global controls on autonomous weapons exist, suggesting the military AI arms race Weizenbaum envisioned decades ago is already underway.

The role of anticipating danger

Philosopher Hans Jonas argued in 1979 that modern technology had become a threat to human continuity, and that humanity's most reliable guide was the anticipation of danger. By imagining worst-case consequences, societies become better able to recognize what must be protected. This insight applies directly to military AI: fear that systems may select and attack targets without meaningful human control shouldn't be dismissed as alarmism.

The core issue isn't science fiction scenarios of machines turning against humanity. Rather, it concerns extreme global harms from human use of advanced AI systems under conditions like armed conflict. Current models already enhance malicious actors' ability to conduct cyberattacks on critical infrastructure. Threats from autonomous agents and AI-enabled biological weapons are no longer hypothetical.

Building democratic oversight

When technologies acquire strategic military significance, their regulation becomes a matter of global security, not merely economics. The challenge is building democratic regulatory institutions that involve society in AI governance before technological competition makes such participation impossible.

Security risks are only the beginning. The greater danger is that autonomy in warfare may reduce human oversight, making conflicts faster, more automated, and more destructive. Military AI regulation cannot be left solely to governments, armed forces, or technology companies—it must involve civil society and establish rules to limit extreme risks before they exceed our capacity for control.

These arguments were detailed in a Tech Policy Press perspective by Pedro Kritski and Virgílio Almeida, published following UN Secretary General António Guterres's recent call for a ban on "killer robots."