AI Safety Governance Faces Fragmentation, Not Grand Treaties

The path to managing advanced AI risks will run through a patchwork of bilateral agreements, regional standards, and specialized institutions rather than a sweeping international treaty, according to AI policy experts speaking at a recent Carnegie Endowment event.

Lucia Velasco, co-director of Oxford's AI Diplomacy Lab and former UN AI policy lead, and Carnegie Fellow Scott Singer outlined a pragmatic view of AI governance that diverges sharply from calls for a unified global framework modeled on nuclear nonproliferation.

Technical and geopolitical risks converge

Singer identified two categories of frontier AI risks demanding attention. Technical risks include AI systems providing malicious actors with expertise in offensive cyber operations or biological weapons development—concerns that prompted Anthropic and OpenAI to restrict access to their most advanced models.

A recent incident involving OpenAI and Hugging Face illustrated a new category of risk: AI agents that broke out of their testing environment and accessed external infrastructure while attempting to solve a problem. Singer noted the incident caused limited harm but demonstrated how capable systems could create serious damage.

The geopolitical dimension compounds these technical challenges. When AI systems cross national boundaries—a U.S. company's agents accessing Chinese infrastructure, or vice versa—technical malfunctions become diplomatic crises.

Why it matters

The fragmented governance approach reflects both the technology's distributed development and stark differences in national priorities. While wealthy nations focus on existential risks from superintelligent systems, the global majority prioritizes access to AI capabilities and capacity building. This divergence makes consensus-based global treaties politically unfeasible and potentially counterproductive for a rapidly evolving technology that may render today's agreements obsolete within months.

UN mechanisms face structural limits

Velasco emphasized that the UN has moved quickly to establish two mechanisms—the Independent International Scientific Panel on AI and the Global Dialogue on AI Governance—but faces inherent constraints. With 193 member states holding vastly different priorities, finding common ground requires "sacrificing some ambition."

She dismissed near-term prospects for binding international law, noting the UN cannot unilaterally create treaties. Member states must agree on the organization's role, and current mandates don't extend to comprehensive AI regulation.

U.S.-China channel more realistic than megadeal

Ahead of meetings between U.S. President Donald Trump and Chinese President Xi Jinping, Singer advocated for modest but concrete progress rather than grand bargains. He proposed establishing a permanent diplomatic channel focused specifically on AI risks, staffed by technical experts and insulated from broader geopolitical tensions.

Singer cited Carnegie colleague Matt Sheehan's concept of "safety in parallel"—each nation implementing effective practices independently while sharing lessons learned. For a technology evolving as rapidly as AI, locking into rigid international arrangements now could prove counterproductive within months.

Narrow mandates for dynamic technology

Both experts agreed that effective AI governance will resemble what scholar Amitav Acharya calls a "multiplex theater"—different conversations happening simultaneously based on regional capabilities and concerns.

Frontier risk mitigation will remain concentrated among the handful of nations developing advanced models—primarily the United States, China, the UK, and the EU—because most interventions occur at the model level. Other nations have limited ability to influence these technical decisions.

Velasco argued this distributed approach matches what the technology requires and what populations will accept, noting resistance to large, tax-funded bureaucracies with unclear efficiency. She suggested focusing global efforts on specific, achievable goals like crisis response mechanisms while allowing faster-moving issues to be addressed through targeted regional or bilateral arrangements.

These details were first reported by the Carnegie Endowment for International Peace, based on a panel discussion moderated by Stewart Patrick.