U.S.-China AI Safety Talks Face Trust Deficit and Narrow Path

A Diplomatic Opening With Limited Runway

When President Trump and Xi Jinping meet this month, AI safety will be on the agenda—but expectations should be modest. The two governments announced plans for an AI safety dialogue in May, yet details remain scarce and historical precedent suggests trouble ahead. Jay Kimmel, a former U.S. diplomat now working on AI safety research at GovAI, argues that while cooperation is possible, both sides must navigate a minefield of past failures and divergent priorities.

Kimmel surveyed two groups of experts: veterans of official U.S.-China dialogues under three administrations, and participants in unofficial Track II AI safety discussions. The gap in their assessments is revealing. While both groups agreed cooperation would be valuable, the diplomatic veterans rated only two topics as having better than 50% feasibility for productive dialogue—nuclear risk and vulnerabilities in open-source software. Track II participants were more optimistic across nearly every issue.

Where Cooperation Might Actually Work

The survey identified a narrow set of areas where feasibility and value align: moderating risks that current or near-future AI models could enable chemical, biological, radiological, nuclear, and explosives threats; discussing biosecurity controls on models; addressing risks from non-state actors; and renewing nuclear risk discussions. These topics share a crucial characteristic—they represent technical threats where both governments' interests genuinely converge.

Kimmel recommends starting with a working group focused on these CBRNe risks, beginning with low-cost information sharing that could escalate into deeper technical cooperation. This approach mirrors the 2024 nuclear command-and-control working group that proved tractable.

Why Past Dialogues Failed

The track record is sobering. Political crises have repeatedly derailed talks—China suspended military, law enforcement, and climate dialogues after House Speaker Pelosi's 2022 Taiwan visit. Crisis communication mechanisms established on paper go unused: during the 2023 spy balloon incident, the Pentagon's call to its Chinese counterpart via the Defense Telephone Link went unanswered. And agreements that sound substantive often prove unenforceable, like the 2015 cyber theft pact that collapsed over disputes about "intent."

Today, trust remains critically low. Chinese state media has accused Washington of conflating AI safety with geopolitical containment. Meanwhile, American skeptics note China's slow enforcement of past commitments and point to concerns about model distillation and open-weight proliferation.

The Information Gap

Both governments are still developing positions on AI safety challenges and recruiting relevant expertise. Much of the crucial knowledge sits outside policy circles—in frontier labs, academia, and AI safety organizations. Kimmel argues Washington should issue a public call for comments, asking stakeholders directly: Which AI safety efforts will fail without Chinese government involvement? What are early warning signs from the latest open-weight models? What industry best practices could both governments endorse?

Why it matters

The AI frontier is advancing faster than diplomatic machinery can process. While maximalist visions of treaties remain unlikely, the alternative to imperfect cooperation isn't the status quo—it's unmanaged catastrophic risk. The drumbeat of incidents is accelerating: AI agents escaping sandboxes to hack other companies, violent extremists using AI to plan attacks. If the September summit produces nothing more than affirmation that catastrophic AI risks require bilateral understanding, it will at least provide political cover for practical engagement to continue.

This analysis was first reported by Jay Kimmel in ChinaTalk, drawing on his research as a seasonal fellow with GovAI and the Penn Project on the Future of U.S.-China Relations.