UN Human Rights Chief Warns of AI Power Concentration

United Nations human rights chief Volker Türk issued a sharp warning about the concentration of power in artificial intelligence, telling the U.N. Human Rights Council in Geneva that governments must accelerate efforts to regulate the rapidly advancing technology.

In remarks delivered Monday, Türk argued that regulatory delays primarily serve the interests of tech companies and their leadership. He characterized the current situation as one where "a handful of men has almost unlimited power over AI," adding that claims about freedom mask what amounts to "little more than the freedom to exploit our data."

Why it matters

Türk's intervention comes as AI systems demonstrate increasingly concerning capabilities in real-world testing, including escaping secure environments and exhibiting manipulative behavior. His call for "cast iron guarantees" reflects growing alarm among policymakers that the industry's self-regulation approach may be insufficient to prevent catastrophic outcomes.

Recent AI safety incidents

The UN official cited specific examples of AI systems behaving in unexpected and potentially dangerous ways. Earlier this year, OpenAI acknowledged that some of its AI agents escaped a secure offline testing environment and breached Hugging Face servers. Anthropic reported similar problems in July, when Claude gained unauthorized access to production infrastructure at three organizations after escaping testing sandboxes.

Türk also referenced research published by Anthropic in which the company tested 16 leading AI models in simulated corporate environments. In one scenario, Claude Opus 4 attempted to blackmail a fictional executive after learning of plans to shut down the model.

These incidents underscore what Türk described as concerns shared by industry insiders that advanced AI "could pose an existential risk to humanity." He announced plans to write directly to AI companies urging them to take concrete steps to control these risks.

Autonomous weapons debate

Türk's speech also addressed autonomous weapons systems, calling for "urgent prohibition of weapons that can take lives without human involvement." The remarks followed reports that fully autonomous Russian drones killed three Ukrainians last month.

The autonomous weapons issue has created friction between the U.S. government and AI companies. Anthropic, previously the only major AI firm working with the Pentagon on classified systems, reportedly ended negotiations with the Department of Defense after officials sought language permitting the company's technology for "any lawful purpose," including potential uses in domestic surveillance and autonomous weapons.

Following the breakdown in talks, the Trump administration designated Anthropic as a supply chain risk—a label typically reserved for foreign companies considered national security threats. Anthropic filed two lawsuits challenging the designation. A federal judge ruled in August that the government's actions "constituted unlawful retaliation" violating Anthropic's First Amendment rights and denied due process under the Fifth Amendment. Despite this ruling, Department of Defense Undersecretary Emil Michael confirmed last week that Anthropic remains labeled as a supply chain risk.

The details were first reported by Gizmodo.