Pope's AI Advisor Calls Big Tech Labs a 'Cartel' on Safety Rules

The Vatican's technology advisor is challenging the narrative coming from leading artificial intelligence companies, arguing that their warnings about superintelligence serve to consolidate control rather than protect the public.

Father Paolo Benanti, a priest who counsels the Catholic Church on AI matters, told WIRED that when a small group of companies dictates industry rules, "this has a really clear name: a cartel." His comments come as labs including OpenAI and Anthropic have issued increasingly dire warnings about AI capabilities potentially surpassing human control.

The Control Problem Behind the Safety Debate

Benanti contends that by framing AI development as so complex that only they possess the expertise to manage it, major labs effectively exclude broader society from crucial decisions about the technology's future. "Ethics cannot be something that belongs to the board of trustees of a company," he said. "It's something that has to be balanced, mitigated in debate in the public square."

The priest takes particular issue with proposals from lab leaders like Anthropic CEO Dario Amodei, who has called for mandatory third-party evaluations and restrictions on techniques like distillation—where competitors train models on outputs from proprietary systems. Rather than limiting development methods, which Benanti fears could stifle innovation, he advocates for standards ensuring humans maintain control over AI systems.

Economic Incentives and 'Divine' Framing

Benanti questions whether grand claims about AI capabilities reflect genuine technical assessment or serve overlapping economic and political interests. Both OpenAI and Anthropic are preparing for public offerings worth billions, while US-China rivalry adds geopolitical complexity to the development race.

He also criticizes how some industry figures discuss superintelligence with religious reverence—citing OpenAI CEO Sam Altman's aspiration to build "magic intelligence in the sky." This framing, Benanti argues, obscures what AI actually is: statistical computation, not sentient intelligence. "We are not building the next divinity," he said.

Why It Matters

The debate over who sets AI safety standards has immediate regulatory implications. If major labs successfully position themselves as the only entities qualified to govern AI development, they gain extraordinary influence over rules that will shape a technology affecting billions. Benanti's intervention represents a significant institutional voice—the Catholic Church—arguing for democratic oversight rather than industry self-regulation. His perspective also highlights how existential risk narratives, whatever their technical merit, can function strategically to centralize decision-making power.

The Vatican's Position

In May, Pope Leo XIV issued his first encyclical on artificial intelligence, warning against "technocratic logic" that treats "efficiency as the ultimate measure of value." The letter called for international collaboration and cautioned against letting "the desire to secure geopolitical or commercial dominance" drive development.

Benanti, who advises the Church as a scholar of philosophy, theology, and technology rather than as a formal Holy See representative, frames AI's emergence as creating "a new role for religions in public debate" by raising fundamental questions about human agency and technological control.

"The first thing is, we need to have brakes, and a wheel," Benanti said. "That means a significant position of control for human beings over the machine."

These details were first reported by WIRED.