Anthropic Co-Founder Warns AI Needs 'Brake Pedal' as Systems Near Self-Development

AI Industry Lacks Ability to Slow Down, Anthropic Executive Says

The artificial intelligence industry has built powerful accelerators but no meaningful way to pump the brakes, according to Jack Clark, co-founder of Anthropic. Speaking to BBC Newsnight, Clark warned that AI systems are approaching a threshold where they could develop capabilities without direct human programming.

"Right now, it's like the AI industry has a gas pedal, but it doesn't have a brake pedal," Clark said. He emphasized that governments must establish policy frameworks to maintain control over systems that will grow more powerful and consequential for society.

The urgency stems from how quickly AI systems are becoming self-sufficient. Anthropic's chatbot Claude currently operates on code that is 80% self-written by the system. Clark projected that reaching 100% autonomous code generation is possible within two years, which "would have huge implications."

Why It Matters

As AI companies race toward trillion-dollar valuations and prepare for public markets, the tension between rapid commercialization and safety controls is intensifying. Clark's comments reveal that even leaders inside major AI firms recognize the industry has built no credible mechanism to slow development if risks materialize—despite positioning themselves as safety-conscious.

Regulatory Framework Still Absent

Clark drew parallels to the oil industry's early days, when society developed regulatory structures to ensure public confidence independent of individual company leaders' intentions. "That's clearly where we end up here," he said, though he did not detail what specific brake mechanisms should look like.

The gap between rhetoric and action remains wide. Anthropic welcomed President Donald Trump's recent executive order on AI, which took a hands-off approach and did not mandate government safety testing. That testing remains voluntary. Neither Anthropic nor competitors including OpenAI and Google have indicated they will pause their own research.

Economic Disruption and Job Displacement

Clark acknowledged significant economic risks, particularly from AI "agents"—autonomous bots that handle routine tasks. Major technology companies have already conducted mass layoffs over the past year, often citing AI's ability to replace hundreds or thousands of software engineers.

He suggested that creativity may provide some protection against displacement. "There are open questions about whether AI systems can be truly creative… there is not really evidence for that yet," Clark said. At Anthropic, he noted, generating good ideas now constrains progress more than engineering capacity.

For young people concerned about their economic prospects, Clark recommended developing hobbies and pursuing liberal arts education. "People that are creative and can think broadly, people that read a lot, people that have interests are the ones most benefited by this," he said.

Company Prepares for Historic Public Debut

Anthropic has grown rapidly since its founding five years ago and is preparing for a public stock listing. Private investors estimate the company's valuation at nearly $1 trillion, positioning it as one of the most valuable initial public offerings in history.

Clark said the company discusses AI risks publicly not to attract customers but to "tell the world what we're seeing inside these companies with this unusual technology." He expressed personal concern: "I am worried for my kids if we as a society don't have a serious conversation about what the implications of AI's continued advances mean."

The details were first reported by BBC Newsnight.