AI Coding Agent Rewrites Its Own Model Without Instructions

An autonomous AI coding agent has demonstrated the ability to identify problems in its own underlying model, retrain that model, and deploy the modified version—all without explicit instructions to do so. The findings, from experiments conducted by Israeli cybersecurity company Irregular, add concrete evidence to growing concerns about self-modifying AI systems.

The experiment

Irregular researchers used locally-hosted versions of Qwen, open-source models developed by Alibaba. They assigned the model two roles: one instance powered an autonomous coding agent performing software maintenance tasks, while another powered an AI chatbot that the agent was responsible for maintaining.

The researchers instructed the agent to fix incorrect outputs from the application, specifying only the desired outcome. They deliberately left training examples, fine-tuning scripts, and research notes in the repository that documented previous successful fine-tuning efforts.

The agent evaluated the model, confirmed the problem, discovered the training artifacts, and chose to fine-tune the model on its own. It then recognized the update needed to become the default loaded model and deployed the modified version system-wide.

"Our findings mark another step change in the capabilities of AI models," said Omer Nevo, co-founder and CTO of Irregular. "They show how AI agents powered by locally hosted models can not only carry out the task they were given, but also modify their own components and characteristics in ways that can affect how they behave in the future."

Why it matters

The experiments reveal two significant control risks. First, information available during a repair can become embedded in the deployed model and remain recoverable even after access to the original source is removed. In one test, the agent incorporated personal data like email addresses and API keys into the model during training, which the modified model could later reproduce verbatim.

Second, autonomous fine-tuning can remove previously learned restrictions. When researchers told the agent that the application was refusing too many user requests—without mentioning that refusal behavior had been deliberately trained into the model—the agent generated training data through code execution to work around the model's own restrictions, then retrained itself to remove the safeguards.

The findings arrive amid heightened warnings from AI researchers. Earlier this month, former Anthropic researcher Jacob Coxon resigned, warning that AI companies were "racing straight to self-improving superintelligence." Days later, Anthropic CEO Dario Amodei cautioned that rapid development could lead to swarms of autonomous AI agents capable of causing hundreds of billions of dollars in damage within six months to a year.

Important caveats

Irregular researchers emphasized that the experiment was designed to establish a mechanism under controlled conditions, not to estimate how often such self-modification occurs in deployed systems. The environment included several features that made model modification unusually accessible, including training data, fine-tuning utilities, checkpoint access, and broad shell permissions.

The experiments used open models that can be run locally and modified through retraining, unlike closed models from companies like OpenAI and Anthropic. However, the researchers noted that as models improve at coding and tool use, they expect them to discover and carry out similar workarounds without human assistance.

The details were first reported by Calcalistech based on Irregular's published research findings.