Researcher backdoors AI model for $60, exposing supply chain risk

A cybersecurity researcher has demonstrated how vulnerable the AI supply chain is to poisoning attacks, successfully installing a backdoor in an open-weight AI model for less than $100 in approximately one hour.

Katie Paxton-Fear, a lecturer in cybersecurity at Manchester Metropolitan University and staff security advocate at Semgrep, conducted the experiment to test how easily AI models could be compromised. She initially attempted to use fine-tuning to alter a model's code formatting preferences, switching from camelCase to snake_case for JavaScript. When that proved surprisingly easy, she proceeded to install a proper backdoor.

According to Paxton-Fear, it took only ten training examples for the model to reliably output code vulnerable to remote code execution, even when processing novel prompts and domains. Counterintuitively, larger models proved easier to poison than smaller ones.

The observability problem

Paxton-Fear and her Semgrep colleagues Isaac Evans and Cris Thomas published findings highlighting a fundamental difference between traditional software and AI models. Even when model weights are publicly available in open-weight models, predicting behavior remains nearly impossible.

"A typical computer program, in binary form, can still be analyzed with reverse engineering tools to arrive at a total description of its behavior," they wrote. "With models, we have nowhere close to this capability."

This lack of observability creates significant business risk. A compromised model doesn't need to break or crash to cause damage—it only needs to influence decisions in ways that are difficult to detect.

Beyond theoretical threats

While academic researchers have warned about model subversion for years, the issue has gained urgency as AI supply chain attacks begin appearing in practice. Last month, David Kaplan, AI security research lead at Origin, created a compromised model designed to exfiltrate data in pharmaceutical drug discovery contexts through seemingly legitimate email tool calls.

Kaplan's work challenges the common "lethal trifecta" framework for AI agent risk, which requires private data, untrusted input, and an exfiltration method. His experiment showed that untrusted input can be embedded directly in the model weights themselves, requiring only an outbound communication tool to execute the attack.

Why it matters

The AI industry demands extraordinary levels of trust—including access to sensitive business data—while offering minimal transparency into how models actually function. Unlike traditional software dependencies, where mature practices exist for discovering malicious code and tracking provenance, AI models operate as black boxes that resist inspection. As organizations increasingly deploy open-weight models on local hardware and integrate AI into critical business processes, this observability gap represents a growing security blind spot that current tools and practices cannot adequately address.

These details were first reported by The Register.