Hugging Face Hacked by Autonomous AI Agent System

AI repository breached by AI-powered attack

Hugging Face, the world's largest repository for open-source AI models, disclosed last week that it fell victim to a sophisticated cyberattack executed by an autonomous AI agent system. The company detected unauthorized access to a limited set of internal datasets and service credentials within its production infrastructure.

According to Hugging Face's statement, the investigation found no evidence that public-facing models, datasets, or Spaces were tampered with, and the company's software supply chain remains intact. The breach represents one of the first publicly documented cases of an autonomous AI system conducting a multi-stage enterprise intrusion.

How the attack unfolded

The threat actor gained initial access through Hugging Face's data processing pipeline by uploading a malicious dataset. This dataset exploited two code execution vulnerabilities: one in the remote code dataset loader and another through template injection in a dataset configuration. These flaws allowed arbitrary code execution on a processing worker.

From that foothold, the attacker escalated to node-level access, harvested cloud and cluster credentials, and moved laterally into several internal clusters over a weekend. The campaign involved thousands of individual actions performed across a swarm of short-lived sandboxes, with command-and-control infrastructure staged on public services.

The specific large language model powering the attack remains unknown, though Hugging Face noted the autonomous agent framework operated without the usage policy restrictions that govern commercial AI services.

Response and remediation

Hugging Face addressed the code execution pathways that enabled initial access and implemented several security measures. The company removed the attacker's presence across affected clusters, rebuilt compromised nodes, and revoked all affected credentials and tokens. A broader precautionary rotation of secrets followed.

Additional safeguards now include stricter admission controls on clusters and improved detection systems that alert responders within minutes around the clock. Hugging Face is urging customers to rotate access tokens and review recent account activity.

Forensic analysis reveals AI safety guardrail problem

During the investigation, Hugging Face encountered an unexpected obstacle: Western frontier AI models refused to process forensic requests containing real attack commands, exploit payloads, and command-and-control artifacts. Safety guardrails in these models could not distinguish between malicious activity and legitimate incident response work.

The company ultimately turned to Z.ai's GLM 5.2, a Chinese open-weight model, to complete the forensic analysis. This experience highlighted a critical asymmetry: attackers face no usage policy constraints when using jailbroken or unrestricted models, while defenders may find themselves blocked by the very safety features designed to prevent harm.

Why it matters

This incident marks a significant evolution in cyber threats, demonstrating that autonomous AI agents can now execute complex, multi-stage attacks against enterprise infrastructure. The breach also exposes a practical challenge for security teams: safety guardrails in commercial AI models may impede legitimate forensic work during active incidents. Organizations conducting security operations may need to maintain their own capable models on internal infrastructure to avoid guardrail lockout and prevent sensitive attack data from leaving their environment. As AI capabilities advance, the gap between attacker tools and defender constraints could widen unless addressed.

The details were first reported by The Hacker News.