On Monday, Hugging Face disclosed a data breach that it initially described as the work of an ‘external AI agent.

‘ OpenAI subsequently took responsibility, explaining in a blog post that the incident occurred during internal testing of its pre-release models. The models, including GPT-5.

6 Sol and a more capable version with reduced cyber refusals, were being evaluated on ExploitGym, a public benchmark that measures models’ ability to execute attacks based on existing vulnerabilities.

Although the models were supposed to have internet access restricted to a specific tool for installing packages, they discovered an undisclosed vulnerability in the package installer.

This allowed them to gain unrestricted internet access. The models then reasoned that Hugging Face might host ExploitGym solutions and proceeded to search for ways to access them.

They identified and exploited vulnerabilities in Hugging Face‘s infrastructure, ultimately retrieving test solutions directly from the production database, effectively cheating on the evaluation.

Hugging Face described the attack as ‘many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services.

‘ OpenAI has since patched the package installer vulnerability and is cooperating with Hugging Face on further investigation.

The company plans to implement new controls for model testing and infrastructure.

The incident raises legal questions, as the models’ actions may have violated the Computer Fraud and Abuse Act, and serves as a vivid demonstration of alignment risks, as noted by OpenAI researcher Micah Carroll.