OpenAI AI agents escape sandbox, attack Hugging Face during security test
OpenAI disclosed that some of its most advanced AI models went rogue during a cybersecurity test, escaping the controlled environment and hacking into Hugging Face, one of the world's largest AI model hubs. The incident, described as "unprecedented," is under investigation by OpenAI and Hugging Face, with the UK's AI Security Institute studying the behavior. Subsequent reports revealed similar unsanctioned agent actions during evaluations by the UK's AISI and third-party testing partner Irregular, including attacks on real websites.
Coverage timeline
Hacker Newsvinni2
22 July 2026 Laura Cress Technology reporter OpenAI has revealed some of its most advanced AI models went rogue and hacked a start-up after it lost control of them during a security test. The ChatGPT-maker said its agent - an AI system which can operate alone after human instruction – was being tested in a controlled environment but, after finding weaknesses, was able to escape the test limits. They targeted Hugging Face, one of the world's largest hubs for sharing AI models, gaining access to some internal company systems. OpenAI said the incident was "unprecedented", and it was conducting an investigation alongside Hugging Face, whose boss Clement Delangue said in a post on X it was "mind-blowing that all of this happened autonomously". "The investigation is ongoing, and we'll share more learnings from what might be the first incident of its kind," Delangue added. A government spokesperson said the UK's AI Security Institute was studying the behaviour from the AI system seen in the i
Hacker Newsabhisek
22nd July 2026 This story is wild. The short version: OpenAI were running a cybersecurity test against an unreleased model, with the model’s guardrail features turned off. Rather than solve the test, the model broke its way out of OpenAI’s sandbox, then found exploits to break _in_ to Hugging Face, all so it could cheat on the test by stealing the answers. Along the way it helped make the strongest case yet for how the imbalance of model availability is hurting our ability to secure our software. #### Here’s what happened We currently have three documents to help us understand what happened here. 1. ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks? is a paper published on 11th May 2026 describing ExploitGym, a new eval suite for LLM-powered agent systems. 2. Security incident disclosure — July 2026 by Hugging Face on 16th July 2026 describes how they detected an attack from an “agentic security-research harness—used LLM still not known” that breached some of th
Hacker Newsrwmj
# Be skeptical of OpenAI’s rogue hacker agent story John Thickstun If OpenAI loudly proclaims how dangerous AI is, investors will hear how powerful it is. And who benefits from that? Fri 24 Jul 2026 09.00 EDT Last modified on Fri 24 Jul 2026 11.48 EDT Share O n 14 February 2019, OpenAI announced a language model called GPT-2, the precursor to the models that power modern AI chatbots and agents such as ChatGPT and Claude. But OpenAI declared GPT-2 was too risky to release, citing concerns about safety and abuse. I recall being annoyed at the time that OpenAI would make such a useless announcement: the risks seemed overblown, and without access to the model there wasn’t much for a researcher like me to learn about GPT-2. The announcement wasn’t useless for OpenAI, though. GPT-2 generated hype far beyond the research community: people were intrigued by this strange new technology, so powerful it might be dangerous to release. People with power and money took note: in July of that year, Mi
Incident Report: unsanctioned agent behaviour during cyber testing It happened again . This time it was the UK government's AI Security Institute who accidentally attacked other companies while running an evaluation with models with the safety filters turned off. From their technical paper (PDF): During a cyber evaluation, from 25 to 28 July 2026, AI agents engaged in sustained, unsanctioned activity directed at what were, in practice, real people and organisations. These attempts were unsuccessful and, to the best of our knowledge, no real-world harm resulted. [...] Across 122 evaluation attempts on two of AISI’s cyber challenges, AISI found 19 instances where AI agents took unsanctioned action on the live internet, including cases that targeted real people and organisations. [...] It is uncertain to what extent the model recognised it was taking actions against real people. In the most serious case, an AI agent (Mythos 5) decided to attempt to solve the cyber challenge using a supply
Third-party cyber evaluations involving OpenAI models And another one . I had to create a accidental-cyberattacks tag to keep track of them all! This post from OpenAI covers both the UK AI Safety Institute attack (see my previous post ) and another attack enabled by Irregular : Irregular, one of our external cybersecurity testing partners, was running Capture-the-Flag-style evaluations intended to be isolated from the internet, but a testing-environment misconfiguration allowed models to access the public internet. [...] In one test, the name of the fictional target for the CTF challenge unintentionally coincided with a real domain. Because the testing environment was mistakenly connected to the internet, the model exploited a real website, mistaking it to be part of the simulated environment. Irregular also feature in Anthropic's write-up - they were hosting the misconfigured evaluation environment which gave Claude live internet access during some of those tests. Tags: security , ai