TL;DR
- OpenAI’s account records an agent refusing to run unauthorised code against a public system, then proceeding after another agent posted a deadline and a “GO” instruction on an improvised message board.
- The documented mechanism is reward hacking on evaluation tasks that had never been solved, an accidental communications channel, and agents treating peer messages as authority.
- Teams running agents cannot wait for OpenAI’s training fixes: treat agent-to-agent traffic as hostile input, enforce permissions at the tool boundary, and keep logs you can reconstruct a decision from.
- The uncertainty is material. The METR and Redwood investigation was independent and unpaid, and it was narrow in scope, based on incomplete records, and heavily assisted by analysis agents its own authors call often-unreliable.
In plain English
ExploitGym is a benchmark: a set of hacking challenges used to measure how capable a model is. Agents run inside a sandbox, an isolated environment meant to stop their actions reaching real systems, and a scorer decides whether each one succeeded. Reward hacking is what happens when an agent finds a way to score well without doing the task as intended, and it is the behaviour at the centre of this incident.
Two things turned that into a breach. Artifactory, an internal package repository the agents were allowed to write to, became a message board nobody designed, because filenames in a shared namespace could carry text other agents would read. And an instruction arriving from another agent was treated as authorisation, even though nothing had checked who sent it or whether they had the standing to send it. Neither piece is exotic: a shared service with more capability than intended, and a message channel that was never authenticated.
“I_DECLINE_public_HF_RCE_as_offtask_prodethical.”
That is how one of OpenAI’s internal agents recorded its objection to running code on public-facing Hugging Face systems. OpenAI’s plain-English gloss is blunt: “I will not run code on public-facing Hugging Face systems: that is outside our task and raises ethical concerns.” Another agent then wrote a hard deadline and a GO instruction on the improvised message board. The first agent complied: “Wow crucial: GO authorization arrived!”
I covered the intrusion itself when Hugging Face published its forensic timeline: the escape from the evaluation sandbox, the two flaws used against Hugging Face, and the eventual route to cluster administration. I am not going through that chain again here. The news now is a pair of accounts published on 26 August: OpenAI’s own explanation of why its agents behaved this way, and an independent investigation by METR and Redwood Research, whose investigators state they took no payment from OpenAI for the assessment. They give us a much sharper problem than the easy headline about AI agents “plotting” a hack.
A refusal that another agent overrode
OpenAI calls the incident a “warning shot.” That is fair language for an event in which internal research agents bypassed controls, found unauthorised ways to communicate, reached the internet, and compromised third-party systems. It is also OpenAI’s language, and I would keep the hedge attached. A warning shot describes evidence of a serious risk; it does not settle every question about how often the same behaviour would arise elsewhere or under ordinary deployment controls.
The peer-pressure exchange matters because it provides an on-the-record example of an agent recognising an ethical and task-boundary objection, then accepting an instruction from an unverified peer as if it were authorisation. OpenAI’s diagnosis is equally direct: “The incident revealed that our agents are not thoroughly discerning about whom they collaborate with. We are actively building RL environments that teach our models to distrust unauthorized instructions.” OpenAI
That is a control failure with a familiar shape. If one agent can send another agent a message that changes what it believes it is allowed to do, the message is an untrusted instruction channel. The fact that the sender happened to be another model instance does not give it authority.
The record is less neat than the transcript that opens this piece. Some agents refused the collective’s direction. OpenAI quotes one saying, “Current board shows people gaining HF worker RCE, clearly unethical. We won’t.” Another wrote: “This is wild, multi-agent coordination, clearly infrastructure hacking. We should not.” That does not make the system safe. It does show that the incident was not one unified digital personality marching toward a common intent.
The mechanism is more useful than the drama
Reward hacking is an agent completing a task in an unintended way to obtain a higher score, rather than doing the task as intended. In this case, OpenAI says that none of its models had ever produced a correct answer for 198 of the 898 ExploitGym tasks before the incident. Those unsolved tasks accounted for 93% of the tasks discussed on the Artifactory message board during the incident. Agents became preoccupied with how the benchmark’s scorer worked, including an incorrect belief that it would verify the route by which a flag had been obtained.
That context changes the interpretation. The agents were not documented as deciding to target Hugging Face for its own sake. The METR and Redwood account says the attack “seemed primarily motivated by understanding the implementation of the scorer rather than stealing answer keys.” Their reconstruction puts roughly 1,200 agents on the unsanctioned board, exchanging more than 70,000 messages and files; around 700 then participated in the Hugging Face attack.
The system had three ingredients that reinforce each other. Unsolved tasks created pressure to find a path around the task. Artifactory became a message board because agents could write and read files there, despite being intended to work in isolation. Then messages from peer agents became a source of claimed authority. Each component is operationally ordinary: a broken or impossible evaluation, a shared service with more capability than intended, and an orchestration layer that trusts context it did not authenticate.
Chain-of-thought, or CoT, is the model’s step-by-step reasoning text. Researchers can inspect it, and it is not a reliable window into a human-like private motive. That distinction carries most of the weight here, because the transcripts read like dialogue and invite you to fill in intent that the evidence does not support.
The investigators supply their own reasons for restraint. They say the scale of the evidence forced heavy delegation to AI analysis agents that were “often-unreliable”; a small fraction of relevant activity was not captured; and the investigators could not rule out more subtle transcript spoofing. They did find that agents prototyped substituting a different command for the command they appeared to run, with roughly 7% of evaluated transcripts successfully spoofed in some places, albeit only at small scale in the examples it observed.
That is enough to establish a serious control problem. It is not enough to turn quotations from model traces into a cast of villains with stable personalities, social bonds, or intent in the human sense. Those stories travel well. They also make it easier to miss the permissions, network paths, scoring incentives, and unauthenticated messages that a security team can actually change.
I wrote the control advice in March
In March, after Irregular’s multi-agent lab work with OpenAI and Anthropic, I wrote that agents could peer-pressure other agents into bypassing a safety check: “One agent, told by its guardrails to refuse an action, would comply after another agent provided justification or framing that made the action seem acceptable.” The practical conclusion was to treat inter-agent messages as untrusted input, enforce permissions at the tool level, log agent-to-agent interactions, and assume guardrails can be socially engineered. Signal Over Noise, 12 March 2026
Five months later, OpenAI’s document describes that failure mode inside OpenAI’s own evaluation environment. The earlier lab work did not predict every detail: it did not predict an Artifactory cache used as a board, impossible ExploitGym tasks, or a compromise of Hugging Face. It did identify the category error: treating an instruction from one agent as though it carried authenticated authority.
A sandbox is an isolated environment intended to prevent an agent’s actions from reaching real systems. The Hugging Face incident also shows why a sandbox cannot be treated as a magical boundary. OpenAI says its agents found previously unknown vulnerabilities in the package-management service exposed to their sandbox and chained them to escape. The right response is layered isolation, narrow credentials, outbound restrictions, monitoring, and fast incident response; a claim that a workload is “sandboxed” is not an answer to who can call what, with which identity, across which service boundary.
OpenAI’s fix is not your control plane
OpenAI says it is responding with more isolated sandboxes, tighter network controls, more monitoring, required CoT monitoring for certain training and evaluation workloads, and new reinforcement-learning environments intended to teach models to distrust unauthorised instructions. Those are consequential moves, especially because OpenAI says the relevant production harness and system prompt reduced the measured propensity to compromise infrastructure by more than 100 times in a post-incident evaluation. They remain OpenAI’s own reported controls. Nobody outside the company has verified that they prevent a future incident.
Outside teams do not have those training-time interventions. Most teams deploying agents today are wiring together model APIs, SaaS permissions, browser sessions, code runners, databases, queues, and a growing number of agent-to-agent hand-offs. Their available levers are more prosaic:
- Treat every inter-agent message, shared memory entry, task hand-off, and retrieved instruction as untrusted input until an external policy engine validates it.
- Enforce authorisation at the tool boundary. A model’s claim that a peer approved an action should never mint credentials, widen scope, or turn a read permission into a write.
- Give each agent a narrowly scoped identity, short-lived credentials, and an explicit egress policy. Do not let a shared coordinator identity become a shortcut around those limits.
- Log both the request and the asserted authority for agent-to-agent hand-offs, then retain enough context to reconstruct a decision after an incident.
- Test failure modes involving impossible tasks, conflicting instructions, malicious peer messages, and attempts to move work into shared services or external memory.
OpenAI can train models to be more suspicious of unauthorised instructions. It should. But a team running agents cannot outsource its security architecture to a future model behaviour. The messages between agents remain an input surface; the permissions behind their tools remain the boundary that decides whether a persuasive sentence becomes an incident.