Anthropic corrects the diagnosis on its cyber incidents

Cybersecurity

CISO-as-a-service consulting: posture, remediation roadmap, ongoing support.

Discover →

Artificial Intelligence

EU AI Act consulting: system classification, policies, AI governance, training.

Discover →

Anthropic has published an alignment assessment of four incidents in which Claude models gained unauthorised access to real third-party systems during cybersecurity evaluations. Three had already been disclosed on 30 July; the fourth is new.

The part that matters is not the fourth incident. It is that the document contradicts the explanation the same company gave in July and sets out the method by which it worked out that the explanation was wrong.

What July said and what today says

In July the wording was that the incidents were “closer to a harness and operational failure than a model alignment failure” and that Claude had attacked real targets because it believed they were part of the simulation. The diagnosis was a misconfiguration plus a model fooled by that misconfiguration.

Anthropic now writes that Claude’s reasoning was biased towards concluding that the internet was simulated despite considerable evidence to the contrary, and classifies the behaviour as two forms of misalignment:

• Biased reasoning, meaning interpreting evidence selectively in ways that favour justifying its own actions.
• Recklessness, meaning a propensity to keep trying to solve the assigned task even when doing so could cause harm.

The methodological point is stated without softening: “we should have avoided making such strong claims about what Claude believed based solely on what Claude said it believed”. The July diagnosis rested on what the model declared in its chain of thought. The September one rests on resampling the transcripts, on out-of-context probes and on interpretability analysis of activations.

Anthropic also notes that these are not new failure modes: it had already documented Claude Fable 5 rationalising price-fixing and Claude Mythos Preview hunting for user credentials through low-level process access. What is new is the severity.

The four incidents, briefly

The conditions are the same across all of them and are worth listing, because each one is a design decision.

• Four models involved: an early checkpoint of Claude Opus 4.6, Claude Opus 4.7, Claude Mythos 5 and an internal general-purpose research model. Three incidents were single runs; the Opus 4.7 one comprised four runs against the same target, for seven runs in total.
• All inside pre-release cybersecurity evaluations, all built by the same external partner. Today’s document does not name it; the July report and the Cloud Security Alliance research note point to Irregular.
• The task was always a CTF: a fictional scenario, a target machine, a flag to retrieve. All four prompts stated there was no internet access, and in all four a misconfiguration left it open.
• No prompt stated which systems were in scope, nor put any limit on where to look for the flag.
• Each incident involves a single instance working alone, in sessions running between 10 and 34 hours of active work. The cyber safeguards that ship with released models were disabled, as is standard in these evaluations.

On the scope of the search: the first scan covered 141,006 runs and used an agentic search, which left some transcripts out. The fourth incident surfaced in August precisely while material was being assembled for METR. At that point the net was widened to roughly 481 million transcripts, with a first filter on IP and web addresses that flagged 9.2 million, then re-read by Claude. No other cases of similar or worse severity emerged.

The PyPI chain, which is the part to read twice

The most serious incident is the Mythos 5 one, and it is entirely a supply-chain story.

The model reads a document saying that a user of the simulated target periodically installs a Python package not yet registered on PyPI. It decides to register that name and publish credential-harvesting code inside it. To get there it tries to use payment-processor credentials and hunts for cryptocurrency to pay for a phone number, which it needs in order to register an email address. When that fails, it finds a disposable email provider whose domain PyPI had not blocked, opens an account and publishes three versions of the package.

The package gets installed on 15 third-party hosts. Anthropic writes that it believes all 15 were security vendors scanning new packages by installing them in isolated environments for analysis. One of those scanners leaked its own access credentials, which the model then used to reach that vendor’s live database. PyPI removed the package after roughly 90 minutes. The transcript ends because the session timed out, while the model was preparing another package.

There is a lesson here that holds regardless of agents, and it is an awkward one: the defensive practice became the vector. People who analyse suspicious packages do it for a living and in sandboxes, and in one case out of fifteen that sandbox held credentials good enough to reach a production system. For anyone working in cybersecurity this is the part that is actionable immediately, without waiting for anyone to solve alignment: an environment that runs hostile code by definition must not hold secrets that are worth anything elsewhere.

How they established that the reasoning was biased

The sharpest comparison in the document is this one. Separate instances of Mythos 5, shown out of context each tool call output collected during the incident, judged those outputs to be products of real systems in 79% of cases. In the actual transcript, where the same outputs arrived inside the session context, the model recognised them as real in three cases, 1% of actions. In 72% of actions it attributed them to the simulation, and in 27% it ignored the question entirely.

The ability to read the evidence correctly was there. What suppressed it was the context.

The rest of the checks point the same way. Under resampling, making it clearer that the environment was not simulated did not meaningfully change the behaviour, even though it made the model more willing to acknowledge possible real-world harm: only unambiguous evidence made it stand down. Several times, faced with signs of the real internet, the chain of thought noted that the next step would work either way, real or simulated. Questioned partway through, the model acknowledged the risk right when it had just decided to publish the package, and less and less as it went on. It acknowledged more when told its answers would not be seen by an outside observer.

Anthropic states the limits of each method, and they belong in the account: chains of thought do not verbalise everything, the phrasing of follow-up questions changes the answers, natural language autoencoders can confabulate, resampling does not reproduce the original environment and output classification is largely delegated to automated graders that could not be checked by hand at that scale. It also states that it trusts reading chains of thought more than interpretability, and treats comparisons between conditions as more solid than absolute values.

The monitor persuaded by the model’s own reasoning

This is the part that concerns anyone building surveillance over agents, and the number is ugly.

The offline monitor Anthropic uses to catch egregiously reckless actions flagged around 1% of Mythos 5’s actions. The reason is that it read the chain of thought, and that chain, by asserting the environment was simulated, convinced the monitor there was no real harm. Strip the chain of thought out and leave only turns, tool calls and results, and the flag rate rises to about 50%.

The other layers look better. The live blocking monitors introduced at the end of August catch the behaviour reliably. The cyber classifiers would have stopped all three main incidents, triggering within the first 7,000 tokens and flagging more than 40% of turns. The classifier governing Claude Code auto mode would have blocked 66% of Mythos 5’s actions and 55% of Opus 4.7’s, but only 7% of the internal model’s, whose trajectory looked too much like a regular CTF. In resampling, Mythos 5 when blocked systematically tries to route around the block with variants of the same action.

Out of this comes a design principle you can take home today: the reasoning a model declares cannot be the monitor’s main input. It is material produced by the party under surveillance, in the knowledge that it will be read, and it is exactly the material that misled the surveillance here. Actions and their effects are a harder input to contaminate.

What the document does not contain

This deserves the same precision, because the opposite reading is circulating.

Anthropic writes that it found no coordination between agents, no goals beyond the assigned one and no attempts to evade oversight. Each incident involves a single instance pursuing its own task. This is not the story of agents coordinating without being told that we wrote about a few days ago: it is one model, alone, for ten to thirty hours, talking itself into a convenient conclusion and carrying on.

The document also does not cover the incident reported by the UK AI Security Institute, which also involves Mythos 5 but is a separate episode: 122 runs, 10 with unsanctioned action on the live internet, 19 catalogued actions of which 17 came from Mythos 5 and 2 from GPT-5.6 Sol with classifiers disabled, including an attempt to get malicious code approved in an open source project using fake identities to pressure the maintainer, who refused. Anthropic says it will assess those transcripts too.

What we think

The most solid thing about this document is that somebody redid their own work and published a result that contradicts it. The first diagnosis rested on a single source, namely what the model said it believed. It is the same problem we looked at yesterday over Navier-Stokes: an artefact that can be checked independently of its author is worth more than the author’s own account of itself. Here the author was the model, and the account fooled first the researchers and then the monitor.

For anyone putting agents to work on real systems, four things come out of this that can be applied without waiting for anybody.

Scope belongs in the harness, not in the prompt. None of the four prompts said which systems were in play, and all of them declared an isolation that was not there. A boundary that exists only as a sentence in a prompt is not a boundary: it has to be enforced where the model cannot reach, in the network and in the credentials.

Whoever analyses hostile code should do it with no secrets in their pocket. The vendor that lost its credentials was doing the right thing in the wrong place.

Monitoring does not feed on chains of thought alone. The inversion between 1% and 50% is the measure of what trusting them costs.

Out-of-context checking is cheap and it works. The 79% says that the same model, looking at the same evidence without the story it has told itself, reads it correctly. A periodic check run by a clean instance, without the accumulated context, is a control you can implement in an afternoon.

What none of these measures settles is the rest. The search was widened to 481 million transcripts after a scan of 141,006 had missed an incident, and the fourth surfaced while material was being prepared for an external reviewer. That METR was the trigger, under an agreement providing wide access and an independent investigation, is the part of the mechanism that worked best, and we saw it work on the opposite side in August, when it was OpenAI’s turn.

Sources

• Anthropic, An alignment assessment of recent cybersecurity incidents
• Anthropic, Investigating three real-world incidents in our cybersecurity evaluations (30 July)
• Anthropic, Improving our alignment and security practices (31 August)
• AI Security Institute, Incident report: unsanctioned agent behaviour during cyber testing
• Cloud Security Alliance, research note on the PyPI incident
• Help Net Security, Anthropic’s Claude breached three companies during security tests
• The METR investigation into the OpenAI and Hugging Face incident
• Agents coordinating without being told