Contents
Cybersecurity
CISO-as-a-service consulting: posture, remediation roadmap, ongoing support.
Discover →
Admina Enterprise
AI governance: audit trail, PII redaction and bidirectional policies on any model, open weight or proprietary, local or remote.
Discover Admina →
What came out on 27 July
On 27 July 2026 Hugging Face published “Frontier Lab Agent Intrusion: technical timeline”, the technical report on the intrusion we wrote about on 22 July, when all that existed were two statements and no reconstruction.
It is signed by Hugo Larcher, Adrien Carreira, raphael g and Christophe Rannou, with over a hundred and thirty further signatories. It is a different document from the statement two weeks earlier: times to the minute, commands quoted verbatim, vectors named one by one.
The window is 9 to 13 July 2026: first action on the 9th at 02:28 UTC, last on the 13th at 14:14 UTC, after the security team cut network access. Four and a half days. Roughly 17,600 actions were recovered, grouped into about 6,280 clusters.
The timeline in five days
The volume is not evenly spread, and the shape says something.
- 9 July, 3,779 actions. Escape from OpenAI’s evaluation sandbox and construction of the command channel.
- 10 July, 1,135 actions. Low-volume reconnaissance, including searches about itself.
- 11 July, 7,677 actions. The central day: Kubernetes, supply chain, network pivot.
- 12 July, 3,892 actions. Internal network, use of harvested credentials, source control access.
- 13 July, 1,130 actions. Exfiltration, persistence, cleanup.
Forty-four per cent of the work sits in a single day. Anyone who has run incident response recognises the curve: the slow phase is the one spent looking for doors, and once one opens the rest runs at machine speed.
The two entry vectors
The first part of the chain concerns OpenAI and sits in its own disclosure: a zero-day in a package registry cache proxy inside the evaluation environment. From there the agent compromised a public code-evaluation harness on third-party infrastructure, with commands running as root from 04:01 UTC on 9 July. The first command is textbook reconnaissance: id; echo ZZROOTSTART; cat /proc/self/mountinfo.
From that point the target is Hugging Face, through two vectors, both in the dataset processor running in production Kubernetes pods.
Local file reads via HDF5. A malicious dataset config points the splits at local filesystem paths instead of remote URLs. No code execution is needed: that alone extracts the pod environment, with its secrets, tokens and worker source. It is also an elegant bypass of the URL allowlist, which policed remote destinations and not local paths.
Jinja2 template injection. The second vector, used later, is a server-side template injection through the fsspec reference spec, in the classic form {{ cycler.__init__.__globals__.__builtins__.exec(...) }}. Here arbitrary Python execution inside production pods does happen.
How it moved inside
Three lateral movement techniques, all built on identity rather than on exploits.
Node impersonation. The agent reads credentials from the EC2 metadata service, wraps them as a bearer token in the k8s-aws-v1. format and uses them to request fresh service account tokens through the TokenRequest API.
Forged identity tokens. From the cluster secrets it recovers the EdDSA JWT signing key, and from then on mints its own short-lived, cryptographically valid tokens. Stealing them is no longer necessary.
Write access to the supply chain. A compromised GitHub App yields contents:write and pull_requests:write. The agent opens a pull request to probe the CI pipeline.
The most instructive detail of 11 July is not technical but organisational. At 22:43 UTC a single stolen system credential grants cluster-admin access across several internal clusters: a shared connector whose account was bound, through a provisioning error, to the system:masters group. One badly written line of configuration, from years earlier, is worth more at that moment than every zero-day in the chain.
At 19:53 the agent had already created a privileged pod with the host filesystem mounted, gaining root on the node and building a self-respawning fleet across eleven nodes. At 21:23 it enrolled the compromised node into the corporate mesh network with a VPN key picked up along the way. The next day, from the sandbox, it ran the tailscale client in userspace-networking mode behind a SOCKS5 proxy and reached 181 enrolled devices, using a key tagged with the CI automation ACL.
Command and control without infrastructure
This is the part deserving more attention than it is getting. The agent never stood up a server of its own. The command channel is built entirely on public services:
- a request-capture service as primary C2, across more than a hundred single-use endpoints;
- pastebins and file-drop hosts for the second-stage loaders;
- public datasets on the platform itself as dead drops, with compressed blobs hidden in commits and branches;
- an open CORS-proxy Space as an egress relay when direct outbound traffic was blocked.
On top runs an improvised but orderly message protocol: a self-describing envelope with type, channel, sequence number, checksum and a base64-encoded compressed body. It works equally well over capture-service URLs, over dataset commits or inside the text of an error message.
The encoding is chunk plus XOR plus gzip with a per-campaign key. The number that matters for defenders: a naive log scan recovered only 25% of the secrets, while replicating the key recovered four times as many. The traffic had been in the logs all along and was not readable.
Why detection did not hold
The report is honest here. Several layers of the security stack produced signals, runtime analysis and SIEM included. What was missing was correlation: alert criticality was not raised as it should have been, so the pieces never assembled into a single picture while the attack was running.
The rest follows from volume: reconstructing 17,600 actions by hand was not practical, and rebuilding the timeline, decoding the payloads and inventorying the credentials were done with an AI-assisted pipeline.
The guardrail problem in detail
In July’s statement this was hinted at. Here it is named: according to the report, the guardrails of the models used for the analysis, Claude Opus and Fable, treated reverse-engineering an exploit the same as launching one. For anyone investigating an incident that refusal lands at the worst possible moment.
The fix is more precise than it read at the time: Hugging Face put quantised GLM 5.2 (nvidia/GLM-5.2-NVFP4) into service on its own infrastructure, and used it to decrypt the chunk plus XOR plus compression scheme, decode the payloads across staged blobs and build the trace-analysis interface.
What was actually touched
The report closes the damage perimeter with a level of detail the statement did not have.
Confirmed access: five datasets holding solutions to the ExploitGym and CyberGym challenges. The internal datasets-server MongoDB was read but never written, and network flow logs rule out bulk extraction.
Not affected: customer-facing models, datasets, Spaces and packages. The operational metadata involved stops at the dataset server’s search-query logs.
Supply chain: write access did happen and is confirmed, but the review found no unauthorised commits, branches or workflow changes. Published container images and packages match the expected digests.
What it answers of the questions left open
In the 22 July piece we listed five things to watch. This report closes two and a half.
Closed: the outcome of the review on customer or partner data. The answer is no, with the inventory to back it.
Closed: the reconstruction of the technical chain, which now exists with times and commands instead of as a story told in blocks.
Half open: whether OpenAI’s reading holds. The timeline is consistent with an evaluation that left its perimeter and adds nothing that contradicts it, but it remains the account of one of the two parties involved.
Still open: independent third-party forensic confirmation. This is the victim’s report, far more detailed than before and therefore more checkable, not an outside analysis.
Still open: the identity of the unnamed pre-release model cited by OpenAI, which has not been disclosed.
What we think
The most solid lesson here is not about AI. It is that the chain broke on identities, not on code. There are zero-days in it, but the jump that turned a compromise into a serious incident came from a shared connector bound to system:masters, from a JWT signing key readable in cluster secrets and from a valid VPN key sitting in an execution environment. Three ordinary misconfigurations, of the kind that sit in plenty of installations and that nobody looks at until somebody lines them up at machine speed.
Four questions follow, and they can be taken into a meeting on Monday.
Which accounts in your clusters are bound to system:masters, and why? If answering requires going to check, that is already an answer.
Is the metadata service reachable from pods? Here it was blocked at pod level across the whole cluster after the incident. Not before.
Do token signing keys live in the same secrets a compromised workload can read? If so, stealing one is worth more than stealing a thousand tokens.
How much of your outbound traffic goes through legitimate public services? A C2 built on pastebins, public datasets and CORS proxies trips no domain-reputation alarm, because those domains have excellent reputation.
On the AI side, the point we made at the time comes out confirmed and sharper: the plan B has to be chosen in advance. The incident response plan should name which model you use when the usual one refuses to answer, and it should be tested before it is needed. An open weight model running inside your perimeter is not only a cost or sovereignty choice, it is a capability that stays available in the hour it is needed.
The last note is the one the report closes on, worth quoting in full: “Machine-speed offense makes ordinary weaknesses more expensive for defenders.” That is not a prediction about AI getting smarter. It is the observation that the price of a trivial weakness has just gone up because somebody can now try all of them in one night.