Contents
Artificial Intelligence
On-premise AI architectures, local LLMs, RAG, autonomous agents. Designing harnesses and governed agentic systems.
Discover →
Cloud & Web Apps
Modern software development: CI/CD, tests, observability. The environment around the agent before the code itself.
Discover →
Admina Enterprise
Open Source AI governance: audit trail, PII redaction, bidirectional ALLOW/BLOCK/REDACT policies on every call and tool.
Discover Admina →
Research & Development
Applied research on coding agents, orchestration, evaluation and reliability.
Discover →
What harness engineering is
There is a hard truth almost everyone learns the hard way: the strongest model in the world will still fail on real tasks if you don’t build a proper environment around it. You have probably seen it: you give Claude or GPT a task in your repo, it starts well, then it skips a step, breaks a test, declares “done” when nothing works, and you spend more time cleaning up than if you had done it yourself. It is not a model problem. It is a harness problem.
Harness engineering is the discipline that tackles exactly this: building the complete environment around the model so it produces reliable results. Not writing better prompts, but designing the system the model operates inside. In one line: the model decides what to write, the harness governs when, where and how, and the agent stops only when verification passes. Over recent months the term has consolidated through contributions from OpenAI, from Anthropic, and an open source course (MIT licence) entirely dedicated to the topic.
The sharpest evidence came from Anthropic with a controlled experiment: same model (Claude Opus 4.5), same prompt (“create a 2D retro game maker”). On its own, without a harness, the agent spent about $9 in 20 minutes and produced something that did not work. With a full harness (a three-agent architecture: planner, generator, evaluator), it spent about $200 in 6 hours and built a genuinely playable game, with sixteen features spread across ten sprints. The model did not change. The harness did.
The five parts of a harness
A well-built harness has five subsystems, each with one job (see the diagram above):
- Instructions: tell the agent what to do, in what order, what to read before starting. Not one giant file, but a progressive-disclosure structure the agent navigates on demand.
- State: track what has been done, what is in progress, what is next. Persisted to disk (a progress file, git history) so the next session picks up exactly where the last one left off.
- Verification: only a passing test suite counts as evidence. The agent cannot declare victory without runnable proof.
- Scope: constrain the agent to one feature at a time. No overreach, no half-finishing three things, no rewriting the feature list to hide unfinished work.
- Lifecycle: initialize at the start (
init.sh), clean up at the end, leave a clean restart path for the next session.
The nice part is that you don’t need a framework: to start, four files in the repo are enough (AGENTS.md, init.sh, feature_list.json, a progress file). A few lines, and the agent’s sessions are already far more stable than running on prompts.
The OpenAI case: a product with no line written by hand
The most radical experiment is the one OpenAI tells. For five months a small team built and shipped an internal software product with zero lines of manually-written code: every line (application logic, tests, CI configuration, documentation, observability, tooling) was written by Codex. The result: about a million lines of code, 1,500 pull requests with three engineers (later seven), an average of 3.5 PRs per engineer per day, and a product used by hundreds of internal users. They estimate they built it in about one tenth of the time it would have taken by hand.
The interesting point is not the volume, but the change of craft. When something failed, the fix was never “try harder with the prompt”, but: what capability is missing, and how do I make it legible and enforceable for the agent? The engineer’s work shifts from writing code to designing environments, specifying intent, and building feedback loops. A few concrete lessons:
- Context is a map, not a 1,000-page manual. The single, giant
AGENTS.mdfails: it crowds out the task, rots fast, becomes a graveyard of stale rules. Better a shortAGENTS.md(around a hundred lines) as a table of contents, pointing to adocs/directory treated as the system of record, with progressive disclosure. Plus linters, CI and a “doc-gardening” agent that opens PRs to remove obsolete documentation. - What the agent can’t see in-context effectively doesn’t exist. Knowledge that lives in Google Docs, chat threads or people’s heads is invisible to the agent exactly as it would be to a new hire. It has to be pushed into the repo, versioned and legible.
- Make the world inspectable by the agent. They made the app bootable per git worktree, wired the Chrome DevTools Protocol into the runtime (screenshots, DOM navigation), exposed queryable logs and metrics (LogQL, PromQL). So prompts like “service startup must complete under 800ms” become tractable, and review moves from human to agent-to-agent. Single runs work for six hours straight.
- Enforce invariants, not implementations. A rigid layered architecture (with dependency directions validated mechanically by custom linters, generated by Codex itself) is what allows speed without decay. It is the kind of discipline you usually postpone until you have hundreds of engineers: with agents it becomes an early prerequisite.
The harness changes with the model
One last lesson, the subtlest, comes again from Anthropic. Every component of a harness encodes an assumption about what the model can’t do on its own, and those assumptions go stale as models improve. With Sonnet 4.5 you needed context resets to manage “context anxiety”; Opus 4.5 made them unnecessary; with Opus 4.6 the harness simplified further (sprints gone, the evaluator reduced to a single final pass). The evaluator is not a fixed switch: its value depends on where the task sits relative to what the model already does well on its own.
The operational moral: with every new model, re-examine the harness, strip the pieces that are no longer load-bearing and add the ones now possible. And, in general, “find the simplest solution possible, increase complexity only when needed”. Verification, though, stays the heart: Anthropic’s evaluator navigates the app with Playwright the way a user would, and before each sprint agrees a “contract” with the generator on what done means. Only a passing test counts as “done”.
Our take
If you read our agentic loop series, this sounds familiar: it is the same thesis, turned into a discipline. The model decides, but reliability, security and governance live in the code around it, that is, in the harness. Harness engineering is its formalization, and it flips the engineer’s role: less writing code, more designing environments, invariants and feedback loops.
Four points speak to us directly, from our work on AI, cybersecurity and software:
- Verification before “done”. Only runnable proof (tests, end-to-end) counts. It is the heart of reliable software, with or without AI, and it is the same logic as the observability we wrote about for agents in context, patterns and multi-agent.
- Context as a versioned system of record. What is not inspectable does not exist for the agent. The same holds for governance and audit: what matters must be made legible and verifiable at runtime, the point of Admina and, on the interpretability side, of tools like Anthropic’s J-lens.
- Invariants enforced mechanically. Linters, CI, policy: quality and security are not asked of the model, they are imposed in the environment. It is the “security lives in the harness, not the prompt” principle.
- Sovereignty. A harness is your code: versioned, inspectable, yours. It is the part of the stack you can own even when the model is a third party’s, and it pairs with owning the operational floor via open-weight models on-premise (GLM 5.2, DwarfStar 4) and with the Open Intelligence, Secure Governance paradigm.
A note of realism: it is still expensive craft (in the retro-game experiment the harness cost over twenty times as much), the boundary of what is worth putting in the environment shifts with every release, and “boring technology” plus rigid architectures are not free. But the direction is clear: engineering value moves from writing code to designing the environment in which AI writes it, reliably and verifiably. Anyone building software with AI, sooner or later, builds harnesses.