a permanent system improvement
Self-directed R&D · inside a live Swiss agency engagement · 2025 — 2026
Engineering an AI Delivery Pipeline.
A year shipping production code with AI coding assistants inside a live Swiss agency engagement — no mandate, no budget line, no playbook. Every model upgrade raised the ceiling on what was possible, but the failure modes stayed constant: scope drift, degrading sessions, siloed agents, plausible work that failed code-owner review. Midway through, the thesis inverted — the bottleneck was orchestration, not intelligence. I stopped optimizing model choice and engineered the layer around it. The industry now has a name for that layer: the harness.
Problem 1 — Scope drift
The agent touches files outside the ticket, “improves” code nobody asked for, and forgets conventions mid-session. The fix was unglamorous: a persistent instruction set — a CLAUDE.md encoding conventions, guardrails and scope discipline, refined every time a failure revealed a missing rule. Prompt-region stability: boring, and responsible for the single largest drop in convention violations I measured all year.
Problem 2 — Long sessions degrade
Sessions slow down and quality drops as context bloats — full interaction history plus large injected files, until the signal drowns. The intervention: session grounding. A SessionStart hook injects a small set of curated, checkpointed context files instead of accumulated history. The hard-won lesson is a trade-off: injected context must be small and stable, or the grounding mechanism itself becomes the bloat.
Drag the diagram sideways to read it
Problem 3 — Siloed swarms
Parallel agents fail in a specific way: each completes its task correctly in isolation, and the results don’t compose, because no agent knows what the others learned. The pattern that fixed it: subagents run in fully isolated contexts and return only distilled summaries to an orchestrating agent. Context stays bounded per agent; knowledge still flows laterally through the synthesis step.
Drag the diagram sideways to read it
Problem 4 — The flywheel
The core of the work: five domain-specific review agents encoding the agency’s conventions — markup semantics, CSS architecture, backend patterns, accessibility, scope discipline. The loop I designed to train them: implement, agents review, the code owner’s PR review — standard practice at the agency for every PR — catches what remains, and I log every correction as structured feedback — ground truth — and update the agents. Repeat. Each cycle the gap shrank — measured, not felt — until the flagship reviewer reached 11/11 agreement with the lead’s real PR corrections. Not testimonial approval; empirical calibration.
Drag the diagram sideways to read it
What came out of it
The harness stopped being a way of working and became two plugins. The first stayed with the client: a conventions plugin for the agency’s Umbraco projects, built over three days in April 2026 — 5 review agents, 8 auto-loading skills, 2 commands. It runs every specialised reviewer in parallel and adds the checks no single reviewer can make: a typo between a Razor js-* hook and the TypeScript querySelector looking for it, a component used in Razor but never registered in the init factory, a token used in SCSS that does not exist in variables.scss. When the code owner corrects a PR, the correction is saved as feedback and encoded as a rule in the relevant agent.
The second is compounder, its open-source successor — rebuilt from scratch and generalised beyond one stack, designed to run on Fable, Opus, Sonnet and Haiku: 11 skills of about a hundred lines each, 7 agents (researcher, four reviewers, adversarial verifier, fork executor), a hook that catches mid-work asides so a message never derails the running task, and /slfg, the whole pipeline as a swarm. Its quality gates are run by an agent that tries to refute the evidence instead of trusting the pipeline’s own self-attestation.
The industry caught up
In February 2026, OpenAI published “Harness engineering”, five months of shipping a product with no hand-written code. The engineer’s job, in their words: “design environments, specify intent, and build feedback loops that allow agents to do reliable work”. In July, researchers published “The Harness Effect”: same enterprise scenarios, same prompts, same six models — only the orchestration layer changed. Roughly 38% fewer tokens, 41% lower cost, 44% less execution time, quality statistically unchanged. Their vocabulary retroactively described this year: token maxing was my degradation problem before session grounding; harness leverage matched what I saw at every model upgrade. The layer I had been building had a name.
Why a design engineer
Harness engineering is interface design. Deciding what context an agent sees is information architecture. Guardrails and scope discipline are constraint design. The flywheel is a usability loop — observe failure, adjust the system, measure again. Nearly thirty years of design practice turned out to be directly load-bearing for a problem that looks, from the outside, like pure engineering.
A calibrated delivery harness, portable across stacks and model vendors: a persistent instruction set, session grounding over checkpointed context, subagents composing through synthesis, and review agents trained by a feedback flywheel against the code owner’s real PR corrections. Shipped twice: as the client’s conventions plugin and as compounder, open source.