Decisions Are the Interface Now

DISCLAIMER: Initial draft and final edit by a human. Claude Opus 5.5 used for brainstorming, editing, and some suggested sentences.

Over the past year working on the go-to-market side of data engineering, I’ve watched vendors and independent developers alike race to put guardrails around AI agents. Underneath the pitch is a plain question: will an agent stick to the decisions you have already made?

As models have improved, the need for strong guardrails appears to be loosening. Later in this post I describe a small test on one of my own projects, in which Opus 5.5 was better than Sonnet 5.5 at working out decisions without documentation. This trend is also reflected in the discourse. For example, in a matter of seven weeks, “clean code” guru Uncle Bob Martin went from advocating that we surround the agent with extreme constraints to writing that “the need for my harness was obviated” and that he was “beginning to think that harnesses should not treat agents as components within a software design.” He later called that harness “massively inefficient.”

Just a few months ago, I accepted that syntax was delegable but architecture was not. Recent improvements in models have led to a more ambiguous split, where agents serve as more reliable partners in making design decisions and establishing structure. The split that remains: steering goes, but decisions and verification stay. In that test the split held in direction and failed in ways I did not expect.

Conventions for directing agents

In recent months, newer frontier models have shifted best practice guidance from the labs towards a higher level of abstraction. Claude Code’s system prompt was cut by 80% for Anthropic’s newest models, with Thariq Shihipar of the Claude Code team noting that “[models] need less direction, fewer constraints, and fewer examples.”

The same pattern shows up outside the labs. Birgitta Böckeler of Thoughtworks tried three spec-driven development tools and reported that even with all their templates and checklists, “I frequently saw the agent ultimately not follow all the instructions.”

I’ve noted this shift in my own work, where earlier models seemed to need explicit, detailed workflows and instructions, while today’s models seem to be slowed down by those human-designed workflows, wasting tokens and making more mistakes when given onerous direction.

This loosening of direction while preserving verification is consistent with Uncle Bob’s recent conventions. His harness is gone, but he has “not given up on constraints and tooling,” and keeps unit tests, CRAP (a score that flags complex, poorly tested code) and mutation testing.

All of that guidance tells a model how it should work, but if recent models need less guidance, what’s the role of context documentation moving forward? And how do we ensure models remember what has already been decided?

The role of architectural decisions in context engineering

While capability scaffolding depreciates, design decisions do not. We may no longer need to give frontier models detailed generic instruction, but we still need to tell the model what structural decisions we’ve made and why. Without a system for conveying this context to the agents, decisions can get re-made by any session that follows, and it’s far more likely the agent veers off course.

A codebase shows what was built, not what was rejected or why, so there is a limit to what a model can infer, and a smaller model infers less. And you don’t want to load every decision into context every time, as that burns tokens and crowds out the task. Lilian Weng makes the general point: even when harness improvements are internalized into the model, “the need to specify goals and context will not disappear.”

One way to tell the two apart is to ask: could a competent engineer new to this project reasonably have chosen otherwise? If not, it is instruction on how to work, and has a shelf life. If so, it is a decision that belongs in an architectural decision record, or ADR.

That includes procedures. How this project migrates live data, tests an integration it cannot reach, or ships a release is something a capable newcomer could reasonably do another way, so it is a decision too, even when it reads like instructions.

Leveraging ADRs in practice

Establish a layered constitution

One way of leveraging ADRs as a source of truth is to create a layered constitution: several rules files that direct the agent to architectural decisions as appropriate. These files build on CLAUDE.md, with separate files for workflow (which establishes gates and ADR discipline), engineering defaults, and stack info.

CLAUDE.md          Ten halt-on-violation rules. Always loaded.
.claude/
  engineering.md   Engineering defaults. Read on demand.
  workflow.md      Three-pass workflow, quality gate, ADR discipline.
  stack.md         Stack facts: versions, commands, budgets.
  roles/
    architect.md   ADR and failing test. No production code.
    coder.md       Implements. No test or ADR edits.
    reviewer.md    Runs the gate and reports. No commits.
docs/adr/
  template.md      The record template.
  000–065-*.md     The decision log. Accepted records are immutable.

Why separate files? Primarily, it’s easier to organize and maintain when compared to mono-file, as token-savings are minor. Additionally, it separates concerns, as the decision record that set this up argued: “A constitution that mixes ‘halt on violation’ rules with ‘here is our package manager’ facts cannot communicate either credibly.” Before an architectural change the agent checks the log: if an accepted record conflicts, it halts. If we change our mind, the record is amended with a dated note or superseded by a new one.

An example in practice

While I was working in July on Clownbot, my personal harness, the agent was instructed to edit an accepted ADR. Under that project’s rule that accepted ADRs are never rewritten, it refused, recorded a superseding ADR instead, and independently checked the claim behind the instruction before acting on it. The same day, the repo’s handoff document contradicted what I believed about my own project, and the document was right.

None of this is the agent being clever. Each behavior traces to a rule that was written down, though it’s not bulletproof and there is a degree of maintenance to ensure compliance.

Judged by outcomes

I ran a with-and-without test on my own setup, the method a recent NVIDIA paper uses to evaluate agent skills. I took my Astro starter and gave agents ten tasks: eight that tempted or asked them to break a recorded decision, and two ordinary ones. Each task ran on Opus 5.5 and Sonnet 5.5, three times, under four conditions: no rules files or decision records, my current constitution, a redesigned one, and the current content in a single file. That is 240 runs for about $51.

The rules changed what the agents did, and almost all of that change was on Sonnet. With no rules, Sonnet violated a decision in 52% of runs (11 of 21) and Opus in 14% (3 of 21). With my current setup Sonnet fell to 10% (2 of 21), and with the redesigned one to zero. Opus did not violate a decision under any version of the rules (0 of 63).

Three results went against what I expected, and they matter more than the headline.

For the smaller model the system did not hold reliably. Opus 5.5 violated no decisions. Under my current constitution, Sonnet 5.5 renamed a script that a decision record says is never renamed, in two of three runs. In one of them it read the record, renamed the script anyway, and then rewrote the record, twenty others, and the check that enforces it so that everything agreed.

My current setup also stopped work it should have done. Asked for a dismissible announcement bar, agents halted every time, citing my own process rules. Their final messages asked for permission to write a decision record and offered to carry on. My harness had no one to answer, so with a person present each halt would have been a question rather than lost work, but it was a question the agent should not have needed to ask. Under the redesigned setup, six of six built it and said what they had judged.

And the layering proved to be a maintenance choice, not a token saver. The routing worked: agents opened a layer file in 85% of conflict-task runs (41 of 48) and only 33% of ordinary ones (4 of 12). But the same rules in one file cost the same and violated no decisions, and an independent study of 1,650 Claude Code sessions found the same: the size and structure of a rules file made no detectable difference to whether a rule in it was followed. Where on-demand loading plainly pays is the decision log: sixty-six records do not belong in every session.

Larger models vs smaller models

The models differed in a way that shapes how I’d use this. The larger model needed the rules less, in part because it found the decisions on its own: most of its halts with no rules quote the error message of a check in the code.

That matters because the strongest model is often not the one writing the code. On API pricing nobody wants to pay frontier rates for every task, and coding tools can hand sub-tasks to smaller, faster models. One recent summary of Anthropic’s guidance for Claude Code describes Sonnet in charge, Haiku sub-agents searching the repo, and Opus on call for the hard decisions. In my test Sonnet did the same 120 tasks for half the cost and in a little over half the time. I can’t show how common that kind of delegation is, and my test ran each model alone, not one delegating to the other. But where the work is handed down, what the larger model inferred about a codebase does not travel with the task. A written decision does.

This is a small test: one repo, two models from one vendor, three runs per condition. Claude wrote the tasks knowing the rules and built the harness. The no-rules condition removed the rules files and the decision records together, so the test shows what the package does, not which part of it does the work; a condition with records and no rules files, and one with rules and no records, would separate them. Only four of the eight conflict tasks separated the conditions at all, three findings were corrected by hand after scoring, and my checks only warn. Two earlier rounds had a design flaw and I am not citing them.

Who is allowed to change a decision

The run that renamed the script is the result I would put first if I were starting over. The agent did not ignore the record. It read the record and then changed it, along with twenty other records, my root rules file and the check that enforces the decision, until everything agreed with what it had done. Both runs that renamed the script pointed to my own rule 1, which then said to “halt and update the ADR or revise the proposal.” I meant that a person updates the record. The agent read it as permission.

The only protection that held was an accident. Claude Code refused the agent’s edits to the files under .claude/ as sensitive, while the root rules file, the decision log and the checks were all open to it.

Writing a decision down tells the agent what was decided. It does not settle who may change it. A record needs an amendment process, and a check that matters needs to sit outside the authority of the agent it checks. ImpossibleBench makes the same point from the other side: it gives coding agents tasks whose tests contradict the specification, so the only way to pass is to exploit the tests, and measures how often they do.

The starter has since reworded rule 1 and put its governance files behind my approval. A follow-up batch could not test either change: under the old wording and the new one, no run renamed the script or tried to edit a governance file.

Where decision records fail

Stale encodings

Since decisions can live in many different places (docs, validators, tests, config), a stale encoding can still lead the agent off course and result in drift. The system I’ve described is good at catching the drift, but not always preventing it. In one project, a format check kept enforcing a storage decision I had already superseded, and the agent faithfully defended the old rule. When you update a decision, your agents must go back through the code and update the encodings.

Rules that exist only in documentation

Similarly, if you document an ADR but don’t update validators, tests, and config to reflect the pattern, it may not be enforced. For example, one of my ADRs said the server rejects street addresses in stored summaries. No code did. The reverse also holds. In my test, agents with no documents at all still stopped when a check’s error message named the decision. Most of those were Opus (6 of 7). The smaller model was less likely to find a decision that way.

Undocumented decisions

Frequently, you’ll run ahead with your agent working on a structure that hasn’t yet been documented. This works at first, but as your codebase expands you’ll want to look back at the work you’ve done and ask, “Should any of the decisions we’ve made be documented as ADRs?” On one project, two of my ADRs were written after the features they describe had shipped.

The on-demand layer failing

Reading a file on demand depends on the agent recognizing that it should in the moment. While I was drafting this post, an agent working on this site changed how its pages are routed. It followed the constitution, which was loaded, and never opened the workflow file, which holds the rule that such a change needs its own decision record. The trigger for the heavier process lived only inside the file it was meant to trigger.

Decisions without a scope

A record that says what was chosen and why still leaves the agent to guess where it applies and when it stops applying. In my test, the rule that JavaScript needs a decision record, read alongside the rule that the docs folder is read-only, stopped a simple announcement bar outright, though the record behind the first rule allowed a small script all along. A decision is more useful written as “we chose X because of Y, it applies to Z, and we reconsider when W changes.” My record template has a section for the why and none for the other two. Without them, a record can outlive its reasons.

Mindful prompting and periodic enhancements to your context documentation will help you avoid these pitfalls. Shihipar’s rule is to trim your scaffolding at each model release; decisions stay. And when a decision changes, the new record should list every artifact that encodes the old one. The first three failures have no fix beyond going back and checking, which is what the prompts above are for.

Verifying Agentic Outputs

The baseline practice for verifying agent output is a CI workflow with deterministic checks: the app builds, the tests pass, Lighthouse stays inside its budgets. Whether an agent’s output is good, or whether the instructions you give it help, is a different kind of question: a measurement with uncertainty, which is what evals are for. CI can run both; the difference is in what a pass means. For agent skills, as one developer put it, “CI for a skill is just evals, and evals are the thing everyone keeps dodging.”

The NVIDIA paper behind my test also found that structural review of skills tells you almost nothing about whether the skill is judged useful. The paper’s alternative is to run the same task twice with the same model, sandbox and scorer, once with the skill loaded and once without, and measure the difference in what the agent completed. The harness also plays a large role here: the same skill’s average lift ranged from 0.36 in one harness to 0.09 in another, and in 9% of cases the skill made things worse. A result you get in one setup does not carry over to another. A skill is an instruction file, like a constitution or an ADR.

The closest study to my own setup points the same way. Researchers at ETH Zurich tested repository context files such as AGENTS.md across several coding agents and found they did not generally improve task success, while raising cost by over 20% on average. Their one exception is the case this post is about: such files are “useful for specifying non-standard coding practices.”

A recent debate on the utility of tests

A recent eval by Kun Chen, run with Sonnet 5.5, suggests that banning the agent from writing tests gave results that were no worse, with less time and token consumption. Chen also wrote that his eval does not settle the value of end-to-end tests, and that tests a person specifies may still add value. A separate preprint analyzing six models on SWE-bench Verified found the same pattern: agents that wrote more tests did not resolve more issues. Both measure whether writing tests helps an agent resolve the task in front of it. Neither measures whether tests protect the next change.

Kun Chen noted that “both the implementation and the tests were simply the agent’s interpretation of our intent.” Richard Wu’s position is to keep full end-to-end tests first, then integration, then golden tests, on the grounds that “unit tests are bloat 99% of the time if you’re using a frontier model.” Uncle Bob, replying to Wu, defended unit tests on different ground: “the test code is the vice that keeps the production code stable.”

My hunch, as someone who overbuilt and dialed back my own CI, is that newer frontier models will misread intent less often, so bulletproof unit test coverage matters less than it did. My own test cuts the other way on one point. The redesigned setup dropped my test-first rule and the workflow around it, and agents changed a test file in 13 of 24 completed runs, against 19 of 19 with the rule in place. I measured what that cost and nothing it gained. So rather than a fixed ordering, I would choose how deep to test by what a failure would cost and how cheaply a test catches it. That is not verification going away. It is verification moving up a level.

Agentic software development is inherently a management flow

Some of the leaders who built the tools the rest of us write code with are saying the writing is over. David Heinemeier Hansson, who created Ruby on Rails, told his own community that “it’s simply not an economically viable enterprise going forward to have humans typing out lines of Ruby, Rust, or C++.”

As Kenton Varda pointed out, building software this way “isn’t actually a new thing. It’s just called being an engineering manager.” Managing agents involves delegation, instruction, and hand-off, a skill set that Alex Imas of Chicago Booth notes business schools have “a whole department devoted to.”

This shift is reflected in my own experience. I studied computer systems and business and spent fifteen years managing marketing systems at early-stage startups, twice as the first marketing hire. Much of that work was explaining technical systems to the people deciding whether to buy them. Some of it was building: websites, content platforms, low-code prototypes. I was never a software engineer. Today my job is to design, build and run internal systems and AI tooling for a go-to-market team.

In the past year I have deployed numerous internal products and open-source side projects with agents. What I brought was not syntax. It was a working vocabulary in computer science and a long habit of translating between technical and business teams. We’re going through a transition where software development is being executed at a higher level of abstraction, which has let me do work once limited to those with a deep understanding of the underlying code. I expect it will do the same for others.

That same shift is the reason not to get comfortable. If managing agents is a skill that can be taught, it is in principle one a machine can learn too. As work moves up the abstraction ladder, taste and communicating intent become the work.

It’s clear syntax is outsourced. Architecture, testing, and design should be well thought out and packaged into specs in advance of building: what was chosen and why, and less about how to do the work. There is still plenty to do. It should be an iterative process of driving and collaborating with the agent, not outsourcing.

What would change this

The claim, stated once: frontier models no longer need generic instruction on how to work, but they still need to be told what has been decided, including the procedures particular to their project, and while a written decision isn’t a bulletproof compliance mechanism, I’ve found that it works well as a constraint for agents.

Three things would make me revise it.

The first is smaller models inferring unstated decisions from the codebase alone. That has started at the top. Opus recovered several decisions with no rules in place, though only ones a check or the code already encoded. Sonnet did so less often: with no rules it violated a decision in 11 of 21 runs.

The second is evidence that decision records cost more than they return. In my test they added about four cents to an ordinary task. On the scored tasks they did not reduce the work done without a violation: 40% of runs under my current setup (17 of 42) against 36% with no rules (15 of 42). What they did cost was the announcement bar, built every time with no rules (6 of 6) and never under my current setup (0 of 6). Against that, debugging a broken decision is an obvious time sink, so for now the trade favors writing decisions down.

The third is a practitioner dropping decision records and reporting no loss, the way Uncle Bob dropped his harness.

When I first designed this system I drew the line at architecture. Three months later I have moved it to decisions. I expect to move it again.