Fast evals for better coding agent decisions

A fast eval is a check a reviewer can run in a few minutes to prove an agent change did what its receipt says it did. It is the cheapest way out of the merge jam that hits every team the same way: agent branches all go green in CI, and the reviewer cannot say what any of them actually did because the replay recipe scrolled off the top of the chat. The fix is a small receipt check, not another review meeting. If you want one number to watch, measure how often reviewers replay a session to understand a change. Everything below is about driving that number down.

See why your queue is slow

Review queues back up because checking is slow, not because writing code is slow. Agents made generation almost free and left verification exactly where it was. So the queue grows while everyone feels faster.

Another prompt template will not clear it. A faster way to check finished work will.

There is a trap worth naming. When green CI becomes the goal, green CI stops measuring the thing you care about, which is whether a person can explain the change. That is Goodhart's law in your pull request queue. The real lever is traceability: can a reviewer trace what happened without you in the room.

Four checks that take minutes, not meetings

Each common failure pairs with one fast eval: a named artifact a reviewer compares against the diff, without ever opening the session.

The first failure is recursive handoff blur. Chained agents hand back summaries that leave out child-owned paths, and the parent approves a mystery diff. The fix is a child receipt block. Every child returns the paths it touched, the commands it ran, and the tests that prove the regression guards hold. The eval is one comparison, receipt against diff, and parents stop approving on tone.

The second is review queue theater. CI is green, yet reviewers keep asking "why this approach?" and find no written answer anywhere. The fix is a decision stub. The PR template forces three lines: constraints considered, alternatives rejected, and the proof you verified it. The debate moves from vibes to written tradeoffs, which is a much shorter meeting than archaeology.

The third is Cursor scope fog. The .mdc rule language reads as precise until two reviewers argue about what it meant. The Cursor agent docs give you the mechanics; the contract is still yours to write. The fix is a scope ledger. The parent chat carries five lines: goal, allowed paths, forbidden paths, verification command, merge owner. Review turns into checking the ledger against the diff.

The fourth is Claude permission creep. On a shared laptop, bash approvals in Claude Code become muscle memory. The fix is a supremacy clause at the top of CLAUDE.md. State which hooks win, which folders need human eyes, and where temporary overrides live. Sessions stop inventing policy mid-run because precedence is written down.

Here is a starting rule file you can paste and adapt:

---
description: Delegation boundary snapshot (adapt globs to your repo)
globs:
  - "**/*"
alwaysApply: false
---

- Cursor: keep scopes explicit in `.mdc`; forbid undeclared MCP domains.
- Claude Code: cite `CLAUDE.md` precedence before expanding bash scope.
- Codex: ensure `AGENTS.md` carries replay-friendly verification notes for CLI runs.

Think of a fast eval as a turnstile, not a tribunal. One person, one receipt, one push through. A change that cannot clear a turnstile was never ready for the merge train.

Run the four gates before you merge

Four gates cover most agent merges. Read this table as the whole job, not a checklist of nice-to-haves.

If your repo cannot state its boundaries plainly, agents will guess. Guessing scales badly.

Try it on one stuck PR

Pick one named fix and turn it into a real artifact this week: a .mdc rule, an AGENTS.md note, or a child receipt template before your next automated run. Our training runs this exact drill on a live repo with your own branches, so bring one stuck PR and leave with a receipt format the team keeps.

Further reading

• NIST AI Risk Management Framework
• Model Context Protocol specification
• OpenAI Plugins repository