In AI software development the bottleneck moves, it does not disappear
Before agents, the slow step for most teams was producing a change. Now the slow step is deciding whether a change is right. That is the whole story of AI software development in one sentence, and almost every rollout problem we see traces back to a team that sped up the first step and left the second one untouched.
The symptom is easy to spot. Pull requests get bigger. Review latency goes up. Someone says "the agent wrote it, it's probably fine" out loud in a standup. Six weeks later there is a class of bug nobody can attribute to a person.
Where agents are reliably good
After training more than a hundred teams, the same tasks keep working:
- Mechanical refactors across many files. Rename, extract, change a signature and fix every call site.
- Test writing against code that already exists and already works.
- Reading unfamiliar code. "Trace what happens when this endpoint gets a 409" is a very strong prompt.
- Migrations with a clear before and after. One framework version to the next, one client library to another.
- Glue: scripts, fixtures, seed data, one-off data checks.
The common thread is that success is checkable. You can run the test suite, diff the output, or read the trace. Where there is no cheap oracle, quality drops fast.
Where it goes wrong
Agents are weak on the parts of the job that depend on things not written down. Which module is load bearing. Which abstraction the team already tried and abandoned. Why the retry count is 3 and not 5. An agent will produce a fluent, well-structured change that quietly reintroduces a design you rejected last year, because nothing in the repo records that you rejected it.
The second failure is scope creep inside a single task. Ask for a bug fix, get a bug fix plus a refactor of the surrounding module plus a new helper file. The change is often defensible in isolation and terrible for review, because now you cannot see the one line that mattered.
The third is confidence in the absence of evidence. If the agent cannot run your tests, it will still tell you the change works. Give it a way to run them.
Practices that actually hold up
Four things, in the order we teach them.
Write the repo instructions file. CLAUDE.md, AGENTS.md, or .cursor/rules depending on your tool. Put in it the things a new senior hire would need on day one: how to run tests, which directories are off limits, what your team does not do. This is the single highest return edit in the whole practice.
Cap the change size. One task, one concern. If a diff exceeds a few hundred lines and was not asked to be mechanical, throw it away and re-scope rather than reviewing it.
Give the agent a feedback loop. A single command that builds and tests, documented in the instructions file, so the agent can check itself before you look:
npm run verify # lint + typecheck + unit tests, exits non-zero on failure
Keep a human accountable per merge. Not "the agent wrote it". A name.
None of that replaces the old discipline. Small pull requests still review better. Tests you do not trust are worse than no tests. A system nobody understands still fails at 3am, and the agent will not be paged. Architecture decisions still need to be argued by people who will live with them. If your team was undisciplined before, agents make the mess arrive faster.
What to do next
Take your last ten merged pull requests. Sort them by review time. If the slowest ones are agent-authored, your problem is scope and instructions, not model quality, and you can fix it this week. Write the instructions file, cap diff size, and re-measure in a month.