How to measure an AI workflow before scaling it
Measure the whole path from an agreed task to an accepted result before you expand an AI workflow across a team. Faster code generation is useful only if the work also meets the requirements and can pass review.
This is our proposed pilot method. It is a way to collect evidence in your team, not a claim that every team will get the same result.
What problem should the pilot solve?
Choose one recurring task that matters to delivery. A small bug fix, a test migration, or a change to an existing feature gives you something concrete to inspect. Avoid starting with an entire backlog or a vague goal such as "use more AI".
Write down the current constraint. Is the delay in understanding the codebase, agreeing requirements, implementing the change, or reviewing it? Include the people who define the work and the people who accept it. An engineer's saved hour may otherwise become a reviewer's extra hour.
What should you record?
Use the same definitions before and during the pilot. Keep the record small enough that people will maintain it.
| Measure | Start and finish | What it helps you see |
|---|---|---|
| Delivery time | Task ready to work on → accepted result | Whether useful work reaches the next stage sooner |
| Review effort | Active time spent checking the change | Whether generation shifts work onto reviewers |
| Rework | Changes needed after the first review | Missed requirements, incorrect assumptions, or weak checks |
| Accepted scope | Requirements met, with supporting checks | Whether a faster result did the intended job |
Record waiting time separately from active work. Also note task size, relevant experience, and missing access or context. These can explain a change in the result without attributing it all to AI.
How do you run the comparison?
- Select several comparable tasks and record the team's current method.
- Define what the agent may do, which context it receives, and who checks its work.
- Run the pilot with the same acceptance standards. Record corrections and unsuccessful attempts as well as completed work.
- Review the results with engineering, product, and the people responsible for quality.
- Keep, adjust, or stop the workflow based on the evidence. Write down the reason.
A before-and-after comparison can guide a local decision. It does not isolate the effect of AI: task difficulty, staffing, and practice may also change. Do not turn a small pilot into a company-wide productivity percentage.
Why not use tool activity as the result?
Prompt counts and active seats show adoption. They do not show whether the team delivered the intended change with less effort. Keep them as supporting information rather than the success measure.
DORA's 2025 research treats AI as an amplifier of the wider organization. Its findings support looking at the surrounding delivery system, rather than treating tool access as the whole intervention.
METR's February 2026 update explains selection and measurement problems in its later developer productivity experiment. That is a useful reason to state what your own comparison can and cannot show. Neither source establishes the result of a Harness workshop.
What should happen after the pilot?
Keep one short team instruction: the task boundary, required context, acceptance checks, and the person who owns the final decision. Test that instruction on another task before extending it to another team.
Our free methodology guide explains Delegate, Review, Own. If the pilot exposes a shared skills gap, bespoke AI team training can focus on that gap and your own tasks.
Does every team need the same measures?
No. Choose measures that reflect the actual constraint. A review bottleneck needs review evidence; unclear requirements need evidence that the result matches the agreed scope. Keep the definitions stable during a comparison.
How long should a pilot run?
Long enough to observe several representative tasks, including review and rework. Agree the review date before starting. If the sample is too small or the tasks differ too much, report that limit instead of declaring a win.