Define what you are trying to improve
Before choosing a metric, choose the outcome. A team may need shorter lead time, less engineering effort, more reliable releases or the capacity to tackle work that previously remained in the backlog.
These outcomes are related but not interchangeable. An agent can produce more code while review becomes slower. A task can use fewer engineering hours without reaching production sooner because an external dependency remains unchanged.
Write down the workflow, its start and end points, the accepted output and the quality criteria. That definition is the basis for a meaningful comparison.
Build a comparable baseline
Use recent, representative work where possible. Record scope, complexity, team experience, tool access and dependencies. If a historical comparison is weak, say so rather than present it as a controlled experiment.
For a modernization slice, distinguish discovery, implementation, validation and migration. For a delivery-team pilot, include the people who review and support the work, not just the person generating code.
The baseline should make clear which effort would have occurred without the pilot and which setup costs are new.
Count the complete effort
Include requirements clarification, context preparation, implementation, testing, review, rework and integration. Track training and workflow setup separately so buyers can see both the pilot investment and the recurring cost.
Do not count the same accepted output twice because an agent produced several drafts. Failed attempts and the work needed to repair them are part of the process being evaluated.
A simple efficiency comparison is baseline total engineering effort divided by pilot total engineering effort for comparable accepted output. The calculation is only useful when scope and quality are sufficiently comparable.
Keep quality beside speed
| Dimension | A useful question |
|---|---|
| Accepted output | Did the work satisfy the agreed requirement? |
| Total effort | How much human engineering time did the complete workflow consume? |
| Lead time | How long did the work take from agreed start to acceptance? |
| Review and rework | How much correction and verification did it require? |
| Quality | What defects, regressions or unmet criteria were found? |
| Repeatability | Can the team reproduce the workflow without exceptional support? |
Use an observation period appropriate to the system. Acceptance on release day may not reveal every operational problem. Explain what has and has not yet been observed.
Interpret a 5× target carefully
PeakROAI’s pilot ambition is 5× greater engineering efficiency in selected workflows: comparable accepted output using one-fifth of the baseline total engineering effort, while meeting agreed quality criteria.
This is a target, not an achieved-result claim or a promise for every engagement. Review, rework and integration effort count. Delivery time is reported separately. A result in one workflow should not be multiplied across a whole program without evidence.
Make the pilot decision explicit
Finish with a recommendation supported by the observations: expand the workflow, refine it and repeat, apply it only to a narrower task class, or stop. Identify what must change before broader adoption.
The value of a pilot is not limited to a positive number. It should help the organization understand where agentic engineering works, what it costs to operate and which responsibilities remain with its people.
Explore team enablement and a measured pilot.