The cost of an accepted code change is more than coding

Most engineering estimates stop at the wrong boundary. They price the typing, perhaps the review, then declare a change complete when it merges. The business pays until that change is specified clearly enough to build, passes the pipeline, reaches production, survives its first real traffic, and stops generating follow up work.

That is why an AI assistant can appear to make a team twice as fast while payroll barely moves and incident load rises. Faster code generation matters, but code generation is only one cost center inside delivery. If you want an honest comparison between human only delivery and assistant led delivery, measure the cost of an accepted code change.

An accepted change is a change that has met an agreed outcome in production, with its operational consequences understood. This definition is stricter than "merged" and more useful than "deployed." It forces the estimate to include the work that founders usually discover only after a missed date: clarification, review repair, failed builds, release work, rollback preparation, monitoring, support, and rework.

Coding is usually the smallest visible slice

Writing the first implementation is often the easiest portion to see because it happens in an editor and has a named owner. The other work is dispersed across product conversations, pull request comments, CI logs, release channels, and support messages. Dispersed work still consumes paid time.

Take a modest request: add a field to an account settings page and include it in an API response. The apparent task is a form control, validation, database storage, and an endpoint update. The accepted change may also require a decision about existing accounts, an API versioning decision, permissions checks, tests for invalid values, review of a migration, a dashboard query update, release notes, a staged rollout, and a support response when an integration sees an empty field.

None of that means every small feature needs a committee. It means the team should stop pretending that all those decisions are free because they happen outside the coding ticket.

DORA defines change failure rate as the share of deployments that require immediate intervention, and separately tracks unplanned deployment rework. Those definitions matter here. A hotfix and a rollback are not bad luck outside the estimate. They are delivery work created by an earlier change.

The distinction people blur is simple: implementation cost is not acceptance cost. Implementation ends when a candidate solution exists. Acceptance ends after the organization has evidence that the candidate produces the promised result without unacceptable damage. If you compare one delivery method by implementation cost and another by acceptance cost, the comparison is worthless.

Define one finish line before you price anything

A team needs a written definition of accepted before it collects numbers. Do not make it abstract. Put it in the engineering handbook and attach it to normal change classes.

For a normal production change, I use this finish line:

1. The requester and delivery owner agree on the intended behavior, exclusions, and success signal.
2. The change passes the required automated checks and receives the required review.
3. The team deploys the intended artifact through its normal release path.
4. The team verifies the relevant production signal during a defined observation period.
5. The change creates no unresolved support issue, regression, or follow up deployment caused by the release.

The observation period does not need to be long for every change. A copy change may need a few minutes of smoke checks. A billing rule, background job, permission change, or migration may need a full business cycle. What matters is that the team declares the period before it ships, rather than treating silence as proof of success.

This is also where teams need change classes. Do not compare a low risk text correction to a schema migration and call the average a productivity metric. Start with four practical classes:

• routine product behavior with no data migration
• integration or API behavior
• infrastructure and reliability work
• changes involving money, permissions, or irreversible data effects

The classes do not need separate bureaucracy. They give you enough context to spot a distorted average. A team that ships many cosmetic changes can claim an excellent average while its risky changes take weeks and regularly come back as incidents.

Google's SRE Workbook makes a related point in its canary release guidance: testing cannot cover every production condition, and production traffic exposes defects that preproduction systems miss. That is why a safe release process includes observing real behavior rather than treating a green test suite as the finish line.

Use a loaded formula that exposes waiting and rework

The calculation does not need a finance department or minute by minute surveillance. It needs a consistent unit and an explicit place for costs that teams usually hide.

Use this formula for each accepted change:

accepted_change_cost =
  specification_cost
+ implementation_cost
+ review_and_repair_cost
+ ci_cost
+ deployment_and_verification_cost
+ rollback_preparation_cost
+ support_and_rework_cost
+ shared_delivery_overhead

Each direct labor component is the sum of a person's loaded hourly cost multiplied by active time. Shared delivery overhead is the portion you allocate from systems that exist for every change: CI runners, monitoring, security scanning, artifact storage, release tooling, and the engineers who maintain them.

Do not turn elapsed calendar time directly into labor cost. A change that waits two days for review does not cost two days of one engineer's salary. It does carry a real economic cost because it delays revenue, learning, and dependent work. Track active time and elapsed wait as separate columns. Blending them produces a number that tells you neither the payroll cost nor the flow problem.

A minimal record can live in a spreadsheet, a warehouse table, or a small internal service:

change_id, class, outcome,
spec_minutes, build_minutes, review_minutes, repair_minutes,
ci_runs, ci_runner_cost, ci_investigation_minutes,
deploy_minutes, verification_minutes, rollback_minutes,
support_minutes, rework_minutes, elapsed_hours

Add assistant_mode with values such as none, drafting, test_generation, or agent_workflow. Do not add it to judge individual engineers. Add it so you can compare the same class of work across delivery modes.

Here is a worked example using deliberately simple figures. A product manager spends 30 minutes clarifying scope. An engineer spends 90 minutes implementing and testing. A reviewer spends 25 minutes, and the engineer spends 20 minutes responding to review. The pipeline takes 18 minutes of runner time, then a flaky integration test costs 15 minutes to investigate and rerun. Deployment and production checks take 30 combined minutes. A support engineer spends 20 minutes answering a customer question caused by a changed default. The team has a small allocated per change cost for shared systems.

The candidate code took 90 minutes. The accepted change consumed 230 active human minutes before shared infrastructure cost. If an assistant reduced implementation from 90 minutes to 45 but created an extra review round and another flaky test investigation, the team did not cut the change cost in half. It cut one component and may have increased another.

This is the comparison founders need. It makes assistant investment a delivery economics question, not a demo contest.

Specification work cannot be delegated to autocomplete

Assistants can turn a vague request into plausible code very quickly. That is exactly why vague requests become more expensive. A human engineer who writes slowly may stop and ask what "delete" means for invoices. An assistant will happily produce endpoints, database mutations, and tests for the first interpretation it receives.

Specification work includes deciding behavior, invariants, exclusions, ownership, and evidence of success. It also includes finding the hidden constraints: existing clients, data already in production, permission rules, compliance commitments, operating limits, and fallback behavior.

Put a small change contract at the top of the ticket or pull request:

Intent: Existing workspace owners can set a default export format.

Must remain true:
- Existing API clients receive the current format unless an owner changes it.
- Members cannot change the workspace default.
- A failed export does not alter the saved default.

Evidence after release:
- Successful export rate stays within the normal range.
- Permission-denied responses do not increase.

Rollback condition:
- Revert the application release. The migration only adds a nullable column.

That artifact is more useful than a page of prose. It tells the engineer what to build, tells the reviewer what to challenge, tells the release owner what to watch, and tells support what should be true for existing users.

Assistant led delivery should reduce the cost of turning an approved contract into a candidate implementation. It should not erase the contract. The popular recommendation to "let the agent inspect the repository and figure it out" is attractive because it removes an uncomfortable product conversation. It is wrong for consequential work. Repository context explains how the system currently behaves. It does not decide what the business should promise next.

A practical rule: if the engineer cannot state the invariant in one or two sentences, do not ask an assistant to write the migration or the authorization logic. Resolve the ambiguity first. You will spend less time than you would cleaning up a believable but incorrect patch.

Review is a control loop, not a ceremonial approval

Review cost has three parts: the reviewer's reading time, the author's response time, and the cost of discovering that the proposed change was too broad or based on a wrong assumption. Teams usually count only the first part because it appears in a pull request system.

Assistant generated patches change the shape of review. They often make a reviewer inspect more code in less familiar patterns, including generated tests that assert the implementation instead of the intended behavior. A 600 line patch may be valid, but it should not receive the same review treatment as a 60 line patch simply because an assistant drafted it quickly.

Reviewers should ask five concrete questions:

• Does the change satisfy the stated contract, including what remains unchanged?
• Did the patch widen the scope beyond the request?
• Can the tests fail for the defect we care about, or do they only repeat the code's assumptions?
• Does the operational behavior match the release and rollback plan?
• Can a maintainer understand and modify this six months later?

The fastest way to reduce review cost is not to demand faster approvals. It is to reduce ambiguity and patch size. Keep assistant tasks narrow enough that a reviewer can identify the intended behavior and its boundaries without reverse engineering a large generated diff.

Measure review repair separately from implementation. When review repair rises after assistant adoption, do not dismiss it as "normal learning." Find the source. The assistant may lack repository instructions, the requests may be underspecified, reviewers may be receiving overlarge patches, or the team may be using the assistant to avoid design decisions.

GitLab's pipeline documentation describes stages as sequential groups, with jobs within a stage able to run in parallel. That is a useful delivery model, but human review usually becomes an unmeasured sequential stage ahead of it. If review waits dominate elapsed time, buying more CI capacity will not improve the delivery path.

CI belongs in the change cost even when the runner bill is tiny

CI cost is not just compute. It includes every failed job someone investigates, every slow test that delays feedback, every pipeline rule that produces a false sense of safety, and every engineer who maintains build images, test fixtures, secrets, caches, and runners.

The runner invoice may be small compared with payroll. The delay and interruption are not. A ten minute failure that forces an engineer to abandon another task can cost more than the compute that found it.

Record CI in three buckets:

1. Execution cost: runner minutes, storage, and paid external checks.
2. Human handling cost: time spent reading failures, retrying, repairing tests, and fixing pipeline configuration.
3. Confidence gap cost: production defects that passed CI because the checks did not test the relevant behavior.

The third bucket is uncomfortable because it requires linking a release problem back to missing or misleading checks. Do it anyway. Otherwise a team celebrates a cheap pipeline while support handles defects that a better contract test or migration test could have caught.

GitLab documents that later stages such as test and deploy do not run after an earlier stage fails. That sequential gate is useful only when the gate is trustworthy. A flaky test turns a safety control into a random delay. A test suite that never catches real regressions turns a safety control into theater.

Assistants can help here, but use them carefully. They are good at drafting test cases from an explicit contract, identifying untested branches, explaining a failure log, and maintaining repetitive pipeline configuration. They are poor substitutes for deciding what behavior matters. A generated test that only confirms an internal helper was called may improve coverage and do nothing for acceptance risk.

Track the ratio of CI failures caused by product defects, test defects, environment defects, and configuration defects. If environment and test failures dominate, your team has a delivery system problem, not a developer discipline problem.

Deployment and rollback are design work

A deployment is not a button click at the end of development. It is the moment a change meets the data, traffic patterns, permissions, integrations, and operational limits that the test environment approximated.

The Google SRE Workbook recommends exposing a change to a limited, time bound portion of production traffic and evaluating it before wider rollout. The point is not ceremony. A canary limits the impact while you obtain evidence from real traffic.

The delivery cost should include preparing the release, selecting the rollout path, checking the relevant signals, and recording the outcome. For a low risk web change that may be a short automated smoke check and a dashboard glance. For a migration, it may mean a compatible schema change, backfill monitoring, a feature flag, and a documented recovery plan.

Do not call a change rollbackable because the deployment tool shows a rollback action. GitLab's deployment documentation is explicit: its rollback redeploys a prior deployment, and the deployment script must define what that means. It also warns that jobs needed to regenerate deployment artifacts may require manual execution. That is the difference between an interface control and a recovery plan.

A rollback plan should answer these questions before release:

• Which artifact, configuration, and data state will change?
• Which parts can be reverted by redeploying old code?
• Which migrations are additive, reversible, or irreversible?
• What signal triggers rollback, and who has authority to act?
• What customer or partner state remains changed after code rolls back?

A common failure looks like this. The team adds a new required state to a workflow, deploys code and a migration, then finds an integration that still creates records without that state. The application rollback succeeds, but the data migration changed validation rules and the old version cannot handle the new records. The team now writes a hotfix under pressure. The original change did not cost one deployment. It cost the original release, diagnosis, hotfix, extra review, another release, and support communication.

Assistant tools often make it easier to produce the migration and the rollback script. They do not make the data lifecycle reversible. Treat generated infrastructure and migration code as high scrutiny work because a concise patch can change a large amount of production state.

Support is feedback on delivery quality

Support cost belongs in the accepted change cost when the support work is caused by the release. This does not mean every customer conversation becomes an engineering charge. It means the organization should connect change records to the first wave of confusion, broken workflows, integration issues, and defect reports that follow.

Use a simple attribution rule. Link an issue to a change when a reasonable investigator can say that the change introduced the behavior, removed expected behavior, changed a documented default, or made a previously working workflow fail. Record the initial support time and the engineering rework time separately.

Support cases reveal costs that review and CI often cannot see:

• users misunderstood a new default or control
• an undocumented API consumer relied on old behavior
• a slow path appeared only with real account sizes
• an alert fired but lacked enough context to identify the changed release
• the release created work for operations or finance teams

The answer is not to charge support against engineers as punishment. That produces defensive behavior and bad data. Use the connection to improve change contracts, observability, release notes, and rollout practices.

DORA's use of failed deployment recovery time is useful because it measures the period from a failed deployment to restored service, not merely the time until someone acknowledges an alert. But for cost control, you also need the human work after restoration: root cause analysis, customer follow up, cleanup, and preventative changes. Restoring service ends the urgent part. It does not settle the invoice.

Compare assistant led delivery against the same acceptance standard

The fair comparison has three columns: human only delivery, human delivery with assistant help, and agent workflow delivery. Each column must use the same request class, definition of accepted, review threshold, test standard, deployment path, and observation window.

Do not compare an assistant's first draft with a human's finished change. Do not compare a human change that passed full review with an agent patch that a tired engineer skimmed. Those are demonstrations, not measurements.

For each class, calculate these measures over a sample of completed changes:

median active cost per accepted change
median elapsed time to acceptance
rework cost per accepted change
support and incident cost per accepted change
share of changes accepted without a follow-up deployment

Median is often more honest than average because a small number of major incidents can distort an average. Keep the outliers, though. An outlier is often where the delivery process has a serious design flaw.

Then inspect the cost movement by stage. If assistant help cuts implementation by 40 minutes but review repair rises by 30 minutes and support rises by 20, the team did not gain capacity. If the same assistant cuts implementation, reduces CI repair through better tests, and leaves production rework flat or lower, you have a real improvement.

The temptation is to measure output: commits, pull requests, lines changed, tickets closed. Those measures become dangerous when assistants increase the supply of code. More candidate code can simply push the bottleneck into specification, review, and operations.

The constraint is different in every company. In some startups, founders are the bottleneck because requirements arrive incomplete. In others, a single experienced reviewer holds operational knowledge. In others, CI takes too long or releases require manual coordination. An assistant program should target that actual constraint, not the most impressive demo.

At AppMaster.io, moving to a small AI augmented engineering group only makes economic sense if the group can own the full path to stable production. Replacing drafting work while leaving release, reliability, and support work unmanaged merely moves cost into a smaller team with less slack.

Build the measurement system from evidence you already have

Most startups should not start with time tracking software. Start by reconstructing twenty to thirty recently accepted changes from the records you already produce: ticket history, pull requests, CI logs, deployment records, incident notes, and support conversations.

Choose a mix of change classes. For each one, ask the owner to estimate active effort after the fact in broad bands such as under 30 minutes, 30 to 90 minutes, 90 minutes to half a day, half a day to a day, and more than a day. Broad bands are enough to expose where the money goes, and they create less false precision than seven minute entries.

After the reconstruction, publish a one page result for leadership:

• median accepted change cost by class
• active effort by delivery stage
• elapsed wait by delivery stage
• top three recurring rework causes
• the one constraint that prevents safe faster delivery

Do not present a single blended cost as a universal rate card. The aim is to decide what to fix. If review repair dominates, improve contracts and patch boundaries. If CI investigation dominates, stabilize the suite and pipeline. If deployment and support dominate, invest in release visibility, flags, migration discipline, and operational ownership.

A Team & AI Audit is useful when the evidence is scattered across tools and people cannot agree where delivery time disappears. The output should be a cost model tied to your actual delivery path, not a generic claim that assistants write code faster.

The first number will be imperfect. That is fine. A consistent imperfect model that includes acceptance, release risk, and rework is more useful than a precise coding estimate that stops before the customer sees the change. Once the team can see the whole cost, it can decide where AI assistance earns its place and where a human decision still saves the most money.