Autonomous coding costs must earn their keep

Autonomous coding saves money only when the cost of accepted, production-worthy changes falls below the cost of producing the same outcomes with the team it replaces or frees. Token spend by itself says almost nothing. A cheap agent that produces rejected patches, long review queues, and recovery work can cost more than an expensive agent that lands clean changes.

The useful number is not cost per token, task, pull request, or agent run. It is total incremental cost per accepted change, measured against labor that the business actually avoids. I have seen founders celebrate a low model bill while senior engineers spend half their week untangling agent output. I have also seen a large inference bill make perfect economic sense because two engineers delivered work that previously occupied a much larger team.

You can calculate the boundary before arguing about models. The calculation needs five inputs: accepted changes, human review hours, retries, inference charges, and avoided payroll or avoided labor hours. It also needs a consistent observation window and a rule for quality failures. Without those, the spreadsheet rewards activity and hides the expensive parts.

Break-even starts with a counterfactual

The break-even point compares two credible ways to deliver the same class and volume of work. One side is the baseline process without autonomous coding. The other is the agent-assisted process, including every human and machine cost required to get changes safely into production. If the two sides do not target comparable outcomes, the result is theater.

Define a period, usually a month or a rolling quarter. Record the baseline cost of producing the observed accepted changes with people, conventional tools, and the old operating process. Then record the new cost: retained engineering payroll, model inference, agent infrastructure, human review, retry waste, and failures attributable to the changes. The difference is savings.

The counterfactual must reflect a choice the company could really make. Last year's payroll can be a useful baseline if scope and output remain similar. A hiring plan can be a baseline when autonomous coding lets you avoid approved hires. An estimate of hours can work for a project that would otherwise go to an agency. Do not compare the current bill with an imaginary ten-person team that nobody intended to hire.

There are two valid forms of avoided cost. Avoided payroll is money the company no longer pays or does not begin paying because the operating model changed. Avoided labor is capacity that existing people can spend on other work. The first affects cash directly. The second has economic value only if the freed capacity produces something useful, removes a real bottleneck, or prevents a hire. Calling every saved hour cash savings overstates the result.

Use this monthly cash equation when headcount or contracted spend changed:

net_savings = baseline_delivery_cost
              - current_human_cost
              - inference_cost
              - agent_infrastructure
              - failure_and_recovery_cost

In this ledger, do not add reviewer payroll again if current_human_cost already includes the reviewers. Track review hours as an operating signal, but counting both their salary and their hours as separate costs would charge the same labor twice.

An accepted change is the denominator

An accepted change is a change that passes the team's normal controls, merges, reaches its intended environment, and survives a defined observation window without rollback or material corrective work. The exact window depends on deployment frequency and risk, but the rule must be written before measurement starts. A pull request opened by an agent is inventory, not value.

Raw accepted-change counts can still mislead. Renaming a variable and replacing a payment workflow are both one merge. If the mix changes sharply between the baseline and the measured period, group work into a few stable classes such as small maintenance, normal product change, and high-risk change. Compare costs within each class or assign simple weights that the team agrees on in advance.

Do not let lines of code into the denominator. Agents can generate lines cheaply, and large diffs often demand more review. Story points are also a poor cross-period unit if the team changes how it estimates. Accepted changes are imperfect, but they connect activity to the actual gate where the business receives maintainable software.

A practical acceptance record needs only a few fields:

• change identifier and work class
• first agent run time and merge time
• inference cost across all attempts
• human review and repair time
• deployment status after the observation window

Count a reverted change as rejected until a replacement survives the same window. If an agent creates three pull requests and the third one lands, record one accepted change with all three attempts attached. This rule prevents a busy agent from looking productive by multiplying artifacts.

Teams sometimes worry that this definition punishes experiments. It should. Experiments have a budget and can produce learning, but they do not belong in a production savings claim unless the comparison baseline also includes equivalent experimental work. Keep a separate research ledger and make its purpose explicit.

Pick one ledger before adding costs

A payroll ledger answers whether the new operating model reduces total delivery spend. A unit ledger answers whether one more accepted change has positive contribution after variable costs. Both matter, but combining parts of them produces a false result.

The payroll ledger works best for a founder deciding whether to hire, reduce contractors, or restructure a team. Start with baseline loaded payroll and contractor cost. Subtract current loaded payroll, recurring agent infrastructure, failure recovery, and inference. The inference ceiling is what remains:

maximum_monthly_inference = baseline_delivery_cost
                            - current_human_cost
                            - agent_infrastructure
                            - failure_and_recovery_cost
                            - required_savings_margin

Suppose the credible baseline costs $180,000 per month and the current team costs $75,000. Agent infrastructure costs $15,000, and incident plus recovery work attributable to agent changes costs $8,000. With no required savings margin, inference can rise to $82,000 before cash savings disappear. If management requires $30,000 of monthly savings to justify transition risk, the ceiling falls to $52,000.

The unit ledger is better for choosing models, autonomy levels, or task classes. Estimate the baseline labor cost per accepted change. Subtract review, human repair, expected failure cost, variable agent infrastructure, and inference for that change. The remainder is contribution. Fixed costs then determine how many accepted changes the system must deliver before the month turns positive.

Do not put full current payroll into the unit ledger and also price every review hour unless you are deliberately allocating payroll by activity. Do not claim avoided payroll in the cash ledger and then add the dollar value of freed hours from the same people. Choose what the question requires, document the choice, and reconcile the ledgers at the end of the period.

Retries belong to the change that survives

Every failed attempt is part of the acquisition cost of the accepted change. This includes model calls that time out, patches rejected by tests, loops caused by missing context, abandoned branches, and runs that a human stops because the agent is going the wrong way. Reporting only the final successful run is the inference equivalent of ignoring scrap in a factory.

Attribute retries by work item. Add input tokens, output tokens, cache charges, tool calls billed separately, and any model-specific fees across every attempt. If a retry supports no accepted change, send its cost to an unsuccessful-work bucket. At month end, allocate that bucket across accepted changes or show it separately in the total ledger. Never delete it.

Retry rate alone is not enough because attempts have different prices. Track retry cost ratio:

retry_cost_ratio = inference_cost_before_final_attempt / total_inference_cost

Also track acceptance yield:

acceptance_yield = accepted_changes / work_items_started

A system can have many cheap retries and still work well. It can also have one enormous context-heavy retry that destroys the economics. Dollars connect the behavior to the decision better than attempt counts.

The popular recommendation to reduce cost by routing nearly everything to the cheapest model is often wrong. It is popular because the invoice rate is visible and easy to compare. If the cheaper model raises review time, expands retry loops, or produces changes that fail later, the accepted-change cost rises. Route by total cost for a work class, not by the price printed beside a million tokens.

Put a hard budget on each work item, but do not treat the cap as the optimization. A cap limits loss when the loop breaks. The useful control is an escalation policy: after a known failure condition, provide missing context, switch the model, narrow the task, or send it to a human. Blindly repeating the same prompt at a lower price is still waste.

Human review can erase the advantage

Review time is usually the first hidden cost to overwhelm inference savings. Senior engineers are expensive, and autonomous output can shift their work from implementation to diagnosis. A patch that took the agent four minutes can consume two hours when the reviewer must reconstruct intent, discover an unsafe assumption, and request several revisions.

Measure active review time, not wall-clock time between opening and merging a pull request. Capture time spent reading, testing locally, commenting, repairing agent output, and verifying revisions. Separate waiting time because it affects cycle time but does not consume labor. A lightweight timer or a short field completed at merge is more reliable than asking engineers to remember the month after it ends.

Use loaded hourly cost when building a unit ledger. Loaded cost includes salary, payroll taxes, benefits, and any overhead convention the company already uses for planning. Do not invent a universal multiplier. Finance should supply the number, and the same convention must apply to baseline implementation and current review.

Review compression has a limit. For low-risk, familiar changes, tests and narrow scopes can reduce review. For security boundaries, data migrations, billing logic, and irreversible operations, reducing review may just move cost into incidents. The goal is not minimal human attention. The goal is the least human attention that keeps expected failure cost within the company's tolerance.

Track median review time by work class and the expensive tail. Averages hide the patch that consumes a day. I use the 90th percentile as a diagnostic even when the financial model uses total hours, because a growing tail often exposes missing context, poor task boundaries, or an autonomy level that the repository cannot support.

Include the preparation that makes review possible. When an engineer cleans the branch, reruns a flaky test, reconstructs a missing migration, or writes a summary because the agent did not explain its choice, that time belongs to the change. Teams often record only minutes spent inside the review screen. The result makes review look cheap while the same work appears as an unexplained drop in engineering capacity.

Reviewer utilization changes the cash meaning of those hours. If the company pays the reviewer a fixed salary and has spare capacity, an extra hour may not create an immediate cash payment. It still consumes capacity and can delay higher-value work. If review forces overtime, contractor help, or another hire, the cash effect becomes direct. Keep the measured hours stable, then let the chosen ledger decide how to value them.

Approval rate needs the same skepticism as acceptance count. A high approval rate can mean that agents produce good changes, or that reviewers have become permissive under queue pressure. Pair it with recovery cost, rollback count, and review time by work class. When approval rises while review time falls and recovery work rises, the process probably removed scrutiny rather than waste.

Review time also reveals when an organization has moved rather than removed a bottleneck. If agents generate changes faster than qualified reviewers can accept them, work piles up, branches age, and conflicts grow. More inference then buys more queue, not more delivery. Stop increasing concurrency until accepted changes rise without a matching increase in review hours.

Quality costs arrive after the model bill

A merged change has not earned its savings if it causes a rollback, support load, security repair, or unplanned follow-up. Quality costs lag behind inference, so a same-day dashboard will always flatter autonomy. Use an observation window and revise the ledger when delayed failures appear.

Expected failure cost works when incidents are infrequent but material:

expected_failure_cost_per_change = failure_probability
                                   * average_recovery_cost

Estimate both terms from your own records and segment them by work class. Include engineer recovery time, emergency vendor spend, credits or direct customer remediation when finance can attribute them, and the cost of the replacement change. Do not invent reputational dollars to make the formula look complete. Explain unpriced risks beside the number instead.

Consider a familiar failure. An agent updates an authorization check and produces a clean diff. Tests pass because fixtures cover authenticated and unauthenticated users but not cross-tenant access. A reviewer scans the small patch in twelve minutes and approves it. Two days later, a support report triggers investigation, two engineers interrupt planned work, the team rolls back, writes the missing test, repairs the code, and reviews it again. The original ledger recorded one cheap accepted change. The corrected ledger records a rejected attempt, its inference and review cost, recovery labor, and one later accepted replacement.

This distinction between syntactic acceptance and economic acceptance matters. A merge proves that the process allowed a change through. Economic acceptance means the change delivered its intended outcome without costs that reverse the claimed saving. Teams that blur the two optimize approval throughput while accumulating repair work.

Do not wait months to learn anything. Use a short provisional window for ordinary changes and a longer review for delayed effects, then mark the recent period as incomplete. Finance already understands accruals and late adjustments. Engineering economics deserves the same honesty.

The spend ceiling changes with volume

The maximum affordable inference spend is not one permanent company number. It moves with accepted volume, work mix, review time, fixed platform cost, and the amount of labor genuinely avoided. A model can be profitable at 120 accepted changes per month and unprofitable at 50 because fixed costs are spread across fewer outcomes.

Take an assumed unit example. Before autonomy, a normal product change requires 6.5 engineering hours at a loaded rate of $140, so the baseline labor cost is $910. With autonomy, each accepted change uses $185 of inference, 1.4 hours of review worth $196, and an expected $35 of recovery work. Monthly agent infrastructure is $24,000.

The variable contribution per accepted change is $494:

910 baseline labor
- 185 inference
- 196 review
-  35 expected recovery
= 494 contribution before fixed cost

The system needs 49 accepted changes to cover $24,000 of fixed cost because 24,000 divided by 494 is 48.6. At 120 accepted changes, contribution after fixed cost is $35,280. This example does not promise that any team will see those rates. It shows exactly where local measurements belong.

Now solve for the inference ceiling instead of using a fixed model price. At 120 accepted changes, fixed cost contributes $200 per change. Baseline value is $910, review costs $196, and expected recovery costs $35. Maximum inference is therefore $479 per accepted change, or $57,480 for the month. Spending above that removes all modeled savings.

At 50 accepted changes, fixed cost allocation rises to $480 per change. The inference ceiling falls to $199 per accepted change, or $9,950 for the month. A finance team looking only at the $82,000 cash-ledger ceiling from a different scenario could approve far too much. The operational unit model explains why volume and yield matter.

Run sensitivity ranges rather than arguing over one forecast. Calculate the result at expected, weak, and strong acceptance yield; then vary review hours and recovery cost. If savings disappear after a small change in one assumption, the system is fragile. Keep autonomy narrow until the observed distribution improves.

A small calculator makes the decision auditable

The calculator below uses the unit ledger and prints the measures a founder needs: contribution before fixed cost, the number of accepted changes required to cover fixed cost, total inference ceiling at the observed volume, and net savings. Replace the sample inputs with one work class from your own ledger.

from math import ceil

accepted_changes = 120
baseline_hours_per_change = 6.5
loaded_hourly_cost = 140
review_hours_per_change = 1.4
repair_hours_per_change = 0.25
inference_per_accepted_change = 185
fixed_monthly_agent_cost = 24000
required_monthly_savings = 0

baseline_value = baseline_hours_per_change * loaded_hourly_cost
review_cost = review_hours_per_change * loaded_hourly_cost
repair_cost = repair_hours_per_change * loaded_hourly_cost
non_inference_variable = review_cost + repair_cost
contribution_before_fixed = (
    baseline_value
    - non_inference_variable
    - inference_per_accepted_change
)

if contribution_before_fixed <= 0:
    break_even_changes = None
else:
    break_even_changes = ceil(
        (fixed_monthly_agent_cost + required_monthly_savings)
        / contribution_before_fixed
    )

inference_ceiling_total = max(
    0,
    accepted_changes * (baseline_value - non_inference_variable)
    - fixed_monthly_agent_cost
    - required_monthly_savings
)
actual_inference_total = (
    accepted_changes * inference_per_accepted_change
)
net_savings = (
    accepted_changes * baseline_value
    - accepted_changes * non_inference_variable
    - actual_inference_total
    - fixed_monthly_agent_cost
)

print({
    'contribution_before_fixed': contribution_before_fixed,
    'break_even_changes': break_even_changes,
    'inference_ceiling_total': inference_ceiling_total,
    'actual_inference_total': actual_inference_total,
    'net_savings': net_savings,
})

With the sample inputs, the output shape and values are:

{'contribution_before_fixed': 494.0, 'break_even_changes': 49, 'inference_ceiling_total': 57480.0, 'actual_inference_total': 22200, 'net_savings': 35280.0}

The calculator assumes each accepted change has comparable baseline value. Run it separately for different work classes if that assumption fails. It also treats repair hours as the expected quality cost; replace that input with a fuller expected recovery amount when you have incident records.

Required savings belongs in the model. A company may demand a margin because the new process carries transition risk, concentration risk, or management overhead that the simple ledger does not price. Setting required_monthly_savings to zero finds accounting break-even. Setting it to the minimum benefit the company expects finds the operating threshold that should govern model budgets.

This is the arithmetic I use in a Team & AI Audit at oleg.is: first make the cost boundary explicit, then test whether the current workflow can stay below it. A polished agent demo cannot substitute for accepted changes and a reconciled ledger.

Manage autonomy by the curve, not the invoice

A monthly model bill tells you what you spent, while the cost curve tells you whether another dollar is likely to buy accepted work. Plot cumulative inference against cumulative accepted changes for each work class. Add review hours on the same time axis. When inference rises faster than acceptance or review hours bend upward, concurrency or autonomy has passed the useful range.

Set budgets at three levels. Give each work item a loss cap, each work class an inference ceiling per accepted change, and the whole program a monthly ceiling based on avoided cost. These controls answer different failures. The task cap stops runaway loops, the class ceiling stops an unsuitable use case, and the monthly ceiling protects the business case.

Make stop conditions concrete. Pause or narrow a work class when its trailing accepted-change cost exceeds the ceiling for two measurement windows, when the review queue grows despite stable demand, or when delayed recovery costs erase the required savings margin. Resume only after changing the context, tests, routing, task boundary, or human gate that caused the loss. Merely resetting the budget starts the same failure again.

Do not chase the lowest possible inference spend. A higher-capability model can reduce total cost if it needs fewer retries and less review. More context can be cheap if it prevents a wrong architectural assumption, or wasteful if every attempt reloads irrelevant history. The decision belongs at the accepted-change level, where all those effects meet.

The first month should produce a range, not a victory claim. Reconcile invoices, review logs, accepted changes, and recovery work. Compare the result with the counterfactual that finance and engineering agreed before the trial. Then fund only the work classes that retain a margin under weak assumptions.

Autonomous coding stops saving money at the point where inference plus the human and failure costs needed to accept its output consume the labor it truly avoids. That point is measurable. If nobody can show the accepted-change ledger, the company does not know whether it has automation or an unusually entertaining expense account.