AI technical debt in code written by agents

AI technical debt is the maintenance cost that accumulates when code agents optimize for a passing task while the team remains responsible for the system. The problem is not that a machine typed the code. The problem is that speed can move uncertainty out of implementation and into review, testing, operations, and the next change. If you measure only delivery volume, that transfer looks like productivity.

I have seen this pattern in ordinary human teams too, but agents amplify it. They can repeat a weak abstraction across twenty files before a reviewer notices the first copy. They can produce convincing tests that confirm the implementation rather than the requirement. They can also remove boring debt faster than a person when the task, boundaries, and verification are explicit. Treating all generated code as suspect is lazy. Treating a green build as proof of maintainability is worse.

A useful measurement system therefore ignores authorship as a verdict. It watches what the repository and the team must carry after a change lands: duplicated decisions, widening interfaces, dependency churn, review effort, fragile tests, unclear ownership, and repairs that recur. Measure those residues every month, compare them with a stable baseline, and assign work when a signal crosses a threshold. That turns a vague argument about AI quality into an engineering control loop.

Measure the residue, not the author

Technical debt describes a future cost created by a present design choice. Martin Fowler's technical debt quadrant adds a distinction that teams still need: debt can be deliberate or inadvertent, and prudent or reckless. Code written by an agent does not create a fifth category. It changes how quickly a choice can spread and how easily nobody feels ownership of it.

Start by separating provenance from condition. Provenance asks whether a person, an agent, a generator, or a migration produced a line. Condition asks whether that line makes the next safe change harder. Only the second question tells you whether repair deserves time. A generated client derived from an API schema may be large and repetitive yet cheap to replace. A hand edited authorization helper may contain ten elegant lines and still be dangerous debt. Raw generated line count gets that comparison backward.

This distinction has an operational consequence. Do not create an ai_debt bucket where every unpopular change goes to die. Use the same maintainability dimensions across the repository, then retain agent involvement as one diagnostic label. ISO/IEC 25010 breaks maintainability into modularity, reusability, analysability, modifiability, and testability. Those names are more useful than a single quality grade because they tell you which future task will hurt.

For each merged change, record three facts that most source control systems can preserve without guesswork: whether an agent materially produced it, which model or workflow produced it, and who accepted responsibility for the result. The responsible engineer matters more than the prompt transcript. Prompts help reproduce a failure, but ownership gets it repaired. Do not infer provenance later from writing style. That produces noisy data and invites arguments about detection instead of code.

The measurement unit should be a repository or service with one accountable owner. Teamwide averages conceal the component that is getting worse. File level scoring creates thousands of alerts and no decision. A service level view is usually narrow enough to act on and broad enough to include code, tests, configuration, and operational behavior.

Generated code leaves recurring signatures

Code from agents often fails in recognizable shapes, even though none of them proves that an agent wrote it. The useful question is whether those shapes are rising after the workflow changed. Look for clusters, not a magic detector.

The first signature is local correctness with global duplication. An agent sees the current file and writes a helper that works there, while a similar helper already exists elsewhere. Simple clone detection catches literal copies. The more expensive form is semantic duplication: several validators encode the same business rule with slightly different names and edge cases. Track duplicate blocks with a tool such as jscpd or PMD CPD, then review the largest new clusters. Do not set a target of zero. Generated clients, fixtures, and migrations need explicit exclusions so the signal describes code people maintain.

The second signature is abstraction spray. One task adds wrappers, adapters, factories, and configuration switches that each have one caller. Agents tend to satisfy anticipated flexibility because it sounds safe in a prompt. It is not free. Count new public symbols, exported types, feature flags, and files per completed change. A rising ratio of public surface to delivered behavior deserves inspection, particularly when most symbols have one reference. Static analysis can find unused exports; a small repository query can find single caller abstractions. Neither should auto delete code. They create a review queue.

The third signature is exception swallowing. Generated handlers frequently catch broad errors, log a plausible message, and return a default value. The happy path passes while production loses the failure's type and context. Search for broad catches, empty catches, ignored promise rejections, and fallback returns added in changed code. Language linters already expose many of these patterns. Measure accepted suppressions too, because an agent can make a warning disappear by adding an ignore comment.

The fourth signature is comment confidence that exceeds code evidence. Long comments may claim idempotency, thread safety, validation, or fallback behavior that no test establishes. Comment volume itself is not debt. A claim without an executable check is. During review, tag claims about concurrency, security boundaries, destructive operations, and compatibility. Require a test or remove the claim. Over time, count how many claim tags reviewers raise and how many return in later changes.

The fifth signature is style compliant inconsistency. The code passes formatting and lint rules, but neighboring modules use different error types, pagination conventions, time units, or transaction boundaries. Generic quality scanners rarely see this. Architecture tests and repository specific lint rules can. Every repeated review comment is a candidate for an executable local rule. If reviewers have written "use the domain error here" five times, the process has already paid for the rule without receiving it.

A baseline keeps the numbers honest

A debt metric without a baseline becomes a morality score. Establish the baseline before changing the agent workflow when possible, or reconstruct it from several months of repository history. The baseline should describe normal variation, not an imagined clean state. Old code contains debt; your immediate concern is whether the rate and repair burden are changing.

Choose a fixed set of maintained services and exclude archived repositories, vendored code, generated artifacts, lockfiles, snapshots, and one time migrations. Store the exclusion file beside the measurement script and review changes to it like code. Otherwise every uncomfortable result will produce a new exclusion, and the trend will improve while the system does not.

Record both stock and flow. Stock is what exists now: duplicate blocks, open suppressions, dependency age, cycle count, public symbols with one caller, and tests coupled to internals. Flow is what changed this month: new debt signals, signals removed, rework after merge, and reviewer interventions. Stock tells you the carrying cost. Flow tells you whether current behavior adds to it. A team can have ugly legacy stock and healthy flow, or a polished repository whose flow is deteriorating fast. Those require different decisions.

Normalize only where the denominator has meaning. Findings per thousand maintained lines can help compare one service with itself, but lines of code do not represent business value or cognitive load. Findings per merged change can reveal workflow drift, but a large migration and a one line fix are not equivalent. Keep absolute counts beside normalized rates. When they disagree, inspect examples rather than picking the friendlier chart.

Use a rolling median for noisy human measures such as review time and revision count. Keep the distribution visible. One incident response change may wait in review for a day because the reviewer was asleep; the median resists that accident. A narrow median can still hide a painful tail, so record the 90th percentile or simply list the five slowest changes with their causes. The purpose is diagnosis, not statistical theater.

Do not compare developers by debt score. People get different work, and attribution becomes murky when one person prompts, another reviews, and a third repairs. Once a metric becomes an individual target, people learn to split changes, suppress findings, or avoid risky components. Compare a service with its own history and ask its owner to explain material movement.

The monthly routine takes one working session

Run the routine on the same date, against the same branches, with pinned tool versions. Monthly measurement is frequent enough to catch drift and slow enough to avoid turning normal daily movement into alerts. A repository owner and one engineering leader can complete the first pass in a focused session; investigation and repair estimates come afterward.

1. Freeze the sample. Record the default branch commit, the date range, included paths, exclusion file hash, and tool versions. Export the month's merged changes and their review events.
2. Run structural checks. Collect duplication, dependency, cycle, lint suppression, complexity, dead export, and test isolation findings. Save machine readable results, not screenshots.
3. Join delivery evidence. Add change size, time to first substantive review, revision rounds, post merge fixes, rollback or revert events, and linked incidents. Label agent involvement from the workflow record.
4. Sample the code. Inspect the largest new finding in each category plus a random set of clean changes. The clean sample catches blind spots in the automated checks. Record false positives rather than silently deleting them.
5. Decide work. Compare stock and flow with the baseline, name the component owner, open bounded repair items, and set an expiry date for accepted exceptions. Publish the scorecard with short notes on causes.

A small shell artifact makes the structural part reproducible. This example records the commit, counts tracked code, finds suppression markers, and emits a tab separated file. Adapt the extensions and exclusions to the repository, then pin the script in source control.

#!/usr/bin/env bash
set -euo pipefail
commit=$(git rev-parse HEAD)
code_files=$(git ls-files '*.ts' '*.tsx' ':!:vendor/**' ':!:dist/**')
printf 'metric\tvalue\tcommit\n' > debt-signals.tsv
printf 'tracked_code_files\t%s\t%s\n' "$(printf '%s\n' "$code_files" | sed '/^$/d' | wc -l | tr -d ' ')" "$commit" >> debt-signals.tsv
printf 'suppression_markers\t%s\t%s\n' "$(printf '%s\n' "$code_files" | xargs grep -E 'eslint-disable|ts-ignore|TODO|FIXME' 2>/dev/null | wc -l | tr -d ' ')" "$commit" >> debt-signals.tsv
npx jscpd -l 8 -k 60 -r json -o reports .

Its output shape is intentionally boring: metric, value, and commit, followed by rows such as suppression_markers, 37, and a commit hash. The jscpd JSON remains a separate artifact because a count without file locations cannot support repair. In a real pipeline, replace npx with a pinned local executable. A moving analyzer version can create a trend even when the code does not change.

The routine does not need a central data warehouse at first. Store timestamped artifacts in a restricted build bucket or an internal repository, then generate a compact scorecard. Automation should reproduce facts. A human still decides whether a finding is debt, an intentional tradeoff, or a faulty rule.

Review tax belongs on the scorecard

Agents can reduce typing time while increasing the time needed to establish trust. That review tax is part of AI technical debt when it persists across later changes. Measure it separately from generic cycle time so a faster first draft does not conceal slower acceptance.

Time to first review is mostly a scheduling measure. Time from first substantive comment to approval tells you more about comprehension and correction. Capture revision rounds, distinct review comments that request behavior changes, and the number of files a reviewer must reopen after an update. Exclude bot comments and simple acknowledgments. Repository APIs can provide timestamps and review events, but the classification needs a small, documented rule set.

Add a recurring correction rate. Search merged changes for follow ups that repair, revert, or clarify work from the prior month. Link them to the originating change when evidence exists. Do not label every nearby edit as rework. A clear link can come from a revert reference, an issue relationship, a commit message, or reviewer confirmation. The manual effort is worthwhile because false rework labels destroy trust in the scorecard.

Review comments are also product data for the agent workflow. Classify repeated interventions into a short taxonomy: missed repository convention, incorrect requirement, unsafe fallback, missing boundary test, unnecessary abstraction, or poor explanation. The counts show where context or verification fails. Feed stable conventions into repository instructions or executable checks. Keep product decisions and ambiguous business rules in human review; stuffing them into a longer prompt creates confidence, not shared understanding.

Do not reward a falling comment count by itself. Reviewers may be tired, rushed, or resigned. Pair comments with post merge corrections, incidents, and the clean sample from the monthly routine. If comments fall while repairs rise, the review gate weakened. If both fall and sampled changes remain understandable, the workflow probably improved.

One useful qualitative measure is reconstruction time. Give an engineer who did not author the change its issue, diff, and tests, then ask them to explain the behavior and likely failure modes. Record where they needed external context. You do not need to run this on every change. A few high impact samples expose generated indirection that static metrics miss.

Dependencies and configuration carry hidden debt

Generated implementation code receives most of the scrutiny, but agents also modify package manifests, permissions, CI files, feature flags, and infrastructure definitions. These small files can create a larger future obligation than an extra helper. Measure them as a separate change class.

Track new direct dependencies, version range changes, duplicated packages, unresolved vulnerability findings, and dependencies used in only one small location. A new package is not automatically debt. It becomes suspect when the team cannot explain its owner, update path, license, or removal cost. Require the change description to state why an existing dependency or a short local implementation was insufficient. That single sentence often reveals when an agent selected the first plausible package from its training context.

Lockfiles need different treatment. Their size and churn are poor quality metrics because package managers generate them. Check whether manifest changes explain lockfile changes, whether the install remains reproducible, and whether the chosen version exists in the approved registry. Never ask reviewers to assess thousands of generated lockfile lines visually. Let the package manager and policy checks verify the artifact.

Configuration debt appears as new switches without owners or expiry conditions. Count flags, environment variables, permission grants, and CI exceptions introduced during the month. Each temporary switch needs an owner, a reason, and a deletion condition. Expiry dates are useful for exceptions, but a date alone just schedules another ignored alert. The deletion condition says which release, migration, or observed behavior makes removal safe.

Pay special attention to defaults. Agents often add an environment variable, handle its presence, and leave absence behavior ambiguous. Test both states. For permissions and network access, fail closed unless the product requirement explicitly says otherwise. For operational tuning, an absent value should produce a documented, observable default. Configuration that only works in the agent's test environment is debt the moment it merges.

Schema and API changes need a compatibility record. Count new optional fields, widening types, duplicate representations, and deprecated paths without removal plans. An agent can make a local compiler happy by accepting both old and new forms forever. That pushes cleanup onto every consumer. Record which consumers exist, how compatibility is tested, and what event permits the old path to disappear.

Generated tests can conceal the risk

Test count and coverage can rise while confidence falls. Agents are good at producing tests that mirror the implementation they can see. Those tests protect the current shape, including its mistakes, and make later refactoring expensive. Measure what the tests observe and how they fail.

Start with behavior versus implementation coupling. Count tests that mock internal modules, assert private calls, snapshot large objects, or reproduce implementation constants. These patterns are sometimes justified, but a rising share means the suite resists structural change. Sample new tests and ask whether they would still pass after a valid refactor. If the answer is no, the test probably protects arrangement rather than behavior.

Mutation testing gives stronger evidence than line coverage for important logic. A mutation tool changes operators, conditions, or return values and checks whether tests fail. Surviving mutations point to assertions that execute code without distinguishing correct from incorrect behavior. Do not run mutation testing across the entire repository on day one. Select authorization, billing, data deletion, state transitions, and other consequential modules. Track surviving mutations in those boundaries and assign concrete tests.

Look for the shared blind spot. When an agent writes implementation and tests in the same context, both may encode the same mistaken assumption. A separate verification prompt can help, but independence comes from different evidence, not different wording. Derive boundary cases from the issue, schema, protocol, or production failure. Have a reviewer add at least one case the implementation context did not suggest.

Flakiness and runtime still matter. Generated tests may add arbitrary sleeps, broad retries, real network calls, or oversized fixtures. Record new quarantined tests, retry counts, suite duration, and failures that pass without a code change. A retry can be an operational containment measure, but it needs an owner and repair item. Otherwise the suite slowly learns to ignore uncertainty.

Delete low information tests when better boundary tests replace them. Teams often keep every generated test because creation looked free. Maintenance is not free: each assertion constrains refactoring, each fixture needs updates, and each slow test spends feedback time. A smaller suite that detects meaningful faults is healthier than a larger suite that confirms getters and mocks.

A scorecard needs decisions, not a blended grade

Do not collapse the measurements into one debt score. A weighted grade looks authoritative while hiding whether risk comes from dependencies, review effort, tests, or architecture. Keep a small set of signals with an owner, baseline, threshold, current value, direction, and named action.

A practical scorecard can use five rows: structural residue, review tax, correction flow, dependency and configuration exceptions, and test effectiveness. Under each row, retain links inside your internal system to raw artifacts and sampled examples. The published summary needs the count and explanation, not every scanner warning. This is the second place where judgment matters: a stable count can still hide old findings being removed as worse findings arrive.

Set thresholds from the service's own history and risk. One approach uses two levels. An investigation threshold asks the owner to explain movement and sample findings. An action threshold requires scheduled repair or an explicit acceptance with an expiry. Avoid universal thresholds such as a fixed complexity number for every language. Tools use different definitions, and a parser naturally differs from a CRUD handler.

Treat an exception as a decision record. Name the finding, the affected component, the reason for acceptance, the owner, the expected cost, and the event that ends acceptance. If the team cannot name an end event, it has accepted permanent complexity and should say so. Honest permanent exceptions are easier to manage than temporary labels renewed forever.

The scorecard review should produce few, bounded tasks. "Improve generated code quality" cannot be completed. "Replace three duplicate entitlement checks with the policy module and add boundary tests" can. Tie each task to the signal it should change, but do not require the metric to reach zero. The repair is successful when the repeated decision disappears and the next change becomes easier to verify.

Keep the scorecard outside performance evaluation. Its job is to direct engineering attention and test whether the agent workflow improves. If leaders use it to rank people, the data will become defensive within a month. Owners should be able to annotate tool changes, migrations, incidents, and intentional investments that explain movement.

Repair the workflow before blaming the model

A deteriorating trend usually calls for a workflow change before a model change. Switching models may alter the symptoms while leaving vague tasks, missing context, and weak verification untouched. Repair the cheapest upstream cause that explains the evidence.

If duplication rises, give the agent a repository map and require a search for existing behavior before implementation. If broad catches rise, add a language rule and require tests for the failure path. If review tax rises on large changes, cap task scope and make the agent produce independent commits for behavior, tests, and mechanical edits. If dependency churn rises, restrict manifest edits or require a human approval with a written reason. These controls are specific enough to test in the next monthly run.

Keep agent instructions short enough to inspect. A giant policy file accumulates contradictions and stale conventions. Move deterministic rules into formatters, linters, architecture tests, permission policies, and CI checks. Leave instructions for context that tools cannot express: where business truth lives, which boundaries carry risk, and who can approve a tradeoff. Every instruction should have an owner and a failure example.

Use a holdout sample when changing the workflow. Apply the new rule to one comparable set of changes and retain the current process for another, then inspect debt flow and review tax. This does not need academic experimental design. It needs enough discipline to avoid crediting the new model for an easy month or blaming it for a migration. Record task type and service risk so the comparison remains honest.

At oleg.is, a Team & AI Audit uses this kind of evidence to identify where a smaller AI augmented engineering team can cut cost without transferring the bill into maintenance. The fixed engagement is useful only if it produces named savings, owners, and controls that survive after the audit.

The first report will be imperfect. Keep its definitions and exclusions stable for the next run, repair the clearest repeated failure, and watch whether both the machine signal and human review burden move. After three monthly cycles, delete measures that never change a decision. A metric earns its place when somebody can point to the code it exposed, the task it created, and the future work that became simpler. Otherwise it is another artifact the team must maintain.