What an AI code review mandate really buys

Amazon's reported move to require senior sign-off on AI-assisted changes is easy to read as a verdict on AI-generated code. It is more useful to read it as a capacity decision. When software creation gets cheaper and faster, review, testing, deployment control, and operational ownership become the scarce parts of the system. A mandate moves money and attention toward those constraints, whether leadership planned that shift or discovered it after an incident.

That lesson applies well below Amazon's scale. A small team does not need a committee, an enterprise license for every developer, or a rule that treats every autocomplete suggestion as hazardous. It needs a review policy based on the damage a change can cause, evidence that the checks actually ran, and enough senior attention for the risky exceptions. Buying an AI reviewer before defining those controls usually creates more comments, not more confidence.

What Amazon reportedly changed, and what it did not prove

The reported policy was a senior approval requirement for AI-assisted changes made by junior and mid-level engineers. Financial Times reporting connected the move to an internal review of incidents with a high blast radius and novel generative AI use. That is a governance response: a company places a more experienced person between a class of changes and production. It does not prove that an AI model wrote the faulty code, that every AI-assisted change carries equal risk, or that senior review alone prevents outages.

Amazon publicly disputed the broad causal story. Its statement said none of the incidents involved AI-written code. It said one incident involved an engineer following inaccurate advice that an AI tool inferred from an outdated internal wiki. That correction matters because advice, code generation, command execution, and production access are different failure paths. Calling all four AI coding hides the control that actually failed.

The stale-wiki case, taken on Amazon's account, points to retrieval and operational authority rather than defective generated source code. A reviewer should ask which document the agent used, whether that document had an owner and expiry date, what command the engineer approved, and what permissions let the command affect production. A line-by-line code review may never see any of those facts.

The policy still sends a clear market signal. Large organizations expect AI-assisted work to raise change volume, and they are willing to spend senior engineering time to control it. They also expect the use of AI to become observable enough to govern. Small companies should copy the risk logic, not the organizational ceremony.

The mandate buys accountability before it buys quality

A required approver gives the company a named decision point. That improves accountability immediately, but code quality improves only if the approver receives useful evidence and has authority to reject the change. A senior engineer who must clear a queue of oversized pull requests becomes an expensive rubber stamp.

This is the first distinction teams often blur: approval is a control, while inspection is an activity. A branch rule can prove that somebody with the right role clicked Approve. It cannot prove that the person understood the change, checked a migration, challenged an agent's assumption, or ran a rollback exercise. The control has value because it creates ownership and a place to attach requirements. It is not evidence of careful inspection by itself.

The second distinction is between author seniority and change risk. A junior engineer can update copy with almost no operational exposure. A principal engineer can approve an agent's destructive infrastructure command with enormous exposure. A policy based only on job level spends review time in the wrong places and can give senior-authored changes an undeserved fast lane.

A workable approval rule names the protected assets and the evidence required to touch them. Authentication, billing, customer data, permissions, infrastructure state, database schemas, and deployment logic deserve a stronger gate. Documentation, isolated tests, and reversible user-interface copy usually do not. The reviewer then knows why a change reached them and what decision they own.

Senior sign-off is useful as an emergency brake because organizations can deploy it quickly. As a permanent design, it needs risk routing, automated evidence, and limits on change size. Otherwise the mandate converts generation speed into a review backlog and calls the queue safety.

AI-assisted is the wrong unit for risk

Teams should classify the effect of a change, not whether an engineer used AI somewhere while making it. AI provenance is hard to measure, easy to underreport, and weakly related to blast radius. A hand-written permission change can be dangerous; an AI-generated unit test can be harmless.

A blanket AI label also becomes meaningless once assistants touch search, planning, code, tests, documentation, and terminal commands. Does asking a model to explain an unfamiliar function make the final patch AI-assisted? What if the developer rejects every suggestion? A rule that depends on self-reporting an ambiguous category will produce inconsistent data and arguments at review time.

Track AI use for cost, vendor, privacy, and training decisions. Do not make it the primary release gate. Route changes using properties that the repository and deployment system can observe:

• Which paths, services, data stores, and permissions changed.
• Whether the change is reversible without restoring data.
• Whether it alters an external contract or a production migration.
• Whether tests cover the failure mode rather than only the happy path.
• Whether the author or an agent can execute against production.

This does not excuse AI-specific controls. An agent needs narrower credentials than a developer usually carries, explicit boundaries on command execution, and logs that retain its request, proposed action, approval, and result. Generated code also deserves an independent check when the generator and reviewer share context or a model family, because they can repeat the same mistaken assumption.

The useful policy sentence is not, All AI code needs senior review. It is, Changes in the high-risk lane need an accountable senior approver, regardless of how they were produced. AI use can raise a change's lane when provenance is missing, the agent acted on external instructions, or the team cannot explain the result. That rule survives the next tool change.

Tooling budgets move downstream from generation

An AI coding budget cannot stop at assistant seats or token charges. Faster creation increases the amount of code and configuration that tests, reviewers, CI workers, security scanners, staging environments, and observability systems must process. The cost center moves from writing to proving and operating.

The first budget line is compute for verification. More pull requests mean more test runs, static analysis, dependency checks, preview environments, and retries. An inexpensive coding seat can generate a surprisingly large CI bill when an agent opens several revisions or changes broad parts of a repository. Put a monthly ceiling and per-pull-request visibility on that compute before adoption expands.

The second line is integration. A reviewer that cannot see repository conventions, service ownership, recent incidents, architecture decisions, or generated artifacts will return generic advice. Connecting those sources takes engineering work and ongoing maintenance. Context that has no owner decays, as Amazon's account of outdated internal guidance illustrates. Retrieval quality belongs in the budget because stale context can make a confident agent less safe than no context.

The third line is senior attention. Price it explicitly. If five developers each save four hours a week generating code but one senior engineer spends twenty more hours reviewing larger diffs, the team has moved labor rather than removed it. The move may still pay if releases improve, but the budget should show the transfer.

The fourth line is control and evidence: audit logs, policy checks, retention, identity, least-privilege credentials, and incident analysis. These items feel indirect beside a coding assistant demo. They decide whether a company can explain who proposed a production action, who approved it, which tests passed, and how the result was rolled back.

Reserve money for evaluation as well. Teams need a representative set of past defects and risky changes to test any reviewer before making it a gate. Vendor accuracy claims cannot tell you whether the tool understands your migration framework, permission model, or failure conventions.

A lightweight policy starts with three risk lanes

A small team can get most of the benefit with three lanes and one required field in its pull request template. The lanes must be simple enough that authors classify changes consistently and concrete enough that CI can challenge a dishonest or mistaken classification.

Low risk covers changes with small, reversible effects: documentation, isolated tests, copy, and internal refactors behind existing behavior. Standard risk covers ordinary product work that changes application behavior but does not touch protected assets. High risk covers authentication, authorization, payments, sensitive data, infrastructure, production commands, schema migrations, public APIs, and changes whose rollback can lose data.

The process can fit on one page:

1. The author selects a lane and writes the worst plausible failure in one sentence.
2. CI compares changed paths with a small risk map and raises the lane when protected areas changed.
3. Low-risk work can merge after tests and normal peer review. Standard work needs a peer plus the service's normal checks.
4. High-risk work needs a named senior approver, a rollback note, and evidence for the relevant failure test.
5. After deployment, the owner records whether the lane was correct and whether any review finding mattered.

Do not create a fourth lane for AI. Add an AI disclosure field for governance and vendor cost, then treat unexplained or autonomously executed changes as high risk. This avoids punishing harmless use while still escalating work that nobody can defend.

The risk map should stay small. Start with paths such as infra/, migrations/, auth/, billing handlers, deployment workflows, and access policy files. A team can add a path after a near miss. If everything becomes protected, the map has stopped making decisions.

Make the pull request carry evidence

A reviewer should not reconstruct intent from a diff. Require the author to state the risk, failure mode, verification, and rollback in a machine-readable block that CI can validate. This is a small artifact with more value than a long AI-generated summary.

change_risk: high
worst_failure: Existing customers lose access after the role migration
verification:
  - test: role_migration_preserves_existing_grants
  - check: staging_migration_and_rollback
rollback: Restore application version, then run migrations/042_down.sql
ai_assisted: true
agent_production_access: false

A simple policy job can reject a high-risk change when the required evidence is absent. Its output should tell the author what failed rather than produce a mysterious red mark:

policy-check: FAIL
risk: high
protected paths: migrations/042_roles.sql, auth/grants.ts
missing: senior approval
verified: rollback note, failure-mode test, AI disclosure

This block does four jobs. It makes the author name a consequence, gives the reviewer a testable claim, lets automation route the pull request, and leaves a compact record for an incident review. It also exposes a common failure: a rollback instruction can be syntactically present but operationally false. For a destructive migration, restoring the old application version may not restore deleted or transformed data. The reviewer must challenge the semantics, not merely check that the field exists.

Keep AI-generated review comments advisory at first. Let the reviewer annotate possible bugs, missing tests, broad exception handling, unsafe queries, or deviations from repository rules. Require a human decision for high-risk findings, and record Accepted, Rejected with reason, or Deferred with owner. That feedback gives you a local precision signal and stops the same noisy rule from consuming attention forever.

Amazon Q Developer's documentation offers a useful caution about scope. It says its review combines generative AI with rule-based automatic reasoning and can check security, secrets, infrastructure as code, code quality, deployment risk, and dependencies. It also says filtering excludes some material, including test code and unsupported languages, depending on review mode. A broad feature list is not proof that every file in your change was inspected. Your CI record should show what the reviewer actually saw.

Review capacity has to be measured as a queue

A mandate fails quietly when leaders count licenses and approvals but ignore waiting time. Review is a queue: changes arrive at a rate, reviewers process them at a rate, and oversized or noisy changes increase service time. Once arrivals regularly outrun capacity, developers split work awkwardly, chase approvers, or merge with shallow inspection.

Measure the queue at the lane level. Track time to first substantive review, time waiting after the last revision, pull request size, the number of review rounds, and high-risk changes per qualified reviewer. Medians hide the painful end, so inspect the oldest open changes and the slowest tenth as well. You do not need a management dashboard with dozens of charts. A weekly table is enough to show whether the high-risk lane is accumulating debt.

Limit work before hiring another reviewer or buying another bot. Smaller pull requests reduce human review time and make automated findings easier to locate. Set a soft size threshold, require an explanation above it, and allow generated files to be excluded from the human diff when their source and reproducible build are reviewed. Never let an agent use a huge change as evidence of productivity.

Protect senior focus with ownership. Route database work to someone who understands recovery, identity work to someone who understands the authorization model, and infrastructure work to someone who understands deployment state. Seniority without domain knowledge is a weak substitute. For a small company, one person may own several domains, but the routing still makes the decision explicit.

An AI reviewer can remove obvious defects before a human sees the pull request. It can also flood the queue with low-confidence comments. Cap comment volume, suppress style advice already covered by deterministic tools, and measure how often developers change code because of a finding. The goal is less reviewer effort per safe change, not more visible reviewing.

Buy gates only after measuring their findings

The metric that matters is not AI review coverage. It is how much credible risk the review removes for the money and time spent. Coverage can reach 100 percent while everyone ignores the output.

Start with four outcome measures. Count accepted findings that prevented a defect or added a meaningful test. Count false positives and repeated comments. Track escaped defects by risk lane, including which evidence was missing or misleading. Track reviewer time and total time from ready-for-review to merge. These measures connect tool behavior to delivery and reliability without pretending that one score captures quality.

Separate deterministic findings from model judgments. Secret detection, forbidden dependency rules, formatting, schema validation, and known unsafe patterns often belong in fast, repeatable tools. Architecture fit, suspicious omissions, misleading names, and whether a rollback plan can work need context and judgment. Paying a model to rediscover lint errors wastes tokens and trains developers to skim all automated comments.

Run a shadow evaluation before making an AI reviewer blocking. Feed it a set of recent merged changes, known defects, and deliberately seeded variations in a controlled branch. Record what it saw, what it found, how often it invented a concern, and how much engineer time adjudication took. Include changes from your actual languages and frameworks.

Then set a narrow blocking policy. A verified secret, a prohibited production permission, or a missing high-risk approval can block. A probabilistic claim about a race condition should request human review unless your local evaluation has earned stronger trust. Blocking every model warning transfers uncertainty into queue time.

Review the tool like any supplier. Ask what code and prompts leave your environment, how long data is retained, which repositories it can access, whether model or rule changes are versioned, and how usage costs scale with diff size and retries. If the answers cannot be tied to logs and contract terms, budget for that uncertainty instead of assuming it away.

Agent permissions matter more than another reviewer

An AI reviewer cannot compensate for an agent that holds excessive production authority. Review happens before a merge, while agents can also read operational guidance, run terminal commands, change cloud resources, update tickets, and retry failed actions. The control boundary must cover what the agent can do, not only what appears in a source diff.

Start with separate identities. An agent should never inherit a developer's broad session by default, and a shared service account makes later investigation needlessly vague. Give each workflow its own identity, repository scope, allowed commands, environment, and short credential lifetime. Read access to production telemetry may be reasonable for diagnosis. Write access to production infrastructure should require a distinct approval path and should remain unavailable to ordinary coding sessions.

Treat retrieved instructions as untrusted input with an owner. Architecture notes, runbooks, tickets, and wiki pages can be wrong, stale, or written for a different environment. Store an owner and review date beside operational documents. When an agent proposes an action based on retrieved text, retain the source revision in the log so an approver can see the exact instruction it followed. A fresh model working from obsolete guidance still produces an obsolete answer.

Command approval must show the resolved action. Asking a person to approve a friendly natural-language plan while hiding the final command, target account, affected resources, or expansion of a wildcard is consent theater. Render those details before execution, reject unresolved variables, and make destructive operations require a second confirmation from someone other than the requesting workflow.

A useful execution record contains the agent identity, initiating user, source revisions, proposed command, resolved target, approval identity, timestamp, exit status, and resulting change. Keep secrets out of the record, but do not reduce it to agent action succeeded. During an incident, the team needs to reconstruct the decision chain without trusting the agent's own summary.

Test the boundary with denied actions. Try to make the coding agent read a repository outside its scope, use an expired credential, alter a production resource, and follow an instruction embedded in an issue that conflicts with policy. A safe result is a clear denial recorded in the audit trail. If the test merely asks the model to behave, you have tested manners rather than permissions.

This is where small teams have an advantage. They can define two or three agent identities and a short list of protected actions without negotiating across hundreds of internal systems. Spend the first control dollars on identity and execution limits. Another layer of review comments will not repair an agent session that can bypass the pull request entirely.

The small-team budget should fund one control loop

A small company should fund a complete loop before buying several overlapping assistants. The loop starts when a change declares its risk, runs deterministic checks and one AI review, routes exceptions to the right person, records the decision, and uses production results to adjust the policy. One connected loop beats three tools that each produce a separate inbox.

A practical first budget has five buckets: one generation tool per active engineer who benefits from it, one repository-level reviewer, additional CI capacity, a modest block of senior review time, and implementation time for policy and measurement. Use your own rates and workloads. Do not copy a large company's seat count or assume the reviewer will pay for itself through comments.

Revisit the allocation after four to six weeks. Cut rules that create noise. Expand deterministic checks for recurring defects. Add context only when somebody owns it. If senior wait time rises, reduce change size and tighten routing before purchasing more generation capacity. If AI findings rarely change code, remove the gate or test another approach.

For founders who cannot see where review time and tooling money go, the Team & AI Audit at oleg.is maps the workflow, costs, and savings opportunities over five business days for a fixed $5,000. The guarantee is at least $50,000 a year in identified savings or the audit is free, which makes the engagement easy to compare with another year of unused seats and hidden senior review time.

Do not imitate Amazon by placing a senior name on every AI-touched patch. Borrow the useful part: faster code creation changes the control budget. Classify risk, make evidence travel with the change, and watch the review queue. If generation grows while those three remain fixed, the mandate arrives eventually, usually after the team has less time to design it well.