Can golden master tests make assistant-led refactors safe?

A coding assistant can rewrite a tangled function faster than your team can agree on where to start. That speed is useful only when the assistant has a boundary it must not cross. Golden master tests give it one: preserve the outputs that users, integrations, and operations already depend on, while changing the internals as aggressively as the work requires.

This is not permission to bless whatever a legacy system does today. It is a way to separate two decisions that teams routinely mash together: whether a refactor preserved current behavior, and whether current behavior deserves to change. Keep those decisions separate and reviews get sharper. Combine them and every refactor turns into an argument about requirements, formatting, historical bugs, and code style.

A golden master records behavior, not truth

A golden master is an approved capture of a system's output for a known input. A test runs the same input, captures the new result, and compares it with the approved result. If the files differ, the test fails until a reviewer decides whether the difference is intentional.

ApprovalTests documentation calls this approval testing and notes that the same approach is also known as snapshot testing or golden master testing. Its important point is easy to miss: the generated file becomes a specification only after a person approves it. The capture is evidence, not an oracle that arrived from nowhere.

That distinction matters most in old code. Suppose an invoice endpoint rounds a tax adjustment down instead of up. If you capture its response and approve it, the master protects the rounding behavior during a database migration. It does not tell you the rounding behavior is lawful, profitable, or even wanted. A product owner can later change the rule in a separate pull request with a visible fixture update.

I want three questions answered before I accept a master:

• Which real consumer would notice if this output changed?
• Which parts of the output form the contract, and which parts are incidental noise?
• Who has enough domain context to approve a diff when it fails?

If nobody can answer the first question, skip the master. You are probably taking a photograph of internal plumbing. If nobody can answer the third, do not let an assistant refactor that path unattended. The missing reviewer is the risk, not the missing test framework.

Traditional assertions and golden masters do different jobs. A focused test can say that a declined card produces a specific error code. A master can preserve the full customer statement generated by a messy reporting pipeline. Do not replace every assertion with a large file. Use an approval capture when the behavior is broad enough that hand-written assertions would be incomplete, unreadable, or so expensive that the team will never maintain them.

Put the master at a boundary people depend on

Capture behavior at API, file, and report boundaries because those are places where a change becomes visible outside the code being rewritten. The best boundary is usually farther out than a unit test and narrower than a whole production environment.

For an API, capture the status, relevant headers, and canonical response body. For a file generator, capture the bytes if exact formatting matters, or a normalized representation if it does not. For a report, capture the final rendered table or export, not the dozen private objects that happened to assemble it.

A practical boundary has four properties. It accepts a controlled input. It produces an inspectable output. It can run without calling live third parties. It has a stable owner who understands why a line changed.

Consider a customer export endpoint. The fragile approach is to ask an assistant to "clean up the export service" and hope existing unit tests cover the details. A better approach is to construct representative input records, run the endpoint through its public handler, and save the CSV. That capture preserves column order, blank-field treatment, escaping, date formatting, and subtotal placement. Those details are often where a refactor breaks a finance or operations workflow.

Do not make the capture so wide that reviewers cannot reason about it. An HTTP transcript that includes routing diagnostics, every security header, tracing IDs, and a megabyte of unrelated configuration creates a noisy test. The response contract is not the same thing as every byte emitted by the running process.

A useful rule is this: capture the last stable representation before another person or system takes over. For an internal algorithm, that might be a domain result object serialized to JSON. For a public endpoint, it is the response. For a document generator, it is the file. For a nightly business report, it is the report that someone downloads and uses.

Stable fixtures need controlled inputs and normalized outputs

A master test fails for a useful reason only when the fixture removes accidental variation. Time, random values, generated identifiers, hostnames, locale, file paths, record ordering, and external data are the usual offenders. Teams often call this brittleness, then abandon the technique. In most cases, they captured a moving target.

Control variation as close to its source as possible. Inject a clock instead of deleting dates from every output. Seed the random generator instead of replacing random values after the fact. Use a fake mail provider or payment gateway rather than replaying a live response. Sort records where the contract does not promise order.

Then normalize the output deliberately. Do not use a broad regular expression that erases every number and call the result stable. You may remove the exact field that would reveal a regression. A normalization rule should state what it removes and why that field is not part of the contract.

This Python example captures an API response after isolating values that should not affect the result:

import json
from copy import deepcopy

VOLATILE_HEADERS = {"date", "x-request-id", "traceparent"}

def canonical_response(response):
    body = deepcopy(response.json())
    body.pop("generatedAt", None)

    for row in body.get("items", []):
        row["id"] = "<generated-id>"

    headers = {
        name.lower(): value
        for name, value in response.headers.items()
        if name.lower() not in VOLATILE_HEADERS
    }

    return json.dumps(
        {
            "status": response.status_code,
            "headers": dict(sorted(headers.items())),
            "body": body,
        },
        indent=2,
        sort_keys=True,
    ) + "\n"

This code makes a claim about the contract. It says request IDs, tracing headers, generated timestamps, and generated record IDs do not matter for this test. That claim may be right for a catalog search response and wrong for an endpoint where the client must later use the generated ID. The test author has to know the difference.

For JSON, parse and reserialize it with stable key order. For CSV, decide whether row order has meaning, then sort only when it does not. For PDFs, do not begin by comparing raw bytes. Metadata, font subsets, and object offsets make raw PDF comparison miserable. Extract the text and page count if that is what users care about, or test a structured source model before the renderer.

Keep fixtures small enough to read in a pull request. A five-hundred-line master can be legitimate, especially for a report. A five-thousand-line master with no visual grouping is a maintenance failure. Split it by scenario, or write a custom serializer that prints the business fields in a useful order.

Byte equality and behavioral equality are different contracts

Teams waste time because they treat every output as if byte-for-byte equality were the only honest comparison. Sometimes it is. Often it is not.

Byte equality is appropriate when another system consumes the literal output and its parser or signature rules make every character meaningful. A fixed-width export, an email template where line wrapping affects rendering, a cryptographic payload, or a canonical interchange file can justify it. In those cases, do not normalize away the formatting that consumers rely on.

Behavioral equality checks meaning after you remove representation details that nobody depends on. JSON object key order is the obvious example. Whitespace in a human-readable report may not matter, while the values, headings, and row sequence do. A command-line tool may include an elapsed duration that belongs in logs but not in a compatibility test.

The dangerous middle ground is a test that pretends to compare behavior while silently strips meaningful fields. I have seen normalizers delete all IDs, all dates, and all currency amounts because they made diffs inconvenient. That test passes through the exact regressions customers report.

Write a short contract note beside each capture. It can be a comment or a test name, but it should answer what equality means. For example:

customer_export_with_closed_accounts_preserves:
- one row per account, ordered by account number
- ISO 8601 dates in the report timezone
- blank closure_date for active accounts
- a final totals row

It ignores:
- export generated timestamp
- request correlation identifier

This is not paperwork. It tells the reviewer why an added field needs discussion and why a changed timestamp does not.

The same distinction applies to errors. An API may promise a status and stable machine-readable code while leaving prose free to improve. Capture the code and status. Do not freeze a badly written message unless clients parse it or support staff relies on the exact wording. If your client does parse the prose, fix that client instead of congratulating the test suite for preserving a bad contract.

The first approval deserves more scrutiny than the refactor

The initial capture is the moment when a team can accidentally make a bug permanent. Review it as if it were a new public API, because from that point forward it will influence what engineers feel safe changing.

Do not have an assistant generate a hundred fixture files and approve them in bulk. That produces the appearance of coverage while nobody has checked the content. Start with a small set of inputs that expose the branching behavior you fear. Include ordinary cases, boundary cases, and one or two awkward records that have caused support work before.

For an account report, that might mean an active account, a closed account with a final adjustment, and an account with an empty optional address line. For a pricing endpoint, it might mean a normal order, a discount that reaches zero, and a currency rounding boundary. Capture cases that force the code to show its decisions.

ApprovalTests documentation describes a received result and an approved result. Preserve that two-file model even if you do not use the library. A failed run writes a received file. A human reads the diff and deliberately promotes it to approved only when the behavior is correct. The normal test command must never overwrite approved output.

A minimal shell workflow looks like this:

./scripts/capture-customer-export --case closed-account

git diff --no-index \
  tests/fixtures/customer-export/closed-account.approved.csv \
  tests/fixtures/customer-export/closed-account.received.csv

mv tests/fixtures/customer-export/closed-account.received.csv \
   tests/fixtures/customer-export/closed-account.approved.csv

The mv is intentionally boring and visible. It forces an explicit act after inspection. Git's git diff documentation also makes clear that --no-index can compare two paths outside the repository index, which fits received-versus-approved files well. In CI, the capture command should fail if it creates a received file or if a received file is already present.

Do not approve an output merely because the old system produced it. Ask whether it exposes private data, encodes an obsolete policy, or locks in a bug that you already intend to remove. If it does, either correct it before the refactor or label the fixture with a tracked requirement and make the policy change a separate, reviewed change.

Let the assistant change code inside a narrow operating area

An assistant-led refactor works when you give the assistant a constrained objective, an executable proof, and a stopping rule. "Modernize this module" has none of those. "Replace the export service internals while all approved export fixtures pass, without changing the public handler" is a real assignment.

I use a sequence like this when the code has enough complexity to tempt broad changes:

1. Run the existing test suite and capture baseline fixture results on a clean branch.
2. Ask the assistant to map the call path and identify the boundary that the fixture covers. Do not ask for code yet.
3. Give it a limited refactor target, such as extracting a formatter, replacing a data access adapter, or removing duplicated transformations.
4. Require it to run focused tests after each coherent change and show the diff for any fixture change.
5. Stop when the intended structural change is complete. Do not let a successful refactor turn into opportunistic cleanup across unrelated modules.

The assistant should not have permission to approve a received fixture. It can explain a diff, group changes by cause, and propose a new expected output. It cannot decide that a different tax total, missing field, reordered report, or altered error code is acceptable. That decision belongs to the reviewer who owns the contract.

This boundary is especially useful with agent workflows that can edit many files before pausing. Give the agent an allowlist of directories, the exact command that must remain green, and an instruction to stop on any fixture difference. A broad agent with write access and auto-approval can turn a useful regression test into a machine for laundering behavior changes.

The popular recommendation to give the assistant all the tests and ask it to "make them pass" is wrong for masters. It encourages the assistant to modify the expected output, loosen a normalizer, or alter test setup until the comparison says yes. Treat approved fixtures as read-only during implementation. If a test needs a new approved result, make that a separate human action after code review.

A changed fixture is a product decision waiting for an owner

When a master fails, classify the diff before touching the fixture. The classifications are simple: intended product change, accidental regression, accepted correction of old behavior, or test noise. The classification determines the next move.

An intended product change needs a requirement or a clear reviewer decision, then an approved fixture update. An accidental regression means fix the code and leave the fixture alone. A correction of old behavior needs its own explanation because the fixture now documents both the old contract and the decision to abandon it. Test noise means tighten the setup or normalization so future runs stop producing meaningless diffs.

Here is a failure that shows why this discipline pays for itself. A team refactors an order summary service to reduce repeated database reads. The API master changes in three places: an item list appears in a different order, an absent discount field becomes null, and the total changes from 19.99 to 20.00.

The first difference may be harmless if the API says items are unordered. The second can break a mobile client that distinguishes missing from null. The third is not a formatting detail. It may expose a decimal conversion error, a newly corrected rounding rule, or an old defect that nobody noticed. If the reviewer approves the entire fixture because "the refactor should not change anything," the team either preserves a financial bug or hides a regression.

Split the diff by cause. Fix the total first. Decide whether field presence belongs to the compatibility contract. Only then decide whether sorting belongs in the serializer or the fixture normalizer. This creates more commits, but each commit says something a future engineer can understand.

A good pull request description names the expected fixture changes in plain language. It does not say "snapshots updated." It says "the export now prints inactive accounts after active accounts because the report requirement changed" or "the response no longer exposes an internal source field." If the author cannot write that sentence, reviewers should not approve the file change.

Large masters need review ergonomics, not blind trust

A large output can be a legitimate contract. It still needs structure that helps a person inspect it. The test should print useful names, isolate cases, and make a diff point to business meaning instead of raw serialization accidents.

Use one fixture per scenario rather than one giant fixture for every variation. Give scenarios names that state the condition and expected behavior. invoice_with_tax_exemption beats case_07. Put stable sections in a stable order. For a report, group header, rows, totals, and warnings rather than dumping an internal object graph.

Custom serializers earn their keep here. ApprovalTests documentation points out that layout affects how maintainable approval files are when failures happen. That is correct. A serializer that puts one meaningful field per line can prevent a one-word change from appearing as a hundred-line reflow.

For structured output, include only fields that are part of the boundary. A full ORM dump is a bad master because internal additions and refactors create noise. Instead, create a test representation designed for review:

{
  "report": "monthly_revenue",
  "period": "2026-06",
  "currency": "USD",
  "rows": [
    {"account": "A-102", "recognized": "1250.00", "deferred": "0.00"},
    {"account": "B-451", "recognized": "0.00", "deferred": "400.00"}
  ],
  "totals": {"recognized": "1250.00", "deferred": "400.00"}
}

This representation is not a lie if the real report renders the same business information. It is an adapter for testing. Keep a smaller number of end-to-end checks for the final CSV, spreadsheet, or PDF so the adapter itself does not become a false comfort.

Reviewers also need a limit. If a fixture change is too large to read, ask the author to partition it or provide a second comparison that summarizes field-level changes. Do not accept "the assistant says the output is equivalent." The assistant may be correct, but a claim of equivalence is not review evidence.

Masters complement focused tests instead of replacing them

Golden masters are poor at explaining why a result is right. They tell you that the result differs. Keep direct assertions for rules that carry high risk or need precise diagnosis: authorization, validation, money arithmetic, state transitions, retry behavior, and error classification.

Use masters where the amount of observable detail is large. A human-readable document, a data export, a response with many computed fields, or a migration compatibility path are good candidates. Use focused tests where a one-line assertion captures the actual rule. A test named applies_state_tax_after_discount is easier to understand than discovering that the rule is buried in line 183 of a report fixture.

This split makes assistant work safer too. The focused tests expose local rules. The golden captures expose system-level effects. When the assistant changes a calculation, the direct test points to the broken rule and the master shows which downstream output changed. Neither test type has to carry the entire burden.

At AppMaster.io, the practical version of this is not asking AI-augmented engineers to be careful in the abstract. It is giving them repeatable production boundaries, fast tests, and a review trail that makes intended changes obvious. Teams that need to identify where those boundaries are missing can start with a Team & AI Audit before they hand large refactors to agents.

Start with the path your team avoids touching

The best first target is not the cleanest service. It is the behavior everyone treats as dangerous because the code is old, the outputs are fussy, and no one wants to be blamed for breaking it. Capture three or four representative cases at its public boundary. Read every line of the first approved files. Then let the assistant make one contained internal change.

If the output stays the same, you gained confidence and a reusable guardrail. If it changes, the diff gives you a concrete question to answer. That is a better place to spend review time than reading a hundred changed lines and hoping nobody forgot an edge case.