Context engineering for AI coding teams

Context engineering decides what an AI system gets to know for the next decision, when it learns it, and when that information leaves. Prompt engineering tunes an instruction. That still matters, but a polished instruction cannot rescue a session buried under stale files, repeated tool output, conflicting rules, and a chat transcript that has become its own source of noise.

Teams feel this first as cost or latency. The deeper damage is decision quality. The model notices an obsolete convention, misses the test failure that matters, or spends most of its attention reconciling duplicate evidence. A larger context window delays the visible failure while making waste easier to ignore.

I treat context as an engineered runtime resource, with an owner, a budget, admission rules, provenance, and an expiry policy. That framing changes the work. Instead of asking, "How do we fit more into the prompt?" the team asks which evidence the current decision requires and what should be removed before the next one.

Prompt engineering ends where context engineering begins

Prompt engineering controls the request; context engineering controls the entire input state that surrounds it. That state includes system rules, repository instructions, retrieved documents, tool results, conversation history, memory, examples, generated plans, and any summary carried from earlier work. The prompt is one item in that set.

This distinction has a practical consequence. When an agent edits the wrong file, repeating "edit only the billing module" with stronger wording may do nothing. The agent may have retrieved two billing modules, received an old migration guide, and inherited a plan that points to the retired one. The instruction is clear. The evidence is not.

A useful context inventory names every channel and its authority:

• System and policy hold stable boundaries and mandatory rules. The main risk is that minor rules hide the few that govern the task.
• Repository guidance holds local commands, architecture, and ownership. A broad note can contradict a local instruction.
• Retrieval holds evidence selected for the present question. Similar text often arrives without the conditions that make it valid.
• Tool output holds fresh observations from commands and services. Raw output can flood the session or lose the time and environment that gave it meaning.
• Working memory and conversation hold active decisions, user intent, and negotiated constraints. Guesses can harden into facts, while old intent can compete with a later correction.

Authority and freshness must travel with the content. A database schema read from the running environment should outrank a design note from six months ago design note. A user correction should outrank the initial task summary. A instruction for a directory instruction should govern files in that directory, even when a broad repository note says something else. If the agent must infer these precedence rules from prose order, the context design has already failed.

Prompt work asks whether the instruction is understandable. Context work also asks whether the model can identify the current source of truth. You need both, but they solve different defects.

A context window is capacity, not a budget

The advertised context window is a hard ceiling; a context budget is an allocation for useful evidence. Filling the ceiling is like filling every byte of RAM before starting a process. The fact that the system accepts the input says nothing about whether the allocation is sensible.

Set budgets by role before a run. A coding task might reserve space for stable policy, the task contract, a small working set of source files, test or compiler feedback, and a response margin. The exact counts depend on the model and job, so fixed universal percentages are false precision. What matters is that every category competes for a declared allowance.

A simple manifest makes the contract inspectable:

context_plan:
  objective: fix invoice rounding without changing tax rules
  budget_tokens: 48000
  reserve_for_response: 8000
  always_include:
    - repository_policy
    - task_acceptance_tests
  retrieve_on_demand:
    - invoice_domain_files
    - current_tax_schema
    - failing_test_output
  exclude:
    - generated_build_artifacts
    - archived_migrations
    - unrelated_chat_history
  stop_when:
    - evidence_conflicts
    - budget_remaining_below: 6000

The numbers are an example, not a recommended default. The useful part is the separation between material admitted at startup and material fetched after the agent forms a question. The stop rule also prevents a common failure: the agent keeps retrieving after conflicting evidence appears, then improvises a resolution instead of escalating it.

Budgeting works at decision boundaries, not only at session start. Before planning, the agent needs the task contract and enough architecture to find the work. Before editing, it needs the selected files and local invariants. Before verification, it needs the changed surface, the relevant test commands, and fresh results. Carrying all discovery material through every phase wastes tokens and keeps rejected hypotheses alive.

Reserve output space explicitly. Models often produce abrupt, weak answers when input consumes nearly all available capacity. More importantly, an agent needs room for tool calls, error recovery, and a final check. A session that has enough space for the intended edit but none for the first failed test is underbudgeted.

Retrieval hygiene starts before the search

Good retrieval begins with a clean corpus and a precise request; ranking alone cannot repair stale, duplicated, or material without context material. Teams often judge retrieval by whether a returned passage resembles the query. Similarity is only the first gate. Evidence also needs authority, freshness, scope, and enough surrounding text to interpret it.

I use four admission questions for every retrieved item:

1. Does it answer a question the agent has actually formed?
2. Can the agent tell where it came from and when it was last valid?
3. Does its scope match the component, version, tenant, or environment under discussion?
4. Does it add evidence that is not already present in a better form?

If an item fails one of those checks, leave it out or label the limitation. Dumping the top ten results into context feels safe because nothing appears omitted. In practice, it transfers the filtering job to the model and charges tokens for every bad candidate.

Chunk construction matters as much as search. A function body without its signature, a policy sentence without its exception, or an error line without the command that produced it can rank well and still mislead. Index units that preserve the smallest meaningful boundary: a function with its contract, a runbook step with its preconditions, or a decision record with its status and superseding link expressed as metadata rather than a clickable URL.

Deduplicate before admission. Exact hashes catch copied content; normalized hashes catch whitespace and formatting changes; semantic checks catch two documents that say the same thing in different words. Keep the version with the clearest authority and provenance. Do not ask the model to arbitrate three nearly identical runbooks.

Freshness is not merely a date field. A document can be old and authoritative, such as a stable protocol rule. It can be new and already wrong, such as a generated architecture summary created before a rename completed. Record the event that invalidates the item: a schema version, deployment, file change, ownership transfer, or explicit replacement. Expiry tied to a real event is more useful than deleting everything after an arbitrary number of days.

Retrieval should return less when confidence falls. If two authoritative sources disagree, the retriever should expose the conflict and stop broadening the search. More passages do not settle an authority dispute. A person or a fresh tool observation must do that.

Layer context by lifetime and scope

The most reliable design separates context into layers that expire at different rates. Stable policy should not share a lifecycle with a compiler error. A task decision should not become permanent project memory merely because it appeared in a successful session.

Use four layers as a starting point:

• The policy layer contains stable safety, legal, and repository boundaries. Change it deliberately and keep it short.
• The task layer contains the requested outcome, acceptance tests, constraints, and current plan. Replace it when intent changes.
• The evidence layer contains retrieved files, tool observations, and environment state. Attach provenance and expire it when the underlying state changes.
• The scratch layer contains hypotheses, partial calculations, rejected plans, and temporary notes. Clear it aggressively at phase changes.

These layers should not have equal write permissions. Tool output may update evidence, but it should not silently rewrite policy. A model may propose a memory, but a deterministic filter or human review should decide whether that memory survives the run. Without that boundary, a plausible guess can become a permanent instruction after one compaction cycle.

Scope each layer as narrowly as the work allows. Repository policy applies broadly. A package convention applies only inside that package. A specific to one customer fact applies only to that customer and should disappear before the next tenant's task. A tool result applies to the environment and time in which the command ran. Put those qualifiers in fields, not in a paragraph the agent must reinterpret.

Phase transitions are the best cleanup points. After discovery, keep the chosen path and the evidence supporting it; discard search branches that lost. After editing, keep the diff, acceptance criteria, and current failures; discard the prose that narrated every keystroke. After verification, preserve results and unresolved risks; discard raw successful logs unless an audit requirement says otherwise.

Summaries belong in the task layer only when they distinguish observation from inference. "Tests pass" is an observation if it carries the command, environment, timestamp, and exit status. "The change is safe" is an inference and should remain labeled as one. Flattening those statements into the same confident summary creates false certainty.

Token waste usually hides an evidence failure

A bloated session often starts with a reasonable request and then accumulates evidence without replacement. Consider an agent asked to change retry behavior in a monorepo. It loads the root instructions, searches for "retry," opens six matching packages, retrieves an old incident note, and runs the entire test suite. The test output alone fills a large part of the remaining window.

The first failure is not token count. The task never established which service owns the behavior. Retrieval selected lexical matches before resolving ownership. The incident note describes the retired queue client, but it contains the same error phrase as the current implementation. The full test suite then produces thousands of passing lines and one truncated failure from an unrelated package.

Compaction makes the situation worse if it summarizes frequency instead of authority. The retired client appears in several files and in the incident note, so the summary calls it the main implementation. The current client appears once behind an interface. The agent edits the retired package, writes a test beside it, and reports success because that isolated test passes. Every local step looks plausible. The context pipeline aimed the work at the wrong system.

A disciplined run changes the sequence:

1. Resolve ownership from the current dependency graph and package metadata before searching implementations.
2. Retrieve the interface, the active binding, and the narrow package instructions. Exclude packages absent from the active graph.
3. Run the smallest test that reproduces the requested behavior, retaining failures and a compact pass summary.
4. After the edit, rerun that test and the nearest integration boundary. Expand only when changed dependencies justify it.
5. Compact the run into decisions, cited observations, changed files, and open risks. Omit rejected search results.

Notice what this does not do: it does not ask the model to "pay more attention." It changes the evidence path. It also reduces tokens, but token savings are a result of better control rather than the sole objective.

The popular recommendation to load the whole repository is attractive because it avoids building retrieval. It is wrong for most nontrivial repositories. Generated code, vendored packages, fixtures, archived migrations, and parallel implementations create collisions. Start with maps and ownership, then admit source around the chosen execution path. Loading the whole repository context is reasonable only when the repository is genuinely small enough that every file can affect the decision and you can prove there are no stale branches.

Measure decisions, not just tokens

Token totals reveal cost, but they do not tell you whether context helped. Measure the path from admitted evidence to a decision. A cheap run that edits the wrong component is worse than an expensive run that stops on a real conflict.

Log context events in a form you can inspect without storing hidden reasoning. The record needs the item identity, source, authority, scope, freshness basis, token estimate, admission reason, and eventual disposition. It does not need a transcript of private model deliberation.

{"event":"context_admit","item_id":"src/billing/rounding.ts@8f31","source":"repository","authority":"runtime_code","scope":"billing","freshness":"commit:8f31","tokens":742,"reason":"implements selected interface"}
{"event":"context_evict","item_id":"docs/rounding-plan.md@19ac","reason":"superseded by runtime code"}
{"event":"decision","decision_id":"d17","evidence":["src/billing/rounding.ts@8f31"],"result":"edit_selected"}

That shape supports useful ratios. Admission yield is the share of retrieved items that survive the hygiene checks. Evidence use is the share of admitted items cited by a plan, edit, verification decision, or escalation. Carryover ratio is the share of one phase's context copied into the next. Conflict rate counts runs where authoritative items disagree. Rework rate counts decisions reversed after new evidence arrives.

Do not turn these into universal score targets. A low admission yield may mean retrieval is noisy, or it may mean the task required broad discovery. A high evidence use rate can look efficient while revealing that the agent admitted too little and missed an alternative. Compare similar task classes, inspect outliers, and read a sample of failed runs.

Track latency and money beside quality signals. Separate tokens fetched, tokens admitted, tokens served from cache, tokens generated, and tokens discarded during compaction. That breakdown tells you where an intervention belongs. Better indexing reduces fetched noise. Admission rules reduce loaded noise. Stable prefixes improve cache reuse. Phase cleanup reduces carryover. A shorter final answer changes none of those upstream defects.

The strongest metric is preventable rework tied to context. Label each reversal with a cause such as stale evidence, wrong scope, missing authority, lost user correction, or truncated tool output. Once a cause repeats, fix the pipeline stage that admitted or dropped the evidence.

Design patterns that survive long tasks

Long tasks need controlled disclosure, replacement, and handoff. Merely extending the conversation preserves too much accidental state. Three patterns cover most systems I build.

Map, query, fetch

Give the agent a compact map of components, owners, interfaces, and available evidence. Let it form a specific question, then fetch the narrow material that answers it. This is better than injecting a large starter pack because the question creates an admission test.

The map must describe where evidence lives without pretending to contain the evidence itself. A package summary can say that billing owns rounding and exposes an invoice calculator. It should not paraphrase every rounding rule, because the paraphrase will drift from code. Fetch the rule when the task reaches it.

Replace, do not append

When a tool result becomes stale, replace it in working context. When the user corrects a constraint, rewrite the task contract and mark the old version superseded. When a plan changes, keep the current plan plus a short decision record, not two complete competing plans.

Logs that only append logs still belong in observability storage. They do not belong wholesale in the model's active context. The active view should represent current state, while the external log preserves history for debugging and audit. Confusing those two needs is a major source of waste.

Handoff packets

When one agent delegates to another or a session must restart, pass a bounded packet rather than the full transcript. Include the objective, acceptance tests, current plan, confirmed observations with provenance, changed artifacts, unresolved conflicts, and the next permitted action. Exclude conversational filler, rejected hypotheses, and raw successful logs.

A receiving agent should validate the packet before acting. It must check that cited files still match their identifiers, tool observations still apply to the environment, and the user has not changed the task. Handoff is a cache, not fresh truth.

These patterns also help multiple agent pipelines. Give each worker the smallest context that lets it own a decision, then require structured evidence in its return packet. Broadcasting every worker's transcript to every other worker multiplies cost and spreads weak guesses across the group. Shared state should contain accepted facts and current coordination, not everyone's scratchpad.

Cache stable context and invalidate it on events

Context caching saves money only when the cached prefix stays authoritative and the system knows exactly what invalidates it. Caching a long starter prompt because it changes infrequently can reduce repeated input processing. Caching a mixed bundle of policy, repository summaries, generated plans, and recent tool output creates a cheap way to repeat stale assumptions.

Build cache units around ownership and lifetime. Stable system policy can occupy one unit. Repository instructions tied to a commit can occupy another. A component map tied to the dependency graph can use a third. Task state and live tool results should stay outside those units unless the cache identifier includes every event that can make them stale. Smaller units create more bookkeeping, but they let the system reuse policy without preserving a retired package map.

Put the invalidation event in the cache contract. Repository material should expire when the referenced commit or generated index changes. Environment observations should expire after a deployment, configuration change, or movement to another environment. User intent should expire when the user revises the objective or constraint. A duration can be a safety limit, but elapsed time cannot detect most meaningful changes.

Stable prefix ordering also matters. If the provider reuses processed input for an identical prefix, placing volatile timestamps or random identifiers near the beginning can destroy reuse for everything after them. Put stable policy and stable project instructions first, then scoped task material, then current evidence. This ordering must still respect authority. Never move a later user correction behind an earlier cached instruction merely to improve reuse.

Cache hits need the same observability as retrieval. Record which units were reused, their source identifiers, the invalidation conditions checked, and the tokens avoided. A high cache hit rate is not success if stale units cause rework. Pair reuse with the context defect labels from failed decisions.

Do not cache summaries without their evidence identifiers. A summary can outlive the files that supported it, and fluent prose hides that drift. Store the summary as a derived view linked to exact source versions. When one source changes, regenerate the view or evict it. If regeneration costs more than fetching the narrow original evidence, skip the summary.

Sensitive context needs admission boundaries

Privacy and security controls belong in the context pipeline before retrieval, not in a final instruction asking the model to ignore sensitive data. Once a secret, customer record, or restricted design enters active context, later wording cannot undo the exposure. Admission must enforce purpose, tenant, environment, and retention rules.

Start with least context access. A worker that edits a public documentation page does not need production credentials merely because another tool in the same agent can reach them. An analyst working on test failures may need an error type and stack frame but not the request payload containing personal data. Shape tool responses for the decision instead of returning the richest object available.

Redaction must preserve diagnostic meaning. Replacing every value with the same mask can hide whether two identifiers match or whether a value changes between retries. Use consistent scoped tokens where equality matters, retain safe type and length information where useful, and remove fields that do not contribute to the decision. Never place raw secrets in a visible to the model transcript so the model can redact them afterward.

Tenant boundaries require separate retrieval scopes and separate memory. Metadata on a chunk is not enough if the search service can rank an item from the wrong tenant before filtering. Apply authorization before similarity search when the storage system permits it, or search only an already authorized partition. Clear specific to one customer working state at the end of the task and test that the next task cannot retrieve it.

Tool permissions should narrow with the phase. Discovery may need read access across an allowed repository scope. Editing needs write access only to the selected files. Verification needs the approved commands and environment, not every credential used by deployment. This reduces damage when stale context points to the wrong action.

Log the policy decision without copying the sensitive value. A useful event says that a field was excluded because its classification exceeded the task purpose, records the policy version, and names the tool that enforced it. That gives operators evidence that the control ran while keeping the forbidden content out of the audit trail.

Treat context deletion as a tested operation. Ending a chat is not proof that provider caches, agent memory, retrieval indexes, temporary files, and observability records follow the same retention rule. Map each copy, set its owner and expiry mechanism, and run deletion tests that verify the item disappears from future retrieval.

Make context ownership an engineering job

Context quality needs an owner, review rules, and failure tests. If prompts belong to one person, retrieval to another, agent tooling to a third, and cost reporting to finance, nobody sees the full input path. Assign one technical owner for the context contract even when several teams maintain its parts.

Start by collecting failed or expensive runs, not by rewriting the system prompt. For each run, reconstruct what the agent knew at the decision that went wrong. Mark each item as necessary, distracting, stale, conflicting, or missing. Then trace why the pipeline admitted the bad item or omitted the needed one. This produces a backlog you can test.

Turn each recurring failure into an evaluation. Keep a case where a retired module has more textual matches than the active module. Keep one where a user correction arrives late. Keep one where a successful command produces huge output before the relevant failure. A context change passes only if the agent still resolves the right authority and scope while using fewer unnecessary tokens.

When I reduced an engineering operation from 25 people to two AI-augmented engineers, the hard part was not finding clever prompts. The hard part was making constraints, current evidence, verification, and ownership available at the exact decision where each mattered, without carrying the entire company history into every run.

A Team & AI Audit is useful when a founder needs this mapped across the whole delivery system: it takes five business days, costs $5,000, and guarantees at least $50,000 a year in identified savings or it is free. The result should still be judged like engineering work, by which context defects it finds, how they affect delivery, and which controls will prevent them.

Do not begin by buying a larger context window. Take one failed run and locate the first decision made with stale, missing, or wrongly scoped evidence. Fix that admission path, add the case to evaluation, and measure whether rework falls. That is context engineering in practice.