How spec-driven development keeps AI agents honest

A good prompt can produce an impressive patch. It cannot tell your team whether that patch is the right product change. For serious engineering work, the durable unit of intent is a versioned specification with acceptance criteria, constraints, and a defined way to prove completion. The prompt merely tells an agent what to do next with that specification.

I learned this distinction the expensive way. An agent can follow a detailed chat instruction, pass the obvious tests, and still change an API contract, weaken an authorization boundary, or invent behavior that nobody approved. The failure is rarely that the model cannot write code. The failure is that the team gave it prose without a contract and treated plausible output as evidence.

Spec-driven development fixes that control problem. It does not mean writing a giant requirements document before anyone experiments. It means placing intent in repository artifacts that humans can review, tools can validate, and agents must treat as inputs. The smallest useful spec is precise enough to produce failing checks before implementation and stable enough that a fresh agent session reaches the same interpretation.

A prompt expires when the session ends

A prompt is an instruction to one model in one context window. A specification is shared project state. That difference matters as soon as work lasts longer than a single exchange, crosses an ownership boundary, or carries production risk.

Chat history creates the illusion of continuity. The agent remembers the clarification you gave twenty minutes ago, until context gets compressed, a new session starts, or another engineer takes over. Even when the entire transcript survives, important decisions sit among guesses, corrections, tool output, and abandoned approaches. Nobody reviewing a pull request should have to reconstruct the accepted behavior from that stream.

A repository spec gives every participant the same starting point. It records what users can do, what the system must reject, which interfaces may change, and which evidence closes the work. A prompt can then stay small: read the approved change, implement task two, run the required checks, and stop if the spec conflicts with the codebase.

This also makes model choice less consequential. Different agents will still produce different implementations, but they face the same boundary. Without a spec, switching models often changes the implied product decision. With one, switching models mostly changes the route to an agreed result. That is a much safer variable to optimize.

Do not turn this into a semantic argument about whether a long prompt counts as a spec. Location and lifecycle settle it. If the text lives only in a conversation, cannot be reviewed as a diff, has no owner, and disappears from the change record, it behaves like a prompt. If it is versioned, validated, and tied to acceptance evidence, it behaves like a spec.

A useful spec closes five kinds of ambiguity

A useful spec removes the decisions that an implementation agent should not make. It can leave coding choices open, but it must close ambiguity about behavior, boundaries, failure, quality, and proof.

Start with observable behavior. Name the actor, precondition, action, and result. "Add CSV export" is a feature label. "An account administrator can export the currently filtered invoice set as UTF-8 CSV" gives the agent a testable surface. Add an example when words can support two reasonable readings.

Then state boundaries. Name what is in scope and what is deliberately out. If export applies to invoices but not credit notes, say so. If the public API cannot change, record that constraint. Agents tend to repair nearby inconsistencies because that often improves benchmark tasks. In a production repository, an unrequested cleanup can multiply review and regression risk.

Failure behavior deserves its own requirements. Define what happens when the dataset is empty, the requester lacks permission, a field contains a comma, or generation exceeds a size limit. Happy-path specs invite the agent to invent error semantics. Those inventions become accidental product policy once clients depend on them.

Quality constraints must be measurable. "Fast" and "secure" cannot fail a check. A better requirement names a budget, an authorization rule, a supported platform, or a prohibited data flow. When you cannot choose a number honestly, define a comparison or an observation procedure instead of fabricating precision. For example, require the new query plan to avoid a sequential scan on the production-shaped fixture.

Finally, define proof. Each requirement needs at least one acceptance scenario, and each scenario should map to an automated test or an explicit human review. I use RFC 2119 words sparingly: MUST for a condition that blocks release, SHOULD for an intentional default with understood exceptions, and MAY for permitted variation. The standard warns that these terms should appear only where interoperability or harm requires them. Turning every sentence into MUST produces ceremony, not clarity.

A compact requirement can look like this:

Requirement: Filtered invoice export
Actor: Account administrator
Precondition: At least one invoice matches the active filters
Behavior: The system MUST export exactly the matching invoices
Format: UTF-8 CSV with one header row and RFC 4180 field escaping
Failure: A non-administrator receives 403 and no export job is created
Out of scope: Scheduled exports and credit notes
Evidence: API contract test, permission test, CSV fixture comparison

The field names are not sacred. The closed decisions are.

Keep intent, design, tasks, and evidence separate

One enormous spec file usually becomes a bad prompt with headings. Separate artifacts let people review decisions at the right level and prevent implementation detail from quietly rewriting product intent.

I use four layers. The product spec owns behavior and exclusions. The technical plan owns architecture, data changes, interface contracts, migration strategy, and operational risks. The task list owns execution order and dependencies. Evidence records which tests, measurements, or approvals satisfied each acceptance criterion.

That separation gives you traceability without a heavyweight requirements database. Give each behavioral requirement a stable ID such as EXP-003. The plan references that ID when it selects an endpoint or queue. Tests include the ID in their name or metadata. The pull request reports the result. If the requirement changes, a search shows the artifacts that may now be stale.

A practical repository layout is small enough to understand at a glance:

specs/
  current/exports.md
  changes/exp-014-filtered-csv/
    proposal.md
    spec.md
    plan.md
    tasks.md
    evidence.md
contracts/
  exports.openapi.yaml
tests/acceptance/
  exp_014_export_test.ts

Keep current truth separate from a proposed delta. OpenSpec formalizes this with openspec/specs/ for accepted behavior and openspec/changes/ for proposals, tasks, optional designs, and spec deltas. That model fits established products because reviewers can see both the stable contract and the exact proposed change. After approval and implementation, archiving folds the delta into current truth.

GitHub Spec Kit takes a broader staged approach: constitution, specification, plan, tasks, and implementation. Its documentation describes the core flow as Spec to Plan to Tasks to Implement, with clarification and cross-artifact analysis available around it. I like the separation, but no command can decide whether your acceptance criteria express the correct business policy. The harness enforces sequence and shape; an accountable person still owns meaning.

Avoid copying the same sentence into all four layers. Duplication creates disagreement disguised as documentation. Reference requirement IDs, let each artifact answer its own question, and make validation reject missing references.

Enforcement turns documents into a control system

A spec that an agent may ignore is background reading. Enforcement begins when the repository blocks work that lacks an approved change, rejects malformed requirements, and refuses a merge without acceptance evidence.

The first gate is structural. Validate that every change has an owner, scope, exclusions, acceptance scenarios, and traceable IDs. OpenSpec includes an openspec validate <change> command for its required delta format. Spec Kit offers checklist and analyze phases to find unclear requirements and inconsistencies across its generated artifacts. You can use either tool, or a small local validator, as long as failure produces an actionable message.

The second gate is agent instruction. Put one short rule in the repository instructions: agents may implement only approved task IDs, must cite requirement IDs in changed tests, and must stop when code and spec conflict. Do not bury this policy in twenty pages of style guidance. The instruction should point to the versioned artifacts, not repeat them.

The third gate is continuous integration. A simple pipeline can verify the contract before it spends minutes on the full test suite:

spec-check:
  script:
    - ./bin/spec-lint specs/changes/$CHANGE_ID/spec.md
    - ./bin/trace-check $CHANGE_ID
    - npm test -t $CHANGE_ID

contract-check:
  script:
    - ./bin/openapi-diff contracts/base.yaml contracts/exports.openapi.yaml
    - ./bin/verify-evidence specs/changes/$CHANGE_ID/evidence.md

spec-lint should report locations and missing fields, not a vague invalid result. A useful output shape is spec.md:24 EXP-003 missing failure behavior. trace-check should fail when a task or acceptance test references an unknown ID. The API diff should distinguish an additive change from a breaking one and require explicit approval for the latter.

The fourth gate is merge policy. GitHub documents that protected branches can require status checks before merge and can require review from code owners. Assign owners to the spec paths and contract files, then make the spec, contract, and acceptance checks required. Also protect the CODEOWNERS file itself; GitHub calls that out because an unprotected ownership rule can be edited in the same change it is meant to control.

Do not ask the model to grade its own compliance and call that enforcement. An agent review can find gaps, but deterministic checks and named human approvals decide whether the gate opens.

The agent workflow needs explicit stop points

An effective agent workflow alternates generation with approval and verification. Giving an agent a spec and permission to work until it feels done recreates the original prompt problem at a larger scale.

Use a bounded loop. First, a human or planning agent drafts the change spec from the product request. A responsible reviewer resolves open questions and approves the behavior. A planning agent then maps requirements to components, contracts, risks, and tasks. Another check compares the plan against the spec before any production code changes.

Implementation proceeds one task or one coherent task group at a time. The agent reads the relevant spec, current repository instructions, and only the code needed for that task. It writes or updates the acceptance test, changes the implementation, runs scoped checks, and records evidence. A failing test may trigger another implementation attempt. A conflict with the approved behavior must stop the loop and return to the spec owner.

The stop conditions matter more than elaborate orchestration. Stop when the agent discovers an unmentioned data migration, a public contract change, an authorization decision, a destructive operation, or a requirement that cannot be tested as written. Those are product or architecture decisions. More tokens will not supply legitimate authority.

Keep permissions narrower than scope. A spec authorizes a product change; it does not automatically authorize deployment, production data access, dependency upgrades, or edits outside the task boundary. Tool policy must enforce those limits separately. Teams routinely blur behavioral authority with operational capability, then wonder why a well-specified agent still caused damage.

For parallel agents, partition by artifacts with explicit ownership. One agent can implement the endpoint while another writes a client adaptation only if the contract is already approved. Do not let both redefine the shared schema. Their work should converge through contract tests and requirement IDs, not through mutual chat summaries.

This workflow is slower than free-form generation for the first hour. It is faster across review, rework, handoff, and incident recovery. That is the time horizon a CTO should measure.

A worked change exposes the missing decisions

Consider a billing product with an existing endpoint, POST /exports, that starts an asynchronous invoice export. The request currently accepts a date range. A founder asks an agent: "Let users export the invoices they filtered in the UI, preserve the existing endpoint, and add tests." That sounds detailed. It still leaves the dangerous decisions open.

Does "filtered" include search text, status, customer, currency, and sort order? Does the export capture filters or the exact matching record set? What happens if invoices change while the job runs? May a user export invoices outside their account by guessing a customer ID? Is an empty result a valid file or an error? A coding agent can choose answers and produce clean code. Clean code does not make those answers authorized.

Write the behavioral delta first:

Change: EXP-014 Filtered invoice export

EXP-014-1
Given an account administrator with active date, status, and customer filters
When the administrator creates an invoice export
Then the job stores the normalized filter values
And the worker exports invoices matching those values at execution time

EXP-014-2
The service MUST derive account_id from the authenticated session.
It MUST reject a customer_id that does not belong to that account with 404.

EXP-014-3
An empty match MUST produce a CSV containing the header row.
Search text, currency filters, and sort order are out of scope.

The choice to evaluate filters at execution time is now explicit. A snapshot export would need a different design and storage cost. The 404 response avoids confirming that another account owns the guessed customer ID. The header-only result lets users and automation distinguish a successful empty query from a broken export.

Next, change the interface contract before application code:

ExportRequest:
  type: object
  additionalProperties: false
  properties:
    date_from: { type: string, format: date }
    date_to: { type: string, format: date }
    status: { type: array, items: { enum: [draft, open, paid, void] } }
    customer_id: { type: string, format: uuid }
  required: [date_from, date_to]

additionalProperties: false prevents a misspelled customer_id from being silently ignored. The enum prevents the UI and service from drifting on status names. The plan can now identify the request validator, authorization query, job payload, worker query, CSV fixture, and compatibility test.

The task sequence follows the risk: add contract tests, extend request validation, add the tenant-scoped customer lookup, persist normalized filters, update the worker query, add the empty fixture, then run the legacy date-only tests. Each task references the relevant requirement.

Suppose the agent discovers that jobs currently store only date_from and date_to in fixed database columns. Adding filters requires a migration. The original request did not authorize a schema strategy, so the agent stops and proposes two plans: nullable columns for the bounded filter set, or a versioned JSON payload with stricter application validation. The reviewer chooses based on expected change and query needs. This interruption is success. The spec workflow surfaced a decision before code made it expensive to reverse.

Evidence closes the change with more than a green unit suite:

EXP-014-1: acceptance/export_filters.test.ts passed
EXP-014-2: acceptance/export_tenant_boundary.test.ts passed
EXP-014-3: fixtures/export_empty.csv byte comparison passed
API compatibility: additive request fields, existing request fixture passed
Migration: upgrade and rollback tested on a production-shaped fixture
Human approval: billing owner reviewed execution-time semantics

A new agent can now inspect the change without reading the original conversation. That is the operational payoff of the spec.

Tests are evidence, but they are not the whole spec

Tests prove selected observations about an implementation. They do not fully explain why the behavior exists, which alternatives were rejected, who owns a policy, or what remains outside scope. Treating tests as the entire specification leaves product intent trapped in test mechanics.

The reverse mistake is worse: acceptance prose with no executable checks. Given-When-Then text helps reviewers reason about behavior, but it does not prevent drift unless a test, contract validator, measurement, or named reviewer evaluates it. Every acceptance criterion needs an evidence type. Not every criterion needs a unit test. Accessibility may need automated checks plus keyboard review. A migration may need rehearsal and row-count reconciliation. An operational limit may need a load test with a recorded fixture.

Link evidence to requirements rather than files alone. Files move and tests get renamed. Stable requirement IDs let a validator build a coverage table: requirement, implementation task, check, result, and approval. Reject unknown IDs and duplicate IDs. Warn about requirements with no evidence. Do not require one test per sentence; one end-to-end scenario can cover several related assertions if the mapping stays visible.

Beware of generated tests that merely mirror generated code. If the same agent invents behavior, writes the implementation, and writes assertions against that invention, the suite proves internal consistency rather than correctness. Anchor important tests in an independently reviewed contract or fixture. For high-risk boundaries, have another person or agent derive adversarial cases from the approved spec before seeing the implementation.

Production feedback must also update the spec. If an incident reveals that retries create duplicate exports, fix the idempotency behavior in current truth, add the regression scenario, then change code. Otherwise the test gains a mysterious case while the product contract remains wrong. Specs that never absorb operational learning become historical fiction.

Tooling should fit the age of the codebase

Choose tooling based on how your system changes, not on which repository has the loudest launch. Greenfield products, established systems, APIs, and regulated workflows need different amounts of structure.

GitHub Spec Kit suits teams that want an end-to-end staged flow and are willing to maintain a constitution, feature specs, plans, tasks, and analysis. Its explicit phases help a new project establish conventions before many conflicting patterns exist. The official documentation now positions it as an extensible harness that can run phases separately or as automated workflows. That flexibility is useful, but automation increases the need for deliberate approval points.

OpenSpec is attractive for brownfield work because it separates current truth from proposed changes and validates requirement scenarios and deltas. Its proposal, apply, and archive cycle maps cleanly to an existing product where small changes must preserve a large body of behavior. It also supports instructions for several coding assistants, which reduces dependence on one agent interface.

Kiro's spec workflow organizes feature work around requirements, design, and tasks. It can be a reasonable fit when the team wants those phases inside an integrated agent environment. OpenAPI, AsyncAPI, JSON Schema, Protocol Buffers, database migration tools, and policy engines solve narrower parts of the same control problem. Use them for machine-checkable interfaces rather than rewriting exact constraints in prose.

Plain Markdown plus a validator is often enough. A small team does not need a branded framework to assign requirement IDs, review spec diffs, run contract tests, and protect a branch. It does need consistent paths, templates that ask hard questions, and CI that fails when evidence is absent. The tool should reduce interpretive freedom, not add a folder of generated prose that nobody reviews.

Do not let a spec tool become another source of truth beside tickets, design files, API definitions, and code. Decide which artifact owns each fact. Link them with stable IDs and generate derived views where possible. If two files both claim to own response semantics, they will eventually disagree.

Adoption works when one risky change proves it

Roll out spec-driven development on changes where ambiguity is expensive and acceptance is observable. An authorization rule, billing behavior, data migration, or public API change will teach the method faster than a cosmetic component.

Set a narrow policy for the pilot. The change needs an owner, behavioral requirements, exclusions, a technical plan, traceable tasks, acceptance evidence, and required CI checks. Track review cycles, escaped requirement defects, rework, and elapsed delivery time against similar work. Do not judge the method by lines of documentation or agent output volume.

Expect the first specs to reveal organizational gaps. Product may not know who decides empty-state behavior. Engineering may lack a stable API contract. Tests may depend on fixtures that cannot represent production scale. Those discoveries can feel like process overhead because the old workflow hid them inside code review or production incidents. Fix the ownership and tooling rather than deleting the questions from the template.

After two or three changes, remove fields that never affect a decision and strengthen gates around recurring failures. Keep an exception path for urgent production fixes, but require the spec and evidence to catch up in the same incident workflow. An emergency label must not become a permanent way around product review.

For companies moving to smaller AI-augmented teams, this discipline is not optional management theater. Fewer engineers can ship more changes only when intent, authority, and evidence remain inspectable. I use a Team & AI Audit at oleg.is to identify where specifications, agent permissions, CI gates, and ownership break across the delivery path; the useful output is the operating change, not another AI policy deck.

Prompts will keep improving, and agents will need less procedural guidance. That makes durable specifications more important, because faster implementation amplifies every unresolved product decision. Put the next risky change in a versioned contract, make CI demand its evidence, and see whether a fresh agent can complete it without inheriting the chat. If it cannot, the missing context belongs in the system, not in somebody's memory.