How a release evidence packet keeps two engineers honest

A two-engineer team does not need a release committee. It does need a record that answers an unpleasant question quickly: what exactly changed in the production release that caused this behavior?

A green CI pipeline is not that record. It proves that jobs passed under whatever conditions the pipeline had at that moment. It usually does not put the tested commit, migration outcome, dependency delta, production deployment, and usable rollback path in one place. When an incident starts, people pull those facts from separate browser tabs, terminal histories, and half-remembered messages. That is slow when the customer is waiting and dangerous when someone is tempted to roll back first and investigate later.

A release evidence packet fixes the gap by making the release itself the unit of record. CI creates it from the release commit, attaches it to the pipeline, and treats an incomplete packet as an incomplete release. This is not paperwork. It is a small operational product with a clear customer: the engineer on call, including the person who deployed it six weeks ago and has forgotten the details.

The packet must prove a release, not describe one

A release packet is useful only when its claims come from commands, APIs, or immutable CI variables. A hand-written note saying "tests passed and migrations look good" is a status update. It is not evidence.

Every packet needs a single release identity. Use an immutable version plus the full commit SHA. If your deployable is a container image, include its digest too. A mutable tag such as latest, production, or even v1.8 without a digest does not tell you what actually ran after someone retags an image.

The minimum questions are straightforward:

• Which repository commit created this release?
• Which test suites ran, and did CI receive their results?
• Did the database reach the expected migration version?
• Which direct and transitive dependencies changed?
• Which environment received which artifact, at what time, through which deployment job?
• Can an engineer run the rollback, and is the database compatible with it?

Notice what does not belong on this list: a prose recap of every pull request, raw production logs, screenshots from a dashboard, or a copy of an approval chat. Those things make the packet larger without making it more trustworthy. Capture a short machine-generated change summary if it helps humans orient themselves, but do not confuse narrative with proof.

The distinction matters during failures. An engineer sees an error after deploying commit c7e.... The packet says the image digest, the migration version, the test report files, and the previous production target. That engineer can compare facts before touching production. Without the packet, they start reconstructing a release under pressure, which is how weak assumptions become production changes.

One release ID has to travel through every job

Use the CI pipeline to create a release ID once, then pass it to every later job. Do not let test, build, deploy, and packet jobs independently invent labels. The packet becomes unreliable when the deployment says 2026.07.22.3, the image says main-abc123, and the migration log contains only a timestamp.

For GitLab CI, a small dotenv artifact is a practical transport mechanism. GitLab documents that a dotenv report makes values available to later jobs in the pipeline. It also warns that dotenv reports are accessible to pipeline users, which is why they must contain identifiers and URLs, never credentials or tokens.

Create a file with the identity you intend to preserve:

mkdir -p evidence
RELEASE_VERSION="${CI_COMMIT_TAG:-${CI_COMMIT_SHORT_SHA}-${CI_PIPELINE_IID}}"
IMAGE_REF="registry.example.internal/app@${IMAGE_DIGEST}"
PREVIOUS_RELEASE="${PREVIOUS_RELEASE_VERSION}"

cat > evidence/release.env <<EOF
RELEASE_VERSION=$RELEASE_VERSION
COMMIT_SHA=$CI_COMMIT_SHA
PIPELINE_ID=$CI_PIPELINE_ID
PIPELINE_URL=$CI_PIPELINE_URL
IMAGE_REF=$IMAGE_REF
PREVIOUS_RELEASE=$PREVIOUS_RELEASE
EOF

The output shape should stay boring:

RELEASE_VERSION=4f1d9a2-841
COMMIT_SHA=4f1d9a28545170b87e45d34a8d6f4c8e2cbe8c57
PIPELINE_ID=1841
PIPELINE_URL=https://git.example.internal/team/app/-/pipelines/1841
IMAGE_REF=registry.example.internal/app@sha256:8ad4...
PREVIOUS_RELEASE=02bc91e-837

PREVIOUS_RELEASE deserves more care than teams give it. It must mean the last release confirmed in the target environment, not the prior successful build on the default branch. A skipped production deployment, a hotfix branch, or a failed deploy can make those different values. Have the deploy job query the deployment system or a small release registry before it changes anything, then save the returned target as evidence.

This is also where two-person teams often make a costly shortcut. They use a commit SHA as the whole release identity and skip the immutable artifact reference. That works until the build is not reproducible, a base image moves, or configuration packaging differs between two builds of the same source. Source identity and runtime artifact identity answer different questions. Keep both.

Test evidence needs results, not a green badge

Test evidence should show which reports CI received and whether the job passed, failed, or never ran. A green badge is a summary computed by the CI system. The packet should preserve the underlying test result files and a compact summary derived from them.

GitLab accepts JUnit XML reports and displays them in pipeline and merge request test views. Its documentation also makes an easy-to-miss point: test reports do not determine job status. Your test command still has to exit nonzero when tests fail. That distinction catches a surprisingly common bad setup: a wrapper generates XML, swallows a test failure, and leaves the pipeline green.

Make test execution and report collection explicit:

unit_tests:
  stage: test
  script:
    - mkdir -p evidence/test-results
    - ./scripts/test-unit --junit evidence/test-results/unit.xml
    - ./scripts/summarize-junit evidence/test-results/unit.xml > evidence/test-summary.json
  artifacts:
    when: always
    access: maintainer
    expire_in: 180 days
    paths:
      - evidence/test-results/
      - evidence/test-summary.json
    reports:
      junit: evidence/test-results/*.xml

Use when: always for the report artifact. If a test job fails, the failed report is often the most useful artifact in the entire release investigation. Do not, however, let the packet job run after a failed test and present the release as eligible. Create a packet for failed attempts if you need diagnostic history, but mark it release_status: blocked and prevent deployment.

A compact test summary should contain counts and pointers, not thousands of test names pasted into Markdown:

{
  "suite": "unit",
  "status": "passed",
  "tests": 486,
  "failures": 0,
  "errors": 0,
  "skipped": 7,
  "report": "evidence/test-results/unit.xml",
  "job_url": "https://git.example.internal/team/app/-/jobs/9921"
}

Run each class of test that can block a production release, but label it accurately. Unit tests, integration tests, migrations against an empty database, migrations against a production-shaped restore, browser tests, and smoke checks are different claims. Do not collapse them into tests: passed. If browser tests did not run because the preview environment failed to start, the packet should say not_run, not silently omit them.

You also need a policy for quarantined tests. A quarantined test is an admitted gap. It is not a passing test, and it should appear in the packet with its reason and expiry owner. If nobody owns the expiry, you have not quarantined a flaky test. You have permanently lowered the release bar.

Migration status must come from the database after deployment

A migration file in the repository proves only that somebody committed a migration. A successful migration command proves that a process exited successfully. Neither proves the target database ended at the version you expected.

Collect two records. Before deployment, save the ordered migration plan and the current schema version. After the migration job completes, query the target database through the migration tool's read-only status command and save the resulting version. The post-deploy status is the evidence that belongs in the packet.

The exact command depends on the migration tool, but the record should follow one stable shape:

{
  "environment": "production",
  "database_alias": "primary",
  "tool": "your-migration-tool",
  "before": "202607180915_add_invoice_index",
  "planned": ["202607220840_add_delivery_state"],
  "after": "202607220840_add_delivery_state",
  "status": "applied",
  "job_url": "https://git.example.internal/team/app/-/jobs/9940"
}

Do not include hostnames, connection strings, SQL bind values, table contents, or credentials. The point is to prove state, not to make an incident responder hunt through a sensitive archive.

The important engineering question is rollback compatibility. Every migration gets one of three labels in the packet:

• backward_compatible: old application code can run against the new schema.
• requires_expand_contract: deploy new code before destructive cleanup, then remove old schema in a later release.
• irreversible: reverting application code does not reverse the database state.

Teams often treat a down migration as proof that rollback is safe. It is not. A down migration can destroy data written after the forward migration, block because another release depends on the new schema, or take far longer than the incident allows. For most production systems, the safer plan is expand-contract: add the new field or table, ship code that reads both forms, migrate data if needed, switch writes, then remove the old form in a later release. The packet should state which phase this release is in.

Require an explicit exception when an irreversible migration ships. The exception needs a recovery method, such as restoring a tested backup to a separate environment, replaying a queue, or accepting that only application code can be reverted. If the team cannot write that sentence before release, it should not discover the answer after release.

Dependency changes need an inventory and a comparison

A lockfile diff is not a dependency report. It is a source-level input that frequently includes ordering noise, registry metadata, and indirect resolution changes that nobody can explain quickly. Use it for developer review. Use a normalized inventory for release evidence.

An SBOM is a useful inventory format because it records software components and their relationships. The CycloneDX SBOM guide describes an SBOM as an inventory of components and services plus dependency relationships. That is enough structure to compare the exact release artifact with the previous production artifact, rather than guessing from a pull request.

Generate the inventory from the release workspace or built artifact, then retain both inventories and a computed diff. Do not generate it later from the current default branch. A basic diff file should identify the package, old version, new version, scope, and why it appeared if your tooling can tell you.

{
  "base_release": "02bc91e-837",
  "target_release": "4f1d9a2-841",
  "added": [
    {"name": "example-parser", "version": "3.4.0", "scope": "runtime"}
  ],
  "changed": [
    {"name": "example-http", "from": "2.8.1", "to": "2.9.0", "scope": "runtime"}
  ],
  "removed": []
}

Keep runtime and development dependencies separate. A linter plugin can affect CI without changing production behavior. A runtime library can change request handling even when application code barely moved. That distinction makes the packet useful for debugging and security response.

Do not turn dependency evidence into a ceremonial vulnerability scan. Scanners are worth running, but their output has a different meaning. The dependency inventory says what was included. A scan says what one scanner concluded about known issues at a particular time, under a particular policy. Preserve them as separate files. Otherwise someone will see 0 findings and falsely infer that the artifact contains no new or risky software.

Deployment evidence starts before the deploy command

A deployment record must say what CI intended to deploy, what the target system accepted, and what verification observed afterward. Many scripts save only the first item because the command line looked correct. That is not enough when the platform substituted an image tag, rolled back automatically, or routed traffic to a different revision.

Capture these fields in the deployment job:

{
  "environment": "production",
  "requested_release": "4f1d9a2-841",
  "requested_image": "registry.example.internal/app@sha256:8ad4...",
  "deployment_id": "deploy-7b93e1",
  "started_at": "2026-07-22T16:18:04Z",
  "completed_at": "2026-07-22T16:21:17Z",
  "result": "succeeded",
  "observed_image": "registry.example.internal/app@sha256:8ad4...",
  "health_check": {"endpoint": "/healthz", "status": 200},
  "job_url": "https://git.example.internal/team/app/-/jobs/9945"
}

The post-deploy observation is where weak release automation usually fails. A deployment API returning success may mean it accepted a request, not that new instances became healthy. Query the platform for the active revision or image digest, wait for the deployment's terminal state, and run a minimal health check against the normal traffic path. Keep the check narrow. A packet is not a performance test report, and it should not contain customer data scraped from production.

GitLab job artifacts fit this workflow well because later pipeline jobs can retrieve artifacts from earlier stages, and needs:artifacts can restrict a job to the files it requires. GitLab also supports artifact access controls, including maintainer-only access. Use those controls for release packets, because deployment records and dependency inventories can reveal operational detail that does not belong in a public pipeline.

Be deliberate about retention. GitLab notes that artifact expiry follows the configured expire_in or an instance default, while the latest successful pipeline can be retained separately depending on configuration. A release packet that disappears before the typical support window is useless. A packet retained forever may create a storage and disclosure problem. Pick a period tied to how long you support deployed versions, then preserve exceptional releases under your normal incident or customer record policy.

Rollback instructions have to be executable and conditional

A rollback note is trustworthy only if an engineer can follow it without inventing missing details. "Redeploy the previous version" fails this standard because it hides the target, the database condition, and the verification step.

Generate a rollback file during the release, after the deployment system returns the previous confirmed target. It should name the exact prior image or revision, the deployment action, the migration rule, and the success test. Keep it specific to the environment.

# Rollback for release 4f1d9a2-841

Target: registry.example.internal/app@sha256:17cf...
Deployment action: ./scripts/deploy-production --image registry.example.internal/app@sha256:17cf...
Database condition: compatible. Migration 202607220840_add_delivery_state is expand phase only.
Verification: wait for deployment status `succeeded`, then request /healthz and confirm the observed image digest is sha256:17cf...
Stop condition: do not run schema rollback. Escalate if the target image is unavailable or the health check fails.

The release job must validate the target before it publishes this file. Check that the prior image exists, that the deployment principal can fetch it, and that the target is allowed in the environment. A rollback target that was deleted by image retention policy is not a rollback target.

Do not automate all rollback decisions. Automation can deploy a known prior artifact when a narrow, agreed signal fires, but it cannot decide whether newly written data remains compatible with older code. For a two-engineer team, a good rule is simpler: automate evidence and mechanics, keep the decision visible when data compatibility or customer impact is unclear.

This is one place where the popular recommendation to "always roll back on errors" is wrong. Some errors come from an upstream provider, a bad feature flag, or a migration that made backward movement unsafe. The packet gives the person deciding enough context to choose rollback, forward fix, flag disablement, or containment without guessing.

Build the packet as a final CI job

The packet job should consume evidence from test, migration, dependency, and deployment jobs, validate that required files exist, and render one readable document plus the source JSON and XML. Do not have every job append to a shared Markdown file. Concurrent jobs create ordering problems, and a failed append can leave a document that looks complete.

A simple GitLab pipeline arrangement looks like this:

stages:
  - prepare
  - test
  - build
  - migrate
  - deploy
  - evidence

release_packet:
  stage: evidence
  needs:
    - job: release_identity
      artifacts: true
    - job: unit_tests
      artifacts: true
    - job: dependency_inventory
      artifacts: true
    - job: migrate_production
      artifacts: true
    - job: deploy_production
      artifacts: true
  script:
    - ./scripts/validate-release-evidence evidence/
    - ./scripts/render-release-packet evidence/ > evidence/release-packet.md
    - sha256sum evidence/release-packet.md > evidence/release-packet.sha256
  artifacts:
    access: maintainer
    expire_in: 180 days
    paths:
      - evidence/

Your validator should fail for absence, ambiguity, or mismatch. It should reject a deployment record whose observed image differs from the requested image. It should reject a migration record with status: unknown. It should reject a rollback file missing a verified target. It should also reject an evidence file whose release version does not match the identity file. A packet generator that fills gaps with "N/A" creates a document that looks reassuring while hiding broken CI wiring.

Keep one short human-written field if you need it: a release note limited to intended customer impact, a feature flag name, or an approved exception. Put it in version control and require a reviewer for exceptions. The evidence itself should remain generated.

For teams running GitLab CI/CD and Sentry, this packet can point to the same version string used for a release in error tracking, but it should not copy an incident dashboard into CI artifacts. Sentry's release model supports associating deploys with releases, which can help correlate a production error with the deployment record. The packet remains the record of what your pipeline proved, while monitoring remains the record of what the system did afterward.

A two-engineer team should make exceptions painfully visible

The first version of this system should be small enough to finish in a week: identity, JUnit reports, migration status, dependency inventory, deployment record, rollback file, and one rendered packet. Do not wait for a perfect internal developer portal. The artifact archive and a predictable folder layout are enough.

Then review the last five packets. Look for fields that nobody used, commands that failed but did not block release, and manual facts that keep appearing in chat. Fix the pipeline rather than adding a longer checklist. Repeated manual explanation is a signal that CI does not yet collect the right evidence.

A two-engineer team also needs a plain rule for exceptions: if a normal evidence item cannot be produced, the release must say why, who accepted the gap, and when the team will remove it. An exception is not a blank field. It is a decision with an owner.

This is the kind of work that makes a smaller engineering team more capable, not more bureaucratic. AppMaster.io's production experience showed me that output does not require a large team when the team removes repetitive uncertainty from the delivery path. The release packet does that in a narrow, practical way: it leaves the next engineer with facts instead of a reconstruction project.