AI migration release checklist for safer production changes

Why generated migrations need extra care

AI can draft a migration in seconds. Your database still pays the full price if it is wrong.

One generated UPDATE with a loose filter can change 50,000 rows before anyone notices. A bad DELETE can do even more damage, faster. Many of these migrations look fine in review. The SQL is tidy, the names look sensible, and the logic reads well. Then the script meets real production data, where old records, null values, duplicates, broken formats, and years of odd edge cases are waiting.

A simple example is a new default value. On paper, changing a column from null to active sounds harmless. In production, that default might mark old trial accounts as paying customers, trigger the wrong emails, or break reporting for past months. The query ran exactly as written. The data still ended up wrong.

Some failures have nothing to do with business logic. A migration can lock a busy table longer than expected. A column type change can rewrite a large table and slow the app to a crawl. An index build can block writes during a release window that looked generous. Teams often learn this only after users can no longer save changes.

The usual risks are simple: data loss from deletes or overwrites, bad defaults copied into live rows, locked tables that stop traffic, and long downtime from slow backfills or table rewrites.

AI makes drafting faster. It does not make changes safe on its own. Models are good at producing likely SQL patterns, but they do not know the messy history inside your database unless someone checks every assumption. Treat every generated migration like a power tool - useful, fast, and fully capable of wrecking production records in one pass.

Set the risk before you ship

Generated migrations often fail in ordinary ways. The SQL runs, no syntax errors appear, and the data still ends up wrong. So treat the change as a business risk first, not just a code change.

Start with a plain map of what the migration can touch. Write down the tables, the columns, and a rough estimate of how many records may change. If the script fills a new column from old data, note that too. Small edits can spread farther than they seem, especially when one table feeds billing, reports, or admin tools.

A short risk note is enough if it answers four questions: what data the migration will read and write, which user actions depend on that data, how many records may change, and who decides whether the release continues or stops.

Describe failure in human terms. "Migration failed" is too vague. A useful note says what people would actually see: orders with the wrong status, empty invoices, staff unable to search for customers, or support seeing duplicate records. If you can picture the bad outcome, you can spot it faster during release.

Set a stop rule before anyone starts. Do not invent it under pressure. You might stop the release if a validation query returns the wrong row count, if the migration runs far longer than expected, or if one acceptance test fails on production-like data. The rule should be simple enough that nobody argues about it at 2 a.m.

Approval also needs names, not floating roles. Pick one person to approve the release, one person to run it, and one person who can cancel it on the spot. In a small team, two of those jobs may belong to the same person. Write it down anyway.

Ten minutes spent naming the risk is far cheaper than two days spent guessing which records changed and who should make the call to stop.

Freeze the exact change set

Many migration failures start before release day. A team tests one set of files and ships a slightly different one. With generated migrations, that drift happens fast because someone reruns a prompt, edits the SQL, or slips in a small change that nobody checked.

Freeze the release candidate before final testing begins. Save the exact migration files, the prompt that created them, and every manual edit in one place. If you use Git, pin the commit. If a config flag changes how the app reads or writes data, pin that too.

Keep the frozen set simple: one app version, one schema version, one config set, one approved migration folder, and one short release note.

That note should explain the intent in plain English. Something like "split full_name into first_name and last_name, copy existing values, keep the old column for one release" gives everyone the same target. Reviewers can compare that note with the SQL and the app code without guessing what the model meant.

Remove extra edits that never went through review. Small changes cause real trouble here. A renamed index, a changed default, or a quick fix in app code can make your dry run look clean while production behaves differently. If the test used one config and production uses another, you did not test the real release.

A frozen change set also makes handoff easier. If one engineer prepares the migration and another runs the release, they should see the same files, the same prompt history, and the same release note. No guesswork. No "latest version." No mystery patch living on one laptop.

Dry run the migration step by step

Use a safe environment with recent data that is close to production. A toy dataset does not tell you much. SQL that looks fine on a few sample rows can slow down, fail, or damage edge cases when it meets real volume.

Start with a fresh copy of the database structure and realistic data shape. If you need to mask sensitive fields, keep the odd records intact: old accounts, blank values, duplicates, and half-finished rows. Those cases expose bad assumptions in generated migrations all the time.

Run the migration once exactly as you plan to run it on release day. Use the same order, the same scripts, and the same app version. Time the full run. If it takes 12 minutes in a safe environment, do not assume production will finish faster.

After the run, inspect the data itself, not just the success message. Compare row counts before and after. Check for unexpected null values, changed defaults, and truncated text. Confirm that indexes exist and constraints still work. Then open the app and test the flows that depend on the changed data.

That last step matters more than many teams expect. A migration can finish with no SQL errors and still break signup, search, billing, or an admin page because the app expects the old data shape.

Then reset the environment and run the whole process again from a clean copy. The second run often exposes hidden state, missing preconditions, or manual steps nobody wrote down. If the result changes between runs, the process is not ready.

For teams moving quickly with AI, a repeatable dry run is still one of the cheapest ways to catch a bad migration before it touches production.

Take a snapshot you can restore fast

Take the snapshot as late as you safely can. If you create it hours before release, new orders, edits, or account changes can slip through the gap. For a migration that changes live records, a snapshot from five or ten minutes before the run is usually far more useful than one from earlier in the day.

Teams often skip this because they see a finished backup job and assume they are covered. They are not. A backup that nobody has restored is still a guess.

Rehearse once before release day. Restore the snapshot into a separate environment and make sure the database starts, the tables are readable, and the app can connect if needed. You do not need a perfect copy of production for this check. You need proof that the snapshot is real and usable.

Record four facts during that rehearsal: when you created the snapshot, how long the backup took, how long the restore took, and the exact command or steps needed to restore it.

Those numbers matter. If backup takes 12 minutes and restore takes 18, the team now knows the real recovery window. That changes when you schedule the deploy, who stays online, and how long you can wait before calling a rollback.

Store the restore command somewhere the whole release team can reach in seconds. Put it in the release notes, the runbook, or the deployment ticket. Do not leave it in one engineer's shell history.

A small example makes the point. Say a generated migration splits one customer table into two tables and rewrites status values. The team takes a snapshot at 6:55, starts the release at 7:00, and restores that snapshot in staging first. They learn the restore needs 14 minutes, not the five they guessed. That one fact can save a miserable night if production data goes sideways.

Practice the rollback

A rollback plan is part of the release. It is not a rescue note you write after trouble starts.

If a generated migration changes live records, the team needs clear steps before release day. Write them down in order, with exact commands, stop points, and the person who gives the go-ahead.

Use the same safe copy you used for the dry run. Rollback can look fine on a toy dataset and fail on data with real duplicates, nulls, old formats, or partial updates. Run the forward migration first, confirm the change, then run the rollback exactly as you would under pressure.

Some changes do not reverse cleanly. A rollback script can put back a column or undo a schema change, but it often cannot recreate deleted rows or separate records that were merged. If the migration combines two customer records into one, you need a saved map of what changed. Without that map, you may restore the table shape and still lose the original data state.

A good rollback test answers four plain questions: who starts the rollback, what signal tells the team to stop and reverse, how long rollback takes, and what data it cannot recover on its own.

Time matters more than many teams expect. If rollback takes 18 minutes and cache rebuild takes another 12, half the release window is already gone. Record the full time, not just the database step. Include checks after rollback too, such as row counts, spot checks on recent records, and one business action that proves the app still works.

This is where the checklist earns its keep. AI can draft a reverse migration, but it cannot tell you how much downtime your team can tolerate or who actually has production access at 11 p.m. Teams with tight release habits usually handle this better because every role and step is explicit.

If the rollback feels slow, confusing, or incomplete in rehearsal, delay the release. That is cheaper than discovering the gaps on production data.

A simple example from a release

A small SaaS team needed to add a required field called customer_segment to its customers table before a billing update could ship. An AI tool wrote the migration: add the column, backfill it from billing history and account notes, then mark it NOT NULL. One engineer prepared the change, and the CTO had the single go or no-go approval for the release window that night.

The dry run changed the plan. The team restored a recent copy of production data into staging and ran the migration step by step. The schema change worked, but the backfill left hundreds of rows empty because many older customers had no billing history and messy notes. If they had pushed that script straight to production, the final constraint would have failed halfway through the release.

So they split the release in two. First, they added the new column. Then they backfilled only rows with clear source data and sent rows with missing values to a review queue.

That made the release slower, but much safer.

Before the real deployment, the team took a fresh snapshot and tested a restore on another server. Then they practiced rollback. The engineer timed how long it took to stop the app job, remove the new write path, and restore changed rows from the snapshot. The full sequence took about 12 minutes, which fit inside their 30-minute window.

That practice paid off. During production, the revised backfill started writing the wrong segment for a small batch of archived test accounts because the join condition was too loose. The team caught it from row counts they had written down before the release. They stopped the job, rolled back the app change, restored the affected rows from the snapshot, and closed the window without damaging customer records.

The useful part was not the generated migration by itself. It was the discipline around it: dry run first, snapshot before release, and a rollback they had already rehearsed.

Mistakes teams make right before release

Most release failures do not start with exotic bugs. They start when a team trusts generated SQL because it looks tidy and the test run passed once.

The first mistake is simple: nobody really reads what the migration will touch. A generated UPDATE may look correct, but the affected rows tell a different story. If a query changes 200 rows in staging and 2 million in production, you no longer have the same release. Inspect row counts, sample records, and the exact filters before you run anything for real.

The next problem is test data. Many teams use a clean database or an old copy that lacks weird records, nulls, duplicates, and half-finished states from old app versions. Generated migrations often look fine on neat data. Production data is rarely neat.

Backups create false confidence too. A backup that nobody has restored is just a file. Teams take a snapshot, feel safe, and move on. Then the release goes wrong, the restore takes far longer than expected, or the recovered data no longer matches the app version they shipped.

Timing checks get skipped for the same reason. A migration that runs in 12 seconds on a laptop can run for an hour on a busy table with indexes, locks, and live traffic. That delay hurts twice: users feel it, and the release window closes while the team is still waiting.

The messiest failures happen when app code and schema changes land out of order. A team deploys code that reads a new column before the migration has created or backfilled it. Some requests hit the old shape, others hit the new one, and errors start showing up in places that seem unrelated.

Picture a team adding a required status field. They ship the app first and trust the backfill. The backfill stalls on unexpected nulls in old records. New requests start failing because the code already expects status to exist everywhere.

Right before release, skepticism beats speed. Read the affected rows, test with messy data, restore the snapshot once, measure runtime, and make sure the app can survive while the schema changes.

A short release day checklist

Release day should feel boring. That is usually a good sign.

Before anyone runs the migration, confirm a few plain facts:

• The dry run passed against a recent copy of production data.
• The team took a fresh snapshot and proved they can restore it quickly.
• The rollback steps are written in plain language, and each step has an owner.
• The release window is agreed in advance.
• Post-release checks are ready before the migration starts.

Small details matter here. Use the exact migration file you tested, not a last-minute edit. Keep the SQL, app version, environment settings, and runbook in one place so nobody has to guess under pressure.

A simple rule helps: if one person has to explain the whole plan from memory, the plan is too loose. Write it down. Put names next to tasks. Pick a clear stop point where the team says "go," "pause," or "roll back."

If any item still feels vague, delay the release. Shipping an hour later is cheaper than cleaning up damaged production data for a week.

What to do next

A safe release process should not depend on who happens to be online that day. Turn your migration checklist into a short template that every team member can follow the same way.

Keep that template next to the migration file, release notes, and rollback commands. When pressure rises, people make fewer mistakes if the plan is already written down.

For every generated migration, add one more review before approval. One person should read the change itself. Another should confirm the release plan: dry run results, snapshot timing, restore steps, and the exact point where the team stops and rolls back.

That review does not need a long meeting. Ten careful minutes before release can save hours of cleanup later.

After each deployment, write down what actually happened. Record the real runtime, any warnings or failures, what the team fixed, and whether the rollback plan still looked realistic once the change was live.

A short template can include the migration name and owner, the dry run result and expected runtime, snapshot status, restore test results, the rollback command, and notes from the live release.

These notes shape the next release. If one migration took 90 seconds longer than expected, or if a restore test felt slower than your release window allowed, the next plan should change.

Keep the process lean, but do not remove the parts that protect production data. A short checklist that people trust is better than a long document nobody reads.

If your team wants a second set of eyes before a risky release, Oleg Sotnikov at oleg.is works as a Fractional CTO and startup advisor and often helps small teams tighten release and rollback processes around AI-driven software changes.