Why the obvious migration takes you down
During any rolling deploy, old and new code run at the same time for a window. A migration that renames or drops a column assumes the schema and the code change atomically — they don't. The old instances hit a column that no longer exists and 500. And some operations (adding a non-null column with a default on a large table, certain index builds) take a heavy lock that blocks reads/writes while they run.
Expand and contract
The pattern decouples the schema change from the code change into backward-compatible steps. Instead of one breaking migration, you expand the schema to support both old and new, migrate the code and data, then contract by removing the old shape once nothing uses it.
- Expand: add the new column/table — additive only, so old code is unaffected.
- Backfill: copy data to the new shape in batches, not one giant locking UPDATE.
- Migrate code: deploy code that writes to both and reads from the new column.
- Contract: once no code reads the old column, drop it in a later deploy.
-- Renaming email_addr -> email, zero downtime
-- Step 1 (deploy A): add new column, no default rewrite
ALTER TABLE users ADD COLUMN email text;
-- Step 2: backfill in batches to avoid a long lock
UPDATE users SET email = email_addr WHERE id BETWEEN $1 AND $2;
-- Step 3 (deploy B): app writes both, reads `email`
-- Step 4 (deploy C): once nothing reads email_addr
ALTER TABLE users DROP COLUMN email_addr;Every individual step is safe to run and safe to roll back on its own. That's the whole point: you never have a moment where the deployed code and the live schema disagree.
The locking traps
- Backfill in batches with a short transaction each — a single UPDATE over millions of rows holds a lock and bloats the WAL.
- Build indexes concurrently (CREATE INDEX CONCURRENTLY in Postgres) so writes aren't blocked.
- Adding a NOT NULL column with a volatile default can rewrite the whole table — add nullable, backfill, then add the constraint.
- Set a short lock_timeout so a migration that can't get its lock fails fast instead of queuing every query behind it.
Key takeaways
- Old and new code overlap during rollout — migrations must be backward-compatible.
- Expand → backfill → migrate code → contract, each step independently safe and reversible.
- Backfill in batches; build indexes concurrently; avoid full-table-rewrite operations.
- Zero-downtime schema change is a discipline, not a maintenance window.