Your database provider says backups are enabled. That is useful. It is not yet proof that you can recover a customer’s work.

A backup is an input. Recovery is the result.

Do not ask whether a backup exists. Ask which failure you can recover from, to what moment, and how long it takes.

Name the data you would have to rebuild

Begin with an inventory. A production app rarely lives in one database.

List:

  • relational database records;
  • uploaded files and generated assets;
  • authentication identities and team membership;
  • billing identifiers and entitlement state;
  • search indexes and vector stores;
  • queued jobs and scheduled work;
  • configuration and infrastructure definitions;
  • encryption and signing keys;
  • domain, email, and third-party integration settings;
  • the application release that can read the restored data.

Mark the system of record for each item. A search index may be disposable if it can be rebuilt from the database. A user upload is not disposable merely because its metadata row survives.

This inventory exposes a common trap: the founder protects the obvious database while the irreplaceable files sit elsewhere.

Decide how much loss the product can survive

Two plain questions define the recovery target.

How much recent work can be lost? If the last usable recovery point is four hours old, can the product honestly lose four hours of customer changes? This is often called the recovery point objective, or RPO.

How long can the product remain unavailable? If restoration and verification take six hours, can customers wait six hours? This is often called the recovery time objective, or RTO.

Do not borrow targets from a large company’s compliance page. Choose them from your product’s real promise.

A weekend planning tool and a system holding a clinic’s live appointments do not need the same answer. A free beta can accept a rough recovery process if users understand the limit. A paid product that claims continuous availability cannot quietly depend on a founder finding a three-day-old dump.

Write the targets down before choosing the backup method. Otherwise the provider’s defaults become the policy by accident.

Understand what the provider actually saves

Open the current production settings and documentation. Record:

  • what is included and excluded;
  • backup frequency;
  • retention period;
  • the oldest and newest available recovery point;
  • whether point-in-time recovery is supported;
  • whether backups share the same account, region, or failure boundary;
  • encryption and access controls;
  • restore destination and overwrite behavior;
  • the plan, quota, and payment conditions that keep backups available;
  • who can start a restore;
  • whether deletion of the production project also deletes its backups.

“Daily backups” can mean nearly twenty-four hours of lost writes. “Point-in-time recovery” still has a retention window. A snapshot of a database does not include an object-storage bucket unless the provider says it does.

PostgreSQL’s current backup and restore documentation separates logical dumps, file-system backups, and continuous archiving because they have different requirements and recovery behavior. The founder does not need to operate PostgreSQL by hand to learn the lesson: the word backup does not describe one interchangeable thing.

Protect the backup from the production failure

A backup reachable through the same compromised account can disappear with production. A bad migration can be copied faithfully into every later snapshot. A billing failure can close the whole provider account. A leaked application credential may expose both live data and the copy meant to rescue it.

Use separation in proportion to the risk:

  • a different storage location or account;
  • credentials not available to the running application;
  • retention that ordinary deletion cannot immediately erase;
  • encryption with a recoverable key;
  • more than one recovery point;
  • an export format you can inspect without the original application.

CISA’s ransomware guidance recommends offline, encrypted backups and regular tests of their availability and integrity. A small startup may implement that principle with managed snapshots plus a separate periodic export. The important property is that one failure cannot erase every copy.

Do not place raw production data on a founder’s unencrypted laptop and call it redundancy. The rescue copy deserves at least the privacy promised for the live system.

Restore into an isolated place

Never test by overwriting the only known production copy.

Create a temporary recovery environment with restricted access and no ability to send customer email, charge cards, trigger webhooks, or run live scheduled jobs. Use separate credentials. Disable outbound integrations before the restored application starts.

Then perform the recovery from the written instructions, not from memory:

  1. Choose a named recovery point.
  2. Record the start time.
  3. Restore the database, files, and required configuration.
  4. Deploy the compatible application release.
  5. Run migrations only when the recovery procedure requires them.
  6. Verify representative records and relationships.
  7. Open uploaded files and generated assets.
  8. Sign in with a controlled test identity.
  9. Complete the product’s core read path.
  10. Record the finish time and every manual intervention.

AWS’s Well-Architected guidance on periodic recovery testing makes an important distinction: starting a restore is not enough. The recovered resource must be queried and checked for accessible, uncorrupted data.

The same standard applies to a two-person startup. A green provider notification proves that a restore job ended. It does not prove that the app can use the result.

Verify meaning, not only row counts

Row counts can catch an empty table. They cannot prove that the records still form a usable product.

Prepare a small recovery fixture before launch:

  • one account with a completed core workflow;
  • one team with two different roles;
  • one uploaded file;
  • one paid test subscription or safely mocked entitlement;
  • one deleted or archived object;
  • one record created near the chosen recovery point.

After restoring, verify ownership, relationships, timestamps, file access, and the visible result. Confirm that one account still cannot read another account’s data. Check that encrypted values can be decrypted with the recovered keys.

If the product cannot start because one secret, schema version, or storage bucket is missing, the drill has found a real part of the recovery system.

Test the failures you are likely to cause

Catastrophe is not limited to a data-center fire. Young products are more likely to suffer a bad command, broken migration, faulty import, permission mistake, or destructive feature.

Practice a few narrow cases:

  • restore one accidentally deleted object;
  • recover the database to just before a bad batch job;
  • rebuild a disposable search index;
  • restore files whose database rows still exist;
  • roll back the application while keeping newer data readable;
  • revoke the credential that performed the destructive action.

The old Hacker News discussion of a production database being deleted accidentally still captures the practical question founders must answer: how do you test restoration without destroying the trustworthy copy? Isolation and a written drill are the answer.

Test the paths your architecture supports. Do not claim point-in-time recovery if you have only proved a full restore from last night.

Write the runbook while the evidence is fresh

Keep the recovery note short enough to use under pressure.

It should state:

  • which failure has occurred;
  • who may declare recovery;
  • how writes or destructive jobs are stopped;
  • where recovery credentials are kept;
  • how a recovery point is selected;
  • the exact provider actions or commands;
  • how integrations are kept disabled;
  • which checks prove the data is usable;
  • how production traffic is moved safely;
  • how customers are informed;
  • how temporary restored data is destroyed afterward.

Attach the date, duration, recovery point, operator, result, and discovered gaps from the last drill. “Sam knows how” is not a runbook.

Connect this note to the launch-day runbook. A rollback returns application code to a safer version. A restore returns data to a recoverable state. Sometimes you need one, sometimes the other, and sometimes both.

Do not confuse backup with undo

Restoring the whole database to recover one deleted record may erase every valid change made afterward.

For common user mistakes, build a smaller recovery path where the risk justifies it:

  • soft deletion with a clear retention period;
  • version history for important documents;
  • idempotency around repeated destructive requests;
  • export before bulk mutation;
  • confirmation that names the exact object and consequence;
  • an administrator path to restore one item with an audit record.

These controls do not replace backups. They reduce how often the largest recovery tool must be used.

If a customer can delete an account, explain what is immediate, what remains recoverable, and when deletion becomes final. Recovery language is part of the public trust promise, not an internal technical detail.

Repeat after meaningful change

Run the drill before launch, then after changes to the database engine, hosting account, storage layout, encryption keys, authentication system, billing model, or destructive workflows. Repeat it on a schedule that matches the consequence of failure.

Automate the backup. Monitor whether it ran. Keep a human recovery drill.

A discussion on Hacker News about why testing backups is harder than saying it points to the real obstacle: restoration depends on old configuration, forgotten knowledge, and systems that have changed since the copy was made. Rehearsal exposes that decay before an incident does.

The pre-launch proof

Before real users depend on the product, you should be able to show:

  • an inventory of irreplaceable data;
  • a stated tolerable loss window and recovery time;
  • documented backup scope and retention;
  • at least one copy outside the simplest production failure path;
  • a successful isolated restore;
  • checks that prove the restored product is usable;
  • the actual time recovery took;
  • a short runbook another founder can follow;
  • a date for the next drill.

Add this evidence to the production checklist for an AI-built app. Generated code can create a backup button or a provider setting. It cannot prove that yesterday’s customer work can return.

The first restore test may fail. That is a good result before launch. You have lost an afternoon and gained the truth.