Accepted (2026-10-02). Fix for review finding Issue 3 on PR #2168 (refs #2132). Applies to server/src/services/backupService.ts (createBackup, executeRestore), server/src/services/restoreSwap.ts (the synchronous swap, its marker and startup recovery) and the one recovery call in server/src/plugins/db.ts. The HTTP API is unchanged: no path, shape, status code or ErrorCode changes.

EPIC-19 shipped backup and restore without an ADR. This is the first one for the feature.

Two product trade-offs are recorded as decisions in §9 with the architect's defaults. The user may revisit either one; neither choice is load-bearing for the rest of this design.

At beta 39c19241 the shipped restore could not work in the shipped container, and the shipped backup produced torn archives:

#	Defect
a	executeRestore ran fs.rename(dataDir, dataDir + '.backup-<ts>'). /app/data is a Dockerfile VOLUME mount point, so renaming it fails with EBUSY. Even where it succeeds, the sibling /app/data.backup-* lands on the container layer.
b	The extraction temp dir was join(dirname(BACKUP_DIR), '.restore-<ts>'). With the default /backups that is /.restore-<ts>: / is root-owned, so the non-root user gets EACCES. Were it writable, the later rename into the data volume would cross devices (EXDEV).
c	createBackup wrote a consistent snapshot via db.backup() and then tarred the whole data directory, so the archive also held the live cornerstone.db, -wal and -shm, copied non-atomically. Restore used the live (possibly torn) copy, because the db plugin opens basename(DATABASE_URL); the consistent snapshot rode along as an unused extra file.
d	After getClient(db).close(), any failure left the process running with a closed connection and no rollback.

Unit tests passed throughout, because they ran against temp directories that are neither mount points nor on separate devices, as a user that can write anywhere it needs.

The archive keeps its single top-level directory, <basename(dataDir)>/ (normally data/), so v1 and v2 archives share a root. archiveStem is the archive filename without .tar.gz.

1. Take the snapshot with db.backup(dataDir/<archiveStem>.db), under the operation lock, on the data volume.
2. Write the manifest dataDir/cornerstone-backup-manifest.json: { "formatVersion": 2, "database": "<archiveStem>.db", "createdAt": "<iso>" }. Its presence is what marks an archive as v2.
3. tar.create({ gzip: true, cwd: dirname(dataDir) }, [basename(dataDir)]) with a filter that drops, at the top level of dataDir only (excluding a directory excludes its subtree):
   • the live database <dbName> and its sidecars <dbName>-wal, <dbName>-shm, <dbName>-journal;
   • every cornerstone-backup-*.db other than this backup's snapshot (strays from a crashed backup);
   • every restore-reserved name (§3): .restore-*, .pre-restore-* and lost+found.
4. One finally deletes the snapshot and the manifest from dataDir. A failed tar also deletes the partial archive.

File modes. The process's umask can narrow these modes but never widen them.

• 0600: the archive (tar.create's mode option sets only the output file's mode, not the modes of the entries), the snapshot (chmod right after backup()), the manifest, the BACKUP_DIR write-probe file, and the restore marker's temp file. A leftover temp file from a crash is deleted before it is reopened, so it cannot keep an older mode.
• 0700: a BACKUP_DIR that the backup creates, .restore-staging-<ts>/ and .pre-restore-<ts>/. An existing, operator-provided BACKUP_DIR keeps its mode, and so do archives written before this change.

The archive holds the whole database, including password hashes and sessions, so no other local user may read it. Off-host sync tools must run as the container's UID or as root; the backup guide explains this.

A v2 archive therefore contains exactly one database, a consistent backup() copy with no WAL dependency. Its WAL-mode header is harmless: opening it without a -wal file is valid.

Single source of truth for names. restoreSwap.ts exports the reserved-name predicate (isRestoreReservedEntry), the archive-exclusion predicate built on it (isExcludedFromArchive), and the name constants. The archive filter, the swap and the startup sweep all use them, so the three lists cannot drift. restoreSwap.ts imports nothing from backupService.ts, because the db plugin imports it.

Every directory the restore creates is a direct child of dataDir, so it is on the same filesystem as the data, every move is a same-device rename, and nothing outside dataDir (other than reading the archive from BACKUP_DIR) is ever written. dataDir itself is never renamed, so it does not matter whether it is a mount point. Config already rejects a BACKUP_DIR inside dataDir, so the swap never touches the backup directory.

This removes defects (a) and (b) together: no EBUSY (the mount point stays put), no EACCES (no write outside the volume the process owns) and no EXDEV (no cross-device rename).

Name in dataDir	Purpose
.restore-staging-<ts>/	Extraction target
.pre-restore-<ts>/	Current contents, moved aside during the swap
.restore-state.json	Crash-recovery marker { phase, staging, preRestore }
.restore-state.json.tmp	Atomic-rewrite temp file for the marker
lost+found	Present when the volume is a raw filesystem root; root-owned, never moved

isRestoreReservedEntry(name) is startsWith('.restore-') || startsWith('.pre-restore-') || name === 'lost+found'. Reserved entries are never archived and never moved by the swap.

Runs asynchronously after the 202, under the operation lock. Any failure leaves the live system untouched: staging is removed, the error is logged as RESTORE_FAILED, the lock is released and the server keeps serving the old data.

1. mkdir dataDir/.restore-staging-<ts> (not recursive, mode 0700).

2. Pre-pass (preflightArchive). tar.list reads the archive's headers without writing anything. Each entry header declares its size, and extraction cannot write more than that, so the sum of the declared sizes is the exact extracted size. A gzip bomb is therefore caught before a byte lands on disk. The restore fails:

   • with "Backup archive could not be extracted" if the archive is unreadable, has zero entries, or has more than RESTORE_MAX_ENTRIES (1,000,000) entries. A zero count also covers a file that is not a tar archive at all: tar.list resolves without an error on non-tar input and simply reports no entries;
   • with "Not enough free disk space to restore this backup" if sum(declared sizes) + entries × 4 KiB + 64 MiB is greater than statfs(dataDir).bavail × bsize. The 4 KiB is one filesystem block per file or directory. The 64 MiB margin covers the marker writes and the live database's growth while Phase A runs.

   There is no fixed byte cap. A cap would reject legitimate large instances, and choosing its value is a product decision; the free-space check is the limit that actually matters.

3. tar.extract({ cwd: staging, strip: 1, filter }). The filter records the first segment of the first entry as the root and rejects any entry with a different root, any entry with no second segment, and any SymbolicLink or Link entry. A rejected entry fails the restore after extraction. node-tar already blocks .. and absolute paths; the filter is defence in depth.

4. Delete any reserved name that came out of the archive (prevents a lost+found or marker collision).

5. Resolve the database by the precedence in §7, rename it to staging/<dbName>, and delete every other snapshot and the manifest.

6. Validate the staged database (§6).

Phase B is synchronous, so no request handler interleaves with it.

1. Write the marker with phase moving-aside. If that write fails, Phase B has not started: remove the marker, its temp file and staging, and fail as in Phase A ('Restore could not be started'). The database is still open and the server keeps serving.
2. stopScheduler(), only after the marker write succeeded. Stopping it earlier and then failing step 1 would leave a server that keeps serving with automatic backups silently off until the next restart. A scheduled backup that fires before this point fails harmlessly on the held operation lock.
3. getClient(db).close(). A clean close checkpoints and removes -wal/-shm.
4. mkdir .pre-restore-<ts>, then rename every non-reserved entry of dataDir into it. Flip the marker to moving-in.
5. Rename every entry of staging into dataDir. Flip the marker to swapped.
6. Finalize: delete staging, delete .pre-restore-<ts>, delete the marker last. A failure here is logged at warn and left to startup recovery.
7. Flush the logger and process.exit(0).

Each marker flip happens only after its loop completes. That gives the invariant startup recovery depends on: while the marker reads moving-aside, every non-reserved entry of dataDir is an original; while it reads moving-in, every non-reserved entry of dataDir is a restored one, and every original is in .pre-restore-*. A single "swapping" phase (the first draft of this design) could not tell the two apart after a SIGKILL.

The marker is written atomically: write .restore-state.json.tmp, fsync it, rename it over the marker, fsync the directory. It stores basenames only, and the reader rejects any staging or preRestore value that lacks its reserved prefix or contains a path separator. A crash between the temp write and the rename leaves the previous phase in place, which is still a correct description of the directory because the flip comes after the loop.

Disk first, memory second. Every flip writes the marker and only then updates the in-memory state.phase, so memory is never ahead of the disk. The reverse can happen. If the rename lands and the directory fsync then throws, the marker on disk already holds the new phase while memory still holds the old one, and the exception propagates into the rollback. This "disk ahead" case is why the rollback starts with step 0 below.

Swap failure (an exception in steps 3 to 5): roll back, log at error with both directory names, flush and process.exit(1). The connection is already closed, so the process cannot keep serving; the restart reopens the original data.

Rollback (rollbackSwap):

0. Free space, then make the marker on disk agree with the in-memory phase, before anything destructive.

   • 0a. Delete staging. Staging is never the only copy of anything. In moving-aside nothing has moved in yet, and the archive is still in BACKUP_DIR. In moving-in, what remains in staging is copies of restored data, and the originals are in .pre-restore-*. If the disk is ahead at swapped, staging is already empty. Deleting it first gives back the space that most likely caused an ENOSPC, before any marker write.
   • 0b. Rewrite the marker to the in-memory phase only if the phase on disk differs, meaning a flip's rename landed but its directory fsync threw. Otherwise write nothing. In particular, startup recovery reads its state from the disk, so it writes nothing here. A redundant write on a full volume would turn every startup recovery into ENOSPC and crash-loop the container under restart: unless-stopped.

   Without 0b, a rollback from moving-in whose disk marker already says swapped would delete the restored entries in step 1. If it then died, the next start would see swapped, finalize, and delete the originals in .pre-restore-*, so both copies would be lost. If 0b's own write fails, only staging has been touched, and the marker on disk still describes the directory: swapped means every staged entry was moved in, so recovery finalizes and keeps the restored data.

1. Phase moving-in only: delete every non-reserved entry of dataDir (all of them are restored entries). Restored entries are deleted rather than moved back into staging: staging is deleted next anyway, the archive still exists, and deletion cannot hit a name collision. Then flip the marker to moving-aside before step 2. After the deletion loop the directory satisfies the moving-aside invariant (no restored entry remains), and the flip is what makes the rollback safe to re-run. Without it, a rollback that dies part-way through step 2 (an exception, which keeps the marker and exits 1, or a SIGKILL) leaves the marker at moving-in while some originals are already back in dataDir, and the next startup's rollback would delete those originals in its step 1.

2. Both phases: move every entry of .pre-restore-* back into dataDir. A dataDir entry with the same name can only be a re-creation by a request that was in flight during the swap; delete it first.

3. Delete staging; rmdir .pre-restore-* (non-recursive, so a non-empty one throws and the marker is kept); delete the marker last.

If the rollback itself throws, the marker is kept with whatever phase is on disk at that moment, both errors are logged at error, and the process exits 1. Because of step 0 and the step-1 flip, that phase always matches the directory. Startup recovery then finishes the rollback, or, in the single case where step 0's own write failed with the disk at swapped, finalizes the restore and keeps the restored data.

A swapped marker is never rolled back; that is a caller error.

• Open it read-write with { fileMustExist: true }, not readonly. A readonly open of a WAL-header file can leave -wal/-shm in staging, which the swap would then move into dataDir. A read-write open plus a clean close also checkpoints a staged v1 -wal (precedence step 4) into the file. Mutating the staging copy is harmless. Opening a non-SQLite file throws, which is mapped to the same failure.
• PRAGMA quick_check must return ok.
• The _migrations table must exist.
• Newer-version check: the greatest applied migration name must not sort after the greatest bundled migration file (listMigrationFiles() in server/src/db/migrate.ts, shared with runMigrations). Otherwise the restore fails with "Backup is from a newer version of Cornerstone". An older level is fine, because startup runs the pending migrations.

There is no fallback. If the chosen database fails validation, the restore fails. Silently falling back to the live copy in a v1 archive would reinstate defect (c) invisibly.

A v1 archive (created before this ADR) holds data/<dbName> (live, possibly torn), possibly its -wal/-shm, and data/<archiveStem>.db (the consistent snapshot, named after the archive). Phase A step 5 picks the database in this order:

1. Manifest present (v2): use manifest.database. Unparseable JSON, a non-number formatVersion, a database that is not a bare cornerstone-backup-*.db name, or a missing file → "Backup manifest is invalid". formatVersion > 2 → "Backup is from a newer version of Cornerstone".
2. <archiveStem>.db present: the archive's own v1 snapshot. It is the consistent copy; the live file was copied after it and may be torn. (archiveStem already begins with cornerstone-backup-, so the file is <archiveStem>.db, exactly the name createBackup has always used, filename.replace('.tar.gz', '.db').)
3. Exactly one cornerstone-backup-*.db present: use it (the archive was renamed by hand).
4. <dbName> present: best effort for hand-made archives. Its -wal stays and is checkpointed by the read-write validation open.
5. Otherwise → "Backup archive contains no database".

Whenever the chosen file is not <dbName>, the staged <dbName> and its sidecars are deleted before the rename, so a v1 archive's live -wal can never be replayed onto the snapshot.

recoverInterruptedRestore(dataDir, dbName, logger) runs in the db plugin after mkdirSync(dataDir) and before new Database(dbPath), on every start. It is synchronous and costs one readdir. It is skipped for :memory: (dirname(':memory:') is the cwd). It covers what no in-process catch can: a SIGKILL, OOM kill or power loss in the middle of the swap.

Marker	State of dataDir	Action
none	Normal, possibly with leftovers from a crashed backup or Phase A	Sweep the top level: delete .restore-staging-*, .restore-state.json.tmp, the manifest, and every cornerstone-backup-*.db except <dbName>. A .pre-restore-* is rmdir'd if empty; a non-empty one is left in place with a warn, because every path deletes the marker last, so one cannot legitimately exist, and if it does it may be the only copy of the user's data.
moving-aside	Non-reserved entries are all originals; the rest are in .pre-restore-*	rollbackSwap (§5), then warn "Interrupted restore was rolled back; original data reinstated".
moving-in	Non-reserved entries are all restored; all originals are in .pre-restore-*	rollbackSwap (§5, including the flip to moving-aside after deleting the restored entries), then the same warn.
swapped	Restored data in place	finalizeSwap, then info "Completed cleanup of a finished restore".
unparseable, unknown phase, or a name failing validation	Unknown	Throw; startup fails with a message naming the marker. Guessing which side to delete risks data loss, so a human inspects it.

All recovery actions are idempotent, so a crash during recovery is recovered by the next start.

• Restart policy. Restore ends with process.exit(0), and a rolled-back swap with exit(1). The container must run with restart: unless-stopped (as docker-compose.yml ships) or always. on-failure does not restart after exit 0, and a bare docker run without --restart stays stopped until started by hand. Testcontainers has no restart policy, so the container-level E2E round trip restarts the container itself and asserts State.ExitCode === 0 first.

• Requests during the swap may fail with 500 between close() and exit. Accepted for under 5 users; there is no maintenance mode.

• Photos. The swap moves every non-reserved top-level entry, so the default PHOTO_STORAGE_PATH ({DB_DIR}/photos) is restored together with the database, one rename per directory rather than a copy. A PHOTO_STORAGE_PATH outside dataDir is neither backed up nor restored. That is unchanged by this ADR and now documented as a limitation.

• Full replacement. Anything in dataDir that is absent from the archive (including photos added since the backup) is removed.

• Disk space. A restore temporarily needs free space on the data volume of about the extracted archive size. The current data is moved, not copied. The Phase A pre-pass (§4 step 2) refuses a restore that would not fit, before anything is extracted. Renames need no free space, but every marker write needs a block. If the volume still fills up during Phase B (for example from other writers):

  • an ENOSPC on the first marker write fails as "Restore could not be started";
  • an ENOSPC on a later flip rolls back. The rollback deletes staging before any marker write (§5 step 0a). A rollback from moving-aside writes no marker. A rollback from moving-in writes its next marker only after deleting the restored entries.

• Synchronous validation is accepted. PRAGMA quick_check on the staged database (§6) runs on the event loop and blocks request handling for its duration, which grows with database size. This is accepted rather than moved to a worker thread.

  • On success, it runs after the 202 and right before a planned process.exit(0). The service is already restarting, so blocking costs nothing extra.
  • Only an archive rejected in Phase A gives the blocked time back to a running server. That is a rare, admin-only action.
  • A worker would need a separate build path, because ts-jest's ESM setup cannot load a .ts worker.
  • A size cap on the staged database would reject legitimate large databases, and that would be a product decision.

  Revisit this if restores of multi-GB databases become routine.

Decision A (recorded default; the user may revisit): Phase A rejections stay log-only. The client shows "Server is restarting" after the 202. If the archive is corrupt, from a newer version or has no database, the server logs RESTORE_FAILED, releases the lock and keeps serving the old data, and the admin sees no signal beyond the server not restarting. The alternative, extracting and validating before replying, puts a large-archive extraction under the HTTP request timeout and needs a client change. This behaviour is unchanged from before this ADR.

Decision B (recorded default; the user may revisit): no undo copy is kept. On success, .pre-restore-* is deleted immediately. The previous documentation promised a data.backup-<ts> copy "until the next restart"; that promise was never kept (defect a) and is removed. Keeping the copy would double disk use on photo-heavy instances. The documentation tells users to take a fresh manual backup before restoring.

Alternative	Why rejected
Keep renaming dataDir, document "do not mount a volume"	The shipped Dockerfile declares VOLUME ["/app/data"], so every real deployment has a mount point. Asking users not to persist their data is not an option.
Stage under BACKUP_DIR or /tmp	Cross-device rename into the data volume (EXDEV) would force a copy, which is not atomic per entry and doubles the time and space for photos.
Copy files into place instead of renaming	Not atomic; a crash mid-copy leaves a mixed directory that no marker phase can describe.
One swapping marker phase	After a SIGKILL, recovery cannot tell restored entries from originals in dataDir (§5).
Restore from the live database in v1 archives	Reinstates the torn-copy defect (c).
Validate before replying (synchronous restore up to the swap)	See Decision A.
Keep .pre-restore-* as an undo copy	See Decision B.
Treat an invalid marker as "no marker" and sweep	The sweep could delete the only surviving copy of the data. Failing startup loudly is the safer default for a state that should never occur.

• Restore works in the shipped container: non-root user, /app/data as a volume mount point and /backups on another device. The container-level E2E round trip (e2e/tests/admin/backup-restore-roundtrip.spec.ts) is the regression test for exactly that setup.
• Every archive written from now on restores a consistent database.
• A crash at any point in the swap is recovered on the next start without manual work.
• Old (v1) archives still restore, preferring their consistent snapshot.

• The restore machinery owns names in dataDir. Any future feature that writes top-level entries into dataDir must avoid the reserved prefixes, and anything it writes there is archived and swapped.
• An invalid marker blocks startup until someone inspects it. That is deliberate.
• Startup now does a readdir of dataDir before opening the database. The cost is negligible.

• PR #2168 review, Issue 3; issue #2132
• server/src/services/restoreSwap.ts, server/src/services/backupService.ts, server/src/plugins/db.ts, server/src/db/migrate.ts
• Architecture › Backup & Restore
• API Contract: POST /api/backups, POST /api/backups/:filename/restore
• ADR-003 (SQLite and better-sqlite3)

Date	Section	Deviation	Resolution
2026-10-02	§5 marker writes and rollback	PR #2169 hardened the swap after this ADR was written: every flip persists to disk before memory is updated, and rollbackSwap first makes the disk marker agree with the in-memory phase. That closes a data-loss path when a flip's rename lands but the directory fsync throws. The ADR described neither, and it said the rollback keeps "its current phase".	Added "Disk first, memory second", rollback step 0 and its failure semantics, the reworded "rollback itself throws" paragraph, and the ENOSPC note in §9. Step 0 is specified as a write only when the disk is ahead, the form the PR #2169 review requires (finding D1).
2026-10-02	§1 file modes, §4 pre-pass, §5 rollback step 0, §9	PR #2169 review round 1 (architect D1, security S1/S2). Step 0 always rewrote the marker, which is redundant at startup, and on a volume that staging had just filled it turned every startup recovery into ENOSPC. Archives and the restore working directories were created with the default (umask-derived) modes. Phase A extracted the archive without bounding the entry count or the extracted size, and validation runs synchronously.	Step 0 now deletes staging first and rewrites the marker only when the disk phase differs (0a/0b). Added the pre-pass (1,000,000-entry cap, free-space check on header-declared sizes, new message "Not enough free disk space to restore this backup"), the file modes (0600 files, 0700 directories), and the recorded acceptance of the synchronous quick_check.
2026-10-02	§4 step 2 (pre-pass)	PR #2169 review round 2. The implementation also rejects an archive with zero entries, because tar.list resolves without an error on non-tar input. The pre-pass description named only an unreadable archive and the entry cap.	The zero-entry rejection and the reason for it are added to the pre-pass list.