ADR-022: Diary Drafts via Status Column on diary_entries

Status

Accepted

Context

Issue #1426 reported that diary photos attached on the create form (DiaryEntryCreatePage) are buffered in component state and only uploaded after the entry is POSTed. If any subsequent upload fails (offline, server error, tab closed) the user's photos are silently lost, and the original File objects are no longer accessible. Several users have lost on-site photos this way that cannot be retaken.

The agreed remedy (issue #1426 acceptance criteria) is to flip the create flow so a diary entry exists server-side before any photo is uploaded:

1. Auto-create a server-side draft entry on first meaningful interaction.
2. Upload photos immediately to the draft via the existing POST /api/photos endpoint.
3. Auto-save subsequent field edits to the draft.
4. Promote the draft to a saved entry when the user clicks Save (and validation passes).
5. Drafts are visible in the diary list with a clear badge, but excluded from automatic aggregations, the dashboard "Recent diary" tile, and any export workflows.

The edit flow already does the right thing (it uploads photos immediately to an existing entry id) and is out of scope.

We need a model for representing "draft" entries that:

• Preserves a single id across draft → saved (so already-uploaded photos do not need re-attachment).
• Reuses the existing photo cascade, document_links, polymorphic source-entity references, and auto-event services unchanged.
• Minimises divergence in the CRUD endpoints (the bug fix should not require a parallel API surface).
• Lets us relax validation for draft writes without contaminating the saved-entry contract.
• Allows a small, focused migration (this is a bug fix, not a new epic).

Decision

1. Add a status column to diary_entries, do not introduce a separate diary_drafts table

diary_entries.status TEXT NOT NULL DEFAULT 'saved' CHECK(status IN ('draft', 'saved'))

• Default is 'saved' so all existing rows (created before this migration) remain in their current "fully validated" state with zero data migration risk.
• A separate diary_drafts table was rejected because promoting a draft to a saved entry would either (a) reuse the draft id and require migrating photos and document links across tables, or (b) issue a new id and require updating every photo and document link that already references the draft. Both are operationally awkward and create a window where the entry is neither in one table nor the other.
• A status column keeps the single-id invariant, lets photos.entity_id and document_links.entity_id continue to reference diary_entries.id unchanged, and means the cascade delete on draft discard works through the same path as a normal entry delete.

2. Relax server-side validation inline for draft writes — no separate endpoint

POST /api/diary-entries accepts an optional status: 'draft' | 'saved' (default 'saved', backward-compatible). When status === 'draft':

• body is optional and may be empty.
• entryDate is optional; if omitted, the server defaults it to today (new Date().toISOString().split('T')[0]).
• entryType remains required (drafts are pinned to a type at creation — see decision #7).
• Type-specific metadata fields (e.g., inspectorName for site_visit) are not validated.
• Metadata shape validation (e.g., weather must be an enum value) is still enforced — we only relax presence requirements, not data integrity.

PATCH /api/diary-entries/:id accepts the same relaxed validation when the entry is status === 'draft'. A new PATCH /api/diary-entries/:id/promote endpoint runs the full saved-entry validation and flips status to 'saved' atomically. Promoting an entry that fails validation returns 400 VALIDATION_ERROR with field-level details and leaves the entry as a draft.

This keeps the API surface narrow (one new endpoint, one new optional field on existing endpoints). A separate POST /api/diary-entries/draft was rejected because it would duplicate the entire JSON schema, the metadata validator branch table, and the controller — for a bug fix.

3. Auto-save strategy: blur for text, immediate for everything else

• Text inputs (title, body, free-text metadata fields): debounced on blur with a 1-second safety debounce after typing stops to catch users who tab away quickly.
• Selects, checkboxes, signature add/remove, photo attach/remove: immediate (no debounce) — these are discrete user actions that the user expects to "stick" right away.
• The client coalesces concurrent in-flight PATCHes against the same draft id (cancel previous, send latest). This is implemented in the frontend draft hook; the server treats each PATCH independently.

4. Orphan cleanup: daily cron, 30-day retention, hard delete with photo cascade

• A node-cron job (the dependency is already in the tree, used by the backup scheduler) runs daily at 03:00 server time. Implementation reuses the same lifecycle pattern as backupService.initializeBackupScheduler / stopScheduler.
• The job deletes diary entries where status = 'draft' AND updated_at < datetime('now', '-30 days').
• Cascade: each deleted draft also deletes its attached photos (via deletePhotosForEntity, same as deleteDiaryEntry does today).
• Configurable via DIARY_DRAFT_RETENTION_DAYS env var (default 30). Setting it to 0 disables the cleanup job (useful for tests).
• Hard delete (not soft delete) — drafts have no audit value once abandoned and disk space is the dominant concern.

PO leaning was 90 days. 30 days is a tighter default chosen because (a) construction-site photos are large and a draft can accumulate dozens of them and (b) a draft sitting untouched for a month is almost certainly abandoned. The env var lets self-hosters extend it.

5. List visibility: drafts included by default, badged, filterable

• GET /api/diary-entries returns drafts and saved entries by default (no breaking change in response shape; clients see status: 'draft' | 'saved' on each item).
• A new query parameter status=draft|saved filters one or the other. Omitting it returns both.
• The diary timeline UI renders drafts with a "Draft" badge using the existing Badge shared component (variant="warning" or equivalent — UX designer to confirm).
• Clicking a draft in the list navigates to /diary/<id>/edit (the same route as a saved entry — the form decides whether to show "Save" or "Save changes" based on status).

PO leaning was "included by default", which matches the diary-list-as-inbox model.

6. Read paths that must filter to status = 'saved'

Drafts are work-in-progress and must not leak into aggregations, automatic-event chronology, dashboard tiles, or report-like surfaces. Concrete read paths to filter:

Read path	Location	Filter
Dashboard "Recent diary" tile	client/src/pages/DashboardPage/DashboardPage.tsx calls listDiaryEntries	Pass status: 'saved' query parameter
Diary list when explicitly filtered to saved	GET /api/diary-entries?status=saved	Server enforces
Auto-event chronology (future PDF/report endpoints)	Any new listDiaryEntries-style queries from reporting code	Default status = 'saved' unless caller opts in
Signed-document workflows	Drafts can carry signatures; the lock starts at promote (#2124). isSigned is display-only, and lock decisions use isDiaryEntrySignatureLocked (@cornerstone/shared, true only when signed and status !== 'draft'). Any future "find signed entries" query that means locked/binding records must add WHERE status = 'saved'	Required
Source-entity backlink listing (future "diary for this work item" UI)	Any new "find diary entries for entity X" service	Filter to status = 'saved' when surfacing to other entities' detail pages

There is no PDF export endpoint today (it was removed during EPIC-13 UAT — confirmed in server/src/routes/diary.test.ts:473-490). When/if it is re-added, it must filter to status = 'saved' by default.

The base GET /api/diary-entries endpoint does not filter by status by default — the diary list is the user's draft inbox. Callers that want only saved entries pass status=saved explicitly.

7. Entry type is locked once the draft exists

Superseded by #2220 (EPIC-21 story 2.1, API contract 5). A draft's entryType can change through PATCH /api/diary-entries/:id; metadata keys the new type does not know are dropped, then the rest is validated for the new type. A saved entry still rejects entryType (400 VALIDATION_ERROR). See the Deviation Log. The text below is the original decision.

• The entryType is required at draft creation and cannot be changed via PATCH. The existing updateDiaryEntry service already rejects entryType changes (it is not in the update schema). This behaviour extends to drafts unchanged.
• If the user wants to switch types, they must discard the draft and start a new one. This is enforced by UI (the type selector is read-only on a draft) and by the API (entryType is not in UpdateDiaryEntryRequest).
• Rationale: type-specific metadata fields differ across types, and a type change could silently invalidate or strand metadata and photos. The discard-and-restart cost is small and matches the PO leaning.

8. Concurrent edits: last-write-wins (no version field)

• This is a 1-5 user self-hosted product. The realistic concurrency case is "the same user opens the same draft on a phone and a laptop" — not a true multi-user collaboration problem.
• No version column, no If-Match header, no conflict detection. Every PATCH overwrites the prior state for the fields it touches. The server returns the resulting entry; the client reconciles its local state to the response.
• If users repeatedly report lost edits from cross-device concurrency, revisit with a version integer column and 412 Precondition Failed semantics. Not built today to keep the bug fix focused.

9. Photo upload back-pressure: cap at 3 concurrent uploads, per-photo retry

• Client-side queue with a concurrency limit of 3 parallel POST /api/photos requests; remaining photos wait. Implemented inside the existing PhotoUpload component (or a new PhotoUploadQueue wrapper that owns the queue).
• Per-photo state: queued | uploading | succeeded | failed. The original File is retained in client memory for failed photos so the user can click "Retry" or "Remove" — that is the core fix for issue #1426.
• A failure on one photo does not block other queued photos.
• Failures are surfaced inline next to each photo (per-photo error UI), not as a single toast.
• 3 parallel uploads is a reasonable balance: enough to feel responsive on a 5G/wifi connection, low enough not to saturate a slow upload pipe with 30 simultaneous large requests.

10. URL transition: /diary/new → /diary/<id>/edit once the draft is created

• /diary/new is a thin shell that renders the type selector. Once the user picks a type and makes a first interaction that creates the draft, the page calls navigate('/diary/<id>/edit', { replace: true }).
• Using replace: true (not push) so the back button takes the user to /diary, not back to /diary/new (which would otherwise show the type selector again).
• The existing DiaryEntryEditPage becomes the host for both saved entries and drafts. The page reads entry.status and shows the appropriate buttons (Save to promote a draft vs Save changes for a saved entry; Discard draft vs Delete entry).
• No new route is introduced (no /diary/<id>/draft) — drafts and saved entries are the same resource at the same URL, differentiated by their status field.

Migration

A new migration 0033_diary_entry_status.sql adds the status column with default 'saved' and indexes draft-filtering queries:

ALTER TABLE diary_entries ADD COLUMN status TEXT NOT NULL DEFAULT 'saved'
  CHECK(status IN ('draft', 'saved'));

CREATE INDEX idx_diary_entries_status_updated
  ON diary_entries (status, updated_at)
  WHERE status = 'draft';

The partial index supports the orphan cleanup query (WHERE status = 'draft' AND updated_at < ?) without indexing the much larger saved-entry set. Backfill is trivial — DEFAULT 'saved' applies to all existing rows during the ALTER.

Consequences

Easier

• Photos are durable from the moment they leave the client. The class of bug described in #1426 disappears.
• The create and edit flows converge on a single page (DiaryEntryEditPage) — the create page becomes a thin type-selector shell that auto-creates a draft and forwards.
• The single-table-with-status model means zero new ORM relationships, no id remapping, no parallel API surface.
• Reusing node-cron (already in the dependency tree) for cleanup adds no new dependencies.
• The 30-day cleanup window is configurable per deployment via env var.

Harder

• Every read path that surfaces diary entries outside the timeline must remember to filter by status = 'saved'. We mitigate this with: (a) an explicit checklist in this ADR, (b) a service-level helper (listSavedDiaryEntries) wrapping listDiaryEntries with the filter baked in for any non-timeline caller, and (c) review-time vigilance on any new code that touches diaryEntries for reporting purposes.
• The frontend must implement an auto-save hook, an upload queue with per-photo state, and a "draft promoted to saved" UI transition. This is non-trivial frontend work but contained to two files (DiaryEntryCreatePage and DiaryEntryEditPage).
• The POST /api/diary-entries and PATCH /api/diary-entries/:id endpoints become slightly more complex: their validation branches on status. We accept this as the cost of avoiding endpoint duplication.
• Last-write-wins concurrency means the rare cross-device edit conflict silently overwrites. Documented as a known limitation; we revisit if users complain.

Future extensibility

• If we later need true multi-user collaboration on diary entries, add a version integer column and If-Match/412 semantics — the status column model does not block this.
• If we re-introduce a PDF export endpoint, it must filter to status = 'saved' by default (called out in this ADR).
• If draft volume becomes a problem before the 30-day cleanup runs (very heavy use of the create form), we can shorten the default retention without a schema change.

References

• Issue #1426 — bug report
• ADR-020 (Construction Diary Architecture) — the model this ADR extends
• ADR-019 (Discretionary Funding) — precedent for adding a status column with a backward-compatible default
• server/src/services/backupService.ts — node-cron lifecycle pattern reused for the orphan cleanup job

Deviation Log

Date	Section	Deviation	Resolution
2026-10-01	Read-path checklist (Signed-document workflows)	The row stated "A draft can't be signed today (no signature path in create flow)". Drafts have carried signatures since the auto-draft edit page gained the signature pad (#2088), and #2124 makes that the intended model.	Premise corrected: drafts can carry signatures and the lock starts at promote (#2124). Lock decisions use isDiaryEntrySignatureLocked; isSigned is display-only. The WHERE status = 'saved' filter for any future locked/binding-signature query is now required, not defensive. This table did not previously exist on this page and was created by this entry.
2026-10-19	§7 Entry type is locked once the draft exists	#2220 (P2.1, API contract 5): the capture screen (#2221) lets the user switch type after the draft exists, because the draft is created on the first character or photo. Discard-and-restart would lose that content.	Superseded. PATCH accepts entryType while status = 'draft'. Stranded metadata, the §7 rationale, is handled by dropping keys the new type does not know (per-type key lists in API contract 5; general_note keeps none) and validating the rest; photos belong to the entry, not the type, so they stay. A saved entry rejects entryType, even the stored one. A type change never issues an Undo token.
2026-10-19	§2 Relax server-side validation (saved-entry content)	§2 relaxed body for drafts and implied that a saved entry needs a non-empty body. #2220 (contract 5, §2.0 #21) lets an entry save with a title alone, or with photos alone.	Superseded. A saved entry (create, PATCH of a saved entry, promote) needs a non-blank title, a non-blank body or at least one photo (400 VALIDATION_ERROR, details.fields = ["title", "body", "photos"]). Deleting a photo is never blocked by the rule. A supplied entryDate must be a real calendar date, drafts included. The §2 sentence "drafts are pinned to a type at creation" no longer holds after the draft exists (see the §7 row).
2026-10-19	§10 URL transition (first interaction)	"The first interaction that creates the draft" included picking a type.	Refined by #2220/#2221: the draft is created on the first typed character (title or text) or the first chosen photo; picking a type alone creates nothing. This is a client rule; the server still accepts a content-free draft, which the photo-first path needs.