workout data pipeline - bitcobblers/wod-wiki GitHub Wiki
How workout data is created, stored, and read after V21, and what changed from the old dual-store model.
Engine execution (IOutputStatement[])
│
▼
EventRecord rows in `events` ← THE single statement/metric store
│
├── projectEventToFacts(record) → AnalyticsDataPoint (WQL filters/aggregates)
└── eventsToStoredLogs(events) → StoredOutputStatement[] (display, replay)
Session rows in `sessions` ← execution metadata only (no statements)
Session carries identity and totals (id, noteId, block content, start/end, duration, completed). EventRecord carries every statement and metric. Nothing stores the statement stream twice.
Workout execution runs in the CodeMirror editor or fullscreen timer:
-
apps/playground/src/components/organisms/editor/NoteEditor.tsxand.../review/FullscreenTimer.tsxdrive the run. - The Whiteboard runtime (
@bitcobblers/wod-wiki-engine) emitsIOutputStatements (reps, loads, elapsed time, HR). - On completion,
resultRecorder.record(...)hands the payload to the persistence seam.
The editor and fullscreen timer await the recorder before showing saved results or dismissing a reported run. Next-driven completion counts even without starting the playback loop. Stop preserves a partial run; stopping after the stack drains preserves completion. Live output counts mean captured, not saved.
A rejected write keeps the overlay open with its payload and result ID. Retry save reuses that ID and the existing session-results query. Badges and mounted Sessions listings refresh after commit.
Session metadata, event rows, and summaries share the note mutation's IndexedDB transaction in IndexedDBNotePersistence.mutateNote / IndexedDBContentProvider.updateEntry. A rejected write aborts the transaction:
-
Session row —
saveSessionwrites the flattened metadata record (scalars only). -
Event rows —
toEventRows(logs, identity)converts each statement to anEventRecord(grain: 'event', id${resultId}:${seq}) andtoSummaryEventRowsfolds Tier-2 outputs intograin: 'summary'rows;appendEvents+finalizeSummariespersist them.
The event rows are the archival record. If a later re-derivation runs (rederiveResultAnalytics), it purges the result's event rows and re-appends the replayed set, so the store converges to exactly one row set.
QueryService fetches EventRecord[] by time range, result, note, or block content, then flattens each row with projectEventToFacts(record) into one AnalyticsDataPoint per numeric metric. Filters, buckets, rollups, and cross-workout joins run on those facts. Joins are relational: resultId → session, noteId → note (and through page_notes to pages), blockContentId → same workout across notes and days.
eventsToStoredLogs(events) is the inverse of toEventRows: it rebuilds an ordered StoredOutputStatement[] from a session's event rows. Consumers:
-
sessionToPayload(session, events)reconstructs theresultspayload aHistoryEntryexposes, so review grids and analytics views keep working. -
replayResultAnalytics(block, events)feeds the headless engine to recompute Tier-1/Tier-2 outputs after edits or RPE capture.
The live runtime still hands getAnalyticsFromLogs its in-memory statement stream directly; nothing round-trips through IndexedDB mid-run.
| Removed | Why |
|---|---|
Session.data.logs |
The statement stream had two homes; event rows are now the only one. |
factRowsToEventRows / inMemoryEventStoreFromFacts
|
Legacy fixture adapters that faked event rows from flat facts. Fixtures now build EventRecord rows directly. |
| Contribution extraction from session rows | Field-catalog contributions are maintained by the event-row write paths and the catalog backfill's events pass. |
The V21 upgrade copies each session's scalar fields onto the row, projects any unprojected legacy logs into event rows (idempotent, deterministic ids), then drops data.
| Stage | Type | Location | Purpose |
|---|---|---|---|
| Execution | IOutputStatement |
runtime (in memory) | Live workout output. |
| Storage (metadata) | Session |
sessions store |
Identity, totals, lifecycle. |
| Storage (statements) | EventRecord |
events store |
The archival statement/metric rows. |
| Query | AnalyticsDataPoint |
in-memory (projectEventToFacts) |
Filterable/aggregatable facts for WQL. |
| Display/replay | StoredOutputStatement |
in-memory (eventsToStoredLogs) |
Reconstructed stream for grids, analytics views, and replay. |