Repeating Task Evidence Developer Guide - edmozley/freeitsm GitHub Wiki

🛠️ Repeating task evidence - Developer Guide

How a task keeps its history, how a repeat keeps its rule versions, and how the evidence report turns that record into "every occurrence that should have happened, and what happened to each". Built for #181; the user guide is Repeating task evidence and the design (with Dr4inder's three clarifications in section 9) is Recurring task evidence - design.

The one-line version: TaskHistory (includes/services/task_history.php) writes an append-only record with no foreign keys, through one status hook every status writer calls; TaskEvidence (includes/task_evidence.php) reads it - with #182's run history - and judges each occurrence. Viewing never writes a task.

1. The files

🗄️ schema · 📖 read · ✏️ write · 🔗 cross-module · 🖥️ UI / help · 🧪 tests

File	🎨	What happens there
`database/freeitsm.sql`, `includes/db_verify_schema.php`	🗄️	`task_events`, `task_recurrence_versions`, `task_removed_records`, `task_checklist_item_events`, `checklist_template_versions`; task columns `scheduled_date`, `previous_occurrence_id`, `created_via`, `completed_by_id`, `skipped_datetime`; `result_mode` / `result` / `exception_note` on checklist items, `version_no` on templates, `template_version` on task checklists
`api/system/db_verify.php`	🗄️	Baselines: one `baseline` rule version per existing series, version 1 of each existing template; the four `tasks_evidence_*` settings
`includes/services/task_history.php`	✏️📖	`TaskHistory`: `record()`, `afterStatusChange()` (the hook), `recordRuleVersion()`, `recordRemoval()` + `snapshot()`, `forTask()`
`includes/services/tasks.php`	✏️	Calls the hook from `updateTask` and `moveTask`; date / assignee events; `skipOccurrence()`; the delete setting and `recordRemoval()` in `deleteTask`; `assertCanWrite()` on every write (Read only)
`api/tasks/reorder.php`, `api/tasks/toggle_subtask.php`	✏️	The other two status writers - they call the hook too
`includes/services/task_recurrence.php`	✏️	`onTaskClosed()` records the successor expectation before creating it, at most one successor per occurrence; `createOccurrence()` sets `scheduled_date` / `created_via` / `previous_occurrence_id` and records `created` with the job run; `deactivate()` records `finished`; "today" is the install's date
`api/tasks/save_recurrence.php`	✏️	Rule versions: `created`, `changed`, `resumed`, `stopped`
`includes/services/checklists.php`	✏️	Step history (`recordStepEvent`), `setTaskItemResult()`, `assertTaskChecklistChangeAllowed()`, `recordTemplateVersion()`, template version on copy, checklist removal recorded with its content
`checklists/api.php`, `checklists/edit/index.php`	✏️🖥️	Saving a template records a version; the editor's Ask for a result toggle
`includes/task_evidence.php`	📖	`TaskEvidence::report()`, `seriesList()`, `seriesHeader()`, `explainGap()`, `recordExport()`, `describeRule()`
`tasks/evidence/index.php`, `tasks/evidence/report.php`, `assets/css/task-evidence.css`	🖥️	Tasks → Evidence, and the report (screen and print)
`api/tasks/evidence_export.php`	📖	The three CSVs, each recorded
`api/tasks/history.php`, `api/tasks/skip.php`, `api/tasks/get.php`	📖✏️	History tab; Skip; `can_skip`, `can_evidence`, `can_write`, `completed_by_name`
`assets/js/tasks.js`, `assets/css/tasks.css`	🖥️	History tab, Repeats (scheduled date, Execution history, Skip), result buttons, read-only styles
`tasks/settings/manifest.php`, `tasks/settings/index.php`, `api/tasks/get_settings.php`, `api/tasks/save_settings.php`	🖥️	The Evidence tab, `Cap::TASKS_EVIDENCE` (tab), `Cap::TASKS_EVIDENCE_REPORT` (everyday work)
`includes/functions.php` (`MODULES_WITH_READ_ONLY`), every `api/tasks/*` writer	🔗	Read only for Tasks: `requireModuleWriteJson('tasks')`
`includes/documents.php`, `api/documents/attach	unlink	save.php`
`includes/job_runs.php`	🔗	`JobRun::current()` - the run an occurrence was made by (Background jobs - Developer Guide)
`tasks/help.php`, `lang/en/tasks.php` (`history`, `evidence`, `help.evidence_*`)	🖥️	In-app help and strings
`tests/task-evidence.php`, `tests/task-read-only.php`	🧪	Below

2. The record outlives what it describes

🔴 None of the five new tables has a foreign key. The auditor's question has to survive an occurrence being deleted, the series being stopped or its tasks removed, and the checklist template being deleted (Dr4inder's third clarification). A CASCADE would delete the evidence with the thing it is evidence about. Rows are append-only.

Table	One row per	Notes
`task_events`	thing that happened to a task	`event_type`, actor (`analyst_id` / `actor_kind` / `actor_name` as it was then), `from_value` / `to_value`, `detail_json`, `job_run_id`. `task_id = 0` for a series-level event (`evidence_exported`)
`task_recurrence_versions`	save, stop, resume or finish of a rule	`rule_json` snapshot, `change_kind`, plus the series' title, first task and company - so a series with no tasks left can still be listed and scoped
`task_removed_records`	deleted occurrence	`snapshot_json` from `TaskHistory::snapshot()`: the task, checklists with every item, step history, events, audit, comments, subtasks, document references
`task_checklist_item_events`	tick, untick, value or result	before / after JSON, `after_completion`
`checklist_template_versions`	template save	full snapshot; `checklist_templates.version_no` is the newest

Event types

created (detail.via: manual / repeat_completion / repeat_schedule / repeat_catchup) · status_changed · completed (to_value = the due date in force) · reopened (detail: the completion it undid) · skipped (to_value = reason) · due_date_changed / start_date_changed (detail task_closed) · assigned / team_assigned · successor (detail expected, reason, existing_task_id; to_value = expected due) · checklist_attached / checklist_removed (detail after_completion; removal carries the checklist) · removed · evidence_exported.

3. The one status hook

A task's status is written in four places:

Writer	Path
Task window, REST `PATCH`	`TasksService::updateTask()`
REST `POST /tasks/{id}/move`	`TasksService::moveTask()`
The board's drag	`api/tasks/reorder.php` (not on the service - it sends client positions)
Ticking a subtask	`api/tasks/toggle_subtask.php`

Each one reads TaskHistory::before() before its write, then - inside the same transaction as the status write - calls TaskHistory::recordStatusChange($conn, $ctx, $taskId, $before, $newStatusId), commits, and only then calls TaskHistory::afterCommit($conn, $ctx, $result).

Half	When	What	On failure
`recordStatusChange()`	In the transaction	`status_changed`; on completing, `completed_by_id` and `completed` with the due date then in force (or `skipped`); the procedure texts (#190); for a top-level occurrence, whether a successor is now expected (`TaskRecurrence::successorExpectation()`); on reopening, `reopened` and what it undid	Throws `ServiceError evidence_not_recorded`; the caller rolls back, so the status does not change
`afterCommit()`	After the commit	Makes the successor (`TaskRecurrence::createSuccessor()`)	Logged. The completion stands, and the expectation is already recorded, so the report shows the successor as Missing

Atomic evidence (Dr4inder's review, 20 Oct)

Until this, the hook ran after the commit and every history write caught its own errors: a database hiccup at the wrong moment left a task completed, moved or deleted with no record of it. For evidence, usually recorded is not recorded.

TaskHistory::recordStrict() writes an event and throws on failure; record() (unchanged, logs and carries on) is kept for non-evidence callers. Both share one writer. Before Database Verification (TaskHistory::ready() false) strict writes do nothing - there is no evidence system to fail.
updateTask() now runs its whole write in one transaction (its own, or the caller's): the update, tags, the completion override (recordTaskCompletionOverride(..., strict: true)), the status hook, and the date and assignment events. Workflow events, the successor and the calendar follow the commit.
moveTask(), skipOccurrence(), api/tasks/reorder.php, api/tasks/toggle_subtask.php: the evidence half before their commit.
Deleting an occurrence: the retained record is built first with TaskHistory::snapshot(..., strict: true) - a failed read throws instead of becoming an empty section (a table that does not exist on the install, SQLSTATE 42S02, still reads as empty) - and written in the delete's transaction (recordRemoval(..., $snap, strict: true)). No retained record, no delete. The calendar clean-up runs between the two, outside the transaction, because it is a network call; if the write then fails, the calendar entry is gone but the task is not, and the next save of the task puts the entry back.
TaskForms, TaskProcedures and TaskFollowUps snapshots, TaskProcedures::freeze() and the completion override all take the same strict flag.

The person sees "Nothing was changed: the record of this change could not be saved" (tasks.history.not_recorded).

🔴 TRAP: a fifth status writer must call the hook. Before #181 only two of the four paths called the repeat, so dragging a card into a closed column never made the next occurrence of an after-completion repeat - the exact gap Dr4inder's second clarification is about. tests/task-evidence.php drives api/tasks/reorder.php over HTTP to keep it fixed.

skipOccurrence() goes through the same hook with a reason, which makes it record skipped instead of completed.

4. Successors (after completion)

onTaskClosed(), in order:

Rule stopped → successor with expected: false, reason: stopped. Nothing created.
Rule produces no date or has run out → expected: false, and the rule is deactivated with a finished version.
A task already has previous_occurrence_id = this one → expected: true, existing_task_id, nothing created. This is what stops reopen-then-complete making a second successor.
Otherwise record successor with expected: true and the date - before creating - then createOccurrence(..., 'repeat_completion', $taskId).

So if creation fails, the expectation is still on the record, and the report shows the gap: an occurrence whose latest successor event says expected, with no task (live or removed) whose previous_occurrence_id points back at it, is Missing. A reopened after the successor event cancels the expectation.

5. Judging an occurrence

TaskEvidence::judge() - one function for live occurrences and for frozen ones, so a removed occurrence is judged exactly as it would have been.

Original due = scheduled_date (or, for occurrences made before #181, due_date, flagged scheduled_not_recorded).
Due at completion = to_value of the latest completed event (after any reopened). Without one, the current due_date, flagged due_history_not_recorded.
Completion day = completed_datetime converted to the install's time zone (date_default_timezone_get()), because due dates are calendar days.
Outcome: completion day > due at completion → late; else > original → on_time_rescheduled; else on_time. Skipped if skipped_datetime or the latest close was skipped. Open / overdue against the current due date.
Flags: rescheduled (due at completion ≠ original, or current ≠ original while open), due_changed_after_completion (any due_date_changed after the close - never re-judges), created_late (repeat_catchup, or created after the begin date), changed_after_completion, the three not-recorded flags.
Exceptions: failed result steps, and the task_audit row Completed with checklist steps outstanding.

Missing, fixed schedule

expectedDates() walks from the series' first occurrence with TaskRecurrence::nextDate(), using the rule version in force at each date, stops at an end date or count, and drops dates whose begin date falls in a stopped span (a stopped/finished version until the next resumed/changed). A date with no occurrence and no removed record is Missing; one whose begin date is today is not created yet; future ones are not listed. Where the first version is a baseline, nothing before its date is called missing - the rule was not recorded then - and the report says so.

What the run history says

explainGap($fromDate, $createdUtc, $runId) reads job_runs for task_recurrence between the day work should have begun and creation (or now). Codes: history_not_available (before_recording - before job_runs_recording_since; beyond_retention), no_recorded_run, job_failed, ran_but_not_created, created_by_catchup[_after_runs], created_late_runs_seen, and for successors successor_not_created. Never "did not run" - Dr4inder's wording on #182.

6. Settings and permissions

Setting key	Values	Read by
`tasks_evidence_completed_checklist`	allow / refuse	`ChecklistsService::assertTaskChecklistChangeAllowed()` - tick, result, attach, remove
`tasks_evidence_delete_occurrence`	allow / refuse	`TasksService::deleteTask()` - `occurrence_delete_refused`
`tasks_evidence_skip_status_id`	a closed status id, or empty	`TasksService::skipStatusId()` - empty = last closed status by `display_order`
`tasks_evidence_report_access`	anyone / permission	`Cap::TASKS_EVIDENCE_REPORT`'s `restrict` - `analystMayAct()`

Read only - analystModuleLevel($conn, $id, 'tasks') === 'read': TasksService::assertCanWrite() throws read_only on every write method; every api/tasks/* writer calls requireModuleWriteJson('tasks') (tests/module-access-coverage.php fails if one does not); the pages get body.module-read-only and hide their writes; drags do not start.

7. Traps

TRAP: tests must delete the history they write. No foreign keys means deleting a test's tasks leaves its task_events and rule versions behind - which then show up in Tasks → Evidence as a real series. Delete events and versions by task id and by recurrence id, before the tasks. (Found when "ZZREC one and done" appeared in the list after a test run.)
TRAP: scheduled_date is set once. Only createOccurrence() and the first save of a repeat (from the task's due date) write it. Never let an edit path touch it - it is what keeps a moved deadline visible.
TRAP: compare the completion as a local DAY. Comparing the UTC datetime with a due date calls a task done at 00:30 BST on the 2nd "on time for the 1st".
TRAP: an occurrence on a fixed schedule is matched on the date it was made FOR. occurrenceExistsFor() reads scheduled_date (falling back to due_date for old rows) and removed records - so moving an occurrence's due date, or deleting it, can never make the job create it again.
TRAP: tasks.created_by_id is NOT NULL. An occurrence the repeat makes still carries the first task's creator there; created_via and the created event are what say "the repeat". Display code must check created_via.
TRAP: skipping has no checklist gate, deliberately - not doing the work is the point.

8. Tests

php tests/task-evidence-atomic.php [base-url] - failure injection (Dr4inder asked for it): fails each evidence write - completing, a due date change, a /move, a successor expectation, a skip, and deleting an occurrence (a failed snapshot read, a failed retained-record write, a failed removed event) - and proves the task is left exactly as it was, each with a control that succeeds with its record. In-process it uses TaskHistory::$failNextWrite, a test-only hook. With a base URL it creates a temporary database trigger that refuses task_events rows for two ZZ tasks, and drives the board's drag and a subtask tick through the web server - so the failure there is a real database refusal - then drops it.
php tests/task-evidence.php [base-url] - builds a ZZ fixed-schedule series with one occurrence of each outcome (Dr4inder's 1st → 8th → 7th example among them), a due date moved after completion, a removed occurrence with a failed step, a missing month; asserts the report creates nothing; the delete setting; an after-completion series: one successor, no second on reopen, a missing successor named. With a base URL, drives the board's drag over HTTP. Tidies everything, history included.
php tests/task-read-only.php [base-url] - Read only for Tasks, with positive controls, in the service and over HTTP.
Also run: task-recurrence-spawn, task-recurrence-company-scope, task-recurrence-dates, calendar-sync-tasks, module-access-coverage, document-permissions.