Catalogue Request Approvals Developer Guide - edmozley/freeitsm GitHub Wiki

Catalogue Request Approvals — Developer Guide

How catalogue-request approvals work, and why the shape is the way it is. Shipped as #928 (the gate + approver inbox + auto-raise) and #929 (the requester's dashboard view). Approval by a team, with a quorum came in 4.3.0 (GH #220) - see section 8.

The user-facing page is Catalogue Request Approvals.

1. 📁 The files involved

Colour key: 🗄️ schema · ⚙️ engine · 🔌 API · 🖥️ UI · 🎨 CSS · 🌍 i18n · 📄 docs

🎨	File	What it does
🗄️	`database/freeitsm.sql`	`forms.requires_approval` / `approver_id`; `form_submissions.approval_status` / `approver_id` / `approval_decided_by_id` / `approval_decided_datetime` / `approval_comment`; the four FKs to `analysts`
🗄️	`includes/db_verify_schema.php`	the same columns (columns only — FKs live in `freeitsm.sql`)
⚙️	`includes/catalogue_approvals.php`	the engine. `catalogueApprovalDecide()`, `catalogueCreateTicketFromSubmission()`, `catalogueApprovalsList()`, `catalogueApprovalGate()`, `catalogueSubmissionBodyHtml()`
⚙️	`includes/services/forms.php`	`FormsService::submitForm()` gates the submission; `saveForm()` stores the per-item settings
🔌	`api/forms/catalogue_approvals.php`	the inbox list (`?filter=mine\|all\|decided`)
🔌	`api/forms/decide_catalogue_approval.php`	approve / reject one request (a vote, in team mode); `override: true` for an administrator's override (#220)
🔌	`api/forms/move_catalogue_approval.php`	an administrator moves a waiting request to another team or person, with a reason (#220)
🔌	`api/forms/approval_teams.php`	active teams with member counts, for the approval window and Move (#220)
🗄️	`form_submission_votes`, `form_submission_approval_events`	one row per team vote; one row per override or move (#220)
✅	`tests/catalogue-group-approvals.php`	the team engine end to end, including two votes landing at once (#220)
🔌	`api/forms/save_form.php`	unchanged — passes `requires_approval` / `approver_id` straight through to the service
🔌	`api/forms/get_forms.php`	returns the approval columns + `approver_name` for the list
🔌	`api/self-service/get_dashboard.php`	returns the requester's `requests[]` with their state (#929)
🖥️	`forms/index.php`	the 🛡️ shield button, the Approval settings modal, the amber pill
🖥️	`forms/approvals.php`	the approver inbox (cloned from `change-management/approvals.php`)
🖥️	`forms/includes/header.php`	the Approvals nav item
🖥️	`self-service/index.php`	the Your requests dashboard section (#929)
🌍	`lang/en/forms.php` + `lang/pt-BR/forms.php`	the `approval` block, same commit
🌍	`lang/en/self-service.php` + `lang/pt-BR/self-service.php`	the dashboard `req_*` keys (#929)
📄	`CHANGELOG.local.md`, this wiki	#928 / #929

There is no new table: a catalogue item is a forms row and a request is a form_submissions row, so the approval state hangs off columns on those, mirroring how is_portal_visible was added.

2. 🔑 Why a designated analyst, and why it clones (not shares) the Change model

The classic version routes a request to the requester's line manager. Portal users (users) have no manager relationship — only tenant_id (their company) — so v1 reuses Change Management's single-approver idea instead: one analyst signs a catalogue item off.

The Change approval code is a good template but nothing is shareable: it's welded to the changes table and to analysts, with status-name string literals. So this is a parallel, smaller implementation of the approver + inbox pattern from Change Management.

3. ⚙️ Gating a submission

FormsService::submitForm() decides at submit time whether to gate, through one function for both ways of deciding:

$gate = catalogueApprovalGate($form, $portalUserId);
$approvalStatus = $gate !== null ? 'pending' : 'not_required';

catalogueApprovalGate() returns the policy to snapshot (approver_id, approval_team_id, approvals_required, rejections_required, approval_exclude_requester) or null. Three deliberate rules live in it:

Portal only ($portalUserId !== null). The feature auto-raises a ticket for the requester; an analyst filling a form internally has none, so it's never gated.
Unconfigured ≠ gated. requires_approval on with neither a team nor an approver_id falls through to not_required — a form must never strand a request with nobody able to clear it.
The policy is snapshotted onto the form_submissions row here, not read from the form at decision time. Editing the catalogue item later re-routes future requests, never ones already waiting - and never lowers the bar for one.

🔑 A gated submission fires catalogue_request.submitted, NOT form.submitted. An admin's create-ticket workflow rule on form.submitted would otherwise raise the ticket immediately and jump the gate. The outcome events are catalogue_request.approved / .rejected. All three are best-effort (guarded by class_exists('WorkflowEngine') and try/catch) — notification is a bonus, never the mechanism.

4. ⚙️ Deciding, and the auto-raise

catalogueApprovalDecide($conn, $actorId, $submissionId, $decision, $comment):

Loads the submission; refuses if it isn't pending.
Refuses unless the actor is approver_id — or sessionIsAdmin(). (Team mode is different: section 8.)
On approve, inside one transaction: raise the ticket, then stamp approval_status='approved', approval_decided_by_id, timestamp, comment and ticket_id. On reject: the same minus the ticket.

catalogueCreateTicketFromSubmission() deliberately mirrors api/self-service/create_ticket.php — the correct portal path:

requester resolved by users.id (never an email string — that's the bug in WorkflowEngine::action_create_ticket());
company from the requester's own tenant_id, falling to NULL/Default exactly as the portal's new-ticket path does;
status Open, the install's default priority;
body built by catalogueSubmissionBodyHtml() — the answers as a fully-escaped table, so a customer's field values can't inject markup into the analyst's reading pane.

🔑 This is also where promote-to-ticket finally landed. form_submissions.ticket_id had been a reserved "not yet actioned" column that nothing wrote. The approve path is its first writer.

🧩 A form's own action lists (#95)

forms.submission_actions is one TEXT column holding JSON:

{ "submitted": [ {"type":"create_ticket","args":{…}} ], "approved": […], "rejected": […] }

Each list is {type, args} in the workflow engine's own action shape, because they are workflow actions — WorkflowEngine::runActionList() runs them through the same executeActionList() loop a stored workflow uses, so chaining ({{last.ticket_id}}), error isolation and the step log all behave identically. JSON-in-TEXT for the same reason workflows.actions is: no migration each time a new action kind lands.

Runs are written to workflow_executions with workflow_id NULL and the form's name in workflow_name (the column is nullable and carries a name snapshot exactly so a run can have no parent row). Without that, a form's automation would be the one kind nobody could audit — and "it didn't do anything and I can't see why" is the worst failure mode automation has.

Executed from FormsService::submitForm() and catalogueApprovalDecide(), never from an adapter, so the UI, the portal and REST API v1 behave identically without any of them knowing the feature exists.

🔑 NULL is not the same as `[]` — and that is what removes the migration

Stored	Means	Behaviour
`NULL`	never configured	pre-#95 behaviour: nothing on submit; an approval raises a ticket the hard-coded way
`[]`	somebody opened the panel and chose nothing	obeyed — an approval raises no ticket
a list	configured	the list runs instead of the hard-coded raise

This is the whole safety argument. Every form that existed before the upgrade has NULL and keeps doing exactly what it did, so no data migration seeds anything and nothing can be half-migrated. The alternative — seeding every gated form with a create_ticket action — had two failure modes (a form missed by the seed silently stops raising tickets; a form seeded and still running the hard-coded path raises two) and this design has neither.

⚠️ A configured approved list raises its ticket AFTER the commit, not inside the decision transaction like the hard-coded path. An action list can send email, and a rolled-back email is still an email somebody received. The submission's ticket_id is stamped afterwards from the create_ticket step's result — that column is what the requester's Your requests panel reads, so a request showing no ticket after one was raised would look like a failure.

⚠️ A gated submission runs no submitted list, for the same reason it fires a different event: raising a ticket at submission time steps straight over the gate.

⚠️ submission_actions is written by createVersion(). Leaving it out would repeat the exact fault below — a per-form setting that Save deletes — except this time the form would quietly stop raising tickets.

⚠️ Saving validates every type against WorkflowEngine::availableActions(). An unknown type stored here would fail at run time inside a form submission, where the person who typed it will never see the error.

⚠️ Three faults this shipped with — fixed under discussion #95

All three were found while working out where form-submission actions should live. None had been reported: each one fails silently, which is what made them survive.

1. Editing a gated form removed its gate

FormsService::createVersion() copies the leaf into a new row, and its INSERT named title, description, is_active, is_portal_visible — not requires_approval or approver_id. requires_approval defaults to 0, so the new version came out ungated.

The catalogue lists leaves, so the ungated copy is the one the portal offers from that moment. Two behaviours changed at once and neither announced itself: sign-off stopped being required, and — because the approve path is the only thing that raises the ticket — requests stopped becoming tickets at all.

🔑 The general shape: a per-form setting that does not travel through createVersion() is a setting that Save deletes. The fix carries both columns; approver_id is passed as NULL rather than 0, because "nobody assigned" is a real state the gate reads as unconfigured.

⚠️ Not retrospectively repairable. A form edited before the fix lost the value at the moment of editing; there is nothing to recover it from. Any install that used approvals needs its forms checked by hand.

2. All three approval events fired into nothing

catalogueApprovalDecide() dispatches catalogue_request.approved / .rejected, and FormsService::submitForm() dispatches catalogue_request.submitted for a gated request. None of the three was in WorkflowEngine::availableTriggers().

dispatch() matches on trigger_event, and the editor's trigger dropdown is built from availableTriggers() — so no workflow could ever hold one of those values, and every dispatch matched zero rows. The code comments describe these as the hook to notify the approver and to tell the requester the outcome. Neither could happen; an approver had to think to go and look.

🔑 A dispatch() call is only half a trigger. The catalogue entry is the other half, and nothing links them — no error, no warning, no failing test. Adding a dispatch() without the availableTriggers() line produces an event that exists and can never be heard.

⚠️ The event is catalogue_request.**rejected**, not .declined. The name is built as 'catalogue_request.' . $decision, and $decision is validated against ['approved','rejected'] — the same word the column, the API contract and the UI button all use.

3. The decision events couldn't see the answers

The decide dispatch carried form and request (comment, ticket id, ticket number) but no submission. So {{submission.fields.Device type}} resolved to nothing on approve/reject while working fine on form.submitted — the same merge code meaning two different things depending on when it ran.

catalogueSubmissionAnswerMap() now supplies the label-keyed map plus the first email answer, and the four form-shaped triggers share one SUBMISSION_FIELD_TRIGGERS const so the merge-code hint and the editor's unknown-variable warning can't drift apart.

⚠️ catalogueSubmissionAnswerMap() and catalogueSubmissionAnswers() are not duplicates and must not be merged. The map is keyed by label because a merge code addresses an answer by name; the list is ordered because the approval card renders every row. A map cannot represent two fields sharing a label — the second would silently swallow the first — so the card keeps its list.

⚠️ Values in the map go through catalogueAnswerText(), so a checkbox reads Yes/No. form.submitted still flattens raw (1/0). Unifying them changes what already-built rules receive, so it is a deliberate decision rather than a tidy-up.

5. 🖥️ The three surfaces

Config (forms/index.php): the shield button opens a self-contained modal (#approvalModal). Active analysts are fetched once from api/tickets/get_analysts.php. Save posts only {id, requires_approval, approver_id} — a partial update, the same contract as the portal togglePortal, so nothing else about the form is touched.
Approver inbox (forms/approvals.php): cloned from change-management/approvals.php — filter sidebar (mine/all/decided) + card list, each card showing the answers and Approve/Reject with an optional note. showToast comes from the shared header, so the page doesn't load toast.js itself.
Requester view (self-service/index.php, #929): a Your requests section, hidden until the dashboard payload carries any. api/self-service/get_dashboard.php reads them in a guarded query — a pre-upgrade instance without the approval columns degrades to no requests rather than a broken dashboard.

6. ✅ How this was verified

php -l across every changed file.
Engine end-to-end against the dev DB: seed a pending request → catalogueApprovalsList shows it → catalogueApprovalDecide(approve) raises a ticket, stamps ticket_id, sets approved; requester (user_id), company and the answers-in-body all correct; re-approve and wrong-approver both refused with their specific messages.
Both analyst pages rendered with a forged analyst session (HTTP 200, no PHP errors), inline JS parsed clean in headless Chrome.
#929: get_dashboard.php returns the three states (pending / approved+ticket_number / rejected) via a forged portal session; the dashboard renders clean.

7. Extending it

Manager-based routing — the headline next step. Needs users.manager_id (or equivalent) and a way to populate it (LDAP/SSO/manual), then gate on the requester's manager instead of a fixed analyst. That relationship is its own mini-project.
Several stages ("line manager, then finance") - one request, one policy today. A chain would need an ordered list of policies per form and a stage number on the request; approval_round is a starting point, not that.
Show the rejection reason to the requester — the note is stored (approval_comment); surfacing it on the portal is a product decision (flip the approver's note from internal to requester-visible).
Attachments — forms have no file fields today; if they gain uploads, carry them onto the ticket in catalogueCreateTicketFromSubmission() (the pattern is in create_ticket.php).
A "raise ticket" button for NON-gated submissions — forms/submissions.php still has none; the approve path only covers gated ones.

8. Approval by a team (GH #220, 4.3.0)

The design, the questions and Ed's answers are on Catalogue Group Approvals - Design. This section is how the code does it.

Schema

Where	Columns	Notes
`forms`	`approval_team_id` (FK `teams`, SET NULL), `approvals_required` (default 1), `rejections_required` (default 1), `approval_exclude_requester` (default 0)	the policy as configured. `approver_id` is kept when a team is chosen, so switching back restores the person
`form_submissions`	the same four (nullable), plus `approval_round` (default 1)	the snapshot. `approval_team_id` set = team mode; NULL = person mode, exactly as before
`form_submission_votes`	`submission_id`, `approval_round`, `analyst_id`, `decision`, `comment`, `was_member`, `voted_datetime`	unique (submission_id, approval_round, analyst_id) - one vote per person per round, enforced by the database
`form_submission_approval_events`	`submission_id`, `analyst_id`, `action` (`moved` / `override_approved` / `override_rejected`), `detail`, `reason`, `event_datetime`	the administrator's interventions

🔑 Nothing existing changes on upgrade. Every new column defaults to "behave as before": no team, one approval, one rejection, requester not excluded. A form with a named approver produces a submission with approval_team_id NULL and goes down the original path. No data migration.

⚠️ Before Database Verification the columns do not exist. catalogueTeamApprovalAvailable() checks for them once per request, and every team read and write is behind it - the install then behaves exactly as 4.2.0.

The policy, from the snapshot

catalogueApprovalPolicy($sub) reads only the submission row: mode, team, the two numbers, exclude_requester (always true in team mode) and the round. Nothing at decision time reads the forms row.

Who may vote

catalogueEligibleMembers() - active members of an active team, less the requester - is asked at every vote, not snapshotted. Membership is the thing an organisation changes when somebody leaves, and a leaver should lose their vote on what is still waiting. Votes already cast keep counting; was_member records that the voter was eligible when they voted.

The requester is matched by email (catalogueAnalystIsRequester()): users and analysts are separate tables, so the only link between a portal requester and a member of staff is the address. Lower-cased and trimmed; an empty address never matches.

Deciding a vote - one lock, decided once

catalogueApprovalDecide() runs in a transaction that starts with catalogueLockSubmission() (SELECT ... FOR UPDATE). In team mode it:

refuses unless the request is still pending;
refuses unless the actor is in catalogueEligibleMembers();
refuses a rejection with no reason;
inserts the vote - a duplicate-key error becomes You have already decided this request;
recounts with catalogueApprovalProgress(); if approvals or rejections reached their number, finishes the decision inside the same transaction; otherwise returns decision: 'recorded' with the progress.

🔑 TRAP: the row lock is what stops two tickets. Two final votes landing together would otherwise each see "one short", each tip it, and each raise a ticket. With the lock the second waits, then finds the request decided. tests/catalogue-group-approvals.php fires two votes from two processes at once and checks for exactly one ticket.

Finishing is split in two, as the hard-coded path always was:

catalogueFinishDecision() - inside the transaction: raise the ticket (or nothing on reject) and stamp the submission.
catalogueAfterDecision() - after the commit: the workflow dispatch and the form's own When approved / When rejected list. Email sent from inside a transaction that then rolls back is still email somebody received.

Stuck

catalogueApprovalProgress() sets stuck when neither number can be reached any more: approvals + still-able-to-vote < approvals_required and rejections + still-able-to-vote < rejections_required. Also for person mode with no approver (a deleted team or analyst set it NULL) or an excluded requester. A stuck request is only ever flagged - never auto-approved, auto-rejected or re-routed. That is an administrator's call.

Administrators

Override (catalogueApprovalOverride(), override: true on the decide endpoint): administrator only, team mode only, reason required. It finishes the decision through the same two functions and writes an override_* event. On a person request an administrator's click already decides it, so there is nothing to override.
Move (catalogueApprovalMove(), api/forms/move_catalogue_approval.php): administrator only, reason required, to a team (with its own approvals_required) or a person. It rewrites the snapshot and increments approval_round - earlier votes stay in the table but are no longer counted, because the new approvers were not the ones asked. Writes a moved event with catalogueRoutingLabel() of before and after.

🔑 An administrator's ordinary Approve in team mode is not a vote. That was Ed's decision: an admin who happens to pass by should not quietly count as one of the "two different people". If the admin is a team member they vote in For me like anyone else; otherwise the card says the click overrides and needs a reason.

Getting in to the inbox

catalogueApproverCanEnter() and sessionIsFormApproverOnly() let a member of an approving team open Forms → Approvals without Forms access, the same as a named approver (catalogueAnalystApprovesAsTeam()). catalogueApprovalsList()'s mine adds team requests the analyst can still vote on; decided adds requests they have voted on. Each item carries mode, progress, can_vote, can_decide, can_override, can_move, events and approval_team_name, and the page shows buttons only where they would work.

Saving the policy

FormsService::saveApprovalPolicy() runs after saveForm()'s update and insert. It touches only the keys sent, checks the team exists and is active, and bounds both numbers to 1-20. copyApprovalPolicy() carries the four columns through createVersion() - the lesson of fault 1, not repeated.

Workflow events

catalogue_request.submitted carries approval_team in team mode, beside approver in person mode. .approved / .rejected fire once, when the request is decided - not on each vote.