Production Go‐Live Checklist - RodrigoMattosoSilveira/EnterpriseRemoteSystems GitHub Wiki

This checklist is the production readiness gate for ERS audit and approval hardening.

It is intentionally operational. Each item should be checked in dev, then tst, then prd, using the normal promotion flow. Do not skip tst verification unless an emergency rollback/fix-forward decision is explicitly made.

Scope

Bite 19F validates that the Bite 19 hardening work is ready for production use:

  • Bite 19A — structured correction reasons for sensitive operations
  • Bite 19B — immutable audit hardening
  • Bite 19C — recent reauthentication requirement
  • Bite 19D / 19D.2 — optional and configurable second-person approval
  • Receipt-status and receipt-backfill enforcement already present in the current-account workflows

This checklist does not introduce application behavior by itself. It records the manual and automated checks required before production go-live.

Environments

Use the standard ERS environment order:

  1. Local development
  2. Development server
  3. Test server
  4. Production server

Expected public health endpoints:

curl -i https://dev.enterpriseremotesystems.com/api/v1/healthz
curl -i https://tst.enterpriseremotesystems.com/api/v1/healthz
curl -i https://app.enterpriseremotesystems.com/api/v1/healthz

Expected response: HTTP 200.

Required local checks before promotion

Run from the project root:

git status
git diff --check
go clean -testcache
go test ./...

If frontend files changed in the same promotion, also run the current frontend checks:

cd frontend
npm install
npm run lint
npm run test:e2e

For Bite 19F itself, the expected change is documentation only.

Migration verification

Confirm all Bite 19 migrations applied successfully in each promoted environment.

Expected Bite 19 migrations include, at minimum:

000022_add_structured_correction_reasons.up.sql
000023_enforce_immutable_authz_audit_logs.up.sql
000024_add_second_person_approval_metadata.up.sql
000025_configurable_second_person_approval.up.sql

For each environment, inspect the backend deployment logs and confirm:

  • migration runner started normally
  • each Bite 19 migration was detected or confirmed as already applied
  • no migration returned an error
  • backend started successfully after migrations
  • /api/v1/healthz returned HTTP 200

Database-hardening verification

Immutable audit logs

Audit logs must be append-only.

In a non-production environment first, verify that direct mutation attempts against authz_audit_logs are blocked:

UPDATE authz_audit_logs
SET decision = 'tampered'
WHERE id = '<existing-audit-log-id>';

Expected result: update rejected by database trigger.

Then verify delete is blocked:

DELETE FROM authz_audit_logs
WHERE id = '<existing-audit-log-id>';

Expected result: delete rejected by database trigger.

Finally confirm the original row remains unchanged.

Do not run destructive verification directly in production unless it is an explicitly approved operational test against a known safe audit row.

Structured correction reason columns

Confirm the current-account ledger schema includes structured reason columns used by Bite 19A:

correction_reason_code
correction_reason_text

Confirm journey settlement metadata includes the second-person approval and reason metadata columns introduced during Bite 19:

reason_code
reason_text
second_approved_by
second_approved_at
second_approval_notes

Column names should match the final schema in the repository migrations.

Authorization and actor checks

For smoke testing protected endpoints, use the same actor headers used by the current ERS transitional authorization model.

Example dev-only header setup:

BASE_URL="https://dev.enterpriseremotesystems.com"
ACTOR="[email protected]"
TENANT_ID="default"

ADMIN_HEADERS=(
  -H "Content-Type: application/json"
  -H "X-Authorized-By: ${ACTOR}"
  -H "X-Tenant-ID: ${TENANT_ID}"
  -H "X-Actor-Permissions: *"
)

Expected protected-endpoint behavior:

  • missing actor is rejected
  • unauthorized actor is rejected
  • authorized actor succeeds
  • denied attempts are audited when the operation reaches the authorization/audit path

Second-person approval policy checks

Read policy

Run in dev first:

curl -i \
  "${ADMIN_HEADERS[@]}" \
  "$BASE_URL/api/v1/current-accounts/settings/second-person-approval"

Expected response: HTTP 200.

Expected body shape:

{
  "data": {
    "tenantId": "default",
    "required": false
  }
}

The exact required value may differ if the environment was intentionally configured.

Enable policy in dev or tst only

curl -i \
  -X PUT \
  "${ADMIN_HEADERS[@]}" \
  -d '{"required":true}' \
  "$BASE_URL/api/v1/current-accounts/settings/second-person-approval"

Expected response: HTTP 200 with required: true.

Verify required policy blocks missing second approval

Using a safe dev or tst sensitive operation payload, include the required Bite 19A reason fields but omit secondApproval.

Expected result:

  • operation rejected
  • error indicates second-person approval is required
  • no ledger mutation is committed
  • denial is auditable if the request reaches the authorization/audit path

Verify valid second approval succeeds

Using a separate safe dev or tst payload, include a second approver different from the primary actor:

{
  "reasonCode": "GO_LIVE_SMOKE_TEST",
  "reasonText": "Go-live smoke test for second-person approval.",
  "secondApproval": {
    "approvedBy": "[email protected]",
    "notes": "Approved for controlled go-live smoke test."
  }
}

Expected result:

  • operation succeeds
  • ledger/correction metadata includes second approval where applicable
  • audit metadata includes second approval where applicable

Reset policy after smoke test

For dev or tst smoke tests, reset the setting to the intended environment default:

curl -i \
  -X PUT \
  "${ADMIN_HEADERS[@]}" \
  -d '{"required":false}' \
  "$BASE_URL/api/v1/current-accounts/settings/second-person-approval"

Do not change the production value without an explicit go-live decision.

Recent reauthentication checks

Bite 19C requires recent reauthentication for sensitive operations.

Fresh reauthentication headers:

REAUTH_AT="$(date -u +"%Y-%m-%dT%H:%M:%SZ")"

REAUTH_HEADERS=(
  -H "X-Reauthenticated-At: ${REAUTH_AT}"
  -H "X-Reauthentication-Method: password"
)

For a safe dev or tst sensitive operation, send the request with:

"${ADMIN_HEADERS[@]}" "${REAUTH_HEADERS[@]}"

Expected result when all other validations pass: operation is allowed.

Send the same safe operation shape without X-Reauthenticated-At.

Expected result:

{
  "error": {
    "code": "recent_reauthentication_required",
    "message": "Recent reauthentication is required for this sensitive operation"
  }
}