Testing and QA - Fighter90/career-ops-ui GitHub Wiki
| Layer | Count | Command | What it covers |
|---|---|---|---|
| Unit / integration | 3210 |
npm test (node --test tests/*.test.mjs tests/acceptance/*.test.mjs) |
In-process createApp() hit with fetch on an ephemeral port; registry invariants; sanitizers; provider parity suites (stubbed transport, no network) |
| Playwright browser | 101 | npm run test:e2e:browser |
Narrow-viewport (320px overflow, v1.227.5), smoke, full-cycle, forms, locale sweep, theme toggle, persistent widgets (docs FAB + usage HUD) |
| Smoke E2E | 21 |
npm run test:e2e (tests/e2e.mjs) |
Real server in a child process, every route walked |
| Comprehensive E2E | 23 |
npm run test:e2e:full (tests/e2e-comprehensive.mjs) |
Long full-surface pass โ catches SPA regressions unit tests can't |
-
Coverage floor: 80 % on non-trivial logic; the actual baseline is ~93 % line / ~83 % branch (
npm run test:coverage). -
Never hardcode port 4317 in tests โ always
server.listen(0). - TDD when adding behavior (red โ green โ refactor); skipped only for pure refactors already under full coverage.
-
No mocks of internal collaborators โ fake the parent by pointing
CAREER_OPS_ROOTat amktemp -dwith the minimal files the path under test needs.
npm run test:ci = npm test + three deterministic gates:
-
scripts/check-no-also-leftovers.mjsโ doc-hygiene check. -
scripts/check-changelog-parity.mjsโ all 16 translated CHANGELOGs at the current version. -
tools/i18n-audit.mjsโ locale dictionary integrity.
Plus in the workflow matrix: Node 18/20/22 runs, CodeQL, and the Playwright suites. Key i18n/docs parity tests: tests/i18n-locale-files.test.mjs, tests/i18n-coverage.test.mjs (snapshot at tests/fixtures/i18n-dict.snapshot.json), tests/canonical-docs-coverage.test.mjs, tests/help-ru-config-section.test.mjs, tests/help-ui.test.mjs, tests/manifesto-link.test.mjs (help bundle 32 H2 / 122 H3 structure gates), and โ since v1.227.5 โ tests/help-source-counts.test.mjs + tests/help-banner-strip.test.mjs, which guard the help bundles' CONTENT rather than their headings: the ยง17 source total and EN/RU breakdown are asserted against the live registry (they had silently contradicted each other for several releases), the paragraph may pin no version, and the GitHub-only provider banner must never survive the help ingress into the page or the docs-assistant corpus.
The hard gate is ci.yml, not the pre-commit hook. The pre-commit AI review is advisory; a green pre-commit with a red CI is possible โ watch the CI run.
Tests cannot assume the parent career-ops project exists. The two rules that cost releases when violated:
-
PATHSresolves once per process.server/lib/paths.mjscomputesPROJECT_ROOTat import time. A test that setsCAREER_OPS_ROOTinbefore()must load every paths.mjs carrier (server/index.mjs,prompts.mjs,store.mjs,en-scanner.mjs,ru-scanner.mjs,paths.mjs) via dynamicimport()insidebefore(). A top-level static import runs before the env is set, pins the REAL parent, and leaks writes (e.g.PUT /api/profile) into the user's real files. Guards:tests/paths-once.test.mjs,tests/test-root-isolation.test.mjs. -
Bootstrap the minimal layout.
CAREER_OPS_ROOT=$(mktemp -d)+ write only what the test needs (cv.md,portals.yml, โฆ). Fixtures live undertests/fixtures/โ never real user data. - New FS-write helpers take an explicit path param โ otherwise the once-per-process resolution leaks writes into the real parent.
Two dedicated reviewer subagents enforce conventions: web-ui-route-reviewer (routes/security envelope) and test-isolation-reviewer (CI isolation, no live network, no port collisions).
Every release ships a QA regression prompt โ qa/QA-REGRESSION-PROMPT-v<version>.md โ a delta driver covering only what that release added. Exactly one lives at the top of qa/: archiving the previous one into qa/archive/superseded-prompts/ is part of shipping a release, so "which prompt is current" never needs thought. That rule was written at v1.137.0 and then not applied for 95 releases โ by v1.231.1 the top level held 98 stale prompts, archived in v1.231.2.
Three perennial prompts sit beside it and read package.json::version rather than pinning one: FUNCTIONALITY-CHECK.md (does it actually work), UX-AUDIT-PROMPT.md (is it good UX) and DESIGNER-EXPORT-PROMPT.md (design-system + key-flow export). v1.231.2 retired five more drivers that called themselves current and were not โ REGRESSION-FINAL.md chief among them, whose ยงยง11โ15 are five closed cycle ledgers (v1.55.x โ v1.59.7) and whose stated help baseline was 28 H2 / 103 H3 across 16 locales against 32 / 122 and 17 today.
Structure of a delta driver (e.g. v1.118.0):
- A header pinning the version under test,
parentVersion, route-module count, adapter count, and the unit-test baseline. - One ยง per shipped feature with concrete, executable checks: exact endpoints to hit, exact test files to run, exact UI states to verify (e.g. "
GET /api/scan/sourcesโ the EN list contains all 9 new values", "POST /api/tracker {status:'Hired'}โ row lands; unknown status degrades toEvaluated"). - A docs & i18n fan-out ยง (parity scripts + locale-file tests + help H2/H3 counts).
- A sign-off ยง:
npm testgreen (โฅ baseline),npm run test:cigreen, Playwright green, both E2E suites green, CI matrix (Node 18/20/22 + CodeQL) green โ with the known CodeQL false-positive dismissal rationale pre-stated.
Operational rules learned the hard way (see also Troubleshooting & FAQ):
-
Never
npm test 2>&1 | grep โฆโ grep masks the exit code; two releases shipped failing tests this way. Run the suite, capture$?, grep separately. - GET-only when smoke-testing a live deployed server โ no write-API calls against a real parent.
- Baselines only ratchet up: the next ship must keep all four suite counts โฅ the previous floor.