Accepted

Cornerstone needs end-to-end (E2E) browser tests that validate complete user workflows against a fully running application. ADR-005 selected Playwright as the E2E framework but deferred the detailed architecture decisions. With the introduction of a dedicated e2e-test-engineer agent, we need to formalize:

1. Test environment setup: How does the E2E suite spin up the Cornerstone application and its dependencies (OIDC provider, reverse proxy)?
2. Multi-viewport testing: The application must be responsive across desktop, tablet, and mobile. How do we systematically validate responsive layouts?
3. Test data management: How is test data seeded and isolated between test runs?
4. Project structure: Where do E2E test files, page objects, and fixtures live?
5. CI integration: How do E2E tests run in the GitHub Actions pipeline, and how are failure artifacts collected?

• Responsive design: The requirements document specifies "desktop and tablet support, mobile-friendly" (Section 3.2). E2E tests must verify layouts at representative viewports.
• OIDC authentication: OIDC is the primary auth mechanism. E2E tests must exercise the OIDC flow against a real provider, not just mocked API calls.
• Reverse proxy deployment: Production deployments sit behind a reverse proxy (nginx, Traefik, Caddy). The trustProxy configuration (PR #70) must be tested end-to-end.
• Single container architecture: Cornerstone runs as a single Docker container with SQLite. The test environment must replicate this production topology.

A. Static Docker Compose for test environment

Define a docker-compose.test.yml that starts the app, OIDC provider, and proxy. Tests connect to the running stack.

• Pros: Simple to set up; familiar tooling; easy to run manually.
• Cons: Cannot vary configuration per test suite (e.g., OIDC vs. local auth); lifecycle management is external to the test runner; teardown is error-prone (orphaned containers).

B. Testcontainers for programmatic container management (chosen)

Use the testcontainers library to start/stop containers from within the test setup code.

• Pros: Full control over container lifecycle; can start different service combinations per test suite; automatic cleanup via Ryuk sidecar; aligns with the test-as-code philosophy.
• Cons: Adds a dependency; requires Docker on the CI runner; slightly more complex setup code than Compose.

C. No containers — run Fastify in-process + mock OIDC

Start the Fastify server in the same Node.js process as Playwright. Mock the OIDC provider with nock or a fake.

• Pros: Fastest startup; no Docker dependency for E2E tests.
• Cons: Does not test the production Docker image; does not validate Dockerfile, static asset serving, or production Node.js configuration. Mocking OIDC skips real token exchange and cookie handling. Does not test reverse proxy behavior at all.

We adopt testcontainers (option B) for E2E test environment management, with Playwright projects for multi-viewport coverage. The E2E test suite lives in a top-level e2e/ directory, separate from the unit/integration tests that are co-located with source code.

All containers are managed programmatically via the testcontainers Node.js library. Containers start before the first test and stop after the last test in a suite run.

Managed services:

Network topology:

Playwright browser
       |
       | HTTP
       v
  [upstream-proxy]  (nginx)
       |
       | HTTP (proxied)
       v
  [cornerstone-app]  (Fastify + SQLite + React SPA)
       |
       | HTTP (OIDC discovery, token exchange)
       v
  [oidc-provider]  (mock OIDC server)

All three containers share a Docker network created by testcontainers. Playwright connects to the upstream proxy's published port, which forwards to the Cornerstone app.

Container lifecycle:

• Global setup (globalSetup in Playwright config): Builds the Docker image (docker build -t cornerstone:e2e .), starts all three containers, waits for health checks to pass, and writes the base URL and OIDC configuration to a shared state file.
• Global teardown (globalTeardown in Playwright config): Stops and removes all containers.
• Per-test isolation: Each test gets a fresh database state via the seeding strategy described below.

Playwright's projects array in e2e/playwright.config.ts defines one setup project and three viewport projects:

Only the desktop project runs the whole suite. The tablet and mobile projects filter with grep: /@responsive/, so a test is exercised on all three viewports only when its title carries the @responsive tag. All three viewport projects reuse the admin storageState written by auth-setup. The tablet and mobile projects get doubled test, expect and action timeouts (see Decision 5) because WebKit is slower than Chromium.

Browser selection: Chromium for desktop, WebKit (through the iPad and iPhone device descriptors) for tablet and mobile. Firefox is not tested; it can be added as a separate project if browser-specific bugs emerge, but is not required by the project's requirements (which specify "Modern browsers — Chrome, Firefox, Safari, Edge" as a compatibility target, not a testing mandate).

cornerstone/
  e2e/
    package.json              # Workspace package for E2E deps (playwright, testcontainers)
    tsconfig.json             # TypeScript config (extends base, targets e2e/ only)
    playwright.config.ts      # Playwright configuration (projects, timeouts, reporters)
    containers/
      setup.ts                # Testcontainers global setup (start all services)
      teardown.ts             # Testcontainers global teardown (stop all services)
      cornerstoneContainer.ts # Cornerstone app container definition
      oidcContainer.ts        # Mock OIDC provider container definition
      proxyContainer.ts       # Upstream proxy container definition
    fixtures/
      auth.ts                 # Authentication fixture (login helper, session setup)
      seed.ts                 # Database seeding utilities
      testData.ts             # Shared test data constants
    pages/
      loginPage.ts            # Page Object Model for login page
      setupPage.ts            # Page Object Model for setup page
      ...                     # One POM per page, added as features are implemented
    tests/
      auth/                   # Auth flow tests (login, logout, OIDC, setup)
      ...                     # Organized by feature/epic, added as stories are completed

The e2e/ directory is an npm workspace (@cornerstone/e2e) with its own package.json and tsconfig.json. It depends on @cornerstone/shared for type definitions but has no dependency on @cornerstone/server or @cornerstone/client — it tests the built Docker image as a black box.

File naming conventions:

• Test files: *.spec.ts (Playwright convention, distinct from Jest's .test.ts)
• Page Object Models: PascalCasePage.ts (e.g., LoginPage.ts, SetupPage.ts)
• Fixtures and helpers: camelCase.ts
• Container modules: camelCase.ts

Recommended approach: API-based seeding via Playwright fixtures.

Each test (or test group) seeds its data by calling the Cornerstone API through the running application, using Playwright's request context (HTTP client). This approach:

• Validates the API: Seeding through the API exercises the same code paths as real user interactions, catching bugs in the creation flow.
• Is maintainable: Seed functions use the same API contract documented on the wiki, so they stay in sync with the application.
• Supports isolation: Each test creates its own data with unique identifiers, avoiding conflicts between parallel tests.

SQL fixtures are used only for initial admin setup, where API-based seeding is not possible (the setup endpoint only works once). A SQL seed file pre-populates the admin user and a basic dataset in the SQLite database mounted into the container.

Test data lifecycle:

1. Before suite: Containers start with a clean SQLite database. A SQL seed file creates the initial admin user.
2. Before each test: A Playwright fixture authenticates as the admin user (via API login) and seeds any test-specific data (via API calls).
3. After each test: No explicit cleanup is required if tests use unique identifiers. If a test modifies shared state, the fixture must restore it.
4. After suite: Containers are stopped and destroyed, including the ephemeral database.

Shared install-wide state needs a Playwright lock; restore and serial mode are not enough. Step 3's finally restore does not stop two concurrent tests from interleaving snapshot, mutate and restore. test.describe.configure({ mode: 'serial' }) only orders the tests of one project's job: under fullyParallel: true the desktop, tablet and mobile projects still run the same serial block at the same time. A test that mutates install-wide state, or asserts a value that another test mutates, must name a lock in its test.describe details ({ lock: '<name>' }, Playwright 1.63+). Tests that share a lock name never run concurrently across files, workers or projects. Keep serial where the block's internal ordering matters, and keep the finally restore.

• Prefer isolation to a lock. A test that can use a dedicated user or entity (as i18n.spec.ts and change-password.spec.ts do) needs no lock.
• Locks are scheduler state of one Playwright run, so they do not span CI shards. Each shard starts its own container stack and database in globalSetup, so no cross-shard guard is needed.
• Lock names in use: household-settings (the /api/settings singleton, navigation/settings-manage.spec.ts) and admin-account (TEST_ADMIN.displayName, profile/update-display-name.spec.ts, admin/edit-user.spec.ts, admin/user-list.spec.ts). Reuse an existing name for the same state rather than coining a second one.

Why not pure SQL fixtures? SQL fixtures bypass the application layer. They do not validate that the API can create the data correctly, they couple tests to the database schema (requiring updates when migrations change), and they cannot exercise business logic (e.g., computed fields, validations). SQL is only used where the API cannot fulfill the need (initial setup).

E2E tests run as a separate job in the GitHub Actions CI pipeline (ci.yml), after the existing quality-gates and docker jobs succeed. This ensures:

• The Docker image is buildable before attempting E2E tests.
• Unit/integration tests pass before the slower E2E suite runs.
• E2E failures do not block fast feedback from linting, type-checking, and unit tests.

Job configuration:

e2e:
  name: E2E Tests
  runs-on: ubuntu-latest
  needs: [quality-gates, docker]

  steps:
    - name: Checkout
      uses: actions/checkout@v6

    - name: Setup Node.js
      uses: actions/setup-node@v6
      with:
        node-version-file: .nvmrc
        cache: npm

    - name: Install dependencies
      run: npm ci

    - name: Install Playwright browsers
      run: npx playwright install --with-deps chromium webkit # webkit for tablet/mobile
      working-directory: e2e

    - name: Build Docker image for E2E
      run: docker build -t cornerstone:e2e .

    - name: Run E2E tests
      run: npx playwright test
      working-directory: e2e

    - name: Upload test results
      if: always()
      uses: actions/upload-artifact@v4
      with:
        name: e2e-results
        path: |
          e2e/test-results/
          e2e/playwright-report/
        retention-days: 14

The block above is the original single-job sketch. The live .github/workflows/ci.yml on beta differs in shape, not in intent: an e2e-warmup job caches browsers (Chromium and WebKit) and the image, e2e-smoke runs the smoke subset and feeds the Quality Gates check, and the full suite runs as e2e across 16 shards (SHARD_INDEX/SHARD_TOTAL, blob reports merged by e2e-merge-reports) feeding the separate E2E Gates check. Neither E2E job depends on quality-gates; they need detect-changes, docker and e2e-warmup. Treat ci.yml as authoritative for job wiring.

Artifact collection:

Parallel execution: Playwright's built-in fullyParallel: true setting runs test files across multiple workers. Locally the number of workers defaults to half the available CPU cores; on CI it is fixed at 2 per shard (workers: process.env.CI ? 2 : undefined). Every test runs once on desktop, and each @responsive-tagged test runs twice more (tablet and mobile), so parallelism and sharding are important for keeping CI times reasonable.

Timeout configuration:

Key settings for playwright.config.ts:

{
  testDir: './tests',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,          // Fail if .only() is left in CI
  retries: process.env.CI ? 1 : 0,       // 1 retry in CI for flake resilience, 0 locally
  workers: process.env.CI ? 2 : undefined,// Limit CI workers; auto-detect locally
  reporter: process.env.SHARD_INDEX      // Sharded CI: blob reports merged across shards
    ? [['blob', { outputDir: 'blob-report' }], ['list']]
    : [['html', { outputFolder: 'playwright-report' }], ['dot']],
  use: {
    baseURL: process.env.APP_BASE_URL || 'http://localhost:3000',
    trace: 'retain-on-failure',           // Capture trace only on failures
    screenshot: 'only-on-failure',        // Capture screenshot only on failures
    video: 'retain-on-failure',
    actionTimeout: 5000,
    navigationTimeout: 10000,
  },
  globalSetup: './containers/setup.ts',
  globalTeardown: './containers/teardown.ts',
  projects: [ /* auth-setup + desktop, tablet, mobile as defined in Decision 2 */ ],
  timeout: 15_000,
  maxFailures: process.env.E2E_FAIL_FAST ? 1 : undefined, // fail-fast on main-targeted PRs
  globalTimeout: process.env.CI ? 30 * 60 * 1000 : undefined,
}

The e2e-test-engineer consults this ADR for structural decisions and escalates to the product-architect only when deviating from the established design.

• Realistic testing: Tests run against the actual production Docker image, catching issues that in-process testing would miss (static asset serving, migration runner, container configuration, healthcheck).
• OIDC coverage: A real mock OIDC provider exercises the full authorization code flow — discovery, redirect, token exchange, cookie setting — without mocking internals.
• Responsive confidence: Running every test on desktop and every @responsive-tagged test on tablet and mobile catches layout regressions across form factors, and on two browser engines (Chromium and WebKit), without manual testing.
• Deterministic environments: Testcontainers ensures a clean environment per suite run. No "works on my machine" issues from stale containers or port conflicts.
• CI artifact debugging: Traces and screenshots on failure provide actionable debugging information without needing to reproduce locally.

• Slower CI: E2E tests with container startup add 2-5 minutes to the pipeline. The separate job avoids blocking fast feedback from unit tests.
• Docker dependency: Running E2E tests locally requires Docker. This is already a project prerequisite for builds.
• 3x multiplication for responsive tests: A test tagged @responsive runs three times (desktop, tablet, mobile); an untagged test runs once, on desktop only. Execution time scales with the number of tagged tests, so the tag is reserved for tests whose behaviour depends on layout. This is mitigated by Playwright's parallel execution and CI sharding. The flip side is that an untagged test gives no tablet/mobile or WebKit coverage.
• Container build time: The E2E job must build the Docker image before running tests. On CI, Docker layer caching reduces rebuild time for incremental changes.
• Test data coupling: API-based seeding depends on the API contract. Changes to the API may require updating seed functions. This is intentional — seed functions serve as additional validation of the API's correctness.