End to End Tests - Enterprise-CMCS/macpro-mako GitHub Wiki
Our E2E Testing Philosophy
E2E tests are the final layer of our testing strategy. They provide the highest level of confidence that our application is working as expected for our users.
Our philosophy is to create tests that:
- Mimic Real User Behavior: Tests should follow a logical path that a real user would take through the application.
- Are Environment-Aware: Tests should be able to run against different environments, from a local development machine to the live production server.
- Are Stable and Reliable: E2E tests can be prone to flakiness. We strive to write tests that are resilient and provide consistent results.
Tools We Use
| Tool | Purpose |
|---|---|
| Playwright | Our framework for writing and running E2E tests. It automates a real web browser to perform user actions and verify outcomes. |
How to Run E2E Tests
Running a Full Test Suite
E2E tests are organized into "projects," where each project is configured for a specific environment. The most common project you will use for development is local.
To run all E2E tests for the local project, use the following command:
bunx playwright test --project=local
Running a Specific Test File
When working on a specific feature, you can save time by running only the test file related to that feature.
For example, to run only the tests in my-feature.spec.ts, you would use:
bunx playwright test tests/e2e/my-feature.spec.ts --project=local
Using Test Tags
We use tags to categorize our E2E tests. This allows us to run a subset of tests based on their functionality or purpose. Tags are added to test.describe blocks.
For example:
test.describe("Dashboard page", { tag: ["@dashboard"] }, () => {
// ... tests
});
How to Run Tests with Tags
You can run tests that have a specific tag using the --grep flag.
# Run all tests with the @dashboard tag
bunx playwright test --grep @dashboard
You can also exclude tests with a specific tag using the --grep-invert flag.
# Run all tests except those with the @dashboard tag
bunx playwright test --grep-invert @dashboard
Our Tagging Strategy
| Tag | Purpose |
|---|---|
@CI |
Tests that are run in our Continuous Integration pipeline. |
@dashboard |
Tests related to the user dashboard. |
@e2e |
General end-to-end tests. |
@faq |
Tests for the FAQ page. |
@home |
Tests for the home page. |
@profile |
Tests for the user profile page. |
@smoke |
A small set of critical tests that are run to ensure basic functionality is working. |
@smprod |
Smoke tests that are run against the production environment. |
@SPA |
Tests related to Single Page Applications. |
Understanding Our Test Projects
We have several E2E test projects, each designed for a specific purpose:
| Project Name | Purpose & Environment | When to Use |
|---|---|---|
local |
Runs tests against your local development server. | Use this for daily development and testing of new features. |
ci |
Runs tests in our Continuous Integration (CI) pipeline. | This is run automatically by our CI server on every code change. |
eua & mfa |
Runs tests against the production environment using specific user roles. | Use these with caution for manual verification on production. Requires special environment variables for authentication. |
smoke-tests |
Runs a small, critical set of tests against the production environment. | Used to quickly verify that the most important features are working after a deployment. |
A Note on Production Testing: The eua and mfa projects run on the live production environment. Always be cautious when running these tests. It is recommended to run only specific, non-destructive tests unless you are intentionally verifying a data-modifying feature.
E2E Test Structure
This document outlines the structure and organization of our End-to-End (E2E) tests.
Our E2E tests are located in the test/e2e directory and are organized by page or feature. This structure makes it easy to find the tests related to a specific part of the application.
For example:
-
dashboard/: Tests for the user dashboard. -
faq/: Tests for the FAQ page. -
forms/: Tests related to form submissions. -
home/: Tests for the home page. -
package-actions/: Tests for package actions. -
smoke/: Smoke tests for different environments.
When adding new tests, please follow this structure. If you are adding tests for a new feature, create a new directory for that feature.