Identifying What to Test - spryker-projects/cypress-boilerplate GitHub Wiki

End-to-end (E2E) testing with Cypress is a powerful way to ensure that your application functions correctly from the user's perspective. By simulating real user interactions and testing the entire stack of your application, E2E tests help identify issues that could impact your users. This guide will help you understand what aspects of your application should be covered by E2E Cypress tests, as well as what should not be covered.

What to Cover with E2E Cypress Tests

1. User Journeys

Mission-Critical Paths

The first and most important areas to cover with E2E tests are the "mission-critical" paths in your application. These are the core functionalities that must always work correctly because they are essential to your application's purpose. For example:

  • Login/Authentication: Ensure users can log in and access their accounts.
  • Purchasing/Checkout: Test the entire purchasing process, from adding items to the cart to completing the payment.
  • Sign-Up/Registration: Verify that new users can register successfully.

Testing User Journeys

A user journey is the sequence of steps a user takes to accomplish a goal in your application. For instance, in an e-commerce application, a user journey might include searching for a product, adding it to the cart, entering shipping information, and completing the purchase.

Why Test User Journeys?

Testing user journeys with a single E2E test ensures that all the components of your application—front-end, back-end, database, APIs—work together seamlessly. A single test covering the entire user journey helps:

  • Validate that all pieces of the application work in unison.
  • Identify issues that might arise from the integration of different layers in your tech stack.
  • Provide confidence that critical user flows are functioning as expected.
Example: E-commerce Checkout

An example E2E test for an e-commerce checkout process might include:

  1. Searching for a product.
  2. Adding the product to the shopping cart.
  3. Proceeding to checkout.
  4. Entering shipping and payment information.
  5. Completing the purchase.

By testing this entire flow in one test, you ensure that every step works correctly and that the overall process is smooth for users.

Covering API Endpoints

Сover used API endpoints with at least one positive and one negative user journey. When testing APIs, use response schema validation instead of direct assertions to ensure the structure and data types are correct.

Example: API User Journey for an E-commerce Shop

An example of an API user journey for an e-commerce shop might include:

  1. Get Access Token: Authenticate the user and obtain an access token.
  2. Search for Product: Use the token to search for a specific product.
  3. Add Product to Cart: Add the searched product to the user's cart.
  4. Checkout-Data: Retrieve all the available shipping and payment methods.
  5. Place an Order: Finalize the purchase and place the order.

By testing both positive and negative scenarios for these API endpoints, you can ensure that the APIs are functioning correctly and handling errors appropriately.

2. New Features

Testing New Features

When implementing a new feature, it’s crucial to start by understanding its end goal. What problem does this feature solve? What does it need to do?

Break Down the Feature

Break down the feature into smaller, testable steps. Each of these steps can be translated into a test that verifies a specific part of the feature's functionality. This approach allows you to:

  • Ensure that the feature works as intended from the start.
  • Refactor code with confidence, knowing that tests will catch any issues introduced during refactoring.
Example: Adding a Wishlist

For a new "wishlist" feature in an e-commerce site, your tests might include:

  1. Adding a product to the wishlist.
  2. Viewing the wishlist.
  3. Removing a product from the wishlist.
  4. Moving a product from the wishlist to the cart.

These tests validate that the feature works as expected, both in isolation and as part of the broader application.

3. Bugs

Writing Tests for Bugs

Whenever a bug is discovered, it's a best practice to write a test that reproduces the bug before fixing it. This test should initially fail, confirming that the bug exists. Once the bug is fixed, the test should pass.

Why Test Bugs?
  • Prevent Regressions: The test ensures that the bug does not reappear in the future.
  • Document the Fix: The test serves as documentation for the bug and its resolution.
  • Confidence in Stability: Passing tests after a fix provides confidence that the application remains stable.
Example: Fixing a Login Bug

Suppose users were unable to log in under certain conditions. You would:

  1. Write a test that simulates those conditions and attempts to log in.
  2. Confirm that the test fails, proving the bug exists.
  3. Fix the bug in the code.
  4. Run the test again to ensure it now passes.

This process not only fixes the current issue but also guards against similar issues in the future.

What Not to Cover with E2E Cypress Tests

While E2E tests are powerful, they are not the best tool for every type of testing. Here’s what you should avoid covering with E2E Cypress tests:

1. Detailed UI Testing

E2E tests should focus on user flows and critical paths, not the fine details of the user interface (UI). For example:

  • Visual Styles: Don’t use E2E tests to check specific colors, fonts, or layout details. These are better covered by visual regression tests or unit tests.
  • Minor UI Elements: Avoid testing the presence or styling of minor UI elements, such as icons or text alignment, which don’t affect the core user journey.

2. Extensive Data Validations

E2E tests should not be used for deep validation of data structures or database states. For example:

  • Complex Data Relationships: Avoid using E2E tests to validate complex data models or relationships. These are better suited for integration or unit tests.
  • Data Consistency Across Multiple Systems: Testing data consistency between multiple systems (e.g., microservices, third-party APIs) should be handled by integration tests rather than E2E tests.

3. Backend Logic

Avoid using E2E tests to validate backend logic in isolation. For example:

  • Business Rules: Use unit tests to validate business logic and rules rather than relying on E2E tests.
  • Performance Metrics: E2E tests are not ideal for measuring performance metrics such as response times or load testing. Specialized performance testing tools are more appropriate.

4. Non-Critical Flows

Avoid covering non-critical or edge-case flows with E2E tests, especially if they do not significantly impact the user experience. For example:

  • Rarely Used Features: Focus on the core user journeys and critical paths rather than rarely used features.
  • Internal Tools or Admin Panels: Unless these tools are mission-critical, they may not need E2E test coverage.

5. Highly Dynamic Content

Avoid testing content that changes frequently or is highly dynamic. For example:

  • Randomized Content: Testing elements with randomized or frequently changing content can lead to brittle tests that fail often.
  • Third-Party Integrations: Third-party content or integrations that are beyond your control may not be stable enough for reliable E2E testing.

6. Visiting External Sites

Avoid trying to visit or interact with sites or servers you do not control. Only test websites that you control. Visiting or requiring a third-party server can introduce variability and flakiness in your tests, making them less reliable.

Conclusion

E2E tests are critical for ensuring that your application works as intended from the user's perspective. Focus on covering mission-critical user journeys, new features, and bugs, while avoiding detailed UI testing, extensive data validations, backend logic, non-critical flows, highly dynamic content, and external sites. Additionally, cover API endpoints with at least one positive and one negative user journey, using response schema validation for robust and reliable testing. By targeting the right areas and avoiding common pitfalls, you can build a robust test suite that provides confidence in the stability and reliability of your application.

Happy testing!