Testing Strategies - bounswe/bounswe2024group11 GitHub Wiki

General Testing Plan

The general test plan for the project includes the following testing strategies:

  1. Unit Testing: Individual components will be tested in isolation as they are being developed to ensure they function as expected. Mock data will be used to simulate external dependencies where necessary.

  2. Integration Testing: The interaction between different modules and components will be tested to verify their proper integration and ensure data flows correctly across the system with mock data.This testing will be implemented as two features that interact with each other is developed.

  3. Mock Data: Mock data will be used during testing to simulate real-world inputs and edge cases, ensuring the application performs as expected under various scenarios.

  4. Regression Testing: After each change to the main branch, existing features will be re-tested to ensure new modifications don’t negatively affect the system.

  5. End-to-End Testing: Complete workflows will be tested from start to finish in the last weekend of the last Customer Demo to ensure the product functions as a whole in real-life usage scenarios.

  6. Performance Testing: The system's and external API's responsiveness and stability will be evaluated under load to ensure scalability and reliability.

This strategy aims to ensure comprehensive coverage and quality assurance at all levels of the development cycle.

Backend

@MucahitErdoganUnlu @ArdaVural @ozankrkya @ceydanursen @meminciftci

We will use Django's built-in testing framework, specifically APITestCase and APIClient, to ensure robust testing for the backend. These tools are well-suited for unit, integration, and end-to-end testing in the context of RESTful APIs and database interactions.

Unit Tests

Overview

Unit tests will focus on individual functions and methods in the backend logic, ensuring that they perform as expected in isolation. We will use Django's TestCase for unit testing backend functions and methods. This framework provides database rollbacks after each test, ensuring clean test execution.

Organization
  • Unit tests will be organized under ./backend/core/tests/unit/.
  • Focus will be on individual components such as utility functions, serializers, and model methods.
Acceptance Criteria
  • Unit tests are written for all new utility functions, serializers, and model methods.
  • All unit tests are located in ./backend/core/tests/unit/.

Integration Tests

Overview

Integration tests will validate interactions between different parts of the backend, such as models, serializers, and views. These tests will cover edge cases, error handling, and data transformations, ensuring the correctness of API endpoints and their responses.

Organization
  • Integration tests will be organized under ./backend/core/tests/integration/.
Acceptance Criteria
  • Integration tests are written for each key API endpoint (e.g., User Management, Quizzes, Forum Questions).
  • Integration tests validate serializer behavior and view logic comprehensively.
  • All integration tests are located in ./backend/core/tests/integration/.

End-to-End Tests

Overview

End-to-end tests will simulate real-world workflows, validating that APIs support complete user scenarios, such as registering a new user, taking a quiz, and posting to a forum.

Organization
  • End-to-end tests will be located in ./backend/core/tests/e2e/.
  • These tests will cover workflows that span multiple API endpoints.
Acceptance Criteria
  • At least one end-to-end test validates a complete user workflow for critical features (e.g., User Registration and Login, Quiz Submission, Forum Posting).
  • All end-to-end tests are located in ./backend/core/tests/e2e/.
Additional Notes
  • Testing will include validation of edge cases, error handling, and performance for critical endpoints.
  • Code coverage goal should be at least 80% coverage across all backend components.
  • CI/CD pipelines should execute all tests automatically to ensure regressions are caught early.

Example Unit and Integration Tests Results :

image

Web

@umitcan07 @hks1444

We will continue using Vitest as our primary framework for unit testing and will extend its use to integration testing as well. Additionally, we will incorporate Playwright for integration and end-to-end (E2E) testing to simulate real-world user interactions.

Unit tests will focus on runtime TypeScript functions that have no external dependencies. Integration tests will validate the behavior of features and components in conjunction with APIs and browser environments. End-to-end tests will ensure that full application flows, such as navigating through pages or completing quizzes, function as expected under simulated user conditions.

Unit Tests

We have selected Vitest because it integrates seamlessly with our existing Vite setup, offering excellent performance and an intuitive API. It’s particularly well-suited for testing isolated utility functions in a clean and efficient manner.

Current unit tests are located in ./client/src/tests/ but will be migrated to ./client/src/tests/unit for better organization.

Current Unit Tests

We will make sure to add new unit tests to cover all of the new pure functions. We will not formally assess the test coverage because of the scarce resources.

Acceptance Criteria:

  • Unit tests are written for all new utility functions introduced with features.
  • All unit tests are moved to ./client/src/tests/unit.

Integration & End-to-End Tests

We will leverage Playwright for integration and E2E testing. Its ability to emulate various browsers and environments makes it ideal for testing the interaction of components, API integrations, and user workflows. By simulating user behavior, we can validate the robustness of features such as quizzes, forum posts, and user profiles. We will focus on the component-level testing, ensuring that each feature behaves as expected under different scenarios. This includes testing for rendering, props, and state management.

Since we are no longer using MSW for API mocking, we will need to update our Playwright tests to use the fetch API directly. This will allow us to simulate API responses and ensure that our application functions as expected.

Integration and end-to-end tests are currently located under ./client/src/tests. In the future, integration tests will be moved to ./client/src/tests/integration.

Acceptance Criteria:

  • Each key route and feature (e.g., Profile, Quizzes, Quiz, Forum) has at least one integration test.
  • Integration tests are moved to ./client/src/tests/integration.
  • At least one end-to-end test validates a complete user workflow for critical application features.

Mobile

@Meminseeker @yunuskaydin

We will use Vitest for unit testing and the React Native Testing Library for integration testing to validate mobile application behavior. These tools ensure that the mobile app performs as expected, both in isolation and under simulated real-world user interactions.

Unit Tests

Overview

We have chosen Vitest for unit testing because it integrates well with our existing ecosystem and provides fast, efficient, and straightforward testing of standalone utility functions. Unit tests will focus on pure JavaScript/TypeScript functions that do not depend on the mobile environment.

Organization
  • Unit tests will be located in ./mobile/app/tests/unit/.
  • They will focus on isolated TypeScript logic that does not rely on React Native or device-specific functionality.
Acceptance Criteria
  • Unit tests are written for most of the new utility functions introduced with features.
  • All unit tests are moved to ./mobile/app/tests/unit/.

Integration Tests

Overview

The React Native Testing Library will be used for integration testing. This library is specifically designed for React Native apps and allows us to validate component rendering, state management, and interactions with the React Native environment.

Integration tests will focus on:

  • Component-level validation, ensuring forms handle state correctly.
  • Navigation functionality.
  • Components interacting with APIs or mock APIs properly.
Organization
  • Integration tests will be located in ./mobile/app/tests/integration/.
Acceptance Criteria
  • Integration tests are written for each key feature (e.g., Profile, Quizzes, Forum Questions).
  • Integration tests validate component rendering, state changes, and API interactions.
  • All integration tests are moved to ./mobile/app/tests/integration/.
Additional Notes
  • Device compatibility will be verified by running E2E tests on both Android and iOS emulators, as well as physical devices when possible.
  • We will monitor code coverage, it should be targeting at least 80% coverage across all tests.
  • CI/CD pipelines will try to execute all tests to ensure reliability and performance in diverse environments.

As stated in the frontend section; the unit tests are only effective in areas without dependencies on external systems and since most mobile functionality is tightly coupled with either the server or browser APIs, we focused on testing vanilla TypeScript utilities, namely a function combining the different types of hints fetched from the backend.

The tests can be found under ./mobile/app/tests

Here is a sample test result from the CLI using Vitest:

Screenshot 2024-11-27 at 21.25.31