Test Plan - CankayaUniversity/ceng-407-408-2023-2024-Insightio GitHub Wiki
1. Introduction
1.1 Version Control
| Version No | Description of Changes | Date |
|---|---|---|
| 0.0.1 | Pre-Alpha | May 9, 2024 |
| 0.1.0 | Alpha | May 26, 2024 |
1.2 Overview
The Insightio platform provides crowd/object counting and tracking solutions for businesses.The use case of the system had been explained in the SRS document. The characteristics of the Insightio will be tested on the system in various settings.
1.3 Scope
This document contains the test design specifications, the test cases in accordance with the test plan, and the test plan for the use cases. As a result, the parts that follow will describe the test requirements and how the test cases will be implemented.
1.4 Terminology
| Acronym | Definition |
|---|---|
| SRS | Software Requirement Specification |
| SDD | Software Design Document |
| UI | User Interface |
| CS | Client-Server |
| AI | Artificial Intelligence |
| SP | Security and Performance |
| CUDA | Compute Unified Device Architecture |
2. Features To Be Tested
In this section, we will explain our test plan and provide general information of the test cases for the distinctive aspects of our project.
2.1 UI Interactions of the Application (UI)
Interface Specifications Testing entails assessing the Insightio platform's user interface (UI) and interaction components to make sure they are simple to use, intuitive, and compliant with requirements. The purpose of this testing is to confirm that all interface elements work as intended and offer a smooth user experience. Features such as account creation, camera setup, and access to statistical data are checked for accuracy and usability.
2.2 Client-Server Communication (CS)
Client-Server Communication Testing evaluates the effectiveness and dependability of data transfer between the client and server of the Insightio platform, with a special emphasis on camera setup and user authentication procedures. Through accurate transmission of requests and responses over the network, secure user authentication, and smooth camera configuration, these tests guarantee consistent and safe platform operations.
2.3 AI Driven Object Tracking (AI)
Using cutting-edge artificial intelligence algorithms, AI-Driven Object Tracking Testing assesses the platform's ability to precisely identify and track designated objects within predetermined zones. In order to ensure that the system provides high-performance object recognition and statistical reporting that meets user expectations, this testing confirms the accuracy of zone definition, the dependability of object counting and tracking under various conditions, and the overall accuracy of the data.
2.4 Security and Performance
The purpose of Security and Performance Testing is to evaluate how well the Insightio platform safeguards user data and performs under different load scenarios. This entails testing the system's response times to guarantee quick data processing, the strength of password hashing algorithms for safe authentication, and RAM optimization to keep performance levels high without sacrificing security or speed.
3. Features Not To Be Tested
In this section, we described features that will not be tested.
3.1 External System Integrations
Due to the unpredictability of third-party service availability and configurations, as well as the presumption that these services carry out their own thorough testing, testing integrations with external systems—such as third-party APIs or services—falls outside our immediate purview. Instead of concentrating on the external systems per se, we continue to pay attention to the essential features of the application and the interfaces that connect it to them.
3.2 Computers Without CUDA
For the purpose of carrying out the intricate parallel computations required for AI and machine learning tasks, our application critically depends on CUDA. As these systems essentially lack the hardware capability needed to run our application, testing on computers without CUDA hardware is prohibited and is therefore irrelevant to our target user base and hardware requirements.
3.3 Real-Time Multi-User Interaction
Although our platform facilitates user-to-user interactions, the complex scenarios that entail synchronous, real-time actions and data transfers between users are not covered in the preliminary testing stages. This choice was made with an emphasis on core functionality and stability under typical usage conditions due to the difficulty of accurately simulating real-world interaction patterns and the substantial resources required.
4. Item Pass/Fail Criteria
A test case must satisfy all success requirements without resulting in any failure conditions in order to pass. A review and possible rework of the tested feature or function is required if there are no pass criteria or if there are any failure criteria.
4.1 Exit Criteria
- 100% of the test cases are executed
- 75% of the test cases passed
- All High Priority test cases passed
5. References
- CENG408_Group6_SRS, Dec 08, 2023. Available: https://github.com/CankayaUniversity/ceng-407-408-2023-2024-Insightio/wiki/SRS
- CENG408_Group6_SDD, Jan 22, 2023. Available: https://github.com/CankayaUniversity/ceng-407-408-2023-2024-Insightio/wiki/SDD
6. Test Design Specifications
6.1 UI Interactions of the Application
6.1.1 Subfeatures to be tested
6.1.1.1 Account Edit (UI.AE)
Enables users to edit their account by entering personal information like a username, email address, and password; includes data integrity and security validations.
6.1.1.2 Camera Setup (UI.CS)
Enables users the ability to add, set up, and control camera devices on the platform. This includes creating monitoring zones and object tracking parameters.
6.1.1.3 Statistical Data (UI.SD)
Based on the object tracking data collected, it provides users with statistical reports and visual analytics that provide insights through customizable time frames, graphs, and charts.
6.1.2 Test Cases
| TC ID | Priority | Scenario Description |
|---|---|---|
| UI.AE.01 | M | Enter a valid user account picture, name, surname, email and password. |
| UI.CS.01 | H | Select camera then enter a camera name and type. Finally, to draw zones select the target |
| UI.SD.01 | M | Displaying statistics with charts based on the configured cameras and tracking settings. |
6.2 Client-Server Communication
6.2.1 Subfeatures To Be Tested
6.2.1.1 Camera Configuration (CS.CC)
Makes it easier for the client interface and the server to exchange camera settings and parameters, guaranteeing that cameras are set up correctly for object tracking in accordance with user requirements.
6.2.1.2 Authenticate Users (CS.AU)
Oversees the safe exchange of login credentials between the client and server, enabling platform access while guarding against illegal use and reset password with having "Reset Password" option.
6.2.1.3 Account Operations (CS.AO)
Enables users to add/edit their accounts to server by entering personal information like a username, email address, account picture and password; includes data integrity and security validations.
6.2.2 Test Cases
| TC ID | Priority | Scenario Description |
|---|---|---|
| CS.CC.01 | H | Add a camera which is not already in db. |
| CS.CC.02 | H | Add a camera which already exists. |
| CS.CC.03 | M | Edit a camera. |
| CS.AU.01 | H | Enter invalid credentials for login. |
| CS.AU.02 | H | Enter valid credentials for login. |
| CS.AU.03 | M | Reset password with "Forget Password" option. |
| CS.AO.01 | M | Add Account Picture. |
6.3 AI Driven Object Tracking (AI)
6.3.1 Subfeatures to be tested
6.3.1.1 Zone Definition (AI.ZD)
Improves the accuracy of data collected by enabling users to designate particular regions inside the camera's field of view for targeted object tracking and counting.
6.3.1.2 Object Counting and Tracking (AI.OCT)
Uses artificial intelligence (AI) algorithms to precisely identify, count, and track specific objects throughout the defined zones, enabling real-time analytics and monitoring.
6.3.1.3 Data Accuracy (AI.DA)
Guarantees the object counting and tracking process's dependability by continuously improving the algorithms used in data reporting and analysis to achieve high accuracy. Guarantees the object counting and tracking process's dependability by continuously improving the algorithms used in data reporting and analysis to achieve high accuracy.
6.3.1.4 Real-Time Object Recognition (AI.RTOR)
Utilizes real-time object recognition techniques to identify and classify objects accurately and efficiently.
6.3.2 Test Cases
| TC ID | Priority | Scenario Description |
|---|---|---|
| AI.ZD.01 | H | Verify that users can accurately define polygonal zones on camera images for targeted object tracking. |
| AI.OCT.01 | H | Ensure that the system accurately identifies, counts, and tracks specific objects throughout the defined zones using AI algorithms. |
| AI.OCT.02 | M | Verify that the system maintains accurate object tracking even in low-light conditions or with partially obscured objects. |
| AI.DA.01 | H | Validate that the object counting and tracking process consistently achieves high accuracy through continuous algorithm improvements. |
| AI.RTOR.0.1 | H | Verify that the system can accurately recognize and classify objects in real-time, maintaining high accuracy. |
6.4 Security and Performance
6.4.1 Subfeatures To Be Tested
6.4.1.1 Response Time (SP.RT)
Monitors the speed at which the system responds to queries and produces results, with the goal of minimizing latency to guarantee a seamless and effective user experience.
6.4.1.2 RAM Usage (SP.RAM)
Keeps track of and adjusts how much RAM the application uses while it is running to make sure system resources are used effectively without sacrificing performance.
6.4.1.3 Password Hashing (SP.PH)
Protects user credentials against illegal access and data breaches by using secure cryptography techniques to transform user passwords into hashed formats.
6.4.1.4 Access Control (SP.AC)
Implements access control mechanisms to restrict unauthorized access to sensitive features and data within the platform.
6.4.1.5 Audit Logging (SP.AL)
Logs all user actions and system events to provide a comprehensive audit trail for security monitoring and forensic analysis.
6.4.2 Test Cases
| TC ID | Priority | Scenario Description |
|---|---|---|
| SP.RT.01 | H | Test the system's response time to queries and ensure it meets performance benchmarks for seamless user experience. |
| SP.RT.02 | M | Verify that the application effectively manages RAM usage to optimize system resources without compromising performance. |
| SP.PH.01 | H | Ensure that user passwords are securely hashed to protect against unauthorized access and data breaches. |
| SP.AC.01 | H | Verify that access controls are enforced consistently across all system features, preventing unauthorized access to sensitive data. |
| SP.AL.01 | M | Validate that audit logs capture all user actions and system events accurately. |
7. Detailed Test Cases
7.1 UI.AE.01
| TC_ID | UI.AE.01 |
|---|---|
| Purpose | Edit User Account |
| Requirements | Login into application successfully. |
| Priority | Medium |
| Estimated Time Needed | 2 Minutes |
| Dependency | Login into application test cases should pass. |
| Setup | Application must be provided. |
| Procedure | - Go to Account page. |
| - Enter an account picture by choosing file from computer. | |
| - Enter a valid username, surname, email and password. | |
| - Click on the “Save” button. | |
| - Observe that the edit is successful and the account information has changed. | |
| Cleanup | Close the application |
7.2 UI.CS.01
| TC_ID | UI.CS.01 |
|---|---|
| Purpose | Make Camera Configuration |
| Requirements | Login into application successfully. |
| Priority | High |
| Estimated Time Needed | 3 Minutes |
| Dependency | Selecting targets must be done before drawing the zones. |
| Setup | Application and at least one camera must be provided. |
| Procedure | - Go to Camera Configuration page. |
| - Enter a camera name. | |
| - Choose Camera Type. | |
| - Select a Camera. | |
| - Select Targets. | |
| - Draw Zones. | |
| - Observe that the configuration is successful and the camera setup successfully made. | |
| Cleanup | Close the application |
7.3 UI.SD.01
| TC_ID | UI.SD.01 |
|---|---|
| Purpose | Display Statistics |
| Requirements | Login into application successfully. |
| Priority | Medium |
| Estimated Time Needed | 1 Minute |
| Dependency | Selecting targets and time frame must be done before displaying statistics. |
| Setup | Application and at least one camera view must be provided. |
| Procedure | - Go to the Dashboard page. |
| - Pick a camera view (if you have more than one camera that has been set). | |
| - Select target. | |
| - Select time frame to display specific data. | |
| - Select Chart type to display. | |
| - Observe that the display statistics are successful and accurate. | |
| Cleanup | Close the application |
7.4 CS.CC.01
| TC_ID | CS.CC.01 |
|---|---|
| Purpose | Add a new camera to the application |
| Requirements | Login into application successfully |
| Priority | High |
| Estimated Time Needed | 5 Minutes |
| Dependency | Camera name, type, and selected targets must be provided |
| Setup | Application and at least one camera must be provided |
| Procedure | - Go to the Camera Configuration page. |
| - Enter a camera name. | |
| - Select a camera type. | |
| - Select a camera. | |
| - Select Targets. | |
| - Draw Zones. | |
| - Observe that the settings for the new camera transmitted to the server successfully. | |
| Cleanup | Close the application |
7.5 CS.CC.02
| TC_ID | CS.CC.02 |
|---|---|
| Purpose | Try to add a new camera configuration with the same camera from the device to the application |
| Requirements | Login into application successfully |
| Priority | High |
| Estimated Time Needed | 5 Minutes |
| Dependency | Camera name, type, and selected targets must be provided |
| Setup | Application and at least one camera that is already saved in the system must be provided |
| Procedure | - Go to the Camera Configuration page. |
| - Enter a camera name. | |
| - Select a camera type. | |
| - Select a camera that is already selected and saved to the database. | |
| - Select Targets. | |
| - Draw Zones. | |
| - Observe that the application is not allowing for the new camera configuration with the same camera from the device. | |
| Cleanup | Close the application |
7.6 CS.CC.03
| TC_ID | CS.CC.03 |
|---|---|
| Purpose | Edit camera settings in application |
| Requirements | Login into application successfully |
| Priority | Medium |
| Estimated Time Needed | 3 Minutes |
| Dependency | Existing Camera Configuration must be provided |
| Setup | Application and at least one camera configuration must be provided |
| Procedure | - Go to the Camera Configuration page. |
| - Select Edit Option from the panel. | |
| - Select a camera configuration that already exists in the device. | |
| - Enter a camera name. | |
| - Select Targets. | |
| - Draw Zones. | |
| - Observe that the edited settings for the camera are transmitted to the server successfully. | |
| Cleanup | Close the application |
7.7 CS.AU.01
| TC_ID | CS.AU.01 |
|---|---|
| Purpose | Check if invalid credentials may be logged in to the application |
| Requirements | - |
| Priority | High |
| Estimated Time Needed | 2 Minutes |
| Dependency | - |
| Setup | Application must be provided |
| Procedure | - Go to the Login page. |
| - Enter invalid username. | |
| - Enter invalid password. | |
| - Click the login button. | |
| - Observe that logging in to the application with invalid credentials is not allowed by the server. | |
| Cleanup | Close the application |
7.8 CS.AU.02
| TC_ID | CS.AU.02 |
|---|---|
| Purpose | Logged in to the application with valid credentials |
| Requirements | User should be created |
| Priority | High |
| Estimated Time Needed | 2 Minutes |
| Dependency | User must exist in the database |
| Setup | Application must be provided |
| Procedure | - Go to the Login page. |
| - Enter valid username. | |
| - Enter valid password. | |
| - Click the login button. | |
| - Observe that logging in to the application with valid credentials is done. | |
| Cleanup | Close the application |
7.9 CS.AU.03
| TC_ID | CS.AU.03 |
|---|---|
| Purpose | Reset Password |
| Requirements | User should be created |
| Priority | Medium |
| Estimated Time Needed | 1 Minutes |
| Dependency | User must exist in the database |
| Setup | Application must be provided |
| Procedure | - Go to the Login page. |
| - Click to "Reset Password" button. | |
| - Observe that resetting of password is done. | |
| Cleanup | Close the application |
7.10 CS.AO.01
| TC_ID | CS.AO.01 |
|---|---|
| Purpose | Add Account Picture |
| Requirements | User should be created |
| Priority | Medium |
| Estimated Time Needed | 1 Minutes |
| Dependency | User must logged into system. |
| Setup | Application must be provided |
| Procedure | - Go to the Account page. |
| - Click to "Account Picture" button. | |
| - Observe that saving picture to database is successfull. | |
| Cleanup | Close the application |
7.11 AI.ZD.01
| TC_ID | AI.ZD.01 |
|---|---|
| Purpose | Verify that users can accurately define polygonal zones on camera images for targeted object tracking |
| Requirements | Login into application successfully |
| Priority | High |
| Estimated Time Needed | 10 minutes |
| Dependency | User must have access to camera configuration settings |
| Setup | User must be logged into the application with access to camera configuration settings |
| Procedure | - Navigate to the Camera Configuration page. |
| - Enter the camera name. | |
| - Select a camera type. | |
| - Draw polygonal zones on camera images to define tracking areas. | |
| - Observe that the system allows users to accurately define polygonal zones without encountering any errors. | |
| Cleanup | Close the application |
7.12 AI.OCT.01
| TC_ID | AI.OCT.01 |
|---|---|
| Purpose | Ensure that the system accurately identifies, counts, and tracks specific objects throughout the defined zones using AI algorithms |
| Requirements | Login into application successfully |
| Priority | High |
| Estimated Time Needed | 15 minutes |
| Dependency | User must have access to object tracking settings |
| Setup | User must be logged into the application with access to object tracking settings |
| Procedure | - Navigate to the Object Tracking page. |
| - Select the desired object classes for tracking. | |
| - Draw polygonal zones on camera images to define tracking areas. | |
| - Observe that the system accurately identifies and tracks specific objects within the defined zones without errors. | |
| Cleanup | Close the application |
7.13 AI.OCT.02
| TC_ID | AI.OCT.02 |
|---|---|
| Purpose | Verify that the system maintains accurate object tracking even in low-light conditions or with partially obscured objects |
| Requirements | Login into application successfully |
| Priority | Medium |
| Estimated Time Needed | 15 minutes |
| Dependency | User must have access to object tracking settings |
| Setup | User must be logged into the application with access to object tracking settings |
| Procedure | - Navigate to the Object Tracking page. |
| - Select the desired object classes for tracking. | |
| - Draw polygonal zones on camera images to define tracking areas. | |
| - Observe that the system accurately maintains object tracking even in low-light conditions or with partially obscured objects. | |
| Cleanup | Close the application |
7.14 AI.DA.01
| TC_ID | AI.DA.01 |
|---|---|
| Purpose | Validate that the object counting and tracking process consistently achieves high accuracy through continuous algorithm improvements |
| Requirements | Login into application successfully |
| Priority | High |
| Estimated Time Needed | 20 minutes |
| Dependency | User must have access to object tracking settings |
| Setup | User must be logged into the application with access to object tracking settings |
| Procedure | - Navigate to the Object Tracking page. |
| - Select the desired object classes for tracking. | |
| - Draw polygonal zones on camera images to define tracking areas. | |
| - Observe that the system consistently achieves high accuracy in object counting and tracking without errors. | |
| Cleanup | Close the application |
7.15 AI.RTOR.01
| TC_ID | AI.RTOR.01 |
|---|---|
| Purpose | Verify that the system can accurately recognize and classify objects in real-time, maintaining high accuracy. |
| Requirements | Login into application successfully. |
| Priority | High |
| Estimated Time Needed | 10 minutes |
| Dependency | User must have access to real-time object recognition settings. |
| Setup | User must be logged into the application with access to object tracking settings. |
| Procedure | |
| - Navigate to the Object Tracking page. | |
| - Ensure the camera feed is active and streaming. | |
| - Observe that the system accurately recognizes and classifies objects in real-time without errors. | |
| Cleanup | Close the application |
7.16 SP.RT.01
| TC_ID | SP.RT.01 |
|---|---|
| Purpose | Test the system's response time to queries and ensure it meets performance benchmarks for a seamless user experience |
| Requirements | Successful login into the application |
| Priority | High |
| Estimated Time Needed | 10 minutes |
| Dependency | User must have access to the system's query interface |
| Setup | User must be logged into the application with access to the query interface |
| Procedure | - Execute a query within the application. |
| - Observe the system's response time and ensure it meets performance benchmarks without delays. | |
| Cleanup | Close the application |
7.17 SP.RT.02
| TC_ID | SP.RT.02 |
|---|---|
| Purpose | Verify that the application effectively manages RAM usage to optimize system resources without compromising performance |
| Requirements | Successful login into the application |
| Priority | Medium |
| Estimated Time Needed | 15 minutes |
| Dependency | User must have access to system monitoring tools |
| Setup | User must be logged into the application with access to system monitoring tools |
| Procedure | - Monitor RAM usage while the application is running. |
| - Observe that the application effectively manages RAM usage without excessive consumption or performance degradation. | |
| Cleanup | Close the application |
7.18 SP.PH.01
| TC_ID | SP.PH.01 |
|---|---|
| Purpose | Ensure that user passwords are securely hashed to protect against unauthorized access and data breaches |
| Requirements | Successful login into the application |
| Priority | High |
| Estimated Time Needed | 5 minutes |
| Dependency | User must have access to the password management interface |
| Setup | User must be logged into the application with access to the password management interface |
| Procedure | - Change the user password through the application interface. |
| - Verify that the system securely hashes the password and stores it in the database. | |
| Cleanup | Close the application |
7.19 SP.AC.01
| TC_ID | SP.AC.01 |
|---|---|
| Purpose | Verify that access controls are enforced consistently across all system features, preventing unauthorized access to sensitive data |
| Requirements | Successful login into the application |
| Priority | High |
| Estimated Time Needed | 15 minutes |
| Dependency | User must have access to sensitive data within the application |
| Setup | User must be logged into the application with access to sensitive data |
| Procedure | - Attempt to access sensitive data without proper authorization. |
| - Verify that the system enforces access controls and prevents unauthorized access to sensitive data. | |
| Cleanup | Close the application |
7.20 SP.AL.01
| TC_ID | SP.AL.01 |
|---|---|
| Purpose | Validate that audit logs capture all user actions and system events accurately |
| Requirements | Successful login into the application |
| Priority | Medium |
| Estimated Time Needed | 15 minutes |
| Dependency | User must have access to audit logs |
| Setup | User must be logged into the application with access to audit logs |
| Procedure | - Perform various actions within the application, such as accessing data or modifying settings. |
| - Verify that all user actions and system events are accurately recorded in the audit logs. | |
| Cleanup | Close the application |
8. Test Results
| TC ID | Result | Priority | Explanation |
|---|---|---|---|
| UI.AE.01 | Pass | M | User inputs taken successfully via system. |
| UI.CS.01 | Pass | H | User can set up the camera successfully. |
| UI.SD.01 | Pass | M | Statistics based on the tracking data displayed successfully. |
| CS.CC.01 | Pass | H | New camera configuration transmitted server successfully. |
| CS.CC.02 | Pass | H | Trying for new Camera Configuration with the same camera from the device is not permitted. |
| CS.CC.03 | Pass | M | Edited camera settings transmitted server successfully. |
| CS.AU.01 | Pass | H | Logging in to the application with invalid credentials is not allowed. |
| CS.AU.02 | Pass | H | Logged in to the application with valid credentials successfully. |
| CS.AU.03 | Fail | M | Resetting password is unsuccessful. |
| CS.AO.01 | Fail | M | Adding picture to database is unsuccessful. |
| AI.ZD.01 | Pass | H | Users were able to accurately define polygonal zones on camera images. |
| AI.OCT.01 | Pass | H | The system accurately identified, counted, and tracked specific objects. |
| AI.OCT.02 | Pass | M | The system successfully maintained accurate object tracking even in challenging conditions. |
| AI.DA.01 | Pass | H | The object tracking process consistently achieved high accuracy. |
| AI.RTOR.01 | Pass | H | The system accurately recognized and classified objects in real-time, maintaining high accuracy levels during testing. |
| SP.RT.01 | Pass | H | The system responded promptly to queries, meeting performance benchmarks. |
| SP.RT.02 | Pass | M | The application effectively managed RAM usage. |
| SP.PH.01 | Pass | H | User passwords were securely hashed. |
| SP.AC.01 | Pass | H | Access controls were consistently enforced across all system features. |
| SP.AL.01 | Pass | H | Audit logs accurately captured all user actions and system events. |