Test Results - CankayaUniversity/ceng-407-408-2024-2025-ReSort GitHub Wiki

Test Results

1. INTRODUCTION

1.1 Version Control

Version No Description of Changes Date
1.0 First Version March 23, 2025

1.2 Overview

This test plan was prepared to evaluate the functionality and accuracy of the waste separation system developed within the scope of the Resort project. The project covers data collection, classification, recording and interface display processes.

1.3 Scope

This document outlines the test strategy for the software and hardware components of the Intelligent Recycling System including image classification, sensor-actuator interactions, waste sorting mechanisms, and performance under real-world conditions.

1.4 Terminology

Acronym Definition
YO YOLOv11 Detection
RPi Raspberry Pi Controller
ARD Arduino Sensors & Actuators
WM Waste Sorting Mechanism
UX User Experience
PERF Performance & Scalability
SEC Security & Privacy
ROI Return on Investment & Market Acceptance

2. FEATURES TO BE TESTED

2.1 YOLOv11 Waste Detection (YO)

  • Test the accuracy and consistency of YOLOv11 model to detect waste types.

2.2 Raspberry Pi Control System (RPi)

  • Test if the Raspberry Pi runs inference correctly and controls connected components.

2.3 Arduino Sensors & Actuators (ARD)

  • Test if Arduino collects sensor data (metal, proximity) and controls motors, servos.

2.4 Waste Sorting Mechanism (WM)

  • Test if the physical system correctly directs waste to the appropriate bin.

2.5 User Experience (UX)

  • Test the usability and user interaction for setting up and maintaining the system.

2.6 Performance & Scalability (PERF)

  • Test how the system behaves under high load and stressful conditions.

2.7 Security & Privacy (SEC)

  • Test the system's resistance against unauthorized access and data security.

2.8 ROI & Market Acceptance (ROI)

  • Evaluate the system's cost efficiency, market acceptance, and ROI.

3. FEATURES NOT TO BE TESTED

  • Cloud logging system.
  • External database integration.

4. ITEM PASS/FAIL CRITERIA

A test case passes if all expected outcomes are met without errors. A failure is recorded if the observed result deviates from the expected behavior.

4.1 Exit Criteria

  • All High Priority test cases executed.
  • Minimum 70% success rate on waste classification.
  • Correct waste sorting confirmed in functional tests.
  • Usability rating of at least 80% positive feedback.
  • Security tests passed without critical vulnerabilities.

5. REFERENCES

  • Software Requirement Specification (March 2025)
  • Software Design Description

6. TEST DESIGN SPECIFICATIONS

6.1 YOLOv11 Waste Detection (YO)

Subfeatures:

  • Plastic Detection (YO.PL)
  • Metal Detection (YO.MT)
  • Glass Detection (YO.GL)
  • Paper Detection (YO.PP)
  • Model Accuracy Testing (YO.AC)
  • Confusion Matrix Evaluation (YO.CM)
  • False Positive/Negative Analysis (YO.FP)

Test Cases: image

6.2 Raspberry Pi Control System (RPi)

Subfeatures:

  • Model Inference Trigger (RPi.INF)
  • Camera Interface Test (RPi.CAM)
  • GPIO Pin Test (RPi.GPIO)

Test Cases: image

6.3 Arduino Sensors & Actuators (ARD)

Subfeatures:

  • Proximity Sensor (ARD.PS)
  • Servo Motor Control (ARD.SV)
  • Metal Sensor Response (ARD.MS)
  • Motor Speed and Direction Control (ARD.MD)

Test Cases: image

6.4 Waste Sorting Mechanism (WM)

Subfeatures:

  • Conveyor Belt Mechanism (WM.CB)

Test Cases: image

6.5 User Experience (UX)

Subfeatures:

  • Usability Testing (UX.US)
  • Task Efficiency (UX.TE)
  • Error Recovery and Feedback (UX.ER)

Test Cases: image

7. DETAILED TEST CASES

YO.PL.01

  • Purpose: Detect plastic waste correctly
  • Requirements: 3.1
  • Priority: High
  • Dependency: YOLOv11 model trained with plastic class
  • Setup: Load sample images containing plastic waste
  • Procedure:
    • [A01] Run YOLOv11 on plastic images
    • [V01] Verify detection includes correct plastic labels and bounding boxes
  • Cleanup: Log detection confidence scores and bounding box coordinates

YO.MT.01

  • Purpose: Detect metal waste correctly
  • Requirements: 3.2
  • Priority: High
  • Dependency: YOLOv11 model trained with metal class
  • Setup: Load sample images containing metal waste
  • Procedure:
    • [A01] Run YOLOv11 on metal images
    • [V01] Confirm model detects metal items accurately
  • Cleanup: Generate accuracy report

YO.GL.01

  • Purpose: Detect glass waste correctly
  • Requirements: 3.3
  • Priority: Medium
  • Dependency: YOLOv11 model includes glass category
  • Setup: Provide test images with glass waste
  • Procedure:
    • [A01] Execute model inference on glass images
    • [V01] Check that glass objects are correctly identified
  • Cleanup: Note any missed or misclassified glass items

YO.PP.01

  • Purpose: Detect paper waste correctly
  • Requirements: 3.4
  • Priority: Medium
  • Dependency: YOLOv11 model supports paper class
  • Setup: Select images featuring paper waste
  • Procedure:
    • [A01] Perform detection using YOLOv11
    • [V01] Validate accurate classification of paper objects
  • Cleanup: Record results and annotate errors if any

YO.CM.01

  • Purpose: Evaluate confusion matrix of classification
  • Requirements: 3.6
  • Priority: Medium
  • Dependency: YOLOv11 test results generated
  • Setup: Use classification results from test dataset
  • Procedure:
    • [A01] Calculate confusion matrix
    • [V01] Analyze true positives, true negatives, false positives, and false negatives
  • Cleanup: Document confusion matrix analysis

YO.FP.01

  • Purpose: Analyze false positives and false negatives
  • Requirements: 3.7
  • Priority: Medium
  • Dependency: Confusion matrix calculated
  • Setup: Identify misclassified items
  • Procedure:
    • [A01] Analyze false positives and false negatives
    • [V01] Document causes of misclassifications
  • Cleanup: Propose model improvements

RPi.INF.01

  • Purpose: Trigger YOLOv11 model on Raspberry Pi and return result
  • Requirements: 4.1
  • Priority: High
  • Dependency: YOLOv11 model deployed on Raspberry Pi
  • Setup: Raspberry Pi booted and camera connected, YOLOv11 inference script ready
  • Procedure:
    • [A01] Trigger image capture via control signal
    • [A02] Run YOLOv11 inference on captured image
    • [V01] Validate detection result is returned successfully
  • Cleanup: Log inference response and timing

RPi.CAM.01

  • Purpose: Validate camera initialization and image capture
  • Requirements: 4.2
  • Priority: Medium
  • Dependency: Camera connected and powered
  • Setup: Raspberry Pi with camera module connected
  • Procedure:
    • [A01] Initialize camera via test script
    • [A02] Capture image
    • [V01] Verify image is saved/displayed correctly
  • Cleanup: Delete captured test images

RPi.GPIO.01

  • Purpose: Test GPIO pin response for sensor communication
  • Requirements: 4.3
  • Priority: Medium
  • Dependency: Sensors connected to GPIO pins
  • Setup: Connect mock sensors or test circuits to GPIO pins
  • Procedure:
    • [A01] Send/receive signal on GPIO pins
    • [V01] Verify correct read/write operation
  • Cleanup: Propose model improvements

ARD.PS.01

  • Purpose: Detect object presence with proximity sensor
  • Requirements: 5.1
  • Priority: High
  • Dependency: Arduino board and proximity sensor connected
  • Setup: Place object in proximity range
  • Procedure:
    • [A01] Monitor sensor output
    • [V01] Validate detection triggers correct signal
  • Cleanup: Propose model improvements

ARD.SV.01

  • Purpose: Activate servo motor for sorting
  • Requirements: 5.2
  • Priority: Medium
  • Dependency: Servo motor connected and powered
  • Setup: System idle and item detected
  • Procedure:
    • [A01] Send activation signal to servo
    • [V01] Validate movement to correct bin position
  • Cleanup: Reset motor to neutral position

ARD.MS.01

  • Purpose: Detect metal object and signal system correctly
  • Requirements: 5.3
  • Priority: Medium
  • Dependency: Metal sensor connected
  • Setup: Place metal item near sensor
  • Procedure:
    • [A01] Monitor sensor output
    • [V01] Validate correct system signaling
  • Cleanup: Log detection output

ARD.MD.01

  • Purpose: Test motor speed and direction control
  • Requirements: 5.4
  • Priority: Medium
  • Dependency: Motor driver and DC motor connected
  • Setup: Load test firmware to Arduino
  • Procedure:
    • [A01] Send speed and direction commands
    • [V01] Validate motor rotates accordingly
  • Cleanup: Stop and reset motor

WM.CB.01

  • Purpose: Conveyor directs item to correct bin based on classification
  • Requirements: 6.1
  • Priority: Medium
  • Dependency: Fully integrated system with actuators
  • Setup: Item placed on conveyor belt
  • Procedure:
    • [A01] Run classification and sorting routine
    • [V01] Observe if item moves to correct bin
  • Cleanup: Reset conveyor for next item

UX.US.01

  • Purpose: Evaluate usability with end users
  • Requirements: 7.1
  • Priority: High
  • Dependency: UI functional and accessible
  • Setup: Conduct usability testing session with test users
  • Procedure:
    • [A01] Ask users to complete waste sorting task
    • [V01] Collect feedback on interface clarity and ease of use
  • Cleanup: Summarize findings in usability report

UX.TE.01

  • Purpose: Measure time-to-task efficiency
  • Requirements: 7.2
  • Priority: Medium
  • Dependency: Interface deployed
  • Setup: Set stopwatch or log interaction timing
  • Procedure:
    • [A01] Instruct user to perform sorting or monitoring task
    • [V01] Record task completion time
  • Cleanup: Compare with baseline targets

UX.ER.01

  • Purpose: Test error messages and system feedback
  • Requirements: 7.3
  • Priority: Medium
  • Dependency: Error handling implemented in UI
  • Setup: Simulate invalid inputs and system failures
  • Procedure:
    • [A01] Trigger known error conditions
    • [V01] Verify appropriate error messages and guidance displayed
  • Cleanup: Reset error states and retry flows

Figures

  • Figure 12: Waste Detection
    image

  • Figure 13: Waste Classification
    image

  • Figure 14: Waste Detection Area image

  • Figure 15: Detection Percentages
    image

  • Figure 16: Raspberry Pi and Camera Module
    image

  • Figure 17: New Data Insert
    image