vision based gap calculation - VforVitorio/F1_Strat_Manager GitHub Wiki

Vision-based Gap Calculation

VforVitorio/F1_Strat_Manager

powered by

Devin

Vision-based Gap Calculation

This document explains the computer vision system used to accurately calculate gaps between Formula 1 cars from video footage. The system leverages YOLOv8 object detection to identify cars and their teams, then calculates real-world distances and time gaps to support strategic decision-making. This data feeds into the gap analysis rules covered in Gap Analysis Rules and the gap analysis view described in Gap Analysis View.

1. System Overview

The gap calculation system processes F1 race footage to extract real-time information about the relative positions of cars on track. This information is crucial for identifying undercut opportunities, defensive positioning, and overall race progress.

2. YOLOv8 Model for Team Identification

The system uses a customized YOLOv8 model specifically fine-tuned to identify all 10 Formula 1 teams. This model is essential for tracking cars and ensuring gap measurements are accurate and meaningful by identifying which teams are involved.

2.1 Model Architecture and Training

The model was fine-tuned from YOLOv8m (medium) base architecture with customized hyperparameters to optimize for F1 car detection. Special attention was given to teams with similar liveries that were prone to misclassification.

2.2 Model Performance and Thresholds

The model achieves an overall mAP50 of 0.815 with varying performance across teams. To handle teams with poorer detection reliability, team-specific confidence thresholds were implemented.

Team mAP50 Confidence Threshold Notes
Williams 0.450 0.90 Requires very high threshold
Alpine 0.811 0.90 Requires very high threshold
Kick Sauber 0.710 0.40 Lower recall (0.5)
Racing Bulls 0.995 0.40 Excellent detection
Ferrari 0.788 0.40 Good detection
Mercedes 0.995 0.50 Excellent detection
McLaren 0.775 0.30 Lower threshold, distinctive livery
Haas 0.995 0.40 Excellent detection

3. Video Processing Pipeline

The video processing system handles frame-by-frame analysis, car detection, and tracking across frames to maintain consistent car identities.

3.1 Frame Processing and Detection

For each frame, the system:

  1. Resizes the frame to a standard processing size
  2. Applies YOLOv8 detection with appropriate thresholds
  3. Processes detection results to extract bounding boxes
  4. Assigns car identity through tracking algorithm
  5. Calculates gaps between detected cars

3.2 Object Tracking System

The tracking system maintains car identities across frames using position-based matching and classification history to improve stability.

4. Gap Calculation Algorithm

The gap calculation component is responsible for converting pixel distances into real-world measurements in both meters and time.

4.1 Distance Calculation Method

The system uses a scale-based approach leveraging the known dimensions of F1 cars (5.63 meters) to convert pixel distances to real-world measurements.

4.2 Implementation Details

The calculate_gap function determines the real-world gap between cars:

  1. Takes two car bounding boxes as input
  2. Calculates the center points of each box
  3. Computes the Euclidean distance between centers in pixels
  4. Uses the average width of the cars as a scaling factor
  5. Converts to real-world distance in meters
  6. Calculates time gap assuming 300 km/h (83.33 m/s) speed

The implementation assumes that car width in pixels is proportional to its real-world size, enabling accurate scaling even with perspective changes.

5. Data Extraction System

The gap calculation system can operate in two modes:

  1. Real-time visualization: Processes video and shows gaps between cars with visual overlays
  2. Gap data extraction: Systematically samples frames to build a dataset of gaps

5.1 Sampling Approach

The data extraction mode samples frames at regular intervals (typically 10 seconds) to build a comprehensive dataset of gap evolution throughout a race.

5.2 Output Format and Integration

The gap data is exported as a CSV file with the following structure:

Column Description
frame Frame number in video
timestamp Time in seconds from start of video
car1_id ID of the first car (closer to camera)
car2_id ID of the second car
car1_team Team classification of first car
car2_team Team classification of second car
distance_meters Gap distance in meters
gap_seconds Gap time in seconds at 300 km/h

This structured data feeds directly into the F1 Strategy Engine's gap analysis rules, providing critical information for race strategy decisions.

6. Visualization Capabilities

The system includes comprehensive visualization features that assist in verification and analysis:

  1. Bounding boxes: Color-coded by team
  2. ID tracking: Consistent identifiers for each car
  3. Gap overlays: Visual display of distance and time gaps
  4. Classification confidence: Display of detection confidence

7. System Integration

The gap calculation system integrates with the broader F1 Strategy Manager system by providing structured gap data that feeds into the expert rule system and gap analysis visualization.

8. Limitations and Considerations

  • Detection accuracy: While the model achieves good overall performance, certain teams (Williams, Alpine) require higher thresholds due to classification challenges.
  • Speed estimation: Gap time calculations assume a constant speed of 300 km/h, which is a simplification.
  • Perspective effects: Distances can be less accurate when cars are far from the camera or in unusual orientations.
  • Occlusion: Partially visible cars may affect detection and measurement accuracy.

These limitations are mitigated through the classification history system and team-specific thresholds.

On this page