Architecture Overview - theweebcoders/CommercialBreaker GitHub Wiki

Architecture Overview

This document provides a technical overview of the CommercialBreaker & Toonami Tools system architecture, component relationships, and data flow.

System Overview

CommercialBreaker & Toonami Tools is a modular Python application designed to automate the creation of Toonami-style anime marathon channels. The system processes anime episodes, detects commercial break points, and integrates authentic Toonami bumps to create seamless viewing experiences.

Core Philosophy

  • Modular Design: Each tool handles a specific aspect of the pipeline
  • Database-Driven: SQLite manages metadata and processing state
  • Interface Flexibility: Multiple UIs for different use cases
  • Platform Agnostic: Works with various media servers and channel platforms

Application Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                     Entry Point (main.py)                   β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚Interface Selection: --tom | --webui | --clydes | --combreak β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                           β”‚
                           β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚   GUI Layer     β”‚   Web Layer     β”‚   CLI Layer     β”‚
β”‚                 β”‚                 β”‚                 β”‚
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
β”‚ β”‚ TOM.py      β”‚ β”‚ β”‚Absolution.pyβ”‚ β”‚ β”‚ clydes.py   β”‚ β”‚
β”‚ β”‚ (Tkinter)   β”‚ β”‚ β”‚ (REMI/Web)  β”‚ β”‚ β”‚ (Console)   β”‚ β”‚
β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜
          β”‚                 β”‚                 β”‚
          β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                            β–Ό 
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”          
β”‚               FrontEndLogic.py (Orchestrator API)           β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ β€’ LogicController class - Central API for all UIs           β”‚           
β”‚ β€’ State management via SQLite database via DataManager      β”‚
β”‚ β€’ Threading for background operations                       β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜           
        β”‚    β”‚  β–²          β”‚    β–²
        β”‚    β”‚  β”‚          β”‚    β”‚
        β”‚    β”‚  β”‚          β–Ό    β”‚  
        β”‚    β”‚  β”‚   β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
        β”‚    β”‚  β”‚   β”‚                  Supporting API Modules                   β”‚
        β”‚    β”‚  β”‚   β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
        β”‚    β”‚  β”‚   β”‚  FlagManager.py          β”‚  MessageBroker.py              β”‚
        β”‚    β”‚  β”‚   β”‚ β€’ Platform compatibility β”‚ β€’ Real-time communication      β”‚
        β”‚    β”‚  β”‚   β”‚ β€’ Global Flags           β”‚ β€’ In-memory pub/sub            β”‚
        β”‚    β”‚  β”‚   β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
        β”‚    β”‚  β”‚   β”‚  NetworkManager.py       β”‚  ErrorManager.py               β”‚
        β”‚    β”‚  β”‚   β”‚ β€’ Network validation     β”‚ β€’ Centralized error handling   β”‚
        β”‚    β”‚  β”‚   β”‚ β€’ Config persistence     β”‚ β€’ Error history tracking       β”‚
        β”‚    β”‚  β”‚   β”‚   (update config.py)     β”‚ β€’ UI error broadcasting        β”‚
        β”‚    β”‚  β”‚   β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
        β”‚    β”‚  β”‚   β”‚  DatabaseValidator       β”‚  FilenameParser                β”‚
        β”‚    β”‚  β”‚   β”‚ β€’ S.A.R.A. orchestration β”‚ β€’ Centralized parsing logic    β”‚
        β”‚    β”‚  β”‚   β”‚ β€’ 20 step validators     β”‚ β€’ Episode name extraction      β”‚
        β”‚    β”‚  β”‚   β”‚ β€’ Integrity validation   β”‚ β€’ Year handling (2002)         β”‚
        β”‚    β”‚  β”‚   β”‚ β€’ Phase-aware tracking   β”‚ β€’ Consistent across tools      β”‚
        β”‚    β”‚  β”‚   β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
        β”‚    β”‚  β”‚   β”‚  NetworkUtils.py                                          β”‚
        β”‚    β”‚  β”‚   β”‚ β€’ Curl-based HTTP client  β€’ Wikipedia table parser        β”‚
        β”‚    β”‚  β”‚   β”‚ β€’ Response interface      β€’ Request exceptions            β”‚
        β”‚    β”‚  β”‚   β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
        β”‚    β”‚  β”‚                                               β–² 
        β”‚    β–Ό  β”‚                                               β”‚
        β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”   β”‚
        β”‚  β”‚                    Database                    β”‚   β”‚ 
        β”‚  β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€   β”‚
        β”‚  β”‚ DatabaseManager.py                             β”‚   β”‚
        β”‚  β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€   β”‚
        β”‚  β”‚ β€’ Thread-safe connections β€’ Transaction supportβ”‚   β”‚
        β”‚  β”‚ β€’ Automatic retry logic   β€’ Simplified API     β”‚   β”‚
        β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜   β”‚
        β”‚     β–²  β”‚                                              β”‚
        β”‚     β”‚  β”‚                β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
        β–Ό     β”‚  β–Ό                β”‚
   β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
   β”‚                   Core Processing Layer                     β”‚
   β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
   β”‚   ComBreak/     β”‚ ToonamiTools/   β”‚     ExtraTools/         β”‚
   β”‚                 β”‚                 β”‚                         β”‚
   β”‚ β€’ Commercial    β”‚ β€’ Show Detectionβ”‚ β€’ Manual Tools          β”‚
   β”‚   Detection     β”‚ β€’ Lineup        β”‚ β€’ Utilities             β”‚
   β”‚ β€’ File Cutting  β”‚   Generation    β”‚ β€’ Debugging             β”‚
   β”‚ β€’ Cutless Mode  β”‚ β€’ Bump Encoding β”‚                         β”‚
   β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                           β”‚
                           β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚              Platform Integration Layer                       β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚   Plex API      β”‚   Platform APIs     β”‚  ComBreakDirect       β”‚
β”‚                 β”‚                     β”‚                       β”‚
β”‚ β€’ OAuth Auth    β”‚ β€’ External REST     β”‚ β€’ Self-contained      β”‚
β”‚ β€’ Library Scan  β”‚ β€’ Channel Creation  β”‚ β€’ Direct streaming    β”‚
β”‚ β€’ Timestamps    β”‚                     β”‚ β€’ M3U/XMLTV gen       β”‚
β”‚ β€’ Smart Retry   β”‚                     β”‚ β€’ Plex discovery      β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Plex Integration

CommercialBreaker includes a Plex client implementation located in API/utils/:

Plex Client Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                 PlexConnectionHelper.py                 β”‚
β”‚  β€’ Smart connection retry (local β†’ direct β†’ relay)     β”‚
β”‚  β€’ Server discovery and connection management          β”‚
β”‚  β€’ Automatic failover between connection URLs          β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                 β”‚
      β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
      β”‚                     β”‚
      β–Ό                     β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ PlexClient.py    β”‚  β”‚ PlexServer.py    β”‚
β”‚                  β”‚  β”‚                  β”‚
β”‚ β€’ PlexAuthClient β”‚  β”‚ β€’ SimplePlexServ.β”‚
β”‚ β€’ OAuth PIN flow β”‚  β”‚ β€’ Library access β”‚
β”‚ β€’ Account client β”‚  β”‚ β€’ Section queriesβ”‚
β”‚ β€’ Resource list  β”‚  β”‚ β€’ Episode data   β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
         β”‚                      β”‚
         β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                    β–Ό
         β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
         β”‚   CurlHttpClient       β”‚
         β”‚   (ToonamiTools/)      β”‚
         β”‚  β€’ HTTP operations     β”‚
         β”‚  β€’ Stdlib based        β”‚
         β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Key Benefits:

  • Minimal Dependencies: Uses Python stdlib urllib and CurlHttpClient
  • Connection Reliability: Automatically tries all available server URLs
  • Lightweight: Only implements features actually used by CommercialBreaker
  • Stability: Direct control over Plex API interactions

Integration Points:

  • LoginToPlex.py - OAuth authentication and server selection
  • GetTimestampPlex.py - Fetch "Skip Intro" timestamps
  • PlexAutoSplitter.py - Split merged episodes
  • RenameSplitPlex.py - Rename split episodes to match file names

In-Memory Message Broker

A unified in-memory message broker is responsible for all real-time communication between the LogicController and the various user interfaces (GUI, Web, CLI).

  • Channel-Based Communication: All UIs subscribe to relevant channels for updates and publish user actions/events.
  • Decoupled Integration: Interfaces interact with the LogicController exclusively through the message broker, ensuring modularity and testability.

Database Management

The CommercialBreaker uses a centralized DatabaseManager for all database operations, providing:

  • Thread-safe connections: Each thread gets its own database connection
  • Automatic retry logic: Handles database locks with exponential backoff
  • Transaction support: Atomic operations with automatic commit/rollback
  • Simplified API: Common operations wrapped in convenient methods
  • Dictionary-based operations: Queries return lists of dictionaries (fetchall_as_dicts(), bulk_insert_dicts(), etc.)

All modules access the database through get_db_manager() from API.utils.DatabaseManager, ensuring consistent error handling and preventing database lock issues in multi-threaded scenarios.

Centralized Error Handling

The system employs a centralized error handling mechanism via ErrorManager.py, which provides:

  • Global Error Tracking: All errors are logged and can be retrieved by any interface
  • UI Broadcasting: Errors can be sent to all interfaces for real-time user feedback
  • Error History: Maintains a history of errors for debugging and user support
  • Modular Error Handling: Each module can raise errors that are caught and processed by the ErrorManager
  • Custom Error Types: Allows for specific error handling based on module needs

S.A.R.A. Validation System

The S.A.R.A. (System Analysis and Reporting Assistant) validation system provides comprehensive database validation and diagnostics. It follows the philosophy: "Validate everything, trust nothing."

Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                    DatabaseValidator                        β”‚
β”‚               (Main Orchestration Layer)                    β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ β€’ Coordinates 20 validators (18 step + 2 integrity)         β”‚
β”‚ β€’ Consolidates issues intelligently                         β”‚
β”‚ β€’ Reports to ErrorManager β†’ broadcasts to all UIs           β”‚
β”‚ β€’ Phase-aware progress tracking                             β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                       β”‚
                       β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                    BaseValidator                            β”‚
β”‚            (Abstract Base for All Validators)               β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ β€’ Database access utilities (via DatabaseManager)           β”‚
β”‚ β€’ Bump vs anime detection (consistent with main tools)      β”‚
β”‚ β€’ Mode detection (cutless vs traditional)                   β”‚
β”‚ β€’ Platform detection (DizqueTV, Tunarr, ComBreakDirect)     β”‚
β”‚ β€’ Issue creation and management                             β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                       β”‚
        β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
        β”‚              β”‚              β”‚              β”‚
        β–Ό              β–Ό              β–Ό              β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚Step Validatorβ”‚ β”‚  Integrity   β”‚ β”‚PhaseTracker  β”‚ β”‚ValidationRes.β”‚
β”‚  (18 total)  β”‚ β”‚ Validators   β”‚ β”‚              β”‚ β”‚              β”‚
β”‚              β”‚ β”‚  (2 total)   β”‚ β”‚β€’ Maps steps  β”‚ β”‚β€’ Result      β”‚
β”‚β€’ Platform    β”‚ β”‚              β”‚ β”‚  to phases   β”‚ β”‚β€’ Issue       β”‚
β”‚β€’ ToonamiCheckβ”‚ β”‚β€’ Lineup      β”‚ β”‚β€’ Progress    β”‚ β”‚β€’ Level       β”‚
β”‚β€’ LineupPrep  β”‚ β”‚  Integrity   β”‚ β”‚  tracking    β”‚ β”‚β€’ Status      β”‚
β”‚β€’ CommercialBrβ”‚ β”‚β€’ Referential β”‚ β”‚β€’ Conditional β”‚ β”‚              β”‚
β”‚β€’ (15 more)   β”‚ β”‚  Integrity   β”‚ β”‚  step logic  β”‚ β”‚              β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Key Features

Step Validation:

  • Each pipeline step has a dedicated validator
  • Validates table existence, structure, and data quality
  • Checks cross-table relationships and referential integrity
  • Detects platform compatibility issues

Integrity Validation:

  • LineupIntegrityValidator: Validates bump placement rules (multibumps β†’ anime, intro β†’ matching BLOCK_ID, etc.)
  • ReferentialIntegrityValidator: Validates cross-table consistency (BLOCK_IDs exist, file paths valid, etc.)

Phase-Aware Tracking:

  • Maps individual steps to logical workflow phases (0-6)
  • Tracks which phase is complete vs in-progress
  • Accounts for conditional steps (PlexAuth, BumpCalculator, etc.)

Issue Consolidation:

  • Groups related issues to prevent overwhelming users
  • Three consolidation strategies:
    1. Version-based (e.g., "Version 2: missing data, Version 3: invalid timestamps")
    2. List-items (e.g., "3 folder issues: β€’ ANIME_FOLDER missing β€’ BUMPS_FOLDER missing β€’ ...")
    3. Generic numbered list for other multi-issue scenarios

Mode Detection:

  • Automatically detects cutless vs traditional mode
  • Identifies platform (DizqueTV, Tunarr, ComBreakDirect)
  • Warns about incompatibilities (e.g., cutless mode with Tunarr)

UI Integration: Page8 Diagnostics

S.A.R.A. is accessible through Page8 in the GUI interfaces (TOM and Absolution):

Access Method:

  • Hidden Panic Button: Click any page title 5 times in 2 seconds

Features:

  • Pipeline status with completion checklist (βœ“/βœ— for each step)
  • Run full validation button
  • Refresh status button
  • Copy results to clipboard (for bug reports)
  • Real-time status updates during validation
  • Grouped issues by severity (CRITICAL, ERROR, WARNING, INFO)

Hidden Panic Button

Implementation: Track clicks on page title labels. 5 clicks within 2 seconds β†’ navigate to Page8.

Purpose: Provides quick access to diagnostics without memorizing menu structure, especially useful during critical errors.

Location: Implemented on all pages except Page5 in TOM (complex layout).

Validation Output

Each validation issue includes:

  • Where: Which step, table, and row (if applicable)
  • What: User-friendly message describing the problem
  • Why: Technical details for developers
  • How to Fix: Actionable suggestion
  • When: Timestamp

Example Issue:

[ERROR] CommercialBreaker β†’ cuts
Message: 3 episodes missing commercial break timestamps
Details: Files: Naruto S01E05.mkv, Naruto S01E06.mkv, Bleach S02E03.mkv
Suggestion: Re-run CommercialBreaker in normal mode (not low power) to detect breaks

Integration with FrontEndLogic

Methods Added to LogicController:

def run_full_validation(self):
    """Run comprehensive database validation (background thread)"""

def get_pipeline_status(self) -> PipelineStatus:
    """Get quick pipeline status without full validation"""

For complete S.A.R.A. documentation, see S.A.R.A. Validation System.


Filename Parsing

The FilenameParser utility provides centralized, consistent episode filename parsing across all modules.

Location: /ToonamiTools/utils/FilenameParser.py

Features:

  • Automatic year stripping from show names (e.g., "Naruto (2002)" β†’ "Naruto")
  • Comprehensive SxxExx pattern extraction
  • Season and episode number parsing (handles single and double digits)
  • Validation and error reporting

Used By:

  • VirtualCut (cutless mode operations)
  • DirectoryScanner (file discovery)
  • BlockMaker (BLOCK_ID generation)
  • PlexToDizqueTV (channel creation)
  • All validators (consistency check)

Benefits:

  • Single source of truth for filename parsing
  • Consistent behavior across entire codebase
  • Automatic year handling from show names
  • Easier to maintain and test

Plex Connection Reliability

Enhanced Plex connection reliability with retry logic and progressive timeouts.

Improvements:

  • Retry Logic: Up to 3 attempts with configurable delay
  • Progressive Timeout: Increases with each retry (base + 20s per attempt)
  • Status Broadcasting: Real-time updates to user during retries
  • Configuration-Driven: config.PLEX_RETRY_ATTEMPTS, config.PLEX_RETRY_DELAY, timeouts

Modified Files:

  • LoginToPlex.py - PlexLibraryManager and PlexLibraryFetcher
  • PlexServer.py - SimplePlexServer with configurable timeout

Why: Intermittent failures on slower networks or when servers are starting up. Retry logic makes connection more reliable without user intervention.


Data Flow

1. Initialization Phase

User Input β†’ Interface Selection β†’ Configuration Loading β†’ Platform Selection β†’ Plex Authentication (conditional)

Note: Platform selection (DizqueTV, Tunarr, or ComBreakDirect) determines whether Plex authentication is required. ComBreakDirect users skip Plex authentication entirely.

2. Content Discovery Phase

Plex Library Scan β†’ Show Detection (IMDB/Wikipedia) β†’ Bump Analysis β†’ Database Population

3. Processing Phase

Episode Selection β†’ Commercial Detection β†’ File Processing β†’ Metadata Generation

4. Integration Phase

Lineup Generation β†’ Platform Channel Creation β†’ Playback Optimization

Platform Options:

  • External Platforms: Traditional platform integration via REST APIs
  • ComBreakDirect: Self-contained streaming server (new)

5. ComBreakDirect Streaming Phase (Optional)

ComBreakToComBreakDirect β†’ ComBreakDirect Server β†’ Live MPEG-TS Stream β†’ Plex/Clients
                                  β”‚
                                  └─▢ LineupExtender (per channel)
                                         β”‚  threading.Timer
                                         β”‚  fires INFINITE_EXTEND_LEAD_MS
                                         β”‚  before channel end
                                         β–Ό
                                  InfiniteChannelExtender
                                  (ShowScheduler + CutlessFinalizer)
                                         β”‚
                                         β–Ό
                                  FactoryFloor.extend_channel
                                  (append-only, persists channels.json)

ComBreakDirect channels are never finite. The LineupExtender's per-channel timer fires on wall-clock time (independent of whether a client is streaming) and appends a fresh ~40 hour chunk before the existing one runs out. The channel survives indefinitely until the LineupExtender is explicitly disabled or _infinite_meta.enabled is set to false on the channel.


ComBreakDirect Architecture

ComBreakDirect is an optional self-contained streaming server using Studio β†’ FIFO β†’ Broadcast FFmpeg β†’ BroadcastTower architecture. It provides:

  • Continuous MPEG-TS Streaming: True broadcast-style streaming with seamless program transitions
  • BroadcastTower Multi-Client: Efficient distribution to multiple simultaneous viewers
  • WebUI: Toonami-themed landing page with setup instructions
  • M3U8 Generation: Dynamic playlist creation for Plex/Jellyfin
  • XMLTV Guides: Full EPG data with show metadata
  • Plex/Jellyfin Discovery: HDHomeRun-style auto-discovery
  • Commercial Injection: Server-side ad break management with pre-rendering
  • Intelligent Audio Selection: Configurable audio track selection (defaults to English for Toonami)
  • Auto Lifecycle: Studio and broadcast FFmpeg start/stop based on client connections
  • Infinite Channel Extension: Per-channel threading.Timer arms ahead of channel end, runs ShowScheduler+CutlessFinalizer to append the next chunk, then reschedules β€” wall-clock based

ComBreakDirect Components

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚   ComBreakDirect Server (Flask + Studio + BroadcastTower)       β”‚
β”‚                        Port 8083                                 β”‚
β”‚                                                                  β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”   β”‚
β”‚  β”‚                    WebUI (UI/)                           β”‚   β”‚
β”‚  β”‚  β€’ Landing page (setup instructions, copy-to-clipboard) β”‚   β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜   β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                              β”‚
            β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
            β–Ό                 β–Ό                 β–Ό
  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
  β”‚  Loading Dock    β”‚ β”‚Factory Floor β”‚ β”‚ Unloading Dock   β”‚
  β”‚   (docks/)       β”‚ β”‚  (docks/)    β”‚ β”‚   (docks/)       β”‚
  β”‚                  β”‚ β”‚              β”‚ β”‚                  β”‚
  β”‚ β€’ Process lineup β”‚ β”‚ β€’ Store data β”‚ β”‚ β€’ Studio thread  β”‚
  β”‚ β€’ Inject ads     β”‚ β”‚ β€’ Generate   β”‚ β”‚ β€’ Broadcast      β”‚
  β”‚ β€’ Pre-render     β”‚ β”‚   M3U8/XMLTV β”‚ β”‚   FFmpeg         β”‚
  β”‚   breaks         β”‚ β”‚ β€’ Persist    β”‚ β”‚ β€’ BroadcastTower β”‚
  β”‚                  β”‚ β”‚   channels   β”‚ β”‚ β€’ Multi-client   β”‚
  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
           β”‚
           β–Ό
  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
  β”‚          Supporting Utilities (utilities/)                  β”‚
  β”‚  β€’ BroadcastTower - Multi-client streaming engine          β”‚
  β”‚  β€’ AudioTrackSelector - Intelligent audio selection        β”‚
  β”‚  β€’ CommercialBreakRenderer - Pre-rendering system          β”‚
  β”‚  β€’ CleanupManager - Automatic file maintenance             β”‚
  β”‚  β€’ configuration - Path resolution                         β”‚
  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
  β”‚   Infinite Channel Extension (docks/ + ToonamiTools/)       β”‚
  β”‚  β€’ LineupExtender (docks/) - Per-channel threading.Timer   β”‚
  β”‚    watchdog, fires before channel end                       β”‚
  β”‚  β€’ InfiniteChannelExtender (ToonamiTools/) - Builds one    β”‚
  β”‚    extension chunk via ShowScheduler + CutlessFinalizer    β”‚
  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

  Broadcasting Architecture (Unloading Dock):

  Studio Thread β†’ FIFO β†’ Broadcast FFmpeg β†’ BroadcastTower β†’ Antennas
     (full tape,     (pipe)   (+genpts,        (distributes    (per-client
      normalizes              continuous)       to all)         buffers)
      programs)

Key Features:

  • Loading Dock (docks/LoadingDock.py): Processes cutless lineup data, injects commercials between bumps, pre-renders breaks. format_extension reuses the program-builder to splice extension chunks onto the existing channel timeline.
  • Factory Floor (docks/FactoryFloor.py): Generates M3U8 playlists and XMLTV guides, stores channel configurations. extend_channel appends new programs under the factory lock; _maybe_spawn_extender arms a LineupExtender per infinite channel at store_channel and on _load_channels restart.
  • Unloading Dock (docks/UnloadingDock.py): Manages studio threads, broadcast FFmpeg processes, and BroadcastTower distribution
  • BroadcastTower (utilities/BroadcastTower.py): Multi-client streaming engine with Antenna pattern
  • WebUI (UI/WebUI.py): Landing page with setup instructions and copy-to-clipboard buttons
  • Audio Selector (utilities/AudioTrackSelector.py): Configurable audio track selection (defaults to English)
  • Commercial Renderer (utilities/CommercialBreakRenderer.py): Pre-renders breaks to eliminate startup delays
  • Cleanup Manager (utilities/CleanupManager.py): Automatically removes old pre-rendered breaks
  • Lineup Extender (docks/LineupExtender.py): Per-channel threading.Timer scheduler, fires INFINITE_EXTEND_LEAD_MS (default 3h) before the last program ends, then reschedules after each extension
  • Infinite Channel Extender (ToonamiTools/InfiniteChannelExtender.py): One-shot orchestrator the LineupExtender calls β€” runs ShowScheduler with continue_from_last_used_episode_block=True against a per-extension SQLite table, then CutlessFinalizer, then returns the rows for LoadingDock to format

Technical Architecture:

  • Studio Thread: Calculates channel position, spawns FFmpeg per program with normalization, writes to FIFO
  • Broadcast FFmpeg: Reads FIFO with +genpts, creates continuous stream, rate limits broadcasting
  • BroadcastTower: Receives stream, distributes to all Antenna objects (one per client)
  • Antenna Buffer: Per-client 132-chunk buffer (~2 seconds) with rate limiting
  • Auto Lifecycle: Studio and broadcast FFmpeg only run when clients connected
  • Real-time Sync: All clients synchronized to same channel position
  • Wall-Clock Extension: LineupExtender's timer fires on real time, independent of playback β€” channels grow even when no client is streaming. The watchdog reads the last program's stop ISO timestamp, not Studio's current_index.
  • Cursor-Driven Continuation: ComBreakToComBreakDirect._seed_episode_cursor populates last_used_episode_block from the initial lineup before the channel is POSTed, so the first extension picks up at E+1 of each show (not the same E0 again, not a random jump)
  • Append-Only Growth: FactoryFloor.extend_channel only ever calls programs.extend() β€” never replaces or reorders β€” so the Studio's programs[i % len(programs)] reads stay coherent across the seam without read-side locking

Integration Points:

  • ComBreakToComBreakDirect - Pushes cutless lineup to server via REST API; sets infinite=True + infinite_meta on the payload for ComBreakDirect channels
  • LogicController._ensure_combreakdirect_server() - Auto-starts server
  • Flask REST API for channel management and streaming
  • HDHomeRun discovery for Plex DVR integration
  • WebUI served at root path for easy access
  • config.INFINITE_EXTEND_LEAD_MS - Tunable lead time (default 10 800 000 ms = 3h) for when the LineupExtender fires ahead of channel end

For detailed ComBreakDirect documentation, see ComBreakDirect.md.


This architecture supports the system's goals of modularity, reliability, and extensibility while maintaining the performance necessary for processing large media libraries efficiently. The addition of ComBreakDirect provides an alternative deployment model that eliminates external platform dependencies.