Solution Architecture View - RogerThattt/Data-Flywheel GitHub Wiki

🏗️ Solution Architecture on Databricks 🔷 High-Level Architecture

            +------------------------+
            |   Telecom Data Sources |
            | (CDRs, CRM, OSS/BSS)   |
            +----------+-------------+
                       |
                       v
  +-----------------------------------------+
  |       Ingestion Layer (AutoLoader, Kafka, DLT)      |
  | - Real-time: Kafka / Auto Loader                    |
  | - Batch: Ingest from S3, ADLS, GCS                  |
  +----------------+------------------------+
                   |
                   v
  +----------------+------------------------+
  |        Bronze Layer (Raw Landing)       |
  | - Raw CDRs, events, customer logs       |
  | - Auditable, schema-preserved           |
  +----------------+------------------------+
                   |
                   v
  +----------------+------------------------+
  |       Silver Layer (Cleaned + Joined)   |
  | - Feature extraction                    |
  | - De-duplication, anonymization         |
  | - Temporal joins, aggregation           |
  +----------------+------------------------+
                   |
                   v
  +----------------+------------------------+
  |     Gold Layer (ML Features & KPIs)     |
  | - Churn prediction dataset              |
  | - Real-time scoring features            |
  +----------------+------------------------+
                   |
                   v
  +----------------+------------------------+
  |    ML & BI Consumers                    |
  | - MLflow for training & deployment      |
  | - Serving via Model Registry + REST     |
  | - BI dashboards (PowerBI, Tableau)      |
  +-----------------------------------------+