Ingesting SAP Data into the Bronze Layer 🔁 Objective: - RogerThattt/SAP-Retail-Data-Databricks GitHub Wiki

🔁 Objective: Capture SAP source data (POS and Product Master) into raw Delta format, with minimal transformation — preserving fidelity and lineage.

⚙️ Mechanism: Delta Sharing or Custom SAP Extract SAP → Raw Data Extraction:

Use ODP extractors or SAP Data Services to extract POS & master data tables.

Export as Parquet/CSV files or push directly via Delta Sharing REST API.

Delta Sharing Setup:

Establish Delta Sharing Provider in SAP-connected system (cloud bucket).

On Databricks, configure CREATE SHARE, then CREATE RECIPIENT via Unity Catalog.

Bronze Layer Table Creation:

sql Copy Edit CREATE TABLE sap_pos_transactions_bronze USING DELTA LOCATION '/mnt/bronze/sap/pos_transactions';

CREATE TABLE sap_product_master_bronze USING DELTA LOCATION '/mnt/bronze/sap/product_master'; Schema Design:

Maintain loosely typed columns (e.g., strings for SAP codes).

Preserve load timestamps, SAP record IDs, and source system tags.