Data pipeline with Apache Airflow - up1/training-courses GitHub Wiki

Course outline :: Data pipeline with Apache Airflow

Software requirements

  • Docker
  • Python 3+

Topics

  • Basic of Apache Airflow
    • Overview of Apache Airflow
    • Key Features and Benefits
    • Installation and Setup
      • Docker
  • Building data pipeline with Apache Airflow
    • Core concepts
      • DAGs(Directed Acyclic Graph), Tasks, Operators, Sensors, Executors and Hooks
    • Design your data pipeline
    • Data processing tasks
    • Manage dependencies and scheduling
    • Error Handling and Retries
    • Monitoring and Logging
  • Working with Datasources (ETL => Extract, Transform, Loader)
    • Connecting to Datasources
    • Data loading techniques
    • Data extraction
    • Data transformation
    • Data quality
  • Workshops