Airflow Initial Setup UI - Pardhu448/clix_dashboard_backend_AF GitHub Wiki

Following are the steps involved in setting up airflow from UI:

1. Update 'start_date' in clix_dashboard_batchprocess.dag to current date on which airflow is first initialised and scheduled. Please note airflow follows UTC timezone and process is scheduled at 10pm UTC everyday
2. Open HostIP:Port on which airflow is hosted, in a browser
3. In the source code, open 'clix_variables.json' with the following details:
   • current_update_date_* : date on which this setup is done
   • prev_update_date_* : date since when syncthing data is available (With these variables set, all data uploaded till today since the start of syncthing is processed as soon as we switch on the dag)
4. Open 'Variables' in 'Admin' dropdown menu. Upload 'clix_variables.json' file which was created while setting up airflow docker and then click import
5. Go to 'DAGS' section, we will see all the data pipeline 'dags' listed here. Switch On the dags - clix_dashboard_barchprocess_dag and clix_static_visuals_dag

Now both data processing pipelines for school level visuals and static state level visuals are scheduled.