Apache Airflow is an open-source workflow orchestration platform for authoring, scheduling, and monitoring data pipelines as code. DAGs (Directed Acyclic Graphs) written in Python define your pipeline logic. Available as an open web service in Eyevinn Open Source Cloud, Airflow gives you a visual web UI for triggering and monitoring runs, a scheduler, and a metadata database to persist state.

• If you have not already done so, sign up for an Eyevinn OSC account

Airflow requires a metadata database. Create a PostgreSQL instance first by navigating to the PostgreSQL service in the OSC web console. Click Create psql-db and enter a name and password.

Once running, note the IP and port from the instance card. The connection string is:

postgres://postgres:<password>@<IP>:<PORT>/postgres

It is good practice to store credentials as a service secret rather than entering them in plain text. Navigate to the Airflow service page, open the Service Secrets tab, and click New Secret. Name the secret dburl and paste in the full PostgreSQL connection string.

Go to the My airflows tab on the Airflow service page and click Create airflow.

Click Create. Wait for the instance status to turn green.

Click the instance card to open the Airflow web UI. Log in with:

• Username: admin
• Password: the AdminPassword you set in Step 3

Change the password after first login under Security → Users.

Deploy DAGs by placing Python files in the dags/ directory of a repository and pointing the agent at it. Or use the Airflow REST API to trigger runs programmatically:

curl -X POST \
  -H "Content-Type: application/json" \
  -u admin:<password> \
  https://<instance>.apache-airflow.auto.prod.osaas.io/api/v1/dags/<dag_id>/dagRuns \
  -d '{"conf": {}}'

osc create apache-airflow myairflow \
  -o AdminPassword="mysecretpassword" \
  -o DatabaseUrl="{{secrets.dburl}}"

• Apache Airflow documentation
• GitHub repository
• Airflow service on OSC