Automated pipelines creation - anodot/daria GitHub Wiki

agent apply command automatically creates or updates sources and pipelines using .json files that contain pipeline and source configs.

In order to use this script you need to:

  1. Create the main directory (with any name) and sources and pipelines subdirectories anywhere on your system. sources and pipelines directories can have any number of subdirectories, the script will recursively walk through all of them and try to create sources and pipelines using configs found in there.
  2. Put your .json files with source and pipeline configs to the corresponding directories
  3. Copy this directory to the container
  4. Run apply command inside the container

After doing these steps use the command and specify the directory inside the container where you put the files

Example for docket

mkdir agent-data
mkdir agent-data/sources
mkdir agent-data/pipelines
...
docker cp ./agent-data anodot-agent:/usr/scr/app/agent-data
docker exec anodot-agent agent apply -d /usr/scr/app/agent-data

Example for kubernetes

mkdir agent-data
mkdir agent-data/sources
mkdir agent-data/pipelines
...
POD=$(kubectl get pod -l app.kubernetes.io/name=anodot-agent -o jsonpath="{.items[0].metadata.name}")

DIR=/usr/src/app/agent-data
kubectl exec -it $POD -- rm -rf $DIR
kubectl cp ./agent-data $POD:$DIR
kubectl exec -it $POD -- agent apply -d $DIR