Versatile Data Kit Control Service - vmware/versatile-data-kit GitHub Wiki
Versatile Data Kit includes Control Service server enabling deploying, managing and monitoring Data Jobs. You can see the REST API Swagger documentation here
Install locally
Local installation is used only for development and prototyping purposes or getting to know the Control Service. It's easier to setup and install but not very mature. You must use the Helm Chart for production deployment.
Prerequisites
Then run:
vdk server --install
This will install Control Service on your local machine and configure local installation of VDK to use it.
Install in Production
In production, use the helm chart to install and configure it.
Prerequisites
- Install helm
- Kubernetes
Then run
helm repo add vdk-gitlab https://gitlab.com/api/v4/projects/28814611/packages/helm/stable
helm install my-release vdk-gitlab/pipelines-control-service
For full production installation, it's recommended to follow this example
Read more about the installation in the Versatile Data Kit Control Service Chart Github repository
See list of all released versions of the helm chart here
Use
# see Help to see what you can do and start playing around
vdk --help
# for example create a new job:
vdk create -u <URI-YOU-GOT-FROM-helm-install-OUTPUT>
For more advanced deployment options with custom VDK SDK see here