SAKK Kubeflow use example - Evanto/qna GitHub Wiki

Kubeflow use example

Once you successfully logged into the EKS cluster via kubectl and accessed Kubeflow UI, you'll have to pass several configuration windows first. Configure your Namespace name to complete the setup:

After that, you'll see Kubeflow dashboard. It looks like this:

To access Kubeflow Pipelines in the UI, click Pipelines. Kubeflow offers a few samples to let you try pipelines quickly. Alternatively, you can upload your own pipelines using advanced features of AWS and Kubeflow.

To learn about using Kubeflow on AWS, please check the official Kubeflow documentation. You can start by trying a demo module with one of the built-in AWS SageMaker algorithms. To do so, create a folder for managing separate Terraform states with resources related to pipeline executions and add the main.tf file with the following content:

module kmeans_mnist {
  source = "path/to/kmeans-mnist-pipeline/folder/at/root/of/the/project"

  cluster_name = "<your-cluster-name>"
  username     = "<your-kubeflow-username>"
}

After that you need to execute Terraform:

terraform init
terraform apply

Terraform will generate a training_pipeline.yaml file. Upload it to Kubflow through UI:

Now that you have your first pipeline and prepared a Kubernetes service account that matches your Kubeflow username with the required permissions for AWS, please specify it on creating a run: