Ubuntu 18.04 Kubeflow 설치 및 예제 - jinwooklim/my-exp GitHub Wiki

참조

1. https://lsjsj92.tistory.com/580?category=891065

2. https://data-newbie.tistory.com/481

3. https://www.kubeflow.org/docs/pipelines/sdk/install-sdk/

Ubuntu 18.04 minikube 설치

1. sudo apt-get update && sudo apt-get dist-upgrade && sudo apt-get autoremove
2. sudo apt-get -y install gcc make linux-headers-$(uname -r) dkms
3. sudo apt-get update && sudo apt-get install -y apt-transport-https
4. curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
5. sudo touch /etc/apt/sources.list.d/kubernetes.list
6. echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
7. sudo apt-get update
8. sudo apt-get install -y kubectl
9. kubectl version kubectl cluster-info curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/
10. minikube start
11. minikube version # for check
12. minikube status

Kubeflow 설치

1. minikube start --cpus 4 --memory 4096 # minikube 시작
2. cd ~/Downloads
3. wget https://github.com/kubeflow/kfctl/releases/download/v1.0.1/kfctl_v1.0.1-0-gf3edb9b_linux.tar.gz tar -xvf kfctl_v1.0.1-0-gf3edb9b_linux.tar.gz
4. echo 'export PATH=$PATH:/home/USER/mlpipe' >> ~/.bashrc
5. echo 'export KF_NAME="USER_kubeflow"' >> ~/.bashrc
6. echo 'export BASE_DIR=/home/USER/mlpipe/kubeflow' >> ~/.bashrc
7. echo 'export KF_DIR=${BASE_DIR}/${KF_NAME}' >> ~/.bashrc
8. echo 'export CONFIG_FILE=${KF_DIR}/kfctl_k8s_istio.v1.0.1.yaml' >> ~/.bashrc (아래 build 때 생성되는 yaml 파일과 일치되게) , (현재는 1.0.1 버전 바뀌면 수정해줘야함)
9. echo 'export CONFIG_URI="https://raw.githubusercontent.com/kubeflow/manifests/v1.0-branch/kfdef/kfctl_k8s_istio.v1.0.1.yaml"' >> ~/.bashrc
10. mkdir -p ${KF_DIR}
11. cd ${KF_DIR}
12. mv ~/Downloads/kfctl ${BASE_DIR}
13. kfctl build -V -f ${CONFIG_URI} # 어느 디렉토리에서 kubeflow를 설치할 것인지 지정
14. kfctl apply -V -f ${CONFIG_FILE}# kubeflow 설치
15. kubectl -n kubeflow get all # Status가 ConatainerCreating 에서 모두 Running이 될 때 까지 기다리면 됨
16. export NAMESPACE=istio-system
17. kubectl port-forward -n ${NAMESPACE} svc/istio-ingressgateway 8080:80 # 쿠버네티스 환경에 접속하기 위한 네트워크 환경 설정
18. minikube stop # minikube 정지

Miniconda 환경에서의 Kubeflow Pipelines SDK 설치

1. pip install kfp --upgrade --user
   

2. echo 'export PATH=$PATH:~/.local/bin' >> ~/.bashrc
   

3. which dsl-compile
   >> /<PATH_TO_YOUR_USER_BIN>/miniconda3/envs/mlpipeline/bin/dsl-compile