$ wget https://repo.anaconda.com/archive/Anaconda3-2022.05-Linux-x86_64.sh

$ sudo apt update

$ sudo apt install docker.io

4.1 Install docker-compose in a separate directory

$ mkdir soft
$ cd soft

4.2 To get the latest release of Docker Compose, go to https://github.com/docker/compose and download the release for your OS.

$ wget https://github.com/docker/compose/releases/download/v2.5.0/docker-compose-linux-x86_64 -O docker-compose

4.3 Make it executable

$ chmod +x docker-compose

4.4 Add to path

$ nano .bashrc
# In .bashrc, add:
export PATH="${HOME}/soft:${PATH}"
# Run
$ source .bashrc

$ sudo groupadd docker
$ sudo usermod -aG docker $USER

$ docker run hello-world

Guide to using GCP service

1. Create a new Gmail account for $300 free credit on GCP
2. Create an account on GCP and setup a billing

Name: mlops-demo

Configuration:

• Name: demo-vm
• Region: us-east1
• Zone: us-east-b
• Series: N2
• Machine Type: n2-standard-2
• Boot disk:
  • Image: Ubuntu 22.04 LTS
• Firewall: Check both
  • Allow HTTP traffic
  • Allow HTTPS traffic

View Network details > IP addresses > Reserve a static address

Generate a SSH key
Below for Mac OS Terminal / Linux OS,
Windows users please update commands as per need.

ssh-keygen -t rsa -f ~/.ssh/mlops-demo -C mlops_zoomcamp

Copy the content of mlops-demo.pub

cat ~/.ssh/mlops-demo.pub

On GCP console, enter the copied public key

Setting > Metadata > SSH Key

Replace your instance public IP address

ssh -i ~/.ssh/mlops-demo [email protected]

Optional, Add host to hostname file for easy login

Enter the following into ~/.ssh/config

nano ~/.ssh/config

Host gcp-mlops_demo
    HostName 35.196.19.116 # VM Public IP
    User mlops_zoomcamp # VM user
    IdentityFile ~/.ssh/mlops-demo # Private SSH key file
    StrictHostKeyChecking no

Further Use

ssh gcp-mlops_demo

• Anaconda

cd ~
wget https://repo.anaconda.com/archive/Anaconda3-2022.05-Linux-x86_64.sh
bash Anaconda3-2022.05-Linux-x86_64.sh

• Docker

sudo apt install docker.io

• Docker-Compose

sudo apt install docker-compose

• Add current user to docker group

sudo usermod -aG docker mlops_zoomcamp

• Verify Installation

which python
# /home/mlops_zoomcamp/anaconda3/bin/python

which docker
# /usr/bin/docker

which docker-compose
# /usr/bin/docker-compose

docker run hello-world

• Run Jupyter Notebook

jupyter notebook

Copy the Password or Token

• Install Remote SSH extension
• Enable port forwarding 8888 to localhost 8888

Refer instructor video for detailed steps

Visit localhost:8888 to start using jupyter Past the copied Password or Token

Sign in to the Azure portal.

• Enter virtual machines in the search or click Virtual machines under Azure services.

Under Services, select Virtual machines.

• In the Virtual machines page, select Create and then Azure Virtual machine.

The Create a virtual machine page opens.

• In the Basics tab, under Project details, make sure the correct subscription is selected and then choose to Create new resource group. Enter <name-of-your-choice> for the name. mlops will be used in this tutorial

• Under Instance details, enter azurevm for the Virtual machine name, and choose Ubuntu 20.04 LTS - Gen2 for your Image. Leave the other defaults. The default size and pricing is only shown as an example. Size availability and pricing are dependent on your region and subscription.

  The Standard_E2s_v3 - 2 vcpus, 16 GiB memory will be used in this tutorial

• Set Authentication type to SSH public key

  Set Username, Generate new key pair and set key pair name.

• Set Inbound port rules Allow selected ports and select 'HTTP (80)' and 'SSH (22)'

Leave the remaining defaults and then select the Review + create button at the bottom of the page.

• It will take you to a service pricing webpage. Go through it. When you are done, select Create.

• When the Generate new key pair window opens, select Download private key and create resource. Your key file will be download as azurevmkey.pem. Make sure you know where the .pem file was downloaded; you will need the path to it in the next step.

• Afte VM deploys finish, click on Go to resource.

• Copy Public IP address

Create an SSH connection with the VM.

If you are on a Mac or Linux machine, open a Bash prompt. If you are on a Windows machine, you can use a PowerShell prompt but in this tutorial, we will use Git bash. Click this link

In your bash prompt, open an SSH connection to your virtual machine. Replace the IP address with the one from your VM, and replace the path to the .pem with the path to where the key file was downloaded.

ssh -i .\<pem-directory>\azurevmkey.pem [email protected]

For easy ssh login, do the following:

• Open a GNU nano editor by entering nano ~/.ssh/config in the bash prompt

• Setup SSH config file

  Host azure-mlops-zoomcamp
    HostName 20.106.112.123 # VM Public IP
    User azureuser # VM user
    IdentityFile C:\Users\User\.ssh\azurevmkey.pem # Private SSH key file
    StrictHostKeyChecking no
    ServerAliveInterval 60 # To prevent ssh from disconnecting (optional)
    ServerAliveCountMax 525600
  

Username gotten from 👇👇👇

Public IP gotten from 👇👇👇

Anytime you want to login to ssh instance again, type ssh azure-mlops-zoomcamp

• Download and install the Anaconda distribution of Python Ensure you are in home directory, if not cd $HOME

  wget https://repo.anaconda.com/archive/Anaconda3-2022.05-Linux-x86_64.sh
  bash Anaconda3-2022.05-Linux-x86_64.sh
  

• Restart ssh, if base is not in vm name like this 👇👇👇.
  (base) azureuser@azvm:~$

• Update existing packages using

  sudo apt update

• Install Docker using

  sudo apt install docker.io

• To run docker without sudo:

  sudo groupadd docker
  sudo usermod -aG docker $USER
  

• Install docker-compose in a separate directory

  mkdir soft
  cd soft
  

• Download Docker Compose wget https://github.com/docker/compose/releases/download/v2.5.0/docker-compose-linux-x86_64 -O docker-compose

  Make it executable

  chmod +x docker-compose

  Go back to HOME path cd $HOME Add to the soft directory to PATH. Open the .bashrc file with nano:

  nano .bashrc In .bashrc, add the following command to the last line:

  export PATH="${HOME}/soft:${PATH}"

  Save it and run the following to make sure the changes are applied:

  source .bashrc

which python
#/home/azureuser/anaconda3/bin/python

which docker
#/usr/bin/docker

which docker-compose
/home/azureuser/soft/docker-compose

docker run hello-world

If you get

docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: 
Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/create": dial unix /var/run/docker.sock: connect: permission denied. error

Restart your VM instance in azure's portal.

Jupyter notebook is installed with anaconda

To run Jupyter Notebook, run the following:

• Create Jupyter notebook Directory

cd notebooks
mkdir notebooks

• run Jupyter Notebook

jupyter notebook

Take note of the port after localhost(http://127.0.0.1:/

• Open Vscode
• Install Remote SSH extension
• Connecting to Remote SSH host

Go to the bottom left of your Vscode and click the icon that contains mlops-zoomcamp. You might not see any name in yours yet that's because this is your first time.

• After clicking, a prompt pops up, Click on Connect to host and select your ssh hostname

• A new Vscode window opens, Go to View in the menu bar and click on terminal or simply enter `ctrl + ``

• Under Terminal, go to PORTS AND forward port 8888

• Jupyter Homepage