Set up Microk8s Kubernetes cluster and Rancher on NREC with Ansible - uib-ub/uib-ub-monorepo GitHub Wiki

Set up Microk8s Kubernetes cluster and deploy Rancher on NREC with Ansible

In microk8s-NREC-deployment repo, an Ansible role "microk8s" is implemented to set up Microk8s Kubernetes cluster on 3 NREC instance nodes.

"microk8s" role

In microk8s/defaults/main.yml, the user can configure the version of the Microk8s and the domain for Rancher, eg:

microk8s_version: 1.28

rancher_domain: rancher-test.testdu.uib.no  # This is just an example domain.

In microk8s/task/main.yml, the user can see the main structure of the role with all runtime variables defined to run the role, including install_snapd for installation of snap, configure_firewall_ports for setting up ports for Microk8s, etc.:

- include_tasks: install-snapd.yml
  when: install_snapd | default (false) | bool

- include_tasks: setup-firewall-ports-microk8s.yml
  when: configure_firewall_ports | default (false) | bool

- include_tasks: install-microk8s.yml
  when: install_microk8s | default (false) | bool

- include_tasks: setup-cluster.yml
  when: create_microk8s_cluster | default (false) | bool

- include_tasks: enable-addons.yml
  when: enable_addons | default (false) | bool

- include_tasks: config-cluster-remote-access.yml
  when: config_cluster_remote_access | default (false) | bool

- include_tasks: setup-rancher.yml
  when:
    - setup_rancher | default (false) | bool
    - ansible_default_ipv4.address == hostvars[play_hosts[0]].ansible_host

- include_tasks: install-metricbeat.yml
  when:
    - install_metricbeat | default (false) | bool
    - ansible_default_ipv4.address == hostvars[play_hosts[0]].ansible_host

- include_tasks: install-filebeat.yml
  when:
    - install_filebeat | default (false) | bool
    - ansible_default_ipv4.address == hostvars[play_hosts[0]].ansible_host

- include_tasks: connect-microceph.yml
  when: connect_microceph | default (false) | bool

Ansible inventory

Here is an example of Ansible inventory file to define a group of instances to set up Microk8s cluster, and this group is defined in ./inventory/hostfile

[hono_api_prod]
hono-api-prod-01 ansible_host=xxx.xxx.xxx.xxx ansible_user=rocky
hono-api-prod-02 ansible_host=xxx.xxx.xxx.xxx ansible_user=rocky
hono-api-prod-03 ansible_host=xxx.xxx.xxx.xxx ansible_user=rocky

Playbook `setup-microk8s.yml`

The user of the role "microk8s" can create a playbook, eg. setup-microk8s.yml having:

- name: "Playbook for setting up microk8s and rancher"
  hosts: "hono_api_{{ group }}"

  tasks:
    - import_role:
        name: "microk8s"

Run playbook and Microk8s role

We suggest running the playbook with the runtime options step by step, eg. (note: the vault pass (file) "api" used to run the ansible playbook is stored as a repository variable (ANSIBLE_API_VAULTPASS) in Github Settings of microk8s-NREC-deployment repo):

install snap:

ansible-playbook setup-microk8s.yml --vault-id ../secrets/api -e group=prod -i inventory/hostfile -e install_snapd=true

configure firewall ports for Microk8s Kubernetes:

ansible-playbook setup-microk8s.yml --vault-id ../secrets/api -e group=prod -i inventory/hostfile -e configure_firewall_ports=true

install Microk8s on NREC instances

ansible-playbook setup-microk8s.yml --vault-id ../secrets/api -e group=prod -i inventory/hostfile -e install_microk8s=true

create Microk8s Kubernetes cluster:

ansible-playbook setup-microk8s.yml --vault-id ../secrets/api -e group=prod -i inventory/hostfile -e create_microk8s_cluster=true

enable Microk8s addons: cert-manager and ingress:

Although we can run the playbook to enable these addons, sometimes, for the cert-manager addon, it takes longer to finish or might have errors. So, if you want to use the playbook, then run :

ansible-playbook setup-microk8s.yml --vault-id ../secrets/api -e group=prod -i inventory/hostfile -e enable_addons=true

or, we suggest logging in to one of the instances by ssh, and manually enabling addons:

microk8s enable cert-manager and microk8s enable ingress,

and then run microk8s status to check the status and addons.

configure remote access to Microk8s Kubernetes cluster

This step is used to generate a configuration file: ~/.kube/config on each NREC instance node, so that we can use it for remote access to the cluster, such as Github actions to access the cluster.

ansible-playbook setup-microk8s.yml --vault-id ../secrets/api -e group=prod -i inventory/hostfile -e config_cluster_remote_access=true

Deploy Rancher with Microk8s Kubernetes cluster

After finishing all 6 steps above, we can deploy Rancher (with the default username admin) to Microk8s Kubernetes cluster:

ansible-playbook setup-microk8s.yml --vault-id ../secrets/api -e group=prod -i inventory/hostfile -e setup_rancher=true

Deploy Metricbeat on Microk8s Kubernetes cluster

Run: microk8s enable metrics-server on one of the cluster nodes
Then run:

ansible-playbook setup-microk8s.yml --vault-id ../secrets/api -e group=prod -i inventory/hostfile -e install_metricbeat=true

Deploy filebeat on Microk8s Kubernetes cluster

Run:

ansible-playbook setup-microk8s.yml --vault-id ../secrets/api -e group=prod -i inventory/hostfile -e install_filebeat=true

Connect existing MicroCeph cluster