Flocker volumes for the Portable Multi Node Kubernetes cluster (deprecated) - eddytruyen/kube-deploy GitHub Wiki

This fork of the docker-multinode project contains a small extension that integrates Kubernetes with Flocker. The extension ensures that the necessary Flocker environment variables and security credentials are set in the hyperkube containers of Kubernetes.

Prerequisites

Linux machines with Docker 1.11.0 or higher

A working installation of Flocker on every Kubernetes node is required. The flocker control service is typically installed on the Kubernetes master node.

Installing Flocker

https://github.com/ScatterHQ/flocker

Configuring Flocker

There was some ambiguity in the Section Configuring cluster authentication. The idea is that you generate a node certificate and associated private key for each node to which volumes should be attached. You have to generate these node certificates from the node where the flockercli package is installed (typically the control service node of Flocker), and in the directory where the cluster.key file is stored. Then scp a pair of node certificate and private key to each node under directory /etc/flocker and rename these files to node.crt and node.key. Also scp the cluster.crt file to each node under /etc/flocker.

Integration with Flocker in OpenStack to manager Cinder volumes

Flocker depends on an underlying dataset manager for attaching volumes. We have tested Flocker with the Cinder volume service in a private OpenStack cloud.

The clue is that you specify the right configuration for this dataset manager in an agent.yml file in the /etc/flocker/ directory.

The configuration syntax depends on the underlying dataset manager. For Openstack: see agent.yml file.

Installing Flocker client

Install the flockerctl command at the control-service node.

To run flockerctl, you need an APIUser certificate.. You also need to set 3 environment variables:

export FLOCKER_CERTS_PATH=/etc/flocker
export FLOCKER_USER=kubernetes
export FLOCKER_CONTROL_SERVICE=172.17.13.43

Integrating Flocker and docker-multinode

On every Kubernetes node where Flocker volumes will be attached, one or more environment variables and files must be created in order to mutually authenticate Kubernetes and Flocker against each other: Kubernetes as a user of the Flocker cluster and the Flocker cluster as a Flocker volume manager to Kubernetes. The environment variables can be set in the shell session where you will run the master or worker scripts of Kubernetes:

  • Three optional environment variables can be specified:

    • Optional: FLOCKER_USER_CA_DIR should refer to the directory where the necessary keys and certificates for authenticating kubernetes and flocker are stored. This defaults to /etc/flocker. If you don't want that sensitve information, stored in the guest host's /etc/flocker directory, are passed to the hyperkube container, you better use another directory.
    • Optional: the FLOCKER_CONTROL_SERVICE_PORT defaults to 4523, but if the Flocker control service listens on another port you must specify this.
    • Optional: The FLOCKER_CONTROL_SERVICE_HOST defaults to the ${MASTER_IP}

  • In the FLOCKER_USER_CA_DIR directory you need to store three files

    • the cluster.crt file. This file is the certificate of the Flocker cluster.
    • the kubernetes.key and kubernetes.crt files. These files are the api client key and certificate that Kubernetes uses to talk to the Flocker control service.

    If you have given other names to these files, then you have to specify these names in other environment variables:

    • FLOCKER_CONTROL_SERVICE_CA_FILE should refer to the full path to the cluster certificate file
    • FLOCKER_CONTROL_SERVICE_CLIENT_KEY_FILE should refer to the full path to the api key file for the API user
    • FLOCKER_CONTROL_SERVICE_CLIENT_CERT_FILE should refer to the full path to the api certificate file for the API user

Final Tips

You have to specify a name when creating a dataset. Otherwise the kubernetes agent will not be able to find the dataset.

To create a dataset with a name you have to specify the following command

flockerctl create -m name=my-volume -s 50G -n 265a0498

Flocker will then complain to Kubernetes that it can't find the dataset by its datasetID, but this warning is skipped by Kubernetes. Next the Pod will start its containers and the volume is linked with a subdirectory of /flocker directory.

Curl example for the REST API of flocker

curl -XGET --cacert /etc/flocker/cluster.crt --cert /etc/flocker/kubernetes.crt --key /etc/flocker/kubernetes.key https://172.17.13.43:4523/v1/configuration/datasets
curl -XDELETE --cacert /etc/flocker/cluster.crt --cert /etc/flocker/kubernetes.crt --key /etc/flocker/kubernetes.key https://172.17.13.43:4523/v1/configuration/datasets/cb681701-01f6-4be0-9e08-ab12e694f915