Etcd - kamialie/knowledge_corner GitHub Wiki
etcd uses RAFT consensus algorithm to agree what the current state is - each node's state is either what the majority of nodes agree it is or one of previously agreed states. This algorithm requires a majority (or quorum) for the cluster to progress to the next state. Disconnected node may catch up, if it loses a connection with a cluster.
etcd is usually deployed with an odd number of instances. This is because odd number still ensures a majority is left to allow a cluster to continue working. For example, both 3 and 4 instances configurations allow single node failure. 3 - 2 nodes are left and quorum is present. 4 - 3 nodes left are able to reach quorum; if 2 nodes fail, the remaining 2 do not reach quorum. To allow 2 node failure use 5 node configuration, and so on.
# data directory of etcd daemon
$ sudo grep data-dir /etc/kubernetes/manifests/etcd.yaml/etc/kubernetes/pki/etcd - directory containing TLS keys and certificates;
needed for TLS connection with etcdctl.
$ cd /etc/kubenetes/pki/etcd
$ echo * # since etcd image could be minimized and don't have ls, find, etc# kubernetes connection
$ kubectl -n kube-system exec -it <etcd_container_name> -- sh \
-c "ETCDCTL_API=3 \ # Version to use
ETCDCTL_CACERT=/etc/kubernetes/pki/etcd/ca.crt \ # Pass the certificate authority
ETCDCTL_CERT=/etc/kubernetes/pki/etcd/server.crt \ # Pass the peer cert and key
ETCDCTL_KEY=/etc/kubernetes/pki/etcd/server.key \
etcdctl <command>" # The command to test the endpoint# database health
$ etcdctl endpoint health
# list databases part of cluster
# production usually 3 or 5
$ etcdctl member list
$ etcdctl member list -w table
# back up database
# /var/lib/etcd is available from the node as well
$ etcdctl snapshot save /var/lib/etcd/snapshot.db
# on control plane node
$ mkdir $HOME/backup
$ sudo cp /var/lib/etcd/snapshot.db $HOME/backup/snapshot.db-$(date +%m-%d-%y)
$ sudo cp /root/kubeadm-config.yaml $HOME/backup/
$ sudo cp -r /etc/kubernetes/pki/etcd $HOME/backup/