Contrail ConfigDB DR(Backup Restore) Procedure - ganeshahv/Contrail_SRE GitHub Wiki
NOTE: The restore procedure is meant for debugging purposes only and not configDB restore in a production environment.
Backup procedure
1]. On the first controller, create a backup directory.
mkdir /tmp/db-dump
2]. Get inside the config_api docker.
docker exec -it config_api_1 bash
3]. Use the exim utility to export the db-dump.
python /usr/lib/python2.7/site-packages/cfgm_common/db_json_exim.py --export-to db-dump.json
4]. Exit the docker and copy the contrail_api conf file and the db-dump to the backup directory on the node.
docker cp config_api_1:/etc/contrail/contrail-api.conf /tmp/db-dump/
docker cp config_api_1:/db-dump.json /tmp/db-dump/
Recovery procedure
1]. On the controller node, stop all the contrail services which register with Zookeeper.
for i in `` `docker ps -a | grep config | awk '{print $1}'` ``; do docker stop "$i"; done
for i in `` `docker ps -a | grep analytics | awk '{print }'` ``; do docker stop "$i"; done
for i in `` `docker ps -a | grep database | awk '{print }'` ``; do docker stop "$i"; done
for i in `` `docker ps -a | grep vcente | awk '{print }'` ``; do docker stop "$i"; done
for i in `` `docker ps -a | grep control | awk '{print }'` ``; do docker stop "$i"; done
for i in `` `docker ps -a | grep webui | awk '{print }'` ``; do docker stop "$i"; done
for i in `` `docker ps -a | grep redis | awk '{print }'` ``; do docker stop "$i"; done
2]. Backup the Zookeeper and Cassandra datastore and manually delete the data files.
cp -R /var/lib/docker/volumes/config_database_config_zookeeper/_data/version-2 /var/lib/docker/volumes/config_database_config_zookeeper/_data/version-2-slave
rm -f /var/lib/docker/volumes/config_database_config_zookeeper/_data/version-2/*
cp -R /var/lib/docker/volumes/config_database_config_cassandra/_data /var/lib/docker/volumes/config_database_config_cassandra/_data-slave
rm -rf /var/lib/docker/volumes/config_database_config_cassandra/_data/*
3]. Start only the config cassandra and zookeeper services.
for i in `` `docker ps -a | grep config | grep cassa  | awk '{print $1}'` ``; do docker start "$i"; done
for i in `` `docker ps -a | grep config | grep zoo  | awk '{print $1}'` ``; do docker start "$i"; done
4]. Start a new config_api docker and copy the saved config and db-dump into it.
docker image ls | grep config-api
docker run --rm -it -v /tmp/db-dump/:/tmp/ --network host --entrypoint=/bin/bash REPOSITORY:TAG
5]. Restore the config and db-dump.
cd /usr/lib/python2.7/site-packages/cfgm_common
python db_json_exim.py --import-from /tmp/db-dump.json --api-conf /tmp/contrail-api.conf
6]. Exit the docker and on the controller node, start all the contrail services which were stopped earlier.
for i in `` `docker ps -a | grep config | awk '{print $1}'` ``; do docker start "$i"; done
for i in `` `docker ps -a | grep analytics | awk '{print $1}'` ``; do docker start "$i"; done
for i in `` `docker ps -a | grep database | awk '{print $1}'` ``; do docker start "$i"; done
for i in `` `docker ps -a | grep vcente | awk '{print $1}'` ``; do docker start "$i"; done
for i in `` `docker ps -a | grep control | awk '{print $1}'` ``; do docker start "$i"; done
for i in `` `docker ps -a | grep webui | awk '{print $1}'` ``; do docker start "$i"; done
for i in `` `docker ps -a | grep redis | awk '{print $1}'` ``; do docker start "$i"; done