Upgrading your LA portal - AtlasOfLivingAustralia/documentation GitHub Wiki

Strategies to upgrade a LA portal

In general there are two main strategies to migrate your LA services and servers:

To install new servers and services from scratch using new versions of everything (like the OS), redeploy with ansible & ala-install and restore dbs and /data from backups or rsync or similar. In general the philosophy of devops (and ansible and ala-install) is to not touch servers manually, so if you have a good LA inventory (these generated by the la-toolkit, for instance) and you didn't install or configured nothing manually, it's quite easy to reproduce and restore servers and services or upgrade them. You'll need extra servers, and it's quite safe because you don't switch to production the new servers until you are sure that are working. For this you can play with your hosts file to verify the upgrade of the services and later to switch to use them in production via a real DNS change.
To upgrade using the same servers. This is more risky, but sometimes easy, when you are doing small version upgrades. You can also do OS upgrades that way, but the same, it's more risky as you are upgrading production servers.

In general with option 1) you play with /etc/hosts, or DNS and/or proxies until you are sure that the updated service is working and you change the DNS/proxies to put that new servers in production.

In which order we need to upgrade a LA Portal?

We don't have an unique order to upgrade modules, but there are some LA modules that are more easy to upgrade than others.

So you can try to upgrade LA services that do not have a db backend theyself, like regions or bioacache-hub, bie-hub, biocache-service... These are more safe to migrate or test and to downgrade if you detect some issue.

Also sometimes you have to migrate some services to maintain compatibilities and dependencies.

Keeping your /data/ configs well configured

If you managed your /data/*/config manually for some reason instead of using ansible and you want to use ala-install instead, you have to create a correct ansible inventory with all these manual /data/ configurations translated into the corresponding ansible variables. The la-toolkit can help you to get an initial version of your inventories, update your config and see the differences. For this is good to keep track of you configurations. We tried to describe this (and other related recommendations) here.

Database Backup and Restore

Here are instructions for backing up (dumping) and restoring databases from MySQL, PostgreSQL, and MongoDB. Typically you dump the db in the old server and restore the dump in a new machine. If the service is using tomcat, it's better to have it stopped during the restore.

MySQL:

1. Backup (Dump):

Open your terminal and type the following command:

mysqldump -u [username] -p [database_name] > [filename].sql

Replace [username], [database_name], and [filename] with your MySQL username, the name of the database you want to backup, and the desired filename for your backup file, respectively. You will be asked to enter your password.

2. Restore:

To restore the backup to another MySQL server, copy the [filemane].sql to the new machine and use the following command:

mysql -u [username] -p [new_database_name] < [filename].sql

Replace [username], [new_database_name], and [filename] with your MySQL username, the name of the new database you want to restore to, and the filename of your backup file, respectively. You will be asked to enter your password.

PostgreSQL:

1. Backup (Dump):

The same with PostgreSQL, open your terminal and type the following command:

pg_dump -U [username] -W -F t [database_name] > [filename].tar

Replace [username], [database_name], and [filename] with your PostgreSQL username, the name of the database you want to backup, and the desired filename for your backup file, respectively. You will be asked to enter your password.

2. Restore:

To restore the backup to another PostgreSQL server, copy the [filename].tar and use the following command:

pg_restore -U [username] -d [new_database_name] -F t [filename].tar

Replace [username], [new_database_name], and [filename] with your PostgreSQL username, the name of the new database you want to restore to, and the filename of your backup file, respectively. You will be asked to enter your password.

MongoDB:

1. Backup (Dump):

Open your terminal and type the following command:

mongodump --db [database_name] --out [directory_path]

Replace [database_name] and [directory_path] with the name of the database you want to backup and the desired directory path for your backup data.

2. Restore:

To restore the backup to another MongoDB server, use the following command:

mongorestore --db [new_database_name] [directory_path]

Replace [new_database_name] and [directory_path] with the name of the new database you want to restore to and the directory path of your backup data.

Apache Solr:

1. Backup (Dump):

In Solr, you can create a backup of a core or a collection (if you're using SolrCloud) using the BACKUP command via the Collections API or Core Admin API.

For a single core:

curl http://[solr_server]:[solr_port]/solr/[core_name]/replication?command=BACKUP&location=[backup_directory]&name=[backup_name]

For SolrCloud:

curl "http://[solr_server]:[solr_port]/solr/admin/collections?action=BACKUP&name=[backup_name]&collection=[collection_name]&location=[backup_directory]"

Replace [solr_server], [solr_port], [core_name], [collection_name], [backup_directory], and [backup_name] with your Solr server's address, port number, the name of the core/collection you want to backup, the directory where you want the backup to be stored, and a name for your backup, respectively.

2. Restore:

You can restore a backup using the RESTORE command via the Collections API or Core Admin API.

For a single core:

curl http://[solr_server]:[solr_port]/solr/[core_name]/replication?command=RESTORE&location=[backup_directory]&name=[backup_name]

For SolrCloud:

curl "http://[solr_server]:[solr_port]/solr/admin/collections?action=RESTORE&name=[backup_name]&collection=[collection_name]&location=[backup_directory]"

Replace [solr_server], [solr_port], [core_name], [collection_name], [backup_directory], and [backup_name] with your Solr server's address, port number, the name of the core/collection you want to restore, the directory where your backup is stored, and the name of your backup, respectively.

Apache Cassandra:

1. Backup (Dump):

Cassandra does not have a built-in backup tool like mysqldump or pg_dump. Instead, you need to manually copy the data files (SSTables) stored in the data directory.

cp -R /var/lib/cassandra/data/[keyspace]/[table]-* /backup/location

Replace [keyspace], [table], and /backup/location with the keyspace of the database you want to backup, the name of the table you want to backup, and the directory where you want to store the backup, respectively.

2. Restore:

To restore the backup, simply copy the backed up data files back to the data directory.

cp -R /backup/location/[table]-* /var/lib/cassandra/data/[keyspace]

Replace /backup/location, [table], and [keyspace] with the directory where your backup is stored, the name of the table you want to restore, and the keyspace where you want to restore the table, respectively.

Remember, you must stop Cassandra before restoring the backup, and clean up the commit log and saved caches directory afterwards.

Casandra schemas migration

From time to time our cassandra schema changes. We can use ALTER TABLE statements to update it, however the biocache-store detects missing fields and adds them automatically.

Migration notes

And remember

Backup, backup, backup