Upgrading 4CAT - digitalmethodsinitiative/4cat GitHub Wiki

If you have already installed 4CAT and want to upgrade to the latest release, you can use the following methods. If you are a system administrator managing a 4CAT server with many users or on a remote server, you may want to also read the additional notes.

1. Log in with an admin account
2. Navigate to the 'Control Panel' via the navigation bar at the top of the interface
3. Click the 'Restart or Upgrade' button at the top
4. If a new version is available, click the 'Upgrade' button. Make sure to carefully read the instructions on the page first.

If upgrading via the web interface fails, and you're using Docker Desktop to run 4CAT, the following alternative method can be used to upgrade to the latest 4CAT release unless otherwise noted in the release notes:

🎥 View a video screen cast of the steps below

1. In the Docker Desktop sidebar, click 'Images' to open the list of downloaded Docker images
2. Look for the 'digitalmethodsinitiative/4cat' image in the list. Click the three dots under 'actions' and choose 'Pull' to download the latest version of the image. Once you see the 'Status: Image is up to date for digitalmethodsinitiative/4cat:stable' notice pop up, the image has been upgraded to the latest version.
   

   

   
   
3. Open a terminal and navigate to the location of your 'docker-compose.yml' file. If you're unsure where this is, go to the 'Containers' tab in Docker Desktop and click the '4cat' container name. The file path is noted at the top of the details screen that opens.
   

   

   
   For example, to navigate to the path listed above, use the following terminal command:

   cd /Users/stijn/PycharmProjects/4cat
   

4. Run the following command:

   docker compose down && docker compose up --detach
   

5. Wait until the command completes. This can take a while, because it may need to download the updated 4CAT from the internet. Once it completes, 4CAT will have restarted with the latest version of the software; you can close the terminal window and resume using 4CAT via the web interface.

If you are running 4CAT via Docker but not using Docker Desktop, or if you want to migrate to a specific version of 4CAT, or in case of substantial changes that require a full re-build of 4CAT, use the following method:

🎥 View a video screen cast of the steps below, with MacOS and Docker Desktop

1. If you use Docker Desktop, make sure that it is running.

2. Find the folder from which you originally built 4CAT. If you don't remember where this is, and you are using Docker Desktop, you can find the path to the folder by clicking the 'Containers' tab in the side bar and then the '4cat' stack. You can then find the path at the top of the screen (🎥 video). If you cannot find the path, create an empty folder somewhere to work in.

3. If files named docker-compose.yml or .env are in this folder, rename them so they will not be overwritten in the next steps. If .env seems missing but docker-compose.yml is present, it may be hidden; see here.

4. Download the docker-compose.yml and the .env files associated with your desired version and save them to the folder.

   • You can find these in the .zip file you download via the releases page or get the very latest versions of .env and docker-compose.yml. Right-click those links and choose 'Save link as...' to download. Your browser might suggest to save the .env file as Untitled, env.env, env.txt, or similar: in that case, manually adjust the filename back to .env. Double-check that the file has the correct name afterwards and if not, change it to .env.

   • Optionally, edit the .env file as necessary (compare with settings from your old .env file). This is usually unnecessary if you did not edit it when you first installed 4CAT. In particular, if in your old file there is a line like POSTGRES_TAG=[a number] near the top, make sure that line is carried over to the new file. If the line says POSTGRES_TAG=latest, you do not need to copy it.

5. Open a terminal and navigate to the folder in which the files you downloaded in step 1 are located. For example, type the following command, then press Enter (adjust the path to the folder as necessary):

   cd /Users/stijn/Documents/4cat
   

6. Run the following command. This will not delete your data or settings, which are stored separately. Docker will download the 4CAT version designated in the .env file as DOCKER_TAG; for example, DOCKER_TAG=stable will install the latest released stable 4CAT version. This might take a while!

   docker compose down && docker compose pull && docker compose up --detach
   

   You should see something like:

   % docker compose down && docker compose pull && docker compose up --detach
   [+] Running 5/5
    ✔ Container 4cat_frontend   Removed              1.2s 
    ✔ Container 4cat_backend    Removed              0.6s 
    ✔ Container 4cat_memcached  Removed              0.0s 
    ✔ Container 4cat_db         Removed              0.1s 
    ✔ Network 4cat_default      Removed              0.2s
   
   [+] Pulling 19/19
    ✔ frontend Skipped - Image is already being pulled by backend                 0.0s 
    ✔ memcached Pulled                                                            3.2s 
    ✔ db Pulled                                                                   3.3s 
    ✔ backend Pulled                                                              153.3s 
      ✔ b62d650e32c8 Pull complete                                                0.9s 
      ✔ 9c627570eae0 Pull complete                                                0.6s
      [...] 
      ✔ 4f3f963c14fc Pull complete                                                1.4s 
   
   [+] Running 5/5
    ✔ Network 4cat_default      Created              0.0s 
    ✔ Container 4cat_db         Healthy              5.9s 
    ✔ Container 4cat_memcached  Healthy              10.9s 
    ✔ Container 4cat_backend    Started              11.0s 
    ✔ Container 4cat_frontend   Started              11.1s 
   

7. Wait until the command completes. Once it completes, 4CAT will have restarted with the latest version of the software; you can close the terminal window and resume using 4CAT via the web interface.

If afterwards 4CAT refuses to start, you may need to rebuild with a specific database version. See these instructions.

See Additional Notes for more information and troubleshooting.

1. Open a terminal and navigate to the folder 4CAT is running from
2. (Optionally) activate the virtual environment
3. Run python3 helper-scripts/migrate.py --release --restart and follow the instructions

Optionally, you can run the latest development version (i.e. the master branch of this repository) by running python3 helper-scripts/migrate.py --branch master --restart. Note that this is UNSUPPORTED and that this branch may contain more bugs than the regular releases.

You can also update 4CAT to test new features or take advantages of bug fixes prior to a full release. Our master branch is always our latest development branch where we push any fixes.

To upgrade to a branch, you must enable this feature in the Control Panel -> Settings under "User privileges". Be sure to first view settings for the desired user or group (i.e. admin) at the top where it says "Editing global configuration. View for tag instead: ...". Enable "Can upgrade to development branch". Then navigate to Control Panel -> Restart or Upgrade, set the desired branch (e.g. master) under "Branch" and click "Update to branch". Watch the logs for any issues. Note that this is UNSUPPORTED and that this branch may contain more bugs than the regular releases.

When re-building 4CAT's Docker containers, initially 4CAT may fail to run because the new containers use a more recent database version than the one you originally installed 4CAT with. In Docker Desktop, the '4cat_db' container may fail to run or you may see an error message similar to:

The data directory was initialized by PostgreSQL version 14, which is not compatible 
with this version 15.3 (Debian 15.3-1.pgdg120+1).

This does not mean your data has been lost. You simply need to make 4CAT use the correct (older) version instead. You can do so the following way:

🎥 View a video screen cast of the steps below, with MacOS

0. (Follow steps 1-4 of "Upgrading with Docker via the command line")

1. Open a terminal and run the following command:

   docker run --rm -it -v 4cat_4cat_db:/var/lib/postgresql/data alpine cat /var/lib/postgresql/data/PG_VERSION
   

2. The command (it may take a moment to run) will output a number, e.g. '17'. This is your database version.

3. Open the .env file in a text editor (like Notepad or TextEdit). Look for the line near the top starting with POSTGRES_TAG=. Edit it to set the database version you found with the previous command, for example:

   POSTGRES_TAG=17
   

4. Save the file and repeat the upgrade steps starting at step 5.

• Open the 'Command Prompt' or 'Terminal' application that comes with Windows; both work for our purposes.

• Open the 'Terminal' application that comes with macOS.

• Linux comes in many shapes and sizes. Look for an application with 'Prompt', 'Terminal', or 'Shell' in its name.

• Click the 'Terminal' button on the lower right of the Docker Desktop window. When opening the terminal for the first time, you need to click 'Enable' to enable it.

• In Finder, press Command + Shift + . (dot).

• Linux file explorers comes in many flavours. Ctrl + H is a common hotkey combination to toggle the display of hidden files in the file explorer.

• On Windows, files starting with a dot are not hidden.

In Docker, the 4CAT web interface upgrade feature modifies both the volume data (which persists) and the container itself (which persists when stopping and starting containers, but NOT when building or redeploying from an existing 4CAT image). You will thus notice no difference if you keep the existing Docker container and stop and start it as needed. However, if you run 4CAT from the a Docker image, you may experience slowdowns after upgrading. When deploying with an older existing image, 4CAT will read the persistent volume information and upgrade the containers again to the current version. Depending on your setup, it may be faster/more efficient to download the most recent 4CAT image and deploy using that image. I.e., to follow the command line instructions on updating 4CAT.

Note: Early Docker V1 used the docker-compose command, but V2 version use docker compose. There are differences between the two commands, but they should not effect the following commands to upgrade 4CAT. I.e., older Docker versions can replace docker compose with docker-compose.

The helper-scripts/migrate.py script works based on a .current-version file that is created in the 4CAT directory. This file is not maintained by Docker and is lost on rebuild. Normally this does not matter as Docker creates the environment itself, however, it is possible that necessary changes do not take place (particularly with the database). You can force migrate to run by deleting the .current-version file and running helper-scripts/migrate.py. E.g.:

docker exec -it 4cat_backend /bin/bash
python3 4cat-daemon.py stop
rm .current-version
python3 helper-scripts/migrate.py
python3 4cat-daemon.py start