D. Docker file and image - nmik/Xgam GitHub Wiki

Step 1. Install Docker

Docker is a set of platform as a service (PaaS) open source products that use OS-level virtualization to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries and configuration files. For more information see https://www.docker.com/resources/what-container. The elements needed to run a container are represented by a Docker image.

In order to create or pull your image and run a container with the Xgam package and dependencies included you need to install the Docker suite.

An overview of the installation can be found here: https://docs.docker.com/install/overview/. Installation instruction for specific OS:

Step 2. Create the Docker Image

We provide a Docker file that contains the instruction needed in order to build the Xgam Docker image. The image can be created by opening the terminal and running this command in the directory where the Dockerfile is present:

docker build -t xgam .

The final dot is necessary. This image is tagged as "xgam" but you can choose you own tag by changing the name after the -t option. The image creation will take some minutes; you will need an internet connection and 10 GB of disk space.

NOTE: a repository from which you can pull a pre-built image will be established in the near future.

In order to check that everything is fine, you can simply run:

docker images

This command will return the list of your images, which should include xgam:

REPOSITORY          TAG                             IMAGE ID            CREATED             SIZE
...
xgam                latest                          6f1681bec903        24 hours ago        7.05GB
...

Step 3. Run the container and the Xgam apps

You can choose to run your image interactively or in background. In both ways, the Docker image needs to access files in the host file system; in order to do that you should link the folder where you wish to store your data and results to the /run_xgam/home/ container directory by mounting an appropriate volume (-v option).

In order to run your container in an interactive session the command is:

docker run -it -v /your_path/:/run_xgam/home/ xgam:latest

A bash session from the container will replace your current local session (the Docker container is based on Ubuntu 18.04). As you can notice, the /your_path/ folder is matched with the /run_xgam/home/, and you should be able to access all its content. If you tagged your image with a different name you should just replace "xgam:latest" with your tag. You can also tag your container execution with the option --name your_container

A brief look to the other options:

--interactive , -i  Keep STDIN open even if not attached
--tty         , -t  Allocate a pseudo-TTY
--volume      , -v  Bind mount a volume

Now you are able to run the Xgam apps in the /run_xgam/ folder by typing for example:

python /run_xgam/Xgam/bin/mkdataselection.py --config=/run_xgam/CONFIG/your_config.py

Alternatively you can run your container non-interactively by dropping the -it option. In this case you should pass some script to be executed by the container. The script must be present in the mounted volume and must be linked with the path within the Docker container. You can write for instance a simple data selection pipeline:

docker run -v /your_path/:/run_xgam/home/ xgam:latest /run_xgam/home/bash_script.sh

We provided an example with the bash_script.sh file:

#!/bin/bash

python /run_xgam/Xgam/bin/mkdataselection.py --config=/run_xgam/home/CONFIG/config_dataselection.py

python /run_xgam/Xgam/bin/mkforeground.py --infile=/run_xgam/Xgam/fits/gll_iem_v07.fits --nsideout=1024

python /run_xgam/Xgam/bin/mksmartmask.py --config=/run_xgam/home/CONFIG/config_datafluxmaps.py --srccat=/run_xgam/Xgam/fits/gll_psc_v20.fit --srcextcat=/run_xgam/Xgam/fits/gll_psc_v20.fit --irfs=P8R3_SOURCEVETO_V2 --evtype=3 --ltfile=/run_xgam/home/output/output_gtltcube/tag_outofltcube.fits --outflabel=tag

python /run_xgam/Xgam/bin/mkdatafluxmaps.py --config=/run_xgam/home/CONFIG/config_datafluxmaps.py

Of course you can run your scripts also in the interactive session.

IMPORTANT: by default the photon and spacecraft folders are assumed to be in the /run_xgam/home/ folder. If you choose to move your data in a non-default folder for Xgam, you must always export the path both if you choose the interactive or the background execution (in this case you should include the export lines in the script). The folder must in any case be present in the mounted volume, otherwise the container cannot access it.

Example: photon data are in /home/user/data/fermidata/, you can mount the volume -v /home/user/:/run_xgam/home/ and run export P8_DATA=/run_xgam/home/data/fermidata/. The same is valid also for other input/output folders e.g. X_OUT.

To end the session, simply type:

exit

More on Docker

After you exit the session, the container is no longer running, but it still exists. To list all the containers execute:

docker ps -a

You can restart and stop a container with this commands:

docker restart container_id
docker stop container_id

or if you like to have another interactive session:

docker exec -it container_id

In order to remove a container no longer useful, you can type:

docker rm container_id

Similarly, you can delete Docker images:

docker rmi image_id