This page has instructions for running iEBE-MUSIC simulations on the Wayne State Grid.

First, one needs to initialize an interactive node (with 4 CPU cores)

srun -q primary -N 1 -n 4 --mem=16G -t 24:0:0 --pty bash

Run simulations with native setup

1. Download and compile code package

First, you can download the code package by git clone https://github.com/chunshen1987/iEBE-MUSIC. Please use codes/get_code_packages.sh to download the individual code packages and codes/compile_code_packages.sh to compile all the packages before event-by-event simulations.

git clone https://github.com/chunshen1987/iEBE-MUSIC -b dev
cd iEBE-MUSIC/codes
./get_code_packages.sh
./compile_code_packages.sh

2. Generate work directory

cd ..
cp config/parameters_dict_user_IPGlasma.py .
./generate_jobs.py -w [runFolder] -c wsugrid -n 64 -n_hydro 16 -n_urqmd 4 --n_threads 4 -par parameters_dict_user_IPGlasma.py --nocopy

You need to pick a name for your [runFolder], where simulations will be performed.

3. Submit the simulation jobs to the cluster scheduler

./submit_all_jobs.sh [runFolder] [queueName]

The queueName has options: primary(default), secondary, requeue, and debug.

Run simulations with the singularity container

On the Wayne State Grid, one can perform simulations with the pre-build singularity container. Such a setup would be preferred if a fixed version of the iEBE-MUSIC package were used. One can follow the following steps to perform calculations,

1. Download the singularity image to a local directory

singularity pull [imageName.sif] docker://chunshen1987/iebe-music:dev

2. Download the iEBE-MUSIC code package from the Github repository,

git clone -b dev https://github.com/chunshen1987/iEBE-MUSIC
cd iEBE-MUSIC/codes
./get_code_packages_dev.sh

3. Now you can create the working directory from the iEBE-MUSIC folder,

./generate_singularity_jobs.py -w [runFolder] -c wsugrid -n 10 -n_hydro 4 -n_th 4 -par parameterFile.py -singularity [path_to_singularity_image]

You must replace the [path_to_singularity_image] with the path to the singularity image file and [runFolder] with your simulation folder name.

The full help message can be viewed by ./generate_singularity_jobs.py -h,

Usage: generate_singularity_jobs.py [-h] [-w] [-c] [-n] [-n_hydro] [-n_th] [-par] [-singularity] [-exe] [-b] [-seed]

⚛ Welcome to iEBE-MUSIC package

optional arguments:
  -h, --help            show this help message and exit
  -w , --working_folder_name 
                        working folder path (default: playground)
  -c , --cluster_name   name of the cluster (default: local)
  -n , --n_jobs         number of jobs (default: 1)
  -n_hydro , --n_hydro_per_job 
                        number of hydro events per job to run (default: 1)
  -n_th , --n_threads   number of threads used for each job (default: 1)
  -par , --par_dict     user-defined parameter dictionary file (default: parameters_dict_user.py)
  -singularity , --singularity 
                        path of the singularity image (default: iebe-music_latest.sif)
  -exe , --executeScript 
                        job running script (default: Cluster_supports/OSG/run_singularity.sh)
  -b , --bayes_file     parameters from bayesian analysis (default: )
  -seed , --random_seed 
                        Random Seed (-1: according to system time) (default: -1)

4. Submit the simulation jobs to the cluster,

./submit_all_jobs.sh [runFolder] [queueName]

Collect and analyze the simulation results

After all the simulation jobs are finished, we can do the following steps to collect the results into a single hdf5 database file,

1. Collect results

mkdir -p resultsFolder
./collect_events.sh [runFolder] resultFolder

Here, the [runFolder] needs to be replaced by the actual path of your simulation folder.

If one runs simulations with the singularity container, then please use the following command to collect the results,

mkdir -p resultsFolder
./collect_events_singularity.sh [runFolder] resultFolder

2. Analyze the simulation results

There are analysis scripts under the iEBE-MUSIC/analysisKit folder. You can use them to compute various experimental observables.