Using FDS on roihu.csc.fi - firemodels/fds GitHub Wiki

These notes provide basic information about using FDS on roihu.csc.fi, of CSC, Finland. It is assumed you have an account and have logged in for the first time.

Create ssh keys
```
ssh-keygen -t ed25519 -C "[email protected]" 
```
and add the public key to your Github ssh keys. See generating ssh keys.
Clone the firemodels/fds repo to one of your /projappl/project_xxx folders. Use these notes.
In the directory Build/ompi_gnu_linux, run the script to compile
```
./make_fds.sh
```
Prepare a SLURM script like this in a folder under your /scratch/project_xxx (in this example, the name of the script is job_name_script.sh):
```
#!/bin/bash 
#SBATCH --job-name=ductFlow
#SBATCH --account=project_xxx
#SBATCH --time=05:00:00
#SBATCH --mem-per-cpu=1G
#SBATCH --partition=small
#SBATCH --ntasks=8
#SBATCH --cpus-per-task=4
#SBATCH --tasks-per-node=8
export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
srun /projappl/project_xxx/firemodels/fds/Build/ompi_gnu_linux/fds_ompi_gnu_linux duct_flow.fds
```
This script starts 8 tasks (8 mesh) and four OpenMP threads for each task. Roihu has 486 CPU nodes, and 2 x 192 cores per node. All tasks (i.e. 32 threads) can fit into a single node. One GB memory is reserved for each thread, and five hours for the whole thing.

Add the following command to the end of job_name_script.sh to print out the maximum memory requirement during the job:
```
sacct -o reqmem,maxrss -j $SLURM_JOBID
```
Another way to collect information about job is to add seff command.
Run the job by submitting the script:
```
$ sbatch job_name_script.sh
```
To run the FDS verification suite, there are two options:

To run using the queuing system, go to fds/Verification/scripts and run
```
$ export SLURM_MEMPERCPU=4G
$ export SBATCH_ACCOUNT='project_XXX'
$ export USE_MAX_CORES=1
$ ./Run_FDS_Cases.sh -q small -w 06:00:00
```
These commands will first set the memory requirement for computing nodes, and then run both normal and benchmarking cases on the 'serial' queue. It assumes the jobs can be run in 6 h. To check how close to the 4 GB limit you got, you can list the memory usages between 3 and 4 GB since date YYYY-MM-DD
```
sacct -o maxrss -S YYYY-MM-DD --units=G | grep '3...G'
```
Another option is to use interactive shell:
```
$ sinteractive --account project_XXX --time 48:00:00 --mem 16000 --tmp 100
$ ./Run_FDS_Cases.sh -q none 
```
Once run the cases, create Smokeview images by running
```
$  ./Make_FDS_Pictures.sh -p ~/bin -X 
```