Tutorials - calab-ntu/gpu-cluster GitHub Wiki

Tutorials of conducting various tasks on Eureka.

Running GAMER

For experienced users

  • Use the machine configuration file configs/eureka_intel.config on eureka or configs/spock_intel.config on spock (see this wiki page for details).
  • Example job scripts can be found at example/queue/submit_eureka.job and example/queue/submit_spock.job. To request a different number of nodes, modify nodes=1 but keep everything else the same.

For beginners

Here is a basic but complete example based on the 3D Blast Wave test. For details, see GAMER wiki.

1. Go to your work1 folder.

cd /work1/$USER

2. Download GAMER.

git clone https://github.com/gamer-project/gamer.git

3. Compile GAMER.

cd gamer/src
cp ../example/test_problem/Hydro/BlastWave/generate_make.sh .

Edit generate_make.sh.

  • eureka
PYTHON=python3
${PYTHON} configure.py --machine=eureka_intel --model=HYDRO --hdf5=true --mpi=true --gpu=true
  • spock
PYTHON=python3
${PYTHON} configure.py --machine=spock_intel --model=HYDRO --hdf5=true --mpi=true --gpu=true

Generate the executable gamer.

sh generate_make.sh
make clean
make -j 4

4. Prepare a working folder.

cd ../bin/
cp -r ../example/test_problem/Hydro/BlastWave .
cd BlastWave
cp ../gamer .

5. Run GAMER.

  • eureka
cp ../../example/queue/submit_eureka.job .
# Edit `submit_eureka.job` to
#  (1) put your email after `#PBS -M` and
#  (2) [optional] replace `nodes=1` with the number of nodes you want to use
qsub submit_eureka.job # It will return a unique job ID (e.g., `4305.eureka00.gpucluster.calab`)
  • spock
cp ../../example/queue/submit_spock.job .
# Edit `submit_spock.job` to
#  (1) put your email after `#PBS -M` and
#  (2) [optional] replace `nodes=1` with the number of nodes you want to use
qsub submit_spock.job # It will return a unique job ID (e.g., `4305.spock00.gpucluster.calab`)

You can monitor your run by qstat:

Job id                    Name             User            Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
4305.eureka00              gamer            fish                   0 R workq

The status R (the character under S) suggests that the task is Running. This small test should take less than 1 min. The status will switch to C once the task is Complete.

Job id                    Name             User            Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
4305.eureka00              gamer            fish            00:10:20 C workq

7. Visualize data with yt.

cd plot_script
python3 plot_slice_gas.py -s 10 -e 10
...
display Data_000010_Slice_z_density.png # replace `display` with `feh` on spock

images/blastwave.png

Running FLASH

TBF

Links

Home