HPC facilities

1. Summer

ssh -X [email protected]

Useful links:

• http://summer.intra.cea.fr/wiki/doku.php/summer:main
• http://summer.intra.cea.fr/qmon

Example submission script for VASP calculation:

#!/bin/bash
#PBS -S /bin/bash
#PBS -l walltime=48:00:00,nodes=4:ppn=32
#PBS -m n
#PBS -q csp
#PBS -N jobName

ulimit -s unlimited
module purge
module load gcc/7.1.0 ompi/3.1.1 mkl/18

cd $PBS_O_WORKDIR

mpirun -np 128 /W/av245900/Codes/VASP_5.4.4_gcc_skylake/vasp.5.4.4/bin/vasp_std

---

2. Irene

ssh -X [email protected]

For more information see: https://www-tgcc.ccc.cea.fr/fr/tgcc/User_documentation.html

Email address for user support: [email protected]

Useful paths

cd $CCCSCRATCHDIR

cd $CCCWORKDIR

cd $CCCSTOREDIR

Warning: Jobs are run from the scratch directory, but files not accessed for over 60 days will be deleted. The work directory is not purged, but is also not backed up! The store directory is the best for long term storage of data. (See later for adding rsync backup to your .bashrc)

Useful commands on irene

For submitting a job: ccc_msub

For checking jobs in the queue: ccc_mpp -u $USER

For checking remaining project hours: ccc_myproject

To kill a job: scancel BATCHID

Transferring data to irene

To avoid issues with permissions (that sometimes arise from using scp or tar decompression), using rsync to transfer data with specific options is a good solution. The following alias can be added to the .bashrc on your local machine. Change PROJECTCODE accordingly and this function can be run from the shell just as irene_rsync what-you-what-to-send where-you-want-to-receive-it

# rsync for irene (issue with gitbit permissions with scp or tar files)
# Two args: source, destination
function irene_rsync {
	rsync -a ${1} ${2} --chmod=Dg+s --chown=:PROJECTCODE
}

Submitting jobs

• Skylake nodes: 48 cores each
• Submit script with: ccc_msub
• Check queue with: ccc_mpp (or 'ccc_mpp -u $USER' to just see yourself!)
• Execute binary with mpi: ccc_mprun
• To kill a job: scancel BATCHID (where BATCHID can be found from the queue or when submitting the job)
• For more information on job submission, see: https://www-fr.ccc.cea.fr/docs/irene.info.html#Job_submission

Example submission script for VASP calculation:

For VASP, must email [email protected] to request a new user form, you will then be able to load the vasp module, as in the example submission script below.

#!/bin/bash
#MSUB -q skylake
#MSUB -A gen10765
#MSUB -r jobName
#MSUB -n 48
#MSUB -T 6000
#MSUB -m work

# If you are working on several project codes, edit and uncomment the line below
#module switch dfldatadir/PROJECTCODE

module load vasp
cd ${BRIDGE_MSUB_PWD}
ccc_mprun vasp

Extra notes:

• Time (#MSUB -T) is in seconds on irene
• Using the test queue: #MSUB -Q test (Note: QoS is limited to 2 jobs of maximum 30 minutes and 10 nodes each)

---

Optional additions to your .bashrc

Aliases:

Save yourself some typing with custom commands, e.g. alias q='ccc_mpp -u $USER'

Misc. useful things for HPC usage

SSH-keys:

For setting up ssh keys (for password-less login), see: https://confluence.atlassian.com/bitbucket/set-up-an-ssh-key-728138079.html

tree:

tree -L 3 e.g. is like ls for file depth of 3, very pretty and already installed on irene.