Marseille mésocentre - aancel/admin GitHub Wiki

Tips for using oar and tools on Marseille mésocentre from the CIRM.
Somme useful links on the official website:

  • Available submission queue: here
  • Hardware configuration: here
  • Installed software: here
  • Tutorials: here

SSH access

The login machine for Marseille HPC center is named login.ccamu.u-3mrs.fr. To ease the ssh connection to the access node, you can add the following information in your .ssh/config file:

Host rheticus
    User username
    HostName login.ccamu.u-3mrs.fr

After this change, the following command:

ssh rheticus

Will be equivalent to:

ssh [email protected]

Access to the custom built modules for your project

Initial configuration

To access the custom built modules for your project, add the following lines to your .bashrc (or equivalent configuration file for your current shell):

# source configuration for modules
PREVPATH=`pwd`
cd /softs/cemracs_2016/config/etc
FILE_TO_SOURCE=feelpprc.sh
if [[ -f "${FILE_TO_SOURCE}" ]]; then
source ${FILE_TO_SOURCE}
else
echo "File to source (${FILE_TO_SOURCE}) does not exist"
fi
cd ${PREVPATH}

Then when issuing a module avail command the new modules should appear.

Project profiles

Regarding to the modules you asked for, they can be listed with the module avail command. To ensure that you don't encounter any issue with compatibility, a special module named project.profile is also created, where project is the name of your project. This specific module will load all the software you asked for.

Useful module commands

The module command allow you to load environment variables that makes new software accessible to you. Those pieces of software are software built outside of the packaging system.

  • module avail: list the applications/libraries that can be loaded in your environment
  • module list: list the applications/libraries that are currently loaded for your environment
  • module load modulename: loads the module named modulename and its associated libraries/applications
  • module unload modulename: unloads a module from teh environment
  • module purge: unloads all the currently loaded modules

Oar

  • Oar usage: https://equipex-mesocentre.univ-amu.fr/oar/

  • List all nodes with their properties: oarnodes

  • List only node names: oarnodes -l

  • Interactive job: oarsub -I ...

    • Example: oarsub -I -l cpu=8,walltime=5:00:00 ./script.sh

  • Batch: oarsub -S ./mon_script.oar (TODO: chmod 755 ./mon_script.oar)

  • List jobs for user: oarstat -u <user>

  • More descriptive list for user: oarstat -u <user> -f

  • Connect to the node on which the job runs: oarsub -C <jobid>

  • SSH to the nodes running a job: OAR_JOB_ID=<jobid> oarsh <node>

GPU Computing

  • Allocate an interactive job on 1 node with 1 gpu: oarsub -I -l host=1/gpunum=1 -p gpu

Visualisation

Debug your running app:
top, strace -p et gstack

Troubleshooting modules

To ensure that modules, especially Python modules, works correctly with cmake, check that the CMAKE_PREFIX_PATH is set in the module definition. Don't hesitate to re-create modules from the system if they seem to be missing the CMAKE_PREFIX_PATH.
You can for example end up with detection error in cmake, with partial detection of a loaded Python module (e.g. Python/2.7.3) and partial detection of the system install python (2.6). This can cause errors like _PyUnicodeUCS4_AsDefaultEncodedString, where the two libraries have different encodings.

Example: Loading into a Python module: PYTHONHOME, PYTHONPATH, PATH is not enough to ensure correct detection from cmake (e.g. VTK cmake step)

SSHFS on OS X:

Install:

Add a custom footer
⚠️ **GitHub.com Fallback** ⚠️