2021

Connect to or-slurm-login.ornl.gov

Install mcvine

Install mcvine using conda

$ module purge
$ module load anaconda3/5.1.0
$ conda config --add channels conda-forge
$ conda config --add channels diffpy
$ conda config --add channels mantid
$ conda config --add channels mcvine
$ conda create -n mcvine3 -c mcvine/label/unstable mcvine=1.4.0 mcvine-core=1.4.1.dev python=3 openmpi
$ source activate mcvine3

Create ~/.mantid if necessary

$ git clone https://github.com/yxqd/dotmantid.git ~/.mantid

Test job script 1

#!/bin/bash

#SBATCH -N 1
#SBATCH -n 2
#SBATCH -c 1
#SBATCH -A birthright
#SBATCH -J ttt
#SBATCH -p burst
#SBATCH [email protected]
#SBATCH --mail-type=BEGIN,END,FAIL
#SBATCH --mem=0
#SBATCH -t 00:10:00
#SBATCH -o ./output.txt
#SBATCH -e ./error.txt
module purge
module load anaconda3/5.1.0

#source activate mcvine
export PATH=$HOME/.conda/envs/mcvine3/bin:$PATH
which python
which mcvine

# mcvine
mpirun -np 2 python -c "from mpi4py import MPI; print(MPI.COMM_WORLD.Get_rank())"

More examples

See https://github.com/mcvine/training/blob/master/cades/

2019: old. does not seem to work now

Connect to virtues.sns.gov

Install mcvine

$ module purge
$ module load anaconda2
$ conda config --add channels conda-forge
$ conda config --add channels diffpy
$ conda config --add channels mantid
$ conda config --add channels mcvine
$ conda create -n mcvine python=2
$ source activate mcvine
$ conda install openmpi mcvine

Create ~/.mantid/instrument/Facilities.xml

$ mkdir -p ~/.mantid/instrument

Then download Facilities.xml from https://github.com/mantidproject/mantid/blob/master/instrument/Facilities.xml

Test mpi

Submit the following script

#!/bin/bash

#PBS -N test_mpi4py_mcvine
#PBS -l nodes=5:ppn=36
#PBS -l walltime=0:00:10:00
#PBS -W group_list=cades-virtues
#PBS -A sns
#PBS -l qos=std
#PBS -q skylake


## main program ##

module purge
module load anaconda2
source activate mcvine
module load PE-gnu/3.0
module load openmpi

# Switch to the working directory (path of your PBS script)
cd $PBS_O_WORKDIR

# Test run
mpirun -np 180  python -c "from mpi4py import MPI; print MPI.COMM_WORLD.Get_rank()"

# exit
exit 0

If things are working correctly, we should see in the output random integers from 0 to 179.