Installing dependencies on NERSC Machines - ProfessorBrunner/rc3-pipeline GitHub Wiki

Some of the packages in the Astromatic suite (SExtractor, STIFF) is already installed in NERSC modules. I followed Theodore Kisner's instructions on how to load these modules: On Edison:

source /project/projectdirs/cmb/modules/hpcports_NERSC.sh

hpcports shared_gnu

module load astromatic-hpcp

So additionally, we need Montage and Montage-wrapper

wget http://montage.ipac.caltech.edu/download/Montage_v3.3.tar.gz

gunzip Montage_v3.3.tar.gz

tar -xf Montage_v3.3.tar

cd Montage_v3.3

make

Print the file path using the pwd command, then add the address to your .bashrc.ext file (which should be in your $HOME directory): PATH=$PATH:/global/scratch2/sd/dorislee/Montage_v3.3/bin

Loading in the python module gives us the basics scientific Python stuff (Numpy..etc):

module load python

Now we need to install Astropy and Montage wrapper:

Pip is kind of weird on NERSC. First we need to run a script which is inside

mk_pip_cabundle.sh

If this file is not found append this to the PATH variable: PATH=$PATH:/usr/common/usg/bin

Then any package could be installed by using the flags
pip --cert $HOME/.pip/cabundle install --user astropy
pip --cert $HOME/.pip/cabundle install --user montage-wrapper
pip --cert $HOME/.pip/cabundle install --user astroquery

To test that this is actually working:

dorislee@edison04:~> python
Python 2.7.9 (default, Jan  8 2015, 22:29:31) 
[GCC 4.9.1 20140716 (Cray Inc.)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
import astropy
import montage_wrapper as montage
import astroquery

Now everything should be ready for running bulk_run.py

Building with MPI

To compile with MPI, we need to first ask for a node in the interactive queue: qsub -I -l mppwidth=24' Then we need to load in the GNU compilers and necesaary modules:

module load openmpi-serial

Now you should be able to find the command mpicc :

> mpicc
gcc: fatal error: no input files
compilation terminated.

Inside the Makefile add in the line:

mpicc = mpicc

The montage-wrapper should be updated