Software Modules - icl-utk-edu/cluster GitHub Wiki

Managing Your Environment With Modules

When using a Linux or unix-like system, there are a number of shell variables and command aliases that determine the when the system reacts to your command-line activity. For example, when you type a command, the shell will search the $PATH variable (which consists of a list of directories) in order to find an executable file with the same name as the command that you typed. When a binary application is executed, it may look through directories in the $LD_LIBRARY_PATH variable to find the library files that it needs to load. Also, when you type "man ls", the "man" application searches a list of directories in the $MANPATH variable to find the manual page that you have requested to read. You can list all the variables in your current environment with the "env" shell command, and you can print the contents of a variable with the "echo" command:

[user@newton0 ~]$ echo $PATH
/opt/sge/bin/lx24-amd64:/usr/lib64/qt-3.3/bin:/data/apps/pgi/linux86-64/6.0/bin:/data/apps/openmpi/1.4.2-intel/bin:/usr/kerberos/bin:/opt/intel/fce/10.1.018/bin:/opt/intel/idbe/10.1.018/bin:/opt/intel/cce/10.1.018/bin:/usr/local/bin:/bin:/usr/bin:/home/user/bin

In addition to environment variables, the shell also has command aliases. You can list these with the "alias" command.

In the course of your work on ICL systems, you may want to modify these variables or aliases. For instance, if you want to execute the command "R" that is located at /data/apps/R/R-2.11.1/bin/R, you would either have to type in the full path each time, or you could add the directory /data/apps/R/R-2.11.1/bin/ to your environment's $PATH variable. While it is possible to directly modify the variables and shell aliases, we offer an better system, know as "Modules".

Modules

The modules system provides a uniform way of managing your environment. Each module generally corresponds to a single version of a particular application. The module contains all the information needed to modify your shell environment in order to support use the the application. Modules allow you to easily choose among the multiple versions of an application that are available. For example, we offer multiple versions of !OpenMPI on the system as modules. If you choose a particular !OpenMPI version to load into your environment, the modules system will modify your $PATH, $LD_LIBRARY_PATH, $MANPATH, and other shell variables that are needed for !OpenMPI to function correctly. You may also unload a module from your environment, reversing these changes.

You use the Modules system with the "module" command. Typing "module" will provide you with a command usage summary. You can also use "man module" for more detailed usage information.

Listing available modules

To list the available modules on the system, use the "avail" sub-command:

[user@newton0 ~]$ module avail

------------------------------------- /data/apps/Modules/versions --------------------------------------
3.2.7

--------------------------------- /data/apps/Modules/3.2.7/modulefiles ---------------------------------
454/454                           hmmer/2.3.2-MPI-0.92              openmpi/1.4.1-intel
FFmpeg/061611                     hmmer/3.0                         openmpi/1.4.2-gcc
Git/1.7.5.4                       idl                               openmpi/1.4.2-gcc-static
Modules                           intel-compilers/11.1.072(default) openmpi/1.4.2-intel
R/2.11.1                          intel-compilers/2011.5.220        openmpi/1.4.3-gcc
R/2.12.0                          java/jdk1.6.0_25_i586             openmpi/1.4.3-gcc-psm
R/2.12.0-32bit                    java/jdk1.6.0_25_x64              openmpi/1.4.3-gcc4.4.6-psm
R/2.13.0                          lammps/21Dec11                    openmpi/1.4.3-intel
R/2.14.0                          libevent/2.0.17                   openmpi/1.4.3-intel-psm(default)
amber/11                          libxml2/2.7.1                     openmpi/1.4.3-intel-psm-test
amber/11-mpi                      libxslt/1.1.19                    perlmodules/5.8.8
atlas/3.9.17                      maple/14                          pgi/6.0-i686
biopython/1.58                    mathematica/7.0.1                 pgi/6.0-x86_64
blast/2.2.22(default)             mathematica/8.0.1                 pgi/6.1-x86_64
blast/2.2.24                      mathematica/8.0.2(default)        phyml/3.0
charmm/c35b1r1                    matlab/R2010a                     proj4/4.7.0
charmm/c35b1r1-mpi                mercurial/1.6.2                   python/3.2.1
clustalw/2.0.12                   module-cvs                        python2/2.5.5
clustalw/mpi-0.13-1               module-info                       qt/4.3.1-i686
cmake/2.8.3                       modules                           qt/4.3.1-x86_64
coils/2.2                         mothur/1.12.0                     raxml/7.2.6
comsol/4a                         mothur/1.12.0-mpi                 root/5.26
defaults                          mothur/1.13                       rubygems/1.8.21
emacs/23.2                        mothur/1.13-mpi                   sas/9.2TS2M0
fftw/2.1.5                        mpfr/2.3.2                        sas/9.2TS2M3
fftw/3.2.2                        mpfr/3.0.1                        schrodinger/2010-02
fgsl/0.9.4                        mrbayes/3.1.2                     scipy/0.10.1
fluent/13.0                       mummer/3.21                       signalp/3.0
fluka/2010.2pre                   muscle/3.7                        site_utilities
gamess/2010-10(default)           namd/2.6-intel-openmpi            spss
gamess/2011-04-29-noMPI           namd/2.6-threaded                 sqlite/3.7.6.3
gaussian/g03(default)             namd/2.8-intel-openmpi            subversion/1.6.17
gaussian/gaussview                ncurses/5.9                       tmhmm/2.0c
gcc/4.4.6                         null                              udunits/1.12.9
gdal/1.8.1                        numpy/1.6.1                       udunits/2.1.11
gmp/5.0.2                         nwchem/6.0(default)               use.own
grace/5.99                        nwchem/6.0-patched                vim/7.3
grass/6.4.2RC1                    nwchem/6.1                        visit/2.0.1
gromacs/4.5.1-mpi                 openfoam/1.5-i386                 visit/2.1.0
gromacs/4.5.1-threaded            openfoam/1.5-x86_64               visit/2.3.0
gsl/1.15                          openmpi/1.2.8-intel               vmd/1.8.7
hdf5/1.6.10                       openmpi/1.4                       wxPython/2.8
hdf5/1.6.10-gcc                   openmpi/1.4.1-gcc                 yasm/1.1.0

Each module has a name followed by zero or more version identifiers separated by slashes. Some modules have multiple available versions. You can list the versions for a specific module:

[user@newton0 ~]$ module avail hmmer

---------------------- /data/apps/Modules/3.2.7/modulefiles -----------------------
hmmer/2.3.2-MPI-0.92 hmmer/3.0

Loading a module

Once you have identified the specific module and (optionaly) version that you want to use, load the module using the "load" or "add" sub-commands:

[user@newton0 ~]$ module load mothur/1.13
[user@newton0 ~]$ which mothur
/data/apps/Mothur/1.13/mothur

You can see that when typing "mothur" the system is able to find the location of the application in a non-standard directory because the directory was added to your "$PATH variable by the module.

Listing loaded modules

The sub-command "list" will show the modules that are currently loaded into your environment:

[user@newton0 ~]$ module list
Currently Loaded Modulefiles:
  1) openmpi/1.4.2/intel(default)   3) mothur/1.13
  2) intel-compilers/11.1.072

You can see that mothur appears here as well as two other modules which are loaded by default on our system.

Automatically loading modules

Certain modules that you use often may be automatically loaded at login. It is recommended that you do this by creating a ~/.modulerc file. This file must contain the special modulefile token "#%Module" on the first line followed by one or more module commands:

#%Module
module load mothur/1.13

This will cause mothur version 1.13 to be automatically loaded when you log in.

Unloading a module

Using the "unload" sub-command, you can often "undo" any changes that it has made to your environment. Some changes are not fully reversible though. Here, we unload the mothur module and show that the application will no longer be picked up when typing "mothur":

[user@newton0 ~]$ module unload mothur
[user@newton0 ~]$ mothur
-bash: mothur: command not found

Replacing a module

If you have a currently loaded module which you would like to replace with a different verison, use the "switch" sub-command:

[user@newton0 ~]$ module switch openmpi/1.4
[user@newton0 ~]$ which mpirun
/data/apps/openmpi/1.4/bin/mpirun
[user@newton0 ~]$ module list
Currently Loaded Modulefiles:
  1) openmpi/1.4                2) intel-compilers/11.1.072   3) mothur/1.13

Using modules in batch jobs

When running compute jobs through the batch-queue system, it is recommend to load the application's module within the batch submit file. Here is an example batch file:

#$ -N jobname
#$ -q short*
module load mothur/1.13
mothur < inputfile

Module dependencies

Some modules have dependencies that need to be met before the module can be loaded. For example, Mother version 1.13-mpi requires that one of the openmpi modules be loaded first because this version of mothur is linked against the MPI libraries. Here, you can see the prerequisite error and resolution:

[user@newton0 ~]$ module load mothur/1.13-mpi
mothur/1.13-mpi(16):ERROR:151: Module 'mothur/1.13-mpi' depends on one of the module(s) 'openmpi/1.4.2-intel openmpi/1.4.2-gcc-static openmpi/1.4.2-gcc openmpi/1.4.1-intel openmpi/1.4.1-gcc openmpi/1.4 openmpi/1.2.8-intel'
mothur/1.13-mpi(16):ERROR:102: Tcl command execution failed: prereq openmpi
[user@newton0 ~]$ module load openmpi mothur/1.13-mpi

Conflicts

When you load a module, the system will load any dependencies automatically. If one of these automatically loaded modules conflicts with a module that you already have loaded, you will get a message like this:

$ module load test
intel-compilers/2016u3(2):ERROR:150: Module intel-compilers/2016u3 conflicts with the currently loaded module(s) intel-compilers/2013.1.3
intel-compilers/2016u3(2):ERROR:102: Tcl command execution failed: source /data/apps/Modules/modulefiles/../baselib.tcl

In this case, you must use "module switch intel-compilers/2016u3" to change your current version of the intel-compilers module to the version that is required by the "test" module that you tried to load. Then you can execute "module load test" again.

Making custom module files

If you compile a custom software package and install it in your home directory (or another non-standard location), you can create a user-specific modulefile to make it easy to use the new software:

  1. Add export MODULEPATH=$HOME/modules:${MODULEPATH} to your .bashrc file.
  2. Log out and back into the system to activate the change.
  3. Create a new directory in which to put your custom module files: mkdir ~/modules
  4. Copy an example modulefile to the new directory: 
    cp /data/apps/Modules/3.2.7/modulefiles/fluka/2010.2pre ~/modules/fluka-custom
  5. Modify $HOME/modules/fluka-custom as needed to use your new application.
  6. Execute module load fluka-custom to load it into your environment.

Further Reading

Users can create custom modules for privately installed applications and can group modules into projects for easy management of different computing environments. Consults the following resources for more information:

⚠️ **GitHub.com Fallback** ⚠️