access_tildeAccess - ACCESS-NRI/accessdev-Trac-archive GitHub Wiki
~access is the shared directory for all users of the ACCESS system. Files that get used by multiple different groups should get installed here. It is for permanent, long term storage.
Directories
~access/apps
apps
is where ACCESS-specific progams & utility libraries are stored. It uses the same structure as the supercomputer's root /apps
directory - programs are installed to directories named after the program name & version. For instance version 1.2.3
of the foo
library would be installed to the path ~access/apps/foo/1.2.3
. Researchers shouldn't access these programs directly - instead they should load the app into their environment using the module system.
~access/modules
In order to load a program a user should load it into their environment using the module
command. The module system will look for a corresponding script in ~access/modules
which defines the environment variables to set. This means that we can avoid hard-coding paths - to change the version of a library getting used by a simulation you just need to change the module that gets loaded.
A module is configured using a plain-text file, stored in a directory named after the program or library. For instance to find the module file for foo/1.2.3
you would look in the directory ~access/modules/foo
for the file named 1.2.3
.
The most basic module file looks like
#%Module
set help "Library for foo-ing"
set install-contact "Scott Wales <[email protected]>"
set install-date 2013-07-23
set url foo.example.com
set prefix /projects/access/apps/foo/1.2.3
source /projects/access/modules/common
The module file includes a small amount of meta-data, including who installed it and where it came from. It also sets a variable prefix
, which defines where the library was installed. The final thing any module should do is to source the file ~access/modules/common
. This file uses the prefix variable to set up the user environment, setting variables including:
- PATH: applications ($prefix/bin)
- CPATH: headers ($prefix/include)
- LD_RUN_PATH: libraries ($prefix/lib)
- MANPATH: documentation ($prefix/man)
To add additional PATHs you can use the prepend-path
command in the module file. For instance a Python library may wish to add to the module file:
prepend-path PYTHONPATH $prefix/lib/python2.7/site-packages
When the module gets loaded the directory is added to the PATH variable, when it is unloaded it gets removed.
~access/data
Larger model input files, such as ancillary data for UM runs, gets placed in the data
directory. Currently this is a mirror of vayu:/data/projects/access
, a hand edit exists on accesscollab to make a UM job use this new format (~access/raijin/data-paths.sh).
~access/umdir
Files & scripts specifically for the UM. This includes the small execs and default run scripts for each model version.
Installed Modules
Top level ACCESS modules include:
- cdat-lite: Python library for climate data processing
- use version 6.0rc2-fixed. See https://github.com/MartinDix/cdat_lite_test for the fixes in this version.
- cylc: Suite scheduler
- fcm: Met Office build system
- gcom: MPI abstraction library
- iris: Python library for climate data processing
- rose: Model configuration tool
- um: UM environment settings
- This sets the $DATAOUTPUT and $UMDIR environment variables
- oasis3: Coupling library
- oasis3-mct: Coupling library
- mars: mars client for accessing OpenDAP archive on raijin
- first export GRB_ROOT=/path/to/data/archive, default is /short/$PROJECT/$USER/opendap
- ar_mars.sh to archive, where file must be pxnnn in grib format, with filename as per output from the model, and x = a..g but not pi
- mars.sh to retrieve. Set GRB_ROOT to access other users' archive, eg. /share/dp9/yix548/opendap
- ODB:
- odb/32.0: based on ECMWF version 32 for use in APS0/1
- odb/29.1: updated version to match lastest versions from UKMO and ECMWF for use in OPS29.1+
- cawcr-utils: various cawcr-utils such as um/grib/bufr utils and metview4
To use these you must first enable the access modules, then load them normally:
module use ~access/modules
module load rose
Additionally a number of python libraries have been installed, generally as pre-requisites for other models.
- Cython
- PIL
- ScientificPython
- Shapely
- cartopy
- mock
- netCDF4
- nose
- pep8
- pyke
- pyshp
These are in the pythonlib
subdirectory, load them either like
module load pythonlib/netCDF4
or do a 'module use
' on the pythonlib directory
module use ~access/modules/pythonlib
module load netCDF4
Requesting modules
To request a program or library be installed please email [email protected]
Installing modules
Members of the access.admin group are able to set up modules in the ~access space, following the directory structure above.
Python libraries can be installed using a special script
~access/admin/install-python-lib.sh LIBNAME
This script will download and install a library from PyPI - if there are no prerequisites this happens automatically. If the library uses C code that needs to be compiled this should use gcc, rather than the intel compilers (the intel compilers cause library conflicts)