Trilinos - shawfdong/hyades GitHub Wiki
The Trilinos Project is an effort to develop algorithms and enabling technologies within an object-oriented software framework for the solution of large-scale, complex multi-physics engineering and scientific problems. A unique design feature of Trilinos is its focus on packages.
The Trilinos 11.12 release uses the CMake build system, which requires CMake version 2.8.11 or greater. Trilinos requires BLAS and LAPACK. We'll build Trilinos 11.12.1 with Intel MKL & Intel MPI using Intel compilers.
Download Trilinos 11.12.1:
$ cd /scratch $ wget http://trilinos.org/oldsite/download/files/trilinos-11.12.1-Source.tar.bz2 $ tar xvfj trilinos-11.12.1-Source.tar.bz2
Create a build directory:
$ mkdir trilinos-11.12.1-Build $ cd trilinos-11.12.1-Build
Configure Trilinos[1]:
$ cmake \ -D CMAKE_CXX_FLAGS:STRING=" -DMPICH_IGNORE_CXX_SEEK -DMPICH_SKIP_MPICXX" \ -D CMAKE_BUILD_TYPE:STRING=RELEASE \ -D CMAKE_VERBOSE_MAKEFILE:BOOL=ON \ -D Trilinos_VERBOSE_CONFIGURE:BOOL=ON \ -D TPL_ENABLE_MPI=ON \ -D MPI_BASE_DIR=/opt/intel/impi/4.1.3.045/intel64 \ -D MPI_C_COMPILER:FILEPATH=mpiicc \ -D MPI_CXX_COMPILER:FILEPATH=mpiicpc \ -D MPI_Fortran_COMPILER:FILEPATH=mpiifort \ -D Trilinos_ENABLE_ALL_PACKAGES=ON \ -D BLAS_INCLUDE_DIRS:PATH=$MKLROOT/include/ \ -D BLAS_LIBRARY_DIRS:PATH=$MKLROOT/lib/intel64/ \ -D LAPACK_LIBRARY_DIRS:PATH=$MKLROOT/lib/intel64/ \ -D BLAS_LIBRARY_NAMES:STRING="mkl_intel_lp64;mkl_core;mkl_sequential" \ -D LAPACK_LIBRARY_NAMES:STRING="mkl_intel_lp64;mkl_core;mkl_sequential" \ -D BUILD_SHARED_LIBS:BOOL=ON \ -D CMAKE_INSTALL_PREFIX=/pfs/sw/parallel/impi_intel/trilinos-11.12.1 \ /scratch/trilinos-11.12.1-SourceNote:
- Trilinos 11.12.1 is installed at /pfs/sw/parallel/impi_intel/trilinos-11.12.1 (-D CMAKE_INSTALL_PREFIX).
- I use the C++ compiler flags -DMPICH_IGNORE_CXX_SEEK -DMPICH_SKIP_MPICXX (-D CMAKE_CXX_FLAGS) to work around a problem in the MPI-2 standard[2][3]. Without them, we'll get the following error:
catastrophic error: #error directive: "SEEK_SET is #defined but must not be for the C++ binding of MPI. Include mpi.h before stdio.h"
Compile and install Trilinos:
$ make -j8 install
Initially we built Trilinos 11.12 with the Netlib reference BLAS & LAPACK, which are not optimized for performance. Here is what I did:
$ cmake \ -D CMAKE_CXX_FLAGS:STRING=" -DMPICH_IGNORE_CXX_SEEK -DMPICH_SKIP_MPICXX" \ -D TPL_ENABLE_MPI=ON \ -D MPI_BASE_DIR=/opt/intel/impi/4.1.3.045/intel64 \ -D MPI_C_COMPILER:FILEPATH=mpiicc \ -D MPI_CXX_COMPILER:FILEPATH=mpiicpc \ -D MPI_Fortran_COMPILER:FILEPATH=mpiifort \ -D Trilinos_ENABLE_ALL_PACKAGES=ON \ -D TPL_BLAS_LIBRARIES:FILEPATH=/pfs/sw/serial/gcc/lapack-3.5.0/lib/libblas.a \ -D TPL_LAPACK_LIBRARIES:FILEPATH=/pfs/sw/serial/gcc/lapack-3.5.0/lib/liblapack.a \ -D CMAKE_INSTALL_PREFIX=/pfs/sw/parallel/impi_intel/trilinos-11.12.1 \ /scratch/trilinos-11.12.1-Source
There is extensive online documentation for Trilinos at its official site. There is also a hands-on tutorial hosted hosted on Google Code.
Trilinos 11.12.1 is installed at /pfs/sw/parallel/impi_intel/trilinos-11.12.1. To facilitate the usage of trilinos, I've created a module trilinos to set up environment for the synonymous library[4].
To display information about the module:
$ module show trilinos
To load the module:
$ module load trilinos