Trilinos - shawfdong/hyades GitHub Wiki

The Trilinos Project is an effort to develop algorithms and enabling technologies within an object-oriented software framework for the solution of large-scale, complex multi-physics engineering and scientific problems. A unique design feature of Trilinos is its focus on packages.

Table of Contents

Building Trilinos 11.12.1

With Intel MKL

The Trilinos 11.12 release uses the CMake build system, which requires CMake version 2.8.11 or greater. Trilinos requires BLAS and LAPACK. We'll build Trilinos 11.12.1 with Intel MKL & Intel MPI using Intel compilers.

Download Trilinos 11.12.1: 

$ cd /scratch
$ wget http://trilinos.org/oldsite/download/files/trilinos-11.12.1-Source.tar.bz2
$ tar xvfj trilinos-11.12.1-Source.tar.bz2

Create a build directory: 

$ mkdir trilinos-11.12.1-Build
$ cd trilinos-11.12.1-Build

Configure Trilinos[1]: 

$ cmake \
-D CMAKE_CXX_FLAGS:STRING=" -DMPICH_IGNORE_CXX_SEEK -DMPICH_SKIP_MPICXX" \
-D CMAKE_BUILD_TYPE:STRING=RELEASE \
-D CMAKE_VERBOSE_MAKEFILE:BOOL=ON \
-D Trilinos_VERBOSE_CONFIGURE:BOOL=ON \
-D TPL_ENABLE_MPI=ON \
-D MPI_BASE_DIR=/opt/intel/impi/4.1.3.045/intel64 \
-D MPI_C_COMPILER:FILEPATH=mpiicc \
-D MPI_CXX_COMPILER:FILEPATH=mpiicpc \
-D MPI_Fortran_COMPILER:FILEPATH=mpiifort \
-D Trilinos_ENABLE_ALL_PACKAGES=ON \
-D BLAS_INCLUDE_DIRS:PATH=$MKLROOT/include/ \
-D BLAS_LIBRARY_DIRS:PATH=$MKLROOT/lib/intel64/ \
-D LAPACK_LIBRARY_DIRS:PATH=$MKLROOT/lib/intel64/ \
-D BLAS_LIBRARY_NAMES:STRING="mkl_intel_lp64;mkl_core;mkl_sequential" \
-D LAPACK_LIBRARY_NAMES:STRING="mkl_intel_lp64;mkl_core;mkl_sequential" \
-D BUILD_SHARED_LIBS:BOOL=ON \
-D CMAKE_INSTALL_PREFIX=/pfs/sw/parallel/impi_intel/trilinos-11.12.1 \
/scratch/trilinos-11.12.1-Source
Note:
  1. Trilinos 11.12.1 is installed at /pfs/sw/parallel/impi_intel/trilinos-11.12.1 (-D CMAKE_INSTALL_PREFIX).
  2. I use the C++ compiler flags -DMPICH_IGNORE_CXX_SEEK -DMPICH_SKIP_MPICXX (-D CMAKE_CXX_FLAGS) to work around a problem in the MPI-2 standard[2][3]. Without them, we'll get the following error:
catastrophic error: #error directive: "SEEK_SET is #defined but must not be for the C++ binding of MPI. Include mpi.h before stdio.h"

Compile and install Trilinos: 

$ make -j8 install

With Netlib LAPACK

Initially we built Trilinos 11.12 with the Netlib reference BLAS & LAPACK, which are not optimized for performance. Here is what I did: 

$ cmake \
-D CMAKE_CXX_FLAGS:STRING=" -DMPICH_IGNORE_CXX_SEEK -DMPICH_SKIP_MPICXX" \
-D TPL_ENABLE_MPI=ON \
-D MPI_BASE_DIR=/opt/intel/impi/4.1.3.045/intel64 \
-D MPI_C_COMPILER:FILEPATH=mpiicc \
-D MPI_CXX_COMPILER:FILEPATH=mpiicpc \
-D MPI_Fortran_COMPILER:FILEPATH=mpiifort \
-D Trilinos_ENABLE_ALL_PACKAGES=ON \
-D TPL_BLAS_LIBRARIES:FILEPATH=/pfs/sw/serial/gcc/lapack-3.5.0/lib/libblas.a \
-D TPL_LAPACK_LIBRARIES:FILEPATH=/pfs/sw/serial/gcc/lapack-3.5.0/lib/liblapack.a \
-D CMAKE_INSTALL_PREFIX=/pfs/sw/parallel/impi_intel/trilinos-11.12.1 \
/scratch/trilinos-11.12.1-Source

Using Trilinos

There is extensive online documentation for Trilinos at its official site. There is also a hands-on tutorial hosted hosted on Google Code.

Trilinos 11.12.1 is installed at /pfs/sw/parallel/impi_intel/trilinos-11.12.1. To facilitate the usage of trilinos, I've created a module trilinos to set up environment for the synonymous library[4].

To display information about the module: 

$ module show trilinos

To load the module: 

$ module load trilinos

References

  1. ^ Trilinos Configure, Build, Test, and Install Quick Reference Guide
  2. ^ I get compile errors saying "SEEK_SET is #defined but must not be for the C++ binding of MPI".
  3. ^ Error when compiling C++ aps using Intel® MPI Library compilation driver mpiicpc
  4. ^ Environment Variables Affecting GCC
⚠️ **GitHub.com Fallback** ⚠️