PnetCDF - shawfdong/hyades GitHub Wiki

Parallel netCDF (PnetCDF) is a library providing high-performance parallel I/O while still maintaining file-format compatibility with Unidata's NetCDF, specifically the formats of CDF-1 and CDF-2. Although NetCDF supports parallel I/O starting from version 4, the files must be in HDF5 format. PnetCDF is currently the only choice for carrying out parallel I/O on files that are in classic formats (CDF-1 and 2).

In addition, PnetCDF supports the CDF-5 file format, an extension of CDF-2, that supports more data types and allows users to define large dimensions, attributes, and variables (>2B elements).

NetCDF gives scientific programmers a self-describing and portable means for storing data. However, prior to version 4, netCDF does so in a serial manner. By making some small changes to the netCDF APIs, PnetCDF can use MPI-IO to achieve high-performance parallel I/O.

Building PnetCDF 1.5.0

We'll build PnetCDF 1.5.0 with Intel MPI using Intel compilers[1].

Download the latest stable release of PnetCDF: 

$ cd /scratch
$ wget http://cucis.ece.northwestern.edu/projects/PnetCDF/Release/parallel-netcdf-1.5.0.tar.bz2
$ tar xvfj parallel-netcdf-1.5.0.tar.bz2
$ cd parallel-netcdf-1.5.0

Configure PnetCDF 1.5.0: 

$ export MPICC=mpiicc
$ export MPICXX=mpiicpc
$ export MPIF77=mpiifort
$ export MPIF90=mpiifort
$ export CC=icc
$ export CXX=icpc
$ export FC=ifort
$ export F77=ifort
$ export CXXFLAGS="-g -O2 -DMPICH_IGNORE_CXX_SEEK -DMPICH_SKIP_MPICXX"
$ export FFLAGS="-g -O2"
$ export FCFLAGS="-g -O2"
$ export F90FLAGS="-g -O2"
$ ./configure --prefix=/pfs/sw/parallel/impi_intel/parallel-netcdf-1.5.0
Note
  1. The defaults for CFLAGS & CXXFLAGS are -g -O2; but the defaults for FFLAGS & FCFLAGS & F90FLAGS are -g.
  2. We set the Fortran compiler flags to -g -O2 too.
  3. I use the C++ compiler flags -DMPICH_IGNORE_CXX_SEEK -DMPICH_SKIP_MPICXX (CXXFLAGS) to work around a problem in the MPI-2 standard[2][3]. Without them, we'll get the following error:
catastrophic error: #error directive: "SEEK_SET is #defined but must not be for the C++ binding of MPI. Include mpi.h before stdio.h"

Compile PnetCDF 1.5.0: 

$ make all

I got an error! 

/opt/intel/impi/4.1.3.045/intel64/bin/mpiicpc -c -g -O2 -DMPICH_IGNORE_CXX_SEEK -DMPICH_SKIP_MPICXX -I. -I. -I../lib -DHAVE_CONFIG_H  ncmpi_notyet.cpp
ncmpi_notyet.cpp(7): error: identifier "printf" is undefined

There is a bug in src/libcxx/ncmpi_notyet.cpp: the C++ codes calls repeatedly the C function printf, but it doesn't include the header stdio.h. The bug can be easily fixed. Add the following line to src/libcxx/ncmpi_notyet.cpp: 

#include <stdio.h>

Then recompile PnetCDF: 

$ make all
It succeeded without a hitch.

Install PnetCDF 1.5.0: 

$ make install

Using PnetCDF

There is extensive online documentation as well as a quick tutorial for PnetCDF at its official site.

PnetCDF 1.5.0 is installed at /pfs/sw/parallel/impi_intel/parallel-netcdf-1.5.0 on Hyades. To facilitate the usage of PnetCDF, I've created a module pnetcdf to set up environment for the PnetCDF library[4]. Note the environment variable FPATH sets the path for include and module files for Intel Fortran Compiler.

To display information about the module: 

$ module show pnetcdf

To load the module: 

$ module load pnetcdf

References

  1. ^ PnetCDF Installation Guide
  2. ^ I get compile errors saying "SEEK_SET is #defined but must not be for the C++ binding of MPI".
  3. ^ Error when compiling C++ aps using Intel® MPI Library compilation driver mpiicpc
  4. ^ Environment Variables Affecting GCC
⚠️ **GitHub.com Fallback** ⚠️