A more complete CMake build system for QuantLib - pkovacs/QuantLib GitHub Wiki

A more complete CMake build system for QuantLib

QuantLib currently utilizes three overlapping build systems: GNU Autotools, CMake and MS Visual Studio.

This effort reimplements the CMake build system for QuantLib so that it can be used in almost all scenarios, using a wide variety of available CMake generators on many diverse platforms. Much of this effort centered around making the Windows DLL build possible with CMake. This was a challenge, since QuantLib has more than a few global symbols that must be managed and a sessionId implementation that needed to be reworked. The result is a single, unified build system for all platforms.

The changes I've made use a very light touch -- just enough to address the completeness of build system and the Windows DLL problem. I did not rewrite any core implementation or algorithmic code, in fact, I went out of my way to avoid doing so.

Managing global data symbols for building a Windows DLL

CMake provides a means to export Windows symbols automatically using the flag CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS. This setting simplifies creation of a Windows DLL when using MSVC tools. That flag, however, does not export global data symbols -- those must be exported explicitly with __declspec(dllexport) and then imported with __declspec(dllimport).

Identifying the global data symbols

On Linux, you can use the nm tool to inspect global data symbols found in the read-only data section of the ELF binary:

nm -gC libQuantLib.so.0.0.0 | grep ' R ' lists the demangled global symbols, with

nm -gC libQuantLib.so.0.0.0 | grep ' R ' | wc -l reporting a count of 148, so we have 148 items to manage manually.

A problematic C++ pattern

Below is a recurring C++ pattern in QuantLib which generates incidental, unneeded globals. It should be apparent that the constants a1_ and a2_ need not be exposed by the class in this manner. Fortunately this pattern is easily modified to one which eliminates the globals:

// foo.hpp
namespace QuantLib {
class Foo {
public:
    Foo() {}
    static int Compute(int i) {
        return i + a1_ + a2_;
    }
private:
    static const int a1_;
    static const int a2_;
};
}

// foo.cpp
namespace QuantLib {
    const int a1_ = 1;
    const int a2_ = 2;
}

A better C++ pattern using constexpr

The pattern below moves the a1_ and a2_ members into the implementation unit as constexpr items and places them in a private namespace devoted to that particular class. The use of the intermediate anonymous namespace is optional. It ensures that those constants cannot be seen outside the .cpp unit. Organizing the constants in this manner also allows for the common situation where several classes are implemented in one unit. We thus know which constants belong to which classes.

This pattern has several benefits:

  • it removes the unneeded, incidental globals a1_ and a2_
  • it reduces the size of the class objects
  • it removes read-only data that the library needs to initialize when it starts
  • it pushes work back from the linker to the compiler via use of constexpr
  • it uses a very light touch and does not change any algorithmic code

Note that sometimes the new pattern removes the need for an instance to be created at all! Observe that we can issue Foo::Compute() with no instance now.

// foo.hpp
namespace QuantLib {
class Foo {
public:
    Foo() {}
    static int Compute(int i);
};
}

// foo.cpp
namespace QuantLib {
    namespace { // file scope
        namespace FooPrivate {
            constexpr int a1_ = 1;
            constexpr int a2_ = 2;
        }
    }
int Foo::Compute(int i) {
    using namespace FooPrivate;
    return i + a1_ + a2_;
}
}

The array bounds as global data pattern

Sometimes array bounds are implemented in QuantLib as a global constant, as follows:

// foo.hpp
namespace QuantLib {
class Foo {
public:
    Foo() {}
private:
    static const int M;
    int data_array[M];
};
}
// foo.cpp
namespace QuantLib {
    const int M = 100;
}

A better pattern again using constexpr

Here we just replace the global with a static constexpr and the compiler takes care of the rest.

// foo.hpp
namespace QuantLib {
class Foo {
public:
    Foo() {}
private:
    static constexpr int M = 100;
    int data_array[M];
};
}
// foo.cpp
namespace QuantLib {
    // init of M removed
}

QuantLib sessions, multi-threading and C++ static data

C++ static class data breaks through the logical session boundary that QuantLib uses to partition work across sessions. A QuantLib session is a set of singletons sharing the same session id which allows for partitioning of unrelated work. That design allow sessions to operate in different threads, but requires that we avoid using any writeable C++ static data.

In my sweep of global data, I found these two problematic cases:

// ql/money.hpp
namespace QuantLib {
class Money {
public:
    // these statics violate the session concept and create race conditions across threads
    static ConversionType conversionType;
    static Currency baseCurrency;
};
}
// ql/cashflows/iborcoupon.hpp
namespace QuantLib {
class IborCoupon {
private:
    // these statics violate the session concept and create race conditions across threads
    static bool constructorWasNotCalled_;
    static bool usingAtParCoupons_;
};

To manage this problem, I removed the static data from those classes and create new classes: MoneySettings and IborCouponSettings. I then placed instances in the Settings singleton with new access api's for those settings. Other implementations are possible, of course, including creating separate singletons. I did not want to pollute the library with more singletons, however, so I avoided doing so. The api's are substantially the same, except you need to grab the settings singleton and get or set the new sub-settings from there. The client changes become, for example:

// old
if (!IborCoupon::usingAtParCoupons()) {
// new
if (!Settings::instance().iborCouponSettings().usingAtParCoupons()) {

Shared helper instances as global data

Some global data were instances of helper classes shared as static members so that all instances can use the same helper. This approach requires that QuantLib instantiate the helper class when the library initializes. This behavior is not ideal since the helper is instantiated whether or not you actually use it. More importantly, it again creates a multi-threaded and session hazard, as described above. Since the helper classes involved were small, I removed the static designation and thus allow the helper to be instantiated only when the containing class is instantiated. Below is one example.

// ql/math/distributions/normaldistribution.hpp
namespace QuantLib {
class CumulativeNormalDistribution {
    // ...
private:
    // below gaussian is now just a normal member not a static 
    NormalDistribution gaussian_;    
};
}

Static arrays

Exposing naked pointers to internal arrays creates more global data that we need to link explicitly, so I converted these few items to function calls which return const references. After the client grabs the reference, the code is identical.

namespace QuantLib {
// the old pattern
extern "C" const long * const some_array[MAX_SIZE];

// old client code
... = some_array[i];
}
#include <array>
namespace QuantLib {
// the new pattern
const std::array<const long *, MAX_SIZE> & SomeArray()

// new client code
auto & some_array = SomeArray();  // grab reference and proceed unchanged
... = some_array[i];
}

Scope of global variables on Linux vs Windows

After resolving the global data issues above, only a few problems remain for building a QuantLib DLL on Windows. The core issue is the architectural difference for dynamic shared objects (DSO's) on Linux vs Windows. On Windows, global data is scoped to the module (DLL or EXE), but on Linux the runtime linker ensures that global data is scoped and visible to the entire process. When we create a singleton with QuantLib on Linux, as a result, the linker ensures that the instance is seen everywhere in the process. With a Windows DLL, however, this is not the case. Creating a singleton in a DLL creates an isolated instance that is "global" only within the DLL. We thus create distinct singletons in the EXE and DLL and the library does not function.

We can identify the problems again using nm on Linux:

nm -gC libQuantLib.so.0.0.0 | grep ' u ' | grep Singleton

displays the items that Linux guarantees to be globally unique, but Windows doesn't. The data items displayed are the template specializations of the Singleton class that are used as base classes. These items need to be managed.

Managing the singleton classes

In order to ensure that a unique singleton object is used across all modules in a Windows process, we must manually export and import classes derived from template specializations of the Singleton class. Fortunately there are only seven such classes, e.g. Settings, IndexManager, etc.

For these we used the standard DLL import/export technique:

// ql/qldefines.hpp
#if defined(BOOST_MSVC)
#    if defined(QL_COMPILATION)
#        define QL_EXPORT __declspec(dllexport)
#    else
#        define QL_EXPORT __declspec(dllimport)
#    endif
#else
#    define QL_EXPORT
#endif

then we just add the QL_EXPORT macro to those seven classes and, when building the QuantLib library, we set the compiler definition -DQL_COMPILATION to export the singletons. When building clients, e.g. the test suite and examples, that flag is not set and thus we import the singletons instead.

namespace QuantLib {
class QL_EXPORT Settings : public Singleton<Settings> {
// ...
};
}

Managing the sessionId function

The current sessionId implementation cannot work when using QuantLib in a Windows DLL for the architectural reasons described above: a function defined in the EXE will not be implicitly visible in the Windows DLL. Again, process-wide visibility is feature of Linux and similar OS'es, not Windows. My solution fixes this problem while keeping the spirit of the original implementation. See the code below.

My changes implement a defaultSessionId function which is the same function seen in the test suite and example programs. A session_id_function function pointer is added internally to the library and set to defaultSessionId. The original function called sessionId is now a library implemented function which calls the function indicated by the new function pointer. A new api setSessionIdFunction allows the caller to set a session id function for the library, thus altering the behavior of the sessionId function. The default session id function can be reset by calling setSessionIdFunction() with no parameters.

// sample client code
//   i.e. some non-default implementation that you require
QuantLib::Threadkey mySessionId() { return 42; }
QuantLib::setSessionIdFunction(mySessionId);
// ql/patterns/singleton.hpp
#if defined(QL_ENABLE_SESSIONS)
    // A default SessionIdFunction is provided by the library.
    // You may implement a custom sessionId function and set it as follows:
    //     setSessionIdFunction(yourSessionIdFunction);
    // Your function need not be in the QuantLib namespace, but it must have
    // the following signature:
    //     QuantLib::ThreadKey (*sessionIdFunction)()
    // To reset the library to the default sessionId functions:
    //     setSessionIdFunction();
    using SessionIdFunction = std::add_pointer<ThreadKey()>::type;
    ThreadKey sessionId();
    ThreadKey defaultSessionId();
    void setSessionIdFunction(SessionIdFunction = defaultSessionId);
#endif
namespace QuantLib {
// ql/patterns/singleton.cpp
#if defined(QL_ENABLE_SESSIONS)
ThreadKey defaultSessionId() {
    return {};
}

SessionIdFunction session_id_function = defaultSessionId;

ThreadKey sessionId() {
    return (*session_id_function)();
}

void setSessionIdFunction(SessionIdFunction sid_function) {
    QL_ASSERT(sid_function,
        "setSessionIdFunction was called with a null function");
    session_id_function = sid_function;
}
#endif

}

Setting up Boost

Requirements for QuantLib

To build QuantLib, you need only the Boost headers and two libraries: thread and unit_test_framework. The QuantLib library itself requires only Boost headers. The test suite, benchmark and example programs require unit_test_framework and/or possibly thread, depending on the QuantLib configuration.

Installing Boost

The example below installs a minimal library setup for Boost in a non-system directory. I only install the unit_test_framework and thread libraries. Additional libraries may be installed by b2 as dependencies. A word on layout tagging. Boost allows you to tag the library names in several ways. By default b2 will use system tagging which, on Windows, adorns the file names both with a compiler version and a Boost version:

libboost_unit_test_framework-vc142-mt-gd-x64-1_77.lib

I prefer to use the simpler tagged layout:

libboost_unit_test_framework-mt-gd-x64.lib

The commands below use --layout=tagged simply because that is my preference. Adjust it if you prefer different tagging. All viable configurations of static and shared are installed. b2 will issue a warning that the "shared library with static runtime combination " is not allowed. That is normal. The other combinations are viable and we can make use of them. Note too that I prefer to build Boost libraries with position independent code so that I can build PIC libraries and PIE executables which can be important for hardened systems. On Windows, PIC and PIE are not applicable.

# Boost minimal setup on Linux: installing only the thread and unit_test_framework libraries
$ export MY_INSTALL_DIR=$HOME/local
$ cd $HOME
$ mkdir boost-build
$ pushd boost-build
$ wget https://boostorg.jfrog.io/artifactory/main/release/1.77.0/source/boost_1_77_0.tar.bz2
$ tar -xjf boost_1_77_0.tar.bz2
$ cd boost_1_77_0
$ ./bootstrap.sh
$ ./b2 -d+2 --layout=tagged address-model=64 architecture=x86 variant=debug,release \
    threading=multi pch=off link=shared,static runtime-link=shared,static cxxflags=-fPIC \
    toolset=gcc --prefix=$MY_INSTALL_DIR --build_dir=build --stage_dir=stage --with-thread \
    --with-test install
popd
# On Windows, in a Native Tools Command Prompt for VS window, use the following (omit cxxflags and toolset):
$ bootstrap.exe msvc
$ b2.exe -d+2 --layout=tagged address-model=64 architecture=x86 variant=debug,release \
    threading=multi pch=off link=shared,static runtime-link=shared,static --prefix="C:\local" \
    --build_dir=build --stage_dir=stage --with-thread --with-test install

Using the Boost libraries with CMake

CMake supports Boost directly and we can use several different flags depending on our setup. Assume you have unpacked QuantLib into a directory $HOME/ql and you wish to build it in $HOME/ql-build, the following examples show how we inform CMake of the location of the Boost installation (see CMake's FindBoost documentation for more details).

cd ql-build
# Use our local Boost installation and do NOT use the system version of Boost
#    BOOST_ROOT implies a ROOT/include ROOT/lib installation tree
# We link Boost statically to the test and example programs
cmake \
    -DBOOST_ROOT:STRING="$HOME/local" \
    -DBoost_NO_SYSTEM_PATHS:BOOL=ON \
    -DBoost_USE_STATIC_LIBS:BOOL=ON \
    (... other QuantLib config settings ... ) \
    ../ql
# Use our local Boost installation and do NOT use the system version of Boost
#    Here we use specific options to specifiy the Boost include and lib dirs explicitly.
# We link Boost dynamically to the test and example programs
cmake \
    -DBOOST_INCLUDEDIR:STRING="$HOME/local/include" \
    -DBOOST_LIBRARYDIR:STRING="$HOME/local/lib" \
    -DBoost_NO_SYSTEM_PATHS:BOOL=ON \
    -DBoost_USE_STATIC_LIBS:BOOL=OFF \
    (... other QuantLib config settings ... ) \
    ../ql

Auto-link

The new CMake build system does not use Boost auto-link to build the package. Auto-link is turned off with the compiler definition -DBOOST_ALL_NO_LIB. In programs which include ql/auto_link.hpp, I added an additional check for that definition prior to including that header (MSVC only).

// test suite and example programs
#include <ql/qldefines.hpp>
#if !defined(BOOST_ALL_NO_LIB) && defined(BOOST_MSVC)
#  include <ql/auto_link.hpp>
#endif

Using the CMake build system

Requirements

  • C++11
  • CMake 3.15 (current version is 3.21.2 as of this writing)
  • Boost 1.58 with libraries unit_test_framework and thread (current version is 1.77 as of this writing)

On Windows you can install CMake directly with the Visual Studio installer (e.g. VS 2019 installer) which provides a recent version of CMake and a plethora of CMake generators. I prefer the Ninja generators as they are faster than the default VS Studio generator. Try different generators and see which ones work best for you.

Options

Below are the options I've implemented for the CMake build system. Most should be familiar. All QuantLib options are prefixed with QL_ to distinguish them from any CMake or Boost option. I removed the USE_BOOST_DYNAMIC_LIBRARIES because we already have full control of Boost linking with CMake options: Boost_USE_STATIC_LIBS and Boost_USE_STATIC_RUNTIME. I added separate options for building and installing the test suite, benchmark and example programs since you may wish to build them, for example, but not install them, etc. Many other CMake options come into play -- see the Examples section.

# Installation directories
set(QL_INSTALL_BINDIR "bin" CACHE STRING "Installation directory for executables")
set(QL_INSTALL_LIBDIR "lib" CACHE STRING "Installation directory for libraries")
set(QL_INSTALL_INCLUDEDIR "include" CACHE STRING "Installation directory for headers")
set(QL_INSTALL_EXAMPLESDIR "lib/QuantLib/examples" CACHE STRING
    "Installation directory for examples")

# Options
option(QL_BUILD_BENCHMARK "Build benchmark" ON)
option(QL_BUILD_EXAMPLES "Build examples" ON)
option(QL_BUILD_TEST_SUITE "Build test suite" ON)
option(QL_ENABLE_OPENMP "Detect and use OpenMP" OFF)
option(QL_ENABLE_PARALLEL_UNIT_TEST_RUNNER "Enable the parallel unit test runner" OFF)
option(QL_ENABLE_SESSIONS "Singletons return different instances for different sessions" OFF)
option(QL_ENABLE_SINGLETON_THREAD_SAFE_INIT "Enable thread-safe singleton initialization" OFF)
option(QL_ENABLE_THREAD_SAFE_OBSERVER_PATTERN "Enable the thread-safe observer pattern" OFF)
option(QL_ENABLE_TRACING "Tracing messages should be allowed" OFF)
option(QL_ERROR_FUNCTIONS "Error messages should include current function information" OFF)
option(QL_ERROR_LINES "Error messages should include file and line information" OFF)
option(QL_EXTRA_SAFETY_CHECKS "Extra safety checks should be performed" OFF)
option(QL_HIGH_RESOLUTION_DATE "Enable date resolution down to microseconds" OFF)
option(QL_INSTALL_BENCHMARK "Install benchmark" ON)
option(QL_INSTALL_EXAMPLES "Install examples" ON)
option(QL_INSTALL_TEST_SUITE "Install test suite" ON)
option(QL_TAGGED_LAYOUT "Library names use layout tags" ${MSVC})
option(QL_USE_DISPOSABLE "Use the Disposable class template. Not needed for C++11" OFF)
option(QL_USE_INDEXED_COUPON "Use indexed coupons instead of par coupons" OFF)
option(QL_USE_STD_CLASSES "Enable all QL_USE_STD_ options" OFF)
option(QL_USE_STD_SHARED_PTR "Use standard smart pointers instead of Boost ones" OFF)
option(QL_USE_STD_UNIQUE_PTR "Use std::unique_ptr instead of std::auto_ptr" ON)
option(QL_USE_STD_FUNCTION "Use std::function and std::bind instead of Boost ones" OFF)
option(QL_USE_STD_TUPLE "Use std::tuple instead of boost::tuple" OFF)

Library name tagging

QL_TAGGED_LAYOUT is a new option which tags the QuantLib library name in a similar way to that of Boost's b2 with the option --layout=tagged, e.g.

QuantLib-mt-sgd-x64

You can use your own suffix if you choose (auto-link won't recognize custom names however). Simply turn off QL_TAGGED_LAYOUT and manually specify CMAKE_DEBUG_POSTFIX and/or CMAKE_RELEASE_POSTFIX, for example:

cmake \
    -DQL_TAGGED_LAYOUT:BOOL=OFF \
    -DCMAKE_BUILD_TYPE:STRING="Debug" \
    -DCMAKE_DEBUG_POSTFIX:STRING="-experiment42-debug" \
    (... more options ...)
# produces a QuantLib library named
QuantLib-experiment42-debug

Examples

All examples assume QuantLib is unpacked to a directory ql and will be built in a directory ql-build at the same level (out of the source tree). On Windows we open a Native Tools Command Prompt for Visual Studio window.

All examples assume Boost is installed to C:\local on Windows and $HOME/local on Linux.

CMake generators are either single-configuration or multi_configuration. If the single-config (make, nmake, etc.), the CMake variable CMAKE_BUILD_TYPE is used. For multi-config generators (Visual Studio, Ninja Multi-Config, etc.), that option is ignored and the option --config= determines the build type.

Build Windows DLL: Debug w/static Boost, Ninja multi-config generator

cd ql-build
cmake \
    -DBUILD_SHARED_LIBS:BOOL=ON \
    -DCMAKE_INSTALL_PREFIX:STRING="C:\local" \
    -DBOOST_ROOT:STRING="C:\local" \
    -DBoost_USE_STATIC_LIBS:BOOL=ON \
    -DBoost_USE_STATIC_RUNTIME:BOOL=OFF \
    -DQL_BUILD_EXAMPLES:BOOL=OFF \
    -DQL_INSTALL_BENCHMARK:BOOL=OFF \
    -DQL_INSTALL_TEST_SUITE:BOOL=OFF \
    -G"Ninja Multi-Config" \
    ../ql
cmake --build . --config=Debug --verbose --parallel 8
cmake --build . --config=Debug --verbose --target install

Build static Windows lib: RelWithDebInfo w/static Boost, Visual Studio multi-config generator

cd ql-build
cmake \
    -DBUILD_SHARED_LIBS:BOOL=OFF \
    -DCMAKE_INSTALL_PREFIX:STRING="C:\local" \
    -DBOOST_ROOT:STRING="C:\local" \
    -DBoost_USE_STATIC_LIBS:BOOL=ON \
    -DBoost_USE_STATIC_RUNTIME:BOOL=ON \
    -DQL_BUILD_EXAMPLES:BOOL=OFF \
    -DQL_INSTALL_BENCHMARK:BOOL=OFF \
    -DQL_INSTALL_TEST_SUITE:BOOL=OFF \
    ../ql
cmake --build . --config=RelWithDebInfo --verbose --parallel 8
cmake --build . --config=RelWithDebInfo --verbose --target install

Build shared Linux DSO: Release w/dynamic Boost, make single-config generator

cd ql-build
# The rpath options ensure the test suite, bench and examples are linked to our local installation
# in the event either Boost or another version of QuantLib are in the system path
cmake \
    -DBUILD_SHARED_LIBS:BOOL=ON \
    -DCMAKE_BUILD_TYPE:STRING="Release" \
    -DCMAKE_INSTALL_PREFIX:STRING="${HOME}/local" \
    -DCMAKE_INSTALL_RPATH:STRING="$HOME/local/lib" \
    -DCMAKE_INSTALL_RPATH_USE_LINK_PATH:BOOL=ON \
    -DBOOST_ROOT:STRING="${HOME}/local" \
    -DBoost_NO_SYSTEM_PATHS:BOOL=ON \
    -DBoost_USE_STATIC_LIBS:BOOL=OFF \
    -DBoost_USE_STATIC_RUNTIME:BOOL=OFF \
    ../ql
cmake --build . -verbose --parallel 8
cmake --build . --verbose --target install

Build a source package

cd ql-build
cmake \
    (.. your options ...)
    ../ql
# Build all three (tar.gz, .zip and .7Z)
cmake --build . --verbose --target package_source
# Build just 7z (or ZIP or TGZ)
cpack -G 7Z --config CPackSourceConfig.cmake

Philip Kovacs

⚠️ **GitHub.com Fallback** ⚠️