Building from Source - sys-bio/roadrunner GitHub Wiki
For the latest build instructions we refer you to:
Build Instructions for roadRunner
This guide assumes basic familiarity with git and command line. Please consult other tutorials if you need more information on these tools.
- Unless you want to contribute to RoadRunner's development, you may wish to take the easier option of simply installing via PyPI:
pip install libroadrunner
- This guide assumes familiarity with the following tools:
- Git
- CMake (you can use any front-end, e.g. GUI or console)
- If you are not familiar with CMake, check out our intro to CMake
- Since these instructions are platform-generic, you should have experience with your target toolchain. For example, if you are building using Visual Studio, you should know how to build the "INSTALL" target. Roadrunner supports three different toolchains (Visual Studio, Xcode, and Makefiles).
You may want to build the release version without the Visual Studio or other IDE setup that CMake gives you or because other tools like Ninja can parallelize builds faster. On windows this can be a tiny bit tricky if you want to use the MSVC toolchain.
If you run CMake from the command line, like with
cmake -G Ninja ..\..\source\libroadrunner-deps
CMake won't be able to find tools like mc
and cl
. You can't just add them to your path either because their path includes the Visual Studio version number, so when you upgrade you will still be using the old tooling (or worse).
Instead, run that same command under the Developer Command Line. You need to make sure that it's the correct command line for your target. So, if you're on a 64-bit, you need to run the x64 Native Developer Command Line. You can achieve the same thing by just running vcvarsall.bat
with the appropriate argument (e.g. x64
), which will set the appropriate environment variables. It's just a batch script that sets up environment variables with the correct versions of MSVC tools. Then run the CMake build command:
cmake --build . --config Release
-
You must have the message compiler
mc.exe
somewhere on your system. It is most likely underC:\Program Files (x86)\Windows Kits
, so do a search there and be prepared to use the file path to that executable when building dependencies. -
The
CMAKE_BUILD_TYPE
parameter will only work on the latest version of Visual Studio 2017 or higher
- If you get error
LNK 2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL'...
it's probably because one of the projects was built in Debug mode, and another in Release mode.2
is debug, and0
is release. You probably want everything on Release mode. They have different implementations of STL structures, so they are incompatible.
You will want to create a directory to house the entire roadrunner build, including not just this repo but also the libroadrunner-deps repo and LLVM. You should start by creating this file structure somewhere not admin-locked
<root>
├── build
│ ├── llvm
│ ├── libroadrunner-deps
│ └── roadrunner
├── install
│ ├── llvm
│ └── roadrunner
└── source
<root>
is the path to the folder that you put all of this in. We will refer back to it when needed.
Where should root be? Should it be in the roadrunner directory created when the repo was cloned?
You may want to create separate debug and release build and install folders if you are planning to use both build types, for example "build_debug" and "build_release" instead of just "build"
Note: install libroadrunner-deps to '$PREFIX/install/roadrunner' not '$PREFIX/install/libroadrunner-deps'
Building LLVM can take over 25 GB for both Release and Debug build files
Building the entire project: LLVM, roadrunner, and its dependencies for Debug and Release takes about 35 GB in total including source, build files, and installation directory
If you want to have separate build and install directories for debug vs release versions, that can easily go over 40-45 GB. In order to mitigate this, you can build & install the debug/release versions of libroadrunner-deps, since it changes so infrequently, then delete the build files.
There are three repositories: roadrunner, libroadrunner-deps, and llvm. Each of these should live in their respective folders under <prefix>/source
. For now, there will be nothing under build
or install
. Later we will put the CMake generated build files in build
, and after compiling some components will be installed to install
.
If you don't know how to clone a repository from Github, now is a perfect time to learn. Here is a good introduction to command line and Git
- Roadrunner has two sets of dependencies: LLVM and non-LLVM. LLVM can be built via a single CMake script. We also provide a single CMake script for building the rest of the dependencies.
LLVM 6.0 support was added in late 2019. We have created our own fork of the project, since a few things needed to be tweaked to better support roadrunner's use of it. The local fork is at https://github.com/sys-bio/llvm-6.x and was created via the following method (in case anyone needs to reproduce this in the future, if we upgrade to a later version of llvm). After creating a 'llvm-6.x' project on github:
git clone --bare --single-branch --branch release/6.x https://github.com/llvm/llvm-project.git
cd llvm-project.git/
git push --mirror https://github.com/sys-bio/llvm-6.x.git
Now that that was done, all you need to do is:
git clone https://github.com/sys-bio/llvm-6.x.git
Into what directory do we clone this into
If I clone into build I get llvm-6.x with another llvm directory inside that, then the rest of the llvm code inside that directory. what happens to the llvm directory I created in the section above, called RR directories? Is that reserved for later on?
I don't know what to do with the prebuilt binaries
Don't worry, the links are actually dead! So move on and hope for the best
Here are pre-built windows LLVM-6 libraries: llvm-release.tar.gz (233MB) llvm-release.7z (130 MB) llvm-debug.tar.gz (288MB) llvm-debug.7z (439MB)
- First, make sure that you have an up-to date version of CMake that has generators that correspond to the development
environment that you will be using. For example, if you use Visual Studio 2017, you will need CMake at least above 3.0
- Fun note: If you do need to install a newer version of CMake, it won't uninstall the older version, so make sure you're not just using the old version again.
- Make sure that you are using the same generators and compilers for all of the build process
- (We recommend using CMake to do an out-of-source build of LLVM. Install it to a non-system-wide directory. The Roadrunner team never uses Autotools to compile LLVM.)
- In CMake, change the
CMAKE_INSTALL_PREFIX
to<root>/install/llvm
- After you have generated the build files for LLVM, build the
INSTALL
project. - Once it is finished compiling, check
<root>/install/llvm/bin
and verify thatllvm-config
is there
-
CMAKE_INSTALL_PREFIX
set to<prefix>/install/llvm
-
LLVM_TARGETS_TO_BUILD
set toX86
The following options are turned on by default in the forked version of the source, but not in the original:
-
LLVM_ENABLE_EH
turned on. -
LLVM_OPTIMIZED_TABLEGEN
turned on. -
LLVM_ENABLE_RTTI
turned on.
Do the following instructions apply to Windows as well?
- In the directory
<root>/source
, rungit clone https://github.com/sys-bio/llvm-6.x.git
- This will create a directory
<root>/source/llvm-6.x/llvm/
. - Change directory to
<root>/build/llvm
. - Configure with
cmake ../../source/llvm-6.x/llvm/ -DCMAKE_INSTALL_PREFIX=../../install/llvm -DLLVM_TARGETS_TO_BUILD=X86
. - Compile with
make
. - Install with
make install
. This installs LLVM to /install/llvm.
- All dependencies other than LLVM are hosted in their own repository: https://github.com/sys-bio/libroadrunner-deps
- In the directory
<root>/source
, rungit clone https://github.com/sys-bio/libroadrunner-deps.git
- This will create a directory
<root>/source/libroadrunner-deps
. - Run CMake with your desired generator and options. Select
<root>/source/libroadrunner-deps
as your source directory and<root>/build/libroadrunner-deps
as your build directory. Valid generates are Visual Studio 2015 64- or 32-bit, Xcode, and Unix Makefiles. Refer to the CMake manual on how to set options. The required options are as follows:
CMAKE_BUILD_TYPE=Release
CMAKE_INSTALL_PREFIX=<root>/install/roadrunner
- Set
LLVM_CONFIG
to the path to the llvm-config executable, which should be under<prefix>/install/llvm/bin
- If you are on Windows, you will need to pass in the path to your
mc.exe
You can find it underC:\Program Files(x86)\Windows Kits\
in one of the directories. Try searching, and you should find it.
CMAKE_MC_COMPILER=C:/Program Files (x86)/Windows Kits/[SOME OS NUMBER]/bin/x64/mc.exe
- Build and install the project. For Visual Studio, run the INSTALL target. For Xcode, select the install target and hit run. For a Makefile build, run
make install
.
-
LLVM_CONFIG
set to the appropriate path (see above) -
CMAKE_INSTALL_PREFIX
set to<prefix>/install/roadrunner
-
CMAKE_BUILD_TYPE
set to Release -
CMAKE_MC_COMPILER
properly set if on Windows
Note: when generating for Visual Studio on release mode, set VS to Release mode if it is not already. It may
not be by default even if CMAKE_BUILD_TYPE
is set to Release
- Change directory to
<root>/source/
. - Clone libroadrunner-deps:
git clone https://github.com/sys-bio/libroadrunner-deps.git
. - Change directory to
<root>/build/libroadrunner-deps
. - Configure with:
cmake ../../source/libroadrunner-deps/ -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=../../install/roadrunner
. - Make with
make
. - This might fail if libxml2 or other dependencies are missing. If so, install them and try again.
- Install with
make install
. This installs to<root>/install/roadrunner
.
- Navigate to
<root>/source
and rungit clone https://github.com/sys-bio/roadrunner.git
- This will create a directory
<root>/source/roadrunner
. Run CMake with your desired generator (Makefiles, XCode, or Visual Studio). The generator should be the same as the one used for the dependencies. Select<root>/source/roadrunner
as your source directory. Select `/build/roadrunner as your build directory. Set the following options:
CMAKE_BUILD_TYPE=Release (or Debug for a debug build)
CMAKE_INSTALL_PREFIX=<root>/install/roadrunner
LLVM_CONFIG_EXECUTABLE=/path/to/llvm-6.x/bin/llvm-config
THIRD_PARTY_INSTALL_FOLDER=<root>/install/roadrunner
RR_USE_CXX11=OFF
USE_TR1_CXX_NS=OFF
LIBSBML_LIBRARY=<absolute-path-to-libsbml> # May need to be set manually
LIBSBML_STATIC_LIBRARY=<absolute-path-to-static-libsbml> # May need to be set manually
Generate, then open the solution and build INSTALL for a VS build. Run make install
for a Makefile build.
Note: when generating for Visual Studio on release mode, set VS to Release mode if it is not already. It may
not be by default even if CMAKE_BUILD_TYPE
is set to Release
- Change directory to
<root>/source/
. - Clone roadrunner with
git clone https://github.com/sys-bio/roadrunner.git
. - Configure with
cmake ../../source/roadrunner/ -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=../../install/roadrunner/ -DLLVM_CONFIG_EXECUTABLE=../../install/llvm/bin/llvm-config -DTHIRD_PARTY_INSTALL_FOLDER=../../install/roadrunner/ -DRR_USE_CXX11=OFF -DUSE_TR1_CXX_NS=OFF -DCMAKE_CXX_FLAGS="-lcurses"
. The -lcurses option shouldn't be needed but was for me. - Compile with
make
. - Install with
make install
.
- To build the Python wrapper, add the following options. These paths will, in general, be different across systems. Therefore, please make note of how your specific Python installation is configured and substitute the appropriate paths for the interpreter, library, and include directory below. If you are using Python3, this information may be easily obtained from the sysconfig module.
- Note that each of the Python paths must be linked to the same installation, even if they are both 2.7 or 3.4, if you have multiple Python installations, please verify that these paths have the same prefix.
- You will also have to install the SWIG executable and link to it in CMake. Specifically, set:
SWIG_DIR=<swig-main-folder>/Lib
SWIG_EXECUTABLE=<swig-main-folder>/swig.exe
- Building the Python wrapper in Debug mode is difficult because it wants to look for Python debug libraries and you may not have those (see below)
- Sometimes you may have some trouble with your specific installation of Python, but if you have Spyder for Tellurium installed you can always compile against Tellurium's Python installation and you probably won't get Tellurium specific errors
BUILD_PYTHON=ON
PYTHON_EXECUTABLE=/path/to/python-interp (python.exe)
PYTHON_LIBRARY=/path/to/python-lib (the lib that includes the version number, i.e. python36.lib for Python 3.6.
For linux, supply the path to the lib directory with the version number i.e. python3.6 for Python 3.6)
PYTHON_INCLUDE_DIR=/path/to/python/include/dir
- Build and install the project. For Visual Studio, run the INSTALL target. For Xcode, select the install target and hit run. For a Makefile build, run
make install
. - You can verify that your
roadrunner.py
and_roadrunner.pyd
were correctly compiled, using a pre-built Spyder for Tellurium distribution. If you look in the Tellurium install directory under,python-2.7.13.amd64/Lib/site-packages/roadrunner
, you should find those two files. Save a copy of them, and copy your compiled versions into that directory. If you start Tellurium, and can run a model fine, then you have correctly built the Python wrapper.
There are currently some issues with building the Python wrappers in debug mode. Namely, the linker tries to look for Python debug library "python36_d.lib" and creates all sorts of problems during linking and runtime.
However, at least for Windows, there seems to be a way to circumvent this issue by building roadrunner under "RelWithDebInfo" configuration. This prevents the linker from looking for Python debug libraries while still allowing for debugging. Here are the steps:
- Build LLVM and libroadrunner-deps in Release mode
- In the cmake step for roadrunner, set CMAKE_BUILD_TYPE to Release (and enable build Python, etc. as in the previous section)
- Open "rr.sln". In Visual Studio, select RelWithDebInfo configuration on the top bar.
- Now build the INSTALL target.
- Run
python setup.py bdist_wheel
ininstall/roadrunner
and install the wheel usingpip
.
After this, if one wishes to stop at breakpoints in the C++ code while running a Python program:
- In VS, create an empty project in "rr.sln" called "python-debug". In Configuration Properties -> Debugging, set "Command" to the path to
python.exe
(it should be the same interpreter used in building). - Set "Command Arguments" to the path of the Python script to be debugged.
- Set "Working Directory" optionally for I/O.
- Finally, in Solution Explorer, set "python-debug" as startup project. Now when the script is run from the top bar, the debugger will stop at breakpoints set in the C++ source.