Including Dependencies - mjhaskell/cpp_tutorials GitHub Wiki

There are several ways to include dependencies into a project, so we will discuss a few of them here with their pros and cons:

1. Include System Installed Packages
2. Include Projects from Subdirectory
3. Include Online Git Projects

In general, there are more steps than just including the dependency. There are 3 common commands you might need to use, depending on the situation, where the first argument is the target you are configuring:

1. add_dependencies(): specifies build order - list all targets that your target depends on so they are built first
2. target_include_directories(): adds to target's include path - list all directories from which you will need to include header files
3. target_link_libraries(): adds to target's linker path - list all library targets your target uses so the linker knows how to find function definitions

add_dependencies() probably only needs to be used if the dependencies are being built at the same time as your target. Often times if you didn't include this line when you should have then your build will break the first time you try it...but that first attempt likely built the dependency, so a 2nd built attempt can be successful. It is annoying to have to build twice, so if you find this error, make sure to use add_dependencies() to fix the issue.

target_include_directories() (or the global equivalent include_directories()) will most likely be required. If you try to #include header files from your dependency (e.g. #include <Eigen/Core>) then the directory containing the header files will need to be added to your target's include path.

target_link_libraries() will need to be used if you are going to use a library from the dependency that is already compiled. This means that header only libraries, like Eigen, do not need to use this function.

The dependency might have instructions online or in a README that explain how you can properly include it in your project. If not, you might need to dig through their CMakeLists.txt to find the variable names of the targets and directories you need to include.

Packages that have been installed system-wide with a package manager like apt or that were cloned, built, and installed manually can be included with the CMake function find_package(), that is if the package was configured for CMake. Some example packages include Eigen, OpenCV, GTest, and catkin. Here is a simple use case:

find_package(Eigen3)

This function looks on the system path for CMake config files to set up the dependency correctly. A package can be declared as REQUIRED or OPTIONAL. An example of an optional package is GTest, which is only needed if the user wants to build and run the tests. Specific components can be included as well as such:

find_package(catkin REQUIRED COMPONENTS
    roscpp
    stdmsg
    )

There are other things you can do with find_package(), but I won't be able to list them all here. I will at least mention that you can specify a specific version and you can check if the dependency was found using the <package>_FOUND variable. This is useful to set up optional packages, like building tests if GTest_FOUND, and you could potentially write a script to acquire the missing dependency if it isn't on the system. Check CMake's documentation for more details. There are so many versions of the documentation that I am not including a link here, but a quick Google search of "cmake find_package" will bring it up.

Once the package is found, you will need to link the targets and include the directories you want to use from the dependency. Generally, find_package() will define the variables ${<package>_INCLUDE_DIRS} and ${<package>_LIBRARIES}>, so after declaring an executable you would do:

add_executable(my_exe ...)
target_include_directories(my_exe PRIVATE ${EIGEN3_INCLUDE_DIRS})
#target_link_libraries(my_exe ${EIGEN3_LIBRARIES})

I commented out the last line because Eigen is header only and doesn't have a compiled library I need to link to; you could uncomment the last line and everything would work fine because ${EIGEN3_LIBRARIES} is an empty variable. Other libraries will require the last line though, so I just wanted to show an example.

Pros:

• Very easy to use (if the package is installed system-wide)
• System packages are already built so your project doesn't have to spend time compiling the dependency

Cons:

• Causes build errors if REQUIRED packages are not found
• CMake will not do anything to install missing packages
• User doesn't have instructions to aquire missing dependencies
• User may have a different version of the package installed to the system causing incompatibilities

Probably only common, highly used, and stable packages should be installed to the system. The package also needs to be set up with the CMake build system so that find_package() can actually find the package. Some people argue that a build should be fully "Hermetic" (meaning that all dependencies are built with the project) so that anyone using the project is garaunteed to have all of the dependencies. I think this makes sense for a company selling a product because I'm sure customers would be upset if they found out they were missing a dependency somewhere. For a research lab at a university, I don't know if it is as big of a deal as long as a majority of the lab use the package in question regularly. These could be things like Eigen, GTest, and OpenCV, but it might prove difficult to ensure that everyone always has the same version of a package (only a problem if the package doesn't maintain backwards compatibility or if newer features are used). In summary, there may be a few packages a research lab decides to use with find_package() (always including instructions to aquire the dependency in the project's README), but it probably should not just be the "go to" option. Also, it likely isn't the best option for research code to be installed to the system.

Side Note: If you are interested in creating an installable CMake package, you have to set up install instructions in the CMakeLists.txt file and create a file in CMake's path so it can be found. I haven't done it myself and won't go into full detail here, but you can look in places like /usr/lib/cmake and /usr/share/cmake to find examples.

It is common to have subdirectories in a project with their own CMakeLists.txt. In fact, some people say that every directory with source code should have its own CMakeLists.txt. I don't always see a CMakeLists in src, but I have often seen it inside of test and in lib/<package>. In a top-level (or just higher level) CMakeLists.txt, you can add subdirectories with their own CMakeLists.txt file using the add_subdirectory() command. This function simply sources the CMakeLists.txt inside of the subdirectory and adds the configuration to the parent project, meaning that targets (libraries and executables) in the subdirectory are available for use. The usage is fairly straight-forward:

add_subdirectory(test)
add_subdirectory(lib/<package>)

It is quite common for this method to be used in conjunction with Git, such as submodules. Often, libraries are built to be used in several different projects where the library is its own Git repo. The library repo can be cloned into a project's lib directory as a Git submodule (or other similar methods). As long as the library has its own CMakeLists.txt, then the add_subdirectory() command above will include it into the parent project.

It is possible that the subdirectory defined the ${<package>_INCLUDE_DIRS} and ${<package>_LIBRARIES} variables to make it easy for you to use, but you might need to dig through its CMakeLists.txt to find the information for the 3 common functions mentioned at the top of this file.

Pros:

• Very easy to use
• CMake manages the build process of the subdirectory
• Dependency is contained entirely in the project at a specific version so everyone using the project is working with the same code
• Each project can include the subdirectory at a specific version

Cons:

• The subdirectory must explicitly be a part of the project's file system even if the user will never have need to edit files in the subdirectory
• Each project can include the subdirectory at a specific version, so it is possible for a copy of the same code to exist in several locations within a single project (especially when using ROS)

Use add_subdirectory() whenever it makes sense for the subdirectory to exist in the file system of the project. For libraries in lib, this would be something that might be developed in conjuction with the project. In my opinion, Git submodules seem to hint that development is occuring within the submodule that the project wants to track and that might need to be updated. Things that probably will not need to update or be developed by the user don't need to be included in a project's file system. Therefore, in my opinion again, add_subdirectory() is not really the best option in that case. Examples include Eigen, GTest, OpenCV, and any other external library that you aren't developing yourself.

CMake can clone a repo directly into the build directory where it is available and out of sight. There are multiple ways to go about this and the process is somewhat unique to each online dependency, making this method a bit more complicated. This is probably the "best practice" method for dependencies that are not being developed along with the project even though it can be a little more complicated. The 2 main utilities are ExternalProject and FetchContent. For more information on these 2 utilities, see here. You can also look at CMake's documentation.

ExternalProject is a CMake module used to populate content from the online dependency at build time. This means that only configuration happens when you run cmake <location> and the content is cloned and compiled when you run make. Typically, you wouldn't add all of the necessary steps inside of your project's CMakeLists.txt but rather store them in a separate file and placed in a folder called "cmake". I have usually seen the files named as cmake/External_<package>.cmake. To include this file for Eigen in your project you just use this line:

include(cmake/External_Eigen3.cmake)

The file External_Eigen3.cmake would look like this:

cmake_minimum_required(VERSION 3.0.2)

include(ExternalProject)

ExternalProject_Add(eigen_ext
  GIT_REPOSITORY  https://gitlab.com/libeigen/eigen.git
  GIT_TAG         3.3.7
  UPDATE_COMMAND  ""
  BUILD_COMMAND   ""
  INSTALL_COMMAND ""
  LOG_DOWNLOAD ON
  LOG_CONFIGURE ON
  )

ExternalProject_Get_Property(eigen_ext source_dir)
ExternalProject_Get_Property(eigen_ext binary_dir)
set(EIGEN3_INCLUDE_DIRS ${source_dir})
set(EIGEN_INCLUDE_DIR ${source_dir})
set(EIGEN_INCLUDE_DIRS ${source_dir})
set(EIGEN_INCLUDE_DIR_HINTS ${source_dir})

First, notice that you need to do include(ExternalProject) in order to use functions from that module. Then we create a new target called eigen_ext using the ExternalProject_Add() function. This function needs to be configured on a case-by-case basis, which is one of the difficulties of using ExternalProject. This function is where you provide instructions for where the git repository exists online, which branch/tag you want to use, and then steps for how to build, install and other things. After creating the target, you kind of need to replicate that project's CMake configuration by adding the libraries/executables the dependency as IMPORTED targets, and you also need to create all of the CMake variables that your project needs from the dependency. These steps can be tricky to figure out, but you have to do all of this because CMake will be completely unaware of these settings when you run cmake <location> because the files will not be downloaded to your machine until you run make. This can be a bit inconvenient to replicate the CMakeLists.txt of the dependency, but on the other hand, I find it really nice that all of the building and action steps don't happen until I run make...meaning that I know the cmake <location> command will be really quick.

NOTE: you will need to use add_dependencies(<your_target> eigen_ext) if your dependency has source code that needs to be compiled before your target.

It is possible to use ExternalProject in a way that will clone and compile the dependency when you run cmake <location>, but this basically involves having your CMakeLists.txt run terminal commands to do so. This allows all of the CMake setup of the dependency to happen during the cmake command, so all of the project's variables and targets will already be declared. If you want to learn more about this, see how Google does it. I think CMake noticed people doing things like this to make the dependency availble during the cmake command, so they created a module to do it called FetchContent (which only started at CMake version 3.11 and some of its functions only became available with version 3.14+). Note that my Ubuntu 18.04 computer is only on CMake version 3.10.2, but my Ubuntu 20.04 computer is on version 3.16.3. Basically, just remember it is a modern CMake feature.

The main difference here is that the content of the dependency is configured, downloaded, and compiled during the cmake step. So by the time you run make, the dependency is already built. This can make the CMake process a little easier, but it still feels strange to me to have build and action steps occur from running cmake. The cmake step could end up taking a while depending on how big the dependency is. The number of lines it takes to configure a dependency using FetchContent is probably less that when using ExternalProject, but I would still probably recommend putting it into a new file called something like cmake/Fetch_<package>.cmake and then just do include(cmake/Fetch_<package>.cmake) in your project's CMakeLists.txt. For Eigen, the contents of cmake/Fetch_Eigen3.cmake would look like this:

cmake_minimum_required(VERSION 3.14)

include(FetchContent)

FetchContent_Declare(eigen_ext
  GIT_REPOSITORY https://gitlab.com/libeigen/eigen.git
  GIT_TAG 3.3.7
)
FetchContent_MakeAvailable(eigen_ext)
set(EIGEN3_INCLUDE_DIRS ${eigen_ext_SOURCE_DIR})

The minimum version is accurate for using FetchContent, and we have to include(FetchContent) in order to use the module. Then we create a target called eigen_ext using FetchContent_Declare(). FetchContent_MakeAvailable() is the function that actually clones the repo and builds it...and it requires version 3.14+ (although you can find ways to get around this online if you need to), and this makes properties of the dependency available to your project. I believe you should be able to use targets declared in the dependency without any further work, but I haven't fully tested it. In this case, I found that Eigen's variables were not made available and since Eigen is a header-only library there isn't a target we can link to. We need to include Eigen's directories, so I found that I needed to manually set EIGEN3_INCLUDE_DIRS to be the source directory of the external target (which is where the repository is cloned).

Pros:

• CMake manages the dependency rather than git
• The dependency does not show up in the project's file structure, thus it does not suggest that it is being developed as well
• Dependency is contained entirely in the project at a specific version so everyone using the project is working with the same code
• Each project can include the subdirectory at a specific version

Cons:

• The most difficult option to figure out how to use
• Not helpful when you are actively developing the dependency

Use ExternalProject or FetchContent whenever you have a dependency that you are not going to develop (and don't want visible in your file structure) but that you want to be built as part of the project, rather than installed to the system. This is how you can create a fully "Hermetic" build, where all dependencies are built with the project and everyone using the project is guaranteed to be using the exact same code. For research code, it is probably fine to install common packages to your system (things like Eigen, OpenCV, and ROS - ROS actually installs both of the other 2) because others in the lab using the code are likely to have the same packages installed. There is always the risk that someone is using an older version of Ubuntu where versions don't match up, but it probably isn't that big of a deal. For dependencies like OSQP, where it is not likely that everyone has it installed, the best practice would probably be to use ExternalProject.