This FAQ (formerly called Internal FAQ) is intended for developers. Note that some actions require special permissions like e.g. updating the website.

• General information:

  • I am new to OpenMS. What should I do first?
  • What is the difference between a tool and a util in OpenMS?
  • I have written a class for OpenMS I want to contribute. What should I do?
  • Can I use QT designer to create GUI widgets?
  • Where are the binary installers created?

• Build system - Introduction:

  • What is CMake?
  • How do I use CMake?
  • How do I generate a build-system for Eclipse, KDevelop, CodeBlocks etc?
  • What are user definable CMake Cache Variables?
  • How do I switch to Debug/Release configuration?
  • What are the most useful (make) targets?
  • (Windows) What version of Visual Studio should I use?
  • How can I speed up the compile process of OpenMS?

• Builds system - Troubleshooting:

  • OpenMS complains about boost not being found but I'm sure it's there!
  • I changed the contrib path, but re-running CMake won't change the library paths?
  • Can I use another solver than GLPK?
  • CMake can't seem to find a Qt library (usually QtCore)! What now?
  • How can I find out which shared libraries are used by an application?
  • How can I get a list of the symbols defined in a (shared) library or object file?

• Builds system - Adding algorithms and tools:

  • How do I add a new class MyClass to the build system?
  • I have written a class for OpenMS I want to contribute. What should I do?
  • How do I add a new directory MYDIR to the build system?
  • I want to implement a new file adapter. What is to be done?
  • How to add a new TOPP tool?
  • How to add a new TOPP util?
  • Where can I find the definition of the main page?
  • Where can I add a new module?
  • What are the important files for adding a new tutorial section?
  • How to create an icon file for a TOPP tool under Windows?

• Continuous integration

  • What checks/requirements do you have for a pull request?
  • How does travis work?
  • What can we do if travis times out?
  • How does GitHub Actions work?
  • How can I debug a GitHub Actions run?
  • How does Jenkins work?
  • What can we do if Jenkins fails unrelatedly?
  • How can I debug a Jenkins run?

• Testing

  • Class tests and tool tests in OpenMS. What is the difference?
  • How do I add a new TOPP test?
  • How do I add a new test for the class MyClass?
  • How do I add a new GUI test (for QT Gui classes) for the class MyClass?
  • (Linux) When executing 'make test', all tests fail.
  • Why are there no source/TEST and source/APPLICATIONS/TOPP|UTILS folder?
  • How do I run a single test?
  • How can I easily update a lot of test files (e.g., after a small format change in result files)?
  • Can the START_SECTION-macro not handle template methods that have two or more arguments?

• Working in IDE's

  • Can I use QT designer to create GUI widgets?
  • Why are there no source/TEST and source/APPLICATIONS/TOPP|UTILS folder?
  • Visual Studio: I'm getting the error "Error C2471: cannot update program database".
  • Visual Studio: Visual Studio can't read the clang-format file.
  • Eclipse CDT: The indexer gets stuck at some file which #includes seqan
  • Eclipse CDT: The parser is confused after OPENMS_DLLAPI and does not recognize standard C++ headers

• PyOpenMS - Troubleshooting

  • How can I wrap my new method with PyOpenMS?
  • How can I wrap my new class with PyOpenMS?
  • My method has multiple outputs. Can I use output parameters? I have trouble wrapping them for pyOpenMS

• Debugging

  • How do I debug uncaught exceptions?
  • How can I profile my code?
  • (Linux) How do I check my code for memory leaks?
  • (Linux) Why is no core dumped, although a fatal error occured?
  • (Linux) How can I set breakpoints in gdb to debug OpenMS?
  • How can I find out which shared libraries are used by an application?
  • How can I get a list of the symbols defined in a (shared) library or object file?

• Cross-platform thoughts

• Doxygen documentation

  • Where can I find the definition of the main page?
  • Where can I add a new module?
  • How is the parameter documentation for classes derived from DefaultParamHandler created?
  • How is the command line documentation for TOPP/UTILS tools created?
  • What are the important files for adding a new tutorial section?

• Bugs and Issues

  • How to contribute a bugfix?

• Release:

  • How to prepare a new OpenMS release?
  • How to update the openms.de website (e.g., after a release)?

• Misc

CMake got confused. Set up a new build directory and try again. If you build in-source (not recommended), deleting the CMakeCache.txt and cmake directory might help.

• Check out the developement version of OpenMS (see website).
• Try and build OpenMS according to the installation instructions.
• Read the OpenMS Coding Convention.
• Read the OpenMS Tutorial.
• Create a GitHub account
• Register to the open-ms-general and open-ms-developers mailing list. (You can see the developers list only if you are logged in to Sourceforge and if you are a OpenMS developer).

A tool starts its lifecycle in UTILS and may exist without beeing thoroughly tested. Tools may be promoted from UTILS to TOOLS if they are stable enough, are fully tested, fully documented and a test workflow exists.

Basically, you must conform to the OpenMS Coding conventions :

Coding style (brackets, variable names, etc.) must conform to the conventions.

• The class and all the members must be documented thoroughly.
• You can check your code with the tool tools/checker.php. Call php tools/checker.php for detailed instructions.

Please open a pull request and follow the checklist

Login to the wordpress admin area at www.openms.de/wp-admin with your username and password assigned by the current Homepage maintainers.

Yes! If you want to create a class called Widget: Create .ui-File with QT designer and store it as Widget.ui. Add the class to the sources.cmake. From the .ui-File the file include/OpenMS/VISUAL/UIC/ClassTemplate.h is generated by the build system. DO NOT CHECK-IN THIS FILE, AS IT IS GENERATED AUTOMATICALLY, WHEN NEEDED!!! Derive the class Widget from WidgetTemplate. You need to check in the Widget.h and Widget.cpp files.

Put round brackets around the method declaration. Then it should work.

https://abibuilder.informatik.uni-tuebingen.de/archive/openms/OpenMSInstaller/nightly/. Please verify the creation date of the individual installers, as there may have been an error while creating the installer.

CMake builds BuildSystems for different platforms, e.g. VisualStudio Solutions on Windows, Makefiles on Linux etc. This allows us to define in one central location (namely CMakeLists.txt) how OpenMS is build and have the platform specific stuff handled by CMake. See http://www.cmake.org for more information.

See Installation instructions for your platform. In general, you call CMake(.exe) with some parameters to create the native build-system. Afterwards you can (but usually don't have to edit the current configuration using a GUI named ccmake (or CMake-GUI in Windows), which ships with CMake). Note: whenever ccmake is mentionend in this document, substitute this by CMake-GUI if your OS is Windows. You can also edit the CMakeCache.txt file directly.

Type cmake into a console. This will list the available code generators available on your platform, which you can pass to CMake using the -G option.

They allow the user to pass options to CMake which will influence the build system. The most important option which should be given when calling CMake.exe is:

CMAKE_FIND_ROOT_PATH, which is where CMake will search for additional libraries if they are not found in the default system paths. By default we add OpenMS/contrib. If your have installed all libraries on your system already there is no need to change CMAKE_FIND_ROOT_PATH. If you need the contrib, you will need to set this variable. On Windows, you always need the contrib, as there are no system developer packages. To pass this variable to CMake use the -D switch e.g. cmake -D CMAKE_FIND_ROOT_PATH:PATH="D:\\somepath\\contrib" Everything else can be edited using ccmake afterwards.

The following options are of interest:

• CMAKE_BUILD_TYPE Define if you want to build Debug or Release version of OpenMS. Release is the default.

• CMAKE_FIND_ROOT_PATH The path to the contrib libraries. Note that you can also provide more then one value here (e.g., -D CMAKE_FIND_ROOT_PATH="/path/to/contrib;/usr/" will search in your contrib path and in /usr` for the required libraries)

• STL_DEBUG Enables STL debug mode.

• DB_TEST (deprecated) Enables database testing.

• QT_DB_PLUGIN (deprecated) Defines the db plugin used by Qt.

• MT_CUDA_BUILD_TYPE ...

Their description will be displayed when you call ccmake.

Yes, but by default the build system only links against GLPK (this is how OpenMS binary packages must be build!). To use another solver try cmake ... -D USE_COINOR=1 .... And look at the documentation of the LPWrapper class.

For Makefile generators (typically on Linux) you can set the CMAKE_BUILD_TYPE variable to either Debug or Release by calling ccmake. For Visual Studio, this is not necessary as all configurations are generated and you can choose the one you like within the IDE itself. The 'Debug' configuration enabled debug information. The 'Release' configuration disables debug information and enables optimization.

Once a library is found and its location is stored in a cache variable, it will only be searched again if the corresponding entry in the cache file is set to false. You can simply delete the CMakeCache.txt, but all other custom settings will be lost as well.

In Visual Studio you can see all targets on the left. For Makefiles type make help. However, this list is quite long. The most useful targets will be shown to you by calling the targets target, i.e. make targets.

CMake finds QT by looking for 'qmake' in your PATH or for the Environment Variable QTDIR! Set these accordingly. If the problem still persists: do you have a second installation of Qt (especially the MinGW version?)? This might lead CMake to the wrong path (it's searching for the Qt*.lib files). You should only move/delete the offending Qt version if you know what you are doing! A save workaround is to edit the CMakeCache file (e.g. via ccmake) and set all paths relating to QT (e.g. QT_LIBRARY_DIR) manually.

Use the latest if you can. Get the latest CMake, as its generator needs to support your VS. If your VS is too new and there is no CMake for that yet, you're gonna be faced with a lot of conversion issues. This happens whenever the Build-System calls CMake (which can be quite often, e.g., after changes to CMakeLists.txt).

1. Create the new class in the corresponding sub-folder of the sub-project. The header has to be created in src/<sub-project>/include/OpenMS and the cpp file in src/<sub-project>/source, e.g., src/openms/include/OpenMS/FORMAT/NewFileFormat.h and src/openms/source/FORMAT/NewFileFormat.cpp.
2. Add both to the respective sources.cmake file in the same directory (e.g., src/openms/source/FORMAT/ and src/openms/include/OpenMS/FORMAT/).
3. Add the corresponding class test to src/tests/class_tests/<sub-project>/ (e.g., src/tests/class_tests/openms/source/NewFileFormat_test.cpp)
4. Add the test to the executables.cmake file in the test folder (e.g., src/tests/class_tests/openms/executables.cmake).
5. Add them to git git add

1. Create two new sources.cmake files (one for src/<sub-project>/include/OpenMS/MYDIR, one for src/<sub-project>/source/MYDIR), using existing sources.cmake files as template.
2. Add the new sources.cmake files to src/<sub-project>/includes.cmake
3. If you created a new directory directly under src/openms/source, then have a look at src/tests/class_tests/openms/executables.cmake
4. add a new section that makes the unit testing system aware of the new (upcoming) tests,
5. look at the very bottom and augment TEST_executables,
6. add a new group target to src/tests/class_tests/openms/CMakeLists.txt

Class tests are unit tests that typically test the functionality of a class. They get build as standalone "additional" executables that include the class to be tested and our own testing utility classes to test outcomes of single functions of the class in question. Unless you added functions that are intended to be used outside of your new additional mode, you don't need to add anything there.

Tool tests are using the tool executable that the user would also receive. We use those executables to run the full algorithm on a small test dataset, to ensure that from version to version the results stay the same/meaningful.

Each tool test consists of

• an executable call on a test dataset (by using either fixed command line parameters or an ini file)
• a FuzzyDiff call that compares the temporary output file of the last call and a reference test output that you have to provide
• a line to add a dependency of the FuzzyDiff call on the actual executable call (so they get executed after each other) Use e.g., ctest -V -R IDMapper to only test tests that include the regex "IDMapper" (-V is just verbose). Make sure to build the IDMapper and IDMapper_test (if edited) executable first everytime. "ctest" does not have any automatic dependency on the timestamps of the executables.

You should always add a test alongside every new class added to OpenMS.

1. Add the class test to src/tests/class_tests/<sub-project>/ (e.g., src/tests/class_tests/openms/source/NewFileFormat_test.cpp)
2. Add the test to the executables.cmake file in the test folder.
3. Add them to git git add

A test template for your specific class can be generated by the create_test.php script found in tools/.

1. make sure you generated XML files containing the class information make doc_xml
2. call:

php tools/create_test.php /BUILD_DIRECTORY/ /PATH_TO_HEADER/MyClass.h \
  "FIRSTNAME LASTNAME" > ./src/tests/class_tests/openms/source/MyClass_test.cpp

1. Create the MyClass_test.cpp in src/tests/class_tests/openms_gui/source
2. Add it to src/tests/class_tests/openms_gui/CMakeLists.txt in the GUI section.
3. Have a look at existing GUI tests, as they use the QT TestLib framework and not the OpenMS macros.

Please check the LD_LIBRARY_PATH environment variable:

You can print the LD_LIBRARY_PATH with echo $LD_LIBRARY_PATH. If your /lib/ folder is included, check that libOpenMS.so is present. With the ldd command, you can show the libraries used by an executable, e.g. 'ldd /bin/ClassTest_test'.

Build with several threads. If you have several pocessors/cores you can build OpenMS classes/tests and TOPP tools in in several threads. On Linux use the make option -j: make -j8 OpenMS TOPP test_build

On Windows, Visual Studio solution files are automatically build with the /MP flag, such that VS uses all available cores of the machine.

see Preparation-of-a-new-OpenMS-release

Travis is an automated system for continuous integration and each new commit and pull request is automatically run through the travis build system. This is controlled by a .travis.yaml file in the source tree.

The first thing to try is to simply restart travis, it sometimes hangs and since it builds on shared infrastructure, the next build may work better. This needs to be done by a OpenMS core developer.

Since we use extensive caching, the build may take much longer when many files are touched and may never complete in that case (running into the travis time limit). In that case we can rebuild the cache using the following approach:

$ git cherry-pick 89c5cd7f2d9d343b3d63fc6bab18e08dcd969c05
$ git push origin develop
# Now wait for the build to complete
$ git revert 89c5cd7f2d9d343b3d63fc6bab18e08dcd969c05
$ git push origin develop

All source files added to an IDE are associated with their targets. You can find the source files for each test within its own subproject. The same is true for the TOPP and UTILS classes.

Using grep one can simply extract the lines starting with diff FILENAME1 FILENAME2 and replace the diff by copy.

This is a bug in Visual Studio and there is a bugfix: http://code.msdn.microsoft.com/KB946040 Only apply it if you encounter the error. The bugfix might have unwanted side effects!

Depending on the Visual Studio version you're using you might get an error like "Error while formating with ClangFormat!". This is because Visual Studio is using an outdated version of clang-format. Unfortunately there is no easy way to update this using Visual Studio itself.
There is a plugin provided by LLVM designed to fix this exact problem, but the plugin doesn't work with every Visual Studio version. However, you can update clang-format by hand using the pre-build clang-format binary. Both the binary and a link to the plugin can be found here: https://llvm.org/builds/.
To update clang-format by hand just download the binary and exchange it with the clang-format binary in your Visual Studio folder.
For Visual Studio 17 and 19 it should be located at: "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\Llvm\bin"

It seems that SeqAn code is just too confusing for older eclipse C++ indexers. You should upgrade to eclipse galileo (CDT 6.0.x). Also, increase the available memory limit in eclipse.ini, e.g. -Xmx1024m for one gig.

Go to Project -> Properties -> C/C++ Include Paths and Preprocessor Symbols -> Add Preprocessor symbol -> "OPENMS_DLLAPI=". This tells eclipse that the macro is defined empty. In the same dialog you can also add an external include path to e.g. /usr/include/c++/4.3.3/, etc. The issue with C++ headers was fixed in the latest galileo release. Hints to resolve the OPENMS_DLLAPI issue using the cmake generator are welcome!

You can can execute an OpenMS class test using the CTest regular expressions:

> ctest -V -R "^<class>_test" 
To build a class test, you simply call the respective make target in ./source/TEST: 
> make <class>_test

> ctest -V -R "TOPP_<tool>" 
To build the tool, use: 
> make <tool>

There is a mechanism to have a core dumped if an uncaught exception occurs.

To enable it, the environment variable OPENMS_DUMP_CORE has to be set.

Each time an uncaught exception occures, the OPENMS_DUMP_CORE variable is checked and a segmentation fault is caused, if it is set.

Try the ulimit -c unlimited command. It sets the maximum size of a core to unlimited.

Note: We observed that, on some systems, no core is dumped even if the size of the core file is set to unlimited. We are not sure what causes this problem.

Imagine you want to debug the TOPPView application and you want it to stop at line 341 of

SpectrumMDIWindow.C. 
Run gdb: 
shell> gdb TOPPView

Start the application (and close it):

gdb> run [arguments]

Set the breakpoint:

gdb> break SpectrumMDIWindow.C:341

Start the application again (with the same arguments):

gdb> run

Linux: ldd

Windows (Visual studio console): Try "Dependency Walker" (http://www.dependencywalker.com/) (use x86 for 32bit builds and the x64 version for 64bit builds. Using the wrong version of depends.exe will give wrong results!) or dumpbin /DEPENDENTS OpenMS.dll

Linux: nm <library> Use nm -C to switch on demangling of low-level symbols into their C++-equivalent names. nm also accepts .a and .o files.

Windows (Visual studio console): dumpbin /ALL <library>

You can use dumpbin on object files (.o) or (shared) library files (.lib) or the DLL itself e.g. dumpbin /EXPORTS OpenMS.dll

OpenMS runs on three major platforms, each one having its own ways of doing things. Here are the most prominent causes of "it runs on Platform A, but not on B. What now?"

Reading/Writing binary files causes different behaviour ... Usually Linux does not make a difference between text-mode and binary-mode when reading files. This is quite different on Windows as some bytes are interpreted as EOF, which lead might to a premature end of the reading process. Thus, if reading binary files make sure that you explicitly state that the file is binary when opening it!

During writing in text-mode on windows a line-break (\n) is expanded to (\r\n). Keep this in mind or use the eol-style property of subversion to ensure that line endings are correctly checked out on non-Windows systems.

unsigned int vs size_t (UInt and Size) UInt and Size are the same on Linux GCC (i.e. both have the same size, 32bit on 32bit systems, 64bit on 64 bit systems), however on Windows this only holds for 32bit. On a 64bit Windows the UInt type is still 32bit, Size is (obviously) 64bit. This might lead to warnings (at best) or overflows and other nasty stuff. So make sure you do not rely on UInt being equal to Size - because they're not.

Paths and system functions...

This is trivial but hardcoding something like String tmp_dir = "/tmp"; is a big no-no! This must fail on Windows! Use Qt's QDir to get a path to the systems temporary directory if required.

Also calling things like uname which are only available on Linux: don't!

When working with files or directories, it is usually safe to use "/" on all platforms. Even Windows understands that. Take care of spaces in directory names though. You should always quote paths if they are used in a system call to ensure that the subsequent interpreter takes the spaced path as a single entity.

You will have to add an entry to src/pyOpenMS/pxds/CLASS_NAME.pxd with the signature of your new method(s).

You will have to create a new file src/pyOpenMS/pxds/CLASS_NAME.pxd which is explained here.

Python does not support passing primitive types (int, double etc) by reference, there fore void calculate(double &) will not work.

OpenMS/doc/doxygen/public/Main.doxygen

OpenMS/doc/doxygen/public/Modules.doxygen

You have to add your class to the program OpenMS/doc/doxygen/parameters/DefaultParamHandlerDocumenter.cpp. This program generates a html table with the parameters. This table can then be included in the class documentation using the following doxygen command:

@htmlinclude OpenMS_<class name>.parameters

Note that parameter documentation is automatically generated for TOPP/UTILS included in the static ToolHandler.cpp tools list. To include TOPP/UTILS parameter documentation use following doxygen command:

@htmlinclude TOPP_<tool name>.parameters

or

@htmlinclude UTILS_<tool name>.parameters

You can test if everything worked by calling make doc_param_internal. The parameters documentation is written to OpenMS/doc/doxygen/parameters/output/.

The program OpenMS/doc/doxygen/parameters/TOPPDocumenter.cpp creates the command line documentation for all classes that are included in the static ToolHandler.cpp tools list. It can be included in the documentation using the following doxygen command:

@verbinclude TOPP_<tool name>.cli

You can test if everything worked by calling make doc_param_internal. The command line documentation is written to OpenMS/doc/doxygen/parameters/output/.

OpenMS tutorial:

• OpenMS/doc/OpenMS_tutorial/refman_overwrite.tex.in (for PDF tutorials)
• OpenMS/doc/doxygen/public/OpenMS_Tutorial_html.doxygen~ (for html tutorials)

TOPP and TOPPView tutorial:

• OpenMS/doc/TOPP_tutorial/refman_overwrite.tex.in (for PDF tutorials)
• OpenMS/doc/doxygen/public/TOPP_Tutorial_html.doxygen (for html tutorials)

1. Submit the bug as a GitHub issue
2. Create a feature branch (e.g. feature/fix_missing_filename_issue_615) from your (up-to-date) develop branch in your fork of OpenMS
3. Fix the bug and add a test
4. Create a pull request for your branch.
5. After approval and merge make sure the issue is closed

Try IBM's profiler, available for all platforms (and free for academic use): Purify(Plus) and/or Quantify. Windows: this is directly supported by Visual Studio (Depending on the edition: Team and above). Follow their documentation.

Linux:

• build OpenMS in debug mode (set CMAKE_BUILD_TYPE to 'Debug')
• call the executable with valgrind: 'valgrind –tool=callgrind ' Note: other processes running on the same machine can influence the profiling. Make sure your application gets enough resources (memory, CPU time).
• You can start and stop the profiling while the executable is running e.g. to skip initialization steps:
• start valgrind with the option –instr-atstart=no
• call 'callgrind -i [on|off]' to start/stop the profiling
• The output can be viewed with 'kcachegrind callgrind.out.'

• build OpenMS in debug mode (set CMAKE_BUILD_TYPE to Debug)
• call the executable with valgrind: valgrind --suppressions=OpenMS/tools/valgrind/openms_external.supp –leak-check=full <executable> <parameters>

Common errors are:

• 'Invalid write/read ...' - Violation of container boundaries
• '... depends on uninitialized variable' - Uninitialized variables:
• '... definitely lost' - Memory leak that has to be fixed
• '... possibly lost' - Possible wemory leak, so have a look at the code

For more information see the valgrind documentation at http://valgrind.org/docs/manual/