Skip to content

CPP Unit Tests

sharon wang edited this page Jun 19, 2023 · 5 revisions

Quick Start

All of the tests for the C++ code can be run using the rstudio-tests script, which is installed at the top level of the build folder. By default, this script executes all the tests, one after another. If all you want to do is run the entire test suite for your change, you can do so easily by executing this script without any arguments.

$ ./rstudio-tests

Prerequisites

Newer versions of the C++ unit tests require an R package called testthat. This can be installed by launching RStudio and entering install.packages("testthat") into the R terminal.

Note that testthat requires an R version >= 3.1.0.

Core Tests

Execution

The core tests validate behavior in the RStudio core library (src/cpp/core), which contains core functionality shared among the client, server, and most other RStudio executables. You can execute just the core library tests by using the --scope core argument:

$ ./rstudio-tests --scope core

Authoring

The core unit tests are written using the Catch C++ unit test framework, which has been modified somewhat to make its syntax more familiar to users of the testthat R package.

To author a core test, do the following:

  1. Create a file with the same name as the .cpp file that you're testing, but with Tests on the end; for instance, if the code to be tested is in Foo.cpp, you need to create FooTests.cpp.
  2. Write your tests in this file; look at other *Tests.cpp files in src/cpp/core for syntax and examples.
  3. Build RStudio as usual. An rstudio-core-tests executable will be produced in the build folder which contains your new tests.

You can run rstudio-core-tests directly to execute all of the core tests, or use the rstudio-tests wrapper as described above.

Session Tests

The session tests are for most purposes identical to the core tests; they use the same framework and conventions. You can run just the session tests as follows:

$ ./rstudio-tests --scope rsession

Session R Tests

Sometimes it's useful to write session unit tests in R instead of C++, or to write unit tests for the R code that provides functionality at runtime. We have a suite of R tests which use the testthat R package to test session code using a live R session. Note that you will need to install the testthat package to your personal R library in order to run these tests; you can do this with the following R command:

> install.packages("testthat")

Execution in the Console

The R tests can be run as follows:

$ ./rstudio-tests --scope r

If you are working on just one feature, it is also possible to filter the tests so that only the tests for your feature are run. This is useful for rapidly iterating on a single set of tests. If your feature is called foo, you can run tests for just your feature using the following command (note that scope is not used in this case):

$ ./rstudio-tests --filter foo

Execution in RStudio

The R tests can also be run in RStudio by running the following command (replace ~/rstudio/ with the root of the RStudio repo on your machine):

> testthat::test_dir("~/rstudio/src/cpp/tests/testthat")

Finally, if you'd like to iterate on the tests in a single file, you can do so as follows:

> testthat::test_file("~/rstudio/src/cpp/tests/testthat/your-test-file.R")

Authoring

The R tests live in the src/cpp/tests/testthat folder. Add a file to test your feature and name it test-foo.R, where foo is the name of your feature. The test- prefix is important! See Writing tests for syntax and examples.

Note that, unlike tests written in C++, R tests do not need to be compiled. You can change your R test and re-run the test right away with the rstudio-tests script.

Using Valgrind

Valgrind is a useful tool for detecting memory problems and other bad runtime behavior. If you pass any arguments to rstudio-tests other than those listed here, the script will run the tests under Valgrind and pass Valgrind the arguments you give it.

CPP Test Section Output

For temporary debugging purposes, it's possible to edit catch.hpp to print out the test section and location. The output will look something like this:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[2023-06-19T21:01:25.297Z] catch-unit-tests is a Catch v2.13.8 host application.

[2023-06-19T21:01:25.297Z] Run with -? for options

[2023-06-19T21:01:25.297Z] 

[2023-06-19T21:01:25.297Z] -------------------------------------------------------------------------------

[2023-06-19T21:01:25.297Z] ConsoleProcessInfo

[2023-06-19T21:01:25.297Z] -------------------------------------------------------------------------------

[2023-06-19T21:01:25.297Z] linux-pipeline/main/src/cpp/session/SessionConsoleProcessInfoTests.cpp:94

[2023-06-19T21:01:25.297Z] ...............................................................................

...

[2023-06-19T21:01:26.317Z] -------------------------------------------------------------------------------

[2023-06-19T21:01:26.317Z] Session Suspend Filters

[2023-06-19T21:01:26.317Z] -------------------------------------------------------------------------------

[2023-06-19T21:01:26.317Z] linux-pipeline/main/src/cpp/session/SessionSuspendFilterTests.cpp:50

[2023-06-19T21:01:26.317Z] ...............................................................................

...
[2023-06-19T21:01:27.846Z] -------------------------------------------------------------------------------

[2023-06-19T21:01:27.846Z] SessionFind

[2023-06-19T21:01:27.846Z] -------------------------------------------------------------------------------

[2023-06-19T21:01:27.846Z] linux-pipeline/main/src/cpp/session/modules/SessionFindTests.cpp:57

[2023-06-19T21:01:27.846Z] ...............................................................................
...

See commit: https://github.com/rstudio/rstudio/commit/96b0fcbd6da362450415c764565e82a3a2152dd1 on how to enable this.

Since this file was merged from multiple files, we want to avoid making actual edits to the file. So please revert any changes to this file once you're done debugging! :)