Let’s start with an example, following code is a simple test for a getenv().

Each test includes the tst_test.h header and must define the struct tst_test test structure.

The overall test initialization is done in the setup() function.

The overall cleanup is done in a cleanup() function. Here cleanup() is omitted as the test does not have anything to clean up. If cleanup is set in the test structure it’s called on test exit just before the test library cleanup. That especially means that cleanup can be called at any point in a test execution. For example even when a test setup step has failed, therefore the cleanup() function must be able to cope with unfinished initialization, and so on.

The test itself is done in the test() function. The test function must work fine if called in a loop.

There are two types of a test function pointers in the test structure. The first one is a .test_all pointer that is used when test is implemented as a single function. Then there is a .test function along with the number of tests .tcnt that allows for more detailed result reporting. If the .test pointer is set the function is called .tcnt times with an integer parameter in range of [0, .tcnt - 1].

Each test has a default timeout set to 300s. The default timeout can be overridden by setting .timeout in the test structure or by calling tst_set_timeout() in the test setup(). There are a few testcases whose run time may vary arbitrarily, for these timeout can be disabled by setting it to -1.

Test can find out how much time (in seconds) is remaining to timeout, by calling tst_timeout_remaining().

Printf-like function to report test result, it’s mostly used with ttype:

The ttype can be combined bitwise with TERRNO or TTERRNO to print errno, TST_ERR respectively.

Printf-like function to report error and exit the test, it can be used with ttype:

The ttype can be combined bitwise with TERRNO or TTERRNO to print errno, TST_ERR respectively.

There are also TST_EXP_*() macros that can simplify syscall unit tests to a single line, use them whenever possible. These macros take a function call as the first parameter and a printf-like format string and parameters as well. These test macros then expand to a code that runs the call, checks the return value and errno and reports the test result.

The TST_EXP_PASS() can be used for calls that return -1 on failure and 0 on success. It will check for the return value and reports failure if the return value is not equal to 0. The call also sets the TST_PASS variable to 1 if the call succeeeded.

As seen above, this and similar macros take optional variadic arguments. These begin with a format string and then appropriate values to be formatted.

The TST_EXP_FD() is the same as TST_EXP_PASS() the only difference is that the return value is expected to be a file descriptor so the call passes if positive integer is returned.

The TST_EXP_FAIL() is similar to TST_EXP_PASS() but it fails the test if the call haven’t failed with -1 and errno wasn’t set to the expected one passed as the second argument.

The TST_EXP_FAIL2() is the same as TST_EXP_FAIL() except the return value is expected to be non-negative integer if call passes. These macros build upon the TEST() macro and associated variables.

The TEST macro sets TST_RET to its argument’s return value and TST_ERR to errno. The TTERNO flag can be used to print the error number’s symbolic value.

No LTP library function or macro, except those in tst_test_macros.h, will write to these variables (rule LTP-002). So their values will not be changed unexpectedly.

If the return value of wait is positive. This macro will print a pass result and set TST_PASS appropriately. If the return value is zero or negative, then it will print fail. There are many similar macros to those shown here, please see tst_test_macros.h.

Return the given signal number’s corresponding string.

Return the given errno number’s corresponding string. Using this function to translate errno values to strings is preferred. You should not use the strerror() function in the testcases.

Returns string describing the status as returned by wait().

Allows for setting timeout per test iteration dynamically in the test setup(), the timeout is specified in seconds. There are a few testcases whose runtime can vary arbitrarily, these can disable timeouts by setting it to -1.

Flush output streams, handling errors appropriately.

This function is rarely needed when you have to flush the output streams before calling fork() or clone(). Note that the SAFE_FORK() and SAFE_CLONE() calls this function automatically. See 2.4 FILE buffers and fork() for explanation why is this needed.

If .needs_tmpdir is set to 1 in the struct tst_test unique test temporary is created and it’s set as the test working directory. Tests MUST NOT create temporary files outside that directory. The flag is not needed to be set when use these flags: .all_filesystems, .format_device, .mntpoint, .mount_device .needs_checkpoints, .needs_device, .resource_file (these flags imply creating temporary directory).

Safe macros aim to simplify error checking in test preparation. Instead of calling system API functions, checking for their return value and aborting the test if the operation has failed, you just use corresponding safe macro.

Use them whenever it’s possible.

Instead of writing:

You write just:

They can also simplify reading and writing of sysfs files, you can, for example, do:

See include/tst_safe_macros.h, include/tst_safe_stdio.h and include/tst_safe_file_ops.h and include/tst_safe_net.h for a complete list.

Test specific command line parameters can be passed with the NULL terminated array of struct tst_option. The optstr is the command line option i.e. "o" or "o:" if option has a parameter. Only short options are supported. The arg is where optarg is stored upon match. If option has no parameter it’s set to non-NULL value if option was present. The help is a short help string.

Helpers for parsing the strings returned in the struct tst_option.

Helpers return zero on success and errno, mostly EINVAL or ERANGE, on failure.

Helpers functions are no-op if str is NULL.

The valid range for result includes both min and max.

In particular, tst_parse_filesize function accepts prefix multiplies such as "k/K for kilobytes, "m/M" for megabytes and "g/G" for gigabytes. For example, 10K are converted into 10240 bytes.

Testcases for newly added kernel functionality require kernel newer than a certain version to run. All you need to skip a test on older kernels is to set the .min_kver string in the struct tst_test to a minimal required kernel version, e.g. .min_kver = "2.6.30".

For more complicated operations such as skipping a test for a certain range of kernel versions, following functions could be used:

These two functions are intended for runtime kernel version detection. They parse the output from uname() and compare it to the passed values.

The return value is similar to the strcmp() function, i.e. zero means equal, negative value means that the kernel is older than than the expected value and positive means that it’s newer.

The second function tst_kvercmp2() allows for specifying per-vendor table of kernel versions as vendors typically backport fixes to their kernels and the test may be relevant even if the kernel version does not suggests so. See testcases/kernel/syscalls/inotify/inotify04.c for example usage.

Be wary that if the test forks and there were messages printed by the tst_*() interfaces, the data may still be in libc/kernel buffers and these ARE NOT flushed automatically.

This happens when stdout gets redirected to a file. In this case, the stdout is not line buffered, but block buffered. Hence after a fork content of the buffers will be printed by the parent and each of the children.

To avoid that you should use SAFE_FORK(), SAFE_CLONE() or tst_clone().

Results reported by tst_res() are propagated to the parent test process via block of shared memory.

Calling tst_brk() causes child process to exit with non-zero exit value. Which means that it’s safe to use SAFE_*() macros in the child processes as well.

Children that outlive the test() function execution are waited for in the test library. Unclean child exit (killed by signal, non-zero exit value, etc.) will cause the main test process to exit with tst_brk(), which especially means that TBROK propagated from a child process will cause the whole test to exit with TBROK.

If a test needs a child that segfaults or does anything else that cause it to exit uncleanly all you need to do is to wait for such children from the test() function so that it’s reaped before the main test exits the test() function.

The tst_reap_children() function makes the process wait for all of its children and exits with tst_brk(TBROK, …​) if any of them returned a non zero exit code.

When using SAFE_CLONE or tst_clone, this may not work depending on the parameters passed to clone. The following call to SAFE_CLONE is identical to fork(), so will work as expected.

If exit_signal is set to something else, then this will break tst_reap_children. It’s not expected that all parameters to clone will work with the LTP library unless specific action is taken by the test code.

The tst_res() function can be also used from binaries started by exec(), the parent test process has to set the .child_needs_reinit flag so that the library prepares for it and has to make sure the LTP_IPC_PATH environment variable is passed down, then the very fist thing the program has to call in main() is tst_reinit() that sets up the IPC.

As LTP tests are written for Linux, most of the tests involve fork()-ing and parent-child process synchronization. LTP includes a checkpoint library that provides wait/wake futex based functions.

In order to use checkpoints the .needs_checkpoints flag in the struct tst_test must be set to 1, this causes the test library to initialize checkpoints before the test() function is called.

The checkpoint interface provides pair of wake and wait functions. The id is unsigned integer which specifies checkpoint to wake/wait for. As a matter of fact it’s an index to an array stored in a shared memory, so it starts on 0 and there should be enough room for at least of hundred of them.

The TST_CHECKPOINT_WAIT() and TST_CHECKPOINT_WAIT2() suspends process execution until it’s woken up or until timeout is reached.

The TST_CHECKPOINT_WAKE() wakes one process waiting on the checkpoint. If no process is waiting the function retries until it success or until timeout is reached.

If timeout has been reached process exits with appropriate error message (uses tst_brk()).

The TST_CHECKPOINT_WAKE2() does the same as TST_CHECKPOINT_WAKE() but can be used to wake precisely nr_wake processes.

The TST_CHECKPOINT_WAKE_AND_WAIT() is a shorthand for doing wake and then immediately waiting on the same checkpoint.

Child processes created via SAFE_FORK() are ready to use the checkpoint synchronization functions, as they inherited the mapped page automatically.

Child processes started via exec(), or any other processes not forked from the test process must initialize the checkpoint by calling tst_reinit().

For the details of the interface, look into the include/tst_checkpoint.h.

The TST_PROCESS_STATE_WAIT() waits until process pid is in requested state or timeout is reached. The call polls /proc/pid/stat to get this information. A timeout of 0 will wait infinitely.

On timeout -1 is returned and errno set to ETIMEDOUT.

It’s mostly used with state S which means that process is sleeping in kernel for example in pause() or any other blocking syscall.

If you need to use signal handlers, keep the code short and simple. Don’t forget that the signal handler is called asynchronously and can interrupt the code execution at any place.

This means that problems arise when global state is changed both from the test code and signal handler, which will occasionally lead to:

• Data corruption (data gets into inconsistent state), this may happen, for example, for any operations on FILE objects.

• Deadlock, this happens, for example, if you call malloc(2), free(2), etc. from both the test code and the signal handler at the same time since malloc has global lock for it’s internal data structures. (Be wary that malloc(2) is used by the libc functions internally too.)

• Any other unreproducible and unexpected behavior.

Quite common mistake is to call exit(3) from a signal handler. Note that this function is not signal-async-safe as it flushes buffers, etc. If you need to exit a test immediately from a signal handler use _exit(2) instead.

If a signal handler sets a variable, its declaration must be volatile, otherwise compiler may misoptimize the code. This is because the variable may not be changed in the compiler code flow analysis. There is sig_atomic_t type defined in C99 but this one DOES NOT imply volatile (it’s just a typedef to int). So the correct type for a flag that is changed from a signal handler is either volatile int or volatile sig_atomic_t.

If a crash (e.g. triggered by signal SIGSEGV) is expected in testing, you can avoid creation of core files by calling tst_no_corefile() function. This takes effect for process (and its children) which invoked it, unless they subsequently modify RLIMIT_CORE.

Note that LTP library will reap any processes that test didn’t reap itself, and report any non-zero exit code as failure.

There are certain cases where the test needs a kernel part and userspace part, happily, LTP can build a kernel module and then insert it to the kernel on test start for you. See testcases/kernel/device-drivers/block for details.

Returns the size of statically defined array, i.e. (sizeof(arr) / sizeof(*arr))

Aligns the x to be next multiple of a. The a must be power of 2.

Some tests are known to fail on certain filesystems (you cannot swap on TMPFS, there are unimplemented fcntl() etc.).

If your test needs to be skipped on certain filesystems use the .skip_filesystems field in the tst_test structure as follows:

When the .all_filesystem flag is set the .skip_filesystems list is passed to the function that detects supported filesystems any listed filesystem is not included in the resulting list of supported filesystems.

If test needs to adjust expectations based on filesystem type it’s also possible to detect filesystem type at the runtime. This is preferably used when only subset of the test is not applicable for a given filesystem.

It is safe to use library tst_res() function in multi-threaded tests.

Only the main thread must return from the test() function to the test library and that must be done only after all threads that may call any library function has been terminated. That especially means that threads that may call tst_brk() must terminate before the execution of the test() function returns to the library. This is usually done by the main thread joining all worker threads at the end of the test() function. Note that the main thread will never get to the library code in a case that tst_brk() was called from one of the threads since it will sleep at least in pthread_join() on the thread that called the tst_brk() till exit() is called by tst_brk().

The test-supplied cleanup function runs concurrently to the rest of the threads in a case that cleanup was entered from tst_brk(). Subsequent threads entering tst_brk() must be suspended or terminated at the start of the user supplied cleanup function. It may be necessary to stop or exit the rest of the threads before the test cleans up as well. For example threads that create new files should be stopped before temporary directory is be removed.

Following code example shows thread safe cleanup function example using atomic increment as a guard. The library calls its cleanup after the execution returns from the user supplied cleanup and expects that only one thread returns from the user supplied cleanup to the test library.

Some tests needs a block device (inotify tests, syscall EROFS failures, etc.). LTP library contains a code to prepare a testing device.

If .needs_device flag in the struct tst_test is set the tst_device structure is initialized with a path to a test device and default filesystem to be used.

You can also request minimal device size in megabytes by setting .dev_min_size the device is guaranteed to have at least the requested size then.

If .format_device flag is set the device is formatted with a filesystem as well. You can use .dev_fs_type to override the default filesystem type if needed and pass additional options to mkfs via .dev_fs_opts and .dev_extra_opts pointers. Note that .format_device implies .needs_device there is no need to set both.

If .mount_device is set, the device is mounted at .mntpoint which is used to pass a directory name that will be created and used as mount destination. You can pass additional flags and data to the mount command via .mnt_flags and .mnt_data pointers. Note that .mount_device implies .needs_device and .format_device so there is no need to set the later two.

If .needs_rofs is set, read-only filesystem is mounted at .mntpoint this one is supposed to be used for EROFS tests.

If .all_filesystems is set the test function is executed for all supported filesystems. Supported filesystems are detected based on existence of the mkfs.$fs helper and on kernel support to mount it. For each supported filesystem the tst_device.fs_type is set to the currently tested fs type, if .format_device is set the device is formatted as well, if .mount_device is set it’s mounted at .mntpoint. Also the test timeout is reset for each execution of the test function. This flag is expected to be used for filesystem related syscalls that are at least partly implemented in the filesystem specific code e.g. fallocate().

In case that LTP_DEV is passed to the test in an environment, the library checks that the file exists and that it’s a block device, if .device_min_size is set the device size is checked as well. If LTP_DEV wasn’t set or if size requirements were not met a temporary file is created and attached to a free loop device.

If there is no usable device and loop device couldn’t be initialized the test exits with TCONF.

The tst_umount() function works exactly as umount(2) but retries several times on EBUSY. This is because various desktop daemons (gvfsd-trash is known for that) may be stupid enough to probe all newly mounted filesystem which results in umount(2) failing with EBUSY.

This function finds a free loopdev and returns the free loopdev minor (-1 for no free loopdev). If path is non-NULL, it will be filled with free loopdev path. If you want to use a customized loop device, we can call tst_find_free_loopdev(NULL, 0) in tests to get a free minor number and then mknod.

This function reads test block device stat file (/sys/block/<device>/stat) and returns the bytes written since the last invocation of this function. To avoid FS deferred IO metadata/cache interference, we suggest doing "syncfs" before the tst_dev_bytes_written first invocation. And an inline function named tst_dev_sync() is created for that intention.

This function finds the block dev that this path belongs to, it uses stat function to get the major/minor number of the path. Then scan them in /proc/self/mountinfo and list 2th column value after ' - ' string as its block dev if match succeeds.

This function gets size of the given block device, it checks the dev_path is valid first, if yes, return the size in MB, otherwise return -1.

This function takes a path to a device, filesystem type and an array of extra options passed to mkfs.

The fs options fs_opts should either be NULL if there are none, or a NULL terminated array of strings such as: const char *const opts[] = {"-b", "1024", NULL}.

The extra options extra_opts should either be NULL if there are none, or a NULL terminated array of strings such as {"102400", NULL}; extra_opts will be passed after device name. e.g: mkfs -t ext4 -b 1024 /dev/sda1 102400 in this case.

Note that perfer to store the options which can be passed before or after device name by fs_opts array.

Some tests have size requirements for the filesystem’s free space. If these requirements are not satisfied, the tests should be skipped.

The tst_fs_has_free() function returns 1 if there is enough space and 0 if there is not.

The path is the pathname of any directory/file within a filesystem.

The mult is a multiplier, one of TST_BYTES, TST_KB, TST_MB or TST_GB.

The required free space is calculated by size * mult, e.g. tst_fs_has_free("/tmp/testfile", 64, TST_MB) will return 1 if the filesystem, which "/tmp/testfile" is in, has 64MB free space at least, and 0 if not.

Some tests need to know the maximum count of links to a regular file or directory, such as rename(2) or linkat(2) to test EMLINK error.

Try to get maximum count of hard links to a regular file inside the dir.

This function uses link(2) to create hard links to a single file until it gets EMLINK or creates 65535 links. If the limit is hit, the maximum number of hardlinks is returned and the dir is filled with hardlinks in format "testfile%i", where i belongs to [0, limit) interval. If no limit is hit or if link(2) failed with ENOSPC or EDQUOT, zero is returned and previously created files are removed.

Try to get maximum number of subdirectories in directory.

This function uses mkdir(2) to create directories in dir until it gets EMLINK or creates 65535 directories. If the limit is hit, the maximum number of subdirectories is returned and the dir is filled with subdirectories in format "testdir%i", where i belongs to [0, limit - 2) interval (because each newly created dir has two links already - the . and the link from parent dir). If no limit is hit or if mkdir(2) failed with ENOSPC or EDQUOT, zero is returned and previously created directories are removed.

Returns non-zero if directory is empty and zero otherwise.

Directory is considered empty if it contains only . and ...

Deletes the contents of given directory but keeps the directory itself. Useful for cleaning up the temporary directory and mount points between test cases or test iterations. Terminates the program with TBROK on error.

Fill a file with specified pattern using file descriptor.

Preallocate the specified amount of space using fallocate(). Falls back to tst_fill_fd() if fallocate() fails.

Creates/overwrites a file with specified pattern using file path.

Create/overwrite a file and preallocate the specified amount of space for it. The allocated space will not be initialized to any particular content.

Some tests require a PID, which is not used by the OS (does not belong to any process within it). For example, kill(2) should set errno to ESRCH if it’s passed such PID.

Return a PID value not used by the OS or any process within it.

Returns number of unused pids in the system. Note that this number may be different once the call returns and should be used only for rough estimates.

tst_cmd() is a wrapper for vfork() + execvp() which provides a way to execute an external program.

argv[] is a NULL terminated array of strings starting with the program name which is followed by optional arguments.

TST_CMD_PASS_RETVAL enum tst_cmd_flags makes tst_cmd() return the program exit code to the caller, otherwise tst_cmd() exit the tests on failure. TST_CMD_TCONF_ON_MISSING check for program in $PATH and exit with TCONF if not found.

In case that execvp() has failed and the enum TST_CMD_PASS_RETVAL flag was set, the return value is 255 if execvp() failed with ENOENT and 254 otherwise.

stdout_path and stderr_path determine where to redirect the program stdout and stderr I/O streams.

The SAFE_CMD() macro can be used automatic handling non-zero exits (exits with TBROK) and ENOENT (exits with TCONF).

The tst_timer_check() function checks if specified clk_id is suppored and exits the test with TCONF otherwise. It’s expected to be used in test setup() before any resources that needs to be cleaned up are initialized, hence it does not include a cleanup function parameter.

The tst_timer_start() marks start time and stores the clk_id for further use.

The tst_timer_stop() marks the stop time using the same clk_id as last call to tst_timer_start().

The tst_timer_elapsed*() returns time difference between the timer start and last timer stop in several formats and units.

The tst_timer_expired_ms() function checks if the timer started by tst_timer_start() has been running longer than ms milliseconds. The function returns non-zero if timer has expired and zero otherwise.

Expiration timer example usage.

The first four functions are simple inline conversion functions.

The tst_timespec_lt() function returns non-zero if t1 is earlier than t2.

The tst_timespec_add_us() function adds us microseconds to the timespec t. The us is expected to be positive.

The tst_timespec_diff*() functions returns difference between two times, the t1 is expected to be later than t2.

The tst_timespec_abs_diff*() functions returns absolute value of difference between two times.

If the test needs additional files to be copied to the test temporary directory all you need to do is to list their filenames in the NULL terminated array .resource_files in the tst_test structure.

When resource files is set test temporary directory is created automatically, there is need to set .needs_tmpdir as well.

The test library looks for datafiles first, these are either stored in a directory called datafiles in the $PWD at the start of the test or in $LTPROOT/testcases/data/${test_binary_name}. If the file is not found the library looks into $LTPROOT/testcases/bin/ and to $PWD at the start of the test. This ensures that the testcases can copy the file(s) effortlessly both when test is started from the directory it was compiled in as well as when LTP was installed.

The file(s) are copied to the newly created test temporary directory which is set as the test working directory when the test() functions is executed.

tst_res is a macro, so on when you define a function in one file:

and call it from another file, the file and line reported by tst_res in this function will be from the former file.

TST_TRACE can make the analysis of such situations easier. It’s a macro which inserts a call to tst_res(TINFO, …​) in case its argument evaluates to non-zero. In this call to tst_res(TINFO, …​) the file and line will be expanded using the actual location of TST_TRACE.

For example, if this another file contains:

the generated output may look similar to:

If you need to detect whether a testcase triggers a kernel warning, bug or oops, the following can be used to detect TAINT_W or TAINT_D:

To initialize taint checks, you have to set the taint flags you want to test for in the taint_check attribute of the tst_test struct. LTP library will then automatically call tst_taint_init() during test setup. The function will generate a TCONF if the requested flags are not fully supported on the running kernel, and TBROK if the kernel is already tainted before executing the test.

LTP library will then automatically check kernel taint at the end of testing. If .all_filesystems is set in struct tst_test, taint check will be performed after each file system and taint will abort testing early with TFAIL. You can optionally also call tst_taint_check() during run(), which returns 0 or the tainted flags set in /proc/sys/kernel/tainted as specified earlier.

Depending on your kernel version, not all tainted-flags will be supported.

For reference to tainted kernels, see kernel documentation: Documentation/admin-guide/tainted-kernels.rst or https://www.kernel.org/doc/html/latest/admin-guide/tainted-kernels.html

CRC32c checksum generation is supported by LTP. In order to use it, the test should include tst_checksum.h header, then can call tst_crc32c().

Some tests may need specific kernel drivers, either compiled in, or built as a module. If .needs_drivers points to a NULL terminated array of kernel module names these are all checked and the test exits with TCONF on the first missing driver.

Since it relies on modprobe command, the check will be skipped if the command itself is not available on the system.

LTP library can be instructed to save and restore value of specified (/proc|sys) files. This is achieved by initialized tst_test struct field save_restore. It is a NULL terminated array of strings where each string represents a file, whose value is saved at the beginning and restored at the end of the test. Only first line of a specified file is saved and restored.

Pathnames can be optionally prefixed to specify how strictly (during store) are handled errors:

• (no prefix) - test ends with TCONF, if file doesn’t exist

• ? - test prints info message and continues, if file doesn’t exist or open/read fails

• ! - test ends with TBROK, if file doesn’t exist

restore is always strict and will TWARN if it encounters any error.

Generally testcases should attempt to autodetect as much kernel features as possible based on the currently running kernel. We do have tst_check_driver() to check if functionality that could be compiled as kernel module is present on the system, disabled syscalls can be detected by checking for ENOSYS errno etc.

However in rare cases core kernel features couldn’t be detected based on the kernel userspace API and we have to resort to parse the kernel .config.

For this cases the test should set the NULL terminated .needs_kconfigs array of boolean expressions with constraints on the kconfig variables. The boolean expression consits of variables, two binary operations & and |, negation ! and correct sequence of parentesis (). Variables are expected to be in a form of "CONFIG_FOO[=bar]".

The test will continue to run if all expressions are evaluated to True. Missing variable is mapped to False as well as variable with different than specified value, e.g. CONFIG_FOO=bar will evaluate to False if the value is anything else but bar. If config variable is specified as plain CONFIG_FOO it’s evaluated to true it’s set to any value (typically =y or =m).

There are some tests that, for different reasons, might need to change the system-wide clock time. Whenever this happens, it is imperative that the clock is restored, at the end of test’s execution, taking in consideration the amount of time elapsed during that test.

In order for that to happen, struct tst_test has a variable called "restore_wallclock" that should be set to "1" so LTP knows it should: (1) initialize a monotonic clock during test setup phase and (2) use that monotonic clock to fix the system-wide clock time at the test cleanup phase.

In some cases kernel has several very similar syscalls that do either the same or very similar job. This is most noticeable on i386 where we commonly have two or three syscall versions. That is because i386 was first platform that Linux was developed on and because of that most mistakes in API happened there as well. However this is not limited to i386 at all, it’s quite common that version two syscall has added missing flags parameters or so.

In such cases it does not make much sense to copy&paste the test code over and over, rather than that the test library provides support for test variants. The idea behind test variants is simple, we run the test several times each time with different syscall variant.

The implementation consist of test_variants integer that, if set, denotes number of test variants. The test is then forked and executed test_variants times each time with different value in global tst_variant variable.

The test library supports guarded buffers, which are buffers allocated so that:

• The end of the buffer is followed by a PROT_NONE page

• The remainder of the page before the buffer is filled with random canary data

Which means that the any access after the buffer will yield a Segmentation fault or EFAULT depending on if the access happened in userspace or the kernel respectively. The canary before the buffer will also catch any write access outside of the buffer.

The purpose of the patch is to catch off-by-one bugs which happens when buffers and structures are passed to syscalls. New tests should allocate guarded buffers for all data passed to the tested syscall which are passed by a pointer.

Guarded buffers can be allocated on runtime in a test setup() by a tst_alloc() or by tst_strdup() as well as by filling up the .bufs array in the tst_test structure.

So far the tst_test structure supports allocating either a plain buffer by setting up the size or struct iovec, which is allocated recursively including the individual buffers as described by an -1 terminated array of buffer sizes.

Some tests may require the presence or absence of particular capabilities. Using the API provided by tst_capability.h the test author can try to ensure that some capabilities are either present or absent during the test.

For example; below we try to create a raw socket, which requires CAP_NET_ADMIN. During setup we should be able to do it, then during run it should be impossible. The LTP capability library will check before setup that we have this capability, then after setup it will drop it.

Look at the test struct at the bottom. We have filled in the caps field with a NULL terminated array containing two tst_cap structs. TST_CAP_REQ actions are executed before setup and TST_CAP_DROP are executed after setup. This means it is possible to both request and drop a capability.

Here we request CAP_NET_RAW, but drop CAP_SYS_ADMIN. If the capability is in the permitted set, but not the effective set, the library will try to permit it. If it is not in the permitted set, then it will fail with TCONF.

This API does not require libcap to be installed. However it has limited features relative to libcap. It only tries to add or remove capabilities from the effective set. This means that tests which need to spawn child processes may have difficulties ensuring the correct capabilities are available to the children (see the capabilities (7) manual pages).

However a lot of problems can be solved by using tst_cap_action(struct tst_cap *cap) directly which can be called at any time. This also helps if you wish to drop a capability at the begining of setup.

If a bug is caused by two tasks in the kernel racing and you wish to create a regression test (or bug-fix validation test) then the tst_fuzzy_sync.h library should be used.

It allows you to specify, in your code, two race windows. One window in each thread’s loop (triggering a race usually requires many iterations). These windows show fuzzy-sync where the race can happen. They don’t need to be exact, hence the fuzzy part. If the race condition is not immediately triggered then the library will begin experimenting with different timings.

Above is a minimal template for a test using fuzzy-sync. In a simple case, you just need to put the bits you want to race inbetween start_race and end_race. Meanwhile, any setup you need to do per-iteration goes outside the windows.

Fuzzy sync synchronises run_a and run_b, which act as barriers, so that neither thread can progress until the other has caught up with it. There is also the pair_wait function which can be used to add barriers in other locations. Of course start/end_race_a/b are also a barriers.

The library decides how long the test should run for based on the timeout specified by the user plus some other heuristics.

For full documentation see the comments in include/tst_fuzzy_sync.h.

Many of the LTP tests need to use hugepage in their testing, this allows the test can reserve hugepages from system only via .request_hugepages = xx.

If set non-zero number of request_hugepages, test will try to reserve the expected number of hugepage for testing in setup phase. If system does not have enough hpage for using, it will try the best to reserve 80% available number of hpages. With success test stores the reserved hugepage number in tst_hugepages. For the system without hugetlb supporting, variable tst_hugepages will be set to 0. If the hugepage number needs to be set to 0 on supported hugetlb system, please use .request_hugepages = TST_NO_HUGEPAGES.

Also, we do cleanup and restore work for the hpages resetting automatically.

or,

Required commands can be checked with .needs_cmds, which points to a NULL terminated array of strings such as:

Using TST_ASSERT_INT/STR(path, val) to assert that integer value or string stored in the prefix field of file pointed by path equals to the value passed to this function.

Also having a similar api pair TST_ASSERT_FILE_INT/STR(path, prefix, val) to assert the field value of file.

Some LTP tests need specific Control Group configurations. tst_cgroup.h provides APIs to discover and use CGroups. There are many differences between CGroups API V1 and V2. We encapsulate the details of configuring CGroups in high-level functions which follow the V2 kernel API. Allowing one to use CGroups without caring too much about the current system’s configuration.

Also, the LTP library will automatically mount/umount and configure the CGroup hierarchies if that is required (e.g. if you run the tests from init with no system manager).

Above, we first ensure the memory controller is available on the test’s CGroup with tst_cgroup_require. We then get a structure, cg, which represents the test’s CGroup. Note that tst_cgroup_get_test_group should not be called many times, as it is allocated in a guarded buffer (See section 2.2.31). Therefor it is best to call it once in setup and not run because run may be repeated with the -i option.

We then write the current processes PID into cgroup.procs, which moves the current process into the test’s CGroup. After which we set the maximum memory size by writing to memory.max. If the memory controller is mounted on CGroups V1 then the library will actually write to memory.limit_in_bytes. As a general rule, if a file exists on both CGroup versions, then we use the V2 naming.

Some controller features, such as memory.swap, can be disabled. Therefor we need to check if they exist before accessing them. This can be done with SAFE_CGROUP_HAS which can be called on any control file or feature.

Most tests only require setting a few limits similar to the above. In such cases the differences between V1 and V2 are hidden. Setup and cleanup is also mostly hidden. However things can get much worse.

Starting with setup; we can see here that we also fetch the drain CGroup. This is a shared group (between parallel tests) which may contain processes from other tests. It should have default settings and these should not be changed by the test. It can be used to remove processes from other CGroups incase the hierarchy root is not accessible.

In run, we first create a child CGroup with tst_cgroup_mk. As we create this CGroup in run we should also remove it at the end of run. We also need to check if it exists and remove it in cleanup as well. Because there are SAFE_ functions which may jump to cleanup.

We then move the main test process into the child CGroup. This is important as it means that before we destroy the child CGroup we have to move the main test process elsewhere. For that we use the drain group.

Next we enable the memory and cpuset controller configuration on the test CGroup’s descendants (i.e. cg_child). This allows each child to have its own settings. The file cgroup.subtree_control does not exist on V1. Because it is possible to have both V1 and V2 active at the same time. We can not simply check if subtree_control exists before writing to it. We have to check if a particular controller is on V2 before trying to add it to subtree_control. Trying to add a V1 controller will result in ENOENT.

We then fork a child process and add this to the child CGroup. Within the child process we try to read memory.swap.current. It is possible that the memory controller was compiled without swap support, so it is necessary to check if memory.swap is enabled. That is unless the test will never reach the point where memory.swap.* are used without swap support.

The parent process waits for the child process to be reaped before destroying the child CGroup. So there is no need to transfer the child to drain. However the parent process must be moved otherwise we will get EBUSY when trying to remove the child CGroup.

Another example of an edge case is the following.

CGroups V2 introduced a feature where memory[.swap].max could be set to "max". This does not appear to work on V1 limit_in_bytes however. For most tests, simply using a large number is sufficient and there is no need to use "max". Importantly though, one should be careful to read both the V1 and V2 kernel docs. The LTP library can not handle all edge cases. It does the minimal amount of work to make testing on both V1 and V2 feasible.

Some tests require more than specific number of CPU. It can be defined with .min_cpus = N.

Test tags are name-value pairs that can hold any test metadata.

We have additional support for CVE entries, git commit in mainline kernel, stable kernel or glibc git repository. If a test is a regression test it should include these tags. They are printed when test fails and exported into documentation.

CVE, mainline and stable kernel git commits in a regression test for a kernel bug:

Glibc git commit in a regression test for a glibc bug:

Testcases for specific arch should be limited on that only being supported platform to run, we now involve a .supported_archs to achieve this feature in LTP library. All you need to run a test on the expected arch is to set the .supported_archs array in the struct tst_test to choose the required arch list. e.g.

This helps move the TCONF info from code to tst_test metadata as well.

And, we also export a struct tst_arch to save the system architecture for using in the whole test cases.

I’ve been hit by this one several times already…​ When you create files with open() or creat() etc, the mode specified as the last parameter is not the mode the file is created with. The mode depends on current umask() settings which may clear some of the bits. If your test depends on specific file permissions you need either to change umask to 0 or chmod() the file afterwards or use SAFE_TOUCH() that does the chmod() for you.

If access(some_file, W_OK) is executed by root, it will return success even if the file doesn’t have write permission bits set (the same holds for R_OK too). For sysfs files you can use open() as a workaround to check file read/write permissions. It might not work for other filesystems, for these you have to use stat(), lstat() or fstat().

Various desktop daemons (gvfsd-trash is known for that) may be stupid enough to probe all newly mounted filesystem which results in umount(2) failing with EBUSY; use tst_umount() described in 1.19 that retries in this case instead of plain umount(2).

Be vary that if a process calls fork(2) the child process inherits open descriptors as well as copy of the parent memory so especially if there are any open FILE buffers with a data in them they may be written both by the parent and children resulting in corrupted/duplicated data in the resulting files.

Also open FILE streams are flushed and closed at exit(3) so if your program works with FILE streams, does fork(2), and the child may end up calling exit(3) you will likely end up with corrupted files.

The solution to this problem is either simply call fflush(NULL) that flushes all open output FILE streams just before doing fork(2). You may also use _exit(2) in child processes which does not flush FILE buffers and also skips atexit(3) callbacks.