LTP Shell Test API

Note	See also Test Writing Guidelines, C Test API.

1 Writing a testcase in shell

LTP supports testcases to be written in a portable shell too.

There is a shell library modeled closely to the C interface at testcases/lib/tst_test.sh.

Warning	All identifiers starting with TST_ or tst_ are reserved for the test library.

1.1 Basic test interface

#!/bin/sh
# SPDX-License-Identifier: GPL-2.0-or-later
# This is a basic test for true shell builtin

TST_TESTFUNC=do_test
. tst_test.sh

do_test()
{
	true
	ret=$?

	if [ $ret -eq 0 ]; then
		tst_res TPASS "true returned 0"
	else
		tst_res TFAIL "true returned $ret"
	fi
}

tst_run

Tip	To execute this test the tst_test.sh library must be in $PATH. If you are executing the test from a git checkout you can run it as PATH="$PATH:../../lib" ./foo01.sh

The shell library expects test setup, cleanup and the test function executing the test in the $TST_SETUP, $TST_CLEANUP and $TST_TESTFUNC variables.

Both $TST_SETUP and $TST_CLEANUP are optional.

The $TST_TESTFUNC may be called several times if more than one test iteration was requested by passing right command line options to the test.

The $TST_CLEANUP may be called even in the middle of the setup and must be able to clean up correctly even in this situation. The easiest solution for this is to keep track of what was initialized and act accordingly in the cleanup.

Warning	Similar to the C library, calling tst_brk in the $TST_CLEANUP does not exit the test and TBROK is converted to TWARN.

Notice also the tst_run shell API function called at the end of the test that actually starts the test.

Warning	cleanup function is called only after tst_run has been started. Calling tst_brk in shell libraries, e.g. tst_test.sh or tst_net.sh does not trigger calling it.

#!/bin/sh
# SPDX-License-Identifier: GPL-2.0-or-later
# Example test with tests in separate functions

TST_TESTFUNC=test
TST_CNT=2
. tst_test.sh

test1()
{
	tst_res TPASS "Test $1 passed"
}

test2()
{
	tst_res TPASS "Test $1 passed"
}

tst_run
# output:
# foo 1 TPASS: Test 1 passed
# foo 2 TPASS: Test 2 passed

If $TST_CNT is set, the test library looks if there are functions named ${TST_TESTFUNC}1, …, ${TST_TESTFUNC}${TST_CNT} and if these are found they are executed one by one. The test number is passed to it in the $1.

#!/bin/sh
# SPDX-License-Identifier: GPL-2.0-or-later
# Example test with tests in a single function

TST_TESTFUNC=do_test
TST_CNT=2
. tst_test.sh

do_test()
{
	case $1 in
	1) tst_res TPASS "Test $1 passed";;
	2) tst_res TPASS "Test $1 passed";;
	esac
}

tst_run
# output:
# foo 1 TPASS: Test 1 passed
# foo 2 TPASS: Test 2 passed

Otherwise, if $TST_CNT is set but there is no ${TST_TESTFUNC}1, etc., the $TST_TESTFUNC is executed $TST_CNT times and the test number is passed to it in the $1.

#!/bin/sh
# SPDX-License-Identifier: GPL-2.0-or-later
# Example test with tests in a single function, using $TST_TEST_DATA and
# $TST_TEST_DATA_IFS

TST_TESTFUNC=do_test
TST_TEST_DATA="foo:bar:d dd"
TST_TEST_DATA_IFS=":"
. tst_test.sh

do_test()
{
	tst_res TPASS "Test $1 passed with data '$2'"
}

tst_run
# output:
# foo 1 TPASS: Test 1 passed with data 'foo'
# foo 2 TPASS: Test 1 passed with data 'bar'
# foo 3 TPASS: Test 1 passed with data 'd dd'

It’s possible to pass data for function with $TST_TEST_DATA. Optional $TST_TEST_DATA_IFS is used for splitting, default value is space.

#!/bin/sh
# SPDX-License-Identifier: GPL-2.0-or-later
# Example test with tests in a single function, using $TST_TEST_DATA and $TST_CNT

TST_TESTFUNC=do_test
TST_CNT=2
TST_TEST_DATA="foo bar"
. tst_test.sh

do_test()
{
	case $1 in
	1) tst_res TPASS "Test $1 passed with data '$2'";;
	2) tst_res TPASS "Test $1 passed with data '$2'";;
	esac
}

tst_run
# output:
# foo 1 TPASS: Test 1 passed with data 'foo'
# foo 2 TPASS: Test 2 passed with data 'foo'
# foo 3 TPASS: Test 1 passed with data 'bar'
# foo 4 TPASS: Test 2 passed with data 'bar'

$TST_TEST_DATA can be used with $TST_CNT. If $TST_TEST_DATA_IFS not specified, space as default value is used. Of course, it’s possible to use separate functions.

1.2 Library environment variables and functions for shell

Similarily to the C library various checks and preparations can be requested simply by setting right $TST_NEEDS_FOO.

Variable name	Action done
TST_NEEDS_ROOT	Exit the test with TCONF unless executed under root.
	Alternatively the tst_require_root command can be used.
TST_NEEDS_TMPDIR	Create test temporary directory and cd into it.
TST_NEEDS_DEVICE	Prepare test temporary device, the path to testing device is stored in $TST_DEVICE variable. The option implies TST_NEEDS_TMPDIR.
TST_NEEDS_CMDS	String with command names that has to be present for the test (see below).
TST_NEEDS_MODULE	Test module name needed for the test (see below).
TST_NEEDS_DRIVERS	Checks kernel drivers support for the test.
TST_TIMEOUT	Maximum timeout set for the test in sec. Must be int >= 1, or -1 (special value to disable timeout), default is 300. Variable is meant be set in tests, not by user. It’s an equivalent of `tst_test.timeout` in C, can be set via tst_set_timeout(timeout) after test has started.

Function name	Action done
tst_set_timeout(timeout)	Maximum timeout set for the test in sec. See TST_TIMEOUT variable.

Note	Network tests (see testcases/network/README.md) use additional variables and functions in tst_net.sh.

Checking for presence of commands

#!/bin/sh

...

TST_NEEDS_CMDS="modinfo modprobe"
. tst_test.sh

...

Setting $TST_NEEDS_CMDS to a string listing required commands will check for existence each of them and exits the test with TCONF on first missing.

Alternatively the tst_require_cmds() function can be used to do the same on runtime, since sometimes we need to the check at runtime too.

tst_check_cmds() can be used for requirements just for a particular test as it doesn’t exit (it issues tst_res TCONF). Expected usage is:

#!/bin/sh

TST_TESTFUNC=do_test
. tst_test.sh

do_test()
{
	tst_check_cmds cmd || return
	cmd --foo
	...
}

tst_run

Locating kernel modules

The LTP build system can build kernel modules as well, setting $TST_NEEDS_MODULE to module name will cause the library to look for the module in a few possible paths.

If module was found the path to it will be stored into $TST_MODPATH variable, if module wasn’t found the test will exit with TCONF.

Alternatively the tst_require_module() function can be used to do the same at runtime.

1.3 Optional command line parameters

#!/bin/sh
# SPDX-License-Identifier: GPL-2.0-or-later
# Optional test command line parameters

TST_OPTS="af:"
TST_USAGE=usage
TST_PARSE_ARGS=parse_args
TST_TESTFUNC=do_test

. tst_test.sh

ALTERNATIVE=0
MODE="foo"

usage()
{
	cat << EOF
usage: $0 [-a] [-f <foo|bar>]

OPTIONS
-a     Enable support for alternative foo
-f     Specify foo or bar mode
EOF
}

parse_args()
{
	case $1 in
	a) ALTERNATIVE=1;;
	f) MODE="$2";;
	esac
}

do_test()
{
	...
}

tst_run

The getopts string for optional parameters is passed in the $TST_OPTS variable. There are a few default parameters that cannot be used by a test, these can be listed with passing help -h option to any test.

The function that prints the usage is passed in $TST_USAGE, the help for the options implemented in the library is appended when usage is printed.

Lastly the function $PARSE_ARGS is called with the option name in the $1 and, if option has argument, its value in the $2.

#!/bin/sh
# SPDX-License-Identifier: GPL-2.0-or-later
# Optional test positional parameters

TST_POS_ARGS=3
TST_USAGE=usage
TST_TESTFUNC=do_test

. tst_test.sh

usage()
{
	cat << EOF
usage: $0 [min] [max] [size]

EOF
}

min="$1"
max="$2"
size="$3"

do_test()
{
	...
}

tst_run

You can also request a number of positional parameters by setting the $TST_POS_ARGS variable. If you do, these will be available as they were passed directly to the script in $1, $2, …, $n.

1.4 Useful library functions

Retrieving configuration variables

You may need to retrieve configuration values such as PAGESIZE, there is getconf but as some system may not have it, you are advised to use tst_getconf instead. Note that it implements subset of getconf system variables used by the testcases only.

# retrieve PAGESIZE
pagesize=`tst_getconf PAGESIZE`

Sleeping for subsecond intervals

Albeit there is a sleep command available basically everywhere not all implementations can support sleeping for less than one second. And most of the time sleeping for a second is too much. Therefore LTP includes tst_sleep that can sleep for defined amount of seconds, milliseconds or microseconds.

# sleep for 100 milliseconds
tst_sleep 100ms

Retry a function call multiple times

Sometimes an LTP test needs to retry a function call multiple times because the system is not ready to process it successfully on the first try. The LTP library has useful tools to handle the call retry automatically. TST_RETRY_FUNC() will keep retrying for up to 1 second. If you want a custom time limit use TST_RETRY_FN_EXP_BACKOFF(). Both methods return the value returned by the last FUNC call.

The delay between retries starts at 1 microsecond and doubles after each call. The retry loop ends when the function call succeeds or when the next delay exceeds the specified time (1 second for TST_RETRY_FUNC()). The maximum delay is multiplied by TST_TIMEOUT_MUL. The total cumulative delay may be up to twice as long as the adjusted maximum delay.

The C version of TST_RETRY_FUNC() is a macro which takes two arguments:

FUNC is the complete function call with arguments which should be retried multiple times.
SUCCESS_CHECK is a macro or function which will validate FUNC return value. FUNC call was successful if SUCCESS_CHECK(ret) evaluates to non-zero.

Both retry methods clear errno before every FUNC call so your SUCCESS_CHECK can look for specific error codes as well. The LTP library also includes predefined SUCCESS_CHECK macros for the most common call conventions:

TST_RETVAL_EQ0() - The call was successful if FUNC returned 0 or NULL
TST_RETVAL_NOTNULL() - The call was successful if FUNC returned any value other than 0 or NULL.
TST_RETVAL_GE0() - The call was successful if FUNC returned value >= 0.

/* Keep trying for 1 second */
TST_RETRY_FUNC(FUNC, SUCCESS_CHECK)

/* Keep trying for up to 2*N seconds */
TST_RETRY_FN_EXP_BACKOFF(FUNC, SUCCESS_CHECK, N)

The shell version of TST_RETRY_FUNC() is simpler and takes slightly different arguments:

FUNC is a string containing the complete function or program call with arguments.
EXPECTED_RET is a single expected return value. FUNC call was successful if the return value is equal to EXPECTED_RET.

# Keep trying for 1 second
TST_RETRY_FUNC "FUNC arg1 arg2 ..." "EXPECTED_RET"

# Keep trying for up to 2*N seconds
TST_RETRY_FN_EXP_BACKOFF "FUNC arg1 arg2 ..." "EXPECTED_RET" "N"

Checking for integers

# returns zero if passed an integer parameter, non-zero otherwise
tst_is_int "$FOO"

Checking for integers and floating point numbers

# returns zero if passed an integer or floating point number parameter,
# non-zero otherwise
tst_is_num "$FOO"

Obtaining random numbers

There is no $RANDOM in portable shell, use tst_random instead.

# get random integer between 0 and 1000 (including 0 and 1000)
tst_random 0 1000

Formatting device with a filesystem

The tst_mkfs helper will format device with the filesystem.

# format test device with ext2
tst_mkfs ext2 $TST_DEVICE
# default params are $TST_FS_TYPE $TST_DEVICE
tst_mkfs
# optional parameters
tst_mkfs ext4 /dev/device -T largefile

Mounting and unmounting filesystems

The tst_mount and tst_umount helpers are a safe way to mount/umount a filesystem.

The tst_mount mounts $TST_DEVICE of $TST_FS_TYPE (optional) to $TST_MNTPOINT (defaults to mntpoint), optionally using the $TST_MNT_PARAMS. The $TST_MNTPOINT directory is created if it didn’t exist prior to the function call.

If the path passed (optional, must be absolute path, defaults to $TST_MNTPOINT) to the tst_umount is not mounted (present in /proc/mounts) it’s noop. Otherwise it retries to umount the filesystem a few times on failure. This is a workaround since there are daemons dumb enough to probe all newly mounted filesystems, and prevents them from being umounted shortly after they were mounted.

ROD and ROD_SILENT

These functions supply the SAFE_MACROS used in C although they work and are named differently.

ROD_SILENT command arg1 arg2 ...

# is shorthand for:

command arg1 arg2 ... > /dev/null 2>&1
if [ $? -ne 0 ]; then
        tst_brk TBROK "..."
fi


ROD command arg1 arg2 ...

# is shorthand for:

ROD arg1 arg2 ...
if [ $? -ne 0 ]; then
        tst_brk TBROK "..."
fi

Warning	Keep in mind that output redirection (to a file) happens in the caller rather than in the ROD function and cannot be checked for write errors by the ROD function.

As a matter of a fact doing ROD echo a > /proc/cpuinfo would work just fine since the ROD function will only get the echo a part that will run just fine.

# Redirect output to a file with ROD
ROD echo foo \> bar

Note the > is escaped with \, this causes that the > and filename are passed to the ROD function as parameters and the ROD function contains code to split $@ on > and redirects the output to the file.

EXPECT_PASS{,_BRK} and EXPECT_FAIL{,_BRK}

EXPECT_PASS command arg1 arg2 ... [ \> file ]
EXPECT_FAIL command arg1 arg2 ... [ \> file ]

EXPECT_PASS calls tst_res TPASS if the command exited with 0 exit code, and tst_res TFAIL otherwise. EXPECT_FAIL does vice versa.

Output redirection rules are the same as for the ROD function. In addition to that, EXPECT_FAIL always redirects the command’s stderr to /dev/null.

There are also EXPECT_PASS_BRK and EXPECT_FAIL_BRK, which works the same way except breaking a test when unexpected action happen.

It’s possible to detect whether expected value happened:

if ! EXPECT_PASS command arg1 2\> /dev/null; then
	continue
fi

tst_kvcmp

This command compares the currently running kernel version given conditions with syntax similar to the shell test command.

# Exit the test if kernel version is older or equal to 2.6.8
if tst_kvcmp -le 2.6.8; then
	tst_brk TCONF "Kernel newer than 2.6.8 is needed"
fi

# Exit the test if kernel is newer than 3.8 and older than 4.0.1
if tst_kvcmp -gt 3.8 -a -lt 4.0.1; then
	tst_brk TCONF "Kernel must be older than 3.8 or newer than 4.0.1"
fi

expression	description
-eq kver	Returns true if kernel version is equal
-ne kver	Returns true if kernel version is not equal
-gt kver	Returns true if kernel version is greater
-ge kver	Returns true if kernel version is greater or equal
-lt kver	Returns true if kernel version is lesser
-le kver	Returns true if kernel version is lesser or equal
-a	Does logical and between two expressions
-o	Does logical or between two expressions

The format for kernel version has to either be with one dot e.g. 2.6 or with two dots e.g. 4.8.1.

tst_fs_has_free

#!/bin/sh

...

# whether current directory has 100MB free space at least.
if ! tst_fs_has_free . 100MB; then
	tst_brkm TCONF "Not enough free space"
fi

...

The tst_fs_has_free shell interface returns 0 if the specified free space is satisfied, 1 if not, and 2 on error.

The second argument supports suffixes kB, MB and GB, the default unit is Byte.

tst_retry

#!/bin/sh

...

# Retry ping command three times
tst_retry "ping -c 1 127.0.0.1"

if [ $? -ne 0 ]; then
	tst_resm TFAIL "Failed to ping 127.0.0.1"
else
	tst_resm TPASS "Successfully pinged 127.0.0.1"
fi

...

The tst_retry function allows you to retry a command after waiting small amount of time until it succeeds or until given amount of retries has been reached (default is three attempts).

1.5 Restarting daemons

Restarting system daemons is a complicated task for two reasons.

There are different init systems (SysV init, systemd, etc…)
Daemon names are not unified between distributions (apache vs httpd, cron vs crond, various syslog variations)

To solve these problems LTP has testcases/lib/daemonlib.sh library that provides functions to start/stop/query daemons as well as variables that store correct daemon name.

Table 1. Supported operations
start_daemon()	Starts daemon, name is passed as first parameter.
stop_daemon()	Stops daemon, name is passed as first parameter.
restart_daemon()	Restarts daemon, name is passed as first parameter.
status_daemon()	Detect daemon status (exit code: 0: running, 1: not running).

Table 2. Variables with detected names
CROND_DAEMON	Cron daemon name (cron, crond).
SYSLOG_DAEMON	Syslog daemon name (syslog, syslog-ng, rsyslog).

Cron daemon restart example

#!/bin/sh
# SPDX-License-Identifier: GPL-2.0-or-later
# Cron daemon restart example

TCID=cron01
TST_COUNT=1
. test.sh
. daemonlib.sh

...

restart_daemon $CROND_DAEMON

...

tst_exit

1.6 Access to the checkpoint interface

The shell library provides an implementation of the checkpoint interface compatible with the C version. All TST_CHECKPOINT_* functions are available.

In order to initialize checkpoints $TST_NEEDS_CHECKPOINTS must be set to 1 before the inclusion of tst_test.sh:

#!/bin/sh

TST_NEEDS_CHECKPOINTS=1
. tst_test.sh

Since both the implementations are compatible, it’s also possible to start a child binary process from a shell test and synchronize with it. This process must have checkpoints initialized by calling tst_reinit().

Shell Test API - pevik/ltp GitHub Wiki

LTP Shell Test API

1 Writing a testcase in shell

1.1 Basic test interface

1.2 Library environment variables and functions for shell

Checking for presence of commands

Locating kernel modules

1.3 Optional command line parameters

1.4 Useful library functions

Retrieving configuration variables

Sleeping for subsecond intervals

Retry a function call multiple times

Checking for integers

Checking for integers and floating point numbers

Obtaining random numbers

Formatting device with a filesystem

Mounting and unmounting filesystems

ROD and ROD_SILENT

EXPECT_PASS{,_BRK} and EXPECT_FAIL{,_BRK}

tst_kvcmp

1.5 Restarting daemons

1.6 Access to the checkpoint interface

⚠️ GitHub.com Fallback ⚠️

Shell Test API - pevik/ltp GitHub Wiki

LTP Shell Test API

1 Writing a testcase in shell

1.1 Basic test interface

1.2 Library environment variables and functions for shell

Checking for presence of commands

Locating kernel modules

1.3 Optional command line parameters

1.4 Useful library functions

Retrieving configuration variables

Sleeping for subsecond intervals

Retry a function call multiple times

Checking for integers

Checking for integers and floating point numbers

Obtaining random numbers

Formatting device with a filesystem

Mounting and unmounting filesystems

ROD and ROD_SILENT

EXPECT_PASS{,_BRK} and EXPECT_FAIL{,_BRK}

tst_kvcmp

1.5 Restarting daemons

1.6 Access to the checkpoint interface

⚠️ **GitHub.com Fallback** ⚠️

⚠️ GitHub.com Fallback ⚠️