This page is part of the TestSuite project and provides a tutorial on how to add new tests to the test suite and a reference guide for developers. It is assumed that you have already gone over TestSuite/Structure, especially the section detailing the naming scheme for the tests.
This document is based on a similar tutorial in NetBSD and a lot of text has been copied form there.
Adding a new test case
To add a new test case to the source tree, check if any existing test program can assimilate your test case. What you are looking for here is a test program with a generic-enough name that can encapsulate the behavior of the new test you want to add. If test programs use the right naming scheme, and if the code you are modifying already has tests, it is very likely that you will find a match.
If you find such a program, bingo! Just add the test case to it: no other changes are required. Otherwise, you will have to create a new test program.
Adding a new test program
If you need to add a new test program to the source tree, you first need to consider the type of test program you will be creating. The recommendation is that you create an ATF-based test program; the rationale being that these are more versatile both from a development perspective and the user's experience. Only resort to adding a plain test program as a transitional step or, especially, if you do not own the code of the test (e.g. you are just hooking a test from a third-party package bundled in the base system).
The procedure to add a new test program is the following:
Locate the appropriate subdirectory in which to put your test program. In general this just means the tests subdirectory located where the code you are interested in testing lives. It is OK and expected to have multiple test programs into the same directory. Avoid creating a directory per test program!
If the tests subdirectory exists:
Choose a sane name for the test program. Avoid naming the test program the same as your test case! If you find yourself in this situation, it probably means that you are choosing a bad name for either of them.
Create the test program source file. Refer to src/share/examples/tests/ for test program sample code.
Add the new test program to the existing Makefile.
If the tests subdirectory does not exist:
Create an empty tests subdirectory.
- Do the same as above.
Create the Makefile for the directory. Refer to src/share/examples/tests/ for Makefile sample code.
Edit the parent Makefile to recurse into the new subdirectory.
Edit src/etc/mtree/BSD.tests.dist to register the new subdirectory. Note that if you are adding tests to, say, src/usr.bin/du/tests/, the directory you register in the mtree file should be /usr/tests/usr.bin/du/; i.e. the layout under /usr/tests/ must match the layout of /usr/src.
FAQ for ATF test programs
Where can I find sample code?
Refer to src/share/examples/tests/tests/atf/ for sample test program code and the supporting build code.
This section provides generic documentation on how to write ATF-based test programs both using the C interface and the shell interface.
Where are the APIs documented?
See atf-c-api(3), atf-c++-api(3) and atf-sh-api(3) for details on the C, C++ and shell APIs respectively. These manual pages are part of the base system.
Do I need to remove any temporary files I create?
No! Kyua does this automatically for you. Every test case is executed in its process and its own temporary subdirectory. Having code to remove files or to perform any other kind of cleanup is usually wrong and you should avoid it. Keep things simple and let the infrastructure do the hard work for you!
When do I use ATF_CHECK and when ATF_REQUIRE?
ATF_CHECK logs errors but does not abort the execution of the test program. ATF_REQUIRE logs errors in a similar way but immediately terminates the execution.
You can use this distinction in the following way: use ATF_REQUIRE to check the code that "prepares" your test case. Use ATF_CHECK to do the actual functionality tests once all the set up has been performed.
FAQ for plain test programs
Where can I find sample code?
Refer to src/share/examples/tests/tests/plain/ for sample test program code and the supporting build code.