Regression tests
Regression tests for nsswitch-modules (i.e. nsswitch-related libc functions) are located in src/tools/regression/lib/libc/nss.
Concepts
Nsswitch regression tests use the ideas, described in ../NsswitchRegressionDescription. Each test operates with the sequence of the particular get***ent() function results. For each item in the sequence it does the straight query (i.e. get**byname()) and reverse query (i.e. get**byid()). It also checks that 2 sequences of get**ent() results are equal. Each test can build a snapshot – write the get**ent() results to a file. The file can later be read by the test and current get**byname(), get**byid() and get**ent() results would be checked to be equal with results from the snapshot.
Snapshots can be very useful to check, for example, that nss-modules and libc separation won’t break the system’s behavior. I.e. snapshots help to check that system behavior doesn’t change after system upgrade.
Implementation details
All nsswitch regression tests use the testutil.h file which contains some general test routines, implemented as several macros. All macros can be divided into several parts:
TEST_DATA_** macros
DECLARE_TEST_DATA(ent) and IMPLEMENT_TEST_DATA(ent) declare the ent##_test_data structure and implement functions, which manipulate it. This structure is the container of the results sequence, where each item is of type “struct ent”. Implemented functions are never called directly – special TEST_DATA_* macros are used instead (TEST_DATA_APPEND(ent, td, d), for example).
TEST_FILE_SNAPSHOT_** macros
DECLATE_TEST_FILE_SNAPSHOT(ent) and IMPLEMENT_TEST_FILE_SNAPSHOT(ent) declare and implement 2 functions, which can be accessed via appropriate macros. TEST_SNAPSHOT_FILE_WRITE(ent, fname, td, f) writes the ent##_test_data structure to a file. It converts each item of the sequence to a string, using the provided function f. TEST_SNAPSHOT_FILE_READ(ent, fname, td, f) reads contents of the file into the ent##_test_data structure. It uses the provided function f to convert char * line to the struct ent.
DO_1PASS_TEST and DO_2PASS_TEST macros
DECLARE_1PASS_TEST(ent) and IMPLEMENT_1PASS_TEST(ent) declare and implement the so called “1 pass” test function. It can be called using the DO_1PASS_TEST(ent, td, f, mdata) macros. It takes each item of the ent##_test_data sequence and applies the provided function f to it, returning f result if it’s not 0.
DECLARE_2PASS_TEST(ent) and IMPLEMENT_2PASS_TEST(ent) declare and implement the so called “2 pass” test function. It can be called using the DO_2PASS_TEST(ent, td1, td2, f, mdata). It simply tests 2 ent##_test_data sequences for equality. The test is called “2 pass” because it is assumed, that filling td1 sequence with data is the first pass and filling td2 is the second pass.
These macros are used in every nsswitch test. Each test can be associated with the particular structure (it is called “ent” in previous macros definitions) – test-getserv is associated with struct servent, for example. For this structures comparison, cloning and deletion routines are implemented. get**byname(), get**byid() and get**ent() tests functions are implemented. There is also a small issue with testing the get**byname() and get**byid() behavior. We use one of get**ent() results to make a query and as a model to check if the get**byname() or get**byid() result is correct. But in some cases this check will always give “false” because of ambiguation. For example, we have service “A” which uses port 1001. And service “B” which uses port 1001. We are testing getservbyport() function with service “B”. It means that we’ll search for all results with port 1001. But the normal behavior is for service “A” to be found instead of service “B”, because it is located earlier in /etc/services. So, If we blindly compare the model and the result, we’ll have the negative result when everything is working correctly. To solve this issue we should do 2-stage check: 1) check the model and the result for equality 2) if they differ, check the result to be present in the whole get**ent() results sequence. Such a check eliminates the issue.
During the testing each received structure is also checked for correctness. Usually correctness means “no NULL pointers and correct (usually – positive) numerical values”.
When using the snapshot, its records are used instead of get**ent() results as the models for testing all get**by**() functions.
Tests interface
Each test has a debug mode (“-d”) which enables debug output. “-n” flag runs the test for get**byname() function, “-e” – for get**ent(). The test for “get**byid()” is associated with different letters – depending on the test (“g” for test-getgr, “u” – for test-getpw, “p” – for test-getserv).
“-s <file>” will cause the snapshot to be built, if it’s not already exist. When “-s <file>” combined with any of the previously defined arguments it causes the <file> contents and not the get**ent() results to be used as the model.