• SummerOfCode2019Projects
• FirewallTestSuite

Firewall Test Suite

• Student: Ahsan Barkati (<ahsanb@freebsd.org>)

• Mentors:

  • Kristof Provost (<kp@freebsd.org>)

  • Tom Jones (<thj@freebsd.org>)

The Code

The Common Firewall Test Suite can be found here: https://github.com/ahsanbarkati/freebsd/tree/projects/firewall_tests/tests/sys/netpfil/common

• The common tests sub-directory within FreeBSD source tree: tests/sys/netpfil/common

Differential revisions Submitted

Bugs Reported

Various blog posts were written during the development of this framework. These can be found here

Project Overview

There are three firewalls in FreeBSD; pf, ipfw and ipf. FreeBSD is one of the very rapidly growing operating system and this fact brings in the need for testing and monitoring of various sub-systems within it. Firewalls are one of the most important components of the network. So writing tests for all the three firewalls is an important task. But because of commonalities in the features of the three firewalls with obvious differences, there will be code duplicacy in the tests of them. This project aims in creating a framework in which a common test can be written and that should be able to test all the three firewalls.

How to use the framework

A framework is designed using which a single test body can be written and the framework will generate separate tests for multiple firewalls. This test design reduces the duplicate code which might have existed if the tests were to be written separately. During the development of this test structure, we were able to catch up a few bugs and this serves as a great motivation for developers to write more tests for the firewalls. The tests can be easily written by following the method described below:

Every test needs to source the utils.subr and runner.subr scripts. The utils.subr contain various utility functions and the runner.subr contains methods to generate tests for multiple firewalls from a single test body. The source is done by the following lines:

. $(atf_get_srcdir)/utils.subr
. $(atf_get_srcdir)/runner.subr

The atf_get_srcdir() is used to get the path of the test directory. After the above mentioned two files are sourced, the tests can be defined. A single test file can contain multiple test cases. For the purpose of illustration, the test cases, testA is considered in the following discussion. The test programs follow this template:

testA_head()
{
    atf_set "descr" "testA for IPv4"
    ...
}
testA_body()
{
    firewall=$1
    firewall_init $firewall
    ...
}
testA_cleanup()
{
    firewall=$1
    firewall_cleanup $firewall
}
setup_tests "testA" \
                "pf" \
                "ipfw" \
                "ipf"

Each test case has three sections, the header, the body and the cleanup routine. The head is the same as that of a general ATF test's head and it may include the test description, the required user or programs etc. The body of the test case is the section is where the test is actually defined. For this testing body to be a general test body, it is passed the name of the firewall during setting up the tests bu setup_tests(). This name is used to specify the configuration for each firewall for testing. For configuring a firewall the firewall_config() function is used. It is used in the following way:

firewall_config "jail_name" ${firewall} \
        "pf" \
            "rule1 for pf" \ 
            "rule2 for pf" \
        "ipfw" \
            "rule1 for ipfw" \
        "ipf" \
            "rule1 for ipf" \
            "rule2 for ipf"

This function has jail_name as the first argument and the name of the firewall as the second argument, followed by the configuration rules for each firewall in the above-mentioned format. The cleanup function is to be used to undo every change that was made during the test. This change can include the creation of jails, creation of files etc. There can be changes specific to firewall as well for some tests and hence it is also passed the name of the firewall by setup_tests() and the function firewall_cleanup $firewall_name is called within the cleanup function(). Afer the head(), body() and cleanup() is defined for every test case, the setup_tests is specified the firewalls for which to run each testcases in the above mentioned format. setup_tests uses the general head(), body() and cleanup() to generate specific tests for each firewall. An example of working test can be found here /usr/tests/sys/netpfil/common/pass_block.sh.

How to run the tests

Once the common test is written and installed it can be run by using kyua. The tests are installed in /usr/tests directory. The common tests after installation can be run by the following command:   kyua test -k /usr/tests/Kyuafile sys/netpfil/common