FreeBSD Pre-Commit CI Makefile
This document describes how developers can run the same Continuous Integration (CI) tests locally as those executed by FreeBSD’s official pre-commit CI system.
The accompanying Makefile automates image builds, kernel/world compilation, smoke tests, and full Kyua test execution inside bhyve or QEMU virtual machines.
Overview
The Makefile provides several CI targets designed to let developers reproduce the project’s automated testing process before committing.
The workflow:
- Build world and kernel for the desired target.
- Create a bootable test image.
- Run smoke or full tests in a VM (bhyve or QEMU).
User-Driven Targets
Target |
Description |
make ci |
Runs the default CI sequence, determined by CITYPE (full or smoke). |
make ci-smoke |
Boots the test image and confirms that it reaches multi-user mode. |
make ci-full |
Runs the complete test suite, collecting Kyua results. |
make clean / make cleandir |
Cleans temporary build directories and images. |
Build Variables
Several variables can be defined to control the test environment:
Variable |
Description |
Default |
TARGET / TARGET_ARCH |
Target platform and architecture. |
Host machine. |
KERNCONF |
Kernel configuration file. |
GENERIC |
USE_QEMU |
Use QEMU instead of bhyve. |
Off if bhyve available. |
CIENV |
CI environment (e.g. local, cirrus, jenkins). |
local |
MAKECONF / SRCCONF |
Override system and source make configuration. |
/dev/null |
CICONF |
Optional CI configuration file. |
tools/ci.conf if present. |
VMSIZE |
Size of virtual disk image. |
6g |
SWAPSIZE |
Swap partition size. |
1g |
VMFS |
Filesystem used in VM image. |
ufs |
FORMAT |
Disk image format. |
raw |
VM_MEM_SIZE |
Virtual memory assigned to VM. |
Auto-detected. |
Toolchain and Prerequisites
To run tests locally, the following packages are required:
qemu-user-static and qemu-system-*
expect
pkg-static
You can bootstrap them automatically:
% make portinstall
This invokes the helper targets:
portinstall-pkg – Bootstraps pkg if missing.
portinstall-qemu – Installs QEMU user and system binaries.
portinstall-expect – Installs Expect for scripted VM interaction.
Typical Workflow
1. **Build the test image**
% make ci-buildimage
2. **Run smoke test (boot check only)**
% make ci-smoke
3. **Run full test suite**
% make ci-full
Note that make ci is also an alias to make ci-full.
All logs are stored under the current directory in files named:
_.<target_arch>.<target>
If a target fails, review the corresponding log for details.
Supported Architectures
The following architectures are supported for CI testing:
amd64
aarch64
armv7
powerpc64
powerpc64le
riscv64
Attempts to test unsupported targets will result in:
Error: <target_arch> is not supported on <type> <revision> <branch>
Supported Versions
At the moment only MAIN, stable/15 and releng/15.0 are supported. Until now it has not been decided whether these will be merged into stable/14 or releng/14.X. But for sure these will not land in 13.X series.
Running Under bhyve vs QEMU
By default, bhyve is used if supported by the host. If the host is itself virtualized (kern.vm_guest != none) or bhyve is unavailable, QEMU will be selected automatically.
- bhyve is preferred for amd64 host architectures.
- QEMU supports cross-architecture testing via user-static emulation.
To force QEMU usage:
% make USE_QEMU=1 ci-full
Output and Reports
The Makefile automatically creates and extracts metadata tarballs used for Kyua report storage. After a successful full run:
Extracted kyua reports to /tmp/meta.XXXXXX
These can be examined manually using kyua report-html.
Cleaning Up
Remove temporary images, disks, and metadata:
% make clean
This will also unset schg flags on created directories before removal.
Troubleshooting
- Ensure at least 4 GB of physical memory is available (8 GB recommended).
- When testing cross-architecture images, verify the correct QEMU binary
is installed (e.g. qemu-aarch64-static).
- For stuck bhyve runs, manually destroy VMs using:
# bhyvectl --vm=<vmname> --destroy
Notes
- Logs are redirected per-target for clarity.
The build honors WITH_META_MODE if filemon is loaded.
The Makefile integrates with the standard release/Makefile.inc1.