Packaging FreeBSD base
Contents
About PkgBase
PkgBase is the FreeBSD base operating system (OS), packaged for use with pkg(8).
pkg is already the norm for installing and updating ported software that is separate from base.
PkgBase aims to replace:
.txz distribution sets, which are used for installation of the OS with bsdinstall(8)
freebsd-update(8) for updates to the OS.
PkgBase complements building and installing from source. In particular:
simple installations of FreeBSD-CURRENT and FreeBSD-STABLE can be updated with pkg – there's no longer a requirement to build from source.
Status
Publication of official packages began in October 2023. Although packages are official:
use of pkgbase remains somewhat experimental – users should subscribe to freebsd-pkgbase.
Packages are published for architectures that are listed in make targets.
Branch |
Frequency |
URL |
main |
twice daily – 12:00 and 00:00 UTC |
https://pkg.freebsd.org/${ABI}/base_latest |
main |
weekly – Sunday at 12:00 UTC |
https://pkg.freebsd.org/${ABI}/base_weekly |
stable/14 |
twice daily – 12:00 and 00:00 UTC |
https://pkg.freebsd.org/${ABI}/base_latest |
stable/14 |
weekly – Sunday at 12:00 UTC |
https://pkg.freebsd.org/${ABI}/base_weekly |
releng/14.0 |
twice daily – 12:00 and 00:00 UTC |
https://pkg.freebsd.org/${ABI}/base_release_0 |
releng/14.1 |
twice daily – 12:00 and 00:00 UTC |
https://pkg.freebsd.org/${ABI}/base_release_1 |
Changes to releng (RELEASE) branches are infrequent.
Publication occurs only if a package changes.
In Bugzilla:
Related resources:
Evilham (Andrés) provides an unofficial repository at https://pkg.freebsd.evilham.com/images.
Overview
PkgBase comprises a few hundred packages with the default build (empty src.conf(5) and make.conf(5)) for amd64. The number of packages depends on several factors.
In most cases, a runtime binary is split into multiple components. Debugging symbols and most shared libraries are packaged individually. 32-bit is bit packaged separately.
Initial setup
Prerequisites:
pkg version 1.20.9 or greater
- FreeBSD 14.0 or greater – the steps below assume 14.0.
If your /usr/local/etc/pkg.conf does not include these two lines, add them:
BACKUP_LIBRARIES=true BACKUP_LIBRARY_PATH=/usr/local/lib/compat/pkg
Library backups, explained:
pkg.conf(5) options
libmd.so.6 and pkgbase (2024-10-07).
Create a directory for repository configuration files, then add a FreeBSD-base.conf file:
# mkdir -p /usr/local/etc/pkg/repos/ # cat <<'EOF' >/usr/local/etc/pkg/repos/FreeBSD-base.conf FreeBSD-base: { url: "pkg+https://pkg.FreeBSD.org/${ABI}/base_release_0", mirror_type: "srv", signature_type: "fingerprints", fingerprints: "/usr/share/keys/pkg", enabled: yes } EOF
alternatively, to have a .1 release from the outset, change base_release_0 to base_release_1.
Update repositories:
# pkg update Updating FreeBSD repository catalogue... FreeBSD repository is up to date. Updating FreeBSD-base repository catalogue... Fetching meta.conf: 100% 163 B 0.2kB/s 00:01 Fetching packagesite.pkg: 100% 33 KiB 33.9kB/s 00:01 Processing entries: 100% base repository update completed. 421 packages processed. All repositories are up to date.
Install base packages from the FreeBSD-base repo:
# pkg install -r FreeBSD-base -g 'FreeBSD-*' New packages to be INSTALLED: FreeBSD-acct: 14.rc4.20231103061237 [base] FreeBSD-acct-dbg: 14.rc4.20231103061237 [base] FreeBSD-acpi: 14.rc4.20231103061237 [base] FreeBSD-acpi-dbg: 14.rc4.20231103061237 [base] ... FreeBSD-zfs-dev-lib32: 14.0p2 [base] FreeBSD-zfs-lib32: 14.0p2 [base] FreeBSD-zoneinfo: 14.rc4.20231103061237 [base] Number of packages to be installed: 421 The process will require 4 GiB more space. 1 GiB to be downloaded. Proceed with this action? [y/N]:y
Finally – do not skip this – attend to:
.pkgsave files
- daemons.
.pkgsave files
If a file on disk differs from what is inside the package holding the file, then pkg(8) will rename the file – using a .pkgsave suffix – before installing what's packaged.
This is the same for config files, pkg(8) will not be able to merge them all if they were modified.
The six steps below are essential.
# cp /etc/ssh/sshd_config.pkgsave /etc/ssh/sshd_config # cp /etc/master.passwd.pkgsave /etc/master.passwd # cp /etc/group.pkgsave /etc/group # pwd_mkdb -p /etc/master.passwd # service sshd restart # cp /etc/sysctl.conf.pkgsave /etc/sysctl.conf
If a system was more broadly customised – beyond sysctl.conf(5) – before initialisation of pkgbase, then attention to other .pkgsave files may be required.
To interactively find and copy all such files, you can run this command (in sh(1)):
# find / -name "*.pkgsave" -type f -exec sh -c "f='{}'; echo '==== OLD ===='; ls -l \${f}; md5sum \${f}; echo '==== NEW ===='; ls -l \${f%.pkgsave}; md5sum \${f%.pkgsave}; cp -vi \${f} \${f%.pkgsave}" \;
A non-interactive approach:
# find / -name \*.pkgsave -delete # rm /boot/kernel/linker.hints
linker.hints was recreated at kernel install time, when we had .pkgsave files of previous modules
a new linker.hints file will be created during the next boot of the OS.
Upgrades
pkg upgrade alone is OK when there is no change to the base version level.
Minor version upgrades
From 14.0 to 14.1, for example.
pkg upgrade --yes --repository FreeBSD-base
- restart the OS
- upgrade ports (using available packages).
Major version upgrades
When attempting a major upgrade of FreeBSD (e.g. from 11 to 12), you may find the following error:
# pkg upgrade -r FreeBSD-base Updating FreeBSD-base repository catalogue... Fetching meta.txz: 100% 268 B 0.3kB/s 00:01 Fetching packagesite.txz: 100% 52 KiB 52.9kB/s 00:01 Processing entries: 0% pkg: wrong architecture: freebsd:12:x86:64 instead of FreeBSD:11:amd64 pkg: repository FreeBSD-base contains packages with wrong ABI: freebsd:12:x86:64 Processing entries: 100% Unable to update repository FreeBSD-base All repositories are up-to-date. pkg: Repository FreeBSD-base cannot be opened. 'pkg update' required Checking for upgrades (0 candidates): 100% Processing candidates (0 candidates): 100% Checking integrity... done (0 conflicting) Your packages are up to date.
For this case, the following command will allow a successful upgrade:
# env ABI=freebsd:12:x86:64 pkg-static upgrade -r FreeBSD-base
Building
To build base and kernel packages and install them to a jail/boot environment directory (14.1 host, 32 cores available):
root@freebsd:~ # cd /usr/src root@freebsd:~ # make -j32 buildworld && make -j32 buildkernel && make -j32 packages root@freebsd:~ # mkdir /root/myjail root@freebsd:~ # mkdir -vp /root/myjail/usr/share/keys/pkg/trusted /root/myjail/usr /root/myjail/usr/share /root/myjail/usr/share/keys /root/myjail/usr/share/keys/pkg /root/myjail/usr/share/keys/pkg/trusted root@freebsd:~ # cp -av /usr/share/keys/pkg/trusted/* /root/myjail/usr/share/keys/pkg/trusted/ /usr/share/keys/pkg/trusted/ -> /root/myjail/usr/share/keys/pkg/trusted /usr/share/keys/pkg/trusted/pkg.freebsd.org.2013102301 -> /root/myjail root@freebsd:~ # mkdir -vp /root/myjail/etc/pkg /root/myjail/etc /root/myjail/etc/pkg root@freebsd:~ # cat /root/myjail/etc/pkg/FreeBSD.conf FreeBSD-base-14.1 { url = file:///usr/obj/usr/src/repo/FreeBSD:14:amd64/14.1 mirror_type = none enabled = yes } Update the repository (full paths required): root@freebsd:~ # pkg --rootdir /root/myjail --repo-conf-dir /root/myjail/etc/pkg update --repository FreeBSD-base-14.1 Find all packages by name and display to standard out: root@freebsd:~ # pkg --rootdir /root/myjail --repo-conf-dir /root/myjail/etc/pkg rquery --repository FreeBSD-base-14.1 --evaluate '%n ~ FreeBSD-*' %n Install to the jail root: root@freebsd:~ # pkg --rootdir /root/myjail --repo-conf-dir /root/myjail/etc/pkg install --repository FreeBSD-base-14.1 --glob --yes '*' root@freebsd:~ # chroot /root/myjail root@freebsd:/ # ls .cshrc COPYRIGH boot etc libexec rescue sbin usr .profile bin dev lib root tmp var
Caveat: /usr/obj/usr/src/repo/FreeBSD:14:amd64/14.1 will not be valid from within the chroot/jail/boot environment and must be populated from the build host in some way.
Outstanding
No method to notify users when pkg upgrade does not include a new base package. If a file moves to a new package, then an upgrade (without the new package) might inappropriately remove the file. This problem was noticed after base 0648b47f0606 (2016), which moved /etc/rc.d/[zfs|zvol] from the FreeBSD-runtime package to the new FreeBSD-zfs package.
To name any base package that is orphaned, and to count the number of all orphaned pacakges:
pkg orphans | grep orphaned\:\ base ; pkg orphans | wc -l
To tell whether any base package is not installed:
pkg install --repository FreeBSD-base --no-repo-update --dry-run -g 'FreeBSD-*' | grep Number\ of\ packages
An example:
% pkg orphans | grep orphaned\:\ base ; pkg orphans | wc -l 54 % pkg install --repository FreeBSD-base --no-repo-update --dry-run -g 'FreeBSD-*' | grep Number\ of\ packages Number of packages to be installed: 16 %
Goals, and additional unresolved issues
The following items need to be addressed before PkgBase can be considered for production use:
bsdinstall(8) needs to be updated to use pkg(8) during installation (this needs to be optional, otherwise the installworld target will need to be pkg(8)-aware, and must be able to handle non-networked systems) (manu@ : working on it)
- pkg(8) needs to be able to be cross-built for non-amd64 architectures to be able to install on non-networked systems
- Infrastructural implementation for publishing packages, similar to pkg.FreeBSD.org
- In addition to infrastructural support, package signing and other related items needs to be implemented
A wrapper to support the functionality provided by freebsd-update(8) needs to be written, and needs to be 1:1 compatible (manu@ : working on it)
- Handbook updates for installation and upgrading
- Updates to the Release Engineering article
- A strategy must be well thought out and implemented on how to handle Security Advisories and Errata Notices, from when a change gets committed to a releng branch to how updated packages are made available on mirrors
- pkg(8) must continuously be backward-compatible against previous versions
- Generated files, such as mandoc.db, must be excluded from packages
- Conflicting files cannot exist, and known conflicting files must be addressed; for example, the FreeBSD-runtime and FreeBSD-runtime-manuals packages both try to install the tree(3) manual page (note: this particular issue is fixed, but still conceptually unresolved)
How to handle /boot/kernel and /boot/kernel.$KERNCONF for multi-kernel support needs to be addressed (Note: while both kernels can be installed simultaneously as packages, the one that gets installed to /boot/kernel is a dependency of the runtime package. Additionally, changing from make KERNCONF="GENERIC GENERIC-NODEBUG" buildkernel to make KERNCONF="GENERIC-NODEBUG GENERIC" does properly replace /boot/kernel and /boot/kernel.GENERIC-NODEBUG with /boot/kernel and /boot/kernel.GENERIC.) (manu@ : working on it)
How to handle ensuring the kernel package is always installed and the system is rebooted before the new userland is installed (Note: Committed fixes for this as r322327, with followup commits r322352 and r322358. brd pointed out that this dependency as-is is inherited by jail(8), so might need some tweaking.) (Note 2: this was reverted. See the commit message for r322545.)
- The kernel pkg will be marked as vital and thus can only be upgraded.
Reboot is not always required, but we will leave that up to the user as is handled currently. Post install script that will notify the user to reboot. Does require a new pkg feature to handle this: pkg#1699
- How to handle removed files/packages (i.e., rcmds removal in 12-CURRENT)
- manu@: pkg autoremove should be just what's needed here
Conversion script to pkgbase to handle registering pkgs without forcing pkg install -f or similar.
IDEA: script shipped with METALOG from a full build that walks the METALOG and does pkg register with the version from freebsd-version as it finds files that belong to pkgs.
- IDEA: Ship the plist and ucl files from the build, since that is what pkg register would need.
Metapkgs: The current is to implement a pkg feature called groups. See: pkg#1697
Retrospective
A call for testing was made in March 2016.
Major blockers and critical issues appeared to be resolved.
Glen Barber's notes from the packaging session at the 2017-08 BSDCam devsummit
Userland packages sets – can someone clarify what exactly this means?
TODO:
Package containing the source tree that a [base] package set was built from. Note: while I think this is a good idea, I have concerns about behavioral consistency between pkg upgrade and svn up ''/usr/src''. Some things that come to mind immediately are:
locally-patched source trees and losing patches on pkg upgrade;
svn up changing checksums recorded in the pkg(8) database;
ultimately "Which one wins?" in an update of /usr/src?
Add SVN rev of source tree to pkg metadata (or equivalent, plus indicate if the tree has been modified similarly to uname(1)) (Note: committed as r322412.)
RDBMS-backed web application to collect file checksums from every official package and allow users to verify files on-line.
Using pkgdb as an IDS? Support pkg audit?
Document FreeBSD 14.1 the source package.