Packaging FreeBSD base
Contents
About PkgBase
PkgBase is the FreeBSD base operating system (OS), packaged for use with pkg(8).
pkg is already the norm for installing and updating ported software that is separate from base.
PkgBase aims to replace:
.txz distribution sets, which are used for installation of the OS with bsdinstall(8)
freebsd-update(8) for updates to the OS.
PkgBase complements building and installing from source. In particular:
simple installations of FreeBSD-CURRENT and FreeBSD-STABLE can be updated with pkg – there's no longer a requirement to build from source.
Status
Publication of official packages began in October 2023. Although packages are official:
use of pkgbase remains somewhat experimental – users should subscribe to freebsd-pkgbase.
Packages are published for architectures that are listed in make targets.
Branch |
Frequency |
URL |
main |
twice daily – 12:00 and 00:00 UTC |
https://pkg.freebsd.org/${ABI}/base_latest |
main |
weekly – Sunday at 12:00 UTC |
https://pkg.freebsd.org/${ABI}/base_weekly |
stable/14 |
twice daily – 12:00 and 00:00 UTC |
https://pkg.freebsd.org/${ABI}/base_latest |
stable/14 |
weekly – Sunday at 12:00 UTC |
https://pkg.freebsd.org/${ABI}/base_weekly |
releng/14.1 |
twice daily – 12:00 and 00:00 UTC |
https://pkg.freebsd.org/${ABI}/base_release_1 |
releng/14.2 |
twice daily – 12:00 and 00:00 UTC |
https://pkg.freebsd.org/${ABI}/base_release_2 |
Changes to releng (RELEASE) branches are infrequent.
Publication occurs only if a package changes.
In Bugzilla:
Related resources:
Evilham (Andrés) provides an unofficial repository at https://pkg.freebsd.evilham.com/images.
Overview
PkgBase comprises a few hundred packages with the default build (empty src.conf(5) and make.conf(5)) for amd64. The number of packages depends on several factors.
In most cases, a runtime binary is split into multiple components. Debugging symbols and most shared libraries are packaged individually. 32-bit is bit packaged separately.
Automatic initial setup
The pkgbasify tool may be used to automatically convert a FreeBSD 14.0 or greater system to use pkgbase.
Follow the simple instructions in the pkgbasify README to download and run the script.
Manual initial setup
Prerequisites:
pkg version 1.20.9 or greater
- FreeBSD 14.0, or greater, for initialisation (or conversion) without a major upgrade
- FreeBSD 13.5-RELEASE, or a more recent 13-STABLE, for a more complex installation of 14 on 13.
The steps below assume 14.0 as a starting point.
If your /usr/local/etc/pkg.conf does not include these two lines, add them:
BACKUP_LIBRARIES=true BACKUP_LIBRARY_PATH=/usr/local/lib/compat/pkg
Library backups, explained:
pkg.conf(5) options
libmd.so.6 and pkgbase (2024-10-07).
Create a directory for repository configuration files, then add a FreeBSD-base.conf file:
# mkdir -p /usr/local/etc/pkg/repos/ # cat <<'EOF' >/usr/local/etc/pkg/repos/FreeBSD-base.conf FreeBSD-base { url = "pkg+https://pkg.freebsd.org/${ABI}/base_release_0"; mirror_type = "srv"; signature_type = "fingerprints"; fingerprints = "/usr/share/keys/pkg"; enabled = yes; } EOFalternatively, to have a .1 release from the outset, change base_release_0 to base_release_1.
Update repositories:
# pkg update Updating FreeBSD repository catalogue... FreeBSD repository is up to date. Updating FreeBSD-base repository catalogue... Fetching meta.conf: 100% 163 B 0.2kB/s 00:01 Fetching packagesite.pkg: 100% 33 KiB 33.9kB/s 00:01 Processing entries: 100% base repository update completed. 421 packages processed. All repositories are up to date.
When installing the FreeBSD-* packages in the next step, run the command as a super user. The password and group files will be temporarily reset to defaults, so if you install the packages using a one-off sudo command, you will lose access to the system.
Install base packages from the FreeBSD-base repo:
# pkg install -r FreeBSD-base -g 'FreeBSD-*' New packages to be INSTALLED: FreeBSD-acct: 14.rc4.20231103061237 [base] FreeBSD-acct-dbg: 14.rc4.20231103061237 [base] FreeBSD-acpi: 14.rc4.20231103061237 [base] FreeBSD-acpi-dbg: 14.rc4.20231103061237 [base] ... FreeBSD-zfs-dev-lib32: 14.0p2 [base] FreeBSD-zfs-lib32: 14.0p2 [base] FreeBSD-zoneinfo: 14.rc4.20231103061237 [base] Number of packages to be installed: 421 The process will require 4 GiB more space. 1 GiB to be downloaded. Proceed with this action? [y/N]:y
Finally – do not skip this – attend to:
.pkgsave files
- daemons.
.pkgsave files
If a file on disk differs from what is inside the package holding the file, then pkg(8) will rename the file – using a .pkgsave suffix – before installing what's packaged.
More specifically: if there are multiple editions of a config file, pkg(8) can not merge the differences.
The six steps below are essential.
# cp /etc/ssh/sshd_config.pkgsave /etc/ssh/sshd_config # cp /etc/master.passwd.pkgsave /etc/master.passwd # cp /etc/group.pkgsave /etc/group # pwd_mkdb -p /etc/master.passwd # service sshd restart # cp /etc/sysctl.conf.pkgsave /etc/sysctl.conf
If a system was more broadly customised – beyond sysctl.conf(5) – before initialisation of pkgbase, then attention to other .pkgsave files may be required.
To interactively find and copy all such files, you can run this command (in sh(1)):
# find / -name "*.pkgsave" -type f -exec sh -c "f='{}'; echo '==== OLD ===='; ls -l \${f}; md5sum \${f}; echo '==== NEW ===='; ls -l \${f%.pkgsave}; md5sum \${f%.pkgsave}; cp -vi \${f} \${f%.pkgsave}" \;
Additionally:
# find / -name \*.pkgsave -delete # rm /boot/kernel/linker.hints
linker.hints was recreated at kernel install time, when we had .pkgsave files of previous modules
a new linker.hints file will be created during the next boot of the OS.
Upgrades
pkg upgrade alone is OK when there is no change to the base version level.
Minor version upgrades
From 14.1 to 14.2, for example.
ee /usr/local/etc/pkg/repos/FreeBSD-base.conf
change base_release_1 to base_release_2
- escape, save
pkg upgrade --yes --repository FreeBSD-base
pkg install --repository FreeBSD-base --no-repo-update --dry-run --glob 'FreeBSD-*'
pkg install --quiet FreeBSD-syscons-data
- restart the OS
- upgrade ports (using available packages).
Major version upgrades
For 14 (versions of which include RELEASE) to 15 (not yet RELEASE), the url line of FreeBSD-base.conf must include one of the following:
base_weekly
base_latest – as shown below.
url = "pkg+https://pkg.freebsd.org/${ABI}/base_latest";
An ABI environment variable is required.
Example commands, for AMD64:
# env ABI=FreeBSD:15:amd64 pkg-static upgrade # env ABI=FreeBSD:15:amd64 pkg-static install -nU -r FreeBSD-base -g 'FreeBSD-*'
When prompted to run pkg bootstrap -f: do not.
The upgrade will be broad – not limited to base. Either:
- avoid using non-base software, such as a desktop environment, during the upgrade period; or
create and mount a new ZFS boot environment, then use the --rootdir option of pkg-static(8) to target the mount point of the environment.
The dry run installation, after the upgrade, lists non-installed base packages. Packages that exist for 15 but not 14 include FreeBSD-ntp.
After the upgrade and any other required installation:
- if you upgraded a new boot environment, make it active or temporarily active
- restart the OS.
13 to 14
There is no pkgbase major upgrade from 13 to 14, because there is no no FreeBSD Project-provided set of base packages for 13.
Instead:
- base packages for 14.2-RELEASE-p2 may be installed on 13.5-RELEASE.
Proceed with caution. An incomplete installation to an active boot environment may result in:
- a running system that can no longer use pkg (to complete the installation); and/or
- a restarted system that is unusable.
With ZFS boot environments, the risk is avoidable. An example, with option -q for quietness during fetch and installation of packages:
create or edit FreeBSD-base.conf, its url line should include base_release_2
prepare /usr/local/etc/pkg.conf for library backups
use bectl(8) to create a new environment
use bectl to mount the new environment at /tmp/up
env ABI=FreeBSD:14:amd64 pkg -r /tmp/up install -Fqy -r FreeBSD-base -g 'FreeBSD-*'
when prompted to run pkg bootstrap -f, do not
env BACKUP_LIBRARIES=true BACKUP_LIBRARY_PATH=/tmp/up/usr/local/lib/compat/pkg ABI=FreeBSD:14:amd64 pkg -r /tmp/up install -qUy -r FreeBSD-base -g 'FreeBSD-*' ; tail -n 2 /var/log/messages
when prompted to run pkg bootstrap -f, do not
rm /tmp/up/boot/kernel/linker.hints
in the new environment, take other steps that are essential – including attention to .pkgsave files and use of pwd_mkdb(8)
use the activate subcommand of bectl with option -t to make the new environment temporarily active (for the next start of the system, not subsequent starts)
- restart the system.
If the installation is good:
use the activate subcommand of bectl without option -t.
Building
To build base and kernel packages and install them to a jail/boot environment directory (14.1 host, 32 cores available):
root@freebsd:~ # cd /usr/src root@freebsd:~ # make -j32 buildworld && make -j32 buildkernel && make -j32 packages root@freebsd:~ # mkdir /root/myjail root@freebsd:~ # mkdir -vp /root/myjail/usr/share/keys/pkg/trusted /root/myjail/usr /root/myjail/usr/share /root/myjail/usr/share/keys /root/myjail/usr/share/keys/pkg /root/myjail/usr/share/keys/pkg/trusted root@freebsd:~ # cp -av /usr/share/keys/pkg/trusted/* /root/myjail/usr/share/keys/pkg/trusted/ /usr/share/keys/pkg/trusted/ -> /root/myjail/usr/share/keys/pkg/trusted /usr/share/keys/pkg/trusted/pkg.freebsd.org.2013102301 -> /root/myjail root@freebsd:~ # mkdir -vp /root/myjail/etc/pkg /root/myjail/etc /root/myjail/etc/pkg root@freebsd:~ # cat /root/myjail/etc/pkg/FreeBSD.conf FreeBSD-base-14.1 { url = file:///usr/obj/usr/src/repo/FreeBSD:14:amd64/14.1 mirror_type = none enabled = yes } Update the repository (full paths required): root@freebsd:~ # pkg --rootdir /root/myjail --repo-conf-dir /root/myjail/etc/pkg update --repository FreeBSD-base-14.1 Find all packages by name and display to standard out: root@freebsd:~ # pkg --rootdir /root/myjail --repo-conf-dir /root/myjail/etc/pkg rquery --repository FreeBSD-base-14.1 --evaluate '%n ~ FreeBSD-*' %n Install to the jail root: root@freebsd:~ # pkg --rootdir /root/myjail --repo-conf-dir /root/myjail/etc/pkg install --repository FreeBSD-base-14.1 --glob --yes '*' root@freebsd:~ # chroot /root/myjail root@freebsd:/ # ls .cshrc COPYRIGH boot etc libexec rescue sbin usr .profile bin dev lib root tmp var
Caveat: /usr/obj/usr/src/repo/FreeBSD:14:amd64/14.1 will not be valid from within the chroot/jail/boot environment and must be populated from the build host in some way.
Outstanding
December 2024: it is not known whether removal of all .pkgsave files is optional (e.g. simple tidiness), or somehow essential before a major upgrade. Following one test upgrade from 14.2-RELEASE to 15.0-CURRENT, it was discovered that the most important .pkgsave files in the ''14.2r'' environment lacked essential data such as user passwords and group memberships.
No method to notify users when pkg upgrade does not include a new base package. If a file moves to a new package, then an upgrade (without the new package) might inappropriately remove the file. This problem was noticed after base 0648b47f0606 (2016), which moved /etc/rc.d/[zfs|zvol] from the FreeBSD-runtime package to the new FreeBSD-zfs package.
To name any base package that is orphaned:
pkg orphans | grep orphaned\:\ base
To tell whether any base package is not installed:
pkg install --repository FreeBSD-base --no-repo-update --dry-run --glob 'FreeBSD-*' | grep Number\ of\ packages
An example:
% pkg orphans | grep orphaned\:\ base ; pkg orphans | wc -l
54
% pkg install --repository FreeBSD-base --no-repo-update --dry-run -g 'FreeBSD-*' | grep Number\ of\ packages
Number of packages to be installed: 16
%
Goals, and additional unresolved issues
The following items need to be addressed before PkgBase can be considered for production use:
Create a tool to let users transition to using pkgbase – milestone 2025, first quarter
Support in the installer to install pkgbase (this needs to be optional, otherwise the installworld target will need to be pkg(8)-aware, and must be able to handle non-networked systems) (manu@ : working on it)
- pkg(8) needs to be able to be cross-built for non-amd64 architectures to be able to install on non-networked systems
- Infrastructural implementation for publishing packages, similar to pkg.FreeBSD.org
- In addition to infrastructural support, package signing and other related items needs to be implemented
A wrapper to support the functionality provided by freebsd-update(8) needs to be written, and needs to be 1:1 compatible (manu@ : working on it)
- Handbook updates for installation and upgrading
- Updates to the Release Engineering article
- A strategy must be well thought out and implemented on how to handle Security Advisories and Errata Notices, from when a change gets committed to a releng branch to how updated packages are made available on mirrors
- pkg(8) must continuously be backward-compatible against previous versions
- Generated files, such as mandoc.db, must be excluded from packages
- Conflicting files cannot exist, and known conflicting files must be addressed; for example, the FreeBSD-runtime and FreeBSD-runtime-manuals packages both try to install the tree(3) manual page (note: this particular issue is fixed, but still conceptually unresolved)
How to handle /boot/kernel and /boot/kernel.$KERNCONF for multi-kernel support needs to be addressed (Note: while both kernels can be installed simultaneously as packages, the one that gets installed to /boot/kernel is a dependency of the runtime package. Additionally, changing from make KERNCONF="GENERIC GENERIC-NODEBUG" buildkernel to make KERNCONF="GENERIC-NODEBUG GENERIC" does properly replace /boot/kernel and /boot/kernel.GENERIC-NODEBUG with /boot/kernel and /boot/kernel.GENERIC.) (manu@ : working on it)
How to handle ensuring the kernel package is always installed and the system is rebooted before the new userland is installed (Note: Committed fixes for this as r322327, with followup commits r322352 and r322358. brd pointed out that this dependency as-is is inherited by jail(8), so might need some tweaking.) (Note 2: this was reverted. See the commit message for r322545.)
- The kernel pkg will be marked as vital and thus can only be upgraded.
Reboot is not always required, but we will leave that up to the user as is handled currently. Post install script that will notify the user to reboot. Does require a new pkg feature to handle this: pkg#1699
- Ensuring updating the kernel leaves a copy of the currently-running kernel (i.e. replacement for installkernel/freebsd-update kernel.old) for recovery, at least between releases, but ideally including patches
- Easy, automatic integration with boot environments like freebsd-update
- How to handle removed files/packages (i.e., rcmds removal in 12-CURRENT)
- manu@: pkg autoremove should be just what's needed here
Conversion script to pkgbase to handle registering pkgs without forcing pkg install -f or similar.
IDEA: script shipped with METALOG from a full build that walks the METALOG and does pkg register with the version from freebsd-version as it finds files that belong to pkgs.
- IDEA: Ship the plist and ucl files from the build, since that is what pkg register would need.
Metapkgs: The current is to implement a pkg feature called groups. See: pkg#1697
Retrospective
A call for testing was made in March 2016.
Major blockers and critical issues appeared to be resolved.
Glen Barber's notes from the packaging session at the 2017-08 BSDCam devsummit
Userland packages sets – can someone clarify what exactly this means?
TODO:
Package containing the source tree that a [base] package set was built from. Note: while I think this is a good idea, I have concerns about behavioral consistency between pkg upgrade and svn up ''/usr/src''. Some things that come to mind immediately are:
locally-patched source trees and losing patches on pkg upgrade;
svn up changing checksums recorded in the pkg(8) database;
ultimately "Which one wins?" in an update of /usr/src?
Add SVN rev of source tree to pkg metadata (or equivalent, plus indicate if the tree has been modified similarly to uname(1)) (Note: committed as r322412.)
RDBMS-backed web application to collect file checksums from every official package and allow users to verify files on-line.
Using pkgdb as an IDS? Support pkg audit?
Document FreeBSD 14.1 the source package.