• PkgBase

Packaging FreeBSD base

About PkgBase

PkgBase is the FreeBSD base operating system (OS), packaged for use with pkg(8).

pkg is already the norm for installing and updating ported software that is separate from base.

PkgBase aims to replace:

• .txz distribution sets, which are used for installation of the OS with bsdinstall(8)

• freebsd-update(8) for updates to the OS.

PkgBase complements building and installing from source. In particular:

• simple installations of FreeBSD-CURRENT and FreeBSD-STABLE can be updated with pkg – there's no longer a requirement to build from source.

Status

Publication of official packages began in October 2023. Although packages are official:

• use of pkgbase remains somewhat experimental – users should subscribe to freebsd-pkgbase.

Packages are published for architectures that are listed in make targets.

Changes to releng (RELEASE) branches are infrequent.

In Bugzilla:

• all open reports with the pkgbase keyword.

Related resources:

• Doug Rabson's FreeBSD container images

• Evilham (Andrés) provides an unofficial repository at https://pkg.freebsd.evilham.com/images.

Overview

PkgBase comprises a few hundred packages with the default build (empty src.conf(5) and make.conf(5)) for amd64. The number of packages depends on several factors.

In most cases, a runtime binary is split into multiple components. Debugging symbols and most shared libraries are packaged individually. 32-bit is bit packaged separately.

Initial setup

Prerequisites:

• pkg version 1.20.9 or greater

• FreeBSD 14.0 or greater – the steps below assume 14.0.

Create a directory for repository configuration files, then add a FreeBSD-base.conf file:

• # mkdir -p /usr/local/etc/pkg/repos/
  # cat <<'EOF' >/usr/local/etc/pkg/repos/FreeBSD-base.conf
  FreeBSD-base: {
    url: "pkg+https://pkg.FreeBSD.org/${ABI}/base_release_0",
    mirror_type: "srv",
    signature_type: "fingerprints",
    fingerprints: "/usr/share/keys/pkg",
    enabled: yes
  }
  EOF

• alternatively, to have a .1 release from the outset, change base_release_0 to base_release_1.

Update repositories:

• # pkg update
  Updating FreeBSD repository catalogue...
  FreeBSD repository is up to date.
  Updating FreeBSD-base repository catalogue...
  Fetching meta.conf: 100%    163 B   0.2kB/s    00:01    
  Fetching packagesite.pkg: 100%   33 KiB  33.9kB/s    00:01    
  Processing entries: 100%
  base repository update completed. 421 packages processed.
  All repositories are up to date.

Install base packages from the FreeBSD-base repo:

• # pkg install -r FreeBSD-base -g 'FreeBSD-*'
  New packages to be INSTALLED:                                  
          FreeBSD-acct: 14.rc4.20231103061237 [base]                                                                             
          FreeBSD-acct-dbg: 14.rc4.20231103061237 [base]                                                                         
          FreeBSD-acpi: 14.rc4.20231103061237 [base]                                                                             
          FreeBSD-acpi-dbg: 14.rc4.20231103061237 [base]
  ...
          FreeBSD-zfs-dev-lib32: 14.0p2 [base]
          FreeBSD-zfs-lib32: 14.0p2 [base]
          FreeBSD-zoneinfo: 14.rc4.20231103061237 [base]
  
  Number of packages to be installed: 421
  
  The process will require 4 GiB more space.
  1 GiB to be downloaded.
  
  Proceed with this action? [y/N]:y

Finally – do not skip this – attend to:

• .pkgsave files

• daemons.

.pkgsave files

If a file on disk differs from what is inside the package holding the file, then pkg(8) will rename the file – using a .pkgsave suffix – before installing what's packaged.

This is the same for config files, pkg(8) will not be able to merge them all if they were modified.

• # cp /etc/ssh/sshd_config.pkgsave /etc/ssh/sshd_config
  # cp /etc/master.passwd.pkgsave /etc/master.passwd
  # cp /etc/group.pkgsave /etc/group
  # pwd_mkdb -p /etc/master.passwd
  # service sshd restart
  # cp /etc/sysctl.conf.pkgsave /etc/sysctl.conf

If a system was more broadly customised – beyond sysctl.conf(5) – before initialisation of pkgbase, then attention to other .pkgsave files may be required.

To interactively find and copy all such files, you can run this command (in sh(1)):

• # find / -name "*.pkgsave" -type f -exec sh -c "f='{}'; echo '==== OLD ===='; ls -l \${f}; md5sum \${f}; echo '==== NEW ===='; ls -l \${f%.pkgsave}; md5sum \${f%.pkgsave}; cp -vi \${f} \${f%.pkgsave}" \;

A non-interactive approach:

• # find / -name \*.pkgsave -delete
  # rm /boot/kernel/linker.hints

• linker.hints was recreated at kernel install time, when we had .pkgsave files of previous modules

• a new linker.hints file will be created during the next boot of the OS.

Upgrades

pkg upgrade alone is OK when there is no change to the base version level.

Minor version upgrades

From 14.0 to 14.1, for example.

1. pkg upgrade --yes --repository FreeBSD-base

2. restart the OS
3. upgrade ports (using available packages).

Major version upgrades

When attempting a major upgrade of FreeBSD (e.g. from 11 to 12), you may find the following error:

• # pkg upgrade -r FreeBSD-base
  Updating FreeBSD-base repository catalogue...
  Fetching meta.txz: 100%    268 B   0.3kB/s    00:01   
  Fetching packagesite.txz: 100%   52 KiB  52.9kB/s    00:01   
  Processing entries:   0%
  pkg: wrong architecture: freebsd:12:x86:64 instead of FreeBSD:11:amd64
  pkg: repository FreeBSD-base contains packages with wrong ABI: freebsd:12:x86:64
  Processing entries: 100%
  Unable to update repository FreeBSD-base
  All repositories are up-to-date.
  pkg: Repository FreeBSD-base cannot be opened. 'pkg update' required
  Checking for upgrades (0 candidates): 100%
  Processing candidates (0 candidates): 100%
  Checking integrity... done (0 conflicting)
  Your packages are up to date.

For this case, the following command will allow a successful upgrade:

• # env ABI=freebsd:12:x86:64 pkg-static upgrade -r FreeBSD-base

Building

To build base and kernel packages and install them to a jail/boot environment directory (14.1 host, 32 cores available):

• root@freebsd:~ # cd /usr/src
  root@freebsd:~ # make -j32 buildworld && make -j32 buildkernel && make -j32 packages
  
  root@freebsd:~ # mkdir /root/myjail
  root@freebsd:~ # mkdir -vp /root/myjail/usr/share/keys/pkg/trusted
  /root/myjail/usr
  /root/myjail/usr/share
  /root/myjail/usr/share/keys
  /root/myjail/usr/share/keys/pkg
  /root/myjail/usr/share/keys/pkg/trusted
  root@freebsd:~ # cp -av /usr/share/keys/pkg/trusted/* /root/myjail/usr/share/keys/pkg/trusted/
  /usr/share/keys/pkg/trusted/ -> /root/myjail/usr/share/keys/pkg/trusted
  /usr/share/keys/pkg/trusted/pkg.freebsd.org.2013102301 -> /root/myjail
  
  root@freebsd:~ # mkdir -vp /root/myjail/etc/pkg
  /root/myjail/etc
  /root/myjail/etc/pkg
  
  root@freebsd:~ # cat /root/myjail/etc/pkg/FreeBSD.conf
  FreeBSD-base-14.1 {
          url         = file:////usr/obj/usr/src/repo/FreeBSD:14:amd64/14.1
          mirror_type = none
          enabled     = yes
  }
  
  Update the repository (full paths required):
  root@freebsd:~ # pkg --rootdir /root/myjail --repo-conf-dir /root/myjail/etc/pkg update --repository FreeBSD-base-14.1
  
  Find all packages by name and display to standard out:
  root@freebsd:~ # pkg --rootdir /root/myjail --repo-conf-dir /root/myjail/etc/pkg rquery --repository FreeBSD-base-14.1 --evaluate '%n ~ FreeBSD-*' %n
  
  Install to the jail root:
  root@freebsd:~ # pkg --rootdir /root/myjail --repo-conf-dir /root/myjail/etc/pkg install --repository FreeBSD-base-14.1 --glob --yes '*'
  
  root@freebsd:~ # chroot /root/myjail
  
  root@freebsd:/ # ls
  .cshrc    COPYRIGH   boot     etc       libexec      rescue     sbin       usr
  .profile  bin        dev      lib       root         tmp        var

Caveat: /usr/obj/usr/src/repo/FreeBSD:14:amd64/14.1 will not be valid from within the chroot/jail/boot environment and must be populated from the build host in some way.

Outstanding

No method to notify users when pkg upgrade does not include a new base package. If a file moves to a new package, then an upgrade (without the new package) might inappropriately remove the file. This problem was noticed after base 0648b47f0606 (2016), which moved /etc/rc.d/[zfs|zvol] from the FreeBSD-runtime package to the new FreeBSD-zfs package.

Goals, and additional unresolved issues

The following items need to be addressed before PkgBase can be considered for production use:

• bsdinstall(8) needs to be updated to use pkg(8) during installation (this needs to be optional, otherwise the installworld target will need to be pkg(8)-aware, and must be able to handle non-networked systems) (manu@ : working on it)

• pkg(8) needs to be able to be cross-built for non-amd64 architectures to be able to install on non-networked systems
• Infrastructural implementation for publishing packages, similar to pkg.FreeBSD.org
• In addition to infrastructural support, package signing and other related items needs to be implemented

• A wrapper to support the functionality provided by freebsd-update(8) needs to be written, and needs to be 1:1 compatible (manu@ : working on it)

• Handbook updates for installation and upgrading
• Updates to the Release Engineering article
• A strategy must be well thought out and implemented on how to handle Security Advisories and Errata Notices, from when a change gets committed to a releng branch to how updated packages are made available on mirrors
• pkg(8) must continuously be backward-compatible against previous versions
• Generated files, such as mandoc.db, must be excluded from packages
• Conflicting files cannot exist, and known conflicting files must be addressed; for example, the FreeBSD-runtime and FreeBSD-runtime-manuals packages both try to install the tree(3) manual page (note: this particular issue is fixed, but still conceptually unresolved)

• How to handle /boot/kernel and /boot/kernel.$KERNCONF for multi-kernel support needs to be addressed (Note: while both kernels can be installed simultaneously as packages, the one that gets installed to /boot/kernel is a dependency of the runtime package. Additionally, changing from make KERNCONF="GENERIC GENERIC-NODEBUG" buildkernel to make KERNCONF="GENERIC-NODEBUG GENERIC" does properly replace /boot/kernel and /boot/kernel.GENERIC-NODEBUG with /boot/kernel and /boot/kernel.GENERIC.) (manu@ : working on it)

  • Multiple versions of the same pkg will be allowed, see: pkg#1698

  • Always install as /boot/kernel.$KERNCONF and symlink /boot/kernel to the correct one via post-install script.

  • Possible bootloader changes required, AllanJude volunteered to help with this.

• How to handle ensuring the kernel package is always installed and the system is rebooted before the new userland is installed (Note: Committed fixes for this as r322327, with followup commits r322352 and r322358. brd pointed out that this dependency as-is is inherited by jail(8), so might need some tweaking.) (Note 2: this was reverted. See the commit message for r322545.)

  • The kernel pkg will be marked as vital and thus can only be upgraded.

  • Reboot is not always required, but we will leave that up to the user as is handled currently. Post install script that will notify the user to reboot. Does require a new pkg feature to handle this: pkg#1699

• How to handle removed files/packages (i.e., rcmds removal in 12-CURRENT)
  • manu@: pkg autoremove should be just what's needed here

• Conversion script to pkgbase to handle registering pkgs without forcing pkg install -f or similar.

  • IDEA: script shipped with METALOG from a full build that walks the METALOG and does pkg register with the version from freebsd-version as it finds files that belong to pkgs.

  • IDEA: Ship the plist and ucl files from the build, since that is what pkg register would need.

• Metapkgs: The current is to implement a pkg feature called groups. See: pkg#1697

Retrospective

A call for testing was made in March 2016.

Major blockers and critical issues appeared to be resolved.

Glen Barber's notes from the packaging session at the 2017-08 BSDCam devsummit

Userland packages sets – can someone clarify what exactly this means?

TODO:

Package containing the source tree that a [base] package set was built from. Note: while I think this is a good idea, I have concerns about behavioral consistency between pkg upgrade and svn up ''/usr/src''. Some things that come to mind immediately are:

1. locally-patched source trees and losing patches on pkg upgrade;

2. svn up changing checksums recorded in the pkg(8) database;

3. ultimately "Which one wins?" in an update of /usr/src?

Add SVN rev of source tree to pkg metadata (or equivalent, plus indicate if the tree has been modified similarly to uname(1)) (Note: committed as r322412.)

RDBMS-backed web application to collect file checksums from every official package and allow users to verify files on-line.

Using pkgdb as an IDS? Support pkg audit?

Document FreeBSD 14.1 the source package.

CategoryProject CategoryTodo