Packaging FreeBSD base

Status

Packaged base (PkgBase) became a beta feature in the FreeBSD 11 branch with r298107. The beta status will continue until 12.0-RELEASE.

A call for testing was made in March 2016. Major blockers and critical issues appear to be resolved. Please note, as with any development branch, this is not yet intended for production environments. Use of virtual machines or dedicated testing machines is strongly encouraged.

PkgBase comprises around 795 packages with the default build (empty src.conf(5) and make.conf(5)) for amd64. The number depends on several factors, but for most cases a runtime binary is split into several components. In particular, most shared libraries are individually packaged, in addition to debugging symbols, profiling libraries, and 32-bit packaged separately.

Outstanding

Please follow up on the freebsd-pkgbase@ mailing list with problems (and successes).

Initial setup

  1. Update your pkg(8) to a version after 1.7.0.

  2. Update your FreeBSD source tree to a revision after r298107 (FreeBSD 11.0-RELEASE or later; FreeBSD 10 and earlier branches do not support PkgBase).

  3. Build the userland and kernel as normal with the buildworld and buildkernel targets. Afterward, packages can be created with the packages Makefile target.

    # cd /usr/src
    # make [make flags] buildworld
    # make [make flags] buildkernel
    # make packages

    The package repository will be created within /usr/obj/usr/src/repo by default.

  4. To enable the repository for use by the local machine, create /usr/local/etc/pkg/repos/FreeBSD-base.conf with the following contents:

    # FreeBSD base system repository
    FreeBSD-base: {
      url: "file:///usr/obj/usr/src/repo/${ABI}/latest",
      mirror_type: "none",
      enabled: yes
    }
  5. To enable the repository for use on remote machines, you can setup an NGINX webserver on the machine that built the packages, adding the following lines to /usr/local/etc/nginx/nginx.conf:

    location /FreeBSD-base {
            alias           /usr/obj/usr/src/repo;
            autoindex       on;
    }

    On remote machines that will use the PkgBase repository, create /usr/local/etc/pkg/repos/FreeBSD-base.conf with the following contents:

    # FreeBSD base system repository
    FreeBSD-base: {
      url: "http://URL.TO.PKGBASE.MACHINE/FreeBSD-base/${ABI}/latest",
      enabled: yes
    }
  6. If installing inside a jail, do not forget to create that jail with allow.chflags set to 1. Without this option, pkg(8) will be unable to overwrite files with the schg flag set, such as /lib/libc.so.7 or /sbin/init

  7. To initially bootstrap the 'FreeBSD-*' packages, they must be forcibly installed. Package registration is not performed during installworld or installkernel, and there are no immediate plans to do this. This can be done by running:

    # pkg update -r FreeBSD-base
    # pkg install -g 'FreeBSD-*'
  8. Use mergemaster(8) or etcupdate(8) after upgrading the base system with pkg(8) to ensure configuration files are merged. This is still required since there is an outstanding issue with configuration file merging in PkgBase.

Kernel packages

It is possible to build multiple kernel packages by listing the kernel configuration names in the KERNCONF variable (separated by spaces) defined in make.conf(5), as long as NO_INSTALLEXTRAKERNELS is also set to no. The first listed kernel in KERNCONF is installed as /boot/kernel, and subsequent kernels in KERNCONF are installed as /boot/kernel.${KERNEL}.

Building GENERIC is not required, as each kernel package is named with the kernel name included.

For example, if make.conf(5) contains:

NO_INSTALLEXTRAKERNELS=no
KERNCONF=MYKERNEL1 MYKERNEL2

The following kernel packages will be created:

Upgrading

After the first time the source tree was built and installed with PkgBase, an upgrade involves the following steps:

  1. Update the FreeBSD source tree and build the userland and kernel as normal with the buildworld and buildkernel targets. Afterward, packages can be created with the packages Makefile target.

    # cd /usr/src
    # svn up
    # make [make flags] buildworld
    # make [make flags] buildkernel
    # make packages
  2. Upgrade the base packages:
    # pkg upgrade -r FreeBSD-base
  3. Check if any new base packages were created, during the build process, that would not be installed during an upgrade; and check if any base packages no longer exist that would not have been removed during an upgrade:
    $ diff <(pkg rquery -r FreeBSD-base %n) <(pkg query -e '%o = base' %n) | grep "^[<>]"

    Lines beginning with < are base packages that were built and exist in the repo but are not installed on your system. Lines beginning with > are base packages that are installed on your system but were not built and do not exist in the repo. Consider whether (a) you need to install or remove these packages; or (b) there is a deliberate difference between the built repo and your system.

  4. Use mergemaster(8) or etcupdate(8) after upgrading the base system with pkg(8) to ensure configuration files are merged. This is still required since there is an outstanding issue with configuration file merging in PkgBase.

Major version upgrades

When attempting a major upgrade of FreeBSD (e.g. from 11 to 12), you may find the following error:

For this case, the following command will allow a successful upgrade:

Project goals and additional unresolved issues

The following items need to be addressed before PkgBase can be considered for production use:

Glen Barber's notes from the packaging session at the 2017-08 BSDCam devsummit

Userland packages sets – can someone clarify what exactly this means?

Kernel package (easy upgrades, rollback).

Boot loader changes to support multiple kernels (allanjude@).

ETA?

TODO:

Package containing the source tree that a [base] package set was built from. Note: while I think this is a good idea, I have concerns about behavioral consistency between pkg upgrade and svn up ''/usr/src''. Some things that come to mind immediately are:

  1. locally-patched source trees and losing patches on pkg upgrade;

  2. svn up changing checksums recorded in the pkg(8) database;

  3. ultimately "Which one wins?" in an update of /usr/src?

Add SVN rev of source tree to pkg metadata (or equivalent, plus indicate if the tree has been modified similarly to uname(1)) (Note: committed as r322412.)

RDBMS-backed web application to collect file checksums from every official package and allow users to verify files on-line.

Using pkgdb as an IDS?

Base system: meta-packages.

PkgBase (last edited 2017-08-24 17:15:08 by GlenBarber)