Packaging FreeBSD base

Contents

  1. Overview
  2. Status
  3. Outstanding
  4. Initial setup
      1. Fixing missing user
    2. Kernel packages
    3. Upgrading
    4. Major version upgrades
  5. Project goals and additional unresolved issues
    1. Glen Barber's notes from the packaging session at the 2017-08 BSDCam devsummit

Overview

PkgBase, as the name implies, is the packaging of the FreeBSD base system. With it, users can upgrade their FreeBSD system using pkg(8) rather than using freebsd-update(8) or building from source.

Status

Packaged base (PkgBase) work is happening in HEAD.

A call for testing was made in March 2016. Major blockers and critical issues appear to be resolved. Please note, as with any development branch, this is not yet intended for production environments. Use of virtual machines or dedicated testing machines is strongly encouraged.

PkgBase comprises around 300 packages with the default build (empty src.conf(5) and make.conf(5)) for amd64. The number depends on several factors, but for most cases a runtime binary is split into several components. In particular, most shared libraries are individually packaged, in addition to debugging symbols, and 32-bit packaged separately.

To aid exploration and adoption, Evilham provides an unofficial pacakge repository.

Outstanding

Please follow up on the freebsd-pkgbase@ mailing list with problems (and successes).

Initial setup

  1. Ensure your pkg(8) version is at least 1.10.5_3.

  2. Update your FreeBSD source tree to HEAD.

  3. Build the userland and kernel as normal with the buildworld and buildkernel targets. Afterward, packages can be created with the packages Makefile target. 

    # cd /usr/src
# make [make flags] buildworld
# make [make flags] buildkernel
# make packages

    The package repository will be created within /usr/obj/usr/src/repo by default.

  4. To enable the repository for use by the local machine, create /usr/local/etc/pkg/repos/FreeBSD-base.conf with the following contents: 

    # FreeBSD base system repository
FreeBSD-base: {
  url: "file:///usr/obj/usr/src/repo/${ABI}/latest",
  mirror_type: "none",
  enabled: yes
}

  5. To enable the repository for use on remote machines, you can setup an NGINX webserver on the machine that built the packages, adding the following lines to /usr/local/etc/nginx/nginx.conf: 

    location /FreeBSD-base {
        alias           /usr/obj/usr/src/repo;
        autoindex       on;
}

    On remote machines that will use the PkgBase repository, create /usr/local/etc/pkg/repos/FreeBSD-base.conf with the following contents: 

    # FreeBSD base system repository
FreeBSD-base: {
  url: "http://URL.TO.PKGBASE.MACHINE/FreeBSD-base/${ABI}/latest",
  enabled: yes
}

  6. If installing inside a jail, do not forget to create that jail with allow.chflags set to 1. Without this option, pkg(8) will be unable to overwrite files with the schg flag set, such as /lib/libc.so.7 or /sbin/init

  7. To initially bootstrap the 'FreeBSD-*' packages, they must be forcibly installed. Package registration is not performed during installworld or installkernel, and there are no immediate plans to do this. This can be done by running: 

    # pkg update -r FreeBSD-base
# pkg install -g 'FreeBSD-*'

    Caution: There is currently no built-in mechanism for bootstrapping pkg info from the existing system, and files (such as /etc/master.passwd) will be replaced with default copies from the newly installed packages. Existing files will be renamed with a .pkgsave extension. Files that may be of particular interest include:

    • /etc/aliases
    • /etc/fstab
    • /etc/group
    • /etc/hosts
    • /etc/mail.rc
    • /etc/master.passwd
    • /etc/ssh/ssh_config
    • /etc/ssh/sshd_config
    • /etc/sysctl.conf
    • /etc/syslog.conf
    • /etc/ttys
    • /root/.cshrc
    • /root/.k5login
    • /root/.login
    • /root/.profile
    • /root/.shrc

WARNINGS:

Fixing missing user

After upgrading, if you have the following message from su 

su: who are you?

It means that you have replaced your master.passwd file during the upgrade, in order to restore your user and rebuild the passwd database, you can do the commands below, then login again: 

# cp /etc/master.passwd.pkgsave /etc/master.passwd
# /usr/sbin/pwd_mkdb -p /etc/master.passwd

Kernel packages

It is possible to build multiple kernel packages by listing the kernel configuration names in the KERNCONF variable (separated by spaces) defined in make.conf(5), as long as NO_INSTALLEXTRAKERNELS is also set to no. The first listed kernel in KERNCONF is installed as /boot/kernel, and subsequent kernels in KERNCONF are installed as /boot/kernel.${KERNEL}.

Building GENERIC is not required, as each kernel package is named with the kernel name included.

For example, if make.conf(5) contains: 

NO_INSTALLEXTRAKERNELS=no
KERNCONF=MYKERNEL1 MYKERNEL2

The following kernel packages will be created:

Upgrading

After the first time the source tree was built and installed with PkgBase, an upgrade involves the following steps:

  1. Update the FreeBSD source tree and build the userland and kernel as normal with the buildworld and buildkernel targets. Afterward, packages can be created with the packages Makefile target. 

    # cd /usr/src
# svn up
# make [make flags] buildworld
# make [make flags] buildkernel
# make update-packages

Note that for make update-packages working correctly you need WITH_REPRODUCIBLE_BUILD=yes in /etc/src.conf as it compares the internal package data.

  1. Upgrade the base packages: 
    # pkg upgrade -r FreeBSD-base
  2. Check if any new base packages were created, during the build process, that would not be installed during an upgrade; and check if any base packages no longer exist that would not have been removed during an upgrade: 
    $ diff <(pkg rquery -r FreeBSD-base %n) <(pkg query -e '%o = base' %n) | grep "^[<>]"

    Lines beginning with < are base packages that were built and exist in the repo but are not installed on your system. Lines beginning with > are base packages that are installed on your system but were not built and do not exist in the repo. Consider whether (a) you need to install or remove these packages; or (b) there is a deliberate difference between the built repo and your system.

Major version upgrades

When attempting a major upgrade of FreeBSD (e.g. from 11 to 12), you may find the following error:

For this case, the following command will allow a successful upgrade:

Project goals and additional unresolved issues

The following items need to be addressed before PkgBase can be considered for production use:

Glen Barber's notes from the packaging session at the 2017-08 BSDCam devsummit

Userland packages sets – can someone clarify what exactly this means?

TODO:

Package containing the source tree that a [base] package set was built from. Note: while I think this is a good idea, I have concerns about behavioral consistency between pkg upgrade and svn up ''/usr/src''. Some things that come to mind immediately are:

  1. locally-patched source trees and losing patches on pkg upgrade;

  2. svn up changing checksums recorded in the pkg(8) database;

  3. ultimately "Which one wins?" in an update of /usr/src?

Add SVN rev of source tree to pkg metadata (or equivalent, plus indicate if the tree has been modified similarly to uname(1)) (Note: committed as r322412.)

RDBMS-backed web application to collect file checksums from every official package and allow users to verify files on-line.

Using pkgdb as an IDS?

CategoryProject CategoryTodo

PkgBase (last edited 2023-04-20T12:37:31+0000 by MateuszPiotrowski)