• pkgng

PKG

Pkg is the Next Generation package management tool for FreeBSD. It is the replacement for the current pkg_info/pkg_create/pkg_add tools that ports use to register local packages and which provide remote packages. Its main goals are to faciliate remote binary package upgrades. It also works with ports without remote binary packages.

Resources

• charter and roadmap

• pkg primer

• https://people.freebsd.org/~bapt/pres/pres-pkgng-bsdcan.pdf

Availability of binary pkgs for Download

The latest supported FreeBSD releases with security updates already includes /etc/pkg/FreeBSD.conf and known public keys.

If you do not have this file see: https://www.freebsd.org/security/advisories/FreeBSD-EN-14:03.pkg.asc

Custom packages can be built with poudriere.

Articles

• http://www.mebsd.com/make-build-your-freebsd-word/pkgng-first-look-at-freebsds-new-package-manager.html

• How to run your own repository

• Full binary upgrade article

• README

• FAQ

• http://blog.des.no/2012/06/return-of-freebsd-desktop.html and more from DagErlingSmørgrav's blog

• http://ivoras.net/blog/tree/2012-07-26.pkgng---best-thing-since-sliced-bread!.html

• tinderbox + pkgng

• tinderbox and pkgng to maintain custom repositories

Links

• puppet provider

• ruby bindings

• salt native support since 0.10: here and there

• CFEngine 3 pkgng package method

• Monitoring PKGNG in Zabbix, Part I - Package Statistics

• Monitoring PKGNG in Zabbix, Part II - Database Audits

• Introduction to writing plugins for pkgng in FreeBSD

• Building packages for pkgng with Jenkins and poudriere

• Libpkg Doxygen Documentation

• perl bindings

• chef support

• Python wrappers for libpkg

Videos

• Pkgng puppet amazon ec2

BSDCan

• 2012 Presentation Video

Development

• IRC channel: #pkgng@freenode

• GIT repo: https://github.com/freebsd/pkg

about

What it is not

pkgng is not:

• a port build tool
• a replacement for portupgrade/portmaster

What it is

pkgng is:

• the new package management tool for ports and binary packages
• a replacement for pkg_* tools
• a tool to query/manage installed packages
• a tool to deal with binary packages
• a tool to upgrade/install packages from a remote repository
• a library that provides all the package management in a safe way so one can write a new frontend

For now the documentation is on the github home page (the man pages are also slowly getting written)

How it works

Package format

A package is a tar archive which can be either uncompressed, or compressed in gzip, bzip2 or xz format. The first elements in a package are the metadata.

Metadata

+MANIFEST

The manifest is a text file in YAML. It contains all the information concerning a package; it also contains the scripts.

name: foo
version: 1.0
origin: category/foo
comment: this is foo package
arch: i386
www: http://www.foo.org
maintainer: foo@bar.org
prefix: /usr/local
licenselogic: or
licenses: [MIT, MPL]
flatsize: 482120
users: [USER1, USER2]
groups: [GROUP1, GROUP2]
options: { OPT1: off, OPT2: on }
desc: |-
  This is the description
  Of foo

  A component of bar
categories: [bar, plop]
deps:
  libiconv: {origin: converters/libiconv, version: 1.13.1_2}
  perl: {origin: lang/perl5.12, version: 5.12.4 }
files:
  /usr/local/bin/foo: 'sha256sum'
  /usr/local/bin/i_am_a_link: '-'
  /usr/local/share/foo-1.0/foo.txt: 'sha256sum'
dirs:
- /usr/local/share/foo-1.0
scripts:
  post-install: |-
    #!/bin/sh
    echo post-install
  pre-install: |-
    #!/bin/sh
    echo pre-install

Scripts

Valid scripts are:

• pre-install
• post-install
• install
• pre-deinstall
• post-deinstall
• deinstall
• pre-upgrade
• post-upgrade
• upgrade

The script MUST be in sh format, otherwise nothing else would work. The shebang is not necessary. pkg-install has/had this rule too, see here and here but did not enforce them as strictly has pkgng does. Also note the args to the scripts are currently:

• PRE-INSTALL
• POST-INSTALL
• DEINSTALL
• POST-DEINSTALL
• PRE-UPGRADE
• POST-UPGRADE

This inconsistency will be resolved once pkg-install tools are no longer supported.

When the manifest is read by pkg_create, files and dirs (yaml nodes) accept another syntax that allow creating packages containing root files without needing the root credentials :

files:
  /usr/local/bin/foo, 'sha256sum'
  /usr/local/bin/bar: {sum: 'sha256sum', uname: baruser, gname: foogroup, perm: 0644 }
dirs:
- /usr/local/share/foo-1.0
- /path/to/directory: {uname: foouser, gname: foogroup, perm: 0755}

+MTREE_DIRS

This is optional; this is used by the package the same way as done by the legacy tools, which means the MTREE is extracted in prefix before each installation.

In the future we hope that mtree will be deprecated in favour of a hier package or in single MTREE that won't be customisable in per package basis and because pkgng supports packing of empty directories, per package MTREE makes no sense anymore.

Content

A list of files and directories in full path (from /)

Local Database

Once a package is installed, it is registered to a SQLite database.

The SQLite database allow fast queries and ACID transactions. It also allows to query the reverse dependencies without a +REQUIRED_BY hack.

In order to save space the MTREE is only stored once, which save 18K per installed package.

pkgng supports a `register' command to register packages into the SQLite database from the ports. The register command can execute the install script, show pkg-message, ...

Repository format

A repository is a tar archive which must be compressed in xz.

It contains a sqlite database (which has all the information about the available packages). The database is mandatory.

It can also contain an optional RSA signature (hash using SHA256) for the sqlite database.

Installing a package with pkgng

1. execute pre_install script if any exists
2. execute install script with PRE-INSTALL argument
3. extract files directly to the right place
4. extract directories directly to the right place
5. execute post_install script if any exists
6. execute install script with POST-INSTALL arguments

Deinstalling a package with pkgng

1. execute pre_deinstall script if any exists
2. execute deinstall script with DEINSTALL argument
3. removes files
4. execute post_deinstall script if any exists
5. execute install script with POST-DEINSTALL arguments
6. extract directories

Upgrading a package with pkgng

A package can be in version 1: not upgrade aware, or in version 2: upgrade aware. If both the installed package and the new package are upgrade aware:

1. execute pre_upgrade script from the old package
2. execute upgrade script with PRE-UPGRADE argument from the old package
3. remove files from the old package
4. remove directories from the old package
5. extract files and directories from the new package
6. execute post_upgrade script from the new package
7. execute upgrade script with POST-UPGRADE argument from the new package

otherwise if falls back to the dumb way:

1. deinstall the old package
2. install the new one

the pkg command

add

Add files from a local package:

$ pkg add ./foo-1.0.txz

if foo has dependencies not already installed, pkg add tries to find them in the directory where the specified package is.

autoremove

When packages are installed, if they are installed as a dependency of other packages they are flagged as automatically installed, whether from ports or from the remote repositories. autoremove proposes to remove those automatic packages if no others depend on them. It also prints the disk space saved.

backup

when used with -d (dump) it exports the whole database in an archive (txz); each package's information is exported as a yaml+mtree file. when used with -r (restore) it reads an archive previously created via pkg backup -d and recreates the database according to the information from the archive.

clean

Clean up the cached packages (packages fetched from remote repositories)

create

This command can create a package from an installed package. It can also create a package from a manifest using a custom directory as root directory (this allows manual package creation).

delete

deinstall a package from local database.

help

show the list of subcommands. if a subcommand is provided as argument then it fires up the manpage concerning the subcommand.

info

show information about local packages. It contains a compatibility layer with the legacy pkg_info for easier migration but the long term purpose is to only keep end-user arguments for info.

example of info command:

$ pkg info -f zsh
Name           : zsh
Version        : 4.3.12
Origin         : shells/zsh
Prefix         : /usr/local
Categories     : shells
Licenses       : ZSH 
Maintainer     : bapt@FreeBSD.org
WWW            : http://www.zsh.org/
Comment        : The Z shell
Options        : 
        ZSH_GDBM: off
        ZSH_MAILDIR: off
        ZSH_MEM: off
        ZSH_MULTIBYTE: on
        ZSH_PCRE: on
        ZSH_STATIC: off
Flat size      : 11 MB
Description    : 
Zsh is a UNIX command interpreter (shell) which of the standard shells
most resembles the Korn shell (ksh), although it is not completely
compatible.  It includes enhancements of many types, notably in the
command-line editor, options for customising its behaviour, filename
globbing, features to make C-shell (csh) users feel more at home and
extra features drawn from tcsh (another `custom' shell).

If you want to use the zsh completion system, you should type the following
commands:

        $ autoload -U compinstall
        $ compinstall

See also zshcompsys(1) manpage. :)

WWW: http://www.zsh.org/

install

use this command to install packages from a remote repository. It will calculate dependencies, and proceed the installation of the whole set of packages.

lock

used to lock packages against reinstallation, modification or deletion.

query

This is the developer-oriented way to query information from packages. It has a custom syntax and allows complex queries:

$ pkg query "%n is installed from %o in version %v it takes %sh on the system" zsh
zsh is installed from shells/zsh in version 4.3.12 it takes 11 MB on the system

$ pkg query "%n is compiled with option %Ok set to %Ov" zsh
zsh is compiled with option ZSH_GDBM set to off
zsh is compiled with option ZSH_MAILDIR set to off
zsh is compiled with option ZSH_MEM set to off
zsh is compiled with option ZSH_MULTIBYTE set to on
zsh is compiled with option ZSH_PCRE set to on
zsh is compiled with option ZSH_STATIC set to off

register

this command registers as installed a packages whose files are already installed. if -l is not provided that it also executes the post-install scripts and shows the pkg-message. if -i /path is provided, then the files located in the /path (considered as the root directory) and listed in the plist as copied to the destination directory (default /).

repo

transform a directory containing a list of packages (recursively) into a pkgng repository. An RSA private key can be provided to sign that repository.

search

Allow searching on remote repositories (same kind of output as in pkg info).

update

Fetches the last version for the remote(s) repository(ies).

updating

print the ${PORTSDIR}/UPDATING entries that concern your installed packages.

upgrade

Find all the installed packages that can be updated and install the new dependencies if needed. linimon note: I fought the wiki syntax to get 'version' displayed correctly for half an hour. good luck. manolis note: I added an invisible character at the end of version (chr 129) - version is defined in moinmoin default css, that's the reason for the wrong rendering

version

same as pkg_version.

which

find which packages installed the given file.

TODO

If you want to help this is here:

• Code review
• Better signing infrastructure (needs a specification from security officer)
• Real solver
• Check vulnerability database signature
• Create users using pw_*, gr_* API, instead of calling pw(8)
• Add a way to replace packages (REPLACE knobs in ports ?)
• Add conflict information during pkg repo to allow detecting as much as possible conflicts prior fetching
• Add pkg tags (key/value) tags

KNOWN FAILURES

This is based on exp-runs (mostly):

---

CategoryPorts