PKGNG Charter and Road Map

Introduction

What the pkgng project is

The pkgng project aims to provide FreeBSD with a state-of-the-art binary package management system, something that has been sadly lacking throughout the existence of FreeBSD. See the pkgng (now "pkg") wiki page for more details.

With release 1.0 on 30th August 2012, pkgng is on track for becoming the standard binary package management system for FreeBSD. This page attempts to lay out how and when that will happen, together with a description of the development process and what guarantees will be made about the stability of APIs, DB Schemata and packaging formats.

Who decides all this stuff?

Anyone can propose changes to pkgng -- appropriate fora are FreeBSD mailing lists, particularly freebsd-ports@freebsd.org; by raising an issue on the pkgng GitHub site or by supplying patches -- git pull requests or the output of git format-patch are preferred, but anything reasonable will do. The FreeNode #pkgng IRC Channel is where most of the day-to-day discussion between developers occurs, and is open to all comers.

Operational control of the pkgng project rests with the pkgng development team, which consists of those people that have commit access to the pkgng GitHub repository. Decisions are taken informally and collectively, with the project leaders (bapt@ and jlaffaye@) having the casting vote in the event of dispute.

The pkgng development team will review all patches and other submissions, decide on their acceptability and correctness and are responsible for adherence to general development policy guidelines, setting development milestones, creating releases, dealing with bug reports and the day-to-day work of committing changes to the git repository.

Anyone that submits particularly good or interesting or frequent patches is at risk of being invited to become a member of the pkgng development team, by consensus of existing members. There's no requirement that pkgng developers also be FreeBSD committers, although that does seem to be the case in general.

FreeBSD portmgr@ has principal oversight of pkgng development policy -- agreeing the overall direction of the project and coordinating related changes to the ports tree in general. Other groups (re@, security@ etc.) are to be consulted wherever pkgng development intersects with their areas of responsibility. Development goals and overall planning are open for discussion by any and all interested parties (not limited to FreeBSD developers) in whatever FreeBSD fora may be appropriate (primarily FreeBSD.org mailing lists, this Wiki, http://forums.freebsd.org/, discussion and presentations at BSD conferences or developer summits). The pkgng project is expected to conform to what has received the support of the majority of FreeBSD developers, as determined informally by such discussions, and resultant policies should be summarized on the pkg wiki page.

Development management

Repository Management

All submissions other than bug-fixes should be supplied as diffs against the master branch, and any substantive code modifications must first be committed to master for a reasonable time before being merged to a release branch. Exceptions may be made where code in master and release branches has diverged significantly, but such changes should be avoided wherever possible. Guidelines on what constitutes a 'reasonable time' are as specified for development in the FreeBSD base system.

Newly written source code should conform to style(9), and reformatting of pkgng specific code in the repo to better conform to style(9) is encouraged. style(9) related commits should be separated from functional changes to code. Code derived from external sources (sqlite, libyaml, etc.) should retain its original formatting. Commit messages should briefly summarize the changes entailed, and should include issue number tags where relevant.

Particularly large or radical changes may, at the discretion of the pkgng development team, be developed within branches created under the principal pkgng GitHub repository. Normally individual contributors would be expected to clone the pkgng git repository and create their own local branches according to the usual git workflow. This can be done by anyone: all-comers are welcome to contribute in this fashion. One reason for holding a branch centrally could be to facilitate several development team members working on it together. Code from development branches must be merged to master before inclusion into any pkgng release.

Code Releases

Release branches will be created at such intervals as the pkgng team decides, by branching from master. All formal production-ready code releases must be made from a release branch and not directly from master. Development snapshots may be made from master in between release cycles. Major code changes, other than those required to fulfill previously agreed milestones should be avoided on release branches prior to the release being made, and are absolutely forbidden afterwards. The GitHub issue tracking facility will be used for tracking code milestones. After release, changes to any release branch are confined to bug-fixes only.

Release numbering will use the standard major.minor.patch three-level convention. Various API, ABI and other stability requirements apply to releases with the same major release numbers, as detailed below. Release branching will apply at the major.minor level (ie the branch is release-1.0, not release-1), with test releases and bug fixes from that branch updating the patchlevel. Prior to release from a branch, the patchlevel field indicates alpha- or beta- test or release candidate status, with version numbers progressing as follows:

package version

meaning

pkg-1.0.a1

The first alpha-test release prior to release 1.0

...

pkg-1.0.a3

Subsequent alpha-test releases

...

pkg-1.0.b1

The first beta-test release

...

pkg-1.0.b7

The seventh beta-test release

...

pkg-1.0.r1

The first release candidate

...

pkg-1.0.r3

The third release candidate

...

pkg-1.0

1.0 Release

pkg-1.0.1

1.0 Release + the first patch

pkg-1.1.a1

The first alpha-test release prior to release 1.1

...

Development snapshots from master should be numbered like so: pkg-1.0.4.20120902 using the version number from the previous release and the snapshot date.

The ports-mgmt/pkg port will track the latest release branch starting from the formal release. ports-mgmt/pkg-devel will track whatever development snapshots may be made, until the branch for the next release is created, when it will track the new release branch with all alpha- and beta- tests and release candidates, up to the point of the final release, at which point it will revert to tracking any snapshot releases. ports-mgmt/pkg will always provide tested production quality software.

API and other Stability Guarantees

libpkg ABI and API Stability

At this early stage in development, while the capabilities of pkgng are still undergoing rapid changes and improvement, less strict controls over changes will apply. The situation will be normalized with pkg-2.0 release, whereupon the usual convention of "don't make backwards incompatible changes" within the lifetime of any major version will apply.

libpkg.so.0

There is no guarantee of API or ABI stability for libpkg.so.0 in releases with major version number 1. The API will only be finalized with pkg-2.0 release.

libpkg.so.1 and subsequent

At pkg-2.0 release, pkgng will offer forward ABI stability for the duration of all 2.x releases, and the same for each subsequent major release. After 2.0 release, the package major version number will be incremented for any backwardly incompatible changes. The libpkg.so ABI version number will be incremented in synchrony with the pkg major version number.

API to be removed must previously have been documented as deprecated for the duration of a whole major version cycle. In general, API forward compatibility must be maintained over any and all releases unless there is very good reason to change.

The API stability promise applies to both the C-language API provided by libpkg.so.N and to the shell programming interface supplied by the pkg(8) program itself, and to any other language APIs that may be added in future as part of a pkgng release.

Major and minor version releases will not be coordinated with the release schedule for the FreeBSD base system, except for the general rule that any production release of pkgng should be avoided during the immediate run-up to any FreeBSD release.

Package and package metadata stability

pkgng packages are compressed tarballs containing exactly one +MANIFEST file in YAML-1.2 format, either zero or one +MTREE files and then all of the files belonging to the package stored with their complete paths from the root of the filesystem. Until release 2.0 these facts about the basic package format may be modified at will. After 2.0, such changes may only be made concurrently with a major version number change. (Meaning that packages may be required to be recreated on major version changes, but not otherwise.)

Package Metadata for pkg-1.x

Any package metadata changes are allowed, so long as it is always possible to upgrade from pkg-1.x to pkg-1.x+1. Best efforts should be made to ensure compatibility is maintained as far as possible.

Package Metadata for pkg-2.0 and subsequent

Packages created with pkg-N.0 should be compatible with any version of pkgng having the same major version number. Packages created with pkg-N.x+1 should at minimum be compatible with pkg-N.x to allow for upgrading. No other compatibility guarantees will be made, although best efforts will be made to preserve compatibility between all versions from the same major release branch.

During the lifetime of a major version number, new sections may be added to package metadata. Older pkgng releases should simply ignore unrecognized sections. New sections may duplicate and modify functionality from older sections, and newer pkgng releases may process data from such additional sections in preference to the originals. However the old sections must be maintained in the metadata for the duration of the entire major release cycle.

Switching a metadata section between mandatory and optional (or vice-versa) may only be done at a major version release.

Database Schema

While it will always be possible, direct interaction via SQL with either ${PKG_DBDIR}/local.sqlite or ${REPOPATH}/repo.sqlite is discouraged and will not be supported in any way. Users are strongly encouraged to interact with pkgng databases only through the published APIs. Arbitrary DB schema changes may be made at any release, and the pkgng team will have little to no sympathy for anyone complaining that such changes "broke their scripts."

The pkg(8) application and any application using libpkg.so will automatically update local.sqlite schema as pkgng itself is updated. Since repository catalogues will generally be downloaded from remote sites outside the administrative control of any one user, repository catalogue schema versioning allows for a degree of forward compatibility: an older, locally installed copy of pkg(8) will frequently be able to use a newer repository catalogue format, and will recognise and provide an informative error message when it cannot.

Similarly people that maintain private package repositories will not have to rebuild all packages on any minor version release, although they may need to rebuild their catalogue (by running pkg repo) -- something that would generally happen automatically as part of the normal processes of repository maintenance.

Road Map

Much of this is pretty tentative.

Target Date

Actual Date

Milestone

Notes

1 Sep 2012

30 Aug 2012

pkg-1.0 release

From here ports-mgmt/pkg tracks releases -- patches for bug fixes only. New ports-mgmt/pkg-devel tracks release candidates for next release.

1 Oct 2012

1 Oct 2012

pkg-1.0.1 release

8 Sep 2012

10 Oct 2012

HEAD (10-CURRENT)

pkgng becomes default. Use WITHOUT_PKGNG in make.conf to stick with previous behaviour.

6 Nov 2012

pkg-1.0.2 release

7 Dec 2012

pkg-1.0.3 release

21 Dec 2012

pkg-1.0.4 release

30 Dec 2012

9.1-RELEASE

Announce deprecation of pkg_install in 9.1 release notes, in advance of 9.2 and 10.0 releases. Last ever release with pkg_install as default

Around Jan 2013

After 9.1-RELEASE

MFC 'pkgng becomes default' to 8/Stable, 9/Stable.

Q1 2013

pkg-1.1.a1

First alpha test for version 1.1

Q1/Q2 2013

pkg-1.1 release

New functionality release

31 Jan 2013

EoL 9.0-RELEASE

Delayed dependent on 9.1-RELEASE

28 Feb 2013

EoL 7.4-RELEASE

(Unknown)

10.0-RELEASE

pkgng as default packaging system, pkg_install still available in base.

(Unknown)

9.2-RELEASE

pkgng as default packaging system, pkg_install still available in base.

(Unknown)

After 10.0-RELEASE

Removal of pkg_install from base in HEAD -- will still be available in ports.

(Unknown)

After 10.0-RELEASE

MFC removal of pkg_install to 9/Stable, 10/Stable approx 1 month after removal from HEAD.

Around Mar 2013

pkg-1.2 release

Subsequent 1.x series releases about every 3 months until planned feature set is complete.

(Unknown)

pkg-2.0 release

Freeze API for libpkg.so.1. From this point, incompatible changes require major version bump.

30 April 2014

EoL 8.3-RELEASE

Earliest date for final removal of pkg_install

pkg_install compatibility will have to be maintained in the ports and old-style pkgs provided until at least the EoL of 8.3-RELEASE (planned for 30 April 2014).

pkgng will be available to users of all current and future FreeBSD releases, but it won't be made the default in any 7.x release, in 8.x up to 8.3-RELEASE or in 9.x up to 9.1-RELEASE.

Targets for availability of pkgng packages for different FreeBSD major versions and CPU architectures from official FreeBSD repositories should be added to this road map.

pkgng/CharterAndRoadMap (last edited 2022-10-03T02:31:44+0000 by KubilayKocak)