Ports and packages summit – BSDCan 2011
May 11, 2011 9:00-12:00am EDT
FlorentThoumie (portmgr - skype)
IonMihaiTetcu (portmgr - skype)
SimonNielsen (so, clusteradm)
ColinPercival (so, portsnap - tentative)
- Ben Kaduk (Athena project)
AdeLovett (skype - tentative)
The FreeBSD project has provided prebuilt ready-to-install binary packages for many years on a best-effort basis. While these packages do work in a large number of cases, there are too many inconsistencies and failure combinations, from the unpredictable update frequency to dependency handling across upgrades, for them to be used on a wider scale. This session centers on a round-table brainstorm that closes with a clear strategy and roadmap on how to improve binary package creation, distribution, installation and upgrading.
See also last years slides for some background.
- Introduction and current state (erwin 15 min)
- Athena project (kaduk 15 min)
- Issues with current system (plosher 10 min)
- brainstorm (erwin 50 min)
- break (15 min)
- pkgng (bapt@ 30 min)
- empty slot (30 min)
- license framework (tabthorpe)
- package cluster (flz 5 min)
- change of VCS?
Note: Not considering src HEAD, only -RELEASE and -STABLE
- EOL policy
- builds both used for building packages and QA
- infrequent builds
- hardware limitations
- number of nodes
- performance of head node
- inconsistent OPTIONS throughout the tree
- distribution – ftp mirroring
- disk space
- inconsistent package sets
- no concept of what is a “set”
- packages-x.y-release vs packages-x-stable
- no QA across ports, only individual ports
- manual log analysis
- package sets coupled with src releases
Scenario 1: Easy out-of-the-box initial installation of prebuilt packages. This works today.
Scenario 2: Installation of additional packages lateron should “just work”. Nothing is upgraded, only added to the existent installed packages. This requires either consistency between dependencies with earlier installed software, whereas today the new packages will have newer dependencies than what is already installed leading to disappointment. Solution is to install additional packages from the same, older set of packages.
Scenario 3: Upgrading should change the pointer to which set of packages to use, so future additions will also “just work”, and upgrades those packages that need upgrading.
How to get there: package sets We need to add consistency to packages as we do for the tree itself. The ports infrastructure only supports installing ports when the whole tree is consistent at a given point in time. The same restriction goes for packages, although this is not enforced which can lead to unexpected results. We therefore need a method to identify a consistent tree in time and encode this identifier into the binary packages. The installation tools can thereby identify the current installed set A and refuse the install a package from a different set B. The package tools should provide the user with a clear way to upgrade between package sets A to B. There needs to be a EOL cycle for how often package sets are released and for how long they will be supported. Download sites need to keep a copy of all Tier 1 architecture packages of all supported releases (not including HEAD) for that time (a CDN may help reduce disc space on the local mirrors, while needing large storage capacity at the backend).
A full tree of packages at given RCS-id within a given major branch.
Requires: single, unique tree-wide revision ID (change RCS or approximate with timestamp)
Weekly package sets.
Frequent enough not to need branching and pullups
EOL policy for package sets.
Keep package sets for 3 months.
EOL policy for FreeBSD releases/branches.
Keep current policy of following so@.
FreeBSD Release packages.
re@ includes latest available package set at the time of release building.
Package building cluster.
What needs to change? (set-id encoding, automated scheduled building).
What needs to change? (set-id checks, set upgrading).