• DevSummit
• 201105
• Ports

Ports and packages summit – BSDCan 2011

May 11, 2011 9:00-12:00am EDT

Attendees

• ErwinLansing (portmgr)

• MarkLinimon (portmgr)

• ThomasAbthorpe (portmgr)

• FlorentThoumie (portmgr - skype)

• IonMihaiTetcu (portmgr - skype)

• SimonNielsen (so, clusteradm)

• ColinPercival (so, portsnap - tentative)

• BrooksDavis (core)

• BaptisteDaroussin (pkgng)

• JulienLaffaye (pkgng)

• PeterLosher (mirror)

• Ben Kaduk (Athena project)

• BradDavis (clusteradm)

• AdeLovett (skype - tentative)

• StevenKreuzer (skype)

• BenLaurie

• WillAndrews

• DianeBruce

Summary

The FreeBSD project has provided prebuilt ready-to-install binary packages for many years on a best-effort basis. While these packages do work in a large number of cases, there are too many inconsistencies and failure combinations, from the unpredictable update frequency to dependency handling across upgrades, for them to be used on a wider scale. This session centers on a round-table brainstorm that closes with a clear strategy and roadmap on how to improve binary package creation, distribution, installation and upgrading.

See also last years slides for some background.

Agenda (tentative)

• Introduction and current state (erwin 15 min)
• Athena project (kaduk 15 min)
• Issues with current system (plosher 10 min)
• brainstorm (erwin 50 min)
• break (15 min)
• pkgng (bapt@ 30 min)
• empty slot (30 min)
  • license framework (tabthorpe)
  • package cluster (flz 5 min)
  • change of VCS?

Current state

Note: Not considering src HEAD, only -RELEASE and -STABLE

Fixed issues

• EOL policy
• ABI/KBI

WIP

• pointyhats
• builds both used for building packages and QA
• infrequent builds
• hardware limitations
  • number of nodes
  • performance of head node

Known issues

• inconsistent OPTIONS throughout the tree
• distribution – ftp mirroring
  • disk space
  • syncing
• inconsistent package sets
• no concept of what is a “set”
• packages-x.y-release vs packages-x-stable
• no QA across ports, only individual ports
• manual log analysis
• package sets coupled with src releases

Brainstorm

Ideal world

Scenario 1: Easy out-of-the-box initial installation of prebuilt packages. This works today.

Scenario 2: Installation of additional packages lateron should “just work”. Nothing is upgraded, only added to the existent installed packages. This requires either consistency between dependencies with earlier installed software, whereas today the new packages will have newer dependencies than what is already installed leading to disappointment. Solution is to install additional packages from the same, older set of packages.

Scenario 3: Upgrading should change the pointer to which set of packages to use, so future additions will also “just work”, and upgrades those packages that need upgrading.

How to get there: package sets We need to add consistency to packages as we do for the tree itself. The ports infrastructure only supports installing ports when the whole tree is consistent at a given point in time. The same restriction goes for packages, although this is not enforced which can lead to unexpected results. We therefore need a method to identify a consistent tree in time and encode this identifier into the binary packages. The installation tools can thereby identify the current installed set A and refuse the install a package from a different set B. The package tools should provide the user with a clear way to upgrade between package sets A to B. There needs to be a EOL cycle for how often package sets are released and for how long they will be supported. Download sites need to keep a copy of all Tier 1 architecture packages of all supported releases (not including HEAD) for that time (a CDN may help reduce disc space on the local mirrors, while needing large storage capacity at the backend).

Discussion points

1. Define “set”.
   A full tree of packages at given RCS-id within a given major branch.
   Requires: single, unique tree-wide revision ID (change RCS or approximate with timestamp)

2. Release frequency.
   Weekly package sets.
   Frequent enough not to need branching and pullups

3. EOL policy for package sets.
   Keep package sets for 3 months.

4. EOL policy for FreeBSD releases/branches.
   Keep current policy of following so@.

5. FreeBSD Release packages.
   re@ includes latest available package set at the time of release building.

6. Package building cluster.
   What needs to change? (set-id encoding, automated scheduled building).

7. pkg_* tools.
   What needs to change? (set-id checks, set upgrading).

8. ...