FLAVORS tools guide

Basics

  1. PORTS_FEATURES containing FLAVORS is a hint from the tree that FLAVORS behavior is expected. This can be looked up with make -f /usr/ports/Mk/bsd.port.mk -V PORTS_FEATURES.

  2. A port origin may or may not have a FLAVORS defined. The list can be looked up with make -C devel/foo -V FLAVORS.

  3. If a port origin has FLAVORS then it can produce a package for each flavor. Depending on the wanted behavior for the tool it may only build the default flavor or loop and build all flavors for the port.

  4. Each flavor listed in FLAVORS defines a unique package from that origin. Meaning each FLAVOR should produce a globally unique PKGNAME.

  5. The chosen FLAVOR for a port origin should always be passed in as a make argument, not in the environment, like make -C devel/foo all FLAVOR=bar

  6. The first flavor listed in FLAVORS is a default flavor and does not need to be specified to produce that flavored package. It can be looked up with make -C devel/foo -V FLAVOR.

  7. The syntax for selecting a FLAVOR in the tree is @flavor which extends to all *_DEPENDS lines.

  8. Packages have a pkg annotation containing which FLAVOR it was built from. It can be looked up with pkg query -F <package.txz> '%At %Av' | awk '$1 == "flavor" {print $2}'

  9. Using the flavor of all should build all of the flavors (not a framework thing).

  10. Using the flavor of - should build the default flavor (not a framework thing).

Minimal support

  1. Support *_DEPENDS containing @flavor after an origin. The lookup for that dependency needs to pass along the chosen FLAVOR for that dependency line. Tools not supporting it would in theory bomb at some point since it is trying to act on a directory with @flavor in it. 

      BUILD_DEPENDS+= foo>0:devel/foo@bar
  LIB_DEPENDS+=   libother.so:devel/other
  2. It is not required that a tool produce all FLAVORS for a given port but the tool does need to support FLAVOR dependencies.

  3. The port@flavor syntax should ideally be supported to pass along the chosen FLAVOR=flavor as a make argument when building that port.

  4. Uniquely reference dependencies in internal dependency graph rather than coalescing all FLAVORS into that origin as a dependency. Meaning that if port A requires port B@foo and port C requires port B, the graph needs to contain both B and B@foo, not just B.

Dependency/performance challenge discussion

The fact that a dependency may have its own flavor means that in a large graph of packages to build, an origin may be needed several times with different flavors. It is also possible that a dependency contains the default flavor for a port. The lookup for that default flavor may then yield the graph containing something like B, B@default, and B@foo, where the default is listed twice. It is likely good to coalesce B and B@default here internally.

This is only really a problem for performance. If in all dependencies found the tool simply goes off and runs make -V for all of the data it needs, then there's nothing more to do. If the tool wants to be efficient then it may benefit from the following.

The way poudriere dealt with this lookup problem is as follows:

  1. Two lookup queues and a build queue are used. The first is "main/default" lookup queue and the 2nd is the "flavor" queue.
  2. Queue items should contain metadata such as:
    1. origin
    2. flavor
    3. should_add_to_build_queue
  3. N threads are spawned.
  4. If there's a list of ports from the command line or a file or whatever, then it is fully added into the "main" queue; flavored packages are added into the "flavor" queue.
  5. Each thread is fed an item from the queue being processed.

  6. When creating the main graph of ports, always ensure that a lookup of the main port is done first. If a dependency line is devel/foo@bar then devel/foo is looked up for all of its:

    1. PKGNAME
    2. FLAVORS
    3. FLAVOR

  7. The "default" lookup for devel/foo is tagged such that it won't be added to the actual build graph (unless something else determines it is needed later).

  8. The actual dependency on devel/foo@bar is delayed into a secondary queue. The queue item is tagged so that it will be added to the main build graph.

  9. The "default" queue is processed until empty, then the "flavor" queue is processed until empty. Then the "default" is checked to see if anything else landed in it. This repeats until both queues are empty.
  10. The N threads are stopped.
  11. The build queue can now be processed.

Ports/FlavorsTools (last edited 2017-11-28T23:42:58+0000 by BryanDrewery)