• EygeneRyabinkin
• PortsUpdatingNg

Making CLI interface to the UPDATING file

A quick note: currently I am concerned with the /usr/ports/UPDATING, but may be eventually /usr/src/UPDATING will also come into the game.

Related work

There is already pkg_updating(1) utility, but is just displays UPDATING entries for the given or all installed ports regardless of their versions. It also lacks processing of shell-like globs in package names inside AFFECTS.

Project phases

• Design (current)
• Development and testing of base tools (not done yet)
• Integration with existing port management utilities (not done yet)

Motivation

Users tend to forget to read /usr/ports/UPDATING. Current tools like portupgrade or portmaster have no support for showing UPDATING entries to their users. Obviously, it is better to have the tool that will be able to query the UPDATING document programmatically (via CLI) and thus all port-management tools that deal with upgrades will get the ability to present the relevant UPDATING entries for their users at the right time.

Survey of current UPDATING

TBD. I should examine all current entries and determine how they are used.

Design of the tool

Minimally, the tool should be able to

• query the ports database (/var/db/pkg) and present all UPDATING entries that will affect the currently-installed ports or their subset
• for the given port (name/version) and, perhaps, the new version of the port (that is going to be installed) give all UPDATING entries that are relevant to the upgrade
  • give ability to choose entries that are to be read before upgrade ("do $THAT before upgrading") and after upgrade ("do $THIS after upgrade"); implies that entries will have metadata on their applicability for the pre-upgrade/post-upgrade phases

Output formats:

• current UPDATING format for the selected subset of entries;
• other forms? selectable or hardcoded format?

Tool distribution:

• base system tool as the part of the pkg_install suite (most logical location, everyone has it out-of-the-box);
• port (people can select if they want this stuff or not).

Design of the data format

Representation of a source UPDATING format

Available options:

• reuse current format: it is RFC822-like with entries separated by label-like timestamps
  • pros:
    • human-readable
    • needs no fancy tools like XML/JSON validator/transformer -- can be easily integrated to the base
  • cons:
    • validator/parser must be written from scratch
    • third parties should learn "just another format" to cope with our UPDATING
• use XML/JSON:
  • pros:
    • existing tools: XML has XSLT for transforms and XML Schema for validation; JSON has JSONT and JSON Schema
    • can easily add own document structure to, for example, ease transforms into HTML (links to manual pages, etc)
    • third parties can use the source for their needs and employ the standard machinery for processing
  • cons:
    • either the tools should be integrated to the base system or we will restrict ourselves to the port or we will distribute the cooked (see below) UPDATING to the end-users
    • structure of the source document is not-so-easily readable due to the lot of tags (XML) or the braces/colons (JSON);
      • perhaps this isn't crucial, since the tool will transform the source to the readable format

Representation of a cooked UPDATING format

By "cooked" I mean the format of the date that is fed to the CLI utility as the input.

Requirements:

• should be easily parsed without much overhead
• should have the extensible structure that doesn't rely on the metadata order: we might need to add other metadata fields later and we should do it in a way that both old and new formats will be understood by all tool versions

Options:

• again, existing UPDATING format: headers will provide extensibility if the tool won't process the entry headers that it doesn't know about, but will transparently push them to the output
• textual format with "property=value" pairs delimited by some symbol or placed at the own lines

Data delivery to the end-user

Ideally, nothing should be changed: user invokes his preferred procedure to update the ports and he received UPDATING as the part of the process.

What format, source or cooked, should be delivered?

• source delivery:
  • source is kept in repository in 100% of cases, so CVS or SVN (or another VCS ports will move to) will deliver it as the part of regular updates
• cooked delivery:
  • either cooked UPDATING should be kept in VCS (not good, data duplication) or cooked UPDATING will be delivered by some other means (breaks POLA a bit)

I'd vote for the source delivery, but this has some implications for the XML/JSON/othername formats -- will need the tools for them, so, most likely, utility will be a port.

Metadata contents and its semantics

There are at least two cases:

• a) port X starting from version N requires user to make some actions before or after its installation b) there were some infrastructural changes that touch big parts of the ports tree (or the whole tree).

Type-"a" entries should have port name and version in the metadata; user should see such entries if he upgrades the port X from version < N to the version >= N.

Entries of type "b" tend to say the following: "if you had ports A, B, C, ... installed before the date Z, then you should do so-and-so to upgrade them without troubles". So, user should see such entries if either A, B, C, ... are upgraded and old version of ports being upgraded were built before the date Z.

Additionally, any entry can have the flags that will tell one if this entry is applicable to the pre-upgrade phase, post-upgrade phase or both.

When to present the data to user

Our CLI should not mess with this question: it is up the tool consumers to choose the right place and time. However, in order to make the consumers happy, we should design the metadata

Conclusion

That's all for now, folks!