libpkg, pkg tools rewrite
Complete packages (dependency inclusive) and package versioning are features that could have an immediate beneficial effect with regards to package distribution and management. However, the state of the pkg tools would only make this possible by layering new hacks on top of old. The creation of a package library would not only facilitate these improvements, but also cleaner code which could allow for more features to added over time.
The new package library (libpkg) will provide an interface for interacting with packages:
- Complete ("fat") package handling. These are packages that are dependency inclusive. The library will allow for these package to be created, and the format must be distiguishable from the current format.
- Versioned dependencies. This is a feature that that ports already supports, and packages benefit from. Instead of breaking packages that need a specific version of a dependency, they will be forced to depend on older versions of a package (possibly created with --no-glob).
- Database locking. For Summer of Code 2008, I worked on adding parallel dependency handling and locking to ports. The library will properly lock on the package registration database for operation that require access to it.
- Seperating the operations and aspects of the package system. This means pulling apart the repository, extraction, installation, deinstallation, database, and information gathering (and writing) parts of the package manipulation process so that the library is based on a model of the package system.
- Optimized routines. Pretty self explanatory. We not only want the library to provide shared code to the pkg tools (and other applications that want to take advantage of it), but we want the library to work as well as possible.
- Buffers and callbacks for status and messages will all be parts of the library. This will allow for the package to be used as a backend for more than just pkg tools.
- Fast access to all aspects and information of a package.
- Take advantage of libarchive directly for faster, more efficient methods package extraction, reading, and creation.
- This library is something that will be portable, expandable, and robust against various uses.
- Enhanced interaction with the package database.
- Maintain as much compatibility with the current pkg tools as possible.
- Simplify the operations of the pkg tools to provide clean, maintainable code.
- The new tool(s) for examining and modifying both packages and the package database.
- Upgrade support. Currently, if pkg_add tries to install a package that is already present on the system, it complains and does not continue with the installation. A graceful upgrade method (not automatic) will allow for package upgrades to be made cleanly and correctly. Package versioning will alert users to compatibility issues during upgrades.
- Pre-fetch for pkg_add. Download a package and all of its dependencies before an installation begins.
- Downward recursive package removal. Currently pkg_delete will recursively remove packages that depend on removed packages. This will add support for removing packages that a deleted package depends on, if it's not required by another package.
- Leveled repository checking for packages. i.e. check for a package locally before fetching it from the ftp mirrors.
- Submit a patch for a complete-package target in ports.
Project Milestones and Schedule (initial projection, likely to be modified)
April 24 - May 23: Library idea collection, pkg tools research, intitial design, redumentary API documentation, and the creation of the library skeleton.
May 24 - May 30: Finish skeleton library (wrap up any initial design issues).
May 31 - June 20: Ship package library skeleton, build basic and important package database interaction aspects of the library, aswell as basic package removal aspects.
Ship basic pkg_info, pkg_version and pkg_delete that demonstrate new features
June 21 - July 11: Add support for fat packages and package installation to both the library and pkg tools.
Ship basic pkg_add and updated pkg_delete
July 12 - July 22: Add versioning and upgrade features to library. Start work on pkg_create.
Ship basic version of pkg_create, aswell as updated pkg_add, pkg_delete, pkg_info, and pkg_version
July 23 - August 3: Catch up on code, fix bugs, add other features that were thought of and added during the planning and design phase, and work on completing pkg tools.
Ship whatever I've got.
August 4 - August 17: Testing and bug fixes.
Final milestone: ship commit ready library and pkg tools.
A few things to note: This timeline will almost definitely be updated through out the summer. While alot of features will be "added" one after another, most are dependent on each other. For a feature to be added really means making a push to get it as near to basic completion as possible. I also wouldn't be suprised if the planning phase was longer than expected.
Please email me if you'd like to add an idea or have a suggestion.