Allowing for parallel builds in the FreeBSD Ports Collection
Proposal: http://code.google.com/soc/2008/freebsd/appinfo.html?csaid=682F3B8616BC4B55 (This link is broken):
Possible alternative on the archive.org: https://web.archive.org/web/20100306065512/http://code.google.com/soc/2008/freebsd/appinfo.html?csaid=682F3B8616BC4B55
Building ports in parallel is considered unsafe and is generally a bad idea. The goal of this project is make building multiple ports at one time as safe as possible, and eventually enable parallel dependency building.
TODO
Updated 07/07/2008
$PKG_DBDIR locks |
COMPLETE |
|
Install awareness |
COMPLETE |
|
Fetch awareness |
COMPLETE |
|
Work awareness |
COMPLETE |
|
$PORT_DBDIR locks |
UNNEEDED(?) |
|
Implement FAKE_J |
COMPLETE |
|
Usable interface |
ABANDONED(?) |
|
Add features to pkg_install suite |
COMPLETE |
|
Project Update 07/07/2008
- bsd.parallel.mk has been added to hold information and the safe fake-pkg target, for the time being.
- Entry lock has been added in the form of BASE_LOCK, which is by default set to ${.CURDIR}/.working. Any action on a port by a user (with ALLOW_PARALLEL defined) immediately acquires this lock and recurses.
- The *-depends targets have been changed so that all dependencies are given their own targets ('depname'-subtarget).
- For all subtargets, a lock on BASE_LOCK in the dependency directory has been added, to that working dependencies aren't simply written off as not found.
- For package registration, a new fake-pkg has been created (in bsd.parallel.mk) that first builds registration information in DUMPDIR (${WRKDIR}/pre_reg) by default, then acquires a lock on PKG_DBDIR, and registers the port. Not yet implemented is a check for working ports what may conflict with a registering port.
- Locks added for distfile and patchfile fetching in do-fetch. A lock file named after the file to be fetched is created in DISTDIR, and removed once fetch has run.
- A FAKE_J flag has been added, and it will be used instead of passing a -j argument It is not working yet.
Project Update 08/14/08
- All modified target moved into bsd.parallel.mk.
- pkg_add and pkg_delete modified to honor locks on PKG_DBDIR.
- FAKE_J implemented, though testing is needed.
- Parallel fetching has been implemented with a modified do-fetch target.