Merging with Subversion
NOTE: This page has been migrated from the wiki to http://www.freebsd.org/doc/en_US.ISO8859-1/articles/committers-guide/subversion-primer.html - any future updates should be made there and not here.
Contents
This deals with merging code from one branch to another (typically, from head to a stable branch). For vendor imports, see SubversionPrimer/VendorImports.
NOTE: In all examples below, $FSVN refers to the location of the FreeBSD Subversion repository, svn+ssh://svn.freebsd.org/base/.
About merge tracking
From the user's perspective, merge tracking information (or mergeinfo) is stored in a property called svn:mergeinfo, which is a comma-separated list of revisions and ranges of revisions that have been merged. When set on a file, it applies only to that file. When set on a directory, it applies to that directory and its descendants (files and directories) except for those that have their own svn:mergeinfo.
It is not inherited - for instance, stable/6/contrib/openpam does not implicitly inherit mergeinfo from stable/6 or stable/6/contrib - as that would make partial checkouts very hard to manage. Instead, mergeinfo is explicitly propagated down the tree. If you merge something into branch/foo/bar, the following rules apply:
If branch/foo/bar doesn't already have a mergeinfo record, but a direct ancestor (for instance, branch/foo) does, that record will be propagated down to branch/foo/bar before information about the current merge is recorded.
Information about the current merge will not be propagated back up that ancestor.
If a direct descendant of branch/foo/bar (for instance, branch/foo/bar/baz) already has a mergeinfo record, information about the current merge will be propagated down to it.
If you consider the case where a revision changes several separate parts of the tree (say, branch/foo/bar and branch/foo/quux, but you only want to merge some of it (say, branch/foo/bar), you will see that these rules make sense. If mergeinfo was propagated up, it would seem like that revision had also been merged to branch/foo/quux, when in fact it hadn't.
Selecting the source and target
Because of mergeinfo propagation, it is important to choose the source and target for the merge carefully to minimize property changes on unrelated directories.
The rules for selecting your merge target (the directory that you will merge the changes to) can be summarized as follows:
- Never merge directly to a file.
Never, ever, merge directly to a file.
Never, ever, ever merge directly to a file.
Changes to kernel code should be merged to sys. For instance, a change to the ichwd(4) driver should be merged to sys, not sys/dev/ichwd. Likewise, a change to the TCP/IP stack should be merged to sys, not sys/netinet.
Changes to code under etc should be merged at etc, not below it.
Changes to vendor code (code in contrib, crypto etc.) should be merged to the directory where vendor imports happen. For instance, a change to crypto/openssl/util should be merged to crypto/openssl. This is rarely an issue, since changes to vendor code are usually merged wholesale.
Changes to userland programs should as a general rule be merged to the directory that contains the Makefile for that program. For instance, a change to usr.bin/xlint/arch/i386 should be merged to usr.bin/xlint.
Changes to userland libraries should as a general rule be merged to the directory that contains the Makefile for that library. For instance, a change to lib/libc/gen should be merged to lib/libc
There may be cases where it makes sense to deviate from the rules for userland programs and libraries. For instance, everything under lib/libpam is merged to lib/libpam, even though the library itself and all of the modules each have their own Makefile.
Changes to man pages should be merged to share/man/manN for the appropriate value of N.
Other changes to share/ should be merged to the appropriate folder under share and not to share/ directly
- Changes to a top-level file in the source tree such as UPDATING or Makefile.inc1 should be merged directly to that file rather than to the root of the whole tree. Yes, this is an exception to the first three rules.
When in doubt, ask. (If 'svn diff' shows you mergeinfo on an individual file, you are in doubt.)
If you need to merge changes to several places at once (for instance, changing a kernel interface and every userland program that uses it), merge each target separately, then commit them together. For instance, if you merge a revision that changed a kernel API and updated all the userland bits that used that API, you would merge the kernel change to sys, and the userland bits to the appropriate userland directories, then commit all of these in one go.
The source will almost invariably be the same as the target. For instance, you will always merge stable/9/lib/libc from head/lib/libc. The only exception would be when merging changes to code that has moved in the source branch but not in the parent branch. For instance, a change to pkill(1) would be merged from bin/pkill in head to usr.bin/pkill in stable/7.
Preparing the merge target
Because of the mergeinfo propagation issues described earlier, it is very important that you never merge changes into a working copy without the proper "ancestry." You must always have a full path back to the root of the branch you will merge into. For instance, when merging from HEAD to 9, you must have a full checkout of stable/9:
(create a directory in a file system that has at least 2 gigs free) $ cd <your directory> $ svn co --depth=empty $FSVN svn $ cd svn $ svn up --depth=empty stable $ cd stable $ svn up --depth=empty 9 $ cd 9
At this point you can either do:
$ svn up --set-depth=infinity
to check out the whole tree (recommended) or you can continue using '--depth=empty' to create proper antecedents of the directory you want to do the merge to. For instance:
$ svn up --depth=empty lib $ cd lib $ svn up libc $ cd libc
To save some typing you may wish to review the '--depth=immediates' option.
The target directory must also be up-to-date and must not contain any uncommitted changes or stray files.
I will give an example below of what happens if it does.
Identifying revisions
You must of course identify the revisions you wish to merge. If the target already has complete mergeinfo, you can ask svn for a list:
$ cd stable/6/contrib/openpam $ svn mergeinfo --show-revs=eligible $FSVN/head/contrib/openpam
If not, you will have to check the log for the merge source - but you probably have to do that anyway.
Merging
The principle
You want to merge
revision $R
in directory $target in stable branch $B
from directory $source in head
$FSVN is svn+ssh://svn.freebsd.org/base/
Assume that revisions $P and $Q have already been merged.
You are standing at the root of an up-to-date working copy of stable/$B.
The existing mergeinfo looks like this:
% svn propget svn:mergeinfo -R $target $target - /head/$source:$P,$Q
You merge like so:
% svn merge -c$R $FSVN/head/$source $target
You check the results like so:
% svn diff target
The mergeinfo record now looks like this:
% svn propget svn:mergeinfo -R $target $target - /head/$source:$P,$Q,$R
If the results are not exactly as shown, ask someone for assistance before committing. You may have made a mistake, or there may be something wrong with the existing mergeinfo, or there may be a bug in Subversion.
A real-life example
I wish to merge an OpenPAM update from head to stable/6 (stable/7 has it already). I have determined that I need to merge r174833, r174836, r175802 and r178818 in contrib/openpam, and revision r174837 in lib/libpam. Let's get started:
$ cd stable/6 $ svn up --set-depth=infinity $ cd contrib/openpam $ svn merge -c174833 $FSVN/head/contrib/openpam .
I'm already in trouble: the first svn merge gives me a conflict in lib/openpam_readline.c. I take a closer look at the logs, and see that I also need to merge r168464, which is what causes the conflict. No problem, I can revert and start over:
$ svn revert -R . $ svn merge -c168464 $FSVN/head/contrib/openpam . $ svn merge -c174833 $FSVN/head/contrib/openpam .
Right? Only in version 1.7. Before then the problem was that r174833 added two files, compile and include/security/openpam_attr.h. When I reverted the first attempt, these two files were left behind; for obvious reasons, svn revert will revert an addition, but not remove the actual file. The second attempt skipped the two files because they were in the way:
--- Merging r174833 into '.': U configure U LICENSE U Makefile.in U include/Makefile.in Skipped 'include/security/openpam_attr.h' U include/security/pam_appl.h U include/security/pam_constants.h U include/security/Makefile.in U include/security/openpam.h U include/security/pam_types.h U include/security/Makefile.am U include/security/openpam_version.h U include/security/pam_modules.h U include/Makefile.am U depcomp Skipped 'compile' U misc/gendoc.pl [...]
This is why it is so important to merge into a clean tree.
To fix this, I have to revert and remove the offending files before starting over:
$ svn revert -R . $ rm compile include/security/openpam_attr.h $ svn merge -c168464 $FSVN/head/contrib/openpam . $ svn merge -c174833 $FSVN/head/contrib/openpam . $ svn merge -c174836 $FSVN/head/contrib/openpam . $ svn merge -c175802 $FSVN/head/contrib/openpam . $ svn merge -c178818 $FSVN/head/contrib/openpam .
Note that I could also have merged all five revisions in one go:
$ svn merge -c168464,174833,174836,175802,178818 $FSVN/head/contrib/openpam .
I also need to merge lib/libpam:
$ cd ../../lib/libpam $ svn merge -c174837 $FSVN/head/lib/libpam .
Let's take a look at the resulting mergeinfo:
$ cd ../.. $ svn propget svn:mergeinfo contrib/openpam lib/libpam contrib/openpam - /head/contrib/openpam:168464,174833,174836,175802,178818 lib/libpam - /head/lib/libpam:174837,178818
The contents of the svn:mergeinfo property should be fairly self-explanatory, except for the addition of 178818, which was not one of the revisions I merged. It was inherited from stable/6:
$ svn propget svn:mergeinfo . /head:178818
Merging into the Kernel (sys)
As stated above merging into the kernel is different from merging in the rest of the tree. In many ways merging to the kernel is simpler because you always have the same merge target.
In this example we'll merge change 210012, which fixed a bug in the hwpmc(4) driver.
$ cd /local/checkedout/fbsd8/src/sys $ svn merge -c210012 svn+ssh://svn.freebsd.org/base/head/sys --- Merging r210012 into '.': U dev/hwpmc/hwpmc_core.c U dev/hwpmc/hwpmc_core.h
Now run an svn diff in the same directory. You may see unrelated property changes, such as:
Property changes on: dev/xen/xenpci ___________________________________________________________________ Modified: svn:mergeinfo Merged /head/sys/dev/xen/xenpci:r210012
These can be safely ignored.
You will, of course, build and test your merged kernel.
Once your tests are complete you can commit the code as normal. Commit messages should include both the original commit information as well as the line MFC r210012:.
Replace the r210012 with revision from your original commit to HEAD. The full commit mesage for this merge would be:
MFC r210012: Fix a panic brought about by writing an MSR without a proper mask. All of the necessary wrmsr calls are now preceded by a rdmsr and we leave the reserved bits alone. Document the bits in the relevant registers for future reference.
Your kernel merge is now complete.
Precautions before committing
As always, build world (or appropriate parts of it).
Check your changes with svn diff and svn stat. Make sure all the files that should have been added or deleted were in fact added or deleted.
Take a closer look at any property change (marked by an M in the second column of svn stat). Normally, you should not see any svn:mergeinfo properties anywhere except on your target directory (or directories).
If something looks fishy, ask for help.
Committing
Make sure to commit a top level directory to have the mergeinfo included as well. Do not specify individual files on the command line.
$ svn commit contrib/openpam lib/libpam