NOTE: This is from the VCS bakeoff.

Please see http://www.freebsd.org/doc/en_US.ISO8859-1/articles/committers-guide/subversion-primer.html for how to merge in the FreeBSD project.

Merging with Subversion

NOTE: Subversion 1.5 has merge tracking built-in. You no longer need svnmerge. See http://subversion.tigris.org/svn_1.5_releasenotes.html#merge-tracking for more information.

Background

A common argument levied against Subversion is that it provides no mechanism to assist in repeated merging. Specifically, if you merge a change from one branch to another in Subversion, and then try and repeat the merge some time later, Subversion does not record the revisions that have already been merged.

This means that the end user has to do some additional work to record information about the revisions that have been merged, otherwise the merge operation will generate many conflicts that need to be resolved by hand.

This argument is, like many, true and false.

It is true that the Subversion API provides no specific mechanism to record this information. And the "standard" command line tools that ship with Subversion do not do this.

However, the Subversion API provides a mechanism to attach arbitrary key/value pairs to entries in the repository. These are called "properties" in the Subversion parlance. These properties provide an "escape hatch" through which extra information can be added to the repository without it needing to be a "first class" item.

Various third party tools use properties to record additional information in the repository. Of particular relevance to this discussion is svnmerge. This ships in the contrib/client-side directory of the Subversion distribution, and is available in both sh(1) and Python implementations. svnmerge uses properties to record merge information, allowing repeated merges to be performed easily.

svnmerge is not the only tool that does this. SVK does something similar. There are probably others.

MFC Walkthrough

This walkthrough shows how svnmerge would be used to implement all the MFC merging functionality required by the FreeBSD project.

Repository creation

First, we need to create a repository and populate it with some files. This example follows the suggested layout described in SVN_Repo_Layout. We create freebsd/src/trunk/, and a projects directory, ready to be populated with two long lived projects.

% svnadmin create repo
% svn checkout file://`pwd`/repo wc
Checked out revision 0.
% cd wc
% svn mkdir freebsd
% svn mkdir freebsd/src
% svn mkdir freebsd/src/trunk
% svn mkdir projects
% svn mkdir projects/trustedbsd
% svn mkdir projects/geom
% svn commit -m 'Initial tree creation'
Adding         freebsd
Adding         freebsd/src
Adding         freebsd/src/branches
Adding         freebsd/src/trunk
Adding         projects
Adding         projects/geom
Adding         projects/trustedbsd

Committed revision 1

With that done we can add some sample files to trunk/.

% cd freebsd/src/trunk/
% echo 'foo' > foo
% echo 'bar' > bar
% echo 'baz' > baz
% svn add foo bar baz
A         foo
A         bar
A         baz
% svn commit -m 'Add initial three files'
Adding         trunk/bar
Adding         trunk/baz
Adding         trunk/foo
Transmitting file data ...
Committed revision 2.

Creating a stable branch

In order to demonstrate MFCs we need a branch to hold -stable.

% cd ../
% svn update
At revision 2.
% svn mkdir branches
A          branches
% svn copy trunk branches/8.x
A          branches/8.x
% svn commit -m 'Create 8.x branch in preparation for 8.0 release'
Adding         src/branches
Adding         src/branches/8.x

Committed revision 3.

Changing -current

Before a change can be MFCd we need a change to MFC. Accordingly, create a simple change to -current.

% cd trunk
% echo 'This must be MFCd' > foo
% svn commit -m 'A change to MFC'
Sending   trunk/foo
Transmitting file data .
Committed revision 4.

As you can see, this change is revision 4. Remember this.

Carrying out MFCs

Now that we have a change to MFC we can go and MFC it.

The first thing we need to do (and this has to be done once per branch) is tell svnmerge that this is a directory we'll be merging to. svnmerge has a similar command structure to svn and cvs. Namely, "svnmerge subcommand subcommand options". The initialisation subcommand is init, and needs to be run in the directory that you will be merging to.

Accordingly, change in to the -stable directory (branches/8.x), and run svnmerge.

% cd ../branches/8.x/
% svnmerge init
property 'svnmerge-integrated' set on '.'

% svn commit -m 'Property changes from svnmerge init'
Sending      8.x

Committed revision 5.

Because Subversion has recorded that branches/8.x was copied from src/trunk we don't need to tell svnmerge where we're merging from -- unless told otherwise it will assumme that the merge is going to happen from the source of the copy.

Running this command has created a new property in this directory called svnmerge-integrated. This property change needs to be committed.

Now that svnmerge knows where we're merging from, we can use the avail subcommand to find out if there are any revisions that we can merge.

% svnmerge avail
4

Indeed there are. As you can see, this has shown that there has been one change on src/trunk, which is revision 4. This is correct. We want to MFC this change, so we use svnmerge to help with this.

% svnmerge merge -r 4
U    foo

property 'svnmerge-integrated' set on '.'

Instead of using the svn merge command we use svnmerge merge. As well as merging the change in to the working copy this also updates the svnmerge-integrated property to note that r4 has been merged. We can now commit the MFC from the working copy.

% svn commit -m 'MFC r4 from trunk'
Sending        8.x
Sending        8.x/foo
Transmitting file data .
Committed revision 6.

More complicated MFCs

You don't have to merge every change, as this example shows.

First, change back on the trunk, and commit three changes. Once of these changes we definitely don't want to MFC, the other two are MFC candidates.

% cd ../../trunk
% echo 'foo - not for MFC' > foo
% svn commit -m 'Change foo.    Not for MFC'
Sending    trunk/foo
Transmitting file data .
Committed revision 7.
% echo 'bar change, for MFC' > bar
% echo 'baz change, for MFC' > baz
% svn commit -m 'Fix bar, MFC after: ...' bar
Sending       bar
Transmitting file data .
Committed revision 8.
% svn commit -m 'Fix baz, MFC after: ...' baz
Sending       baz
Transmitting file data .
Committed revision 9.

Now, go back to the stable branch, and see what we can commit.

% cd ../branches/8.x
% svnmerge avail
7-9

This shows that revs 7, 8, and 9 are available for merging. But we know that one of them (r7) is not an MFC candidate. We can inform svnmerge of this with the block subcommand. We block the revision from showing (which is a property change, so should be committed).

% svnmerge block -r 7
property 'svnmerge-blocked' set on '.'

% svn commit -m 'Prevent r7 from showing as a merge candidate'
Sending       8.x

Committed revision 10.

Now we can use the avail subcommand again to see what's available for merging.

% svnmerge avail
8-9

That no longer shows r7 as a merge candidate.

Now we need to decide how to merge these two revisions. We can either merge and commit them one after another, or we can lump them in to one large change that's going to be merged en-masse. Opting for the later choice, we can use the 8-9 syntax with the merge subcommand.

% svnmerge merge -r 8-9
U    bar
U    baz

property 'svnmerge-integrated' set on '.'

% svn commit -m 'MFC r8 and r9'
Sending        8.x
Sending        8.x/bar
Sending        8.x/baz
Transmitting file data ..
Committed revision 11.

That's it. Pretty much everything you need to know about carrying out MFCs. svnmerge has some additional features that are useful.

Project branch merge walkthrough

MFCs are relatively simple, as they're always one-way (from -current, to one or more of the stable branches). Project branch merges are more complex. A long lived project, like trustedbsd or geom is going to want to periodically carry out MFCs. But they're also going to want to merge their functionality back to -current, either all in one go, or piece by piece as the work becomes appropriate for inclusion in to -current.

This walkthrough assumes you've already carried out the basic MFC walkthrough.

Merge from current to project branch

Lets start the trustedbsd project, make some changes to src/trunk, and merge them in.

Start the project by taking a copy of src/trunk.

% cd ../../../..
% svn update
At revision 11.
% svn copy freebsd/src/trunk projects/trustedbsd
A         projects/trustedbsd/trunk
% svn commit -m 'Start the trustedbsd project'
Adding       projects/trustedbsd/trunk

Committed revision 12.

Now continue making changes to src/trunk.

% cd freebsd/src/trunk
% echo 'ongoing development' > foo
% svn commit -m 'Work continues on trunk' foo
Sending   trunk/foo
Transmitting file data .
Committed revision 13.

So this change to -current was r13.

Switch to the directory that contains the trustedbsd project, make sure it's up to date, and initialise the merge tracking property.

% cd ../../../projects/trustedbsd/trunk/
% svn update
At revision 13
% svnmerge init
property 'svnmerge-integrated' set on '.'

% svn commit -m '"svnmerge init" prop changes'
Sending        trunk

Committed revision 14.

We can make some local changes to the trustedbsd project, and commit them.

% echo 'tustedbsd development' > bar
% svn commit -m 'Special security changes'
Sending        trunk/bar
Transmitting file data .
Committed revision 15.

And we can find out which revisions in src/trunk are available for merging.

% svnmerge avail
13

Merging those changes is easy (so far, this is just like doing an MFC to a -stable branch, as described previously).

% svnmerge merge -r 13
U    foo

property 'svnmerge-integrated' set on '.'

% svn update
At revision 15.
% svn commit -m 'MFC'
Sending        trunk
Sending        trunk/foo
Transmitting file data .
Committed revision 16.

Merge from project branch back to -current

This is more interesting. Suppose that it's now time to merge changes from the project branch back to the trunk. The basic theory is the same.

First, we need to initialise merge tracking. Previously we did this on the copy destination (branches/8.x, and trustedbsd/trunk). This is because these were also the destination of the merges. Now, however, src/trunk is going to be the destination, so that's where we need to initialise the merge tracking info.

This is slightly longer winded, purely because we need to know when the branch was created. This is so we can mark all the changes that happened before the branch as already being merged. Subversion doesn't record quite enough metadata about branches to make this automatic.

Fortunately, this only needs to be done once per merge-target, and is easy to do.

To find out when the branch was created run svn log on the branch. Use the --stop-on-copy option and the log report will stop at the revision in which the copy happened (instead of crossing over the copy). If we do that:

{{{% cd ../../../ % svn update At revision 16. % cd freebsd/src/trunk % svn log --stop-on-copy ../../../projects/trustedbsd/trunk


r16 | nik | 2006-09-22 22:12:24 +0100 (Fri, 22 Sep 2006) | 1 line

MFC


r15 | nik | 2006-09-22 22:11:21 +0100 (Fri, 22 Sep 2006) | 1 line

Special security changes


r14 | nik | 2006-09-22 22:10:44 +0100 (Fri, 22 Sep 2006) | 1 line

"svnmerge init" prop changes


r12 | nik | 2006-09-22 22:09:20 +0100 (Fri, 22 Sep 2006) | 1 line

Start the trustedbsd project


}}}

we see that the trustedbsd project was branched from the trunk at r12.

Now we can run svnmerge init. There are two changes here. First, we need to include a -r option that specifies the revisions that have already been merged. That's 1-12 in this instance.

We also need to specify where we're merging changes from. This is going to the trustedbsd/trunk direectory.

% svnmerge init -r 1-12 ../../../projects/trustedbsd/trunk
property 'svnmerge-integrated' set on '.'

% svn commit -m '"svnmerge init" prop changes'
Sending        trunk

Committed revision 17.

Now the merge tracking information is initialised we can find out whether there are any revisions to merge.

% svnmerge avail
14-16

This doesn't look quite right. r14 was running svnmerge init on the trustedbsd trunk. r15 was an actual change. r16 consisted of merging a change from current (r13) on to the branch. svnmerge calls these reflected revisions, and they can be removed from the list by specifying the -b option.

% svnmerge avail -b
15

As you can see, that's generated the correct output. r15, which was an actual change on the trustedbsd branch, is the only rev shown. It can easily be merged back to -current.

% svn merge merge -r 15
U    bar

property 'svnmerge-integrated' set on '.'

% svn commit -m 'Merge r15 from trustedbsd project'
Sending        trunk
Sending        trunk/bar
Transmitting file data .
Committed revision 18.

Merging from multiple project branches

That's demonstrated how to merge from -current to a project branch, and also from a project branch back to -current. What happens if you have more than one project branch?

Start by creating a second project branch, and using svnmerge init to initialise merge tracking for that branch.

% cd ../../../
% svn update
At revision 18.
% svn copy freebsd/src/trunk projects/geom
A         projects/geom/trunk
% svn commit -m 'Create GEOM project'
Adding         projects/geom/trunk

Committed revision 19.
% cd projects/geom/trunk
% svnmerge init
property 'svnmerge-integrated' set on '.'

% svn commit -m "svnmerge init" prop changes'
Sending        trunk

Committed revision 20.

Now go and make changes in three places -- in the geom project, in the trustedbsd project, and in -current.

% echo 'A geom change' > baz
% svn commit -m 'Change baz to support geom'
Sending        trunk/baz
Transmitting file data .
Committed revision 21.
% cd ../../trustedbsd/trunk
% svn update
At revision 21.
% echo 'A trustedbsd change' > bar
% svn commit -m 'Change bar to support trustedbsd'
Sending        trunk/bar
Transmitting file data .
Committed revision 22.
% cd ../../..
% svn update
At revision 22.
% cd freebsd/src/ctrKtrunk
% echo 'Exciting changes for foo' > foo
% svn commit -m 'Update foo on -current'
Sending        trunk/foo
Transmitting file data .
Committed revision 23.

With changes in both projects, lets find out which revisions are available to merge.

% svnmerge avail -b
22

That doesn't look right. r22 was a trustedbsd change, but r21, a geom change, isn't listed.

As you've probably realised, this is because the merge tracking record that says that projects/geom/trunk can be merged to freebsd/src/trunk has not yet been created.

Again, we need to use svn log --stop-on-copy to find out when the geom project was created. Then svnmerge init can be run, marking all the prior revisions as being merged in.

% svn log --stop-on-copy ../../../projects/geom/trunk
------------------------------------------------------------------------
r21 | nik | 2006-09-22 22:16:36 +0100 (Fri, 22 Sep 2006) | 1 line

Change baz to support geom
------------------------------------------------------------------------
r20 | nik | 2006-09-22 22:16:16 +0100 (Fri, 22 Sep 2006) | 1 line

"svnmerge init" prop changes
------------------------------------------------------------------------
r19 | nik | 2006-09-22 22:15:45 +0100 (Fri, 22 Sep 2006) | 1 line

Create GEOM project
------------------------------------------------------------------------
% svnmerge init -r 1-19 ../../../projects/geom/trunk
property 'svnmerge-integrated' set on '.'

% svn update
At revision 23.
% svn commit -m '"svnmerge init" prop ch
Sending        trunk

Committed revision 24.

With that done, svnmerge avail -b should print something useful.

% svnmerge avail -b
svnmerge: multiple heads found. Explicit head argument (-S/--head) required.
The head values available are:
  file:///usr/home/nik/freebsd-scm/repo/projects/trustedbsd/trunk
  file:///usr/home/nik/freebsd-scm/repo/projects/geom/trunk

As this error explains, now that we've told svnmerge that two branches can be merged on the trunk we need to be more explicit with some commands, and specify which merge source we're talking about. If we re-run this command, specifying the merge source we're interested in we get the correct answer.

% svnmerge avail -S ../../../projects/trustedbsd/trunk -b
22
% svnmerge avail -S ../../../projects/geom/trunk -b
21

This restriction applies to merging revisions as well. We can merge the changes in from the branch, either in one lump, or (as in this case) one by one, but we need to specify the merge source path as well as the revision to svnmerge merge.

% svnmerge merge -r 21
svnmerge: multiple heads found. Explicit head argument (-S/--head) required.
The head values available are:
  file:///usr/home/nik/freebsd-scm/repo/projects/trustedbsd/trunk
  file:///usr/home/nik/freebsd-scm/repo/projects/geom/trunk
% svnmerge merge -r 21 -S ../../../projects/geom/trunk
U    baz

property 'svnmerge-integrated' set on '.'

% svn commit -m 'Merge r21 from geom'
Sending        trunk
Sending        trunk/baz
Transmitting file data .
Committed revision 25.
% svn update
At revision 25.
% svnmerge merge -r 22 -S ../../../projects/trustedbsd/trunk
U    bar

property 'svnmerge-integrated' set on '.'

% svn commit -m 'Merge r22 from trustedbsd'
Sending        trunk
Sending        trunk/bar
Transmitting file data .

Committed revision 26.

So that's r25 (which came from geom) and r26 (which came from trustedbsd) committed. We can now go back to those branches and see that these changes are available to merge back in.

First, trustedbsd.

% cd ../../../projects/trustedbsd/trunk
% svnmerge avail -b
23-25

That's correct, and we could merge those back in to this branch if we so choose. Now for the geom branch.

% cd ../../geom/trunk
% svnmerge avail -b
svnmerge: multiple heads found. Explicit head argument (-S/--head) required.
The head values available are:
  file:///usr/home/nik/freebsd-scm/repo/projects/trustedbsd/trunk
  file:///usr/home/nik/freebsd-scm/repo/freebsd/src/trunk

That looks odd. The geom branch isn't merging directly with the trustedbsd branch, so why is it showing up in this list?

Recall that the geom branch was copied from trunk after trunk had been configured to accept merges from trustedbsd. That information has persisted in the geom branch, even though it's no longer relevant.

Now there's nothing technically wrong with this. The information might be redundant (if we're not planning on doing merges from trustedbsd to geom), but it just means that we need to add -S to some commands. This is a bit tedious, so we can fix this.

Recall that this information is recorded in properties. We can use Subversion's commands to inspect the properties, and edit them.

First, we can find out the properties that are set here.

% svn proplist
Properties on '.':
  svnmerge-integrated

Now we can find out the value of the svnmerge-integrated property.

% svn propget svnmerge-integrated
/freebsd/src/trunk:1-18 /projects/trustedbsd/trunk:1-12,15

That shows that r1-18 from src/trunk are integrated, as are r1-12 and r15 from trustedbsd. By removing the trustedbsd entry we effectively undo the svnmerge init that was inadvertently copied.

% svn propset svnmerge-integrated /freebsd/src/trunk:1-18 .
property 'svnmerge-integrated' set on '.'
% svn commit -m 'Remove bogus record of integration with trustedbsd branch'
Sending      trunk

Committed revision 27

And now svnmerge avail prints the correct list of revisions, without prompting for the -S parameter.

% svnmerge avail -b
23,26

Those revisions can be merged using svnmerge merge.

Conclusion

This has hopefully shown how Subversion, plus svnmerge, can be used to support all the project's merge needs.

Remember, since Subversion doesn't make any distinction between branches and HEAD, with everything being just a directory, it's entirely possible to set up more complicated merge tracking scenarios if necessary. For example, you could have a project branch that merged to another project branch, which then merged to src/trunk. Or, with per-developer directories in the repo, a ports committer could keep their ports in their user/ directory, make changes there, and, when ready, easily merge the changes back in to the correct freebsd/ports/... directory.

SVN_Merging (last edited 2013-05-30 19:54:15 by EdMaste)