Introduction

There are a couple of FreeBSD git repositories that have been converted with various programs and various parameters. This page describes the "official" git-branches of the FreeBSD project that should be used as common repositories to base other work on.

The repositories are provided as a read-only mirror of the state of the Subversion src, doc and, ports repositories at http://svn.FreeBSD.org/base, http://svn.FreeBSD.org/doc, and http://svn.FreeBSD.org/ports respectively. Using git to commit changes to these repositories is not officially supported (but doable), instead it is supposed to serve as a collaboration point by using additional tools like Github, Gitorious, or Gerrit, etc.

Bear in mind that git will not replace Subversion for use in FreeBSD and the repository is only offered on a best-effort basis. The steps to redo the conversion are outlined below, so that people can move the process in-house should they see the need for it.

CAVEAT EMPTOR: The repositories might need to be re-done every once in a while due to bugs or improvements in the converter. This will result in changed commit IDs for the impacted branches. As the actual tree objects for a certain SVN revision will remain identical, it will always be possible to rebase or merge your work on these new branches and not run into any merge conflicts. Ask on the mailing lists if you need help with the required git commands.

The Repositories

The official repository-mirrors are updated at least hourly, should they lag the svn repository by more than that, please contact <uqs@FreeBSD.org> (monitoring still needs to be put into place, help wanted btw).

• git://github.com/freebsd/freebsd.git

• git://github.com/freebsd/freebsd-doc.git

• git://github.com/freebsd/freebsd-ports.git (the preferred sources)

• git://git.freebsd.org/freebsd.git
• git://git.freebsd.org/freebsd-doc.git
• git://git.freebsd.org/freebsd-ports.git (the canonical sources)

• https://code.google.com/p/freebsd (stale, Google Code chokes on pushes that are too large, devs informed and aware of the issue)

The obsolete, non-compatible repositories are:

• A git-svn clone of head and a couple of stable branches, but with truncated history.
  • git://gitorious.org/freebsd/freebsd.git

• A git-svn clone of head, with complete history. It's what you get when doing: git svn clone -Thead svn+ssh://svn.freebsd.org/base/

  • git://git.freebsd.your.org/freebsd-head
  • rsync://repos.freebsd.your.org/freebsd-head.git/
  • git://github.com/freebsd/freebsd-head.git
  • git://gitorious.org/freebsd/freebsd-head.git

  • https://code.google.com/p/freebsd-head

Known Problems

• SVN merges for single revisions (that span only parts of the tree then), are not really representable as merges in git and instead are called cherry-picks there. The converter tries to be smart about whole tree merges however -- this heuristic is only activated for the head/, user/, and projects/ branches. A whole-tree merge of project/foo into head/ should thus be correctly represented in git. Merges from head/ to stable/ or from contrib/ to head/ will be plain commits, i.e. "cherry-picks".
• It seems that with git v1.8.0 the commit hashes of merge commits have changed when using the exporter. It is unclear who's fault it is currently, but it means that the conversion is not repeatable for users of git v1.8.0 and that a re-roll of the whole repositories might be necessary

The first differing commit in the repo went from

% git show --format=raw 42f0fad|head
commit 42f0fadccab6eefc7ffdc1012345b42ad45e36c2
tree de2d7c6726a45428d4a310da2acd8839daf9f85f
parent 5fba0401c23a594e4ad5e807bf14a5439645a358
parent 25062ba061871945759b3baa833fe64969383e40
parent 89bebeef185ed08424fc548f8569081c6add2439
parent 48231afadc40013e6bfda56b04a11ee3a602598f
parent c7d5f60d3a7e2e3c4da23b157c62504667344438
parent e7bc108f0d6a394050818a4af64a59094d3c793e
author rgrimes <rgrimes@FreeBSD.org> 739897097 +0000
committer rgrimes <rgrimes@FreeBSD.org> 739897097 +0000

to

% git show --format=raw e209a83|head
commit e209a83c1e0a387c88a44f3a8f2be2670ed85eae
tree de2d7c6726a45428d4a310da2acd8839daf9f85f
parent 5fba0401c23a594e4ad5e807bf14a5439645a358
parent 25062ba061871945759b3baa833fe64969383e40
parent 89bebeef185ed08424fc548f8569081c6add2439
parent c7d5f60d3a7e2e3c4da23b157c62504667344438
parent e7bc108f0d6a394050818a4af64a59094d3c793e
parent 48231afadc40013e6bfda56b04a11ee3a602598f
author rgrimes <rgrimes@FreeBSD.org> 739897097 +0000
committer rgrimes <rgrimes@FreeBSD.org> 739897097 +0000

Quick Start

$ git clone git://github.com/freebsd/freebsd.git
$ cd freebsd
$ hack, hack, hack
$ git commit

Advanced, useful examples to show the diff between a subdir of a branch, or the commits that introduced these diffs. It is basically what svn mergeinfo --show-revs=eligible does. See git-log(1) for more info.

$ git diff origin/stable/9 master -- usr.sbin/acpi
$ git log --graph --format=oneline --right-only --cherry-pick --no-merges origin/stable/9...origin/master -- usr.sbin/acpi

See any other git howto or documentation if you're stuck on the basics.

Collaboration

There is no endorsed way yet of how users should collaborate with developers on changes they have made. Please bear in mind that git is strictly ancillary at this point and not many committers are using it.

Having your patches/feature-branches published on github for a FreeBSD committer to comment on and pull from seems like the first sensible step. Sending a patch-bomb with git send-email might be an alternative to github or for developers that do not want to use git.

Receiving and integrating these patches can be done in various ways, see below.

Integration

Integrating the changes back into svn is not as trivial as using git-svn(1), as the current repository doesn't have the required meta-information. Instead patches can be applied to a subversion workspace just like any other regular patch.

A more sophisticated approach would be to share the svn and the git workspaces, which can be done by putting .svn in .gitignore and vice versa. This requires subversion 1.7 and you need to know how to recover from out-of-sync workspaces in both subversion and git, or be extra careful to avoid this situation.

Using git-svn (FreeBSD committers only)

The repositories on github also have a branch that is the result of a conversion using git-svn, i.e. the necessary conversion has already been made. To use this, you simply need to:

$ git clone git://github.com/freebsd/freebsd.git
$ cd freebsd
$ git svn init -Thead svn+ssh://svn.freebsd.org/base .
# People w/o write access to SVN can use this instead: git svn init -Thead --rewrite-root=svn+ssh://svn.freebsd.org/base svn://svn.freebsd.org/base .
$ git update-ref refs/remotes/trunk `git show-ref origin/svn_head|cut -d" " -f1` # git-svn really needs this branch
$ git svn fetch  # This will now reconstruct the necessary metadata and should be pretty fast
$ git checkout trunk; git branch -D master # delete master
$ git checkout -b master trunk # and have master hang off trunk
$ git svn rebase # this will now update everything to the latest state in head

Now you can periodically sync to svn using git svn rebase and you can push small changes back to svn using git svn dcommit, but read on.

Things to keep in mind:

• Never git merge branches, unless you know what you're doing.

• Always git rebase your work on top of master, then git svn dcommit can push the top commits to svn.

• Always double-check with git svn dcommit -n to see what would happen.

• While you can use git add for new files just fine, you won't be able to push those upstream, you can however use the patch, apply it to some subversion checkout and do the commit there. This is a shortcoming of our very own Subversion hacks, but hey, it's better than nothing!

• While git-svn now allows you to set svn:mergeinfo when committing, this is so fragile that the FreeBSD projects discourages its use. Please use svn(1) for merging, sorry.

FAQ

• I want authornames!

  • Good for you, and this has been done but abandoned again for various reasons: There's no authoritative, complete and stable mapping of svn users to "Name <email>" pairs. Especially for people without access to freefall. Random contributors are more likely to have a conversion using the current parameters.

• Why the full history and not only since day X
  • It would make the history incompatible with other people's repository and the space savings are simply not worth the hassle. As of Oct/2012, all revisions make up about 933MB.

Everything done is so that everybody can verify the integrity of the conversion using the tools mentioned in this article. Hence the focus on repeat-ability of the process.

Further Reading

It is *really* recommended, that you read Git for computer scientists and skim GitTalkDevSummit, although it's really outdated.

It really helps to understand the data structure of a git commit, because then you know how merging/rebasing works and can fix snafus easily.

Implementation Details

The software used for the conversion is a slightly modified fork of svn2git, as used by the KDE project. It can be found at https://gitorious.org/~uqs/svn2git/uqs-svn2git

It requires a rules file to map svn trees and/or revisions to git branches or tags. The current rules don't make use of tags but simply store them as branches.

See the attached freebsd.rules, freebsd-doc.rules, and freebsd-ports.rules for the current rulesets (subject to change). The command line parameters used for the continuous conversion are:

svn2git/svn-all-fast-export --rules freebsd.rules --add-metadata-notes --identity-domain FreeBSD.org http://svn.freebsd.org/base
svn2git/svn-all-fast-export --rules freebsd-doc.rules --add-metadata-notes --identity-domain FreeBSD.org http://svn.freebsd.org/doc
svn2git/svn-all-fast-export --rules freebsd-ports.rules --add-metadata-notes --identity-domain FreeBSD.org http://svn.freebsd.org/ports