Description

The FreeBSD project has some rather large CVS trees with more than 150000 changesets in ports and lots of files and directories. Therefore we need a fast VCS.

Current Implementation

SVN Implementation

SVN's performance varies a lot depending on how it is used. Remote access over http / WebDAV is slow with the ra_neon module (1.4 and earlier) but much faster with ra_serf (new in 1.5). Running svnserve is much faster than http+WebDAV. We'd be committing and operating with svnserve over ssh, which is the fastest option. Maintaining a local mirror is fast but with caveats.

One of svn-1.4's biggest speed weaknesses is that it puts a LOT of files in a directory with complex repositories. With a converted FreeBSD repository there are roughly 186,000 change set files - one file for each change and they're all in a single directory. This is fixed in svn-1.5 and is trivial to backport.

The next speed limiting factor is that the "fsfs" backend stores deltas going forwards. This means that rev1 has the full contents of the file and subsequent revs have deltas. Checking out rev1 is fastest and files with hundreds or thousands of changes get expensive. However.. svn uses skip-deltas so that it doesn't actually need to apply every single delta. A file with 100 changes would only need to be reconstructed from about 8 change sets instead of all 100. See http://svn.collab.net/repos/svn/trunk/notes/skip-deltas

For FreeBSD.org on the server side of things, the converted repository comes to about 3.5GB and is easy to fit entirely ram resident in a machine that isn't absurdly expensive. When fully cached and ram resident, operations are Very Fast, as one would expect. On a ram limited machine with the full repository locally (typical developer box with replicated repository), svn operations are still quite snappy still.

Formal benchmarking on our FreeBSD-sized workloads still needs to be done.

Hg Implementation

Some figures are available in Ollivier Robert's BSDCan paper. Hg is fast.

Git Implementation

Git is fast. Git is often (but not always) much faster then Mercurial. As a matter of fact a major design goal of git was to be incredibly fast and so it was designed with optimization as a prime driving factor in development. Some benchmarks might be nice: http://weblogs.mozillazine.org/jst/archives/2006/11/vcs_performance.html or http://git.or.cz/gitwiki/GitBenchmarks

Some FreeBSD specific benchmarks exist on the GitConversion page.

VCSFeatureFast (last edited 2008-06-17 21:37:22 by localhost)