Perforce and Summer Of Code
The following is a brief primer for getting started with this system for Google SummerOfCode. Follow these instructions to familiarize yourself with Perforce, and ask your mentor about any problems you have. If you get really stuck, your mentor can get help from the FreeBSD Perforce administrators, who can easily undo anything you may have inadvertently messed up. As always, if your mentor can't help you, ask firstname.lastname@example.org.
Additional documentation, including more detail about some key topics and a number of useful recipes, can be found at: http://www.freebsd.org/doc/en/articles/p4-primer/index.html
If you've used Perforce before and are comfortable with it, please skip straight to Step 10 below to setup a private copy of the FreeBSD src tree.
If you haven't, then we recommend that you actually do all of the steps below. You can't hurt anything; the whole point of source control is that anything can be undone.
A Note Picking Names
A number of objects you will be creating have flat name spaces. Major examples are clients, branches, and projects. In particular clients and branches share a single set of names for all users. As such we have largely adopted a convention of prefacing each name with your username. For example, client (which are associated with a user and host pair) are typically named something like <username>_<hostname>. Branches are similarly prefixed with <username>_.
We similarly encourage projects to be named <username>_<shortname> so people can find your work either by name or topic.
Step 1: Getting an Account
To obtain a Perforce account, send an email to email@example.com (and CC your mentor, please) providing the following information:
- Desired username: Please provide first, second, and third choices. Names should be Unix-style names (no more than 8 characters, all lowercase ASCII letters and digits).
Email address: We'll create a username@FreeBSD.org alias that will forward to this address. All of your Perforce email will go to this address.
- Project name: A short name for your project. We recommend this be all lowercase ASCII letters and digits and not be more than 16 characters, since you (and other people) will type it quite a bit. Your mentor can help you come up with a name that will be meaningful to other people.
- Branch setup: We can create an initial branch for you of the full FreeBSD tree or any part of it. Talk to your mentor to determine how much of the system you need and we'll set up the necessary machinery for you. (Step 10 below explains how you could do this yourself if you wanted. We recommend that you only do this if you have prior experience with Perforce.)
You should get a response within 48 hours with your Perforce username and password. You should:
Send email to firstname.lastname@example.org and verify that it is correctly forwarded to your regular email account. In particular, you may need to adjust spam filters.
- Read the tutorial below and actually do most all of it to make sure you're comfortable with Perforce.
Step 2: Installing Perforce client binaries
Any of the following should suffice to install the client p4 executable on a FreeBSD system:
- portupgrade -N devel/p4
cd /usr/ports/devel/p4 && make && make install
- pkg_add -r p4
Client software for non-FreeBSD systems is available from www.perforce.com.
Step 3: Basic Environment Variables
Create a .p4config file with the following content:
P4USER= username provided above
P4PASSWD= password provided above
Then, add an entry into your .profile or .csh file if desired (recommended):
It's also advisable to check that the EDITOR environment variable refers to your favorite text editor.
Step 4: Create a "client"
In Perforce, a "client" describes your working setup. You need to specify the machine you are using, the local directory where your working files will reside, and what area of the repository they should map to. (If you're used to SVN or similar systems, a Perforce "client" can be thought of as a checkout description.)
To set up a client, check that you have properly set the environment variables above, then type:
$ p4 client
You'll be dropped into an editor with a form to edit. There are a number of fields, but you can ignore most of them for now. The ones that you should pay attention to are:
Client: <client_name> Owner: <username> Host: <local_machine> Root: <your work directory> View: <mappings>
The Client is an arbitrary name that can be used to access this form if you need to edit it later. The Owner should be self-explanatory. The Host is the name of your local machine. This should match the output of hostname on your machine. The default values are usually fine.
The Root directory is the full pathname on your machine where checkouts will occur. On my development machine, I have this set to "/home/tim/p4".
The View part describes which parts of the repository will be checked out and where (under your Root) they will go. It is a series of lines, each with two paths. The path on the left names a part of the repository (three periods ... is used in Perforce to mean "and all files and subdirectories"). The path on the right is a directory on your system, relative to the root directory of your client: For example, if my client is called dark, then I might have:
View: //depot/projects/soc2010/<myproject>/... //dark/<myproject>/...
Which would check out my user area in Perforce to the directory /home/tim/p4/<myproject>.
In the example above you should replace <myproject> with an appropriate name for your project.
You can come back and edit this later to add more lines to the View area. For now, just check out your user area in Perforce to a local directory "user" using a View description similar to that above.
Note: If you work from multiple machines, you should create a separate client for each one. If you ever forget what clients you've created, you can use "p4 clients" to request a list of them.
Step 5: More environment variables
You now should add one more line to your .p4config file:
P4CLIENT=name of your client
Step 6: Your first checkout
The sync command actually updates your local hard disk from the server, using the client defined in the P4CLIENT environment variable:
$ p4 sync
Of course, you don't yet have any files in your user area, so this won't do very much.
Step 7: Add a new file
To make sure everything works, let's add an empty file and commit it:
$ p4 add Milestones $ p4 submit
When you "submit", you'll be given a chance to edit a form that describes this change. The description has to be all indented for Perforce to properly parse the form:
Description: My first commit. Isn't this fun. Files: //depot/soc2010/<myproject>/Milestones # add
You can just delete the "PR:", "Submitted by:", and other canned description elements.
When you are done, save the form and exit the editor.
Note that you can also edit the Files field to change what files get submitted.
Step 8: Modify a file
Unlike some other version control systems, Perforce requires that you tell it before you change any file. By default, files on your local system are write-protected; when you tell Perforce that you want to edit a file, it will unlock the file locally and store a note on the server so that noone else can edit the file:
$ p4 open Milestones
You can now edit the file and make a list of the milestones for your project. When you are done, submit the changes back to the repository:
$ p4 submit Milestones
Note: If you believe you've inadvertently changed some files without using "p4 open", but aren't sure, you can use "p4 diff -se" to locate them. Use "p4 help diff" for more details.
Note: If you decide not to change a file, "p4 revert" can be used to undo your changes and release your lock on the file.
Step 9: Delete a file
As you might guess, you can remove a file by simply:
$ p4 delete foo $ p4 submit
Step 10: Branch a copy of the FreeBSD tree
Note: Many people find this step a little confusing at first. Depending on your project, you may not need this; talk to your mentor to decide what's appropriate. If necessary, your mentor can set this up for you.
Perforce is well-known for its ability to synchronize multiple work areas. You will probably want to keep a copy of the full FreeBSD tree in your local work area and periodically "integrate" new changes from the main tree with your changes.
To do this, you need to create a "branch" description that explains to Perforce how your local tree relates to some other area of the repository.
You create a branch the same way you create a client, by giving Perforce a command and editing a form:
$ p4 branch <mybranch>
As with the client, most of the fields here are pretty straightforward. Each branch description needs a name and an owner and you may find it helpful later on to have a meaningful description.
The interesting part is the View. Just as with clients, the View is a series of lines. Each line has a source path (where changes are copied from) and a target path (where changes will be copied to). For now, we just want to copy the entire FreeBSD source tree into our user area in a directory called src, so we'll need just one line that looks like this:
View: //depot/vendor/freebsd/src/... //depot/projects/soc2010/<myproject>/src/...
When you're done, save and exit the editor.
The branch describes how changes get moved. To actually move the changes, you "integrate" the branch:
$ p4 integrate -b <branchname>
Each time you do this, new changes will get copied over.
Note: How often you integrate new changes will depend on what you're working on and what other work is being done in the main tree. If your work interacts closely with other work happening in the main tree, you may need to integrate changes every couple of days; if you are working in an area where little other activity is happening, you may only need to integrate a few times over the entire summer.
Integrated changes are in a sort of limbo since they may conflict with other changes you've made to your Perforce work area. This has two implications:
- It is much easier if you don't have any unsubmitted changes in your workspace when you integrate. Otherwise, you may find yourself juggling your own unfinished changes along with everyone else's.
- After each integration, you need to "resolve" any possible issues before you "submit" the results.
Perforce has an interactive process for resolving issues that walks you through every possible conflict. With a little practice, this interactive tool can help you to resolve most changes very quickly:
$ p4 resolve
note: using the argument
$ p4 resolve -as
will automatically and safely resolve changes where only one side's copy differs, saving you some sanity.
If there were any issues, you may need to do additional testing before this is completely ready, then you can
$ p4 submit
The traditional Description at this point is "IFC" (Integrate From Current), sometimes with additional comments about any issues you had to resolve.
Note: If you forget what branches you've created, you can ask Perforce:
$ p4 branches -u <username>
Step 11: Your own custom FreeBSD
After integrating, resolving, and submitting, you now have your own custom version of FreeBSD which consists of the CURRENT development tree with any changes you've made.
Remember the FreeBSD build system does support building as a non-root user, so you can build a full system:
$ make buildworld buildkernel
or build any small part of it:
$ cd bin/ls $ make obj $ make
For this to work cleanly, you'll need to ensure that your regular user account has appropriate permissions on the appropriate subdirectory of /usr/obj:
# as root $ mkdir -p /usr/obj/home/<myuser> $ chown <myuser>:<myuser> /usr/obj/home/<myuser>
Note: You can also build FreeBSD without write access to /usr/obj by setting the MAKEOBJDIRPREFIX environment variable:
$ export MAKEOBJDIRPREFIX=~/freebsd/obj
You should set this in your .profile or .cshrc file to ensure it gets set consistently. See man make for details on this and other environment variables used in building.
Of course, you'll want to read UPDATING carefully to actually update your system. Your mentor can give you pointers on how to handle this effectively.
A few additional pointers
Getting help. The p4 command-line program has extensive built-in help. Start from "p4 help" and read carefully.
The Web interface. FreeBSD operates a web interface from which anyone can browse the Perforce repository: http://perforce.freebsd.org/
- Creating diffs to share with others. Unfortunately, we cannot expose the FreeBSD Perforce repository for public checkouts. This means that FreeBSD committers can easily get copies of your work but you'll need to do some additional work to share your efforts with others. Your mentor can help you figure out the best strategy for this, but it's likely to involve generating diffs of your work and posting them somewhere that other people can download them.