Unadulterated Jails, the Simple Way
This document is still WIP.
While several guides on the net call this running jails "the hard way" (meaning, without a helper tool like ezjail or iocage), there really isn't anything hard about it, if you don't care about directory mappings/de-coupling (so called "thin jails"). There are basically five simple steps:
1. Setting up the Filesystem
The first step in creating a jail is deciding where the jail's filesystem will reside. For this little guide we're assuming jails are each in their own dataset under /zroot/jails, so let's create one for "jailname":
zfs create zroot/jailname
Working with UFS instead of ZFS is of course possible as well. You simply mkdir /path/to/jail. Other than not having all the ZFS goodies at your disposal (eg. migrating jails between hosts with zfs send | receive), one gotcha is that you cannot simply rm -rf the jail dir if you want to delete it. There are schg flags on some system bins which would have to be removed first. chflags(1) manpage has more info.
2. Installing the Jail
Installing the jail is as simple as running
bsdinstall jail /zroot/jails/jailname
Or doing it "manually" by unpacking the base.txz tarball fetched from FreeBSD's ftp mirrors. bsdinstall(8) manpage has more info.
3. Configuring the Jail
This step really depends on what the jail will be used for, and is just a regular post-installation configuration step one would take for any FreeBSD installation. For example
Set up login.conf for UTF-8 (don't forget running cap_mkdb /etc/login.conf)
Set up rc.conf to disable sendmail, to setup syslog logging to the host, hostname, etc...
Set up pkg and pkg repo if you have a custom one. Note, you don't need pkg bootstrapped in a jail, pkgs can be manipulated from the host using either -b or -j flags to pkg. See the pkg(8) manpage for more info.
Set up /etc/freebsd-update.conf for subsequent updates for nothing but world and maybe src if you need that
3.1. Using a helper "basejail"
One technique to prepare a boostrapped jail for quickest deployment involves setting up a "basejail", which is a complete, configured jail which you clone every time you need to spawn a new jail. While managing many jails is best done with some kind of configuration automation which, in a way, obsoletes having bootstrapped jails like this, in some cases this can be useful. In short, it boils down to:
# zfs create zroot/jails/_base # bsdinstall jail /zroot/jails/_base # chroot /zroot/jails/_base ... bootstrap and set up the jail # zfs snapshot zroot/jails/_base@11.0-bootstrapped
And then quickly spawning several new jails can be as simple as:
# zfs send -R zroot/jails/_base@11.0-bootstrapped | zfs receive zroot/jails/somejail1 # service jail start somejail1 # zfs send -R zroot/jails/_base@11.0-bootstrapped | zfs receive zroot/jails/somejail2 # service jail start somejail2 # zfs send -R zroot/jails/_base@11.0-bootstrapped | zfs receive zroot/jails/somejail3 # service jail start somejail3
Now, some tutorials suggest ZFS cloning (ie. zfs clone). This in itself indeed is the most simple way to clone a basejail to a production jail, however ZFS clones have certain drawbacks which over time completely negate any benefits. A ZFS clone is basically a snapshot, the filesystem is not physically duplicated and it saves all that space (a base 10.3 jail is some 300MB large). At first. Because as you use the jail and update it, more and more of those files are copied to individual jails as they change. Also, you cannot destroy a snapshot which has existing clones. That means you'd have to keep around all the basejail snapshots, or "promote" cloned jails. So why not just send | receive and copy the basejail which is independent from the start.
Such a basejail exists only as filesystem on disk, ie. you don't create an entry in /etc/jail.conf as explained in the next section, because you don't really need to run this jail (except occasionally to test things).
4. Configuring the jail.conf on the Host
Other than installing the jail filesystem itself, the only other required step is setting up the jail in /etc/jail.conf. In this file we set up common configuration options for all jails, and then for each jail individual options. See jail(8) manpage for more info, as well as jail.conf(5). Below is an example for two jails, "bind" and "nginx", running Bind9 and NGINX respectively. Since the jails bind directly to the public IP, no special configuration is required per jail as the configured defaults cover everything needed.
# Common configs for all jails
allow.nomount;
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
host.hostname = "$name.example.com";
ip4 = inherit;
ip6 = inherit;
mount.devfs;
path = "/zroot/jails/$name";
# The BIND9 jail
bind { }
# The NGINX jail
nginx { }In most cases, however, you would want to separate public and private addresses, for example to have jails communicate between each other via private address space, or generally to have a slightly more complex set up where you can control exactly which service-jail can communicate with what via firewall. Another option is binding jails only to private address space and controlling public access to them via NAT and RDR.
So jails are then controlled by master /etc/jail.conf, and jail service. Adding jail_enable="YES" to /etc/rc.conf will start them all on boot. You can re/start/stop individual jails as simple as running
service jail start jailname service jail restart jailname service jail stop jailname
4.1 Setting up Networking
TODO: Several networking scenarios:
1. Services jails on public IP
2. Services jails with public IP and private LAN
3. Services jail on LAN only, NAT/RDR public access
5. Jail maintenance
Maintaining a jail is not unlike maintaining any FreeBSD installation, anywhere. Whether you do it manually, or through configuration management like Ansible or Salt (or Chef, or Puppet, or ...) is irrelevant. Several gotchas, though:
5.1 Entering a Jail from Host
You can enter a jail from the host by running jexec jailname /bin/tcsh, see more info in the jexec(8) manpage.
5.2 Updating and Upgrading a Jail
You can freebsd-update a jail from the host with -b, eg. freebsd-update -b /zroot/jails/jailname fetch install. Note: freebsd-update relies to uname to check current version. That means upgrading jails, say from 10.3 to 11.0, and assuming you upgraded the host first, as you should because of the kernel, requires you to trick the jail into thinking the host is running the older version. Thankfully, freebsd-update has a switch for that, so in order to upgrade a 10.3 to 11.0, on a host already upgraded to 11.0, you can run:
freebsd-update -b /zroot/jails/jailname --currently-running 10.3-RELEASE -r 11.0-RELEASE upgrade
Fooling freebsd-update like this is required only for the upgrade command or fetch command. Running subsequent install commands can be done normally (with -b for jail dir).
5.3 Managing Packages inside Jails from Host
You can manage packages inside jails with -j or -b switches to pkg. More info in the pkg(8) manual. For example:
pkg -j nginx update pkg -j nginx install nginx pkg -j nginx upgrade
6. Jail migration between hosts
AKA Writing your own Docker with a few lines of code... TODO.
7. Running freebsd-update Reverse Proxy Cache
See freebsd-update Reverse Proxy guide.