Unadulterated Jails, the Simple Way

This document is still WIP.

While several guides on the net call this running jails "the hard way" (meaning, without a helper tool like ezjail or iocage), there really isn't anything hard about it, if you don't care about directory mappings/de-coupling (so called "thin jails"). There are basically five simple steps:

1. Setting up the Filesystem

The first step in creating a jail is deciding where the jail's filesystem will reside. For this little guide we're assuming jails are each in their own dataset under /zroot/jails, so let's create one for "jailname":

zfs create zroot/jailname

Working with UFS instead of ZFS is of course possible as well. You simply mkdir /path/to/jail. Other than not having all the ZFS goodies at your disposal (eg. migrating jails between hosts with zfs send | receive), one gotcha is that you cannot simply rm -rf the jail dir if you want to delete it. There are schg flags on some system bins which would have to be removed first. chflags(1) manpage has more info.

2. Installing the Jail

Installing the jail is as simple as running

bsdinstall jail /zroot/jails/jailname

Or doing it "manually" by unpacking the base.txz tarball fetched from FreeBSD's ftp mirrors. bsdinstall(8) manpage has more info.

3. Configuring the Jail

This step really depends on what the jail will be used for, and is just a regular post-installation configuration step one would take for any FreeBSD installation. For example

3.1. Using a helper "basejail"

One technique to prepare a boostrapped jail for quickest deployment involves setting up a "basejail", which is a complete, configured jail which you clone every time you need to spawn a new jail. While managing many jails is best done with some kind of configuration automation which, in a way, obsoletes having bootstrapped jails like this, in some cases this can be useful. In short, it boils down to:

# zfs create zroot/jails/_base
# bsdinstall jail /zroot/jails/_base
# chroot /zroot/jails/_base

... bootstrap and set up the jail

# zfs snapshot zroot/jails/_base@11.0-bootstrapped

And then quickly spawning several new jails can be as simple as:

# zfs send -R zroot/jails/_base@11.0-bootstrapped | zfs receive zroot/jails/somejail1
# service jail start somejail1
# zfs send -R zroot/jails/_base@11.0-bootstrapped | zfs receive zroot/jails/somejail2
# service jail start somejail2
# zfs send -R zroot/jails/_base@11.0-bootstrapped | zfs receive zroot/jails/somejail3
# service jail start somejail3

Now, some tutorials suggest ZFS cloning (ie. zfs clone). This in itself indeed is the most simple way to clone a basejail to a production jail, however ZFS clones have certain drawbacks which over time completely negate any benefits. A ZFS clone is basically a snapshot, the filesystem is not physically duplicated and it saves all that space (a base 10.3 jail is some 300MB large). At first. Because as you use the jail and update it, more and more of those files are copied to individual jails as they change. Also, you cannot destroy a snapshot which has existing clones. That means you'd have to keep around all the basejail snapshots, or "promote" cloned jails. So why not just send | receive and copy the basejail which is independent from the start.

Such a basejail exists only as filesystem on disk, ie. you don't create an entry in /etc/jail.conf as explained in the next section, because you don't really need to run this jail (except occasionally to test things).

4. Configuring the jail.conf on the Host

Other than installing the jail filesystem itself, the only other required step is setting up the jail in /etc/jail.conf. In this file we set up common configuration options for all jails, and then for each jail individual options. See jail(8) manpage for more info, as well as jail.conf(5). Below is an example for two jails, "bind" and "nginx", running Bind9 and NGINX respectively. Since the jails bind directly to the public IP, no special configuration is required per jail as the configured defaults cover everything needed.

# Common configs for all jails
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
host.hostname = "$name.example.com";
ip4 = inherit;
ip6 = inherit;
path = "/zroot/jails/$name";

# The BIND9 jail
bind { }

# The NGINX jail
nginx { }

In most cases, however, you would want to separate public and private addresses, for example to have jails communicate between each other via private address space, or generally to have a slightly more complex set up where you can control exactly which service-jail can communicate with what via firewall. Another option is binding jails only to private address space and controlling public access to them via NAT and RDR.

So jails are then controlled by master /etc/jail.conf, and jail service. Adding jail_enable="YES" to /etc/rc.conf will start them all on boot. You can re/start/stop individual jails as simple as running

service jail start jailname
service jail restart jailname
service jail stop jailname

4.1 Setting up Networking

TODO: Several networking scenarios:

1. Services jails on public IP

2. Services jails with public IP and private LAN

3. Services jail on LAN only, NAT/RDR public access

5. Jail maintenance

Maintaining a jail is not unlike maintaining any FreeBSD installation, anywhere. Whether you do it manually, or through configuration management like Ansible or Salt (or Chef, or Puppet, or ...) is irrelevant. Several gotchas, though:

5.1 Entering a Jail from Host

You can enter a jail from the host by running jexec jailname /bin/tcsh, see more info in the jexec(8) manpage.

5.2 Updating and Upgrading a Jail

freebsd-update -b /zroot/jails/jailname --currently-running 10.3-RELEASE -r 11.0-RELEASE upgrade

Fooling freebsd-update like this is required only for the upgrade command or fetch command. Running subsequent install commands can be done normally (with -b for jail dir).

5.3 Managing Packages inside Jails from Host

You can manage packages inside jails with -j or -b switches to pkg. More info in the pkg(8) manual. For example:

pkg -j nginx update
pkg -j nginx install nginx
pkg -j nginx upgrade

6. Jail migration between hosts

AKA Writing your own Docker with a few lines of code... TODO.

7. Running freebsd-update Reverse Proxy Cache

See freebsd-update Reverse Proxy guide.


VladimirKrstulja/Guides/Jails (last edited 2018-06-03T01:08:03+0000 by MateuszPiotrowski)