• JailingGUIApplications

Jailing GUI Applications

This is a short tutorial on how to run GUI applications jailed. This is done primarily for Firefox but the same principle can be applied for any other application.

We start by creating a basic jail that depends on your favorite jail abstraction tool. But to unhide all the abstracted details that those tools often do, we're going to do it all manually, and based on ZFS, so feel free to adapt the tasks for whatever jail management tool you're using.

To begin with, we'll dedicate zroot/jails as the base dataset for all jail deployments, and create a basejail in zroot/jails/basejail. The basejail method allows us to simply  zfs clone  it into target jail dataset, which is the most efficient way to share the base jail among other jails. Updating them all then requires updating the basejail and re-creating jail roots. Here's how.

The commands given are all executed as root, on the host, unless explicitly stated differently.

   1 zfs create -o compress=lz4 -o atime=off zroot/jails
   2 zfs create zroot/jails/basejail
   3 bsdinstall jail /zroot/jails/basejail

Here we used the bsdinstall method for convenience. Otherwise downloading and unpacking base.txz and configuring it should suffice. At this point we'd configure the basejail for pkg, like the location of your Poudriere repo if you have it. No other configuration is required for the base as the jails will basically run a single process.

So we snapshot it and create our Firefox jail filesystem:

   1 zfs snapshot zroot/jails/basejail@latest
   2 zfs create zroot/jails/firefox
   3 zfs clone zroot/jails/basejail@latest zroot/jails/firefox/root
   4 zfs create zroot/jails/firefox/var
   5 zfs create zroot/jails/firefox/tmp
   6 zfs create zroot/jails/firefox/home
   7 rsync -a /zroot/jails/firefox/root/var/ /zroot/jails/firefox/var/
   8 zfs set mountpoint=/zroot/jails/firefox/root/var zroot/jails/firefox/var
   9 zfs set mountpoint=/zroot/jails/firefox/root/tmp zroot/jails/firefox/tmp
  10 zfs set mountpoint=/zroot/jails/firefox/root/usr/home zroot/jails/firefox/home

For extra security we want our jail to run with minimum require privilege, so we set some properties on these datasets, which should make obvious why we separated them like this. Of course, these rules are not applicable to every application, as some, unfortunately would like to write or execute to/from paths they shouldn't. For firefox, these suffice, tho'.

   1 zfs set setuid=off exec=off zroot/jails/firefox/var
   2 zfs set setuid=off exec=off zroot/jails/firefox/tmp
   3 zfs set setuid=off exec=off zroot/jails/firefox/home

At this point it's worth observing that when base is to be update, all we need to do is update the basejail and create a new snapshot for cloning. With that, and separate var/home/tmp dirs, it's trivial to update the jails' bases, just zfs destroy root dataset and re-clone it from basejail. This will require unmounting and re-mounting the other datasets, but it can all be easily scripted for simple maintenance.

Next, with the filesystem in place, we install the packages. xauth and firefox are the base minimum, while liberation-fonts-ttf is recommended addition for some nice fonts in Firefox.

   1 pkg -c /zroot/jails/firefox/root install firefox xauth liberation-fonts-ttf

Next, we need to bootstrap launching firefox. The idea here is that all it takes to "launch firefox" is to launch its jail, and when the browser is closed, the jail should shut down too. This is achieved relatively simply. To make bootstrapping easier, we'll now launch the jail and run the commands in it. But first, we need to set up the jail.conf definition of it:

# /etc/jail.conf

allow.nomount;
exec.clean;
mount.devfs;
host.hostname = "$name.your-host-name.lan";
path = "/zroot/jails/${name}/root";
#securelevel = 3;

firefox {
    ip4.addr = "10.0.0.2";
    #exec.start = "/bin/sh /home/firefox/run-firefox";
    #exec.jail_user = "firefox";
    persist;
    devfs_ruleset = 5;
}

At this point, we comment out the exec. directives, and uncomment the persist directive because we want to get inside the jail with no processes running, to bootstrap it. But before we do that, there are two more undefined items here, the devfs ruleset and jail's ip address. So, let's handle those first.

The recommended way to set up the jails' networking is to clone lo0 and give it a dedicated range, then use pf to NAT the traffic out. In short these three things:

   1 # /etc/rc.conf
   2 
   3 #
   4 # Among other thigns you set up in rc.conf, the following is minimum required for jail networking.
   5 #
   6 # We use the 10.0.0.0/29 range just as an example for up to 6 jails
   7 #
   8 cloned_interfaces=lo1
   9 ifconfig_lo1_aliases="10.0.0.1-6/29"
  10 
  11 # And this to enable pf rules for NAT
  12 pf_enable="YES"
  13 pf_rules="/etc/pf.conf"

We enable the network by running  service netif cloneup , and the defined lo1 interface will be cloned with that IP range.

Then the pf rules. These are the minimum required to get NAT going, and it doesn't do anything else, so adjust as needed, or if you already have pf in place, the nat line is all that's required.

   1 # /etc/pf.conf
   2 
   3 # This is for re0 interface, so replace with whatever you have, like em0, igb0, ...
   4 extif = "re0"
   5 intif = "lo1"
   6 
   7 set skip on lo
   8 set state-policy if-bound
   9 
  10 nat on $extif inet from ($intif) to ! ($intif) -> ($extif)

And after that,  service pf start  should start and enable the firewall. If you're doing this over SSH, this article assumes you know what you're doing, missing the appropriate rules.

Next, the devfs ruleset is required to allow audio devices in the jail, so the applications can play audio. We copy the default ruleset for jails (ruleset number 4) from /etc/defaults/devfs.rules and add audio devices, into /etc/devfs.rules:

# /etc/devfs.rules

[devfsrules_desktop_jail=5]
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add path 'mixer*' unhide
add path 'dsp*' unhide

Basically we just added mixer* and dsp* devices in addition to the default jails ruleset. With that file in place, we just have to  service devfs restart  and the ruleset is in effect.

So now we're ready to start the jail and jexec into it for final setup.

   1 # Start the jail
   2 jail -c firefox
   3 
   4 # jexec into it (the commands listed here after this are done inside the jail)
   5 jexec -l firefox
   6 
   7 # First, create a user for firefox (note the exec.jail_user = "firefox" in the jail.conf, so that's the user)
   8 pw useradd firefox -w random -m
   9 
  10 # Write out the "init" script (note the exec.start path in jail.conf, so that's the init script)
  11 cat << EOF > /home/firefox/run-firefox
  12 #!/bin/sh
  13 
  14 export DISPLAY=:0.0
  15 /usr/local/bin/firefox > /dev/null &
  16 EOF
  17 
  18 # We did all this as root, so:
  19 chown firefox:firefox /home/firefox/run-firefox
  20 chmod u+x /home/firefox/run-firefox
  21 
  22 # Prepare the mountpoint for host's X unix socket
  23 mkdir /tmp/.X11-unix
  24 chmod 777 /tmp/.X11-unix
  25 
  26 # Done!
  27 exit

And that's it. We stop the jail with  jail -r firefox , uncomment the exec. bits from jail.conf, comment the persist bit, and the jail is almost ready to run. Finally:

   1 # Allow jails to talk to xorg
   2 xhost +
   3 
   4 # Mount the host's X unix socket into the jail
   5 mount_nullfs /tmp/.X11-unix /zroot/jails/firefox/root/tmp/.X11-unix
   6 
   7 # And finally make the jail's root readonly:
   8 zfs set readonly=on zroot/jails/firefox/root

Done. We start firefox by starting the jail itself:

   1 jail -c firefox

With the persist item commented out, the jail will shut down automatically when firefox is exited

A few gotchas for maintenance

1. The root is mounted readonly, so any pkg -j firefox upgrade operations (and similar) will require remounting it with readonly=off first

2. To update/upgrade the base, do all that's required on the zroot/jails/basejail, make a snapshot,  zfs destroy zroot/jails/${name}/root  (first umount /var, /tmp and /home from it), and re-clone the base into a new root dataset, remount /var, /tmp and /home with zfs from the host.

   • To update basejail:  freebsd-update -b /zroot/jails/basejail fetch install 

   • To upgrade basejail:  freebsd-update -b /zroot/jails/basejail -r 11.2-RELEASE --currently-running 11.1-RELEASE upgrade , though perhaps just untar the base.txz for each upgrade, thatis create a new basejail.

   • To unmount datasets under jail's root:  mount | grep ' on /zroot/jails/firefox/ | awk '{ print $3 }' | sort -r | xargs umount  (replace firefox with whatever jail it is)

   • To re-mount datasets under jail's root:  zfs list -o name | grep -E "^zroot/jails/firefox/" | xargs -n 1 zfs mount  (replace firefox with whatever jail it is)

3. The X unit socket will have to be re-mounted after reboot, ZFS datasets are mounted automatically. An exec.prestart could be added to the jail's config (jail.conf):

exec.prestart = "mount | grep ' on /zroot/jails/${name}/root/tmp/.X11-unix` || mount_nullfs /tmp/.X11-unix /zroot/jails/${name}/root/tmp/.X11-unix"

Integrating host-side browser launchers with jailed Firefox

Various launchers will want to launch local /usr/local/bin/firefox when an URL is clicked or otherwise selected to be open in the default browser. To integrate this with jailed Firefox, one needs a wrapper script that will convert local Firefox calls to jexec calls. Three things are needed for this:

1. Local "firefox" binary that will be used as the wrapper

2. A sudo-enabled NOPASSWD script that's called by the local "firefox" wrapper, and that executes jexec in the jail

3. A sudoer rule allowing the script in #2 to be started with NOPASSWD

As a simple example, assuming local "firefox" will be called with first param being the URL, eg. /usr/local/bin/firefox http://google.com, we then set up (all on the host):

   1 #!/bin/sh
   2 # This is /usr/local/bin/firefox
   3 
   4 if [ -z "$1" ]; then
   5         exit 1
   6 fi
   7 sudo /home/user/bin/jail-firefox-exec $1

   1 #!/bin/sh
   2 # This is /home/user/bin/jail-firefox-exec
   3 
   4 if [ -z "$1" ]; then
   5         exit 1
   6 fi
   7 jexec -U firefox firefox firefox $1

# In sudoers (called by sudo visudo), for example:

%wheel ALL=(ALL) NOPASSWD: /home/user/bin/jail-firefox-exec