Jails

Introduction

Jails were introduced in FreeBSD 4.x by Poul-Henning Kamp

More information:

Patches

Feature

Status

Author

Description

CPU + RAM limits

Not fully working / stalled

ChrisJones

Jails can now have the amount of memory available to their processes' resident sets (RSS)
CPU limiting is implemented by giving each jail a number of CPU shares and tracking the estimated CPU usage of the tasks that run in that jail (SoC 2006 JailResourceLimits)
fix: jtune not showing resource usage patch
fix: jtune not showing resource usage + page faults patch
based on FreeBSD 6.x
Memory Limits for 7.0 by Christopher Thunes patch
Memory Limits for 7.1 by Tom Judge patch
Resource limit for jails (CPU, memory, filedesc, process) patch by Menshikov Konstantin, mailinglist announcement
based on FreeBSD 8.x
Hierarchical Resource Limits SoC 2009 by Edward Tomasz Napierala
based on FreeBSD 9.x

Multi-IPv4/v6/no-IP jails

done / commited

Bjoern A. Zeeb

As an alternate solution to full network stack virtualization, this work shall provide a lightweight solution for multi-IP virtualization. Perforce status overview SVN 188281
HEADS UP: r185435 multi-IPv4/v6/no-IP jails in HEAD
HEADS UP: multi-IPv4/v6/no-IP jails now in 7-STABLE
This update gives you: zero, one or multi-IP jails; IPv4 and IPv6 support; cpuset support for jails; jail names and states to ease administration; 32bit compat on 64bit, jail v1 compat;...
based on FreeBSD 7.1/8.x

Wildcard IP (INADDR_ANY) should not bind inside a jail

done / stalled

Frank Behrens

If there is a jail, the assigned IP address should not considered as valid address for INADDR_ANY in host. With this change an easy jail setup is possible without further modifications of hosts base configuration! PR 84215 patch
based on FreeBSD 6.x
Is covered by Bjoern A. Zeebs work on Multi-IPv4/v6/no-IP jails

jexec - selection by jail name

done / commited

Frank Behrens

jexec(8) needs the numeric id of jail. This id is not constant and changes on jail restarts. Therefore it is desirable to select a jail by it's name. The attached patch is a sample implementation. PR 119305 patch
Extend jexec to accept hostname or ip-number besides jail-id
2008-05-26 commited to 8-CURRENT
MFC after: 2 weeks
based on FreeBSD 7.x

Set priority in rc.d

done / not commited

Jan Srzednicki

A simple rc.d jail patch to enable priority patch
Added documentation PR 124248 patch
based on FreeBSD 7.x

Do not keep JID incrementing forever

done / commited / reverted

Ed Schouten

Jail numbers are incrementing forever, this makes jls/ps output a lot easier. No more 4-5 digit numbers in your test setup, where you only have 10-20 jails. PR 122270
2008-04-11 commited to 8-CURRENT
2008-05-12 MFC commited to RELENG_7
based on FreeBSD 8.x

Per prison process count limit

done / stalled

Alex Lyashkov

Done as part of FreeVPS
based on FreeBSD 6.x

Separated UID hash

done / stalled

Alex Lyashkov

Done as part of FreeVPS
based on FreeBSD 6.x

Separated SysV IPC

done / stalled

Alex Lyashkov

Done as part of FreeVPS
based on FreeBSD 6.x

File handles usage limit

done / stalled

Alex Lyashkov

Done as part of FreeVPS
based on FreeBSD 6.x

Hierarchical jails

work in progress

Jamie Gritton

Hierarchical jails under the new framework (jails inside jail)
jh.diff in mailinglist
jhu.diff in mailinglist (user side)
questions and example use
based on FreeBSD 8.x

Future plans for FreeBSD jails / virtualization

If you have any idea or request, please address the freebsd-jail maililng list.

freebsd-arch mailing list discussions:

Well known or frequent requests, some of these will never be done with jails, some are (or will be) done with alternative technologies:

Netgraph-based networking for jails

Introduction

A few links to other resources:

There are three ways to do networking with jails: (i) using IP addresses (aliases) from the host, (ii) using if_bridge(4) and epair(4) interfaces, and (iii) using netgraph. I will start this how-to with the third method as the second method has already been explained by Mason Loring Bliss.

Before we kick off, we need to understand some new terms that netgraph introduces.

Node
Any object that can connect to another object. This can be a bridge, a virtual interface, but also a VLAN interface or a BPF object.
Edge
The connection between two nodes. You could consider it to be a virtual network cable.
Hook

In netgraph, edges do not really exist per se. Instead, an edge is simply an association of two hooks, one from each node. A node's hooks define how that node can be connected. For a bridge node, the hooks are the virtual interfaces, though for a virtual interface node, hooks can be each protocol supported by that interface.

Path

Every netgraph node is addressable via an ASCII string called a node address or path. Many nodes have names. When a node has a name, it can always be addressed usingthe absolute address consisting of the name followed by a colon. If a node does not have a name, you can construct one from the node's unique ID number by enclosing the number in square brackets. Relative addressing is also possible when two nodes are indirectly connected. A relative address uses the names of consecutive hooks on the path from the source node to the target node. Relative and absolute addressing can be combined.

This is quite confusing. See the example at https://people.freebsd.org/~julian/netgraph.html.

Let's start off with a small network consisting of two jails, called "test1" and "test2", which are bridged to an new, virtual interface on the host machine. We thus need three new virtual interfaces. We will use ngctl to manage all netgraph nodes, hooks, and edges. This tool can be used interactively or you can supply the command on the command line. Below we will use it interactively. Note that this command must be run as the root user.

# ngctl

mkpeer eiface ether ether
mkpeer eiface ether ether
mkpeer eiface ether ether
name ngeth0: ng_host
name ngeth1: ng_test1
name ngeth2: ng_test2

We create three interfaces of type eiface as we need Ethernet interfaces (as opposed to L3 interfaces when using type iface). The man page state they are named ngeth0, etc. We rename them to indicate their purpose.

Note that when referencing a node name, we must append it with a colon. That is because we're specifying a path, not just a node.

Next we need to bridge these interfaces together. We can either create the bridge first and then connect it to the three interfaces, or while creating the bridge, we can already connect it to an interface.

/*
 * OPTION A:
 * To only create the bridge but not connect it to any existing nodes
 */
mkpeer bridge link0 link0

/*
 * OPTION B:
 * Create a new node of type bridge and connect it to the
 * path 'ng_host:'.  Use its hook called 'ether' to connect it to the
 * hook 'link0' of the new node (the bridge).
 */
mkpeer ng_host: bridge ether link0

/*
 * As opposed to the Ethernet interfaces, a bridge does not have a
 * name by default. So we name it here.
 */
name [1e]: br0

/*
 * Now connect the nodes together.  The first connection is only needed
 * when option A was used.
 */
connect br0: ng_host: link0 ether
connect br0: ng_test1: link1 ether
connect br0: ng_test2: link2 ether

The above configuration is pretty straightforward, the exception being the name command. Since the bridge does not have a name, we can't use it to construct a path. Instead we must use its unique ID (between brackets). To determine this ID, we need the list command which I ommitted from the configuration above. Let's show it here.

+ list
There are 6 total nodes:
  Name: ng_host         Type: eiface          ID: 00000007   Num hooks: 1
  Name: ng_test1        Type: eiface          ID: 00000009   Num hooks: 0
  Name: ngctl85802      Type: socket          ID: 0000002b   Num hooks: 0
  Name: <unnamed>       Type: bridge          ID: 0000001e   Num hooks: 1
  Name: ng_test2        Type: eiface          ID: 0000000e   Num hooks: 0
+

The above output shows the unique ID for the bridge (preceding zeros can be ommitted) as well as the number of hooks used, i.e. connections made. I thus created the bridge by directly connecting it to the ng_host interface.

Configuring the jails

I assume you know how to setup a jail, i.e. create a base jail, clone it, etc. If not, please read the Jails chapter of the FreeBSD Handbook.

The below configuration only includes the networking-specific configuration for the two jails. Although shown per jail, you can easily move these two lines to the default configuration using the variable $name to customize the interface name.

test1 {
   vnet;
   vnet.interface = ng_test1;
}
test2 {
   vnet;
   vnet.interface = ng_test2;
}

Once the jails have been started, you can manually configure IP addresses on their interfaces, as well as on the host interface ng_host.

NAT to be covered in the future.

Known issues

Tips

leak of system message buffer from host system to jails

By default jails can read the system message buffer (ie: the console buffer). In many cases this isn't desired. Add the following line to /etc/sysctl.conf to turn it off: security.bsd.unprivileged_read_msgbuf=0

allow ping inside jail

If you need to use ping inside jail, you must set sysctl security.jail.allow_raw_sockets=1 (add security.jail.allow_raw_sockets=1 in hosts sysctl.conf).

installing 32-bit jail in to 64-bit host

Cross compiling FreeBSD and 32-bit jails on a 64-bit system

Running CentOS 5.5 in a jail

FreeBSD Wiki Centos 5.5 jail running on a host running FreeBSD 8-STABLE

Running Debian 6.0 (Squeeze) in a FreeBSD jail

Tutorial: Debian GNU/kFreeBSD in a FreeBSD jail
Can I run Debian GNU/kFreeBSD in a chroot under FreeBSD?

Jail management

Some of these utilities are usable only with ancient versions of FreeBSD (4.x / 5.x).

bastille
Pure shell system for automating deployment and management of containerized applications in jails

sysutils/bastille

BSDploy
Container management and orchestration for FreeBSD jails using ansible

sysutils/bsdploy

cbsd
FreeBSD jail and bhyve management utility

sysutils/cbsd

crate
An application containerizer utilizing jails

sysutils/crate

ezjail
A framework to easily create, manipulate and run FreeBSD jails

sysutils/ezjail

focker

A jail-based image orchestration tool in the vein of Docker

sysutils/py-focker

iocage
Utility for managing jails

sysutils/iocage

iocell

iocage fork focused on supporting classic iocage experience and adding fixes and features requested by the iocage users

sysutils/iocell

jadm
FreeBSD jail framework with zfs, vnet and jail.conf support

sysutils/jadm

jailadmin
A system for managing a set of named jails

sysutils/jailadmin

jailaudit
Script for generating portaudit reports for jails

ports-mgmt/jailaudit

jailctl
Jail management tool

sysutils/jailctl

jailer
Manage FreeBSD jail startup, shutdown and console

sysutils/jailer (This port expired on: 2014-03-10)

jailme
A setuid version of jexec to allow normal users access to jails

sysutils/jailme

jailrc
Improved jail startup/shutdown script

sysutils/jailrc

jailutils
Several utilities for managing jails

sysutils/jailutils

jest
REST API for creating and managing jails

sysutils/jest

jkill
Shutdowns a running jail and all its processes

sysutils/jkill

jps
Wrapper to ps(1) that maps pids to jails

sysutils/jps

jruls
A top-alike utility to display and update FreeBSD jails resource usage

sysutils/jruls

jtop
Wrapper to top(1) that maps pids to jails

sysutils/jtop

mkjail
Easy jail creation, updates and upgrades

sysutils/mkjail

mod_jail
Apache 1.3.x/2.0.xx module to enable an easy alternative to mod_chroot

www/mod_jail (This port expired on: 2017-04-30)

p5-BSD-Jail-Object
An object oriented perl interface to jail(2)

sysutils/p5-BSD-Jail-Object

ploy
Command-line tool to provision, manage, and control servers, including jails

sysutils/py-ploy

pot
Container framework for FreeBSD, based on jails, ZFS and pf

sysutils/pot

qjail
Utility to deploy large number of jails quickly

sysutils/qjail

quickjail
utility to create transient single-command jails

sysutils/quickjail

runj
Proof-of-concept OCI-compatible runtime for FreeBSD jails

sysutils/runj

Other resources


CategorySecurity CategoryHowTo CategoryNeedsContent CategoryStale

Jails (last edited 2023-01-29T14:07:22+0000 by GrahamPerrin)