• Containers

FreeBSD Containers and Orchestration

FreeBSD introduced its container, OS-level virtualization primitive in 1999 in the form of a security-oriented isolation framework and subsystem called Jails. Similar to OpenVZ Containers in 2005, Solaris Zones, LXC, Docker and other implementations, FreeBSD Jails allow isolation of applications or entire stacks with their own processes, filesystems and users, whilst using the same host operating system kernel.

Container Tools: Base System

Out of the box, FreeBSD provides:

• jail command line utility

• jail.conf: configuration file for jails

• rc.conf integration for jail management

Container Tools: Third-Party

These third-party tools aim to simplify and speed up the process of creating and managing FreeBSD jail-based containers on individual hosts, and many include additional support and integration for FreeBSD features such as bhyve virtualization, ZFS, Virtual Networks (VNET), Templating, Import/Export among others.

AppJail

Jail framework written in C and posix shell to create, deploy and maintain jail-based containers.

• AppJail GitHub Repository

• Makejails

• Wiki

• AppJail FreeBSD Port/Package: sysutils/appjail

• AppJail (devel) FreeBSD Port/Package: sysutils/appjail-devel

Supports:

• Supervisor (healthcheckers)

• Parallel startup (Healthcheckers, jails & NAT).

• ZFS support.
• RACCT/RCTL support.
• NAT support.
• Port forwarding.
• IPv4 and IPv6 support.
• DHCP and SLAAC support.
• Virtual networks.
• Bridge support.
• VNET support.
• Text file to make jails: Makejail.
• Netgraph support.

• LinuxJails support.

• Supports thin and thick jails.

• TinyJails - Experimental feature to create a very stripped down jail that is very useful to distribute.

• Startup order control.
• Jail dependency support.

• InitScript for interactive use of jails.

• Commands to import/export jails (ZFS & Tarballs).

• Table interface to easily integrate AppJail with scripts.

• Images.
• Dynamic DEVFS ruleset management.

• Orchestration with AppJail Director.

podman, buildah

A FreeBSD port of the https://github.com/containers stack. Install sysutils/podman-suite from FreeBSD Ports/Packages.

• Suitable for testing and evaluation.
• Container storage using the zfs and vfs storage drivers. ZFS is strongly preferred since its use of snapshots and clones makes it more efficient than vfs.
• Podman provides a CLI which is a drop-in replacement for docker.

• Supports docker-style networking using a port of https://github.com/containernetworking/plugins.

• Container images use the same formats and infrastructure as containerd and can be shared between the two implementations.

runj / containerd / nerdctl

Experimental FreeBSD Jail execution runtime implementation and FreeBSD OCI specification development.

• First Release: 2021¹

• Latest Release: 2022 (No release tags)
• nerdctl provides a Docker-compatible CLI for containerd

Supports:

• containerd support²

• Linux Jail support

• Network Support (Experimental)

• runj FreeBSD Port/Package: sysutils/runj

• runj GitHub Repository

• nerdctl FreeBSD Port/Package: sysutils/nerdctl

• nerdctl GitHub Repository

Bastille

Command-line (shell) tool and automation framework for jail-based containers.

• First Release: 2018 ³

• Latest Release: 2022 (0.9.2022*)

Supports:

• FreeBSD Features: ZFS, VNET
• Other Features: Template Creation, Import, Export
• Has a container / template registry

• BastilleBSD Homepage

• Bastille Port/Package: sysutils/bastille

• Bastille GitHub Repository

pot

• First Release: 2018 ⁴

• Latest Release: 2022 (0.15.3)

Command-line (shell) tool for jail-based containers.

• pot GitHub Repository

• pot Port/Package: sysutils/pot

Supports:

• FreeBSD Features: ZFS, pf, rctl.

• Orchestration with HashiCorp Nomad ⁵

cbsd

Command-line and TUI (shell) tool for jail-based containers.

• First Release: ~2013 ⁶

• Last Release: 2022 (13.1.4)

Supports:

• FreeBSD Features: ZFS, VNET, bhyve, Xen

• Other Features: Templates and Profiles

• cbsd homepage

• cbsd Port/Package: sysutils/cbsd

• CBSD GitHub Repository

None of the above orchestration tools have a "run a plain vanilla linux inside the container" feature, but the start/stop/templating feature of them does not prevent a manually created plain-vanilla-linux-jail.

iocage

Command-line (Python, originally shell) tool for jail-based containers.

• First Release: ~2016 ⁷

• Latest Release: 2019 (1.2)

Supports:

• FreeBSD Features: ZFS (required), VNET
• Other Features: Template Creation, Import, Export

• iocage Homepage

• iocage Port/Package: sysutils/iocage

ioc

Re-implementation (Python) of iocage.

• First Release: Unknown, but ~2017 ⁷

• Latest Release: 2019 (0.8.2)

• ioc GitHub Repository

• ioc Port/Package:sysutils/ioc

iocell

Fork of original (shell) iocage

• First Release: ~2016 ("v2.0.0") ⁷

• Latest Release: 2017 (2.1.2)

• iocell GitHub Repository

• iocell Port/Package: sysutils/iocell

ezjail

Command-line (shell) tool for jail-based containers.

• First Release: 2005 ⁸

• Latest Release: 2015 (3.4.2)

Supports:

• FreeBSD Features: ZFS, VNET (via manual scripting)
• Other Features: Template Creation, Import, Export

• ezjail Homepage

• ezjail Port/Package: sysutils/ezjail

• ezjail Git Repository (gitweb)

Container Orchestration

The container tools above can be used as a base for multi-host orchestration, by exporting containers from one host, and either moving the container to another host, or by utilising shared storage (eg: SAN) or ZFS to switch container datasets from one host to another and importing it on the target host.

Additionally the following FreeBSD container orchestration tools are available:

• HashiCorp Nomad (jail-task driver)

• Nomad Port/Package: sysutils/nomad

Container Orchestration Distributions

XXX: add some FreeBSD derived distributions which have their focus (not only) on container virtualisation.

• CloneOS - WebUI for bhyve, jails, xen, ...