Warning - the following instructions will ravage your machine and then eat all the food in your refrigerator, and unlike the dwarves, they won't be kidding about breaking your dishes.
This example assumes FreeBSD 11 or thereabouts, UEFI, a pair of disks, and a desire to mirror everything, including the EFI System Partition.
Unlike bsdinstall's default for ZFS, it doesn't set explicit mountpoints for datasets, but instead relies on inheritance.
A notable trick is the use of gmirror for the EFI boot partition, a notion gratefully lifted from CentOS. Caveat: Some UEFI firmwares have been known to write files to the ESP. While uncommon, this would result in your ESP mirror being inconsistent. Setting the mirror balance mode to "prefer" and causing a resync at boot might help work around this. (Details forthcoming.)
This variation of the instructions if for FreeBSD 11, where the UEFI bootloader doesn't know about both ZFS and GELI. For a newer example for FreeBSD 12, where loader.efi does understand both GELI and ZFS, see: UEFIandZFSandGELIbyHAND12.
For the legacy boot equivalent, see: LegacyZFSandGELI.
# We're clearing off the disks to start. This is a destructive operation. gpart destroy -F ada0 gpart destroy -F ada1 gpart create -s gpt ada0 gpart create -s gpt ada1 # Add partitions. We'll access them by name everywhere. gpart add -t efi -s 1m -l efi0 ada0 gpart add -t efi -s 1m -l efi1 ada1 gpart add -t freebsd-zfs -l boot0 -a 1m -s 512m ada0 gpart add -t freebsd-zfs -l boot1 -a 1m -s 512m ada1 # If you're not going to use a ZVOL for swap, then: gpart add -t freebsd-swap -l swap0 -a 1m -s 8192m ada0 gpart add -t freebsd-swap -l swap1 -a 1m -s 8192m ada1 # gpart add -t freebsd-zfs -l tank0 -a 1m ada0 gpart add -t freebsd-zfs -l tank1 -a 1m ada1 # Set up gmirrors kldload geom_mirror gmirror label -v -b prefer efi gpt/efi0 gpt/efi1 gmirror label -v swap gpt/swap0 gpt/swap1 newfs_msdos /dev/mirror/efi # There's a bug in GPT label handling. If you run into it, unrelated labels # will disappear as others are consumed by gmirror. For instance, in this set, # after consuming gpt/efi0 and gpt/efi1, I might find that gpt/swap1 has gone # missing. If you hit this, you can set up your mirrors as /dev/ada0p1 rather # than gpt/efi0, using this guide as an example. BZ#230246 # Set up GELI geli init -s 4096 -b -g gpt/tank0 geli init -s 4096 -b -g gpt/tank1 geli attach gpt/tank0 geli attach gpt/tank1 # Create the root pool. zpool create -R /mnt -o cachefile=/tmp/zpool.cache -O mountpoint=/ \ -O atime=off -O canmount=off -O compression=on \ tank mirror gpt/tank0.eli gpt/tank1.eli zfs create -o canmount=off -o mountpoint=none tank/ROOT zfs create -o mountpoint=/ tank/ROOT/default zfs create tank/home zfs create -o canmount=off tank/usr zfs create tank/usr/jails zfs create tank/usr/local zfs create tank/usr/obj zfs create tank/usr/src zfs create tank/usr/ports zfs create tank/usr/ports/distfiles zfs create -o canmount=off tank/var zfs create tank/var/log zfs create tank/var/tmp zfs create tank/tmp # If you want swap on a zvol, look up related tuning as well. # E.g., https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=199189 # and https://wiki.freebsd.org/AndriyGapon/AvgPageoutAlgorithm # Note that /etc/rc.d/zvol will notice the org.freebsd:swap property and # activate this automatically on boot. # As of my testing with 11.1, the system will eventually lock on overfill, # so I don't recommend swap on a zvol as of 11.1. # zfs create -o org.freebsd:swap=on -V 8G tank/swap # Create the boot pool. zpool create -R /mnt -o cachefile=/tmp/zpool.cache -O mountpoint=/zboot \ -O atime=off -O compression=on boot mirror gpt/boot0 gpt/boot1 # Copy over the zpool cache so we'll have it on the running system. cp /tmp/zpool.cache /mnt/zboot # Personal preference - I don't like /usr/home. ln -s /home /mnt/usr/home # Don't forget to take the option of a shell to finish configuration at the # end of the install. exit
The install will proceed.
When offered a post-install shell, take it, and:
# The installer doesn't know about /zboot, so we'll copy over /boot, and put # the cache file in place. mv /boot /zboot ln -s /zboot/boot mv /zboot/zpool.cache /boot/zfs cat > /boot/loader.conf <<END aesni_load="YES" zfs_load="YES" geom_eli_load="YES" geom_eli_passphrase_prompt="YES" geom_mirror_load="YES" zpool_cache_load="YES" zpool_cache_name="/boot/zfs/zpool.cache" zpool_cache_type="/boot/zfs/zpool.cache" vfs.root.mountfrom="zfs:tank/ROOT/default" END cat >> /etc/rc.conf <<END zfs_enable="YES" END cat > /etc/fstab <<END /dev/mirror/efi /boot/efi msdosfs rw,late 0 0 # If you're not using swap on a zvol: /dev/mirror/swap.eli none swap sw 0 0 END # If /dev/ has been depopulated - I see this under 11+ rm /dev/null mount -t devfs devfs /dev # Historically, we didn't manipulate EFI variables. Now we can, although it's # buggy. I'll present both options. I've got one box that doesn't honor the # default \EFI\BOOT\BOOTX64.EFI, hence my exploration. # Preparing for UEFI boot. mkdir /boot/efi mount -t msdosfs /dev/mirror/efi /boot/efi # If we have a box that's happy with the fallback naming, we can just do that. mkdir -p /boot/efi/efi/boot cp /boot/boot1.efi /boot/efi/efi/boot/bootx64.efi # If we want to set entries up with the FreeBSD efibootmgr program, as we can # do that too. kldload efirt mkdir -p /boot/efi/efi/freebsd cp /boot/boot1.efi /boot/efi/efi/freebsd/ efibootmgr -c -L freebsd0 -l ada0p1:/efi/freebsd/boot1.efi efibootmgr -c -L freebsd1 -l ada1p1:/efi/freebsd/boot1.efi # Obviously check to see what numbers were actually assigned first. efibootmgr -a 1 efibootmgr -a 2 efibootmgr -o 1,2 # Note that the ordering command seems not to work, at least on some hardware. # You might be forced to set boot order manually on first boot. Also note that # the documentation is incorrect and shows the Linux tool's syntax. BZ#230871 # Note that activation, implicit in Linux, is explicit here. I've not yet tried # including -a as shown above - I've done it separately. I believe this will # work. If it doesn't, run "efibootmgr -a ..." for each of the boot menu entries # after "efibootmgr -c ...", as a separate step. # You should be good to go at this point. Variations on this can include # setting up keys rather than passphrases for GELI. Also, note that you # can create "degraded arrays" in gmirror by only specifying one device. # This can be useful for initial set-up work where you intend to later # add the other half of a mirror. exit