Installing FreeBSD Root on ZFS using FreeBSD-ZFS partition in a FreeBSD MBR Slice

Contents

Creating a bootable ZFS Filesystem
Installing FreeBSD to the ZFS filesystem
Finish install

1. Creating a bootable ZFS Filesystem

Boot FreeBSD install DVD or USB Memstick
Choose Fixit option in sysinstall
Create MBR disk

If you are installing on an New drive, you will need to create the MBR geometry first:
```
 Fixit# gpart create -s mbr ad0
```

Create Slice using gpart

This example already has two slices defined on the drive.

 Fixit# gpart show ad0
 =>       63  625142385  ad0  MBR  (289G)
          63  209712447    1  !7  (100G)
   209712510     417690    2  !136  (204M)
   210130200  415012248       - free -  (198G)

Create a third slice - gpart will use all the remaining space by default. Then create a BSD partition table on the new slice.

 Fixit# gpart add -t freebsd ad0
 ad0s3 added
 Fixit# gpart create -s BSD ad0s3
 ad0s3 created

And examine the result:

 Fixit# gpart show ad0
 =>       63  625142385  ad0  MBR  (289G)
          63  209712447    1  !7  (100G)
   209712510     417690    2  !136  (204M)
   210130200  415012248    3  freebsd  (198G)

Note:

Sizes and offsets are specified in sectors by default (1 sector is typically 512 bytes) but can have an 'M' and 'G' suffix to use MB or GB.

Set partition active (Optional)

Some BIOSes might require setting a Slice active. Here we set partition 1 (the Windows Partiton) as the active partition.
```
 Fixit# gpart set -a active -i 1 ad0
```

Create partitions (ad0s3a and ad0s3b)

Note that partition order is important. It seems that zfsboot requires the freebsd-zfs partition to be the first, so make sure you add if first, before your swap partition.

 Fixit# gpart show ad0s3
 =>        0  415012248  ad0s3  BSD  (198G)
           0  415012248  ad0s3  - free -  (198G)

 Fixit# gpart add -s 194G -t freebsd-zfs ad0s3
 ad0s3a added

 Fixit# gpart show ad0s3
 =>        0  415012248  ad0s3  BSD  (198G)
   406631421          1         freebsd-zfs  (194G)
   406631421    8380827         - free -  (4.0G)

 Fixit# gpart add -t freebsd-swap ad0s3
 ad0s3b added

 Fixit# gpart show ad0s3
 =>        0  415012248  ad0s3  BSD  (198G)
           0  406631421      1  freebsd-zfs  (194G)
   406631421    8380827      2  freebsd-swap  (4.0G)

Note:

While a ZFS Swap Volume can be used instead of the freebsd-swap partition, crash dumps can't be created on the ZFS Swap Volume.
Sizes and offsets are specified in sectors by default (1 sector is typically 512 bytes) but can have an 'M' or 'G' suffix to use MB or GB.

Load ZFS kernel module

 Fixit# kldload /mnt2/boot/kernel/opensolaris.ko
 Fixit# kldload /mnt2/boot/kernel/zfs.ko

Create ZFS Pool zroot

 Fixit# mkdir /boot/zfs
 Fixit# zpool create zroot /dev/ad0s3a
 Fixit# zpool set bootfs=zroot zroot

Note:

zroot is the name of the ZFS Pool, it could be anything (i.e. tank, data, ...)

Install boot Manager
```
 Fixit# gpart bootcode -b /mnt2/boot/boot0 ad0
 ad0 has bootcode
```
It's ok if you get an "unknown class" message here, but it you get "failed to install" or the like, you'll need to try again after making sure that nothing is mounted from /dev/ad0.
Install ZFS boot

Export zroot, before installing boot code
```
 Fixit# zpool export zroot
```
Install the boot1 stage:
```
 Fixit# dd if=/mnt2/boot/zfsboot of=/tmp/zfsboot1 count=1
 Fixit# gpart bootcode -b /tmp/zfsboot1 /dev/ad0s3
```
This may fail with an "operation not permitted" error message, since the kernel likes to protect critical parts of the disk. If this happens for you, run:
```
 Fixit# sysctl kern.geom.debugflags=0x10
```
Install the boot2 zfs stage into the convienient hole in the ZFS filesystem on-disk format which is located just after the ZFS metadata (this is the seek=1024).
```
 Fixit# dd if=/mnt2/boot/zfsboot of=/dev/ad0s3a skip=1 seek=1024
```
Import zroot to continue the install
```
 Fixit# zpool import zroot
```

2. Installing FreeBSD to the ZFS filesystem

Create ZFS filesystem hierarchy

The fletcher4 algorithm should be more robust than the fletcher2 algorithm.

 Fixit# zfs set checksum=fletcher4                                      zroot

 Fixit# zfs create -o compression=on    -o exec=on      -o setuid=off   zroot/tmp
 Fixit# chmod 1777 /zroot/tmp

 Fixit# zfs create                                                      zroot/usr
 Fixit# zfs create                                                      zroot/usr/home
 Fixit# cd /zroot ; ln -s /usr/home home

 Fixit# zfs create -o compression=lzjb                  -o setuid=off   zroot/usr/ports
 Fixit# zfs create -o compression=off   -o exec=off     -o setuid=off   zroot/usr/ports/distfiles
 Fixit# zfs create -o compression=off   -o exec=off     -o setuid=off   zroot/usr/ports/packages

Note:

If you use nullfs or nfs to mount /usr/ports to different locations/servers, you will also need to nullfs/nfs mount /usr/ports/distfiles and/or /usr/ports/packages.

 Fixit# zfs create -o compression=lzjb  -o exec=off     -o setuid=off   zroot/usr/src

 Fixit# zfs create                                                      zroot/var
 Fixit# zfs create -o compression=lzjb  -o exec=off     -o setuid=off   zroot/var/crash
 Fixit# zfs create                      -o exec=off     -o setuid=off   zroot/var/db
 Fixit# zfs create -o compression=lzjb  -o exec=on      -o setuid=off   zroot/var/db/pkg
 Fixit# zfs create                      -o exec=off     -o setuid=off   zroot/var/empty
 Fixit# zfs create -o compression=lzjb  -o exec=off     -o setuid=off   zroot/var/log
 Fixit# zfs create -o compression=gzip  -o exec=off     -o setuid=off   zroot/var/mail
 Fixit# zfs create                      -o exec=off     -o setuid=off   zroot/var/run
 Fixit# zfs create -o compression=lzjb  -o exec=on      -o setuid=off   zroot/var/tmp
 Fixit# chmod 1777 /zroot/var/tmp

Note:

Compression may be set to on, off, lzjb, gzip, gzip-N (where N is an integer from 1 (fastest) to 9 (best compression ratio. gzip is equivalent to gzip-6).
Compression will cause some latency when accessing files on the ZFS filesystems. Use compression on ZFS filesystems which will not be accessed that often.

Install FreeBSD to zroot

 Fixit# cd /dist/8.0-*
 Fixit# export DESTDIR=/zroot
 Fixit# for dir in base catpages dict doc games info lib32 manpages ports; \
          do (cd $dir ; ./install.sh) ; done
 Fixit# cd src ; ./install.sh all
 Fixit# cd ../kernels ; ./install.sh generic
 Fixit# cd /zroot/boot ; cp -Rlp GENERIC/* /zroot/boot/kernel/

Make /var/empty readonly

 Fixit# zfs set readonly=on zroot/var/empty

chroot into /zroot
```
 Fixit# chroot /zroot
```

Create /etc/rc.conf

 Fixit# echo 'zfs_enable="YES"' > /etc/rc.conf
 Fixit# echo 'hostname="beastie.mydomain.local"' >> /etc/rc.conf
 Fixit# echo 'ifconfig_re0="DHCP"' >> /etc/rc.conf

Note:

Replace re0 with the name of the Network interface for the new system

Create /boot/loader.conf

 Fixit# echo 'zfs_load="YES"' > /boot/loader.conf
 Fixit# echo 'vfs.root.mountfrom="zfs:zroot"' >> /boot/loader.conf

Change root's password
```
 Fixit# passwd
```
Set the local time zone
```
 Fixit# tzsetup
```

Create /etc/mail/aliases.db

 Fixit# cd /etc/mail
 Fixit# make aliases

Exit from the /zroot
```
 Fixit# umount /dev
 Fixit# exit
```

Install zpool.cache to the ZFS filesystem

 Fixit# cp /boot/zfs/zpool.cache /zroot/boot/zfs/zpool.cache

3. Finish install

Create /etc/fstab

 Fixit# cat << EOF > /zroot/etc/fstab
 # Device                       Mountpoint              FStype  Options         Dump    Pass#
 /dev/ad0s3b                    none                    swap    sw              0       0
 EOF

export LD_LIBRARY_PATH

 Fixit# export LD_LIBRARY_PATH=/mnt2/lib

Unmount all zfs filesystems
```
 Fixit# zfs unmount -a
```

Change mount points for zroot pool

 Fixit# zfs set mountpoint=legacy zroot
 Fixit# zfs set mountpoint=/tmp zroot/tmp
 Fixit# zfs set mountpoint=/usr zroot/usr
 Fixit# zfs set mountpoint=/var zroot/var

Exit Fixit mode, and then sysinstall. Remove the FreeBSD install DVD/Memstick and the system will boot using the ZFS root.