Installing FreeBSD Root on ZFS (Mirror) using GPT
1. Creating a bootable ZFS Filesystem
- Boot FreeBSD install DVD or USB Memstick
- Choose Fixit option in sysinstall
- Create GPT Disks
Fixit# gpart create -s gpt ad0 Fixit# gpart create -s gpt ad1
- Create the boot, swap and zfs partitions
Create 3 partitions on both drives ad0 and ad1. The first partition contains the gptzfsboot loader which is able to recognize and load the loader from a ZFS partition. The second partition is a 4 GB swap partition. The third partition is the partition containing the zpool (60GB).
Fixit# gpart add -b 34 -s 512k -t freebsd-boot ad0 Fixit# gpart add -s 4G -t freebsd-swap -l swap0 ad0 Fixit# gpart add -s 60G -t freebsd-zfs -l disk0 ad0
Fixit# gpart add -b 34 -s 512k -t freebsd-boot ad1 Fixit# gpart add -s 4G -t freebsd-swap -l swap1 ad1 Fixit# gpart add -s 60G -t freebsd-zfs -l disk1 ad1
Note:
While a ZFS Swap Volume can be used instead of the freebsd-swap partition, crash dumps can't be created on the ZFS Swap Volume.
- Sizes and offsets are specified in sectors by default (1 sector is typically 512 bytes) but can have 'M' or 'G' suffixes for MB and GB.
- Install the Protective MBR (pmbr) and gptzfsboot loader to both drives
Fixit# gpart bootcode -b /mnt2/boot/pmbr -p /mnt2/boot/gptzfsboot -i 1 ad0 Fixit# gpart bootcode -b /mnt2/boot/pmbr -p /mnt2/boot/gptzfsboot -i 1 ad1
This may fail with an "operation not permitted" error message, since the kernel likes to protect critical parts of the disk. If this happens for you, run:Fixit# sysctl kern.geom.debugflags=0x10
- Load ZFS kernel module
Fixit# kldload /mnt2/boot/kernel/opensolaris.ko Fixit# kldload /mnt2/boot/kernel/zfs.ko
- Create ZFS Pool zroot
Fixit# mkdir /boot/zfs Fixit# zpool create zroot mirror /dev/gpt/disk0 /dev/gpt/disk1 Fixit# zpool set bootfs=zroot zroot
Note:
- zroot is the name of the ZFS Pool, it could be anything (i.e. tank, data,...)
2. Installing FreeBSD to the ZFS filesystem
Create ZFS filesystem hierarchy
The fletcher4 algorithm should be more robust than the fletcher2 algorithm.
Fixit# zfs set checksum=fletcher4 zroot
Fixit# zfs create -o compression=on -o exec=on -o setuid=off zroot/tmp Fixit# chmod 1777 /zroot/tmp
Fixit# zfs create zroot/usr Fixit# zfs create zroot/usr/home Fixit# cd /zroot ; ln -s /usr/home home
Fixit# zfs create -o compression=lzjb -o setuid=off zroot/usr/ports Fixit# zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/distfiles Fixit# zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/packages
Note:
If you use nullfs or nfs to mount /usr/ports to different locations/servers, you will also need to nullfs/nfs mount /usr/ports/distfiles and/or /usr/ports/packages.
Fixit# zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/usr/src
Fixit# zfs create zroot/var Fixit# zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/crash Fixit# zfs create -o exec=off -o setuid=off zroot/var/db Fixit# zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/db/pkg Fixit# zfs create -o exec=off -o setuid=off zroot/var/empty Fixit# zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/log Fixit# zfs create -o compression=gzip -o exec=off -o setuid=off zroot/var/mail Fixit# zfs create -o exec=off -o setuid=off zroot/var/run Fixit# zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/tmp Fixit# chmod 1777 /zroot/var/tmp
Note:
- Compression may be set to on, off, lzjb, gzip, gzip-N (where N is an integer from 1 (fastest) to 9 (best compression ratio. gzip is equivalent to gzip-6).
Compression will cause some latency when accessing files on the ZFS filesystems. Use compression on ZFS filesystems which will not be accessed that often.
- Install FreeBSD to zroot
Fixit# cd /dist/8.0-* Fixit# export DESTDIR=/zroot Fixit# for dir in base catpages dict doc games info lib32 manpages ports; \ do (cd $dir ; ./install.sh) ; done Fixit# cd src ; ./install.sh all Fixit# cd ../kernels ; ./install.sh generic Fixit# cd /zroot/boot ; cp -Rlp GENERIC/* /zroot/boot/kernel/ - Make /var/empty readonly
Fixit# zfs set readonly=on zroot/var/empty
- chroot into /zroot
Fixit# chroot /zroot
- Create /etc/rc.conf
Fixit# echo 'zfs_enable="YES"' > /etc/rc.conf Fixit# echo 'hostname="beastie.mydomain.local"' >> /etc/rc.conf Fixit# echo 'ifconfig_re0="DHCP"' >> /etc/rc.conf
Note:
- Replace re0 with the name of the Network interface for the new system
- Create /boot/loader.conf
Fixit# echo 'zfs_load="YES"' > /boot/loader.conf Fixit# echo 'vfs.root.mountfrom="zfs:zroot"' >> /boot/loader.conf
- Change root's password
Fixit# passwd
- Set the local time zone
Fixit# tzsetup
- Create /etc/mail/aliases.db
Fixit# cd /etc/mail Fixit# make aliases
- Exit from the /zroot
Fixit# umount /dev Fixit# exit
- Install zpool.cache to the ZFS filesystem
Fixit# cp /boot/zfs/zpool.cache /zroot/boot/zfs/zpool.cache
3. Finish install
Using swap
There are 2 ways to use the gpt/swap0 and gpt/swap1 partitions
- Create /etc/fstab to use both swap partitions
Fixit# cat << EOF > /zroot/etc/fstab # Device Mountpoint FStype Options Dump Pass# /dev/gpt/swap0 none swap sw 0 0 /dev/gpt/swap1 none swap sw 0 0 EOF
Use gmirror to mirror the swap partitions
- Load gmirror kernel module
Fixit# kldload /mnt2/boot/kernel/geom_mirror.ko
- Create the gmirror swap
Fixit# gmirror label -b prefer swap gpt/swap0 gpt/swap1
Note:
The 'prefer' balance algorithm can be replaced by 'round-robin'. See the gmirror(8) man page about problem using the 'round-robin' balance algorithm and kernel dumps
- Create /etc/fstab to use the gmirror swap partition
Fixit# cat << EOF > /zroot/etc/fstab # Device Mountpoint FStype Options Dump Pass# /dev/mirror/swap none swap sw 0 0 EOF
- Add gmirror to /boot/loader.conf
Fixit# echo 'geom_mirror_load="YES"' >> /zroot/boot/loader.conf
- Load gmirror kernel module
- Create /etc/fstab to use both swap partitions
- export LD_LIBRARY_PATH
Fixit# export LD_LIBRARY_PATH=/mnt2/lib
- Unmount all zfs filesystems
Fixit# zfs unmount -a
- Change mount points for zroot pool
Fixit# zfs set mountpoint=legacy zroot Fixit# zfs set mountpoint=/tmp zroot/tmp Fixit# zfs set mountpoint=/usr zroot/usr Fixit# zfs set mountpoint=/var zroot/var
- Exit Fixit mode and sysinstall. Remove the FreeBSD install DVD/Memstick and the system will boot using the ZFS root.