Installing FreeBSD Root on ZFS (RAIDZ1) using GPT
1. Creating a bootable ZFS Filesystem
- Boot FreeBSD install DVD or USB Memstick
- Choose Fixit option in sysinstall
- Create GPT Disks
Fixit# gpart create -s gpt ad0 Fixit# gpart create -s gpt ad1 Fixit# gpart create -s gpt ad2
- Create the boot, swap and zfs partitions
Create 3 partitions on drives ad0, ad1 and ad2. The first partition contains the gptzfsboot loader which is able to recognize and load the loader from a ZFS partition. The second partition is a 4 GB swap partition. The third partition is the partition containing the zpool (60GB).
Fixit# gpart add -b 34 -s 128k -t freebsd-boot ad0 Fixit# gpart add -s 4G -t freebsd-swap -l swap0 ad0 Fixit# gpart add -s 60G -t freebsd-zfs -l disk0 ad0
Fixit# gpart add -b 34 -s 128k -t freebsd-boot ad1 Fixit# gpart add -s 4G -t freebsd-swap -l swap1 ad1 Fixit# gpart add -s 60G -t freebsd-zfs -l disk1 ad1
Fixit# gpart add -b 34 -s 128k -t freebsd-boot ad2 Fixit# gpart add -s 4G -t freebsd-swap -l swap2 ad2 Fixit# gpart add -s 60G -t freebsd-zfs -l disk2 ad2
Note:
While a ZFS Swap Volume can be used instead of the freebsd-swap partition, crash dumps can't be created on the ZFS Swap Volume.
- Sizes and offsets are specified in sectors (1 sector is typically 512 bytes) but can also be specified in MB or GB by adding an 'M' or 'G' suffix.
- Install the Protective MBR (pmbr) and gptzfsboot loader to both drives
Fixit# gpart bootcode -b /mnt2/boot/pmbr -p /mnt2/boot/gptzfsboot -i 1 ad0 Fixit# gpart bootcode -b /mnt2/boot/pmbr -p /mnt2/boot/gptzfsboot -i 1 ad1 Fixit# gpart bootcode -b /mnt2/boot/pmbr -p /mnt2/boot/gptzfsboot -i 1 ad2
This may fail with an "operation not permitted" error message, since the kernel likes to protect critical parts of the disk. If this happens for you, run:Fixit# sysctl kern.geom.debugflags=0x10
- Load ZFS kernel module
Fixit# kldload /mnt2/boot/kernel/opensolaris.ko Fixit# kldload /mnt2/boot/kernel/zfs.ko
- Create ZFS Pool zroot
Fixit# mkdir /boot/zfs Fixit# zpool create zroot raidz1 /dev/gpt/disk0 /dev/gpt/disk1 /dev/gpt/disk2 Fixit# zpool set bootfs=zroot zroot
Note:
- zroot is the name of the ZFS Pool, it could be anything (i.e. tank, data, ...)
- A raidz1 pool requires a minimum of 3 disks and a maximum of 9 disks
2. Installing FreeBSD to the ZFS filesystem
Create ZFS filesystem hierarchy
The fletcher4 algorithm should be more robust than the fletcher2 algorithm.
Fixit# zfs set checksum=fletcher4 zroot
Fixit# zfs create -o compression=on -o exec=on -o setuid=off zroot/tmp Fixit# chmod 1777 /zroot/tmp
Fixit# zfs create zroot/usr Fixit# zfs create zroot/usr/home Fixit# cd /zroot ; ln -s /usr/home home
Fixit# zfs create -o compression=lzjb -o setuid=off zroot/usr/ports Fixit# zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/distfiles Fixit# zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/packages
Note:
If you use nullfs or nfs to mount /usr/ports to different locations/servers, you will also need to nullfs/nfs mount /usr/ports/distfiles and/or /usr/ports/packages.
Fixit# zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/usr/src
Fixit# zfs create zroot/var Fixit# zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/crash Fixit# zfs create -o exec=off -o setuid=off zroot/var/db Fixit# zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/db/pkg Fixit# zfs create -o exec=off -o setuid=off zroot/var/empty Fixit# zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/log Fixit# zfs create -o compression=gzip -o exec=off -o setuid=off zroot/var/mail Fixit# zfs create -o exec=off -o setuid=off zroot/var/run Fixit# zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/tmp Fixit# chmod 1777 /zroot/var/tmp
Note:
- Compression may be set to on, off, lzjb, gzip, gzip-N (where N is an integer from 1 (fastest) to 9 (best compression ratio. gzip is equivalent to gzip-6).
Compression will cause some latency when accessing files on the ZFS filesystems. Use compression on ZFS filesystems which will not be accessed that often.
- Install FreeBSD to zroot
Fixit# cd /dist/8.0-* Fixit# export DESTDIR=/zroot Fixit# for dir in base catpages dict doc games info lib32 manpages ports; \ do (cd $dir ; ./install.sh) ; done Fixit# cd src ; ./install.sh all Fixit# cd ../kernels ; ./install.sh generic Fixit# cd /zroot/boot ; cp -Rlp GENERIC/* /zroot/boot/kernel/ - Make /var/empty readonly
Fixit# zfs set readonly=on zroot/var/empty
- chroot into /zroot
Fixit# chroot /zroot
- Create /etc/rc.conf
Fixit# echo 'zfs_enable="YES"' > /etc/rc.conf Fixit# echo 'hostname="beastie.mydomain.local"' >> /etc/rc.conf Fixit# echo 'ifconfig_re0="DHCP"' >> /etc/rc.conf
Note:
- Replace re0 with the name of the Network interface for the new system
- Create /boot/loader.conf
Fixit# echo 'zfs_load="YES"' > /boot/loader.conf Fixit# echo 'vfs.root.mountfrom="zfs:zroot"' >> /boot/loader.conf
- Change root's password
Fixit# passwd
- Set the local time zone
Fixit# tzsetup
- Create /etc/mail/aliases.db
Fixit# cd /etc/mail Fixit# make aliases
- Exit from the /zroot
Fixit# umount /dev Fixit# exit
- Install zpool.cache to the ZFS filesystem
Fixit# cp /boot/zfs/zpool.cache /zroot/boot/zfs/zpool.cache
3. Finish install
- Create /etc/fstab
Fixit# cat << EOF > /zroot/etc/fstab # Device Mountpoint FStype Options Dump Pass# /dev/gpt/swap0 none swap sw 0 0 /dev/gpt/swap1 none swap sw 0 0 /dev/gpt/swap2 none swap sw 0 0 EOF
- export LD_LIBRARY_PATH
Fixit# export LD_LIBRARY_PATH=/mnt2/lib
- Unmount all zfs filesystems
Fixit# zfs unmount -a
- Change mount points for zroot pool
Fixit# zfs set mountpoint=legacy zroot Fixit# zfs set mountpoint=/tmp zroot/tmp Fixit# zfs set mountpoint=/usr zroot/usr Fixit# zfs set mountpoint=/var zroot/var
- Exit Fixit mode, and then sysinstall. Remove the FreeBSD install DVD/Memstick and the system will boot using the ZFS root.