Root On ZFS Configuration using zfsboot (Deprecated)

Note:

  • This guide is deprecated in favor of the RootOnZFS guide

Using zfsboot to create a ZFS Root Only system on a MBR disk using a partition in a slice

1. Boot FreeBSD install CD/DVD

2. Choose Fixit option in sysinstall

3. Create Slice using gpart

If you are installing on an empty drive (e.g. a new VMWare image), you will need to create the MBR geometry first:

Fixit# gpart create -s mbr ad0

Then continue as normal:

Fixit# gpart show ad0
=>       63  625142385  ad0  MBR  (289G)
         63  209712447    1  !7  (100G)
  209712510     417690    2  !136  (204M)
  210130200  415012248       - free -  (198G)

In this example, there are already two slices on the drive. Create a third slice then create a BSD partition table on it:

Fixit# gpart add -t freebsd ad0
ad0s3 added
Fixit# gpart create -s BSD ad0s3
ad0s3 created

And examine the result:

Fixit# gpart show ad0
=>       63  625142385  ad0  MBR  (289G)
         63  209712447    1  !7  (100G)
  209712510     417690    2  !136  (204M)
  210130200  415012248    3  freebsd  (198G)

4. Create partitions (ad0s3a and ad0s3b)

Fixit# gpart show ad0s3
=>        0  415012248  ad0s3  BSD  (198G)
          0  415012248  ad0s3  - free -  (198G)

Fixit# gpart add -s 194G -t freebsd-zfs ad0s3
ad0s3a added

Fixit# gpart show ad0s3
=>        0  415012248  ad0s3  BSD  (198G)
  406631421          1         freebsd-zfs  (194G)
  406631421    8380827         - free -  (4.0G)

Fixit# gpart add -s 4G -t freebsd-swap ad0s3
ad0s3b added

Fixit# gpart show ad0s3
=>        0  415012248  ad0s3  BSD  (198G)
          0  406631421      1  freebsd-zfs  (194G)
  406631421    8380827      2  freebsd-swap  (4.0G)

5. load zfs kernel module

Fixit# kldload /mnt2/boot/kernel/opensolaris.ko
Fixit# kldload /mnt2/boot/kernel/zfs.ko

6. Create Zpool

Fixit# zpool create zroot /dev/ad0s3a
Fixit# zpool set bootfs=zroot zroot

7. Create filesystem hierarchy

Fixit# zfs create -o compression=on     -o exec=on      -o setuid=off                   zroot/tmp
Fixit# chmod 1777 zroot/tmp

Fixit# zfs create                                                                       zroot/usr
Fixit# zfs create                                                                       zroot/usr/home
Fixit# cd zroot ; ln -s /usr/home home
Fixit# zfs create -o compression=on                     -o setuid=off                   zroot/usr/ports
Fixit# zfs create -o compression=off    -o exec=off     -o setuid=off                   zroot/usr/ports/distfiles
Fixit# zfs create -o compression=off    -o exec=off     -o setuid=off                   zroot/usr/ports/packages
Fixit# zfs create -o compression=on     -o exec=off     -o setuid=off                   zroot/usr/src

Fixit# zfs create                                                                       zroot/var
Fixit# zfs create -o compression=on     -o exec=off     -o setuid=off                   zroot/var/crash
Fixit# zfs create                       -o exec=off     -o setuid=off                   zroot/var/db
Fixit# zfs create -o compression=on     -o exec=on      -o setuid=off                   zroot/var/db/pkg
Fixit# zfs create                       -o exec=off     -o setuid=off   -o readonly=on  zroot/var/empty
Fixit# zfs create -o compression=on     -o exec=off     -o setuid=off                   zroot/var/log
Fixit# zfs create -o compression=on     -o exec=off     -o setuid=off                   zroot/var/mail
Fixit# zfs create                       -o exec=off     -o setuid=off                   zroot/var/run
Fixit# zfs create -o compression=on     -o exec=on      -o setuid=off                   zroot/var/tmp
Fixit# chmod 1777 zroot/var/tmp

NOTES:

  1. compression may be set to on, off, lzjb, gzip, gzip-N (where N is an integer from 1 (fastest) to 9 (best compresion ratio. gzip is equivalent to gzip-6).
  2. Compression will cause some latency when accessing files on the ZFS filesystems. Use compression on ZFS filesystems which will not be accessed that often.

8. Install FreeBSD to zroot

Fixit# cd /dist/8.0-20090628-SNAP
Fixit# export DESTDIR=/zroot
Fixit# for dir in base catpages dict doc games info lib32 manpages ports; \
          do (cd $dir ; ./install.sh) ; done
Fixit# cd src ; ./install.sh all
Fixit# cd ../kernel ; ./install.sh generic
Fixit# cd /zroot/boot ; cp -rp GENERIC/* /zroot/boot/kernel/

9. create /etc/fstab, /etc/rc.conf, /etc/src.conf, /boot/loader.conf

Fixit# cat << EOF > /zroot/etc/fstab
# Device                        Mountpoint              FStype  Options         Dump    Pass#
/dev/ad0sb                      none                    swap    sw              0       0
EOF

Fixit# echo 'zfs_enable="YES"' > /zroot/etc/rc.conf

Fixit# echo 'LOADER_ZFS_SUPPORT=YES' > /zroot/etc/src.conf

Fixit# echo 'zfs_load="YES"' > /zroot/boot/loader.conf
Fixit# echo 'vfs.root.mountfrom="zfs:zroot"' >> /zroot/boot/loader.conf

10. Install ZFS aware /boot/loader

Fixit# chroot /zroot
Fixit# mount -t devfs devfs /dev
Fixit# export DESTDIR=""
Fixit# cd /usr/src/sys/boot/
Fixit# make obj
Fixit# make depend
Fixit# make
Fixit# cd i386/loader
Fixit# make install
Fixit# umount /dev
Fixit# exit

11. change root password

Fixit# chroot /zroot
Fixit# passwd
Fixit# exit

12. create zpool.cache

Fixit# mkdir /boot/zfs
Fixit# zpool export zroot && zpool import zroot
Fixit# cp /boot/zfs/zpool.cache /zroot/boot/zfs/zpool.cache

13. export LD_LIBRARY_PATH

Fixit# export LD_LIBRARY_PATH=/mnt2/lib 

14. Change mount points for zroot pool

Fixit# zfs set mountpoint=legacy zroot
Fixit# zfs set mountpoint=/tmp zroot/tmp
Fixit# zfs set mountpoint=/usr zroot/usr
Fixit# zfs set mountpoint=/var zroot/var

- Unmount all zfs filesystems

Fixit# zfs unmount -a

15. Install boot Manager

Fixit# fdisk -B -b /boot/boot0 ad0

It's ok if you get an "unknown class" message here, but it you get "failed to install" or the like, you'll need to try again after making sure that nothing is mounted from /dev/ad0.

16. Install ZFS boot

- Install the boot1 stage:

Fixit# dd if=/mnt2/boot/zfsboot of=/dev/ad0s3 count=1

- This may fail with an "operation not permitted" error message, since the kernel likes to protect critical parts of the disk. If this happens for you, run:

Fixit# sysctl kern.geom.debugflags=0x10

- Install the boot2 zfs stage into the convienient hole in the ZFS filesystem on-disk format which is located just after the ZFS metadata (this is the seek=1024).

Fixit# dd if=/mnt2/boot/zfsboot of=/dev/ad0s3a skip=1 seek=1024

Exit Fixit mode, and sysinstall. Then remove the FreeBSD install CD and the system will boot using the ZFS root.

Alternate Configurations

Instead of using a FreeBSD partition (ad0s3a). A ZFS Root Only system can be created directly on the slice (ad0s3) by skipping step 4, and replacing all instances of ad0s3a with ad0s3.

Fails to boot


Instead of using ad0s3b for a swap filesystem, the swap filesystem can be created on the ZFS Filesystem:

Fixit# zfs create -V 1gb zroot/swap
Fixit# zfs set org.freebsd:swap=on zroot/swap
Fixit# zfs set checksum=off zroot/swap


Instead of using vfs.root.mountfrom (step 9) and setting mount points for /tmp, /usr and /var (step 14), The system can use /etc/fstab to mount the ZFS filesystems. Using the example ZFS system above, this would be the /etc/fstab for it:

# Device                        Mountpoint              FStype  Options         Dump    Pass#
/dev/ad0sb                      none                    swap    sw              0       0
zroot                           /                       zfs     rw,noatime      0       0
zroot/tmp                       /tmp                    zfs     rw,noatime      0       0
zroot/usr                       /usr                    zfs     rw,noatime      0       0
zroot/usr/home                  /usr/home               zfs     rw,noatime      0       0
zroot/usr/ports                 /usr/ports              zfs     rw,noatime      0       0
zroot/usr/ports/distfiles       /usr/ports/distfiles    zfs     rw,noatime      0       0
zroot/usr/ports/packages        /usr/ports/packages     zfs     rw,noatime      0       0
zroot/usr/src                   /usr/src                zfs     rw,noatime      0       0
zroot/var                       /var                    zfs     rw,noatime      0       0
zroot/var/crash                 /var/crash              zfs     rw,noatime      0       0
zroot/var/db                    /var/db                 zfs     rw,noatime      0       0
zroot/var/db/pkg                /var/db/pkg             zfs     rw,noatime      0       0
zroot/var/empty                 /var/empty              zfs     ro              0       0
zroot/var/log                   /var/log                zfs     rw,noatime      0       0
zroot/var/mail                  /var/mail               zfs     rw,noatime      0       0
zroot/var/run                   /var/run                zfs     rw,noatime      0       0
zroot/var/tmp                   /var/tmp                zfs     rw,noatime      0       0

NOTE: Since zroot has it's mountpoint set to legacy, any zfs filesystem created on zroot will need to either be added to /etc/fstab or have it's mountpoint set.

ZFSOnRootWithZFSboot (last edited 2010-12-02 11:53:19 by BruceCran)