Contents
Playing with LiveCDs
Introduction
LiveCDs are without a doubt a very useful concept. They are helpful whether you want to prepare a new system for installation, or recover an existing system from misconfiguration, or simply have a familiar system image that you can carry around. There are a few projects that produce their FreeBSD (or, perhaps, FreeBSD-based?) LiveCDs. Such projects also typically provide SDKs for building your own customized LiveCDs. I used some of them and they worked beautifully. But there is one issue - those things work automatically, auto-magically, mysteriously... I configured a few things, followed some steps, ran a script or two and, voìla, here was my ISO image. But I had zero idea what those scripts did internally, and reverse-engineering the scripts looked too hard.
Demistification
Then, one day I attended a presentation by Mykola Marzhan at KyivBSD where he explained how one could build his own FreeBSD LiveCD. Step-by-step, using command line, no ready-made scripts. That was enlightening and exactly what I needed. And it was very simple. You can find presentation materials here.
Here are instructions for creating the simplest LiveCD:
- Create a directory where a system image will be prepared; e.g.:
mkdir /tmp/R
- Put a pristine FreeBSD tree into that directory; if you use source method, then this could be:
make buildworld make buildkernel
Note: cd9660 filesystem support must be included with kernel, this is already the case for GENERIC
make installworld DESTDIR=/tmp/R make installkernel DESTDIR=/tmp/R make distribution DESTDIR=/tmp/R
- Now time for some basic configuration:
echo 'boot_cdrom="YES"' >> /tmp/R/boot/loader.conf
This will make loader pass -C option to kernel to let it know that it should mount root from CD-ROM (/dev/cd* or /dev/acd*)
- Make ISO image:
cd /tmp/R mkisofs -R -l -ldots -allow-lowercase -allow-multidot -V 'My FreeBSD' -volset 'My FreeBSDs' -hide boot.catalog -o ../R.iso -no-emul-boot -b boot/cdboot .
And this is it! Well, almost 
You have an image of the most trivial LiveCD, it has default configuration (read: no configuration), hostname is not set, network will not be configured, no users, etc. But it will boot and you will get to single-user shell.
Actually, with an additional rather simple change you should be able to boot to multi-user. This change doesn't have to be necessary, but currently FreeBSD tries to re-mount its root filesystem as R/W and if that fails the boot process is interrupted with a message like the following and you are left in single-user mode:
Mounting root filesystem rw failed, startup aborted
FreeBSD tries to do that even if root filesystem is cd9660. So we have to tell it that the root filesystem should stay readonly. In order to do that we should put an entry into fstab (and we have to predict device name for a CD/DVD drive):
/dev/cd0 / cd9660 ro 0 0
Alternatively you can set root_rw_mount="NO" in rc.conf or you can hack etc/rc.d/root script.
How this works
Just in case you wonder why such a simple setup works. By using -b boot/cdboot mkisofs option we specified what should be run by BIOS (firmware) when boot from CD/DVD is chosen. cdboot knows basics of ISO9660, it can find loader(8) and start it. loader also knows how to read ISO9660, it can find and read/load its configuration and kernel with modules, it then executes kernel. kernel in turn also should have ISO9660 filesystem support and it is instructed to use CD as its root, so it mounts root filesystem and kicks off init, which then executes rc scripts that set up a system environment.
You might wonder: some system services need R/W filesystem, but CD is R/O. E.g. daemons create pid files and write to logs. This is taken care of by two rc scripts: rc.d/var and rc.d/tmp that use mdmfs(8) to create in-memory filesystems that are mounted over /var and /tmp if necessary, that is, if those are readonly.
What next
Of course, our simple LiveCD can be enhanced and improved. In rc.conf you can set a hostname and provide some network options (e.g. for DHCP). You can set up resolv.conf or your timezone via etc/localtime. You can disable various services that are enabled by default and that are not needed on a LiveCD.
This what I usually have for example:
update_motd="NO" syslogd_enable="NO" sendmail_enable="NONE" cleanvar_enable="NO" hostid_enable="NO" ip6addrctl_enable="NO" moused_nondefault_enable="NO" sendmail_submit_enable="NO" sendmail_outbound_enable="NO" sendmail_msp_queue_enable="NO" cron_enable="NO" crashinfo_enable="NO" virecover_enable="NO" newsyslog_enable="NO" mixer_enable="NO"
But, of course, only sky is a limit to what you can do. You may want to install some useful thirdparty tools - it's easy. Just chroot into the image directory and do things that you want to do. You might find nullfs(5) to be a convenient way to make directories from host system to be available in the chrooted environment. For example, a directory with packages or ports directory.
Note: things might be not as easy if you are creating a LiveCD for different architecture using cross-compiled world and kernel.
Test before burning
I think I should mention this just in case: it is a good idea to test your LiveCD ISO using likes of emulators/qemu or emulators/virtualbox-ose before actually burning it and trying with real hardware. This way you can discover, troubleshoot and fix potential issues in advance. Sample qemu command line invocation:
qemu -m 256 -cdrom /tmp/R.iso -boot d
Installation
Perhaps it's not immediately obvious, but you can (re-)use the image you created for installation.
Because we didn't mangle our system image much we can transfer it to HDD and have the same system installed there. Of course, first you need to partition the disk, create filesystem(s) and only then transfer the files. With a full-blown system on LiveCD it's quite easy. You can use your favorite way of configuring disks.
You can run gpart and newfs (zpool, zfs), or you can use sade, or even start sysinstall in configuration mode. Don't forget to install necessary boot chain to HDD for your selected configuration.
Once you have filesystems prepared and mounted you can proceed to copying the system files. Here you can encounter a quite unpleasant surprise: copying files from CD is very slow. CD read speed is not very high in general, but especially so when you read many smaller files instead of one large sequential file.
There is an easy solution. Create an archive of your prepared system tree, for example a .tbz2, then add it to the tree before making an ISO image. You will see that extracting files from a compressed archive is much faster than for a plain filesystem.
Note that you can create the archive at different steps of your system tree preparation. You can create it from a pristine tree, or you can create from a final tree with all the configuration and additional software, or something in the middle. You can even replace some files or add new files to the archive, so that you can get a pre-configured system that you want after unpacking.
Try to reboot to a newly installed system - if anything goes wrong you still have your LiveCD that you can use to fix any mistakes. Once your installed system works fine you can proceed to build it up to final configuration.
Hint: when working from LiveCD you might get into a situation where you need to change some file on your (readonly) LiveCD; in this case you can use mdmfs to create an in-memory filesystem and then use unionfs(5) to mount it over a sub-tree where you want to make changes. That way you still have your readonly base files visible, but you can create and modify new versions of them in the in-memory filesystem. Keep in mind that the changes are not persistent between reboots
Improvements
Now everything is OK, but still there are two things of concern:
- We practically have two copies of system tree on the LiveCD, one for the live filesystem itself and one in the installation archive.
- We see that reading from compressed archive is much faster, but our live environment provides slow access to files scattered over CD.
Remember, when you start a program it typically also loads a number of shared libraries, checks some configuration files and so on. This is quite slow when done on a CD.
One popular idea is to compress filesystem image itself and use it with a help of, for instance, geom_uzip(4). In this case we would have the whole system tree stored in a single contiguous compressed file. That is, we kill two birds with one stone - our live filesystem access becomes faster and we don't need a separate archive with the system tree. (We still might want an archive with the overrides for turning live filesystem tree into installation tree).
One thing to keep in mind is that neither BIOS nor cdboot nor loader understand geom_uzip. So all the boot chain up to and including kernel and modules must reside on a regular CD9660 filesystem. But root filesystem can be in a geom_uzip image because kernel (and/or geom_uzip.ko) knows how to handle geom_uzip images.
But here is a small problem: we probably don't want our complete compressed image to be pre-loaded by loader into RAM. First, it would take a quite large chunk of memory (but not too large for modern systems with 1GB of RAM or more). Second, it would take a long time to load the whole image into RAM. And that would happen on any boot, even if we are not going to install anything and will actually run only a handful of tools.
But if our image is not pre-loaded, then the kernel would have to access it first as a file on another filesystem. That, in turn means, that a filesystem inside the image can not be root filesystem; the filesystem holding the image would have to be mounted first.
The solution then is to have a very small root CD9660 filesystem with necessary tools to create a memory disk backed by our uzip image. Once we create the memory disk and mount our system filesystem that it holds, then we can use init_chroot capability [see loader(8)] to chroot to our system tree and, thus, make it appear as root filesystem.
So, step-by-step:
- Create, watch this, a UFS filesystem from our prepared tree:
makefs R.ufs /tmp/R
- Create a uzip disk image from the filesystem image:
mkuzip R.ufs
- Prepare a new tree for ISO image:
copy boot and rescue directories from the prepared original tree
Make sure that you use copy method that preserves hard-links, e.g. tar cf - | tar xf -, otherwise rescue will grow up in size dramatically, this is because it actually has one file with many hard-links to it
This is our boot chain plus basic (and compact) set of tools that we need before chroot
Create empty directories dev and newroot
- Copy R.ufs.uzip that we created above into the root of the new tree
If kernel was compiled without GEOM_UZIP option, then add the following line to boot/loader.conf:
geom_uzip_load="YES"
Configure init_chroot in boot/loader.conf, add the following lines:
init_path="/rescue/init" init_shell="/rescue/sh" init_script="/newroot.rc" init_chroot="/newroot"
- Create newroot.rc script that will prepare new root tree from our system image:
#!/bin/sh #set -x PATH=/rescue mdconfig -a -t vnode -f /R.ufs.uzip -o readonly -u 0 mount -r /dev/md0.uzip /newroot mount -t devfs devfs /newroot/dev kenv init_shell="/bin/sh" echo "newroot setup done" exit 0
- Create new ISO image the same way as before:
mkisofs -R -l -ldots -allow-lowercase -allow-multidot -V 'My FreeBSD' -volset 'My FreeBSDs' -hide boot.catalog -o ../R2.iso -no-emul-boot -b boot/cdboot .
Here you are. One drawback of this approach is that once init chroots you can not access original root CD9660 filesystem should you want to do that for any reason.
Now simple tar | tar copy of the live filesystem to HDD should be almost as fast as it was from a compressed archive.
Further enhancements
One obvious enhancement is to trim some fat. You can remove the modules that are not referenced in loader.conf from the boot CD9660 filesystem. In that case you obviously won't be able to load those modules from loader prompt, but you will still be able to load any modules once booted. So be careful with modules that might be needed for boot.
Also, you can compress your kernel which should improve boot time a little bit. Just go to boot/kernel directory and run gzip kernel.
Another improvement would be to fix the issue of not being able to access original boot/root filesystem after chrooting to live root. I've already made a passing mention of loader's ability to pre-load filesystem images into memory. A kernel with MD_ROOT option is able to use such a pre-loaded image as root filesystem.
So the idea is to use such an image as an original root filesystem, then mount CD in a subdirectory and then use nullfs to make it available in a new (chrooted) live root.
So, we now split what used to be a boot filesystem into a new boot filesystem and a memory root filesystem. In memory root filesystem we need only rescue that provides necessary tools and a script that prepares live root using those tools. In boot CD9660 we have boot directory with loader, kernel, modules and loader configurations. Also, we will need to add memory filesystem images that will be used for memory root and for live root.
So:
- We create some temporary directory.
Put rescue in it.
Put a script named baseroot.rc with the following contents:
#!/bin/sh #set -x PATH=/rescue BASEROOT_MP=/baseroot RWROOT_MP=/rwroot CDROM_MP=/cdrom BASEROOT_IMG=/data/base.ufs.uzip # Re-mount root R/W, so that we can create necessary sub-directories mount -u -w / mkdir -p ${BASEROOT_MP} mkdir -p ${RWROOT_MP} mkdir -p ${CDROM_MP} # Try to mount CD device, currently only trying cd0 and acd0; # can be enhanced to try more device names. mount -t cd9660 /dev/acd0 ${CDROM_MP} || mount -t cd9660 /dev/cd0 ${CDROM_MP} # Mount future live root mdmfs -P -F ${CDROM_MP}${BASEROOT_IMG} -o ro md.uzip ${BASEROOT_MP} # Create in-memory filesystem mdmfs -s 64m md ${RWROOT_MP} # Union-mount it over live root to make it appear as R/W mount -t unionfs ${RWROOT_MP} ${BASEROOT_MP} # Mount devfs in live root DEV_MP=${BASEROOT_MP}/dev mkdir -p ${DEV_MP} mount -t devfs devfs ${DEV_MP} # Make whole CD content available in live root via nullfs mkdir -p ${BASEROOT_MP}${CDROM_MP} mount -t nullfs -o ro ${CDROM_MP} ${BASEROOT_MP}${CDROM_MP} kenv init_shell="/bin/sh" echo "baseroot setup done" exit 0
- Create memory root filesystem image, e.g.:
makefs -b 10% memroot.ufs /tmp/memroot
Note -b 10% option: it makes sure that the filesystem has some free space and we will be able to write to it.
- The image can be compressed the same way as kernel and modules:
gzip memroot.ufs
Then put memroot.ufs.gz into boot directory of CD tree
- Update loader.conf to load the memory filesystem and perform init_chroot trick; complete sample loader.conf:
#geom_uzip_load="YES" #in kernel mfsroot_load="YES" mfsroot_type="md_image" mfsroot_name="/boot/memroot.ufs" #note absense of .gz suffix above init_path="/rescue/init" init_shell="/rescue/sh" init_script="/baseroot.rc" init_chroot="/baseroot"
Note how defined BASEROOT_IMG in baseroot.rc, so create data directory and put live root uzip image into it under name base.ufs.uzip
- mkisofs time now
For the reference, here is a listing of an ISO created this way:
boot/ boot/beastie.4th boot/boot boot/boot0 boot/boot0sio boot/boot1 boot/boot2 boot/cdboot boot/defaults boot/defaults/loader.conf boot/device.hints boot/firmware/ boot/frames.4th boot/gptboot boot/gptzfsboot boot/kernel boot/kernel/kernel.gz boot/loader boot/loader.4th boot/loader.conf boot/loader.help boot/loader.rc boot/mbr boot/memroot.ufs.gz boot/modules/ boot/pmbr boot/pxeboot boot/screen.4th boot/support.4th boot/zfs/ boot/zfsboot data/ data/base.ufs.uzip
Note that various boot files can be safely removed, I haven't done that only because of lazyness. Those files are not used during CD boot and their copies are present in live root. You can even move away cdboot, but don't forget that you need to provide valid cdboot to mkisofs with -b option.
Appendices
Sample sizes
LiveCD created in the most simple way, without installation archive |
~560MB |
Installation archive or uzip filesystem image |
~210MB |
LiveCD create using uzip image, with or without memory root |
~220MB |
Some useful programs
-- AndriyGapon 2010-04-17T20:44:44+0000