Google Summer Of Code 2023: Integrate mfsBSD into the release building tools

Project description

mfsBSD is a toolset to create small-sized but full-featured mfsroot based distributions of FreeBSD that store all files in memory (MFS) [Memory File System] and load from hard drive, usb storage device or optical media.

It can be used for a variety of purposes, including diskless systems, recovery partitions and remotely overwriting other operating systems.

MartinMatuska is the creator of mfsBSD, and he is also the author of the original mfsBSD white paper, which is where the excerpt above was taken from. The upstream mfsBSD is maintained at the GitHub mfsBSD repository.

This project integrates mfsBSD into the FreeBSD release tool set, creating an additional target of mfsBSD images (.img for disc and .iso for optical images) into the /usr/src/release/Makefile. Until now, mfsBSD only existed outside the FreeBSD release tool chain, and only the -release versions have been produced. With this project, mfsBSD images are now available at the official FreeBSD release page, and you can also build mfsBSD yourself by invoking cd /usr/src/release && make mfsbsd-se.img mfsbsd-se.iso, which will then create mfsbsd-se.img and mfsbsd-se.iso at /usr/obj/usr/src/${ARCH}/release/.

Project outcome

Two ways of building mfsBSD have been implemented. The first use case is intended for most use cases, while the second is intended only for release engineering purposes -- e.g. creating all of the FreeBSD installation media for distribution purposes. 

   1 #
   2 # [Use Case 1] If you're a regular user, use this method.
   3 #
   4 
   5 # Build mfsBSD images.
   6 cd /usr/src/release
   7 make mfsbsd-se.img mfsbsd-se.iso
   8 
   9 # The make process creates `mfsbsd-se.img` and `mfsbsd-se.iso` here:
  10 ls -lh /usr/obj/usr/src/${ARCH}/release/

That's it! Basically, it's very similar to mfsBSD's own make process with CUSTOM=1 BUILDWORLD=1 BUILDKERNEL=1 flags. In contrast, the second option shown below is different from the first option in that it invokes not only the mfsBSD image build process, but also all the rest of the FreeBSD release build processes. The first use case only builds mfsBSD itself, and thus is intended for most use cases, while the second use case's purpose is to build all the release media. 

   1 #
   2 # [Use Case 2] Build mfsBSD alongside all release artifacts.
   3 #
   4 
   5 # Build prerequisites.
   6 cd /usr/src
   7 make buildworld buildkernel
   8 
   9 # Build mfsBSD images.
  10 cd /usr/src/release
  11 make release WITH_MFSBSD=1
  12 
  13 # Copy the produced images to ${DESTDIR} and also prepend ${OSRELEASE} to the filenames.
  14 make install WITH_MFSBSD=1 DESTDIR=/usr/src/release/images
  15 ls -lh /usr/src/release/images

The mfsBSD build process is the same for both the first use case and the second use case. However, the second use case also builds all the release artifacts and all of the FreeBSD installation media, such as cdrom, dvdrom, memstick, and mini-memstick. Also, running make install WITH_MFSBSD=1 DESTDIR=/usr/src/release/images copies all of the built images to ${DESTDIR} and appends a prefix to all of the image names ${OSRELEASE}-mfsbsd-se.{img|iso} for distribution purposes.

Example usage of an mfsBSD image

A lot of the example in this section has been taken from the documentation on FreeBSD remote install. This is just one among many possible uses of an mfsBSD image, but it shows you how to install FreeBSD on a system, where there's just a single drive. Thanks to mfsBSD, you can install FreeBSD even without any external storage device, such as an USB, because mfsBSD - by its own definition - runs entirely on the memory file system, which means the original drive that the mfsBSD booted from can then be modified and even completely erased and overwritten with a new operating system.

First, boot into your existing operating system's recovery mode. Then, type the following: 

   1 # List all the disks and find the name of your target drive.
   2 # The name can be different depending on the drive, but suppose we want ada0.
   3 # Listing all the disks is `geom disk list` in FreeBSD, but is likely to be different in other systems.
   4 geom disk list
   5 
   6 # Write the mfsBSD image to your target drive.
   7 dd if=./mfsbsd-se.img of=/dev/ada0 bs=1M
   8 reboot

When rebooting, select the target drive as the booting device. For example, if you're using a Thinkpad, type F12 while booting. 

   1 # Prepare base.txz and kernel.txz for FreeBSD installation.
   2 # Since the image we used (mfsbsd-se.img) is a special edition image, it comes packed with these two files, inside the third partition of mfsBSD.
   3 mkdir /mnt/dist
   4 mount /dev/ada0p3 /mnt/dist
   5 
   6 # Later, we will be using bsdinstall for FreeBSD installation, and bsdinstall looks for the dist files in this specific folder.
   7 mkdir /usr/freebsd-dist
   8 cp /mnt/dist/<version>/*.txz /usr/freebsd-dist/
   9 
  10 # Install FreeBSD.
  11 bsdinstall

Milestones

Development logs

This section explains three major problems encountered in the entirety of the project, and explains what kind of approaches were taken to solve them. The first thing I did for this project was to import mfsBSD into /usr/src/contrib folder and try to build it: 

   1 git clone https://github.com/mmatuska/mfsbsd.git
   2 cp -r ./mfsbsd /usr/src/contrib/
   3 cd /use/src/contrib/mfsbsd
   4 make BASE=/usr/obj/usr/src/${ARCH}/release

This, however, gave me an error 

# First error
Installing configuration scripts and files ...install: /usr/home/fred/mfsbsd/tmp/mfs/etc/motd: No such file or directory.

Interestingly, this error would go away if I mv /usr/src/contrib/mfsbsd /mfsbsd and run from there. The error would reappear if the folder is located at /usr/src/contrib/mfsbsd. So, in the middle of the Makefile, I ran pwd. It showed me the current working directory as /usr/obj/usr/src/${ARCH}/contrib, which it shouldn't be. It was supposed to be pointing at where the mfsbsd folder was located at, which is /usr/src/contrib/mfsbsd, but pwd was pointing to a non-existent, mysterious directory.

How was this problem resolved? This Stack Overflow answer helped me figure out what's happening here. Basically, there's an internal mechanism in make that executes chdir if the directory is located in certain ways. To solve this, I decided to add an extra flag that overrides this unexpected behavior. Here's the code: 

   1 # MAKEOBJDIR flag solved the problem.
   2 cd /usr/src/contrib/mfsBSD
   3 env MAKEOBJDIR=${MFSBSD_DIR} make BASE=/usr/obj/usr/src/{ARCH}/release

The first problem was therefore solved. The second unexpected behavior popped up right after that -- don't get me wrong, I was extremely happy to see this second error and not the first one for the 100th time: 

# Second error
Installing pkgng ...pkg-static not found at: /usr/local/sbin/pkg-static.

Apparently, this is a much more well-known issue compared to the first. There was only one query result on Google regarding the first, while there were dozens regarding this one.

pkg-static is a program that is already included inside pkg. However, my machine was able to notice this instance of error because it was running a fresh install of FreeBSD for testing purposes and was never connected to the internet. When you first run pkg, it goes through an initialization process after asking the user to confirm it. The problem here, however, was that this initialization process requires internet connection, and it is this exact initialization that installs pkg-static. Thus, without internet access, pkg-static doesn't exist, and this was why mfsBSD build process was exiting with an error.

The fix, of course, is unnecessary if the device has an internet connection, and this indeed is most often the case. If you have downloaded the src tree, it most likely means you already have internet access. So, I added a simple if statement in /usr/src/release/Makefile.mfsbsd to check if the pkg-static binary exists, and if not, just bypass that portion of the build program that requires pkg-static. If it does exist, which will be most of the time, proceed as normal. 

   1 . if !exists(${MFSBSD_PKGSTATIC})
   2 # If pkg-static is not present at the device, mfsBSD build process
   3 # cannot proceed further. Thus, creating .packages_done bypasses
   4 # that specific part of the build process that requires pkg-static.
   5         touch ${MFSBSD_DIR}/work/.packages_done
   6 . endif

Finally, the last problem I encountered was: 

# Last error
These libraries should be LIBADD+=foo rather than DPADD/LDADD+=-lfoo: ${_BADLDADD}

The solution was to add -m${MFSBSD_MKINCLUDEDIR} flag. The build process was trying to find mk files from a wrong directory, such as /usr/src/share/mk, so I had to override that with the correct directory /usr/share/mk. The -m flag does exactly that.

What was the cause of this behavior? According to MAKE(1), "the search [for sys.mk and include makefiles] starts with the current directory of the Makefile and then works upward towards the root of the file system."

Update: One more major design decision I had to make was to ditch cd ${MFSBSD_DIR} && make CUSTOM=1 BUILDWORLD=1 BUILDKERNEL=1. It had two issues. (a) Even though mfsBSD images can be built harmlessly at first glance, it actually fails to mount the memory disk when it boots in certain devices. For example, this issue caused the mfsBSD to boot in one of my two amd64 devices, but wouldn't work on my newer amd64 machine. (b) Also, the resultant image's size was considerably larger. Its .img image was around 1.5GB, while its .iso image was around 1.3GB.

Ditching the original method and using a simple cd ${MFSBSD_SRCDIR} && ${IMAKE} buildworld buildkernel along with ${IMAKE} -C ${.CURDIR} obj bootonly solved both of the issues. The mfsBSD images boot correctly without any problem mounting its memory disks, and also the images themselves are around 300MB only.

https://summerofcode.withgoogle.com/programs/2023/projects/Pw6wO1Ng

Sponsored by: Google, Inc. (GSoC 2023)

SummerOfCode2023Projects/IntegrateMfsBSDIntoTheReleaseBuildingTools (last edited 2023-09-26T19:08:57+0000 by SoobinRho)