How to boot FreeBSD from NFS?

Heavily based on https://oshogbo.vexillium.org/blog/28/

Contents

Steps
Extras
FAQ
Resources

1. Steps

1.1. Build FreeBSD from source

You may want to look at the handbook first: https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/makeworld.html

Get the FreeBSD source code. I'll call the root directory of our local source code copy freebsdsrc/.
```
git clone https://github.com/freebsd/freebsd freebsdsrc
```
Go to the directory with the source code.
```
cd freebsdsrc
```

Go to the directory with the kernel's configuration code.

# If your system is 32-bit.
cd sys/i386/conf/
# If your system is 64-bit.
cd sys/amd64/conf/

Get a copy of the GENERIC kernel (I'll call my copy CUSTOM0MP). It will be the base for our customisations.
```
cp GENERIC CUSTOM0MP
```
Now it is time to build the kernel:
```
make -j$(sysctl -n hw.ncpu) buildkernel KERNCONF=CUSTOM0MP -DNO_CLEAN
```
Estimated time on Hasee Q540S running on a one processor: 2 hours.
Estimated time on Dell L702X running on 8 processors: 19 minutes.
Go to the freebsdsrc/ directory and build the world:
```
make -j$(sysctl -n hw.ncpu) buildworld KERNCONF=CUSTOM0MP -DNO_CLEAN
```
Estimated time on Hasee Q540S running on a one processor: 8 hours.
Estimated time on Dell L702X running on 8 processors: 98 minutes.
Let's configure the destination directory for the root file system of your new FreeBSD. I chose the /usr/home/0mp/CUSTOM0MP. Just add the following lines to /etc/src.conf to set it up:
```
.if ${KERNCONF} == "CUSTOM0MP"
DESTDIR?=/usr/home/0mp/CUSTOM0MP
MODULES_OVERRIDE=md ufs
.endif
```
Remember to use spaces not tabs if you wish to indent the code.
Create the root file system now:
```
sudo make distrib-dirs KERNCONF=CUSTOM0MP
```
Estimated time on Hasee Q540S running on a one processor: a few seconds.
```
make distribution KERNCONF=CUSTOM0MP
```
Estimated time on Hasee Q540S running on a one processor: 3 minutes.
Now it's time to install the world and the kernel.
```
sudo make installworld KERNCONF=CUSTOM0MP
```
Estimated time on Hasee Q540S running on a one processor: 5 minutes.
```
sudo make installkernel KERNCONF=CUSTOM0MP
```
Estimated time on Hasee Q540S running on a one processor: a few seconds.
Done. Now it's time to configure you VirtualBox.

1.2. Configure VirtualBox

1.2.1. Configure VirtualBox network

Open File > Preferences > Network > Host-only Networks.
Click the Add host-only network (Ins) button with a flag and a plus.
Select your new network (I'll assume its name is vboxnet0) and click the Edit host-only network (Space) screwdriver button.
Open DHCP Server and uncheck Enable Server.

1.2.2. Configure a virtual machine

Go Machine > New to open the Create Virtual Machine window.
Call your FreeBSD machine however you wish. Choose the proper version of FreeBSD - pay attention to the Version (32-bit or 64-bit).
Give your machine some RAM.
Add a reasonable hard drive.
Select your newly created machine and open Settings > Network > Adapter 1.
Check Enable Network Adapter.
Choose Host-only Adapter from the Attached to: list. Select vboxnet0 in the Name: field.
Click Advanced and put down the value of the MAC Address: field somewhere.

The VirtualBox configuration is also covered by [the oshogbo's tutorial] in the VIRTUALBOX CONFIGURATION section.

1.3. Set up TFTP server

Enable TFTP. Uncomment the

tftp dgram udp wait root /usr/libexec/tftpd tftpd -l -s /tftpboot

line from the /etc/inetd.conf file.

Edit it so that /tftpboot points to your kernel's root directory. In my case the line looks like this:

tftp    dgram   udp     wait    root    /usr/libexec/tftpd      tftpd -l -s /usr/home/0mp/CUSTOM0MP

Add inetd_enable="YES" to /etc/rc.conf and restart the service:
```
sudo service inetd restart
```

Now let's get a copy of pxeboot.

sudo cp /boot/pxeboot /usr/home/0mp/CUSTOM0MP/boot/pxeboot

Test if everything is working fine. Run:
```
 tftp localhost
```
Now in type get boot/pxeboot in the tftp prompt. You should get a message similar to:
```
Received 268289 bytes during 0.1 seconds in 526 blocks
```
You've just downloaded a copy of pxeboot to the current folder - you should probably delete it. See FAQ if you've got pxeboot files scattered around the system.

1.4. Set up NFS server

My custom0mp machine will be assigned the 192.168.56.103 IP address.

Add this line to the /etc/exports file:

/usr/home/0mp/CUSTOM0MP -alldirs -maproot=0 -network 192.168.56.0 -mask 255.255.255.0

Add this line to /etc/rc.conf:
```
nfs_server_enable="YES"
```
Restart the NFS server:
```
sudo service nfsd restart
```

Check if everything is working fine. Run:

showmount -e

You should get an input like:

Exports list on localhost:
/usr/home/0mp/CUSTOM0MP                192.168.56.0

Set up what and where to mount. Add the following line to /usr/home/0mp/CUSTOM0MP/etc/fstab:
```
192.168.56.1:/usr/home/0mp/CUSTOM0MP / nfs rw 0 0
```

Set up loader.conf and rc.conf.

Add the following lines to /usr/home/0mp/CUSTOM0MP/boot/loader.conf:

autoboot_delay="3"
beastie_disable="YES"
boot.nfsroot.options="nolockd,tcp"

Add the following lines to /usr/home/0mp/CUSTOM0MP/etc/rc.conf:

sshd_enable="YES"
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
cron_enable="NO"
devd_enable="NO"

1.4.1. Configure a DHCP server

Install isc-dhcpd. For example with:
```
sudo pkg install isc-dhcp43-server
```

Modify /usr/local/etc/dhcpd.conf to suit your needs. Here's mine configuration:

default-lease-time 600;
max-lease-time 7200;

ddns-update-style none;

# Set the subnet and the mask values the same as in /etc/exports.
subnet 192.168.56.0 netmask 255.255.255.0 {
    authoritative;
    use-host-decl-names on;
    option subnet-mask 255.255.255.0;
    option broadcast-address 192.168.56.255;

    host custom0mp {
        hardware ethernet 08:00:27:22:A9:64;

        # The fixed IP address assigned to the virtual machine.
        fixed-address 192.168.56.103;

        # The path to the boot loader; it is relative to the root-path below.
        filename "boot/pxeboot";

        # The IP address that provides the NFS.
        next-server 192.168.56.1;

        # The path to the root directory over the NFS.
        option root-path "/usr/home/0mp/CUSTOM0MP/";
    }
}

Add these lines to /etc/rc.conf:

dhcpd_enable="YES"
dhcpd_ifaces="vboxnet0"

Start the DHCP deamon:
```
sudo service isc-dhcpd restart
```

This part is also covered in detail by the oshogbo's tutorial in the DHCP SERVER section.

1.5. Boot FreeBSD over NFS

Reboot your host FreeBSD (just to be sure that everything is configured).
Start VirtualBox.
Start the virtual machine to activate vboxnet0.
This step is needed because DHCP probably failed to start since there had been no vboxnet0 interface before a virtual machine was launched. Now, as VirtualBox host-only network is up and running we want to restart the DHCP server.
Restart the DHCP server
```
sudo service isc-dhcpd restart
```
It might fail if vboxnet0 is not yet available.
Start your virtual machine. Hopefully, it will boot over the network. You might need to restart your machine if the DHCP server isn't yet fully functional.

2. Extras

A potentially useful thread on the FreeBSD forum about mounting from NFS (Can't NFS mount exported root).

Here's my NFS related section in /etc/rc.conf:

inetd_enable="YES"
nfs_server_enable="YES"
dhcpd_enable="YES"
dhcpd_ifaces="vboxnet0"

3. FAQ

Why do I have so many pxeboot files scattered around?

Whenever you do this to test whether your NFS is bootable:
1. Run tftp localhost.
2. Enter get boot/pxeboot when prompted by tftp.
3. You should get a message like this:
```
Received 333825 bytes during 0.0 seconds in 655 blocks
```
You actually save a copy of the pxeboot file in the current working director when you do this. Hence the presence of those pxeboot files all over your system.
Why do the boot process freeze?

You might see something similar to
```
NFS ROOT: 192.168.56.1:/usr/home/0mp/CUSTOM0MP/
```
In my case I had to leave it for 3 hours to be able to log in on Hasee Q540S. It takes about a minute to boot over NFS on Dell L702X however.
Why can't I add a user on my virtual machine?

If you are getting an error similar to this one:
```
passwd: could not lock the passwd file: Operation not supported
```
you might be missing boot.nfsroot.options="nolockd,tcp" from the boot/loader.conf file of your NFS.
If it doesn't help then try to tinker with `rpc.lockd(8)` and `rpc.statd(8)`; I stumbled upon them as I was looking for the solution myself.

4. Resources

CategoryHowTo

MateuszPiotrowski/BootFreebsdFromNfs (last edited 2022-11-28T07:16:28+0000 by BenedictReuschling)