• ChrisMoerz
• bhyve_management

bhyve (and jails) manageability workstream

Documents

• bhyve management PRD

• Desired bhyve features

• Desired jails features

bhyve

There is a collection of desired features for bhyve.

As of 2024/03, priorities are

1. Understanding and identifying community needs that could be grouped into a minimum viable product (MVP)
2. figuring out a path forward for state supervision and process handling (UCL, vmstated, bmd, etc.)
3. redesign of snapshot and restore functionality as precursor to live migration capability; Vitaly G previously planned to look into options for redesign. Needs update.
4. measuring and comparing I/O performance of various ways of attaching and using storage in bhyve

There is also some additional tooling to help construct better overall bhyve management tooling

• mk-epair.sh helper script for managing epairs (useful for vnet jails)

• vmstated released as sysutils/vmstated

I'm looking at

• updating the PRD with a focus for TPM emulation implementation (?)
• creating a better overview of bhyve and jails activities across ports and base to help with new people coming to this topic.
• figuring out a path for vmstated
• how to publish the test harness that I set up for I/O performance testing so more people can run and produce measurement data we can match against

Documentation Improvements

We managed to land several updates to the Handbook on the topic of bhyve - big thanks to jrm@ for helping!

There are multiple potential areas for improvements. Some non exhaustive collection:

• adding some examples in /usr/share/examples/bhyve or /usr/share/doc/bhyve

• updating handbook to include more complex networking examples and wiring things up with and without bridges

Snapshot Redesign

There is a mail thread about redesigning the snapshot file format and there have been numerous approaches to implementing live migration; the current consensus as of 2023/11 appears to be that a redesign of the snapshot file format is required first before moving into migration.

Unless file level issues are not fixed, adding migration functionality would only complicate the matter overall. As of 13.2, snapshot functionality is not enabled by default but requires activating an option in kernel configs, adding src.conf options and then recompiling world and kernel.

My tests as of 2024/02 show, that snapshot features do not yet work on and AMD CPU with 13.2-RELEASE. The attempt crashes when vmm tries to dump VHPET data.

There are also multiple limitations

• snapshot / restore only works on identical systems (snapshot source = system on which resume happens)
• only "one of a kind" of emulated hardware may be attached to the guest
• some hardware is not yet supported, i.e. virtio-blk, nvme and others