Virtualization Working Group
Chair Person: Michael Dexter
Agenda:
- Virtualization in general
- Status of the latest bhyve, XEN developments
- Discussion about future bhyve, XEN features
Attendees are welcome to add their own topics to the agenda.
Session Notes
https://etherpad.net/p/2015DevSummitVirtualization
Q: bapt: VirtIO RAM balloon options? Can we overprovision RAM and increment the RAM allocation in a dynamic, coin-operated way? Options with the physmem and max.physmem sysctls? Could they be dynamic? Would a dynamic change be safe on a live system? Does Xen do this?
A: Needs investigation. Top should be adjusted to show the new number of cores and available RAM. Cores should be spin-uppable like on real hardware. Launch with 64 cores with 60 idle. Likely can only add cores, not remove. bapt/kib/Konstantin Belousov have needs/ideas.
Q: Does CHERI (Capability Hardware Enhanced RISC Instructions) support virtualization extensions, making it a target bhyve platform?
A: Not yet.
Q: State of Intel SR-IOV?
A: "We have a broken Intel driver, Jack Vogel and Colin P. know about it." "Our VirtIO does multiqueue by default, which beats Linux." TSO? Does Xen support it? Erick Joiner and GNN may have insights. Has Jack left Intel?
Q: VirtIO/lagg bridge bug? Benjamin Perrault has observed a massive slowdown in host to VM networking with a combination of bhyve VirtIO/lagg and bridge. Works fine to the outside world but not between the host and VM. Must investigate.
Q: bhyve clock sync. Room for improvement?
Q: Vale virtual network switch bhyve/tap hooks? John-Mark Gurney may have netmap insights. (vnet: full virtualization of routing tables)
Q: mkfs/mkimg news?
A: mkfs defaults to UFS1 but supports UFS2. "UFS dies with growfs" bapt is working on powderkeg libvdsk integration. bapt has a technique to powderkeg build a system, export it to an NFS root, format a proper FS and remount the proper file system, perhaps like Linux initrd. Edward Tomasz Napierała may have some bits in head for this.
Current tools to create or convert image files: - qemu-img, VBoxManage, mkimg and libvdsk
John-Mark Gurney's steps
Native qcow, qcow2, VMDK, VHD etc. support is not here yet.
Q: Can CPU Cores be pinned to GELI encryption use?
A: Unknown.
Q: bhyve PXE and USB?
A: The UEFI code defaults to PXE if no boot media can be found. The PXE GSoC project is still in progress.
Q: TunnelFS? Something like VMware's ability to pass a host directory into a VM for file sharing.
A: DFly has DirFS for use with vkernels. SavagedLight has proposed a TunnelFS project. NFS can be used but we prefer avoid RPC calls, opting for VirtIO calls. "Virtual RDMA to guest." Windows Driver? Plan 9 protocol? Used in KVM? John-Mark Gurney and Edward Tomasz Napierała may have insights. Note this, this and this.
Q: State of the "I am virtualized" sysctl?
A: kern.vm_guest appears to be generic. Perhaps should have heuristics to say "I am in bhyve|Xen|VMware". Note smbios sysctl. Note ifconfig device renaming. default=DEFAULT. Could check hypervisor or bare metal and choose between em0 and vtnet0...
Q: Because bhyve leverages or bumps into so many subsystems, how can we help documentation?
A: "See Also:" is always appreciated in man pages and the Wiki. i.e. "See also growfs/How to grow a FS in a VM". growfs was written by John-Mark Gurney and uses sh and awk for name mangling. (Need /bin on /) Colin mentioned the issue of swap on EC2 images, partition order. (All need to be handled correctly)
Q: NanoBSD NG?
A: Crochet has its place. Kamil has worked with Devin/Nathan on various installation scripting/bsdconfig.
Q: Multiboot 64-bit kernel?
A: Currently no chain-loading. We need full iPXE. Local parameters. GRUB kfreebsd pulls these from the loader? bapt and Peter Wemm may have ideas.
Q: bapt is finding HTTP boot quite slow.
A: The loader is lacking network drivers. Need mfsroot, memdisk or SAN boot for faster boot. Issues with signed loaders/secure boot. Edward Tomasz Napierała may have insights. Using iPXE?
Q: Is "isboot/iBFT" boot working?
A: Apparently the new isboot package does the trick.
Q: What's up with the spread out ARM KERNCONF's?
A: That is NetBSD style. They could be unified into more-familiar form. Not necessarily difficult.
Q: zfsd?
A: bapt@ has a call scheduled with Will at Spectra to straighten out the last bits. BSD-licensed code reportedly exists to mimic the Solaris SMF. Note: Avoid double resilvering.
Q: systemd shims for GNOME?
A: bapt says it's dead because the author is too busy. The dbus API is not adequately documented. logind was worked on. "Pass tty to file descriptor" "Not worried about logind, dated, timed, hostnamed."
Q: State of kNFS, possible kCIFSd/CTL/ZFS integration? Goal: Share an iSCSI target etc. from ZFS natively.
A: Illumos has kCIFSd, no? The hook to call Samba depends on the Samba version. Shim is easier than the Solaris version. Different script in base -> wrapper. Allan is interested. Set up Samba, trigger ZFS dataset. Isilon appears to have obtained the "Likewise" open source daemon. Not Sharity. Edward Tomasz Napierała may know where to find the last open source snapshot.
Q: ZFS iSCSI CTL hood?
A: Edward Tomasz Napierała and Allan have ideas. Generate a CIFS config file. FYI: Lots of NFS.
Q: Why does gpart have 90%, not 100% of the valid GPT types?
A: Allan, Ed and bapt care. Michael Dexter recalls the data existing in two locations with hope of one centralized lookup table. Allan Jude is investigating. Note gpt.h, g_part.c.
Q: Kris Moore GRUB fixes?
A: Perhaps one to support 1MB ZFS block size, feature flags.
Q: grub-mount command support?
A: Unknown
Q: Status of the FreeBSD VPS Project?
A: Author is "too busy" and the patch is very intrusive, making incorporation difficult. Can it be broken down?
Q: Is anyone interested in maintaining the Xen port?
A: Unknown. GCC is NOT a runtime dependency and the 4.6 release is quite exciting, warranting -devel builds.
Q: State of WebDAV/SSL?
A: John-Mark Gurney sees value in this given the chaos of other file servers. Not sure about user auth, extended attribute and ACL support. Not sure of the ideal simple web server. Q: Is this a fuse questions? Arla nnfps? Note Linux khttpd. Crazy? What WebDAV server is Apple using? Locking issues? Is IPFS the answer? libsmbfs appears to be from 2002.
Q: State of pNFS? A: Unknown.
Q: Ideal scale-out solution? A: Unknown, most are broken in various ways.
Q: State of installer boot environment/GELI support?
A: Note 64k limit of boot2 loader. pmbr... Kostic, kip, and Baldwin are recommended contacts. Colin Percival is actively working on this.
Q: Would be nice to perform a new snapshot installation directly to a boot environment with no disk partitioning.
A: need more detail
Q: Does FreeBSD support the maximum GPT label length? If so, why not?
A: Allan Jude will investigate. Wants his disk serial numbers in labels.
Q: How's that complete IPMI rewrite coming for Supermicro boards?
A: bapt is working on "portdiag" and freeipmi tool works for what it's worth. Web console is not adequate for virtual media. Is a RAM hog. Linux64 compatibility MAY allow for the browser plugin to work.
Q: State of camcontrol (pass0,ada0)/(ada1/pass1) confusion?
A: Long term goal is camcontrol libxo output. bapt has made progress on the sesutil to blink lights. Desire to support /dev/led Allan is interested.
Q: Should kern.disks sysctl or something like it show non-hard disk block devices like zvols and memory devices?
A: Unknown. geom disk list is the closest.
Q: Does it build with CLANG?
A: GRUB/grub-bhyve/u-boot, UEFI and Xen DO NOT. Attention to this would be appreciated now that we increasingly depend on these third party tools.
Unrelated: Marcello had various PostgreSQL on ZFS input: Number one: Don't let the ARC starve PG of RAM.
Thank you all. It was an honor to run this 1/2 day session. - Michael Dexter