bhyve Windows Virtual Machines
The bhyve hypervisor now supports Microsoft Windows virtual machines as of FreeBSD HEAD SVN tag r288524
Supported Windows Versions
bhyve supports Windows desktop versions Vista, 7, 8/8.1/8.2 and 10, as well as Windows Server versions 2008/2008R2, 2012/2012R2 and 2016 Technical Preview 2 and 3
- A Windows ISO and matching license key
An AutoUnattend.xml automatic installation file
- A Windows VirtIO network driver
A RDP Remote Desktop Protocol client (net/freerdp)
At this time of writing, the Windows Sever 2016 Technical Previews provide the easiest means of testing Windows support with bhyve. The Technical Previews:
- Are free to download and use
- Are automatic installation-friendly
- Include the Special Administration Console (SAC) (The windows serial console)
- Are meant to be tested!
Manually copy or GIT clone (requires the git package) a prepared AutoUnattend.xml file for your version of Windows
git clone https://github.com/nahanni/bhyve-windows-unattend-xml
Fetch the following Windows VirtIO driver ISO
Windows 10 requires virtio-win-0.1.118:
Later releases have various issues with various Windows versions.
Windows Vista requires a previous version of the drivers:
With the Windows ISO, XML auto installation file and VirtIO obtained, the ISO is ready to be remastered. 7z is suggested because it is UDF-friendly, unlike the in-base tar(1).
The remastering steps are as follows:
mkdir -p iso-remaster/virtio # Note the DOS syntax and replace <windows>.iso with the proper file name: /usr/local/bin/7z x <windows>.iso -oiso-remaster # The 2012R2 and later driver: tar xf virtio-win-0.1.96.iso --include="NetKVM/2k12R2/amd64/*" -C iso-remaster/virtio/ cp bhyve-windows-unattend-xml/files/win2016preview_AutoUnattend.xml iso-remaster/AutoUnattend.xml mkisofs \ -b boot/etfsboot.com -no-emul-boot -c BOOT.CAT \ -iso-level 4 -J -l -D \ -N -joliet-long \ -relaxed-filenames -v \ -V "Custom" -udf \ -boot-info-table -eltorito-alt-boot -eltorito-platform 0xEF \ -eltorito-boot efi/microsoft/boot/efisys_noprompt.bin \ -no-emul-boot \ -o install.iso iso-remaster
This will result in a remastered Windows ISO named install.iso
You will also require the EFI loader and a disk image:
pkg install uefi-edk2-bhyve truncate -s 15GB windows2016.img
Note that 15GB will only leave 4GB free space. Consider far more for production use.
Windows ISO Boot
The install.iso is only required for the first boot of Windows Server and must be removed from the bhyve command after the first boot. Desktop editions of Windows require that a null install.iso file remains and it can be created with touch install.iso
The trick is to have the CD configured only for the first boot. It has to be removed from the bhyve command line for the second and subsequent boots, since there isn't currently a boot selection in UEFI and a CD is automatically given preference, resulting in an endless cycle of CD installs. For Windows-desktop installs, the ahci-cd drive *must* be kept there, and a 0-byte file used instead e.g. 'touch null.iso'.
There are currently some slot limitations with UEFI:
- - AHCI devices must be in slots 3/4/5/6 - The PCI-ISA bus aka lpc must be in slot 31 - virtio-net devices can be in any slot
virtio-block devices have an issue with descriptor overflow that is being worked on, so avoid using it for the time being.
A typical bhyve command for Windows is:
bhyve \ -c 2 \ -s 0,hostbridge \ -s 3,ahci-hd,windows2016.img \ -s 4,ahci-cd,install.iso \ -s 10,virtio-net,tap0 \ -s 31,lpc \ -l com1,/dev/nmdm0A \ -l com2,/dev/nmdm1A \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ -m 2G -H -w \ windows2016
Only one or two VCPUs should be used during installation but this number can be increased once Windows is installed.
This should produce many pages of EFI debugging information and land you in the Windows SAC:
Computer is booting, SAC started and initialized. Use the "ch -?" command for information about using channels. Use the "?" command for general help. SAC> EVENT: The CMD command is now available. SAC> EVENT: A new channel has been created. Use "ch -?" for channel help. Channel: SACSetupAct SAC> EVENT: A new channel has been created. Use "ch -?" for channel help. Channel: SACSetupErr
The initial SAC> prompt will appear immediately, followed by the three EVENTs.
Each SAC channel performs a different role. You can list available channels at any time with:
SAC>ch Channel List (Use "ch -?" for information on using channels) # Status Channel Name 0 (AV) SAC 1 (AV) SACSetupAct 2 (AV) SACSetupErr
You can toggle between channels with ESC-TAB.
Some useful SAC commands to shutdown, display the network configuration (once Windows is installed) and set a fixed IP on the 3rd network device are:
SAC>shutdown SAC>i SAC>i 3 192.168.1.230 255.255.255.0 192.168.1.1
The CMD event indicates that the Windows command line is available. Typing cmd will create this channel and should get:
If you only see the CMD channel with another version of Windows, your installation has probably had an error with regard to license keys. To exit the CMD channel back to SAC, type exit.
The SACSetupAct is most useful during installation as it will show the installation steps and is your best indication of success or failure. The SACSetupErr channel will provide the specific error should one occur.
The process will move quickly until the main Windows image extraction begins:
... 2015-10-10 20:58:33, Info [0x06009e] IBS DeployWIMImage:Calling IDepWI MImageResolved::Apply... 2015-10-10 20:58:33, Info [0x0606cc] IBS Calling WIMApplyImage (flags = 0x180)...
This will cause moderate disk activity on the host and high CPU activity because of the image decompression. These can be monitored with gstat and top to get a sense of a "normal" installation. This knowledge will be useful for blind installations of desktop versions of Windows that do not include the SAC.
If you attempt to boot to a blank disk image, you will see:
Booting EFI Network InstallProtocolInterface: 245DCA21-FB7B-11D3-8F01-00A0C969723B BEC7C0E0 .
Windows Hard Disk Boot
Remove or adjust the ahci-cd,install.iso line of the bhyve command as appropriate and allow the VM to boot to the hard disk image. This will involve a SYSPREP boot that will attempt a restart. Boot the system again and you should eventually get a fully-installed system that will default to DHCP networking and be available by CMD in the SAC and RDP.
From the CMD:
Please enter login credentials. Username: Administrator Domain : Password: *******
Note that the delete key does not work and you must type carefully. The default password in the XML file is "Test123"
You can determine the DHCP IP of the VM from the SAC:
SAC>i Net: 3, Ip=126.96.36.199 Subnet=255.255.255.0 Gateway=192.168.1.1 Net: 3, Ip=fe80::b423:80d7:e553:8b02
Note that you might not be able to ping the Windows virtual machine because of the default firewall configuration.
You can initiate an RDP connection to this IP address with:
xfreerdp /v:188.8.131.52 /u:Administrator /p:Test123
The Windows, OSX, iOS and Android RDP clients from Microsoft will also work.
Note that xfreerdp stores host fingerprints in a manner similar to OpenSSH and will prevent you from connecting to a host whose fingerprint has changed. These keys can be managed in ~/.config/freerdp/known_hosts}
Windows Server "Essentials" and other variations have one installation image rather than four. These will give a "missing image" error during installation if you do not set the correct "IMAGE/INDEX" value to "1":
<MetaData wcm:action="add"> <Key>/IMAGE/INDEX</Key> <Value>1</Value> </MetaData>
You can list the available install images on a given ISO with:
7z l iso-remaster/sources/install.wim|more ... Images = 4 ...
This page is an expansion of this documentation: