bhyve Windows Virtual Machines

The bhyve hypervisor now supports Microsoft Windows virtual machines as of FreeBSD HEAD SVN tag r288524

Supported Windows Versions

bhyve supports Windows desktop versions Vista, 7, 8/8.1/8.2 and 10, as well as Windows Server versions 2008/2008R2, 2012/2012R2 and 2016 Technical Preview 2 and 3

Requirements

At this time of writing, the Windows Sever 2016 Technical Previews provide the easiest means of testing Windows support with bhyve. The Technical Previews:

ISO Remastering

Manually copy or GIT clone (requires the git package) a prepared AutoUnattend.xml file for your version of Windows

git clone https://github.com/nahanni/bhyve-windows-unattend-xml

Fetch the stable Windows VirtIO driver ISO

fetch https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/virtio-win-0.1.96/virtio-win-0.1.96.iso

Later releases have various issues with various Windows versions.

Windows Vista requires a previous version of the drivers:

fetch https://fedorapeople.org/groups/virt/virtio-win/deprecated-isos/archives/virtio-win-0.1-94/virtio-win-0.1-94.iso

With the Windows ISO, XML auto installation file and VirtIO obtained, the ISO is ready to be remastered. 7z is suggested because it is UDF-friendly, unlike the in-base tar(1).

The remastering steps are as follows:

mkdir -p iso-remaster/virtio

# Note the DOS syntax and replace <windows>.iso with the proper file name:
/usr/local/bin/7z x <windows>.iso -oiso-remaster

# The 2012R2 and later driver:
tar xf virtio-win-0.1.96.iso --include="NetKVM/2k12R2/amd64/*" -C iso-remaster/virtio/

cp bhyve-windows-unattend-xml/files/win2016preview_AutoUnattend.xml iso-remaster/AutoUnattend.xml

mkisofs \
     -b boot/etfsboot.com -no-emul-boot -c BOOT.CAT \
     -iso-level 4 -J -l -D \
     -N -joliet-long \
     -relaxed-filenames -v \
     -V "Custom" -udf \
     -boot-info-table -eltorito-alt-boot -eltorito-platform 0xEF \
     -eltorito-boot efi/microsoft/boot/efisys_noprompt.bin \
     -no-emul-boot \
     -o install.iso iso-remaster

This will result in a remastered Windows ISO named install.iso

You will also require the EFI loader and a disk image:

pkg install uefi-edk2-bhyve
truncate -s 15GB windows2016.img

Note that 15GB will only leave 4GB free space. Consider far more for production use.

Windows ISO Boot

The install.iso is only required for the first boot of Windows Server and must be removed from the bhyve command after the first boot. Desktop editions of Windows require that a null install.iso file remains and it can be created with touch install.iso

The trick is to have the CD configured only for the first boot. It has to be removed from the bhyve command line for the second and subsequent boots, since there isn't currently a boot selection in UEFI and a CD is automatically given preference, resulting in an endless cycle of CD installs. For Windows-desktop installs, the ahci-cd drive *must* be kept there, and a 0-byte file used instead e.g. 'touch null.iso'.

There are currently some slot limitations with UEFI:

virtio-block devices have an issue with descriptor overflow that is being worked on, so avoid using it for the time being.

A typical bhyve command for Windows is:

bhyve \
      -c 2 \
      -s 0,hostbridge \
      -s 3,ahci-hd,windows2016.img \
      -s 4,ahci-cd,install.iso \
      -s 10,virtio-net,tap0 \
      -s 31,lpc \
      -l com1,/dev/nmdm0A \
      -l com2,/dev/nmdm1A \
      -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
      -m 2G -H -w \
      windows2016

Only one or two VCPUs should be used during installation but this number can be increased once Windows is installed.

This should produce many pages of EFI debugging information and land you in the Windows SAC:

Computer is booting, SAC started and initialized.                               
                                                                                
Use the "ch -?" command for information about using channels.
Use the "?" command for general help.

SAC>
EVENT: The CMD command is now available.
SAC>
EVENT:   A new channel has been created.  Use "ch -?" for channel help.
Channel: SACSetupAct
SAC>
EVENT:   A new channel has been created.  Use "ch -?" for channel help.
Channel: SACSetupErr

The initial SAC> prompt will appear immediately, followed by the three EVENTs.

Each SAC channel performs a different role. You can list available channels at any time with:

SAC>ch
Channel List

(Use "ch -?" for information on using channels)

# Status  Channel Name
0 (AV)    SAC
1 (AV)    SACSetupAct
2 (AV)    SACSetupErr

You can toggle between channels with ESC-TAB.

Some useful SAC commands to shutdown, display the network configuration (once Windows is installed) and set a fixed IP on the 3rd network device are:

SAC>shutdown
SAC>i
SAC>i 3 192.168.1.230 255.255.255.0 192.168.1.1

The CMD event indicates that the Windows command line is available. Typing cmd will create this channel and should get:

X:\windows\system32>

If you only see the CMD channel with another version of Windows, your installation has probably had an error with regard to license keys. To exit the CMD channel back to SAC, type exit.

The SACSetupAct is most useful during installation as it will show the installation steps and is your best indication of success or failure. The SACSetupErr channel will provide the specific error should one occur.

The process will move quickly until the main Windows image extraction begins:

...
2015-10-10 20:58:33, Info       [0x06009e] IBS    DeployWIMImage:Calling IDepWI
MImageResolved::Apply...
2015-10-10 20:58:33, Info       [0x0606cc] IBS    Calling WIMApplyImage (flags 
= 0x180)...

This will cause moderate disk activity on the host and high CPU activity because of the image decompression. These can be monitored with gstat and top to get a sense of a "normal" installation. This knowledge will be useful for blind installations of desktop versions of Windows that do not include the SAC.

If you attempt to boot to a blank disk image, you will see:

Booting EFI Network
InstallProtocolInterface: 245DCA21-FB7B-11D3-8F01-00A0C969723B BEC7C0E0
.

Windows Hard Disk Boot

Remove or adjust the ahci-cd,install.iso line of the bhyve command as appropriate and allow the VM to boot to the hard disk image. This will involve a SYSPREP boot that will attempt a restart. Boot the system again and you should eventually get a fully-installed system that will default to DHCP networking and be available by CMD in the SAC and RDP.

From the CMD:

Please enter login credentials.
Username: Administrator
Domain  : 
Password: *******

Note that the delete key does not work and you must type carefully. The default password in the XML file is "Test123"

You can determine the DHCP IP of the VM from the SAC:

SAC>i
Net: 3, Ip=192.186.1.14  Subnet=255.255.255.0  Gateway=192.168.1.1
Net: 3, Ip=fe80::b423:80d7:e553:8b02

Note that you might not be able to ping the Windows virtual machine because of the default firewall configuration.

RDP Connection

You can initiate an RDP connection to this IP address with:

xfreerdp /v:192.186.1.14 /u:Administrator /p:Test123

The Windows, OSX, iOS and Android RDP clients from Microsoft will also work.

Note that xfreerdp stores host fingerprints in a manner similar to OpenSSH and will prevent you from connecting to a host whose fingerprint has changed. These keys can be managed in ~/.config/freerdp/known_hosts}

Notes

Windows Server "Essentials" and other variations have one installation image rather than four. These will give a "missing image" error during installation if you do not set the correct "IMAGE/INDEX" value to "1":

<MetaData wcm:action="add">
   <Key>/IMAGE/INDEX</Key>
   <Value>1</Value>
</MetaData>

You can list the available install images on a given ISO with:

7z l iso-remaster/sources/install.wim|more
...
Images = 4
...

This page is an expansion of this documentation:


CategoryVirtualization

bhyve/Windows (last edited 2016-07-16 16:12:08 by FabianFreyer)