[GSoC 22] bhyve debug server enhancements documentation
AMD SVM single-stepping support
The GDB stub uses debug exception VM exits caused by the RFLAGS.TF bit to single-step the guest.
Stepping is requested using the VM_CAP_RFLAGS_SSTEP optional VM capability. Activating this capability traces the next instruction as the SVM-specific part of the vmm module sets the guest vCPU's RFLAGS.TF bit to 1. The vmm module also keeps a "shadow" copy of the original TF bit and restores it when the VM_CAP_RFLAGS_SSTEP capability gets deactivated.
Special care was taken to ensure that the shadow TF bit value remains consistent while the guest is executing. Activating the VM_CAP_RFLAGS_SSTEP capability also activates POPF/PUSHF intercepts. After intercepting these instructions, the ~vmm~ module records the intercept, restarts, traces the instruction, and updates the TF bit in the svm.c debug vmexit handler. The ~vmm~ module also updates values written to memory by PUSHF with the shadow TF bit value.
This mechanism suffers from the same problem as its Intel VMX counterpart - single-stepping immediately lands inside a timer interrupt.
GDB stub hardware watchpoint support
The GDB stub supports four hardware watchpoints on all Intel and AMD guests. The limited number of watchpoints stems from hardware limitations - each x86_64 platform will have (at least) four debug registers (DR[1-3]) (excluding AMD 64 bit extensions) intended for hardware watchpoints.
The main goal of this feature is to provide more refined debugging capabilities while making sure that the guest executes correctly. This means that the GDB stub must "release" the debug registers when a guest starts using them.
The GDB stub collects and maintains debug register usage data from all vCPUs. This data is used to track available watchpoints and each watchpoint is mapped to its corresponding debug register (i.e., watchpoint 0 -> DR0, and so on). If there are, for instance, several vCPUs and only one of them is currently using debug register DR0, the first watchpoint is marked as unavailable until that vCPU stops using the debug register.
When setting a watchpoint, the GDB stub selects an available watchpoint and sets the corresponding debug register on all vCPUs. Maintaining the correctness of a guest's execution required userspace and kernel-side support for intercepting and processing debug register accesses (exitcode VM_EXIT_MOV_DR) and debug exceptions (exitcode VM_EXIT_DB). The GDB stub requests forwarding of these exits to userspace using the VM_CAP_DR_MOV_EXIT and VM_CAP_DB_EXIT optional VM capabilities. The platform-dependent parts of the vmm module (vmx.c and svm.c) activate these VM exits, handle them once they occur, and forward them to userspace along with relevant VM exit data. They also emulate MOV DR* instructions after they are intercepted and before the vm exit is forwarded to userspace.
The guest can trigger several events which must be handled when a GDB stub watchpoint is active:
A debug register read
- If the guest attempts to read a debug register that is currently used by the GDB stub, the GDB stub sets the contents of the target GPR to zero to avoid any misbehavior.
If the guest attempts to read the DR7 register, the GDB stub clears each active watchpoint from the target GPR.
- If the guest attempts to read a debug register that is currently used by the GDB stub, the GDB stub sets the contents of the target GPR to zero to avoid any misbehavior.
A debug register write
- If the guest writes a non-zero value to a debug register currently used by the GDB stub,
the watchpoint gets deactivated, marked as "evicted", and cleared from the DR7 registers on all vCPUs. If the guest writes the DR7 register, the GDB stub checks all newly activated guest watchpoints and repeats the previously described watchpoint deactivation procedure. The debug register write handler also attempts to "migrate" all "evicted" watchpoints by placing them into any available debug register.
- If the guest writes a non-zero value to a debug register currently used by the GDB stub,
A guest watchpoint hit
- Each watchpoint hit gets forwarded to the GDB stub. If the hit was caused by a GDB stub watchpoint, all vCPUs get suspended and the stop reason is reported to the remote GDB client. All other watchpoint hits are reflected into the guest.