Session Chair: David Chisnall.
Andrew, open source engagement. Vas, misc optimisation Robin, power management (leading session tomorrow)
Julien (Citrix), Xen things Andy T (FreeBSD), ARMv8 bringup
Xen on Arm
Julien Grall gave a demo of the FreeBSD/ARM/Xen port. Port has been underway for one year. Working for Linaro, decided to port a new OS because Xen on ARM only supported Linux.
Cortex A15 board used for a demo.
Xen on x86 emulates devices via QEMU. On ARM, different approach. Clean support for virtualisation in the ISA. HVM for page tables, interrupt controller. All devices (disk, net) use Xen PV drivers. No device emulation, operating systems must support the devices (already present in FreeBSD/x86). (Quick Xen/x86 update: Able to boot dom0)
Demo, boots FreeBSD on Xen on ARM. Everything works except SMP (for now!). Uses PSCI (firmware) for bringing up second CPUs, not yet supported by FreeBSD.
Demo 2: Boots FreeBSD in dom0 (surprised, but happy, that it works). Boots Linux guest (domU) with FreeBSD dom0 on Xen.
Lots of things missing from FreeBSD/ARM. Some device tree support. Device rebinding is not correctly working. Multiplatform kernel not yet working. The memory layout of the guests isn't fixed.
Most important thing: Need a multiplatform that can run anywhere in memory. Andy T says it sort-of works, as long as the kernel start is aligned on a 1MB boundary. There are some physical addresses embedded in the config file. Discussion to follow on the mailing lists.
Samsung did a fully PV port of Xen to ARM, but the goal for Citrix is to have a thin Xen on ARM.
Device tree files were written by different groups and have diverged. Linux and FreeBSD expect slightly different descriptions. These need unifying.
Uses UEFI bootloader, which loads the kernel in the ARMv8 simulator. Kernel boots to mountroot prompt. Currently missing device drivers for... everything. UART is working through early_printf (hacked up). Missing lots of other things, but basic support now exists. pmap is mostly done, but possibly needs some reworking for efficiency.
pmap code written by implementing all functions as panic(), then implementing them as they were hit. fork() should work. Usermode code will be broken because the model doesn't do anything with caches yet. There's a vector for the exception handlers, but no working signal delivery.
rwatson asks about the state of the toolchain. We need something that's not binutils. LLVM/Clang can compile, but we're currently using a GPLv3 binutils, which can't go into the tree. Especially linker and objcopy. Lldb is expected to work on ARMv8, but might not be feature complete.
Robin mentioned that he replicated the setup, but with the emulator running on FreeBSD's Linux ABI layer. ARM has base system architecture, Robin isn't sure what the bootloader requirements are. Andrew mentions that the SBSA just specifies [U]EFI, but there's heated debate over ACPI vs FDT. ACPI is preferred for enterprise, based on ARM customer / partner design, existing x86 toolsets. ARM doesn't want to see ACPI and FDT in the same device.
What would FreeBSD prefer? Is ACPI preferred? If it can be used from the outset, this would make things easier, but in the mobile space FDT might be more important. Intel Galileo is doing the opposite: trying to add FDT to a platform that's historically ACPI. Pre-ARMv8 the dominant market was mobile, FDT was where this ended up. ARMv8 is pushing more towards the server, where ACPI lives. v8-mobile is still undecided.
It's important to have the discussion early about how FDT and / or ACPI can be maintained in the long run. rwatson mentions that device drivers don't care about ACPI vs FDT. Robin says that the differences can be important for power management. FDT is starting to look quite dated (static configurations of busses don't exist so much).
Help needed for:
- Getting loader working
- Getting real hardware (ARM might be able to help, some Juno boards are starting to appear)
- Interrupt controller support
Foundation model can't be adjusted, the foundation model doesn't do cache simulation by default - might not be enabled for the public releases.
What are the near-term goals? AArch32 support on ARMv8? Not immediately, but soon. Would be useful for package builds (32-bit jail). Probably won't support AArch32 kernel on ARMv8 hardware. Hoping to have a single kernel from the scratch, use ACPI / FDT to detect hardware on boot.
Andrew is trying to find all of the things Linux did wrong so that FreeBSD can learn from their mistakes (and things they did right, so we can do those).
The second session is a brainstorming session for things that need doing.
- Can ARM relicense (or BSD license from the start) optimised memcpy and similar functions?
- How do we get access to hardware? ARM will talk to partners to see if there is anything available.
- Foundation model (emulator) is free-to-use, proprietary. Works on FreeBSD Linux ABI layer. Can we create a port (distfile is not redistributable).
- QEMU ARMv8 model is now present
- AArch64 package building
- QEMU user mode
- Newer v7 cores - AArch32 isn't going away any time soon, worth identifying ones that are interesting to support.
- Is the focus on server or mobile / embedded?
- Server short term.
- Other things longer term.
- 64-bit Chromebook (once it exists) would be very interested
- Server-important features:
- Virtualisation (byhve, Xen) support
- Single kernel for all boards
- SMP (PSCI?)
- Someone to do code review of ARMv8 pmap?
- ARM to identify pitfalls to help avoid them for FreeBSD.
- ARM engineers can answer some questions (over beer?)
- DTrace port?
- Xen port observations
- Can we unify v8 and v7 pmap?
- Xen changes should be the same on both
- LPAE support for ARMv7 would be useful
- Can anything else be shared? Exception levels are very different.
- Interrupt controller? GICv3 closely tied to PCIe, GIC not tied to architecture?