• DevSummit
• 201608
• Toolchain

Toolchain working group

Action items

• bdrewery to change --sysroot (always pass) r304681

• jonathan to add LLVM instrumentation rules to build system

• someone (?) to test use of absolute paths in build
• someone (?) to test turning on BUILD_SHARED_LIBS
• kib to profile Clang startup with BUILD_SHARED_LIBS
• theraven to find CMake option for building LLVM libs as single .so
• anything about arm64 linker package building?
• bdrewery to continue working on cross-OS building
• emaste to work on LLDB for kernel (based on previous GSoC project)

discussion

• build systems
  • proposal (Bryan): always pass --sysroot, even to bootstrapped cross-compiler
    • Q: stop defining sysroot, etc., in cross-compiler?
    • Q: use absolute paths for build failure reproducibility?
    • reproduciblity: already need to give up on paths
    • linking: linker might not be able to handle that many absolute pathnames (command line args too long)
    • brooks: worthwhile experiment, may not work for, e.g., lib32
• LLVM
  • BUILD_SHARED_LIBS?
    • sometimes required to build a Debug Clang on machines with 4GB or 8GB of RAM
    • some LLVM tools (e.g., lldb) broken with 2
    • Apple has an option to build one big LLVM shared library rather than lots of tiny ones
    • if we fix BUILD_SHARED_LIBS for lldb, we can set up a build bot and others might stop breaking it
      • build bots do exist but could stand some improvement
    • we don't want to slow the build down more
      • is it still useful to have a static build toolchain?
      • might need to profile this 20s startup time that brooks has observed (RTLD? FS cache?)
      • job for kib
      • theraven thinks there might be a CMake option to build a single .so
  • instrumentation
  • /usr/bin/opt?
    • goes hand-in-hand with BUILD_SHARED_LIBS!
    • might be tough to support opt, etc., for five years
    • non-default package for packaged base
    • emaste would like llvm-objdump, etc.
  • Linkers
    • lld can link userland on x86_64 (except some funny boot bits), non-booting kernel
    • linker script support getting in good shape
    • will likely be viable for FreeBSD 12
    • self-hosting on arm64, 32b "getting there" ("past hello world")
    • not particularly close on PowerPC, MIPS: might require external toolchain for 12
    • improvements/benefits
      • ten years of better linking
      • LTO
    • requirements
      • version scripts require wildcards on demangled C++ symbols (everything else should be fine)
      • linker scripts (mainly for the kernel)
      • LTO is a nice-to-have, not a requirement, but a *relly nice* to-have.
    • there is an LLVM meta-bug for building FreeBSD
    • 11 for arm64 will ship without a base system linker (old GNU ld doesn't support arm64!)
      • will need to ship packages
      • there are machines to build these packages, but there are... issues (firmware, etc.)
• Cross-OS builds
  • hardest part: bootstrapping system to have FreeBSD-like environment (including a libcompat)
  • two problems
    • need a C++11 compiler (which most modern Linux distros have, but RHEL is a bit out of date)
    • portability
      • need BSD stuff in libc
      • there is a portable version of mtree
      • need a portable crunchgen (or not use crunchgen)
      • NetBSD has this kind of thing

        • cperciva: could we use NetBSD as our compatibility layer?

  • additional problems
    • UFS
      • can't read the filesystem you just built
      • one approach: dump/restore + libarchive
      • another approach: FUSE
      • another approach: rump kernel
      • need a bi-endian UFS
  • "make tools" for cross-OS building can be huge, manual and explicit
    • compat should still be optimized for the case of building on FreeBSD
• LLDB
  • in base for 11 (amd64, arm64, not i386)
  • Missing single step on arm32, horrible user experience
  • missing functionality:
    • we don't parse FP registers in core files
  • gdbserver
    • on Mac OS X and Linux, all debugging is remote debugging
    • Facebook has DS2, which may be a useful target for FreeBSD inclusion once complete
    • Keeping lldb/gdb-server separate makes debugging threaded code cleaner
  • kernel support doesn't exist yet: there was a GSoC project, emaste to work on it
• De-GPL
  • libunwind enabled in -CURRENT
  • lack of tools for debugging libunwind: they don't understand EHABI
  • binutils: still install as, ld, and objdump
    • as: no replacement
    • can just turn it off for amd64
      • 11 ports failed without a /usr/bin/as
      • consensus: force failing ports to use binutils
      • some parts of i386 use /usr/bin/as rather than Clang
        • might be able to switch to .S
    • mips64
      • upstream binutils doesn't understand freebsd-mips64
      • project: extract base system's binutils into separate repository, make port
      • devel-binutils-mips64 as-is doesn't work
    • sparc64
      • no actual relationship to currently-shipping hardware
      • last Ultra3 EOL'ed by Sun in 2008
      • plan
        • emaste to turn of /usr/bin/as for amd64, arm64
        • incrementally migrate to external binutils
        • expunge GNU as
        • lld will become ld
    • llvm-objdump
      • caught up (sort of) to GNU objdump about a week ago
      • different command-line arguments, slightly less polished

      • lack of SafeStack, libssp

        • possibly too broken to care about
    • diff, rcs
      • replaced ident this week
      • there is an OpenRCS
      • strategy: once rcs is all that's left, then we can remove it
    • dtc
      • theraven's dtc currently disabled
      • doesn't always support /include
      • lots of bugs when run over arm dts files
    • libomp
      • llvm's implementation usable (the one Intel open-sourced)
      • discussion ongoing about port vs base
      • possible consensus: use ports
      • minor blockers
        • will need to build it twice
        • currently built as part of the LLVM port, but users might object to installing the LLVM static mega-package (1 GB)
    • libcov
    • gperf
      • build tool for GCC that creates perfect hashes
    • grep: bsdgrep
      • Apple has some grep fixes
      • fairly easy to crash it
    • groff
      • has been removed, man pages need updating
      • LLVM's test-suite needs groff (but it's in ports)
    • libgcc
      • not using compiler-rt for the shared version
      • we should try to keep it up-to-date with modern libgcc (some GNU tools need this)
    • readline
      • only gdb uses this
• LLVM fat binaries
  • Leave LLVM IR into the objects, so we can do later optimisations
  • Research quality for most tools that can consume it, so maybe not priority now
  • Nevertheless, interesting to have the option to, so further experimentation can happen
  • Single-architecture bitcode, not for cross-architecture late generation
  • May be hard to hold on LLVM to micro-optimise, since it defaults to the most common sub-arch not the minimal one
• sanitizers
• modularization
• QEMU

CategoryHistorical