• BuildingFreeBSDWithClang

Building FreeBSD with clang/llvm

FreeBSD world and kernel can be built with clang/llvm. Clang is a compiler built on the LLVM compiler infrastructure. Both clang and llvm are released under a BSD like license.

Status

Clang has been imported into head and stable/9, and is built by default, so there is no need to install the port anymore, unless you want to play with the other tools that llvm provides. Alternatively, you can set WITH_CLANG_EXTRAS in src.conf, to build most of the additional tools.

An unmodified FreeBSD head and stable/9 can completely be built by clang! (Note that you still need to setup /etc/make.conf or /etc/src.conf properly, see below for the details.)

Newer snapshots of clang/llvm are imported regularly into head, and will be merged to stable branches, as appropriate. FreeBSD 9.0 is the first release to contain clang by default.

All of userland (plus clang/llvm itself) and the kernel compiles and runs ok. The amd64, i386, powerpc64 and arm kernels boot multiuser.

ARM works with the current ABI (OABI) and ARM EABI.

PowerPC32 kernel needs a patched version of llvm (not saving FP registers on call func).

A buildbot that runs on-commit builds and boot tests can be found here.

Please note that cross compiling is not yet supported by clang (or at least, it is not easy to do out of the box).

Quickstart

Checkout head:

# svn co http://svn.freebsd.org/base/head src

Add the following lines to /etc/make.conf (if you want to use clang for everything, even ports), or /etc/src.conf (if you want to use clang just for world and kernel):

CC=clang
CXX=clang++
CPP=clang-cpp

(NOTE: Do *not* use "clang -E" for CPP, this will not work correctly. You will almost certainly get RPC-related build errors.)

As of r233419, head should build without -Werror bailing out the build, e.g. there should be no unexpected warnings. If you are attempting to build something earlier, or encounter -Werror bailouts anyway, add the following to /etc/make.conf or /etc/src.conf, as appropriate:

# This setting to build world without -Werror:
NO_WERROR=
# This setting to build kernel without -Werror:
WERROR=

If you are building jails, the following can make life easier by not setting system-immutable flag on various important system files. This may have serious security implications depending on your use, (schg/nouchg are explained in chflags(1), the jail(8) man page exlains how jails implement BSD securelevel):

# Does not set schg bit on various system files, 
# useful for building Jails, has security implications.
NO_FSCHG=

Verify that "/usr/bin/clang -v" works:

FreeBSD clang version 3.0 (tags/RELEASE_30/final 145349) 20111210
Target: x86_64-unknown-freebsd10.0
Thread model: posix

and produces output similar to the above.

Building/installing world (into a chroot).

# make buildworld
# make installworld DESTDIR=/usr/obj/clang
# chroot /usr/obj/clang /bin/echo Hello clang world!
Hello clang world!
# chroot /usr/obj/clang /bin/tcsh

Building/installing kernel.

# make kernel KERNCONF=GENERIC INSTKERNNAME=clang

Boot it.

OK set module_path=/boot/clang
OK boot clang

To update the source tree perform the following steps.

# cd src/
# svn up

Using the static analyzer

On the kernel (This requires lots of space in the output directory, 406M when using "scan-build make MODULES_OVERRIDE=").

# cd /sys/{amd64,i386}/conf/
# config GENERIC
# cd ../compile/GENERIC/
# make depend
# scan-build make MODULES_OVERRIDE=
# {firefox,epiphany,konqueror,lynx,less} /tmp/scan-build-YYYY-MM-DD-N/index.html

On a userland program.

# cd /usr/src/usr.bin/make/
# make cleandir && make obj && make depend
# scan-build make
# {firefox,epiphany,konqueror,lynx,less} /tmp/scan-build-YYYY-MM-DD-N/index.html

Using the analyzer on vanilla sources

Scanning the FreeBSD sources using the static analyzer requires you to:

# svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm
# cd llvm/tools && svn co http://llvm.org/svn/llvm-project/cfe/trunk clang
# cd .. && ./configure --enable-optimized && gmake
# cp -p tools/clang/tools/scan-build/{ccc-analyzer,scan-build,scanview.css,sorttable.js} Release+Asserts/bin
# ln -sf ccc-analyzer Release+Asserts/bin/c++-analyzer

Now you have all the necessary tools in llvm/Release+Asserts/bin. They need to be next to each other, as we will be using the full-path to scan-build later and it must find clang-cc, which it searches in it's own directory and $PATH. As $PATH will be cleansed during buildworld, only the first option remains.

Running the analyzer is now as easy as

# cd /usr/src && /path/to/llvm/Release+Asserts/bin/scan-build -k make buildworld

Weekly scans of buildworld and buildkernel can be studied at http://scan.freebsd.your.org/freebsd-head. The rate of false positives is decreasing and LLVM is working actively on fixing them. You can help by grabbing a report and working out if it's valid or false positive. Then send a bug report to either LLVM or FreeBSD and add it to the list below.

Here's a list of known false positives:

Patches to FreeBSD:

Link Time Optimizations

LLVM supports LTO. It can be used for kernel build by these pseudo steps:

0) build gold and libLLVMgold.so (see http://llvm.org/docs/LinkTimeOptimization.html)
1) CFLAGS+=-emit-llvm
2) setenv LD "/tmp/ld-new --plugin /tmp/libLLVMgold.so"
3) setenv CC clang
4) build the kernel

This was tested on i386. The resulting kernel does boot.

Automatic test generation with KLEE

KLEE is a symbolic virtual machine built on top of the LLVM compiler infrastructure. More about KLEE here and paper here.

Building KLEE

# svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm
# svn co http://llvm.org/svn/llvm-project/cfe/trunk llvm/tools/clang
# svn co http://llvm.org/svn/llvm-project/klee/trunk klee
# cd llvm/
# ./configure --prefix=/usr/local --enable-optimized --enable-targets=host-only --enable-bindings=none
# gmake
# gmake install
# cd ../klee/
# ./configure --with-llvm=$PWD/../llvm
# gmake LLVMGCC=/usr/local/bin/clang
# cp Release+Asserts/bin/* /usr/local/bin/
# cp Release+Asserts/lib/* /usr/local/lib/

Using KLEE

# cd /usr/src/usr.bin/touch/
# clang -c -g -emit-llvm -o touch.bc touch.c
# klee touch.bc
KLEE: output directory = "klee-out-0"
...

More to come later...

Known issues

• Slow build times when two compilers are enabled. Building clang will always be slower than building gcc 4.2.1.

> clang -v
FreeBSD clang version 3.0 (tags/RELEASE_30/final 145349) 20111210
Target: x86_64-unknown-freebsd9.0
Thread model: posix
> gcc -v
Using built-in specs.
Target: amd64-undermydesk-freebsd
Configured with: FreeBSD/amd64 system compiler
Thread model: posix
gcc version 4.2.1 20070831 patched [FreeBSD]
> dmesg | head 
Copyright (c) 1992-2012 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.0-STABLE #0: Sat Mar 17 09:14:25 UTC 2012
    root@leopard1.netperf.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
CPU: Intel(R) Xeon(R) CPU            3050  @ 2.13GHz (2128.04-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x6f6  Family = 6  Model = f  Stepping = 6
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0xe3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM>

Contacts

• irc://irc.oftc.net/#freebsd-clang

• irc://irc.oftc.net/#llvm

• RomanDivacky, PawelWorach, EdSchouten and DimitryAndric

Resources

• Introduction to the LLVM Compiler System

• State of Clang

• LLVM 2.0 and Beyond!(Google Tech Talk Video)

• LLVM New C Front-end part 1234

  • Script to automatically apply the patches in a Tinderbox environment

• For more details on compiling the ports with clang/llvm see: PortsAndClang

CategoryHistorical