Building FreeBSD with clang/llvm

FreeBSD world and kernel can be built with clang/llvm. Clang is a compiler built on the LLVM compiler infrastructure. Both clang and llvm are released under a BSD like license.

Status

Clang has been imported into head and stable/9, and is built by default, so there is no need to install the port anymore, unless you want to play with the other tools that llvm provides. Alternatively, you can set WITH_CLANG_EXTRAS in src.conf, to build most of the additional tools.

An unmodified FreeBSD head and stable/9 can completely be built by clang! (Note that you still need to setup /etc/make.conf or /etc/src.conf properly, see below for the details.)

Newer snapshots of clang/llvm are imported regularly into head, and will be merged to stable branches, as appropriate. FreeBSD 9.0 is the first release to contain clang by default.

All of userland (plus clang/llvm itself) and the kernel compiles and runs ok. The amd64, i386, powerpc64 and arm kernels boot multiuser.

ARM works with the current ABI (OABI) and ARM EABI.

PowerPC32 kernel needs a patched version of llvm (not saving FP registers on call func).

A buildbot that runs on-commit builds and boot tests can be found here.

Build status

amd64

i386

arm OABI

arm EABI

powerpc

powerpc64

mips

mips64

Hello world

OK

OK

OK

OK

OK

OK

unknown

OK

FreeBSD World

OK

OK

OK

OK

OK, PIC missing

OK, asm parser missing

unknown

unknown

GENERIC kernel

OK

OK

OK

OK

OK (with patches)

OK

missing -mno-abicalls

unknown

LINT kernel

OK

OK

OK

OK

unknown

unknown

missing -mno-abicalls

unknown

Please note that cross compiling is not yet supported by clang (or at least, it is not easy to do out of the box).

Quickstart

Checkout head:

# svn co http://svn.freebsd.org/base/head src

Add the following lines to /etc/make.conf (if you want to use clang for everything, even ports), or /etc/src.conf (if you want to use clang just for world and kernel):

CC=clang
CXX=clang++
CPP=clang-cpp

(NOTE: Do *not* use "clang -E" for CPP, this will not work correctly. You will almost certainly get RPC-related build errors.)

As of r233419, head should build without -Werror bailing out the build, e.g. there should be no unexpected warnings. If you are attempting to build something earlier, or encounter -Werror bailouts anyway, add the following to /etc/make.conf or /etc/src.conf, as appropriate:

# This setting to build world without -Werror:
NO_WERROR=
# This setting to build kernel without -Werror:
WERROR=

If you are building jails, the following can make life easier by not setting system-immutable flag on various important system files. This may have serious security implications depending on your use, (schg/nouchg are explained in chflags(1), the jail(8) man page exlains how jails implement BSD securelevel):

# Does not set schg bit on various system files, 
# useful for building Jails, has security implications.
NO_FSCHG=

Verify that "/usr/bin/clang -v" works:

FreeBSD clang version 3.0 (tags/RELEASE_30/final 145349) 20111210
Target: x86_64-unknown-freebsd10.0
Thread model: posix

and produces output similar to the above.

Building/installing world (into a chroot).

# make buildworld
# make installworld DESTDIR=/usr/obj/clang
# chroot /usr/obj/clang /bin/echo Hello clang world!
Hello clang world!
# chroot /usr/obj/clang /bin/tcsh

Building/installing kernel.

# make kernel KERNCONF=GENERIC INSTKERNNAME=clang

Boot it.

OK set module_path=/boot/clang
OK boot clang

To update the source tree perform the following steps.

# cd src/
# svn up

Using the static analyzer

On the kernel (This requires lots of space in the output directory, 406M when using "scan-build make MODULES_OVERRIDE=").

# cd /sys/{amd64,i386}/conf/
# config GENERIC
# cd ../compile/GENERIC/
# make depend
# scan-build make MODULES_OVERRIDE=
# {firefox,epiphany,konqueror,lynx,less} /tmp/scan-build-YYYY-MM-DD-N/index.html

On a userland program.

# cd /usr/src/usr.bin/make/
# make cleandir && make obj && make depend
# scan-build make
# {firefox,epiphany,konqueror,lynx,less} /tmp/scan-build-YYYY-MM-DD-N/index.html

Using the analyzer on vanilla sources

Scanning the FreeBSD sources using the static analyzer requires you to:

# svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm
# cd llvm/tools && svn co http://llvm.org/svn/llvm-project/cfe/trunk clang
# cd .. && ./configure --enable-optimized && gmake
# cp -p tools/clang/tools/scan-build/{ccc-analyzer,scan-build,scanview.css,sorttable.js} Release+Asserts/bin
# ln -sf ccc-analyzer Release+Asserts/bin/c++-analyzer

Now you have all the necessary tools in llvm/Release+Asserts/bin. They need to be next to each other, as we will be using the full-path to scan-build later and it must find clang-cc, which it searches in it's own directory and $PATH. As $PATH will be cleansed during buildworld, only the first option remains.

Running the analyzer is now as easy as

# cd /usr/src && /path/to/llvm/Release+Asserts/bin/scan-build -k make buildworld

Weekly scans of buildworld and buildkernel can be studied at http://scan.freebsd.your.org/freebsd-head. The rate of false positives is decreasing and LLVM is working actively on fixing them. You can help by grabbing a report and working out if it's valid or false positive. Then send a bug report to either LLVM or FreeBSD and add it to the list below.

Here's a list of known false positives:

Title

Example report

Bug

Does not track *argv when 'argv += optind();'

bin/df step 2 + 5

13974

Impossible path in if/else and if/while

Multiple

13426

Patches to FreeBSD:

Title

Example report

PR

Asserts disabled in ncurses

lib.ncurses.menu

misc/171777

LLVM supports LTO. It can be used for kernel build by these pseudo steps:

0) build gold and libLLVMgold.so (see http://llvm.org/docs/LinkTimeOptimization.html)
1) CFLAGS+=-emit-llvm
2) setenv LD "/tmp/ld-new --plugin /tmp/libLLVMgold.so"
3) setenv CC clang
4) build the kernel

This was tested on i386. The resulting kernel does boot.

Automatic test generation with KLEE

KLEE is a symbolic virtual machine built on top of the LLVM compiler infrastructure. More about KLEE here and paper here.

Building KLEE

# svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm
# svn co http://llvm.org/svn/llvm-project/cfe/trunk llvm/tools/clang
# svn co http://llvm.org/svn/llvm-project/klee/trunk klee
# cd llvm/
# ./configure --prefix=/usr/local --enable-optimized --enable-targets=host-only --enable-bindings=none
# gmake
# gmake install
# cd ../klee/
# ./configure --with-llvm=$PWD/../llvm
# gmake LLVMGCC=/usr/local/bin/clang
# cp Release+Asserts/bin/* /usr/local/bin/
# cp Release+Asserts/lib/* /usr/local/lib/

Using KLEE

# cd /usr/src/usr.bin/touch/
# clang -c -g -emit-llvm -o touch.bc touch.c
# klee touch.bc
KLEE: output directory = "klee-out-0"
...

More to come later...

Known issues

Building svn head r233075

rm -rf /usr/obj && time make -j4 buildworld

gcc WITH_CLANG

gcc WITHOUT_CLANG

1:28:48 (h:min:s)

54:22 (min:s)

clang WITH_CLANG

clang WITHOUT_GCC

1:30:33 (h:min:s)

N/A (h:min:s)

> clang -v
FreeBSD clang version 3.0 (tags/RELEASE_30/final 145349) 20111210
Target: x86_64-unknown-freebsd9.0
Thread model: posix
> gcc -v
Using built-in specs.
Target: amd64-undermydesk-freebsd
Configured with: FreeBSD/amd64 system compiler
Thread model: posix
gcc version 4.2.1 20070831 patched [FreeBSD]
> dmesg | head 
Copyright (c) 1992-2012 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.0-STABLE #0: Sat Mar 17 09:14:25 UTC 2012
    root@leopard1.netperf.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
CPU: Intel(R) Xeon(R) CPU            3050  @ 2.13GHz (2128.04-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x6f6  Family = 6  Model = f  Stepping = 6
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0xe3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM>

Contacts

Resources

BuildingFreeBSDWithClang (last edited 2013-03-28 00:56:50 by AndrewTurner)