• linux-kernel

Improving the Linux compatibility layer in the FreeBSD kernel

The goal is to update to 2.6.16 compatible Linux syscalls. Without this compatibility, we are not able to update to a more recent linux_base port.

Test equipment

• Linux Test Project: README and INSTALL are straigt forward, don't forget to mount devfs, linprocfs and linsysfs to the linux-chroot where you compile/test (I'm using gentoo-stage3 to have a complete build environment)

• GNU libc nptl regression test: not as easy as LTP, but doable; precompiled stuff available

• mmap fingerprinter: little tool by Marcin Cieslak, extended by Jung-uk Kim to determine the fingerprint of the mmap() call. Could be used to extend the FreeBSD regression tests too.

LTP

Very easy to set up: install linux_dist-gentoo-stage3, download the tarball into a directory below /usr/local/gentoo-stage3/, mkdir dev and sys in the gentoo-stage3 directory, mount devfs, linprocfs and linsysfs and then do a "chroot /usr/local/gentoo-stage3 bash". To prepare the LTP extract the tarball, cd into the directory, type "make autotools && ./configure && make all install" and then you can run the first tests with "./runltp -v -p -l mylogfilename 2>&1 | tee verboselog.txt".

If you don't have at least revision 1.187 of /sys/compat/linux/linux/misc.c you need to comment out "reboot02" in runtest/syscalls or your system will reboot. "send01" (and some test immediatly after that test) will cause a kernel panic if you don't have at least revision 1.282 of uipc_socket.c (mutex so_snd held when returning to userland).

There are some tests which hang around and do not finish if you use a -current prior to ~20061222, you need to kill those tests there.

Keep an eye on /usr/local/gentoo-stage3/tmp between tests and clean it up. You should also run "ipcs" and remove some leftover (compare before and after each run).

A little funny note: we are even more compatible than a real linux distribution (don't take this literally). LTP 20061222 does not even compile on Ubuntu 6.10. The error message is ../../generate.sh: 60: arith: syntax error: "cnt--" and the line in question is while [ $((cnt--)) -gt 0 ] ; do. If you do a loop unrolling by hand it fails later when you try to run the LTP tests even before it tries the first test. We did not try to proceed further, we want to fix problems on FreeBSD, not on Linux.

Missing stuff

Syscalls

List of all system calls in linux-2.6.16 branch (i386)
List of all system calls in linux-2.6.16 branch (amd64)

This is a list of syscalls used in the linux-2.6 mainline branch. If all the above syscalls have been implemented to ensure 100% compatibility with the linux-2.6.16 branch the next step is to catch up with the linux-2.6 mainline branch syscall wise.

List of all system calls in linux-2.6 mailine branch

Would be nice if somebody could write a script of some sort to make a nice syscall-table out of these src files.

IPC

ioctl

PRs

• 21463: Kris Kennaway suggests the introduction of a sysctl switch which allows/disallows execution of Linux setugid binaries. This would protect FreeBSD from certain Linux userland vulnerabilities.

• 29698: Most of the problems are fixed, only kern.ipc.semmap missing but is present in the code. Bug? We dont have ipcs installed in linux_base but gentoo version works with current code.

• 36952- ldd comand of linux does not work: problem still exists, patch for src needs to be tested

• 39201: If a process gets started with rfork(flags|RFLINUXTHPN) and then traced with ptrace(PT_ATTACH), calling waitpid() fails with ECHILD. Also ptrace(PT_DETACH) might kill the traced process (this hasn't been confirmed though). A patch has been submitted, but is outdated (4.6-RC) and needs to be rewritten.

• 44293: fixed

• 55835: This was commited in rev. 1.54 of compat/linux/linux-ipc.c

• 56451: proc/cpuinfo (linprocfs.c) reports wrong CPU model.

• 72920: seems to be important for Oracle, the fix may be just prepending /compat/linux to the name variable in the args of bind() and connect()

• 73777: we need to lookup the orginal intent of the code which is removed in this patch

• 77710: fixed in 7-current

• 93199: fixed

• 97326: fixed

• 99068: affects linux-java (Oracle installer in this case) fixed? in rev 1.139 of linux_ioctl.c

• 102897: errno problem?: fixed in rev 1.97 of linux_file.c (MFC'd to 6.x)

• 102956: fixed in 8-current (r192203) and MFC'ed to 7-stable (r194281).

• 133144: Linux has gone through two threading model changes. If a Linux application or library has been linked against the old pthreads without fast TLS support or pthreads with internal TLS support libraries it will segfault. This thread describes the threading situation under FreeBSD and Linux in detail: http://lists.freebsd.org/pipermail/freebsd-threads/2003-June/000530.html. This PR is a 2.6.16 stopper since it entirely prevents binaries/libraries compiled against an older Linux threading model from running. As long as this PR doesn't get resolved legacy code (2.4.2 emulation) cannot be removed.

• 138860: linux_socketcall() can lead to buffer overflows.

• 138880: munmap() segfaults after intensive linux_mmap2() stresstest

TODO

in -current

• Futexes lack synchronization with VM and virtual memory independance.
• add the compat.linux.strict_emu sysctl (the name is not set in stone) and use it to enable/disable some functionality we allow in FreeBSD but are not allowed to do in Linux, e.g., allow to open more than one million files, or allow to read() directories
• add detection of unhandled flags to syscalls (e.g. add each detected flag to a variable and compare this variable to the input, if there's a difference print the difference)
• print a message if a known but unhandled flag (e.g. LINUX_O_NOATIME for open()) is used (maybe protected by bootverbose or debug; depending upon the severity and frequence of the use of this flag)
• change the non P1003_1B_MQUEUE part of linux_mq_*() into its own module (the changes for linux aio in p4@114975 can serve as an example)
• SO_PEERCRED: PID handling (compat/linux/linux_socket.c)
• add in the normal device driver entries into the device_handler stuff that are common to all Linux machines
• add in /dev/passX and if possible /dev/sdX support into the device_handler stuff (message "color scanner software for FreeBSD" on emulation@)
• have a look at the linux dev_t issue (HEADSUP message from phk to arch@ in March of 2005)
• cleanup/review of existing code
• Verify that each copyin()/copyout() is handled correctly. There are at least 5 copyout()s where a quick grep revealed ignorance of the return value.

• there are places where a function A calls function B, then calls copyin() on the result of function B, and then checks the return value of function B for errors -> BOOM on error when calling copyin()

• safety-net like in rt_sigpending (but no "new" return values, KASSERT if possible, rt_sigpending needs to be reviewed regarding this)
• CLK_TCK still valid? / XXX comment: both in linux_misc.c
• add doxygen comments to the code
• Use 1:1 threads instead of processes for Linux threads?
• write the MD stuff for amd64 (seems to be done (p4 diff above), modulo bugs, needs testers)

MFC

The following needs to be MFCed (incomplete list):

• mmap() improvements: done

• bugfixes (commits need to be reviewed, telling us about specific stuff/commits would be appreciated): partly done, maybe some are outstanding, we have to check

• utimes() syscall: done

• clock_*() syscalls: done

• rt_*() syscalls: done

• linprocfs improvements: done

• deCOMAPT43ify: done

• PR77710: done

The following will not be MFCed (maybe incomplete list):

• TLS
• NPTL

Bugs

A note to users: Feel free to read the following list of things to fix, but do not make conclusions out of it. The linux compatibility environment runs just fine. There are some broken edge cases which don't affect the daily use. The following list is only meaningful for developers. Just because something is marked as a bug, it doesn't mean it doesn't work. It may be the case that some obscure error condition does not return the expected error value, or that a seldomly used feature is not implemented.

Misc

• the linux-ldd doesn't work when not run with the linux compat shell; if it is run within the linux compat shell, it does print nothing for dynamic executables (it correctly works for libs); because it is a shell script which calls ld-linux.so.2 it may be caused by something which is tested in the LTP... or not

• report by LTP and/or the linux pkill command: "2.4+ kernel w/o ELF notes? -- report this": what are ELF notes and why do we need them - the patch is available.: fixed

• bug regarding mkdir, report by "Steven Hartland" < killing@multiplay.co.uk > on emulation@ at 13 Jul 2006: trailing slash in mkdir() call results in failure: no longer reproducable in current

• remove() does not delete empty directories (PR 102897), also reported with LTP: fixed in current

• mmap problems (errors in LTP runs, not taking PROT_EXEC into account): fixed in current/stable

• broken mmap behavior with PROT_EXEC: fixed in current/stable

• linux_lseek() silently truncates the offset modulo 2^32, lseek() in 2.6.x has error handling for this (EOVERFLOW). Unlike the FreeBSD lseek() (open(), ...) the Linux one can't handle 64bit file sizes.
• the linux "dmesg" utility does not work, missing syscall

• LORs with 2.6.16: fixed in p4

• problems with 2.6.16 and FC6 linux base (not available in the ports collection), primary issue: *at() calls: in HEAD.

• zombie processes with 2.6.16 when closing linux-opera

• utimes changes permissions of files, see http://lists.freebsd.org/pipermail/freebsd-emulation/2006-December/002937.html : fixed in current as of 2006/12/31 13:16:00

Automated test results

• Default testcases of the Linux Test Project

• The ¨ballista¨ testcases of the Linux Test Project should be run when we are finished with fixing bugs.
• The Open POSIX testsuite (comes with the LPI); we need a reference output from Linux to be able to compare this.
• Some of the LTP testcases which are not run by default.
• The glibc regression tests.