FreeBSD Project "ideas" List

Over the years, the FreeBSD Project has built up a list of ideas for implementation work that seems like it might be a good idea, but no hands are available to do it. If you would like to contribute to the FreeBSD Project, you might peruse this list to get a sense of the kinds of work available to do. Obviously, contributions are not limited to this list!

Please do contact us before starting on it though -- sometimes items remain on the list after they are completed, and sometimes they are just ideas, rather than a recipe for success. Searching our mailing list archives may turn up the discussions leading to ideas being put on the list. Frequently the initial goal would be to simply investigate the idea, rather than produce code. Many project ideas list contacts, who it is worth sending an e-mail. Otherwise (and perhaps as well), send e-mail to our hackers@ mailing list.

For Google Summer of Code students:

For Google Summer of Code ideas, visit: SummerOfCode2014, SummerOfCode2013.

This page can be used for inspiration of your Google Summer Of Code idea. Please note: some projects were considered too big to get accomplished in 12-week period.

Contents

  1. FreeBSD Project "ideas" List
  2. Wireless Projects
  3. Embedded Projects
    1. Reduced FreeBSD kernel size for embedded
    2. Make creating a bus easier
    3. Variable hints
    4. ARM cleanup
  4. File System Projects
    1. FUSE (fusefs) cleanups and improvements
    2. Improve the performance of dump/restore
    3. Filesystem decompression layer
    4. EXT2FS improvements
    5. MDFS lockups
    6. Porting HFS+
    7. Port NiLFS to FreeBSD
    8. Port DragonflyBSD's HAMMER file system to FreeBSD
  5. Kernel Projects
    1. Test Kload (kexec for FreeBSD)
    2. Document all sysctls
    3. Document the sound subsystem
    4. DTrace
    5. DWARF2 call frame information (GSoC)
    6. Implementing PCI-Hotplug and ExpressCard support
    7. Collective limits on set of processes (a.k.a. jobs)
    8. Remove procfs dependencies
    9. Suspend to disk
    10. Sync FreeBSD i386 boot code with DragonFly
    11. Memory Compression
    12. Solaris Doors IPC Implementation
  6. Virtualization Projects
    1. Solaris/Illumos jails
  7. Networking Projects
    1. SCPS, Space Communication Protocol Standards
  8. Ports Projects
  9. Security Projects
  10. Userland / Installation Tools Projects
    1. BSD-licensed ELF Tools
    2. BSD-licensed Text-Processing Tools
    3. Build options improvements
    4. boot.kernel.org Compatible Installation Image
    5. lint(1) improvements from OpenBSD
    6. NDMP data server
    7. Port prebind from OpenBSD
    8. Proxy auto-config file support for libfetch
    9. PXE Installer
    10. Regression testing system
    11. Tar output mode for installworld
    12. Improve cron(8) and atrun(8)
    13. libpw
    14. Safe crash dumps
    15. Import syslogd improvements from NetBSD
    16. Add support for usbdump file-format to wireshark and vusb-analyzer
    17. Support for setting base system build options via dialog(1)
    18. RAID and disk monitoring suite
    19. Cross-building FreeBSD from Linux and/or Mac OSX
  11. Global Projects (may touch everything)
    1. EPUB Support in Documentation Build Infrastructure
    2. PerfVisor (PERFormance adVISOR)
  12. Other Projects

Wireless Projects

These are now on WifiIdeasPage.

Embedded Projects

Reduced FreeBSD kernel size for embedded

Technical Contact: imp@

Description

The FreeBSD kernel has been optimized over the years for a server or workstation environment. Memory is plentiful in these environments, so little attention was given to the size of the kernel. There's a number items in the kernel that can be made optional without reducing affecting the functionality needed in an embedded environment. These include things like not compiling in strings into the kernel, less agressively inlining code, making some non-optional features optional and investigating compile time flags. This task requires identifying potentially optional kernel content and building the infrastructure to make that content optional.

Requirements

Make creating a bus easier

Technical Contact: imp@

Description

There's about a dozen busses in the tree now that manage resources and activate children. They are far too hard to create. We need to abstract out the basics for these buses and provide a way to allow these buses to be a subclass of this new base class.

Requirements

Variable hints

Technical Contact: imp@

Description

Often times in the embedded world, you know what kind of built-in devices are on a SoC (System on a Chip) only because you know the specific model of that SoC. It is desirable to have a mechanism that code on these machines can use to load one of several sets of hints, which can then be used to populate the bus.

Requirements

* Good C programming skills

ARM cleanup

Technical Contact: imp@

Description

Adding a new board to the arm code is a lot harder than it needs to be. A lot of benefit could be had by creating tables for memory ranges, etc, and having more generic initialization code. Much of this can also be Machine Independent (MI).

Requirements

File System Projects

FUSE (fusefs) cleanups and improvements

Technical Contact: rodrigc@

Description

FUSEFS has been introduced into base in the summer of 2012. A number of simple cleanups would be desirable:

Requirements

Improve the performance of dump/restore

Description

A performance evaluation of the split cache (as is) and an unified cache (like e.g. NetBSD) would be interesting. More details in this mail to the hackers mailing list. Additional improvements are welcome too.

Requirements

Filesystem decompression layer

Solaris 10 and newer provide dcfs; a read-only decompression stacking filesystem layer for UFS. Files are initially compressed by a userland fiocompress utility. The filesystem layer is very simple, it is implemented in single file that permits transparent decompression without the end user knowing if the file is compressed or not. While the implementation is really simple it is very useful by making possible to install in systems with little memory or for quick compression of files in typical read-only directories like /usr/bin. While the Solaris fiocompress implementation uses zlib, it would be easy to use more modern algorithms like lz4 or snappy.

References: File-System Development with Stackable Layers

EXT2FS improvements

Ext2fs in FreeBSD's Wiki

Description

FreeBSD's EXT2FS implementation has received many improvements and extensive cleanups in the last years; mostly in the lines of updating the code to match better what our native UFS1 implementation does. We now need to take the next steps: adding write support for extents and exploring new features to make it a full-fledged alternative native filesystem.

Requirements

MDFS lockups

Description

Fix MDFS lockups when using async operation modes. Revision 1.115 of md.c has a discussion of the problem.

Requirements

Porting HFS+

HFS Wiki

Description

The Hierarchical File System was developed by Apple Inc. for use in MacOS. With the Release of MacOS X it received many new features, and the source code was made available as part of XNU. An initial FreeBSD 5.3 HFS port was made and although it was subsequently abandoned and support for locking has to be added, it would be excellent reference material for an updated port. A port would also be a good reference for bringing other interesting filesystems from Apple's Darwin.

Requirements

Port NiLFS to FreeBSD

Description

The NiLFS is a file system similar to the BSD LFS (and in some features to ZFS and Flash file system). The project would start (probably but not necessarily) by porting the NetBSD read-only implementation of the NiLFS to FreeBSD and continue by implementing the write support. The practical goal of the project would be to have a read-write NiLFS port which passes FreeBSD's fsx and fstest regression tests.

Requirements

Port DragonflyBSD's HAMMER file system to FreeBSD

Technical Contact: ivoras@

Description

The HAMMER file system is a new file system created as a response to new ideas in file systems initiated by the likes of ZFS and BTRFS. It introduces innovative features and is considered production-ready in DragonflyBSD. It should be comparatively easy to port HAMMER to FreeBSD than any other file system because of the shared ancestry between FreeBSD and DragonflyBSD; however, though the project will bring eternal fame to the student brave enough to tackle it, the task is not for the weak of heart.

Requirements

Note

The project is too complex for a GSoC see: PortingHAMMERFS.

Kernel Projects

Test Kload (kexec for FreeBSD)

Technical Contact: (We'll have to find another mentor for this project too)

Description

https://wiki.freebsd.org/Kload

It's like Linux kexec feature lets you restart the OS very quickly without going via BIOS initialization. You pass the ELF file of the kernel and the current kernel gets overwritten and the new kernel starts. Idea is to actually test it well.

Requirements

Document all sysctls

Technical Contact: mat@, brd@, eadler@

Description

The sysctl(8) utility retrieves kernel states and allows processes with appropriate privilege to change kernel states. On request it is able to display description lines which document the kernel state. Unfortunately not every sysctl is documented. This task is possible to share with other volunteers. mat has done some development in Perforce, in the mat_sysctl_cleanup branch.

Requirements

Document the sound subsystem

Technical Contact: netchild@

Description

Requirements

DTrace

Technical Contact: rwatson@

Homepage: Perforce repository, DTrace for FreeBSD

DTrace is a dynamic tracing facility designed by Sun Microsystems and released in Solaris 10. They have since released the major part of Solaris under the banner of OpenSolaris and the Common Development and Distribution License (CDDL) 1.0.

Requirements

DWARF2 call frame information (GSoC)

Suggested Summer of Code 2011 project idea

The DWARF Debugging Standard

Description

A debug kernel is not able to show stack traces with cross exceptions anymore. This is because we do not emit any dwarf2 call frame information for any assembler code, since gdb switched to the dwarf2 format. A volunteer should annotate every assembler file [*.[sS]] with dwarf2 call frame information.

Requirements

Implementing PCI-Hotplug and ExpressCard support

Technical Contact: freebsd-current@freebsd.org gavin@ bms@ kmoore@

Description

Hotplug PCI, and ExpressCard (Hotplug PCI + powersave) support is sadly missing from FreeBSD. This would require implementing some missing code to notice cards being inserted and removed, along with some code to allocate resources. There is also likely some work to be done with PCIe PHY power saving.

Requirements

Collective limits on set of processes (a.k.a. jobs)

Technical Contact: gabor@, brooks@

In SGI's Irix operating system, there is a concept of a job which is a collection of processes. These processes share a set of resource limits similar to those accessible through the set/getrlimit and getrusage system calls. Schedulers such as Sun Grid Engine currently implement tracking the processes that make up a job and enforcing collective limits on them in an adhoc manner. Having first class kernel support would be useful.

It seems most likely that implementing something like the Irix interface would be the most useful approach since that would enable us to leverage existing code. Implementers will need to make sure that the job construct has no negative performance implications when not enabled (ideally, processes that are not part of jobs should be unaffected) and quantify the impact on perfomance when enabled.

References: http://techpubs.sgi.com/library/tpl/cgi-bin/browse.cgi?cmd=search&coll=0650&db=man&pth=&srch=jid_t&searchpth=man&rpt=20

Requirements

Remove procfs dependencies

Technical Contact: cognet@

Someone needs to finish the support for PT_SYSCALL in the ptrace() subsystem and remove the need for procfs in gcore.

Requirements

Suspend to disk

Implement a suspend/resume from disk mechanism. Possibly use the dump functions to dump pages to disk, then use ACPI to put the system in S4 or power-off. Resume would require changes to the loader to load the memory image directly and then begin executing again.

Requirements

Sync FreeBSD i386 boot code with DragonFly

Technical Contact: jhb@

DragonFly invested a lot of time to clean up and document it. Additionally they fixed some bugs. Interesting files in the DragonFly CVS are sys/boot/i386/bootasm.h, sys/boot/i386/bootasmdef.c, sys/boot/boot0/*, sys/boot/boot2/*, sys/boot/i386/btx/*, sys/boot/i386/cdboot/*, sys/boot/i386/libi386/amd64_tramp.S, sys/boot/i386/libi386/biosdisk.c and sys/boot/i386/loader/main.c. An interested volunteer has to compare and evaluate both implementations and port interesting/good parts.

Requirements

Memory Compression

Technical Contact: theraven@

Something like OSX 10.9's memory compression

MacOS X "Mavericks" does memory compression using the WKdm algorithm when it is low on resources. Brought this up in IRC, theraven said it is something he has had on his mind and he would be willing to mentor someone who wanted to work on this. Leaving this here so the idea isn't lost. -feld

Solaris Doors IPC Implementation

Fast Sockets, An IPC Library to Boost Application Performance

An Implementation of the Solaris Doors API for Linux

Doors provide a mechanism for processes to issue remote procedure calls to functions in other processes running on the same system. The door APIs were developed by Sun Microsystems as a core part of the Spring operating system and were officially available in Solaris 2.6. They are extensively used in Illumos.

In addition to the Solaris/Illumos port, which is well documented, there is also an outdated implementation for linux that can serve for comparison. The project would consist in understanding how the existing code works and designing a completely new, but compatible, implementation for FreeBSD.

Requirements

- Interest in Inter-process Communications

- Ability to understand code and the existing implementations in Illumos and linux.

- Capacity to run your own tests and benchmarks.

Virtualization Projects

Solaris/Illumos jails

FreeBSD has the capacity to create system jail(8)s for environments that can be run natively with ABI extensions. This has been used successfully to provide GNU Linux and Debian kFreeBSD jails. FreeBSD i386 also has the capacity to run some SVR4 binaries but that support was never ported for amd64 or even tested with Solaris binaries. While of little practical use, it would be interesting to build on the previous developments and finish the SVR4 ABI support so that Solaris/Illumos distributions can run in a jail.

Requirements

Networking Projects

SCPS, Space Communication Protocol Standards

SCPS is a protocol suite designed to allow communication over challenging environments. Originally developed jointly by NASA and DoD's USSPACECOM, these protocols are used for commercial, educational, and military environments. A student project in this area would involve implementing various network protocols according to specification (SCPS File Protocol, similar to FTP; SCPS-Transport Protocol, based on TCP; and others.)

Note that European Space Agency has now an ESA Summer of Code in Space program and while FreeBSD is not a mentoring organization, interested students could motivate such a process.

References: Consultative Committee for Space Data Systems

Requirements

Ports Projects

Security Projects

Userland / Installation Tools Projects

BSD-licensed ELF Tools

Technical Contact: jkoshy@, kaiw@

Create BSD-licensed versions of ELF processing tools (e.g., ld, dbx, as and others) using the ELF(3) and GELF(3) API set. Identify overlapping functions in those tools and create a library out of the common functions. Identify parts which can be generated by tools (e.g., machine code parser generators) to support our Tier-1 and Tier-2 architectures.

References:

Requirements

BSD-licensed Text-Processing Tools

Part of Summer of Code 2010, Part of Summer of Code 2008

Technical Contact: gabor@

grep: It has been committed to the base system and available as an alternative of GNU grep. The compatibility is good but the performance is quite behind GNU grep, which prevents us from using it as a default. There are also some problems of regular expressions involved. It is under active development by gabor@.

diff/diff3/sdiff: Many command-line options are supported but some features are still missing. Maybe the three programs can be integrated into a single binary, this should be evaluated. A thorough performance benchmark should also be done. See SummerOfCode2012/JesseHagewood for last status.

mdocml: Some groff features are very hard to implement but they aren't strictly needed to render our man pages. Yet some manuals do not compile with mdocml. Investigate the reasons and create a migration plan.

Requirements

Build options improvements

Technical Contact: netchild@

The "delete-old" and "delete-old-libs" targets in /usr/src should be extended to support the WITHOUT_* knobs, e.g. WITHOUT_RESCUE or WITHOUT_CRYPT, and delete files which are covered by those knobs. Some switches have already been covered. You can view a list of all switches and what effect they have here.

Requirements

boot.kernel.org Compatible Installation Image

boot.kernel.org is a project to provide boot images that can be booted over http with a small boot program (gpxe). A number of Linux distributions have installation media available. It would be useful if FreeBSD's release process could generate appropriate boot images.

Requirements

lint(1) improvements from OpenBSD

OpenBSD has some improvements to lint(1) which may be beneficial to have.

Requirements

NDMP data server

URL: The NDMP Initiative

The NDMP initiative was launched to create an open standard protocol for network-based backup for network-attached storage. Major commercial storage systems come with a compliant service. This allows major commercial backup systems to backup such NAS devices. Including a NDMP disk server into FreeBSD would allow to play nice out of the box (modulo some configuring) regarding backups in a corporate environment.

Requirements

Port prebind from OpenBSD

The OpenBSD prebind is a secure implementation of prelinking that is compatible with address space randomization. Prelinking allows to speed up application startup when a lot of libraries are involved. This should show a noticeable effect with e.g. GNOME/KDE.

Requirements

Proxy auto-config file support for libfetch

A proxy auto-config (PAC) file contains a JavaScript function "FindProxyForURL(url, host)" that determines which HTTP or SOCKS proxy, if any, to use to access a given URL. In most application the file may be specified manually or discovered using the Web Proxy Autodiscovery Protocol. Support for PAC files in libfetch would make fetch more versitle.

Supporting PAC files nominally requires a fairly complete JavaScript implementation. Google's V8 JavaScript engine is BSD Licensed, however it compiles code to native machine code so platform support is an issue. However, the parser etc may provide a good starting point, and other engines may also exist and should be evaluated. A minimalist implementation of the language with commonly used constructs such as if/else, string comparison, and functions would be sufficient in many cases.

References:

Requirements

PXE Installer

It would be great to have a bundled PXE installer. This would allow one to boot an install server from a FreeSBIE live CD-ROM on one box, set the BIOS on subsequent boxes to PXE boot, and then have the rest happen by magic. This would be very helpful for installing cluster nodes, etc.

m@ is working on a bundled PXE installer as part of his BSDInstaller project within the Google Summer of Code 2006. The PXE Installer is working but some non-PXE related issues have to be solved before it can enter the tree.

Requirements

Regression testing system

Technical Contact: netchild@, nik@

Nik has written a regression test infrastructure using Perl. More of the regression tests should be made to work with libtap.

Porting LTP might also be a good idea.

Requirements

Tar output mode for installworld

Technical Contact: cperciva@, kientzle@

Instead of installing using install, mkdir, mtree, etc, directly construct a tarball. This would allow creating install distributions without root access, as setuid etc would never hit the local disk. This would require some retrofitting of our installation mechanisms.

Bsdtar now (8-current, 20080101) has a feature that allows it to create a tar archive from a description provided in the form of an mtree file. This description can specify owner, permissions, and contents for each entry and does not require the files on disk to already have correct ownership. This should make it possible to build a FreeBSD distribution as a non-root user. Talk to Tim Kientzle for details of the new bsdtar features and look at NetBSD, which has a similar facility, for ideas about how to proceed.

Requirements

Improve cron(8) and atrun(8)

Currently, cron(8) and atrun(8) are outdated in their implementation. Here are some directions for improvement:

Requirements

libpw

Technical Contact:

Create a library to be able to manage users/groups easily, it also should have a pam/nss-like plugin framework for different account system.

libutil has pw_* and gr_* undocumented functions which allow user/group manipulation

writing pw_*() and gr_*() manpages

known users for that library: pkgng, pw(8)

Requirements

Safe crash dumps

Technical Contact: GavinAtkinson gavin@

Crash dumps are important for collecting debugging information but are also disabled by default because they can consume much space in /var if the user doesn't pay attention to them.

What we miss is a safe way to enable crash dumps by default without having to worry about them filling up /var.

Requirements

Import syslogd improvements from NetBSD

Technical Contacts: Ed Maste (emaste@), Mark Johnston (markj@)

Note - there is work in progress on this project, please contact Ed or Mark for details.

NetBSD's syslogd has a number of improvements from a former Google Summer of Code project available for porting:

The changes are in NetBSD's repository in src/usr.sbin/syslogd, available for viewing on their cvsweb:

Requirements

Add support for usbdump file-format to wireshark and vusb-analyzer

Technical Contact: hselasky@

Currently it is not possible to graphically see USB traffic. By adding a backend to the above mentioned software packages this can easily be made possible.

The project consists of writing a small amount of C (wireshark) and Python (vusb-analyzer) code.

This is expected to be a 1 month exercise.

Requirements

Support for setting base system build options via dialog(1)

Our ports support setting build options via dialog(1) for ages. Recently, with the pkgng invention, it was made possible also to pack port's build options into binary package - later, a functionality to track dependencies based on the build options may be added.

Our base system build infrastructure now prefers consistent /etc/src.conf options in favor to older ad-hoc /etc/make.conf options. Currently, there are only binary ones, thus possible to map to dialog(1) checkboxes. The idea is to be able to do cd /usr/src && make config to see familiar dialog(1) interface - just as in any port in /usr/ports (it should be complemented with usual make showconfig and make rmconfig, of course).

This idea, amongst direct simplification of average user's life, is valuable in regard to possible future packaging of the FreeBSD base system to one or a few pkgng packages. Such packaging is desired in some environments of large production server farms for easier managing/upgrading servers, see https://github.com/z0nt/pkg for such a project. In the future, however, this could be useful for tracking pkgng packages dependencies based on world's build options (e.g. a port requires a world built with WITH_IDEA or not built WITHOUT_PF, etc.).

Requirements

RAID and disk monitoring suite

Technical Contact:

There have been several organizations that have independently developed RAID and disk failure monitoring tools. These should be gathered together into a unified group of daemons and monitoring scripts to provide a consistent view of disk status.

http://svnweb.freebsd.org/base/user/sbruno/ard/ http://svnweb.freebsd.org/base/user/sbruno/mfid/ http://svnweb.freebsd.org/base/user/sbruno/mptd/

Requirements

Cross-building FreeBSD from Linux and/or Mac OSX

Technical Contact:

FreeBSD's build system is self-contained, but only really designed to be run on FreeBSD as it makes assumptions about the host platform and available tools. Being able to build a fully-functional FreeBSD userland and kernel from either Linux or Mac OSX without having to use FreeBSD inside a virtual machine would allow more people to make use of and build products out of FreeBSD easier.

On Mac OSX an additional complication exists in that the default filesystem is case-insensitive. At least to begin with, creating a case-sensitive filesystem to do this work on is recommended. Some work has already been done here, and it may be desirable to build upon that.

Requirements

Global Projects (may touch everything)

EPUB Support in Documentation Build Infrastructure

Suggested Summer of Code project idea

Enhance the FreeBSD Documentation Project build infrastructure to generate EPUB format output suitable for eBook readers from such as iPads and Kindles.

Requirements

PerfVisor (PERFormance adVISOR)

Coordination: netchild@

The goal of this project is to get a tool which can analyze a system for performance bottlenecks and maybe even give some hints what to do next.

This is not a GSoC project. Maybe small parts of it can be done during a GSoC.

Prerequisites you should know/read to understand the project steps:

Project phases

Initial phase

Improvement phase

Visualization phase

Cloud phase (everything is cloudy nowadays...)

Datawarehouse phase (buzzword bingo!)

Discussion

The first item in the initial phase is a big item. It may be better to see the complete initial phase as an iterative or divide and conquer step. First determine some high level resources and process all steps of the initial phase on them. Then repeat the initial phase again by braking up the high level resources in more detailed items (e.g. first "all CPU in system" as one resource, second "each CPU in system" as a resrouce, third breaking up each CPU via hardware performance counters).

GSoC info

Each phase is a big project of its own. Do not expect to be able to do it during some weekends or during a GSoC. If you think you can, you did not get the full scope, think again. If you still insinst after rethinking, be our guest, but you better prepare a very good outline what you want to do when, how, at which level of detail (e.g. come with a list of resources for the first item in the initial phase), and also include what you will NOT do/cover/handle but could be related (e.g. CPU internal performance counters if you want to handle the CPU performance side of this). You already need to know FreeBSD (you use it already on your server or desktop since several months) and you shall not be afraid to ask questions or discuss on the mailinglists. This is not a project for 2h per day, if you are not motivated to spend time on this, you better chose a different topic.

This can be made "big enough" to be a project suitable for finals at your university (depending on the requirements of your university, with or without extending it over the end of the GSoC).

Other Projects

If you are interested in working on a project not explicitly mentioned above, you may want to contact one of the potential Technical Contacts below:

Additionally, there are a lot of interesting mailing lists that can be used when searching information about specific subjects.

For porting ideas go to WantedPorts and for Beginner tasks check out JuniorJobs

IdeasPage (last edited 2014-02-22 20:19:58 by PedroGiffuni)