Ideas Page
Introduction
Over the years, the FreeBSD Project has built up a list of ideas for implementation work that seems like it might be a good idea, but no hands are available to do it. If you would like to contribute to the FreeBSD Project, you might peruse this list to get a sense of the kinds of work that others have thought might be useful or available to do. Obviously, contributions are not limited to this list!
Please contact us before starting on it though -- sometimes items remain on the list after they are completed, and sometimes they are just ideas, rather than a recipe for success. Searching our mailing list archives may turn up the discussions leading to ideas being put on the list. Frequently the initial goal would be to simply investigate the idea, rather than produce code. Many project ideas list contacts, who it is worth sending an e-mail. Otherwise (and perhaps as well), send e-mail to our hackers@ mailing list.
For Google Summer of Code Students
Ideas on this list are generally unsuitable for Summer of Code projects (too large, too small, no available mentor, etc) but there may still be ideas here that could form the basis of a good GSoC task.
Important: When adding an idea, include a contact method, so interested parties can get in touch.
Contents
Wireless
Wireless ideas can be found on WiFi/IdeasPage.
Embedded
File Systems
Improve the performance of dump/restore
Description
A performance evaluation of the split cache (as is) and an unified cache (like e.g. NetBSD) would be interesting. More details in this mail to the hackers mailing list. Additional improvements are welcome too.
Requirements
- Knowledge of C programming.
- Basic understanding of backup/restore procedures.
OpenBFS
Practical Filesystem Design Book
Description
Before working at Apple, Dominic Giampaolo wrote a 64 bit journalling filesystem for BeOS. The objectives were to have a modern filesystem capable of streaming for multimedia applications. It also supports indexing and Extended Attributes for desktop use. Haiku has implemented a version of the filesystem based on the documentation which is used as their base filesystem and is available under an MIT license. Porting it, or rather using the existing implementation as the basis for a new one, would make an interesting case for an alternative filesystem.
Requirements
- Ability to read and understand foreign C code, familiarity with FreeBSD's VFS is a plus.
- Ability to interpret results from testsuites and find solutions.
- Knowledge of filesystem internals.
Port Hammer2
Description
The Hammer is a recent copy-on-write filesystem developed by Matt Dillon as part of DragonFlyBSD. Due to the differences in evolution in DragonFly it would be very difficult to port the original hammer but it should be possible to port Hammer2, which has most of the same advanced features and is specifically designed to support clustering. Both filesystems are available under a BSD license. The ports tree carries sysutils/hammer2 a read-only port of the kernel driver: figuring out how to finish the write support is being considered for NetBSD.
Requirements
- Strong knowledge of C.
- Understanding of FreeBSD's filesystem interfacing and VFS.
- Willingness to dive into !DragonFlyBSD's locking semantics.
Porting HFS+
Description
The Hierarchical File System was developed by Apple Inc. for use in MacOS. With the Release of MacOS X it received many new features, and the source code was made available as part of XNU. An initial FreeBSD 5.3 HFS port was made and although it was subsequently abandoned and support for locking has to be added, it would be excellent reference material for an updated port. A port would also be a good reference for bringing other interesting filesystems from Apple's Darwin.
Requirements
- Strong knowledge of C.
- Understanding of FreeBSD's filesystem interfacing and VFS.
Kernel
DTrace
Technical Contact rwatson@
Homepage: DTrace for FreeBSD
DTrace is a dynamic tracing facility designed by Sun Microsystems and released in Solaris 10. They have since released the major part of Solaris under the banner of OpenSolaris and the Common Development and Distribution License (CDDL) 1.0.
- We need a clean CTF implementation for FreeBSD to avoid the license (CDDL) that Sun has on their code. A specification about the file format needs to be written, and someone who never looked at the Sun code (and doesn't while doing the work) would have to implement that and write tests for the implementation.
We can always use moreProviders .
Requirements
- Ability to read and understand foreign C code.
- Ability to write C code.
- A good understanding of the FreeBSD kernel.
Kernel Control Flow Integrity (kCFI)
Control Flow Integrity (CFI) is a security feature implemented mostly compilers and is fully supported by Clang/LLVM: the original prototype was indeed implemented for FreeBSD. CFI has greatly evolved and has been implemented in the Linux kernel and Android. CFI is relatively well documented and should be easy to implement on FreeBSD. Using CFI we shouldn't need to use an additional stack protector at all.
Requirements
- Ability to read and understand foreign C code.
- Ability to write C code and definitions.
- Interest in compilers, linkers, and how they work.
IOCTL command checking tool
Technical Contact: brooks@
The ioctl(2) system call takes integer commands which are typically defined using various _IO*() macros in sys/sys/ioccom.h. One component of the command is the amount of data to be copied in and/or out of the kernel (usually specified by a C type to be copied). Any object which changes size between 32 and 64-bit variants of an architecture requires compatibility definitions and special handling (this commonly occurs if a pointer or long is included in the object directly.) Types may also change size due to differing padding rules. Additionally, all types containing pointers require special handling even if a complex definition allows the size not to change. It would be useful to have a tool to check all ioctl(2) commands for size stability and to check objects used as command arguments for pointers.
Requirements
- Ability to read and understand foreign C code.
- Ability to write C code.
- Tool may be written in any language.
Suspend to disk
Implement a suspend/resume from disk mechanism. Possibly use the dump functions to dump pages to disk, then use ACPI to put the system in S4 or power-off. Resume would require changes to the loader to load the memory image directly and then begin executing again.
Requirements
- Good knowledge of C.
- Understanding of the hardware/software interface.
- A laptop that works with ACPI.
- Kernel awareness.
Solaris Doors IPC Implementation
Doors provide a mechanism for processes to issue remote procedure calls to functions in other processes running on the same system. The door APIs were developed by Sun Microsystems as a core part of the Spring operating system and were officially available in Solaris 2.6. They are extensively used in Illumos.
In addition to the Solaris/Illumos port, which is well documented, there is also an outdated implementation for linux that can serve for comparison. The project would consist in understanding how the existing code works and designing a completely new, but compatible, implementation for FreeBSD.
Requirements
- Interest in Inter-process Communications
- Ability to understand code and the existing implementations in Illumos and Linux.
- Capacity to run your own tests and benchmarks.
Develop a netmap vnic for jails that allows connecting jails to vale
Contact: DanielEbdrup
Technical Contact: <freebsd-virtualization AT freebsd.org>
Jail network connectivity has gotten a separate network stack through VNET/VIMAGE, but at present is limited to 10G between jails and the host. Utilizing a netmap based vnic for jails and the vale software switch to make IPC for jails <-> bhyve and the outside world much easier and faster. Additionally, it may very well be possible to adopt a lot of existing code from svn revision 293459.
Requirements
- Interest in IPC/VPC
- Ability to write and/or adopt C code
Virtualization
bhyve gdb-stub/dcons integration
FreeBSD's 'bhyve' Hypervisor has a feature where it allows the kernel debugger to communicate with the outside world via a socket. Unfortunately it is quite slow, being polled one byte at a time. In addition it still uses the kernel on the VM to do all the work. Some avenues of improvement could include using the existing dcons memory buffer driver so that a whole buffer might be transferred at a time, or to make the current driver much faster. One might also intercept some of the commands (memory read for example) and perform them directly in the hypervisor so that cooperation of the virtualized system is not required to examine memory. It is even possible that by manipulating the processor flags appropriately, one could single step or 'break' the guest without its cooperation at all.
Requirements
- Willingness to dig into OS internals
- Knowledge of i486/amd64 assembler and virtualization concepts
- ability to decode intel/AMD processor specs
VirtualBox shared folder support for FreeBSD guests
Technical Contact: lwhsu@, gonzo@, decke@ Contact: vbox@
Description
Oracle VirtualBox does unofficially support FreeBSD as host and guest operating system. VirtualBox shared folder support allows to access folders of your host from within the guest system. This is similar how you would use network shares in Windows networks except that shared folders do not need require networking, only the VirtualBox Guest Additions.
This task was part of GSoC 2013 where the main focus was on porting the code to FreeBSD. The result was almost working read only support but more bugs need to be fixed and read write support needs to be added and tested.
Requirements
- Ability to read and understand foreign C code
- Ability to write C code
- Knowledge of the VFS subsystem
Networking
openPOWERLINK and other industrial ethernet protocols
The manufacturing industry and automation have special requirements for networking in industrial environments: while the cabling has to carry special isolation there is also the requirement to support real-time communications. In such environments, there is a tendency to move towards Ethernet to take advantage of the existing hardware. Several Ethernet types have been developed and standardized to support real-time, with Ethernet/IP and PROFINET being the most common. Ethernet PowerLink (EPL) is particularly interesting as the protocol stack is available under a BSD license.
References: openPowerLink
SCPS, Space Communication Protocol Standards
SCPS is a protocol suite designed to allow communication over challenging environments. Originally developed jointly by NASA and DoD's USSPACECOM, these protocols are used for commercial, educational, and military environments. A student project in this area would involve implementing various network protocols according to specification (SCPS File Protocol, similar to FTP; SCPS-Transport Protocol, based on TCP; and others.)
Note that European Space Agency has now an ESA Summer of Code in Space program and while FreeBSD is not a mentoring organization, interested students could motivate such a process.
References: Consultative Committee for Space Data Systems
Requirements
- Good knowledge of C and TCP.
- Able to understand the FreeBSD TCP/IP stack.
- A testbed with at least two machines.
Porting
Port FreeBSD to new platforms
Porting to new platforms is a good way to learn the internals of FreeBSD and serves to check the general portability of the base system.
While there are important efforts to maintain an external toolchain working it would be ideal to start working on newer platforms that are already supported by the toolchain in base and where the hardware support is either easy to find or well supported through emulation.
Contact: freebsd-arch@
Requirements
- Good Knowledge of C and assembler of the target platform.
- Familiarity with (cross)building FreeBSD.
Port FreeBSD on Tablet device
Porting FreeBSD to arm, intel tablets and next mobile device.
Contact: freebsd-arch@
Requirements
- Some kernel experience
Testing and Continuous Integration
POSIX compliance testing framework
Description
Standards compliance has always been one of the main objectives for the FreeBSD project. In the past we had some efforts to follow regular testing procedures but we haven't followed up the efforts with proper and sustainable infrastructure.
The Open Group has made some testsuites freely available. In particular there used to be a FreeBSD port of the TET testing suite, but this was removed due to a lack of maintenance and other issues. The lsb-vsx Linux testsuite should also be cleaned up so that together we are able to do regular testing on both linux-emulation and FreeBSD-native support.
Any compliance issue should be reported in Bugzilla and an attempt to contact the respective group should be made to draw a plan towards compliance.
Travis Continuous Integration Support for FreeBSD
Technical Contact: rodrigc@,
Description
Travis Continuous Integration is a very popular Continuous Integration system used by projects hosted on GitHub. If a GitHub project has a .travis.yml config file in the root directory, the Travis system will build and test the project if new code is committed to the GitHub project.
Currently, Travis only supports Linux and MacOS X. The Travis project closed issue 1818 which was a request to add FreeBSD support to Travis, due to lack of resources. However, having this support would be very useful, and allow FreeBSD to test many third party projects on GitHub.
Requirements
- Knowledge of Shell scripting in /bin/sh, Python, Ruby
Basic knowledge of REST API's and the GitHub REST API
Knowledge of virtualization systems, such as bhyve, QEMU, KVM
Userland / Installation Tools
Profile Guided Optimization (PGO)
Both GCC and clang offer the option of generating code that is optimized with CPU programming counter data by tools like Linux perf or FreeBSD pmc(3).
With Linux, the recommendation is to use the -b flag to use the Last Branch Record (LBR) to record call chains. This information is then converted with autofdo to generate a profile.
FreeBSD has no similar process. We should enhance tooling to enable generation of profiles in a compatible format.
Documentation
Profile Guided Optimization – Clang Compiler User's Manual
LLVM SPGO profile generation tool:
llvm-profgen14, llvm-profgen(1) https://man.freebsd.org/cgi/man.cgi?query=llvm-profgen14&sektion=1&manpath=freebsd-ports
llvm-profgen15, llvm-profgen(1) https://man.freebsd.org/cgi/man.cgi?query=llvm-profgen15&sektion=1&manpath=freebsd-ports
llvm-profgen16, llvm-profgen(1) manual page installable with devel/llvm16, the online version is not yet available (2023-07-18).
NDMP data server
URL: Network Data Management Protocol (NDMP)
The NDMP initiative was launched to create an open standard protocol for network-based backup for network-attached storage. Major commercial storage systems come with a compliant service. This allows major commercial backup systems to backup such NAS devices. Including a NDMP disk server into FreeBSD would allow to play nice out of the box (modulo some configuring) regarding backups in a corporate environment.
- Evaluate the existing revisions of the NDMP standard.
- Choose an appropriate revision (after checking of supported versions in commercial backup systems).
- Implement at least a NDMP data server.
- Bonus: implement a NDMP tape server (to allow attached tapes to be used).
Requirements
- Access to a commercial backup system with NDMP support (mostly for interoperability testing; since a NDMPcopy application seems to be available, this is not a hard requirement).
- Good knowledge of a programming language which is included in the base system.
- Knowledge about UFS snapshots.
Port prebind from OpenBSD
The OpenBSD prebind is a secure implementation of prelinking that is compatible with address space randomization. Prelinking allows to speed up application startup when a lot of libraries are involved. This should show a noticeable effect with e.g. GNOME/KDE.
Requirements
- Good C knowledge (reading and writing).
Proxy auto-config file support for libfetch
A proxy auto-config (PAC) file contains a JavaScript function "FindProxyForURL(url, host)" that determines which HTTP or SOCKS proxy, if any, to use to access a given URL. In most application the file may be specified manually or discovered using the Web Proxy Autodiscovery Protocol. Support for PAC files in libfetch would make fetch more versitle.
Supporting PAC files nominally requires a fairly complete JavaScript implementation. Google's V8 JavaScript engine is BSD Licensed, however it compiles code to native machine code so platform support is an issue. However, the parser etc may provide a good starting point, and other engines may also exist and should be evaluated. A minimalist implementation of the language with commonly used constructs such as if/else, string comparison, and functions would be sufficient in many cases.
References:
Requirements
- Strong knowledge of secure C programming.
PXE Installer
It would be great to have a bundled PXE installer. This would allow one to boot an install server from a FreeSBIE live CD-ROM on one box, set the BIOS on subsequent boxes to PXE boot, and then have the rest happen by magic. This would be very helpful for installing cluster nodes, etc.
m@ is working on a bundled PXE installer as part of his BSDInstaller project within the Google Summer of Code 2006. The PXE Installer is working but some non-PXE related issues have to be solved before it can enter the tree.
Requirements
- Good PXE knowledge.
Improve cron(8) and atrun(8)
Currently, cron(8) and atrun(8) are outdated in their implementation. Here are some directions for improvement:
- Update cron(8) to ISC cron with security fixes from OpenBSD.
- Integrate the atrun(8) functionality into cron(8), as it was done in NetBSD.
Requirements
- Strong knowledge of the C language and Unix API.
libpw
Technical Contact
Create a library to be able to manage users/groups easily, it also should have a pam/nss-like plugin framework for different account systems.
libutil has pw_* and gr_* undocumented functions which allow user/group manipulation
writing pw_*() and gr_*() manpages
known users for that library: pkgng, pw(8)
Requirements
- Knowledge of C
- Knowledge of pam/nss
Safe crash dumps
Technical Contact: gavin@
Crash dumps are important for collecting debugging information but are also disabled by default because they can consume much space in /var if the user doesn't pay attention to them.
What we miss is a safe way to enable crash dumps by default without having to worry about them filling up /var.
Requirements
- Knowledge of C
- Knowledge of the FreeBSD system internals
Import syslogd improvements from NetBSD
Technical Contacts: emaste@, markj@
Note - there is work in progress on this project, please contact Ed or Mark for details.
NetBSD's syslogd has a number of improvements from a former Google Summer of Code project available for porting:
- new syslog protocol api syslogp(3) that supports structured data and draft-rfc timestamps
- reliable tcp connections with queuing
- encrypted connections (TLS)
The changes are in NetBSD's repository in src/usr.sbin/syslogd, available for viewing on their cvsweb:
Requirements
- Good knowledge about the C programming language.
RAID and disk monitoring suite
Technical Contact:
There have been several organizations that have independently developed RAID and disk failure monitoring tools. These should be gathered together into a unified group of daemons and monitoring scripts to provide a consistent view of disk status.
Requirements
- Knowledge of /etc/rc.d scripts and ordering
- Basic understanding of C and UNIX
- Shell scripting in /bin/sh
Global Projects (may touch everything)
EPUB Support in Documentation Build Infrastructure
Suggested Summer of Code project idea
Enhance the FreeBSD Documentation Project build infrastructure to generate EPUB format output suitable for eBook readers from such as iPads and Kindles.
Requirements
- Ability to work with Makefiles
- Knowledge of SGML/XML transforms
PerfVisor (PERFormance adVISOR)
Coordination: netchild@
Possible Old or No Interest
Document all sysctls
Technical Contact: brd@, eadler@
Description
The sysctl(8) utility retrieves kernel states and allows processes with appropriate privilege to change kernel states. On request it is able to display description lines which document the kernel state. Unfortunately not every sysctl is documented. This task is possible to share with other volunteers. mat has done some development in Perforce, in the mat_sysctl_cleanup branch.
- Find every undocumented sysctl in the kernel.
- Try to determine what this sysctl is for and document it.
Requirements
- Ability to read and understand foreign C code.
BSD-licensed Text-Processing Tools
Part of Summer of Code 2010, Part of Summer of Code 2008
Technical Contact: gabor@
diff/diff3/sdiff: Many command-line options are supported but some features are still missing. Maybe the three programs can be integrated into a single binary, this should be evaluated. A thorough performance benchmark should also be done. See SummerOfCode2012/JesseHagewood for last status.
mdocml: Some groff features are very hard to implement but they aren't strictly needed to render our man pages. Yet some manuals do not compile with mdocml. Investigate the reasons and create a migration plan.
Requirements
- Knowledge of C.
The goal of this project is to get a tool which can analyze a system for performance bottlenecks and maybe even give some hints what to do next.
This is not a GSoC project. Maybe small parts of it can be done during a GSoC.
Prerequisites you should know/read to understand the project steps:
http://dtrace.org/blogs/brendan/2012/12/13/usenix-lisa-2012-performance-analysis-methodology/
http://queue.acm.org/detail.cfm?id=2413037 or http://dtrace.org/blogs/brendan/2012/02/29/the-use-method/
http://dtrace.org/blogs/brendan/2012/03/01/the-use-method-solaris-performance-checklist/
http://dtrace.org/blogs/brendan/2012/03/07/the-use-method-linux-performance-checklist/
http://www.slideshare.net/brendangregg/zfsperftools2012 or http://www.brendangregg.com/Slides/zfsperftools2012.pdf
http://dtrace.org/blogs/brendan/2012/12/10/usenix-lisa-2010-visualizations-for-performance-analysis/
http://dtrace.org/blogs/brendan/2012/03/26/subsecond-offset-heat-maps/
and http://dtrace.org/blogs/brendan/ in general...
Project phases
Initial phase
- Identify/list possible resources in FreeBSD. See the performance checklist postings above for ideas. Those are just a start, this can iteratively be extended to include subitems (e.g. in a first iteration the "network stack" could be an item of a resource, in a second iteration each network interface is a resource, on a third iteration one network protocol could be a resource, on a fourth iteration just the TX queue of a NIC can be a resource, ...). Not everything needs to be identified initially, at one point the work on this resource list should be pushed back to a point where all or some of the following items are handled for the existing resources. When all OS resources are listed, further iterations could even include 3rd party applications (e.g. apache httpd requests, mysql table scans, ...). Interesting values for each resource can be bandwidth (total/percentage), operations per second, latency, usage duration, time until completion, ... Ideally the result is written down in the FreeBSD wiki. For each "high-level" resource (e.g. network stack) try to document some kind of "where to look next if saturated/on-error/..." resource (e.g. network interface, protocol checks, ...)
- For each resource identify/document "utilization", "saturation" and "error" values (see the USE method in the prerequisites), and code points in the source which show them. If there's no code point in the source, mark the resource for later investigation. Hardware provided info, e.g. CPU counters, count as as "code points" in this sense too.
- For each resource determine/document existing tools which provide the required information (saturation/error) and how to use them to get the required info.
- For each resource without an existing way to get the required information, but with a code point in the source which provides the required information, write a dtrace/hwpmc/whatever script/program/whatever to query the required information.
- For each resource without a code point in the source which provides the required information, find a "cheap" way to determine the resource value and add corresponding code readable by sysctl or dtrace (readout variable on existing probe or a specific SDT probe just for this) or whatever (and write a script/program/whatever as above).
Improvement phase
- For each existing code point with an existing script check if a new way of handling this info (sysctl, dtrace probe, ...) would make the sampling of this data cheaper (= less performance impact of the performance monitoring itself).
Visualization phase
- Write a tool (for the ports collection) which visualizes data (heat maps, flame graphs, ...) gathered from the above performance data collection scripts (either stored somewhere or by calling the scripts directly) and provides hints where to look next in case a resource is the bottleneck. The blog of Brendan Gregg gives a lot of ideas how to visualize with various types of graphs. Suggestion: make it system independent, foresee use on *BSD, Solaris and Linux.
Cloud phase (everything is cloudy nowadays...)
- Write an agent which is able to collect the data which the visualizing tool is able to handle and extend the visualizing tool to collect data from agents on other machines (via an encryted and authenticated connection if needed = possible but not mandatory, depending on the environment).
Datawarehouse phase (buzzword bingo!)
- Write a gateway/collector service which a) is able to "handle" agents in a datacenter (query and store data), b) is able to handle bulk-data-retrieval requests from remote locations (transfer data from remote datacenters to the company headquater), and c) is able to store/archive queried data for later use ("How did this look last week/month/year?").
Discussion
The first item in the initial phase is a big item. It may be better to see the complete initial phase as an iterative or divide and conquer step. First determine some high level resources and process all steps of the initial phase on them. Then repeat the initial phase again by braking up the high level resources in more detailed items (e.g. first "all CPU in system" as one resource, second "each CPU in system" as a resrouce, third breaking up each CPU via hardware performance counters).
GSoC info
Each phase is a big project of its own. Do not expect to be able to do it during some weekends or during a GSoC. If you think you can, you did not get the full scope, think again. If you still insinst after rethinking, be our guest, but you better prepare a very good outline what you want to do when, how, at which level of detail (e.g. come with a list of resources for the first item in the initial phase), and also include what you will NOT do/cover/handle but could be related (e.g. CPU internal performance counters if you want to handle the CPU performance side of this). You already need to know FreeBSD (you use it already on your server or desktop since several months) and you shall not be afraid to ask questions or discuss on the mailinglists. This is not a project for 2h per day, if you are not motivated to spend time on this, you better chose a different topic.
This can be made "big enough" to be a project suitable for finals at your university (depending on the requirements of your university, with or without extending it over the end of the GSoC).
Ports
`make db` target in ports
Technical Contact: bapt@
A target in ports tree root that pulls all possible information out of the prots tree into a db, for example, sqlite. From which many things can be query fastly and historical data can be retrieved by querying the snapshots.
References:
Requirements
- Scripting
- Basic knowledge in database
Other
If you are interested in working on a project not explicitly mentioned above, you may want to contact one of the potential Technical Contacts below:
File Systems: ScottLong AlfredPerlstein
Networking: AlfredPerlstein RobertWatson AndreyElsukov
Wireless (802.11) Networking: AdrianChadd BernhardSchmidt
Release Engineering: ReleaseEngineering
Additionally, there are a lot of interesting mailing lists that can be used when searching information about specific subjects.
For porting ideas go to WantedPorts
For Beginner tasks check out JuniorJobs