FreeBSD SMP Network Performance Wiki Page
Not much here yet, other than a pointer to existing web pages:
Contact
TODO
Some basic TODO items for the Netperf project:
- Continue to investigate the impact of preemption on localhost traffic, wherein the netisr preempts the sending user space thread on wakeup, resulting in a series of untimely context switches, reducing performance. In particular, ideally the wakeup won't occur until the sending thread has released higher level locks that then may be acquired by the netisr. Hacking out preemption in this case seems to improve UP performance fairly dramatically, but hurts SMP performance, suggesting that things are more complicated than they seem.
- Prototype changes to permit non-preemptive wakeups of the netisr have been explored, which seem to restore UP performance as desired. However, SMP performance is still hurt in this configuration, and the model is somewhat less clean. The direction currently being explored is how to change the network stack structure to recognize preemptiveness, rather than the scheduler.
- Continue to investigate offering lighter weight clock mechanisms, and using them in context switches. Many network and IPC applications gather frequent timestamps, which can add up substantially. Also, each context switch takes a time stamp. With slower time counters, this makes a very large performance difference.
- Prototype changes to expose CLOCK_REALTIME_FAST and CLOCK_MONOTONIC_FAST to user space have been created. These offer 1/HZ accuracy for user applications requiring fast and fuzzy time stamps. The prototype has now been committed.
- There will be a FreeBSD developer summit session on time counting and time stamps at EuroBSDCon. This happened.
- Prototype ACPI_HPET changes have been posted to offer an alternative ACPI clock.
- Re-working the scheduler in terms of cycles rather than time has been proposed and is being explored. phk is in the process of committing this to CVS HEAD at the time of writing.
- Revisit ifqueue handoff for sending packets -- right now, the sequence of events involves a number of mutex operations, and a small race in ifqueue flagging. Correcting the race is desirable, as is measuring whether the additional granularity provided by mutexes in the ifqueue outweighs the cost in contentious environments.
- Continue to clean up locking in the socket state machine life cycle, which is currently poor due to a combination of splitting coverage of struct socket under two mutexes, and releasing socket layer mutexes when entering the protocol layer due to lock order.
- Continue to investigate alternative synchronization primitives and approaches that allow fast read and slow write, rather than offering similar or equivalent cost to readers and writers. This is especially required where address lists are walked, ifnet lists, etc, frequently, but modified extraordinarily infrequently.