Extensions to the netmap framework
Student: Stefano Duo (<duostefano93 AT SPAMFREE gmail DOT com>)
Mentors:
Giuseppe Lettieri (<g.lettieri AT SPAMFREE iet DOT unipi DOT it>)
Vincenzo Maffione (<v.maffione AT SPAMFREE gmail DOT com>)
Project description
The project aims to:
- Write a kernel module to extend VALE switches to support the IEEE 802.1Q standard (vlans).
- Develop a testing framework for netmap.
- Extend netmap to support buffer offsets.
Approach to solving the problem
VALE vlan kernel module
Write a char device driver which upon receiving a configuration, through a write() system call, will create a VALE switch with modified forwarding logic (through the API that netmap exposes to kernel modules) to handle vlan tags inside Ethernet headers. More precisely, given that VALE can't multicast frames, we need to create a non modified switch for every existing vlan in the configuration and a single switch with modified forwarding logic. The modified switch upon receiving a frame will act differently based on the receiving interface:
- if the interface isn't the trunk port then it tags the frame with the vlan id represented by the switch connected to its other end and forwards it to the trunk port (if one exists).
- if the interface is the trunk port then it reads the vlan id, untags the frame and forwards it to the interface which has its other end connected to the switch representing that vlan id (if one exists).
Per-ring fixed offsets
Given the fixed nature of this type of offsets, the approach I have followed is to let netmap internally handle offset math, thus giving directly the "real" buffer starting address, and then let the application "go backwards" if it needs the offset space.
From the userland perspective a buffer is obtained through the NETMAP_BUF() macro (defined in netmap_user.h), which receives a struct netmap_ring and the index of the requested buffer. Therefore a new field has been created inside the struct netmap_ring to store the offset for that ring. To maintain the kernel-userland ABI the sem field, which is basically a placeholder for new fields, has been resized. After those changes NETMAP_BUF() has been modified to handle the offset internally, thus applications only needs to be recompiled to be able to process offsets passively.
From the kernel perspective a buffer logical (and physical) address is obtained through NMB() and PNMB() (defined in netmap_kern.h). They receive a struct netmap_adapter and a few other things. They have been modified to receive the struct netmap_kring which contains the buffer instead of the adapter (it can be retrieved from the kring). A new field to store the offset value has been added to the struct netmap_kring as well. Moreover NETMAP_BUF_SIZE() now accepts a new parameter: uint64_t offset, which is subtracted to the buffer real size. This way the caller directly receives the size of the space left after applying the offset. By changing the function signature we're sure that all the code which requires a buffer size has been "updated", otherwise netmap would not compile.
Finally a new option has been added to let the user specify if he or she wants to open an interface with offsets.
Per-buffer variable offsets
It follows the same idea of per-ring fixed buffer offsets and its stored inside the struct netmap_slot. Because the ownership over the slots is well defined, this type of offset can be changed any time the kernel (or the userspace applications) owns the slot.
Kernel side NMB() and PNMB() already receives the struct netmap_slot so we only need to the the offset field into account. Buffer overflow protection is accomplished through the offset parameter of NETMAP_BUF_SIZE(), just like per-ring fixed offsets.
From the userspace perspective this type of offset needs to be handled explicitly by the application. It cannot be handled transparently by the NETMAP_BUF() macro, because it doesn't receive a struct netmap_slot as one of its parameters, and the macro cannot be changed otherwise backwards compatibility cannot be obtained. Therefore problems could arise during zero-copy (or even while receiving slots from the kernel, if netmap starts using this type of offset itself). An option has been added to let the application specify that it won't handle per-slot variable offsets. Accordingly to this option the memory allocator gets flagged, so that every buffer from that allocator are used either by application aware of per-slot buffer offsets or by aware application.
Testing framework
functional.c (from the netmap repository) is a CLI tool already capable of sending and receiving UDP packets with specified length and a few of other possible options. It has been extended it with basic functionalities that can be used as building blocks for writing tests (e.g. send/expect packet with a specific payload and/or a specific MAC address etc.).
For more complex behavior multiple calls to functional.c are needed, and between those calls an interface may need to stay up, thus functional.c can start and stop a daemon. This daemon acts like a file descriptor server and can hold netmap interfaces file descriptors and lend them on request.
Furthermore i wrote a bash library which eases and shortens the time required to write a test script. It provides wrappers around the most common operations executing during a test, reducing the amount of boilerplate code needed. The wrappers provided to create VALE persistent ports and VETH interfaces and attach/detach interfaces to VALE bridges automatically handle the cleanup after the test ends (implemented through traps).
Notes
Per-ring fixed offset behavior
This type of offset is usually applied to the rings specified by the register request but there are some exceptions.
- netmap pipes: the offset is applied to every ring of both ends of the pipe. This is needed because the exchange buffers through zero-copy.
- netmap monitors: the offset cannot be specified by the application, but it is retrieved from the monitored rings. This is needed because having a different offset from the monitored rings might cause a packet to require more slots inside the monitor ring.
Deliverables
Implement an external kernel module which extends VALE switches to support virtual lans (IEEE 802.1Q standard) and a CLI tool to easily manage vlan configurations.
Implement the testing framework as a CLI tool with a set of basic functionalities which will be used to write a suite of tests for netmap.
Extend netmap to support per-ring buffer offsets and make use of the inside the vlan external module; then add support for per-buffer offsets.
Milestones
- May 14th: Start of coding
Write a kernel module which implements an hardcoded vlan configuration.
- May 21st: Week 2
Allow configurations to be managed through system calls instead of being hardcoded.
- May 28th: Week 3
Write a CLI tool to manage configurations.
- June 4th: Week 4
Bug fixing and documentation.
- June 11th - June 15th: Mid-term Evaluations #1
- June 18th: Week 6
Extend functional.c to manage file descriptors of interfaces.
- June 25th: Week 7
Extend functional.c with functionalities needed to compose tests.
- July 2nd: Week 8
Write a suite of tests through the new framework.
- July 9th - July 13th: Mid-term Evaluations #2
- July 9th: Week 9
Extend netmap to support per-ring buffer offsets.
- July 16th: Week 10
Extend the VALE vlan kernel module to make use of buffer offsets.
- July 23rd: Week 11
Extend netmap to support per-buffer offsets.
- August 6th: End of coding (soft)
Clean up code and documentation.
- August 14th: End of coding (hard)
Test Plan
The netmap testing framework, developed during this project, will be used to test the VALE vlan kernel module and the buffer offsets implementation.
The Code
https://svnweb.freebsd.org/socsvn/soc2018/sduo/
Future Contributions
Update netmap specific NIC drivers to be offset aware, as they currently do not support any type of offset.