Magic Tunnel Daemon
Note that this project originally started under the name Super Tunnel Daemon. This was later changed to Magic Tunnel Daemon, short mtund.
- 2007-03-24: As part of the proposal an initial implementation has been written. It can tunnel over tcp and udp, load plugins and failover between plugins. See the proposal/source code for more details.
- 2007-04-12: The proposal has been accepted for Summer of Code 2007.
- 2007-04-16: This wiki page was set up.
2007-04-16: Agreed on a name for the project: mtund for Matus/Magic Tunnel Daemon.
- 2007-04-17: Imported source code into the perforce repository.
- 2007-04-23: Started email discussion to shape The Big Plan (design, implementation details,...)
2007-04-25: The Big Plan now lives in perforce: design.txt
- 2007-06-10: Updated design.txt (The Big Plan) with replies from Max and Brooks from 2007-04-23.
- 2007-06-18: First part of libevent rewrite.
- 2007-06-20: Setup of a qemu host for testing.
- 2007-06-24: Added a function in the daemon which plugins use to report errors.
- 2007-07-06: Added a keep-alive/ping mechanism for checking unreliable connections. UDP failures are now successfuly detected.
- 2007-07-07: Submitted a FreeBSD status report.
- 2007-07-09: Added a very basic ICMP plugin.
- 2007-07-19: Added fragmentation support.
- 2007-07-23: Added framing support.
- 2007-07-26: Fixed fragment reassembly - queue(3) rules.
- 2007-08-03: Kernel patch to allow processing of ICMP echo request in user space.
- 2007-08-05: ICMP plugin now simulates ICMP echo request-reply exchanges, but still needs one more rewrite.
- 2007-08-07: ICMP plugin correctly implements the polling strategy and actually works. The client should regularly send a request so that the server could reply with data. If the server does not send back a reply, the next request from the client should follow after a timeout period. If there was data from the server, another request should follow immediately.
- 2007-08-09: Added a TCP_CATCHALL socket patch for binding to all unused TCP ports.
- 2007-08-11: Added a sysctl variable net.inet.raw.udp_catchall to receive unclaimed UDP traffic on a raw socket, i.e. receive on all unused UDP ports.
- 2007-08-18: multi-user support on the server
- 2007-08-18: TCP catchall plugin - listens on all unused TCP ports
- 2007-08-19: UDP catchall plugin - listens on all unused UDP ports
- 2007-08-20: DNS plugin
- 2007-08-23: update kernel patches to -CURRENT. This seems to exhibit new bugs in the user space code, which I am fixing now.
- 2007-08-26: Fixed some off-by-one and other stack-corrupting bugs. mtund now runs also on -CURRENT. Not sure why these bugs were fatal on -CURRENT but not on 6-STABLE.
- 2007-08-26: Sent a patch to iodine's authors fixing some off-by-one bugs.
2007-09-11: EuroBsdCon 2007 poster
- HTTP plugin
- config file format and parsing
- commands to configure tun interface
- plugin loading
- plugin parameters
- configure the tun interfaces (currently calculated from client ID)
- the client provides its view of the network, i.e., getifaddrs(3)
- the server offers a non-conflicting IP address for the tun interfaces
- use byteorder(3) functions, i.e., ntohs/htons, for data sent over the network
- ICMP plugin - firewall probing
- DNS plugin should also support a UDP tunneling mode on port 53 (or pass the connection to the UDP plugin)
- HTTP plugin should also support a TCP tunneling mode on port 80 (or pass the connection to the TCP plugin)
- use sendmsg(2)
- use non-blocking io
- TCP plugin
- man page (mostly config file format)
- port skeleton
- make a release
- code cleanup
- UDP catchall, ICMP, DNS plugins
A snapshot of the code can be obtained from http://www.inf.ethz.ch/personal/mharvan/src/mtund.tar.gz Note that you should also apply the kernel patches to use all features.
The source code is also available in the perforce repository. Use this for a more recent version.
IP can easily be tunneled over a plethora of network protocols at various layers, such as IP, ICMP, UDP, TCP, DNS, HTTP, SSH to name a few. While a direct connection may not always be possible due to a firewall, the IP packets could be encapsulated as payload in other protocols, which would get through. However, each such encapsulation requires the setup of a different program and the user has to manually probe different encapsulations to find out which of them works in a given environment.
The aim of this project is to implement the Super Tunnel Daemon, a tunneling daemon using plugins for different encapsulations and automagically selecting the best encapsulation in each environment. Should the environment change, the user would not notice the transition to a different encapsulation except for a small delay. Connections estabilished within the tunnel would seamlessly be migrated to a different encapsulation. In this way, mobility would be supported as well, even to the extent of changing between different physical network interfaces, e.g. disabling the wireless interface and plugging in an ethernet cable. New encapsulations could easily be added in the future using the plugin interface.
Benefits to the FreeBSD Community
- one tunneling application to do it all
- automated probing of encapsulations
- seamless transition between different encapsulations
- mobility support
- easily extendible by new encapsulations via plugins
- a handy tool for firewall rules verification by checking which encapsulations work
- BSD licensed
- the core tunnel daemon
- tun interface handling (open, close, read, write,...)
- loading plugins
- connecting plugins with the tun interface
- probing plugins to find a working encapsulation
- failover between encapsulations
- parsing a config file to determine which plugins to load
- plugins providing encapsulation inside
- UDP, TCP
- ICMP (echo request, reply)
- distinct configuration files for configurable encapsulations
- manpages for the daemon, plugins and configuration files
- FreeBSD ports Makefile
The daemon and plugins would be written in plain C. A tun(4) interface, for which the daemon would be responsible, would serve as the tunnel endpoint. Plugins would be loaded using dlopen(3) and register their file descriptors with the daemon. The daemon would use select(2) to watch file descriptors for activity, read from the tun device, call the necessary plugin read functions and move data between the tun device and the plugins. From the return value of the send/receive functions the daemon would notice a malfunctioning encapsulation and failover to another one. Each plugin would have to implement functions for initializing and uninitializing the encapsulation as well as sending and receiving data.
To check whether this approach could work, I have implemented the daemon with minimal features. It is available at
It handles the tun device, loads encapsulation plugins, probes for a working encapsulation, successfully tunnels IP traffic and does failover between encapsulations. There are two encapsulation plugins, a TCP and a UDP one. Afer adding blocking rules to the firewall, the daemon successfully does a failover to another encapsulation. Detecting encapsulation malfunction works for TCP, but not yet for UDP. Exchanging some keep-alive traffic might be needed for unreliable protocols such as UDP and ICMP. Functions that a plugin has to implement are in plugin.h, functions available to the plugins from the daemon are in tunneld.h and an overview can be found in the README file. Note that the interfaces are just an initial version and may be changed later if needed.
The ICMP encapsulation could easily be done in C. The plain-text protocol encapsulations such as DNS and HTTP could also be written in C, but might require more protocol features to be implemented to allow HTTP traffic to pass through intermediate proxies and DNS traffic via other DNS servers. In such a scenario, the client would have to regularly poll the server and for DNS a proper DNS zone delegation may be needed. For the ssh plugin, forking an ssh(1) or autossh(1) process and talking to it via a pipe might be a viable option.
Authentication and encryption would likely not be addressed. However, one can always set up IPSec on top of the tun interface.
By using the tun interface as the only kernel-specific feature, the daemon and the plugins would essentially live in the user space, increasing portability. While this may not seem as an advantage to the FreeBSD community, the project would benefit a much broader community and possibly encourage more people to contribute new plugins in the future.
The project can be divided into several parts, for which I would estimate the time needed to completion as follows
- core tunnel daemon, config file parsing, plugin interface - 2 weeks
- probing/checking strategy for unreliable protocols (UDP, ICMP) - 1 week
- TCP, UDP plugins - 1 week
- ICMP plugin - 1 week
- HTTP plugin - 1 week
- DNS plugin - 1 week
- SSH plugin - 1 week
- man pages, ports Makefile - 1 week
The rest of the time would be used for dealing with unforeseen problems and for the writeup. Should things be completed faster, more plugins could be written or authentication/encryption could be looked into.