Contents
HOWTO add SDT probes to DTrace in the kernel
What is DTrace SDT?
SDT stands for Statically Defined Tracing. The SDT provider in DTrace allows a programmer to instrument specific kernel code with his own DTrace probes. As the programmer has more semantic knowledge about kernel code, this allows to add more interesting probes to the kernel, than an automatic code parsing provider (like the fbt provider) can do.
Necessary includes
The following includes are needed if you want to add a SDT probe:
#include "opt_kdtrace.h" #include <sys/kernel.h> #include <sys/sdt.h>
Defining a static provider
Then you need to define a provider name for your static probes:
SDT_PROVIDER_DEFINE(foobar);
The provider is displayed when you list all dtrace providers. As of this writting DTrace in FreeBSD offers "syscall", "vfs", "fbt" and some more providers. The above would add a provider with the name "foobar".
This provider needs to be specified only once, if you have other source files which shall use the same provider, you need only to declare the provider, but not define it again:
SDT_PROVIDER_DECLARE(foobar);
Currently, adding SDT probes to KLD modules is only supported on 10-CURRENT as of r233552. It is not supported in 9.0-RELEASE or what will be 8.3-RELEASE. There is nothing in the code that will prevent you from adding probes to a KLD module but it is fairly easy to panic the system if you do.
Defining static probes
Probes need to be defined first (multiple places in the source can fire the same probe, if the same probe shall be fired from within different source files, the probe needs to be defined at one place globally, and declared in all places where it is used).
Defining a probe (you only need to specify arguments, which you want to provide in the probe):
SDT_PROBE_DEFINE(provider, module, function, probename); SDT_PROBE_ARGTYPE(provider, module, function, probename, 0, "type"); SDT_PROBE_ARGTYPE(provider, module, function, probename, ..., "type"); SDT_PROBE_ARGTYPE(provider, module, function, probename, 4, "type");
The macro to fire a probe is limited to 5 function arguments. TODO: investigate if more arguments can be specified in a different way of probe firing (have a look at the SDT_PROBE macro in sdt.h).
Declaring a probe:
SDT_PROBE_DECLARE(provider, module, function, probename);
Note that FreeBSD currently does not support adding probes defined in one KLD module to a provider defined in a different KLD module. In other words, if foo.ko declares a foo provider, only foo.ko is allowed to define probes in that provider. The kernel linker does not enforce this restriction and if bar.ko tries to add a probe to the foo provider, this will appear to work. However if bar.ko is unloaded its probes will not be correctly destroyed until foo.ko is unloaded, so any attempt to use DTrace after bar.ko is unloaded but before foo.ko is unloaded will cause a panic.
Note that this restriction also applies to providers declared in the kernel. The kernel defines a lockstat provider and you will likely trigger a panic if any KLD module adds probes to the lockstat provider.
Example
Assumption, you have
- a file "source_file1.c" and it contains the function "foo"
- a file "source_file2.c" and it contains the function "bar"
and you want to add probes for
- the function entry point (defacto std name is "entry", see in the example)
- the function return (defacto std name is "return", see in the example)
- an error condition in one of the functions (come up with a sensible name yourself)
See further down the page for the functions where those probes are fired to see what each probe is doing.
#include "opt_kdtrace.h" #include <sys/kernel.h> #include <sys/sdt.h> SDT_PROVIDER_DEFINE(foobar); SDT_PROBE_DEFINE(foobar, source_file1, foo, entry); SDT_PROBE_ARGTYPE(foobar, source_file1, foo, entry, 0, "int"); SDT_PROBE_ARGTYPE(foobar, source_file1, foo, entry, 1, "const char *"); SDT_PROBE_DEFINE(foobar, source_file1, foo, return); SDT_PROBE_ARGTYPE(foobar, source_file1, foo, return, 0, "int"); SDT_PROBE_DEFINE(foobar, source_file2, bar, entry); SDT_PROBE_DEFINE(foobar, source_file2, bar, my_error_condition_name); SDT_PROBE_DEFINE(foobar, source_file2, bar, return);
Instead of "source_file1" and "source_file2" you can also use different names, e.g. in the same source file you can add different names here. This part of the definition tells about the "module" in which you are. In case you have a source file which contains several logical "modules" (whatever a module is in your opinion) you can use different names for those modules. So far we have no rule what to use here, so if you don't have a better idea what to use in your code, you can use the filename without the .c ending here. When such a probe fires, the developer looking at problems quickly get's an idea where he has to look at things. For subsystems like GEOM, it would make sense to use "geom" as the provider name, the core GEOM functions get "core" as the module name, and specific GEOM providers use their name for the module, e.g. geom:mirror:functionX:entry. This is not set in stone, it's just a suggestion. Maybe it makes even more sense to have a provider in each GEOM provider.
Instrumenting code with probes
Example:
int
foo(int a, const char *b) {
int c;
SDT_PROBE(foobar, source_file1, foo, entry, a, b, 0, 0, 0);
if (a == 3) {
SDT_PROBE(foobar, source_file1, foo, return, 1, 0, 0, 0, 0);
return 1;
}
c = a*a;
printf("%s\f", b);
SDT_PROBE(foobar, source_file1, foo, return, c, 0, 0, 0, 0);
return (c);
}
void
bar(void) {
SDT_PROBE(foobar, source_file2, bar, entry, 0, 0, 0, 0, 0);
...
if (trigger_error()) {
/*
* Some kind of "milestone", we want to be able
* to trigger a dtrace action in case this happens.
*/
SDT_PROBE(foobar, source_file2, bar, my_error_condition_name, 0, 0, 0, 0, 0);
return;
}
...
SDT_PROBE(foobar, source_file2, bar, return, 0, 0, 0, 0, 0);
}
Corresponding DTrace script
foobar:source_file2:bar:my_error_condition_name
{
printf("ERROR: %s (%d) triggered error condition X in %s:%s:%s!\n", execname, pid, probeprov, probemod, probefunc);
}