There's an interesting tidbit on the Solaris Internals wiki regarding Dtrace overhead. Under the section "Dynamic Probes," mdb is used to show dissembled code before and after a probe is enabled on a function. Their example uses an AMD64 system; this entry repeats and extends the exercise using a SPARC system.

We can use mdb to examine live kernel code as follows:

[root@stlawrence]# mdb -k
Loading modules: [ unix krtld genunix specfs dtrace ufs ip sctp usba s1394 fctl nca ... ]
> bdev_strategy::dis
bdev_strategy:                  save      %sp, -0xb0, %sp
bdev_strategy+4:                ldx       [%i0 + 0xa8], %i5
bdev_strategy+8:                sethi     %hi(0x18aec00), %i1
bdev_strategy+0xc:              add       %i0, 0xe0, %o3
bdev_strategy+0x10:             ldx       [%i1 + 0xf0], %l6
bdev_strategy+0x14:             clr       %o0
...

Notice that mdb in kernel mode (-k) first displays the supported loadable modules. Most of these will have man page documentation support. Modules such as s1394 may not have a man page, but almost certainly are discussed somewhere on the web.

The command entered at the default > prompt names the function bdev_strategy(), which translates silently to the function's address in the program map. The following dcmd ("dee command") had two parts: a :: prefix and the command. 'dis' is short for disassemble.

We'll explore the meaning of the first output line some other time. For now, we'll enable a Dtrace probe for the function in another terminal window:

dtrace -qn 'fbt::bdev_strategy:entry {}'

and take aanother look at the kernel code:

> bdev_strategy::dis
bdev_strategy:                  ba,a      +0x3adb38   <dt=0x28c3>
bdev_strategy+4:                ldx       [%i0 + 0xa8], %i5
bdev_strategy+8:                sethi     %hi(0x18aec00), %i1
bdev_strategy+0xc:              add       %i0, 0xe0, %o3
bdev_strategy+0x10:             ldx       [%i1 + 0xf0], %l6
bdev_strategy+0x14:             clr       %o0
...

The line save %sp, -0xb0, %sp has been replaced by ba,a +0x3adb38 <dt=0x28c3> which includes a tell-tale Dtrace tag.

Bear in mind it doesn't matter if you modify the dtrace invocation to include actions. The probe allows an entering thread to invoke the actions that might be included, such as grabbing the thread's own process and a timestamp:

dtrace -qn 'fbt::bdev_strategy:entry { trace (execname); trace(timestamp) }'

The time cost of processing these actions is a function of the frequency of the calling threads. The cost of inserting the probe itself, therefore, can be considered negligible until it is used. However, it is not true that loading probes en masse is free; far from it.

What if we tried loading every probe we could get our hands on? Here's one way to get a bunch, using the pid provider on a freshly booted system:

[root@yukon]# dtrace -l | wc -l
   41879
[root@yukon]# ps
   PID TTY         TIME CMD
   548 pts/2       0:00 ps
   540 pts/2       0:00 sh
[root@yukon]# ptime dtrace -P pid540
dtrace: description 'pid540' matched 220327 probes
...
...
real       35.264
user        0.608
sys        23.587
This reported time has a small margin of error to it, owing to the intermittent appliance between my chair and keyboard. But clearly it can be a big purchase to load additional probes (177,448) and insert them.

Nice post. Note, however, that the list of modules that mdb lists is the modules for mdb itself that provide dcmds and walkers. Those are based on the objects loaded in the target (in this case the kernel), but it is by no means a full list of resident kernel modules.
Thanks for stopping by, Adam! I'm flattered. I appreciate the clarification too. Just about all the work I'm doing lately is in JCAPS -- on Windows, no less. I do wish I got to play more and sharpen up on internals, but not much work for me there right now.


Add a comment

Title
Body
HTML : b, i, blockquote, br, p, pre, a href="", ul, ol, li
Math Quiz 8 + 6 = (Helps stop blog spam)
Name
E-mail address
Website
Remember me Yes  No 

E-mail addresses are not publicly displayed, so please only leave your e-mail address if you would like to be notified when new comments are added to this blog entry (you can opt-out later).

TrackBack to http://radio.javaranch.com/michael/addTrackBack.action?entry=1212015094266