DTrace

DTrace

Developer(s)	Sun Microsystems
Initial release	January 2005
Development status	active
Operating system	Solaris, Mac OS X, FreeBSD, NetBSD, Oracle Linux
Type	tracing
License	Common Development and Distribution License
Website	http://opensolaris.org/os/community/dtrace/

DTrace is a comprehensive dynamic tracing framework created by Sun Microsystems for troubleshooting kernel and application problems on production systems in real time. Originally developed for Solaris, it has since been released under the free Common Development and Distribution License (CDDL) and has been ported to several other Unix-like systems.

DTrace can be used to get a global overview of a running system, such as the amount of memory, CPU time, filesystem and network resources used by the active processes. It can also provide much more fine-grained information, such as a log of the arguments with which a specific function is being called, or a list of the processes accessing a specific file.

Description

DTrace is designed to give operational insights that allow users to tune and troubleshoot applications and the OS itself.

Tracing programs (also referred to as scripts) are written using the D programming language (not to be confused with other programming languages named "D"). The language is a subset of C with added functions and variables specific to tracing. D programs resemble awk programs in structure; they consist of a list of one or more probes (instrumentation points), and each probe is associated with an action. These probes are comparable to a pointcut in aspect-oriented programming. Whenever the condition for the probe is met, the associated action is executed (the probe "fires"). A typical probe might fire when a certain file is opened, or a process is started, or a certain line of code is executed. A probe that fires may analyze the run-time situation by accessing the call stack and context variables and evaluating expressions; it can then print out or log some information, record it in a database, or modify context variables. The reading and writing of context variables allows probes to pass information to each other, allowing them to cooperatively analyze the correlation of different events.

Special consideration has been taken to make DTrace safe to use in a production environment. For example, there is minimal probe effect when tracing is underway, and no performance impact associated with any disabled probe; this is important since there are tens of thousands of DTrace probes that can be enabled. New probes can also be created dynamically.

Command line examples

DTrace scripts can be invoked directly from the command line, providing one or more probes and actions as arguments. Some examples:

# New processes with arguments,
dtrace -n 'proc:::exec-success { trace(curpsinfo->pr_psargs); }'
 
# Files opened by process,
dtrace -n 'syscall::open*:entry { printf("%s %s",execname,copyinstr(arg0)); }'
 
# Syscall count by program,
dtrace -n 'syscall:::entry { @num[execname] = count(); }'
 
# Syscall count by syscall,
dtrace -n 'syscall:::entry { @num[probefunc] = count(); }'
 
# Syscall count by process,
dtrace -n 'syscall:::entry { @num[pid,execname] = count(); }'
 
# Disk size by process,
dtrace -n 'io:::start { printf("%d %s %d",pid,execname,args[0]->b_bcount); }'   
 
# Pages paged in by process,
dtrace -n 'vminfo:::pgpgin { @pg[execname] = sum(arg0); }'

Scripts can also be written which can reach hundreds of lines in length, although typically only tens of lines are needed for advanced troubleshooting and analysis. Over 200 examples of open source DTrace scripts can be found in the DTraceToolkit,^[1] created by Brendan Gregg (author of the DTrace book^[2]), which also provides documentation and demonstrations of each.

Supported platforms

DTrace was first made available for use in November 2003, and was formally released as part of Sun's Solaris 10 in January 2005. DTrace was the first component of the OpenSolaris project to have its source code released under the Common Development and Distribution License (CDDL).

DTrace has been ported to FreeBSD^[3] and NetBSD.^[4]

Apple added DTrace support in Mac OS X 10.5 "Leopard", including a GUI called Instruments.^[5] Over 40 DTrace scripts from the DTraceToolkit are included in /usr/bin,^[6] including tools to examine disk I/O (iosnoop) and process execution (execsnoop). Unlike other platforms that DTrace is supported on, Mac OS X has a flag (P_LNOATTACH) that a program may set that disallows tracing of that process by debugging utilities such as DTrace and gdb. In the original Mac OS X DTrace implementation, this could affect tracing of other system information, as unrelated probes that should fire while a program with this flag set was running would fail to do so.^[7] This problem was addressed a few months later in the Mac OS X 10.5.3 update.^[8]

The Linux port of DTrace has been available since 2008;^[9] work continues actively to enhance and fix issues. Standard core providers are available (fbt, syscall, profile), plus a special "instr" provider (some of the Solaris providers are not yet available). The Linux DTrace implementation is a loadable kernel module, which means that the kernel itself doesn't have to be modified, and also allows DTrace to avoid CDDL vs. GPL licensing conflicts.^{[10] [11]}

DTrace is also being developed to support QNX 6.

Oracle added DTrace support for Oracle Linux in 2011.^[12]

Authors and awards

DTrace was designed and implemented by Bryan Cantrill, Mike Shapiro, and Adam Leventhal. The authors received recognition in 2005 for the innovations in DTrace from InfoWorld and Technology Review.^[13][14] DTrace won the top prize in the Wall Street Journal's 2006 Technology Innovation Awards competition.^[15] The authors were recognized by USENIX with the Software Tools User Group (STUG) award in 2008.^[16]

See also

Free software portal

• SystemTap
• IBM ProbeVue

References

• Bryan Cantrill (February 2006). "Hidden in Plain Sight". ACM Queue 4 (1): 26–36. doi:10.1145/1117389.1117401. http://www.acmqueue.org/modules.php?name=Content&pa=showpage&pid=361. Retrieved 2006-09-08. 
• Bryan M. Cantrill, Michael W. Shapiro and Adam H. Leventhal (June 2004). "Dynamic Instrumentation of Production Systems". Proceedings of the 2004 USENIX Annual Technical Conference. http://www.usenix.org/event/usenix04/tech/general/full_papers/cantrill/cantrill_html/. Retrieved 2006-09-08. 

Notes

1. ^ "DTraceToolkit". OpenSolaris.org. http://hub.opensolaris.org/bin/view/Community+Group+dtrace/dtracetoolkit. Retrieved 2010-05-31. 
2. ^ "DTrace: Dynamic Tracing in Oracle Solaris, Mac OS X and FreeBSD". Safari Books. http://my.safaribooksonline.com/book/operating-systems-and-server-administration/solaris/9780137061839. Retrieved 2011-01-03. 
3. ^ "FreeBSD 7.1-RELEASE Announcement". 2009-01-06. http://www.freebsd.org/releases/7.1R/announce.html. Retrieved 2009-01-06. 
4. ^ "NetBSD source changes, 21 February 2010". http://mail-index.netbsd.org/source-changes/2010/02/21/msg006855.html. 
5. ^ "Mac OS X Leopard - Developer Tools - Instruments". Apple Inc.. Archived from the original on 2007-10-24. http://web.archive.org/web/20071024144916/http://www.apple.com/macosx/developertools/instruments.html. Retrieved 2007-10-19. 
6. ^ "Mac OS X DTrace". Apple Inc.. http://www.opensource.apple.com/source/dtrace/. Retrieved 2010-05-31. 
7. ^ "Mac OS X and the missing probes". Leventhal, Adam H.. January 18, 2008. http://dtrace.org/blogs/ahl/2008/01/18/mac-os-x-and-the-missing-probes/. Retrieved 2008-01-20. 
8. ^ "Apple Updates DTrace". Leventhal, Adam H.. June 7, 2008. http://dtrace.org/blogs/ahl/2008/06/07/apple-updates-dtrace/. Retrieved 2008-06-16. 
9. ^ "CRiSP tools download page.". http://www.crisp.demon.co.uk/tools.html. Retrieved 2011-03-02. 
10. ^ "CRiSP Weblog". http://www.crisp.demon.co.uk/blog/index.html. Retrieved 2009-06-13. 
11. ^ "DTrace Weblog". http://crtags.blogspot.com. Retrieved 2011-02-08. 
12. ^ trying out dtrace
13. ^ "Tracing software in real time". Technology Review. MIT. 2005. http://www.technologyreview.com/tr35/Profile.aspx?Cand=T&TRID=91. Retrieved 2007-03-31. 
14. ^ McAllister, Neil (August 2005). "Innovation is alive and well in 2005". InfoWorld. IDG. http://www.infoworld.com/reports/31SRinnovators2005.html. Retrieved 2007-03-31. 
15. ^ Totty, Michael (September 2006). "The Winners Are...". The Wall Street Journal. Dow Jones & Company, Inc.. http://online.wsj.com/public/article/SB115755300770755096-R2Ct41cQ4ZIPMwk4_xh0xU_HnQI_20061011.html?mod=tff_main_tff_top. Retrieved 2007-03-31. 
16. ^ "2008 USENIX Annual Technical Conference (USENIX '08)". 2008. http://www.usenix.org/events/usenix08/index.html. Retrieved 2008-11-26. 

External links

• DTrace Hands On Lab - A step by step course to learn DTrace
• DLight Tutorial — DLight is an interactive graphical observability tool for C/C++ developers based on DTrace technology from the Oracle Solaris Studio
• Exploring Leopard with DTrace -- how to use DTrace for debugging and exploration
• Google Tech Talk on DTrace given by Bryan Cantrill