Legerdemain: A Program Instrumentation and Analysis Framework

Brandon Lucia
University of Washington
Dept. Computer Science and Engineering

---

Legerdemain on GitHub
[.tar.gz Download]

Overview of Legerdemain

Legerdemain (LDM) is a program instrumentation framework. The purpose of LDM is to provide the ability to instrument C and C++ programs with flexibility, simplicity, and low overhead. Here is a list of things that can be done easily using LDM:

• replace/wrap functions
• monitor program events
• collect information
• instrument thread-related events
• temporally sample events in multi-threaded code

LDM is a framework for building instrumentation tools. A user can write a plugin for LDM that replaces certain functions, monitors certain events, or collects information. Which functions, which events, and what information is up to the discretion of the user.

The main use case of LDM is to create plugins that implement dynamic program analysis tools. LDM is a good platform for writing program analysis tools because it can collect information without requiring kernel-level access or requiring changes to the program being instrumented.

LDM is currently under development, and is already useful for building some interesting program analysis tools. This project is relatively young, and I am hoping it will grow as time goes by. The main LDM library is licensed under the GNU Public License.

Using TAPs with Legerdemain

LDM is centered around the idea of a Transparent Analysis Plugins or TAP. TAPs are shared libraries that are built to be loaded at runtime by LDM. A TAP can implement custom instrumentation and analysis. To use LDM, run

LDM_TAPS=ldm:taps:list ldm.sh yourprogram

This command runs your program, and loads TAPs listed in LDM_TAPS. There are several TAPs included with the LDM source code (in the "clientlibs" directory). The following table summarizes what each of them does, to illustrate the utility of LDM.

TAP	Purpose
Scaley	Monitors the rate of memory allocation and the rate at which data is read from stdin. The tool issues a warning when a program allocates one byte or more per byte read from stdin. The purpose of this tool is to detect likely scalability problem in streaming applications.
LockWatcher	Instruments calls to pthread locking routines
MallocWatcher	Instruments calls to malloc
NullThreadWatcher	Serves as basecode for implementing new TAPs
NullSampledWatcher	Serves as basecode for implementing new TAPs that use temporal sampling

Building Your Own TAPs with LDM

Basic TAPs

TAPs extend LDM's core functionality. The simplest kind of TAP is one that replaces calls to a program function with calls to a function in the plugin. To do so, you can use the LDM_REG, LDM_ORIG, and LDM_ORIG_DECL macros.

Example: Profiling Malloc

---

#include <malloc.h>
#include <stdio.h>
#include "legerdemain.h"

static int inited;
static int malloccount;
LDM_ORIG_DECL(void *,malloc,size_t);
void *malloc(size_t sz){
  malloccount++;
  if( !inited ){
    LDM_REG(malloc);
  }
  return LDM_ORIG(malloc)(sz);
}

static void deinit(){
  ldmmsg(stderr,"[MALLOCWATCHER] Unloading fancy malloc plugin\n");
  ldmmsg(stderr,"[MALLOCWATCHER] Malloc was called %d times\n",malloccount);
}

void LDM_PLUGIN_INIT(){
  ldmmsg(stderr,"[MALLOCWATCHER] Loading fancy malloc plugin\n");
  LDM_REG(malloc);
  inited = 1;
  LDM_PLUGIN_DEINIT(deinit)
}

---

The MallocWatcher TAP can be built, by running

gcc -fPIC -shared -o MallocWatcher.so MallocWatcher.c -ldl -L/path/to/ldm/libs -lldm


The plugin is initialized by the special function LDM_PLUGIN_INIT.  This
function will be called when LDM registers the plugin.  In
LDM_PLUGIN_INIT, LDM_PLUGIN_DEINIT(deinit) is called to register a function
that should be called when the plugin is unloaded.

This plugin uses LDM_ORIG_DECL to declare that the function malloc is being
replaced.  Then, in the plugin's version of malloc(), the original version of
malloc() is called using LDM_ORIG(malloc).  Finally, LDM_REG is called in
LDM_PLUGIN_INIT() to register the replacement with LDM.  

In some cases -- like this one -- the function you are replacing may be called
during your plugin's initialization.  You need to be sure it isn't being called
before LDM_REG() is called.  The "if( !inited )" guard in malloc() in the above
example illustrates this.  


To run a program with this TAP enabled, use the command:

LDM_TAPS=MallocWatcher.so ldm.sh yourprogram

Generally, the LDM_TAPS variable loads LDM plugins at runtime, within ldm.sh.
Note that plugins are *not* thread safe, unless you make them thread safe.  For
example, if this plugin is run on a multithreaded program, malloccount would
need to be protected with a lock. 

TAPs for Multithreaded Programs
	
A TAP may also define a thread constructor and destructor.  A thread
constructor is called by every thread just before entering its start_routine.
Thread constructors are declared like this:

void LDM_PLUGIN_THD_INIT(void *arg,void *(*start_routine)(void *));

The "arg" argument is the void * argument passed to pthread_create when the
program created the thread.  The "start_routine" argument is the void *(*)(void
*) argument passed to pthread_create.  The start_routine is exposed to the TAP
so a program may selectively apply instrumentation to only threads executing a
certain function.  Many thread constructors can be applied to a running program
simultaneously, but only one may be specified per TAP.

A TAP may also specify thread destructor routines.  There are several things
that are required to ensure thread destructors are correctly handled.  First,
the thread destructor must be declared, and associated with a special
destructor identifier using LDM_THD_DTR_DECL(id,func).  Second, in the plugin
initializer (the function declared with LDM_PLUGIN_INIT), you must call
LDM_REGISTER_THD_DTR(id,func).  Finally, in the LDM_PLUGIN_THD_INIT function, each
thread must call LDM_INSTALL_THD_DTR(id), to enable the destructor.

An example of a thread-aware TAP is in clientlibs/NullThreadWatcher.  It
illustrates a thread constructor and destructor function.


Sampled TAPs (sTAPs)
LDM provides support to temporally sample events in your program.  There is a
simple tutorial example illustrating how to use this feature in
clientlibs/NullSampledWatcher.  There are several steps:


First write a TAP (see previous section).  

Call sampled_thread_monitor_init() to initialize the sampling subsystem when
  you intialize your plugin (in your LDM_PLUGIN_INIT function).  

Register threads with the sampling system as they are created.  Do this using
  sampled_thread_monitor_thread_init(), which should be called from your 
  plugin's thread initialization function.  

Create a destructor in your TAP (see previous section).  In your plugin's
  thread destructor, you need to call sampled_thread_monitor_thread_deinit().
  This is important, as if you do not, your program will crash.

In order to do anything in your sTAP, you need to register a
  function that gets called when the sampling subsystem 'poke's your threads.  
  To do so, you need to write a function that gets passed as the second 
  argument to sampled_thread_monitor_init(). The returns void, and is passed two
  arguments by the sampling subsystem when it is called: a SamplingState
  argument, that says whether sampling is being turned on (SAMPLE_ON) or off
  (SAMPLE_OFF); and a ucontext_t * argument, which is the architectural context
  of the thread at the point just before it was poked. 

  
Refer to clientlibs/NullSampledWatcher/NullSampledWatcher.c for an illustrative 
  example of a sTAP.