IDB Profiler

Ian,

could you briefly describe how the Profiler works?

One first sight it looks like Black Magic :)

Günther

Günther,

> could you briefly describe how the Profiler works?
>
> One first sight it looks like Black Magic :)

I feel the same way.

OK, a bit of history first (my memory is fading so any corrections welcome).

Back in the early days of Dolphin David Jones, one of the original
developers,  came up with a package named Svengali.  It was a combined
profiler/execution tracer/object allocation monitor - and a few other
interesting bits that I can't remember.  It was written using a combination
of an early Dolphin beta and a (C++ ?) dll to do all the fast graphics
stuff,

David left Intuitve and Dolphin progressed into it's first release.
Svengali didn't work (as well) on the new Dolphin versions so, with David's
permission, I hacked it around to get it working and made it available to
others.  This lasted through a couple of Dolphin releases when I decided the
amount of hacking needed was a bit much and stopped trying.

I then "borrowed" the profiling engine from Svengali, tweaked it a bit, and
added a new framework around it - this became my Profiler package.


The concept is actually very simple - I don't know how much detail you want
so I'll keep the explanation simple as well (and it saves me having to work
it out it detail again :-) ).

When you profile something the code is contained within a block that is
passed to the profiler.  The profiler (see the Profiler class) then creates
a new process at a priority one stop higher than the current process'
priority - this is the profiling process.  Both blocks are started,
synchronized using a semaphore.  The profiling (higher priority) sleeps and
this allows the profiled code (lower priority) to run.

Every so many mS the timer in the profiling process expires and the
profiling process takes control.  It takes a snapshot of the current stack,
uses the frames in the snapshot to walk a tree (see ProfileTreeNode and
ProfilerFrame) and then adds a new sample (instance of ProfilerSample)
pointing at the leaf node of the stack snapshot.  Walking back up the tree
from the leaf to the root will therefore replicate the frames in the
snapshot.

The profiler process then sleeps again so the profiled process can run.

When the profiled process finishes various operations are performed on the
tree that has been created (to work out the timing and count percentages for
each node) and the results, along with the raw samples collected, are
presented in various ways.

The ProfilerSampleSet class a number of different profiler runs to be stored
and compared - although the tool to make the comparison easier never
materialized.

I have actually got a newer version of the Profiler half completed.  It
simplifies some things (no sample sets for instance) but I had some
decisions to make (about how the tree should display nodes) and couldn't
decide what was best - so the project stay's half completed.  One day maybe.


One final note.  Although the timer in the profiler process may expire it
doesn't mean that Dolphin will immediately change processes and take a
snapshot.  Dolphin will only recognize interrupts at certain points -
basically before and after a "normal" message send.  This results in the
sample points used by the profiler being a bit restricted - for example, you
will never see a sample point in an accessor method.  I don't think it
detracts from the sampling process but it can mean that some leaf nodes in
the tree don't look quite right.  A leaf node may be visited 10000 times say
but a method called by that leaf node may never show up in the profiler
tree.

You can get round this by using an ExecutionTracer (basically automating the
Dolphin single step mechanism) but it's very slow and, from my
experimenting, is very difficult to control.


I think that's enough for now :-) If you've got any questions, or need
something explained in more depth, then please ask.  Don't expect an
immediate answer though :-)

--
Ian

Use the Reply-To address to contact me (limited validity).
Mail sent to the From address is ignored.