Re: [Pharo-dev] Status of the VM?

Maybe I'm missing something obvious, but from my point of
view the fundamental difference here is that with JVM's JPDA
triggering of event actually **does not** affect the monitored
system.

If these events are handled by the image, it is quite tricky to
figure out which one is caused by "application" and which one
is caused by the event handling machinery. For instance,
if (non-empty) method is called after a GC, this handler
may trigger another GC, which may...

That brings me to another, if you do in-image profiling (event handling), data could be quite different than what would you get
using out-of-image profiler.


<personal experience:>
I've used (and implemented some :-) both - Smalltalk in-image tools
and out-of-image tools, namely JPDA and DTRACE/STAP. For serious
profiling and monitoring, I always use out-of-image approach if
possible.

Jan

On 01/03/14 23:34, Clément Bera wrote:

Hey,

It's fun because I looked at the JVM list and most point looked
irrelevant as you described.

It seems that the only thing we miss is the GC hook: we could have a
selector in special object array to call at each GC on the 'Smalltalk'
object to trigger a method in the image that would be by default an
empty method.

Field access is a solved problem with slots, definitely.


2014-03-01 22:35 GMT+01:00 Eliot Miranda <[hidden email]

<mailto:[hidden email]>>:

    Hi Stefan,


    On Sat, Mar 1, 2014 at 12:55 PM, Stefan Marr

    <[hidden email] <mailto:[hidden email]>> wrote:

        Hi:

        On 01 Mar 2014, at 19:44, Alexandre Bergel

        <[hidden email] <mailto:[hidden email]>> wrote:

         > The VM is a formidable thing in which everything happen.
        Exposing to the image what’s going on in it is really pushing
        the innovation.

        Perhaps something that could be relevant in case someone decides
        to implement such an interface in Cog:
        Just one, pretty old example:
        http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#EventIndex


    Interesting list.  Assuming the list refers to events one can handle
    in Smalltalk, then let me go through these and see which we need,
    don't need or already have:


          Event Index

      * Breakpoint
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#Breakpoint>

        we already have this.  An illegal bytecode will send an error
        message to the current context.  See my MethodMassage package.
          We use this at Cadence to implement coverage.

      * Class File Load Hook
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ClassFileLoadHook>

        not needed.  classes are loaded by Smalltalk code, not the VM.

      * *Class Load
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ClassLoad>*
        not needed. ditto
      * *Class Prepare
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ClassPrepare>*
        not needed.  ditto
      * *Compiled Method Load
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#CompiledMethodLoad>*

        not needed.  methods are loaded by Smalltalk code, not the VM.

      * *Compiled Method Unload
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#CompiledMethodUnload>*
        not needed.  ditto
      * Data Dump Request
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#DataDumpRequest>

        to what extent is snapshot adequate?
        Has been used for adequate image debugging (and of course we can
        serialize processes so people are using things like Fuel to save
        the stack traces of processes that encounter errors).
        Not adequate for VM debugging: the heap may be invalid and not
        saveable; shapshot load does more than merely fill memory; it
        swizzles etc, and these operations can and will fail on an
        invalid heap.  Personally I think things like -blockOnError
        which causes the VM to block rather than exit on error, allowing
        one to attach gdb to a crashed VM process is more useful.

      * *Dynamic Code Generated
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#DynamicCodeGenerated>*

        not needed.  methods are loaded by Smalltalk code, not the VM.

      * *Exception
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#Exception>*

        not needed.  Smalltalk's exception system has resume semantics,
        not retry, so there's no need to ask to see an exception at
        source; the source of the exception can be examined after the
        fact (unlike Java, which has restart semantics).  In fact folks
        like Avi Bryant (and I believe him) think that the business
        opportunity with hadoop/map-reduce is indeed Smalltalk's
        exception semantics which allow live debugging of errors.

      * *Exception Catch
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ExceptionCatch>*
        not needed.  ditto
      * Field Access
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#FieldAccess>

        Quite possibly.  If one is using classical Smalltalk there is no
        linguistic hook for field access.  With Pharo's slots then
        presumably transforming methods to insert traps on field access
        is trivial (and databases like GemStone have done similar things
        with brute bytecode manipulation).  With Newspeak, which has no
        direct inst var access, this can be subsumed by MethodEntry (see
        below).  So whether this is needed or not depends on either
        language evolution or good bytecode manipulation tools; if these
        are available this is not needed.

      * Field Modification
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#FieldModification>

        Quite possibly.  Again the approaches outlined above can be done
        (as gemStone has done in the past).  But we nearly have this.
          I've implemented per-instance immutability for the old
        Newspeak VM (a variant of the Squeak interpreter) and this is
        easy to fold into Cog (and indeed the Interpreter VM).  GemStone
        engineers prefer per-instance immutability than their own
        bytecode modification.  I think I agree.  I'd rather depend on
        immutability (which has other uses such as immutable literals)
        than a special VM hook.

      * Frame Pop
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#FramePop>

        Hmm.  Possibly.  IIRC, I think method wrappers provide a
        manageable way to do this.

      * Garbage Collection Finish
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#GarbageCollectionFinish>

        Indeed.  There are hacks to get this.  Further there are many
        ways to implement this.  e.g. is it a callback on finishing a
        form of GC, or is it merely the signalling of a semaphore which
        has some Smalltalk process waiting on it (as is the case in VW)?

      * Garbage Collection Start
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#GarbageCollectionStart>

        OK, but what are you going to do when you catch this?  Arguably
        there's no memory with which to do anything at the point that
        you get this.  Give me a scenario and I'll consider it.

      * Method Entry
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MethodEntry>

        Not sure we need this because all method calls are virtual and
        hence one can use proxies (MNU hook) or method wrappers.

      * Method Exit
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MethodExit>
        Ditto.
      * Monitor Contended Enter
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MonitorContendedEnter>

        Not relevant.  Smalltalk doesn't have synchronized methods.

      * Monitor Contended Entered
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MonitorContendedEntered>
        Ditto.
      * *Monitor Wait
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MonitorWait>*
        Ditto.
      * *Monitor Waited
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#MonitorWaited>*
        Ditto.
      * Native Method Bind
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#NativeMethodBind>

        OK.  Not really relevant if we refactor dll loading and function
        lookup as in Alien (and VW) so that the image does the work and
        allows one to implement the hook without VM support.

      * Object Free
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ObjectFree>

        Hmmm.  Is this for finalization or something else?  In either
        case I think that Ephemerons fit the bill and Cog Spur supports
        ephemerons.  So we'll have this before the end of the year.

      * Resource Exhausted
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ResourceExhausted>

        Hmmm, I think this is not relevant.  Resources such as memory
        produce catchable primitive failures.  Likewise for OS
        resources.  if things are engineered right then exhaustion fo
        things like file handles should produce exceptions in the image.

      * Single Step
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#SingleStep>

        Not relevant.  We already have ways of making Smalltalk
        single-step, heh, heh.

      * Thread End
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ThreadEnd>

        If this applies to Process, then easy.  Change fork: et al to
        add code on thread termination.  Native threads are a different
        issue.  But as yet a non-issue.  We don't have them yet,
        although a prototype VM is there to revive when resources allow.

      * Thread Start
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#ThreadStart>
        Ditto.
      * VM Death Event
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#VMDeath>

        Seems like a non-sequitur to me.  If the VM has died then
        there's nothing reliable the system above it can do.  However,
        see below...

      * VM Initialization Event
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#VMInit>

        The existing startUp registration mechanism seems adequate to me...

      * VM Object Allocation
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#VMObjectAlloc>

        AGain lots of hooks to allow this.  The link os pretty vague on
        what this might mean.  They mention using other mechanisms to
        instrument this.

      * *VM Start Event
        <http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#VMStart>*

        Again, the existing startUp registration mechanism seems
        adequate to me...

    However, if this list refers to sub-image level debugging (and
    that's something I do all the time) then again this isn't relevant.
      We have the simulator which is a fabulous tool for intercepting
    any and all of the above.  Gdb is less convenient but can be used by
    someone knowledgeable.

        You can get access to many different information in terms of
        ‘events’ on the JVM.
        For building a profiler, more than sufficient. And, at least in
        my personal opinion also something that would be desirable for
        Cog, perhaps in a slightly more modern design, but with a
        similar flavor.


    So how about responding to the above breakdown with a sketch of what
    events remain, and then speculate on how they might be packaged as
    events.

    --
    best,
    Eliot