Intercepting at:put: problems

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Intercepting at:put: problems

Ana Laura Felisatti
 
Hi!

We're working with Hernan Wilkinson on the Live Typing VM implementation and have recently found a very strange behaviour. We are trying to capture the type of objects stored within an OrderedCollection by intercepting the at:put: method of Object, since that's what ends up being executed internally by the OrderedCollection using an internal Array. To identify whether the at:put: is being executed from that OrderCollection context we inspect the MethodContext nesting.

So far we have intercepted the StackInterpreter commonAtPut: to execute our logic, which attempts to retrieve the current MethodContext, analyse up to 4 levels up looking for an OrderedCollection receiver and in that case store the type information.

The problem is that our code never executes on a regular execution but does while debugging. In fact, it works as expected when we simply hit "Proceed" but fails when we hit "Over" instead, which is quite unexpected.

We would like to know if there's anything we might be missing here about the execution of primitives like at:put: or acquiring the current MethodContext from the VM side.

Thanks!
--
A. Felisatti
Reply | Threaded
Open this post in threaded view
|

Re: Intercepting at:put: problems

timrowledge
 


> On 2019-04-28, at 11:04 AM, Ana Laura Felisatti <[hidden email]> wrote:
>
> So far we have intercepted the StackInterpreter commonAtPut: to execute our logic, which attempts to retrieve the current MethodContext, analyse up to 4 levels up looking for an OrderedCollection receiver and in that case store the type information.
>
> The problem is that our code never executes on a regular execution but does while debugging. In fact, it works as expected when we simply hit "Proceed" but fails when we hit "Over" instead, which is quite unexpected.
>
> We would like to know if there's anything we might be missing here about the execution of primitives like at:put: or acquiring the current MethodContext from the VM side.

It's possible that the commonAtPut() routine has been inlined (looks that way in an ancient copy of interp.c I have lying around) and so if you are trapping in a debugger on the actual routine start then it would be unlikely to hit. If you've modified the commonAtPut() code, then this is unlikely to be a good explanation.

How exactly have you generated the VM? That might tell us something useful.


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: IO: Illogical Or


Reply | Threaded
Open this post in threaded view
|

Re: Intercepting at:put: problems

Ana Laura Felisatti
In reply to this post by Ana Laura Felisatti
 
Hi!

Just to clarify, we've actually modified the StackInterpreter>>commonAtPut: with a call to our code. We are building the Stack Spur VM on Mac 64bits.

To validate whether our code was executed or not we've added printf entries to the C code produced in gcc3x-interp.c which is how we noticed the difference between regular execution and debug with proceed or over.

Here's the modified extract of commonAtPut:

!StackInterpreter methodsFor: 'indexing primitive support' stamp: 'ALF & MDS 4/7/2019 18:57'!
commonAtPut: stringy
    "This code is called if the receiver responds primitively to at:Put:.
     N.B. this does *not* use the at cache, instead inlining stObject:at:put:.
     Using the at cache here would require that callers set messageSelector
     and lkupClass and that is onerous and error-prone, and in any case,
     inlining produces much better performance than using the at cache here."
    | value index rcvr |
    <inline: true> "to get it inlined in primitiveAtPut and primitiveStringAtPut"
    self initPrimCall.
    self keepCollectionTypeInformation.
    rcvr := self stackValue: 2.
    index := self stackValue: 1.
    value := self stackTop.
    ...

And here's an extract of our code highlighting how we attempt to get the current MethodContext.

!StackInterpreter methodsFor: 'dynamic type information' stamp: 'ALF MTQP 4/21/2019 15:46'!
keepCollectionTypeInformation
    <inline: true>
    | senderContext receiverObject isLiveTypingCollectionObject contentTypesArray isSenderContextMethodContext collectionsContentType keepSearchingCollectionSender loopbackIndex maxLoopbackIndex|

    loopbackIndex := 0.
    maxLoopbackIndex := 4.
    keepSearchingCollectionSender := true.
    senderContext := self ensureFrameIsMarried: framePointer SP: stackPointer.
    [keepSearchingCollectionSender and: [loopbackIndex < maxLoopbackIndex]] whileTrue: [
        senderContext = objectMemory nilObject ifFalse: [
            isSenderContextMethodContext := (objectMemory is: senderContext instanceOf: (objectMemory splObj: ClassMethodContext) compactClassIndex: ClassMethodContextCompactIndex)-0.
            isSenderContextMethodContext ifTrue: [
                receiverObject :=  objectMemory fetchPointer: ReceiverIndex ofObject: senderContext.
                receiverObject = objectMemory nilObject ifFalse: [
                    isLiveTypingCollectionObject := (objectMemory is: receiverObject instanceOf: (objectMemory splObj: ClassLiveTypingCollection) compactClassIndex: 0)-0.
                    isLiveTypingCollectionObject ifTrue: [ ...

We are using a custom implementation of OrderedCollection, LiveTypingCollection just to limit the code exposure and have made it a known class to the VM for that reason.

Is there anything we are missing around the at:put: primitive execution and the commonAtPut: implementation it triggers? Is there another way to obtain the current MethodContext?

Thanks

On Mon, Apr 29, 2019 at 9:00 AM <[hidden email]> wrote:
Send Vm-dev mailing list submissions to
        [hidden email]

To subscribe or unsubscribe via the World Wide Web, visit
        http://lists.squeakfoundation.org/mailman/listinfo/vm-dev
or, via email, send a message with subject or body 'help' to
        [hidden email]

You can reach the person managing the list at
        [hidden email]

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Vm-dev digest..."


Today's Topics:

   1. Intercepting at:put: problems (Ana Laura Felisatti)
   2. Re: Intercepting at:put: problems (tim Rowledge)


----------------------------------------------------------------------

Message: 1
Date: Sun, 28 Apr 2019 15:04:13 -0300
From: Ana Laura Felisatti <[hidden email]>
To: [hidden email]
Subject: [Vm-dev] Intercepting at:put: problems
Message-ID:
        <CACxgkY=[hidden email]>
Content-Type: text/plain; charset="utf-8"

Hi!

We're working with Hernan Wilkinson on the Live Typing VM implementation
and have recently found a very strange behaviour. We are trying to capture
the type of objects stored within an OrderedCollection by intercepting the
at:put: method of Object, since that's what ends up being executed
internally by the OrderedCollection using an internal Array. To identify
whether the at:put: is being executed from that OrderCollection context we
inspect the MethodContext nesting.

So far we have intercepted the StackInterpreter commonAtPut: to execute our
logic, which attempts to retrieve the current MethodContext, analyse up to
4 levels up looking for an OrderedCollection receiver and in that case
store the type information.

The problem is that our code never executes on a regular execution but does
while debugging. In fact, it works as expected when we simply hit "Proceed"
but fails when we hit "Over" instead, which is quite unexpected.

We would like to know if there's anything we might be missing here about
the execution of primitives like at:put: or acquiring the current
MethodContext from the VM side.

Thanks!
--
A. Felisatti
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20190428/47ad902b/attachment-0001.html>

------------------------------

Message: 2
Date: Sun, 28 Apr 2019 11:20:19 -0700
From: tim Rowledge <[hidden email]>
To: Squeak Virtual Machine Development Discussion
        <[hidden email]>
Subject: Re: [Vm-dev] Intercepting at:put: problems
Message-ID: <[hidden email]>
Content-Type: text/plain;       charset=us-ascii



> On 2019-04-28, at 11:04 AM, Ana Laura Felisatti <[hidden email]> wrote:
>
> So far we have intercepted the StackInterpreter commonAtPut: to execute our logic, which attempts to retrieve the current MethodContext, analyse up to 4 levels up looking for an OrderedCollection receiver and in that case store the type information.
>
> The problem is that our code never executes on a regular execution but does while debugging. In fact, it works as expected when we simply hit "Proceed" but fails when we hit "Over" instead, which is quite unexpected.
>
> We would like to know if there's anything we might be missing here about the execution of primitives like at:put: or acquiring the current MethodContext from the VM side.

It's possible that the commonAtPut() routine has been inlined (looks that way in an ancient copy of interp.c I have lying around) and so if you are trapping in a debugger on the actual routine start then it would be unlikely to hit. If you've modified the commonAtPut() code, then this is unlikely to be a good explanation.

How exactly have you generated the VM? That might tell us something useful.


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: IO: Illogical Or




------------------------------

Subject: Digest Footer

_______________________________________________
Vm-dev mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/vm-dev


------------------------------

End of Vm-dev Digest, Vol 154, Issue 33
***************************************


--
A. Felisatti
Reply | Threaded
Open this post in threaded view
|

Re: Intercepting at:put: problems

Clément Béra
 
Hi Ana Laura,

Glad you're having fun with the VM :-)

We are using a custom implementation of OrderedCollection, LiveTypingCollection just to limit the code exposure and have made it a known class to the VM for that reason.

Wow. Are you sure it works correctly? Did you check the C code generated? Congrats, it is not that easy.

Is there anything we are missing around the at:put: primitive execution and the commonAtPut: implementation it triggers? 

So you've already disabled the JIT since you have a stack VM.

Look at StackInterpreter>>bytecodePrimAtPut
Basically at:put: sends are compiled specially in the image by the bytecode compiler (specific bytecode based on selector name) which has a quicker path of execution in the interpreter with a local cache. If the send lookup results in the method with the at:put: primitive, the interpretation of the bytecode already executes the at:put: primitive without calling the primitive you edited re-using the local cache information to speed up interpreter performance. 

That's a problem in your case.

One solution is just to replace StackInterpreter>>bytecodePrimAtPut by CoInterpreter>>bytecodePrimAtPut, when using the JIT this at/atput cache logic is disabled (See CoInterpreter>>bytecodePrimAtPut method comment).

Is there another way to obtain the current MethodContext?

Yes and no. As far as I understood, you are looking for the receiver type of the top 4 frames. What you are currently doing (or trying to do, I assume your code works, I only read the 2 lines you highlighted) is to reify the top 4 frames as objects, and then read a field in these objects. This is horribly slow (Specifically 4 instance creation at each at:put: !!!!), but it should work. If that's what you want to do you are good.

The better way is to iterate directly the stack and read the receiver fields directly from the stack. You have access to the current FP with the local variable localFP. You can access the next FP with nextFP := self frameCallerFP: theFP. nextFP is 0 if on top of stack or at a stack page switch point. You can access the receiver of the frame with stackPages longAt: currentFP + FoxReceiver. So you could just make a loop, and either be imprecise (if stack page switch, you ignore that receiver type) or deal with stack page switch (See StackInterpreter>>baseFrameReturn to get inspired on how to do that, the method performs a return across stack pages). That would be way faster and you would certainly be able to scan many more activations than 4 at each at:put while still being much faster that what you are doing.

If you have other questions and you don't get the answers you are looking for in the mailing list after a few days just CC me and/or Eliot.

Best,


On Tue, Apr 30, 2019 at 2:53 PM Ana Laura Felisatti <[hidden email]> wrote:
 
Hi!

Just to clarify, we've actually modified the StackInterpreter>>commonAtPut: with a call to our code. We are building the Stack Spur VM on Mac 64bits.

To validate whether our code was executed or not we've added printf entries to the C code produced in gcc3x-interp.c which is how we noticed the difference between regular execution and debug with proceed or over.

Here's the modified extract of commonAtPut:

!StackInterpreter methodsFor: 'indexing primitive support' stamp: 'ALF & MDS 4/7/2019 18:57'!
commonAtPut: stringy
    "This code is called if the receiver responds primitively to at:Put:.
     N.B. this does *not* use the at cache, instead inlining stObject:at:put:.
     Using the at cache here would require that callers set messageSelector
     and lkupClass and that is onerous and error-prone, and in any case,
     inlining produces much better performance than using the at cache here."
    | value index rcvr |
    <inline: true> "to get it inlined in primitiveAtPut and primitiveStringAtPut"
    self initPrimCall.
    self keepCollectionTypeInformation.
    rcvr := self stackValue: 2.
    index := self stackValue: 1.
    value := self stackTop.
    ...

And here's an extract of our code highlighting how we attempt to get the current MethodContext.

!StackInterpreter methodsFor: 'dynamic type information' stamp: 'ALF MTQP 4/21/2019 15:46'!
keepCollectionTypeInformation
    <inline: true>
    | senderContext receiverObject isLiveTypingCollectionObject contentTypesArray isSenderContextMethodContext collectionsContentType keepSearchingCollectionSender loopbackIndex maxLoopbackIndex|

    loopbackIndex := 0.
    maxLoopbackIndex := 4.
    keepSearchingCollectionSender := true.
    senderContext := self ensureFrameIsMarried: framePointer SP: stackPointer.
    [keepSearchingCollectionSender and: [loopbackIndex < maxLoopbackIndex]] whileTrue: [
        senderContext = objectMemory nilObject ifFalse: [
            isSenderContextMethodContext := (objectMemory is: senderContext instanceOf: (objectMemory splObj: ClassMethodContext) compactClassIndex: ClassMethodContextCompactIndex)-0.
            isSenderContextMethodContext ifTrue: [
                receiverObject :=  objectMemory fetchPointer: ReceiverIndex ofObject: senderContext.
                receiverObject = objectMemory nilObject ifFalse: [
                    isLiveTypingCollectionObject := (objectMemory is: receiverObject instanceOf: (objectMemory splObj: ClassLiveTypingCollection) compactClassIndex: 0)-0.
                    isLiveTypingCollectionObject ifTrue: [ ...

We are using a custom implementation of OrderedCollection, LiveTypingCollection just to limit the code exposure and have made it a known class to the VM for that reason.

Is there anything we are missing around the at:put: primitive execution and the commonAtPut: implementation it triggers? Is there another way to obtain the current MethodContext?

Thanks

On Mon, Apr 29, 2019 at 9:00 AM <[hidden email]> wrote:
Send Vm-dev mailing list submissions to
        [hidden email]

To subscribe or unsubscribe via the World Wide Web, visit
        http://lists.squeakfoundation.org/mailman/listinfo/vm-dev
or, via email, send a message with subject or body 'help' to
        [hidden email]

You can reach the person managing the list at
        [hidden email]

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Vm-dev digest..."


Today's Topics:

   1. Intercepting at:put: problems (Ana Laura Felisatti)
   2. Re: Intercepting at:put: problems (tim Rowledge)


----------------------------------------------------------------------

Message: 1
Date: Sun, 28 Apr 2019 15:04:13 -0300
From: Ana Laura Felisatti <[hidden email]>
To: [hidden email]
Subject: [Vm-dev] Intercepting at:put: problems
Message-ID:
        <CACxgkY=[hidden email]>
Content-Type: text/plain; charset="utf-8"

Hi!

We're working with Hernan Wilkinson on the Live Typing VM implementation
and have recently found a very strange behaviour. We are trying to capture
the type of objects stored within an OrderedCollection by intercepting the
at:put: method of Object, since that's what ends up being executed
internally by the OrderedCollection using an internal Array. To identify
whether the at:put: is being executed from that OrderCollection context we
inspect the MethodContext nesting.

So far we have intercepted the StackInterpreter commonAtPut: to execute our
logic, which attempts to retrieve the current MethodContext, analyse up to
4 levels up looking for an OrderedCollection receiver and in that case
store the type information.

The problem is that our code never executes on a regular execution but does
while debugging. In fact, it works as expected when we simply hit "Proceed"
but fails when we hit "Over" instead, which is quite unexpected.

We would like to know if there's anything we might be missing here about
the execution of primitives like at:put: or acquiring the current
MethodContext from the VM side.

Thanks!
--
A. Felisatti
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20190428/47ad902b/attachment-0001.html>

------------------------------

Message: 2
Date: Sun, 28 Apr 2019 11:20:19 -0700
From: tim Rowledge <[hidden email]>
To: Squeak Virtual Machine Development Discussion
        <[hidden email]>
Subject: Re: [Vm-dev] Intercepting at:put: problems
Message-ID: <[hidden email]>
Content-Type: text/plain;       charset=us-ascii



> On 2019-04-28, at 11:04 AM, Ana Laura Felisatti <[hidden email]> wrote:
>
> So far we have intercepted the StackInterpreter commonAtPut: to execute our logic, which attempts to retrieve the current MethodContext, analyse up to 4 levels up looking for an OrderedCollection receiver and in that case store the type information.
>
> The problem is that our code never executes on a regular execution but does while debugging. In fact, it works as expected when we simply hit "Proceed" but fails when we hit "Over" instead, which is quite unexpected.
>
> We would like to know if there's anything we might be missing here about the execution of primitives like at:put: or acquiring the current MethodContext from the VM side.

It's possible that the commonAtPut() routine has been inlined (looks that way in an ancient copy of interp.c I have lying around) and so if you are trapping in a debugger on the actual routine start then it would be unlikely to hit. If you've modified the commonAtPut() code, then this is unlikely to be a good explanation.

How exactly have you generated the VM? That might tell us something useful.


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: IO: Illogical Or




------------------------------

Subject: Digest Footer

_______________________________________________
Vm-dev mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/vm-dev


------------------------------

End of Vm-dev Digest, Vol 154, Issue 33
***************************************


--
A. Felisatti


--