Smalltalk › Squeak › Squeak VM

normalSend, specialObjectsArray and VM

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

18 messages Options

Mariano Martinez Peck

normalSend, specialObjectsArray and VM

Hi folks. I need to intercept ALL message sends. For the moment, I intercepted in Interpreter >> normalSend

But now, I have a question: I know there is that specialObjectArray that contain objects that may be accessed by the VM.

So, the first question is, they are only ACCESSED and to things with pointers or they also send messages to those objects from the VM?

If the VM really send messages to those object, how that is done ? The code goes also by normalSend ? Or they go from somewhere else that I am not intercepting.

Thank you very much.

Mariano

Eliot Miranda-2

Re: normalSend, specialObjectsArray and VM

On Fri, May 7, 2010 at 2:30 AM, Mariano Martinez Peck <[hidden email]> wrote:

Hi folks. I need to intercept ALL message sends. For the moment, I intercepted in Interpreter >> normalSend

But now, I have a question: I know there is that specialObjectArray that contain objects that may be accessed by the VM.

So, the first question is, they are only ACCESSED and to things with pointers or they also send messages to those objects from the VM?

Only accessed. The Interpreter has specialObjectsArray as one of its object references (along with nil, true, false) and indexes it with indices stored in class variables such as SpecialSelectors, CharacterTable, ClassMessage et al. See implementors and senders of splObj:.

If the VM really send messages to those object, how that is done ? The code goes also by normalSend ? Or they go from somewhere else that I am not intercepting.

At the bottom the VM has to access objects directly to avoid infinite regress. So the only sends are in response to send bytecodes, the perform: primitives, and other edge cases (looking up run:with:in: in the invoke-object-as-method prim and looking up a callback entry point in the Alien FFI).

Thank you very much.

Mariano

chers

Eliot

Igor Stasenko

Re: normalSend, specialObjectsArray and VM

On 7 May 2010 23:04, Eliot Miranda <[hidden email]> wrote:

>
>
>
> On Fri, May 7, 2010 at 2:30 AM, Mariano Martinez Peck <[hidden email]> wrote:
>>
>>
>> Hi folks. I need to intercept ALL message sends. For the moment, I intercepted in Interpreter >> normalSend
>>
>> But now, I have a question: I know there is that specialObjectArray that contain objects that may be accessed by the VM.
>>
>> So, the first question is, they are only ACCESSED and to things with pointers or they also send messages to those objects from the VM?
>
> Only accessed. The Interpreter has specialObjectsArray as one of its object references (along with nil, true, false) and indexes it with indices stored in class variables such as SpecialSelectors, CharacterTable, ClassMessage et al. See implementors and senders of splObj:.
>>
>> If the VM really send messages to those object, how that is done ? The code goes also by normalSend ? Or they go from somewhere else that I am not intercepting.
>
> At the bottom the VM has to access objects directly to avoid infinite regress. So the only sends are in response to send bytecodes, the perform: primitives, and other edge cases (looking up run:with:in: in the invoke-object-as-method prim and looking up a callback entry point in the Alien FFI).

there's also a limited set of selectors, for which VM does not a
normal send, but skips the standard lookup procedure, like with #class
message.

>>
>> Thank you very much.
>>
>> Mariano
>
> chers
> Eliot
>
>

--
Best regards,
Igor Stasenko AKA sig.

Mariano Martinez Peck

Re: normalSend, specialObjectsArray and VM

On Fri, May 7, 2010 at 11:10 PM, Igor Stasenko <[hidden email]> wrote:

On 7 May 2010 23:04, Eliot Miranda <[hidden email]> wrote:

>
>
>
> On Fri, May 7, 2010 at 2:30 AM, Mariano Martinez Peck <[hidden email]> wrote:
>>
>>
>> Hi folks. I need to intercept ALL message sends. For the moment, I intercepted in Interpreter >> normalSend
>>
>> But now, I have a question: I know there is that specialObjectArray that contain objects that may be accessed by the VM.
>>
>> So, the first question is, they are only ACCESSED and to things with pointers or they also send messages to those objects from the VM?
>

> Only accessed. The Interpreter has specialObjectsArray as one of its object references (along with nil, true, false) and indexes it with indices stored in class variables such as SpecialSelectors, CharacterTable, ClassMessage et al. See implementors and senders of splObj:.

>>
>> If the VM really send messages to those object, how that is done ? The code goes also by normalSend ? Or they go from somewhere else that I am not intercepting.
>

> At the bottom the VM has to access objects directly to avoid infinite regress. So the only sends are in response to send bytecodes, the perform: primitives, and other edge cases (looking up run:with:in: in the invoke-object-as-method prim and looking up a callback entry point in the Alien FFI).

there's also a limited set of selectors, for which VM does not a
normal send, but skips the standard lookup procedure, like with #class
message.

Thanks Igor. I could see #class does not the normal way. It was logic as it already has the pointer there ;)

Now I wonder...to avoid those special cases, do you think it makes sense to intercept in commonSend rather than commonSend ? or it would be the same ?

Thanks!

Mariano

>>
>> Thank you very much.
>>
>> Mariano
>

> chers
> Eliot
>
>

--
Best regards,
Igor Stasenko AKA sig.

Mariano Martinez Peck

Re: normalSend, specialObjectsArray and VM

In reply to this post by Eliot Miranda-2

On Fri, May 7, 2010 at 10:04 PM, Eliot Miranda <[hidden email]> wrote:

On Fri, May 7, 2010 at 2:30 AM, Mariano Martinez Peck <[hidden email]> wrote:

Hi folks. I need to intercept ALL message sends. For the moment, I intercepted in Interpreter >> normalSend

But now, I have a question: I know there is that specialObjectArray that contain objects that may be accessed by the VM.

So, the first question is, they are only ACCESSED and to things with pointers or they also send messages to those objects from the VM?

Only accessed. The Interpreter has specialObjectsArray as one of its object references (along with nil, true, false) and indexes it with indices stored in class variables such as SpecialSelectors, CharacterTable, ClassMessage et al. See implementors and senders of splObj:.

Ok, perfect. Thanks for the hints.

If the VM really send messages to those object, how that is done ? The code goes also by normalSend ? Or they go from somewhere else that I am not intercepting.

At the bottom the VM has to access objects directly to avoid infinite regress. So the only sends are in response to send bytecodes, the perform: primitives, and other edge cases (looking up run:with:in: in the invoke-object-as-method prim and looking up a callback entry point in the Alien FFI).

I didn't understand very much this last sentence. I once experimented implementing run:with:in: and changed that in the MethodDictionary...so I tried to see what you said but I didn't understand. I saw it is index 50 of the array and it is done this:

SelectorAttemptToAssign := 50.

But I couldn't see anything else.

Thanks!

Mariano

Thank you very much.

Mariano

chers
Eliot

Igor Stasenko

Re: normalSend, specialObjectsArray and VM

In reply to this post by Mariano Martinez Peck

On 9 May 2010 01:08, Mariano Martinez Peck <[hidden email]> wrote:

>
>
>
> On Fri, May 7, 2010 at 11:10 PM, Igor Stasenko <[hidden email]> wrote:
>>
>> On 7 May 2010 23:04, Eliot Miranda <[hidden email]> wrote:
>> >
>> >
>> >
>> > On Fri, May 7, 2010 at 2:30 AM, Mariano Martinez Peck <[hidden email]> wrote:
>> >>
>> >>
>> >> Hi folks. I need to intercept ALL message sends. For the moment, I intercepted in Interpreter >> normalSend
>> >>
>> >> But now, I have a question: I know there is that specialObjectArray that contain objects that may be accessed by the VM.
>> >>
>> >> So, the first question is, they are only ACCESSED and to things with pointers or they also send messages to those objects from the VM?
>> >
>> > Only accessed. The Interpreter has specialObjectsArray as one of its object references (along with nil, true, false) and indexes it with indices stored in class variables such as SpecialSelectors, CharacterTable, ClassMessage et al. See implementors and senders of splObj:.
>> >>
>> >> If the VM really send messages to those object, how that is done ? The code goes also by normalSend ? Or they go from somewhere else that I am not intercepting.
>> >
>> > At the bottom the VM has to access objects directly to avoid infinite regress. So the only sends are in response to send bytecodes, the perform: primitives, and other edge cases (looking up run:with:in: in the invoke-object-as-method prim and looking up a callback entry point in the Alien FFI).
>>
>> there's also a limited set of selectors, for which VM does not a
>> normal send, but skips the standard lookup procedure, like with #class
>> message.
>
>
> Thanks Igor. I could see #class does not the normal way. It was logic as it already has the pointer there ;)
>
> Now I wonder...to avoid those special cases, do you think it makes sense to intercept in commonSend rather than commonSend ? or it would be the same ?
>

err... commonSend or commonSend? i think it would be the same :)

the other point, where you can try intercept a send is cache lookup.

> Thanks!
>
> Mariano
>
>
>
>>
>> >>
>> >> Thank you very much.
>> >>
>> >> Mariano
>> >
>> > chers
>> > Eliot
>> >
>> >
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>
>
>

--
Best regards,
Igor Stasenko AKA sig.

Mariano Martinez Peck

Re: normalSend, specialObjectsArray and VM

>
> Thanks Igor. I could see #class does not the normal way. It was logic as it already has the pointer there ;)
>
> Now I wonder...to avoid those special cases, do you think it makes sense to intercept in commonSend rather than commonSend ? or it would be the same ?
>

err... commonSend or commonSend? i think it would be the same :)

hahahah sorry, I meant commonSend instead of normalSend.

the other point, where you can try intercept a send is cache lookup.

internalFindNewMethod ?

Thanks

Mariano

> Thanks!
>
> Mariano
>
>
>
>>
>> >>
>> >> Thank you very much.
>> >>
>> >> Mariano
>> >
>> > chers
>> > Eliot
>> >
>> >
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>
>
>

--

Best regards,
Igor Stasenko AKA sig.

Igor Stasenko

Re: normalSend, specialObjectsArray and VM

On 11 May 2010 17:40, Mariano Martinez Peck <[hidden email]> wrote:

>
>
>> >
>> > Thanks Igor. I could see #class does not the normal way. It was logic as it already has the pointer there ;)
>> >
>> > Now I wonder...to avoid those special cases, do you think it makes sense to intercept in commonSend rather than commonSend ? or it would be the same ?
>> >
>> err... commonSend or commonSend? i think it would be the same :)
>>
>
> hahahah sorry, I meant commonSend instead of normalSend.
>
>
>>
>> the other point, where you can try intercept a send is cache lookup.
>>
>
> internalFindNewMethod ?
>

I don't know, maybe :)

> Thanks
>
> Mariano
>
>
>>
>> > Thanks!
>> >
>> > Mariano
>> >
>> >
>> >
>> >>
>> >> >>
>> >> >> Thank you very much.
>> >> >>
>> >> >> Mariano
>> >> >
>> >> > chers
>> >> > Eliot
>> >> >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Best regards,
>> >> Igor Stasenko AKA sig.
>> >
>> >
>> >
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>
>
>

--
Best regards,
Igor Stasenko AKA sig.

Mariano Martinez Peck

Re: normalSend, specialObjectsArray and VM

Hi. I have a related question once again with this topic. I've changed Interpreter >> normalSend to something like this:

normalSend
    "Send a message, starting lookup with the receiver's class."
    "Assume: messageSelector and argumentCount have been set, and that
    the receiver and arguments have been pushed onto the stack,"
    "Note: This method is inlined into the interpreter dispatch loop."
    | rcvr |
    self inline: true.
    self sharedCodeNamed: 'normalSend' inCase: 131.
    rcvr := self internalStackValue: argumentCount.
    ((self isIntegerObject: rcvr) not and: [hasToTrace])
        ifTrue: [
            self internalTurnOnUsedBit: rcvr.
            ].
    lkupClass := self fetchClassOf: rcvr.
    receiverClass := lkupClass.
    self commonSend.

So...if it is not a SmallInetger and if the flag is on, I turn on a bit.

The question is, if I send a normal message to a normal object. Example:

| anObject |
anObject := MyClass new.
anObject foo

Now...I am sure that "anObject" was marked with the bit. But what about:
a) the compiled method MyClass >> #foo
b) MyClass

should they be marked?

In other words:

self deny: (unUsed primitiveGetUsedBit: anObject).
    self deny: (unUsed primitiveGetUsedBit: anObject class).
    self deny: (unUsed primitiveGetUsedBit: (anObject class >> #foo)).

    anObject foo.

    self assert: (unUsed primitiveGetUsedBit: anObject).
    self assert: (unUsed primitiveGetUsedBit: anObject class).
    self assert: (unUsed primitiveGetUsedBit: (anObject class >> #foo)).


should all the asserts pass? I ask because I don't know how CompiledMethods are executed (they receive a normalSend like any other object?) nor how class are accessed.

Thanks in advance,

Mariano

On Tue, May 11, 2010 at 5:43 PM, Igor Stasenko <[hidden email]> wrote:

On 11 May 2010 17:40, Mariano Martinez Peck <[hidden email]> wrote:
>
>
>> >
>> > Thanks Igor. I could see #class does not the normal way. It was logic as it already has the pointer there ;)
>> >
>> > Now I wonder...to avoid those special cases, do you think it makes sense to intercept in commonSend rather than commonSend ? or it would be the same ?
>> >
>> err... commonSend or commonSend? i think it would be the same :)
>>
>
> hahahah sorry, I meant commonSend instead of normalSend.
>
>
>>
>> the other point, where you can try intercept a send is cache lookup.
>>
>
> internalFindNewMethod ?
>

I don't know, maybe :)

> Thanks
>
> Mariano
>
>
>>
>> > Thanks!
>> >
>> > Mariano
>> >
>> >
>> >
>> >>
>> >> >>
>> >> >> Thank you very much.
>> >> >>
>> >> >> Mariano
>> >> >
>> >> > chers
>> >> > Eliot
>> >> >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Best regards,
>> >> Igor Stasenko AKA sig.
>> >
>> >
>> >
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>
>
>

--

Best regards,
Igor Stasenko AKA sig.

Bert Freudenberg

Re: normalSend, specialObjectsArray and VM

What exactly are you trying to do? When do you consider an object to be "used"?

- Bert -

On 03.10.2010, at 12:12, Mariano Martinez Peck wrote:

Hi. I have a related question once again with this topic. I've changed Interpreter >> normalSend to something like this:

normalSend
    "Send a message, starting lookup with the receiver's class."
    "Assume: messageSelector and argumentCount have been set, and that
    the receiver and arguments have been pushed onto the stack,"
    "Note: This method is inlined into the interpreter dispatch loop."
    | rcvr |
    self inline: true.
    self sharedCodeNamed: 'normalSend' inCase: 131.
    rcvr := self internalStackValue: argumentCount.
    ((self isIntegerObject: rcvr) not and: [hasToTrace])
        ifTrue: [
            self internalTurnOnUsedBit: rcvr.
            ].
    lkupClass := self fetchClassOf: rcvr.
    receiverClass := lkupClass.
    self commonSend.

So...if it is not a SmallInetger and if the flag is on, I turn on a bit.

The question is, if I send a normal message to a normal object. Example:

| anObject |
anObject := MyClass new.
anObject foo

Now...I am sure that "anObject" was marked with the bit. But what about:
a) the compiled method MyClass >> #foo
b) MyClass

should they be marked?

In other words:

self deny: (unUsed primitiveGetUsedBit: anObject).
    self deny: (unUsed primitiveGetUsedBit: anObject class).
    self deny: (unUsed primitiveGetUsedBit: (anObject class >> #foo)).

    anObject foo.

    self assert: (unUsed primitiveGetUsedBit: anObject).
    self assert: (unUsed primitiveGetUsedBit: anObject class).
    self assert: (unUsed primitiveGetUsedBit: (anObject class >> #foo)).


should all the asserts pass? I ask because I don't know how CompiledMethods are executed (they receive a normalSend like any other object?) nor how class are accessed.

Thanks in advance,

Mariano

On Tue, May 11, 2010 at 5:43 PM, Igor Stasenko <[hidden email]> wrote:

On 11 May 2010 17:40, Mariano Martinez Peck <[hidden email]> wrote:
>
>
>> >
>> > Thanks Igor. I could see #class does not the normal way. It was logic as it already has the pointer there ;)
>> >
>> > Now I wonder...to avoid those special cases, do you think it makes sense to intercept in commonSend rather than commonSend ? or it would be the same ?
>> >
>> err... commonSend or commonSend? i think it would be the same :)
>>
>
> hahahah sorry, I meant commonSend instead of normalSend.
>
>
>>
>> the other point, where you can try intercept a send is cache lookup.
>>
>
> internalFindNewMethod ?
>

I don't know, maybe :)

> Thanks
>
> Mariano
>

Mariano Martinez Peck

Re: normalSend, specialObjectsArray and VM

On Sun, Oct 3, 2010 at 12:42 PM, Bert Freudenberg <[hidden email]> wrote:

What exactly are you trying to do?

hehehe sorry. I am trying to "detect unused objects".

When do you consider an object to be "used"?

When it receives a message. This is why I changed #normalSend

Thanks in advance,

Mariano

- Bert -

On 03.10.2010, at 12:12, Mariano Martinez Peck wrote:

Hi. I have a related question once again with this topic. I've changed Interpreter >> normalSend to something like this:

normalSend
    "Send a message, starting lookup with the receiver's class."
    "Assume: messageSelector and argumentCount have been set, and that
    the receiver and arguments have been pushed onto the stack,"
    "Note: This method is inlined into the interpreter dispatch loop."
    | rcvr |
    self inline: true.
    self sharedCodeNamed: 'normalSend' inCase: 131.
    rcvr := self internalStackValue: argumentCount.
    ((self isIntegerObject: rcvr) not and: [hasToTrace])
        ifTrue: [
            self internalTurnOnUsedBit: rcvr.
            ].
    lkupClass := self fetchClassOf: rcvr.
    receiverClass := lkupClass.
    self commonSend.

So...if it is not a SmallInetger and if the flag is on, I turn on a bit.

The question is, if I send a normal message to a normal object. Example:

| anObject |
anObject := MyClass new.
anObject foo

Now...I am sure that "anObject" was marked with the bit. But what about:
a) the compiled method MyClass >> #foo
b) MyClass

should they be marked?

In other words:

self deny: (unUsed primitiveGetUsedBit: anObject).
    self deny: (unUsed primitiveGetUsedBit: anObject class).
    self deny: (unUsed primitiveGetUsedBit: (anObject class >> #foo)).

    anObject foo.

    self assert: (unUsed primitiveGetUsedBit: anObject).
    self assert: (unUsed primitiveGetUsedBit: anObject class).
    self assert: (unUsed primitiveGetUsedBit: (anObject class >> #foo)).


should all the asserts pass? I ask because I don't know how CompiledMethods are executed (they receive a normalSend like any other object?) nor how class are accessed.

Thanks in advance,

Mariano

On Tue, May 11, 2010 at 5:43 PM, Igor Stasenko <[hidden email]> wrote:

On 11 May 2010 17:40, Mariano Martinez Peck <[hidden email]> wrote:
>
>
>> >
>> > Thanks Igor. I could see #class does not the normal way. It was logic as it already has the pointer there ;)
>> >
>> > Now I wonder...to avoid those special cases, do you think it makes sense to intercept in commonSend rather than commonSend ? or it would be the same ?
>> >
>> err... commonSend or commonSend? i think it would be the same :)
>>
>
> hahahah sorry, I meant commonSend instead of normalSend.
>
>
>>
>> the other point, where you can try intercept a send is cache lookup.
>>
>
> internalFindNewMethod ?
>

I don't know, maybe :)

> Thanks
>
> Mariano
>

Igor Stasenko

Re: normalSend, specialObjectsArray and VM

Since during message send, you discovered particular method,
it should be also marked as 'used object', and all its literals etc.

On 3 October 2010 13:45, Mariano Martinez Peck <[hidden email]> wrote:

>
>
>
> On Sun, Oct 3, 2010 at 12:42 PM, Bert Freudenberg <[hidden email]> wrote:
>>
>>
>> What exactly are you trying to do?
>
> hehehe sorry. I am trying to "detect unused objects".
>
>>
>> When do you consider an object to be "used"?
>
> When it receives a message. This is why I changed #normalSend
>
> Thanks in advance,
>
> Mariano
>
>
>>
>> - Bert -
>> On 03.10.2010, at 12:12, Mariano Martinez Peck wrote:
>>
>> Hi. I have a related question once again with this topic. I've changed Interpreter >> normalSend to something like this:
>>
>>
>> normalSend
>>     "Send a message, starting lookup with the receiver's class."
>>     "Assume: messageSelector and argumentCount have been set, and that
>>     the receiver and arguments have been pushed onto the stack,"
>>     "Note: This method is inlined into the interpreter dispatch loop."
>>     | rcvr |
>>     self inline: true.
>>     self sharedCodeNamed: 'normalSend' inCase: 131.
>>     rcvr := self internalStackValue: argumentCount.
>>     ((self isIntegerObject: rcvr) not and: [hasToTrace])
>>         ifTrue: [
>>             self internalTurnOnUsedBit: rcvr.
>>             ].
>>     lkupClass := self fetchClassOf: rcvr.
>>     receiverClass := lkupClass.
>>     self commonSend.
>>
>>
>> So...if it is not a SmallInetger and if the flag is on, I turn on a bit.
>>
>> The question is, if I send a normal message to a normal object. Example:
>>
>> | anObject |
>> anObject := MyClass new.
>> anObject foo
>>
>> Now...I am sure that "anObject" was marked with the bit. But what about:
>> a) the compiled method MyClass >> #foo
>> b) MyClass
>>
>> should they be marked?
>>
>> In other words:
>>
>> self deny: (unUsed primitiveGetUsedBit: anObject).
>>     self deny: (unUsed primitiveGetUsedBit: anObject class).
>>     self deny: (unUsed primitiveGetUsedBit: (anObject class >> #foo)).
>>
>>     anObject foo.
>>
>>     self assert: (unUsed primitiveGetUsedBit: anObject).
>>     self assert: (unUsed primitiveGetUsedBit: anObject class).
>>     self assert: (unUsed primitiveGetUsedBit: (anObject class >> #foo)).
>>
>>
>> should all the asserts pass? I ask because I don't know how CompiledMethods are executed (they receive a normalSend like any other object?) nor how class are accessed.
>>
>> Thanks in advance,
>>
>> Mariano
>>
>>
>> On Tue, May 11, 2010 at 5:43 PM, Igor Stasenko <[hidden email]> wrote:
>>>
>>> On 11 May 2010 17:40, Mariano Martinez Peck <[hidden email]> wrote:
>>> >
>>> >
>>> >> >
>>> >> > Thanks Igor. I could see #class does not the normal way. It was logic as it already has the pointer there ;)
>>> >> >
>>> >> > Now I wonder...to avoid those special cases, do you think it makes sense to intercept in commonSend rather than commonSend ? or it would be the same ?
>>> >> >
>>> >> err... commonSend or commonSend? i think it would be the same :)
>>> >>
>>> >
>>> > hahahah sorry, I meant commonSend instead of normalSend.
>>> >
>>> >
>>> >>
>>> >> the other point, where you can try intercept a send is cache lookup.
>>> >>
>>> >
>>> > internalFindNewMethod ?
>>> >
>>>
>>> I don't know, maybe :)
>>>
>>> > Thanks
>>> >
>>> > Mariano
>>> >
>>
>>
>
>
>

--
Best regards,
Igor Stasenko AKA sig.

Bert Freudenberg

Re: normalSend, specialObjectsArray and VM

In reply to this post by Mariano Martinez Peck

On 03.10.2010, at 12:45, Mariano Martinez Peck wrote:

On Sun, Oct 3, 2010 at 12:42 PM, Bert Freudenberg <[hidden email]> wrote:

What exactly are you trying to do?

hehehe sorry. I am trying to "detect unused objects".

What for?

When do you consider an object to be "used"?

When it receives a message. This is why I changed #normalSend

Thanks in advance,

Mariano

Neither the class nor the compiled method "receive a message". Plus there are many ways an object "appears" to receive a message but isn't really. E.g. a Float won't receive #+ because that is short-circuited in the bytecodes. A super send does not use "normalSend". Etc.

Judging from your test cases, if an objects is just passed as an argument, you do not consider it to be used. What if you send it the #class message? What if you identity-compare it (==)? Should that count as "used"?

Maybe if you let us in on the bigger picture we could help you better.

- Bert -

On 03.10.2010, at 12:12, Mariano Martinez Peck wrote:

Hi. I have a related question once again with this topic. I've changed Interpreter >> normalSend to something like this:

normalSend
    "Send a message, starting lookup with the receiver's class."
    "Assume: messageSelector and argumentCount have been set, and that
    the receiver and arguments have been pushed onto the stack,"
    "Note: This method is inlined into the interpreter dispatch loop."
    | rcvr |
    self inline: true.
    self sharedCodeNamed: 'normalSend' inCase: 131.
    rcvr := self internalStackValue: argumentCount.
    ((self isIntegerObject: rcvr) not and: [hasToTrace])
        ifTrue: [
            self internalTurnOnUsedBit: rcvr.
            ].
    lkupClass := self fetchClassOf: rcvr.
    receiverClass := lkupClass.
    self commonSend.

So...if it is not a SmallInetger and if the flag is on, I turn on a bit.

The question is, if I send a normal message to a normal object. Example:

| anObject |
anObject := MyClass new.
anObject foo

Now...I am sure that "anObject" was marked with the bit. But what about:
a) the compiled method MyClass >> #foo
b) MyClass

should they be marked?

In other words:

self deny: (unUsed primitiveGetUsedBit: anObject).
    self deny: (unUsed primitiveGetUsedBit: anObject class).
    self deny: (unUsed primitiveGetUsedBit: (anObject class >> #foo)).

    anObject foo.

    self assert: (unUsed primitiveGetUsedBit: anObject).
    self assert: (unUsed primitiveGetUsedBit: anObject class).
    self assert: (unUsed primitiveGetUsedBit: (anObject class >> #foo)).


should all the asserts pass? I ask because I don't know how CompiledMethods are executed (they receive a normalSend like any other object?) nor how class are accessed.

Thanks in advance,

Mariano

On Tue, May 11, 2010 at 5:43 PM, Igor Stasenko <[hidden email]> wrote:

On 11 May 2010 17:40, Mariano Martinez Peck <[hidden email]> wrote:
>
>
>> >
>> > Thanks Igor. I could see #class does not the normal way. It was logic as it already has the pointer there ;)
>> >
>> > Now I wonder...to avoid those special cases, do you think it makes sense to intercept in commonSend rather than commonSend ? or it would be the same ?
>> >
>> err... commonSend or commonSend? i think it would be the same :)
>>
>
> hahahah sorry, I meant commonSend instead of normalSend.
>
>
>>
>> the other point, where you can try intercept a send is cache lookup.
>>
>
> internalFindNewMethod ?
>

I don't know, maybe :)

> Thanks
>
> Mariano
>

stephane ducasse-2

Re: normalSend, specialObjectsArray and VM

always the same:
we want a kind of new LOOM (Melt in Java) where unused objects are move to disc.

Stef

On Oct 3, 2010, at 1:03 PM, Bert Freudenberg wrote:

> On 03.10.2010, at 12:45, Mariano Martinez Peck wrote:
>
>> On Sun, Oct 3, 2010 at 12:42 PM, Bert Freudenberg <[hidden email]> wrote:
>>
>> What exactly are you trying to do?
>>
>> hehehe sorry. I am trying to "detect unused objects".
>
> What for?
>
>> When do you consider an object to be "used"?
>>
>>
>> When it receives a message. This is why I changed #normalSend
>>
>> Thanks in advance,
>>
>> Mariano
>
> Neither the class nor the compiled method "receive a message". Plus there are many ways an object "appears" to receive a message but isn't really. E.g. a Float won't receive #+ because that is short-circuited in the bytecodes. A super send does not use "normalSend". Etc.
>
> Judging from your test cases, if an objects is just passed as an argument, you do not consider it to be used. What if you send it the #class message? What if you identity-compare it (==)? Should that count as "used"?
>
> Maybe if you let us in on the bigger picture we could help you better.
>
> - Bert -
>
>
>>
>> On 03.10.2010, at 12:12, Mariano Martinez Peck wrote:
>>
>>> Hi. I have a related question once again with this topic. I've changed Interpreter >> normalSend to something like this:
>>>
>>>
>>> normalSend
>>> "Send a message, starting lookup with the receiver's class."
>>> "Assume: messageSelector and argumentCount have been set, and that
>>> the receiver and arguments have been pushed onto the stack,"
>>> "Note: This method is inlined into the interpreter dispatch loop."
>>> | rcvr |
>>> self inline: true.
>>> self sharedCodeNamed: 'normalSend' inCase: 131.
>>> rcvr := self internalStackValue: argumentCount.
>>> ((self isIntegerObject: rcvr) not and: [hasToTrace])
>>> ifTrue: [
>>> self internalTurnOnUsedBit: rcvr.
>>> ].
>>> lkupClass := self fetchClassOf: rcvr.
>>> receiverClass := lkupClass.
>>> self commonSend.
>>>
>>>
>>> So...if it is not a SmallInetger and if the flag is on, I turn on a bit.
>>>
>>> The question is, if I send a normal message to a normal object. Example:
>>>
>>> | anObject |
>>> anObject := MyClass new.
>>> anObject foo
>>>
>>> Now...I am sure that "anObject" was marked with the bit. But what about:
>>> a) the compiled method MyClass >> #foo
>>> b) MyClass
>>>
>>> should they be marked?
>>>
>>> In other words:
>>>
>>> self deny: (unUsed primitiveGetUsedBit: anObject).
>>> self deny: (unUsed primitiveGetUsedBit: anObject class).
>>> self deny: (unUsed primitiveGetUsedBit: (anObject class >> #foo)).
>>>
>>> anObject foo.
>>>
>>> self assert: (unUsed primitiveGetUsedBit: anObject).
>>> self assert: (unUsed primitiveGetUsedBit: anObject class).
>>> self assert: (unUsed primitiveGetUsedBit: (anObject class >> #foo)).
>>>
>>>
>>> should all the asserts pass? I ask because I don't know how CompiledMethods are executed (they receive a normalSend like any other object?) nor how class are accessed.
>>>
>>> Thanks in advance,
>>>
>>> Mariano
>>>
>>>
>>> On Tue, May 11, 2010 at 5:43 PM, Igor Stasenko <[hidden email]> wrote:
>>>
>>> On 11 May 2010 17:40, Mariano Martinez Peck <[hidden email]> wrote:
>>> >
>>> >
>>> >> >
>>> >> > Thanks Igor. I could see #class does not the normal way. It was logic as it already has the pointer there ;)
>>> >> >
>>> >> > Now I wonder...to avoid those special cases, do you think it makes sense to intercept in commonSend rather than commonSend ? or it would be the same ?
>>> >> >
>>> >> err... commonSend or commonSend? i think it would be the same :)
>>> >>
>>> >
>>> > hahahah sorry, I meant commonSend instead of normalSend.
>>> >
>>> >
>>> >>
>>> >> the other point, where you can try intercept a send is cache lookup.
>>> >>
>>> >
>>> > internalFindNewMethod ?
>>> >
>>>
>>> I don't know, maybe :)
>>>
>>> > Thanks
>>> >
>>> > Mariano
>>> >
>>
>>
>>
>
>

Mariano Martinez Peck

Re: normalSend, specialObjectsArray and VM

In reply to this post by Bert Freudenberg

On Sun, Oct 3, 2010 at 1:03 PM, Bert Freudenberg <[hidden email]> wrote:

On 03.10.2010, at 12:45, Mariano Martinez Peck wrote:

On Sun, Oct 3, 2010 at 12:42 PM, Bert Freudenberg <[hidden email]> wrote:

What exactly are you trying to do?

hehehe sorry. I am trying to "detect unused objects".

What for?

Hi Bert. The idea is to be able to detect objects that are not used but referenced (this is why the GC don't collect them). We think there are a lots and lost of objects in this situation: they are referenced but not used for a long time, only agains certain situations, etc. The idea is to detect subgraphs of unused objects and swap them to disk, and let proxies in the image that know how to load back them when necessary.

If you are interested, you may want to see the slides of my ESUG talk: http://www.slideshare.net/esug/swapping-esug2010
(the video is not yet available)

When do you consider an object to be "used"?

When it receives a message. This is why I changed #normalSend

Thanks in advance,

Mariano

Neither the class nor the compiled method "receive a message". Plus there are many ways an object "appears" to receive a message but isn't really. E.g. a Float won't receive #+ because that is short-circuited in the bytecodes. A super send does not use "normalSend". Etc.

Yes, that's one of my problems. I am interested in knowing all these "anormal" situations. And yes, I think the definition "an object is usage when it receives a message" is not clear enough. Because yes, I would like to mark the CompiledMethod, Float when sending #+, and super calls.

Judging from your test cases, if an objects is just passed as an argument, you do not consider it to be used.

Good question. If you pass an object as parameter, you will probably send a message to it. In such case, then yes, it will be marked.
Now...what happens if you pass an object by argument and you don't send any message? I guess it should be marked.

Anyway the mark has not to be THAT strict. I mean, I would like to be like that. But all these step of detecting unused objects is because I will try to swap them. Thus, the less they are usually used, the best. In the worst case, I will swap an object and will be used inmediatly -> overhead.

What if you send it the #class message? What if you identity-compare it (==)? Should that count as "used"?

Good question. I would guess that yes. Is there a way to know all these "anormal" sends? I know #class is in bytecode, etc. But I don't know all of them.

Maybe if you let us in on the bigger picture we could help you better.

Thanks Bert. Please let me know if there is something missing.

Mariano

- Bert -

On 03.10.2010, at 12:12, Mariano Martinez Peck wrote:

Hi. I have a related question once again with this topic. I've changed Interpreter >> normalSend to something like this:

normalSend
    "Send a message, starting lookup with the receiver's class."
    "Assume: messageSelector and argumentCount have been set, and that
    the receiver and arguments have been pushed onto the stack,"
    "Note: This method is inlined into the interpreter dispatch loop."
    | rcvr |
    self inline: true.
    self sharedCodeNamed: 'normalSend' inCase: 131.
    rcvr := self internalStackValue: argumentCount.
    ((self isIntegerObject: rcvr) not and: [hasToTrace])
        ifTrue: [
            self internalTurnOnUsedBit: rcvr.
            ].
    lkupClass := self fetchClassOf: rcvr.
    receiverClass := lkupClass.
    self commonSend.

So...if it is not a SmallInetger and if the flag is on, I turn on a bit.

The question is, if I send a normal message to a normal object. Example:

| anObject |
anObject := MyClass new.
anObject foo

Now...I am sure that "anObject" was marked with the bit. But what about:
a) the compiled method MyClass >> #foo
b) MyClass

should they be marked?

In other words:

self deny: (unUsed primitiveGetUsedBit: anObject).
    self deny: (unUsed primitiveGetUsedBit: anObject class).
    self deny: (unUsed primitiveGetUsedBit: (anObject class >> #foo)).

    anObject foo.

    self assert: (unUsed primitiveGetUsedBit: anObject).
    self assert: (unUsed primitiveGetUsedBit: anObject class).
    self assert: (unUsed primitiveGetUsedBit: (anObject class >> #foo)).


should all the asserts pass? I ask because I don't know how CompiledMethods are executed (they receive a normalSend like any other object?) nor how class are accessed.

Thanks in advance,

Mariano

On Tue, May 11, 2010 at 5:43 PM, Igor Stasenko <[hidden email]> wrote:

On 11 May 2010 17:40, Mariano Martinez Peck <[hidden email]> wrote:
>
>
>> >
>> > Thanks Igor. I could see #class does not the normal way. It was logic as it already has the pointer there ;)
>> >
>> > Now I wonder...to avoid those special cases, do you think it makes sense to intercept in commonSend rather than commonSend ? or it would be the same ?
>> >
>> err... commonSend or commonSend? i think it would be the same :)
>>
>
> hahahah sorry, I meant commonSend instead of normalSend.
>
>
>>
>> the other point, where you can try intercept a send is cache lookup.
>>
>
> internalFindNewMethod ?
>

I don't know, maybe :)

> Thanks
>
> Mariano
>

Bert Freudenberg

Re: normalSend, specialObjectsArray and VM

On 03.10.2010, at 13:32, Mariano Martinez Peck wrote:

On Sun, Oct 3, 2010 at 1:03 PM, Bert Freudenberg <[hidden email]> wrote:

On 03.10.2010, at 12:45, Mariano Martinez Peck wrote:

On Sun, Oct 3, 2010 at 12:42 PM, Bert Freudenberg <[hidden email]> wrote:

What exactly are you trying to do?

hehehe sorry. I am trying to "detect unused objects".

What for?

Hi Bert. The idea is to be able to detect objects that are not used but referenced (this is why the GC don't collect them). We think there are a lots and lost of objects in this situation: they are referenced but not used for a long time, only agains certain situations, etc. The idea is to detect subgraphs of unused objects and swap them to disk, and let proxies in the image that know how to load back them when necessary.

If you are interested, you may want to see the slides of my ESUG talk: http://www.slideshare.net/esug/swapping-esug2010
(the video is not yet available)

When do you consider an object to be "used"?

When it receives a message. This is why I changed #normalSend

Thanks in advance,

Mariano

Neither the class nor the compiled method "receive a message". Plus there are many ways an object "appears" to receive a message but isn't really. E.g. a Float won't receive #+ because that is short-circuited in the bytecodes. A super send does not use "normalSend". Etc.

Yes, that's one of my problems. I am interested in knowing all these "anormal" situations. And yes, I think the definition "an object is usage when it receives a message" is not clear enough. Because yes, I would like to mark the CompiledMethod, Float when sending #+, and super calls.

Judging from your test cases, if an objects is just passed as an argument, you do not consider it to be used.

Good question. If you pass an object as parameter, you will probably send a message to it. In such case, then yes, it will be marked.
Now...what happens if you pass an object by argument and you don't send any message? I guess it should be marked.

Anyway the mark has not to be THAT strict. I mean, I would like to be like that. But all these step of detecting unused objects is because I will try to swap them. Thus, the less they are usually used, the best. In the worst case, I will swap an object and will be used inmediatly -> overhead.

What if you send it the #class message? What if you identity-compare it (==)? Should that count as "used"?

Good question. I would guess that yes. Is there a way to know all these "anormal" sends? I know #class is in bytecode, etc. But I don't know all of them.

Maybe if you let us in on the bigger picture we could help you better.

Thanks Bert. Please let me know if there is something missing.

Mariano

Well, that explanation helped.

IMHO, sends are too high-level to give accurate "usage" information. You might have to do it in the fetch/store functions (or even the memory access macros), which probably would have a serious performance impact.

You could still try to do it in the sends and primitives and byte codes, if you then propagate usage info to related objects. You're only interested in finding larger image subgraphs that are unused anyway, so this might be okay. You'll find out pretty promptly if you swapped out something that was still needed ;)

- Bert -

johnmci

Re: normalSend, specialObjectsArray and VM

In reply to this post by Mariano Martinez Peck

Craig Latta has done all this work, talk to him.

On 2010-10-03, at 4:32 AM, Mariano Martinez Peck wrote:

On Sun, Oct 3, 2010 at 1:03 PM, Bert Freudenberg <[hidden email]> wrote:

On 03.10.2010, at 12:45, Mariano Martinez Peck wrote:

On Sun, Oct 3, 2010 at 12:42 PM, Bert Freudenberg <[hidden email]> wrote:

What exactly are you trying to do?

hehehe sorry. I am trying to "detect unused objects".

What for?

Hi Bert. The idea is to be able to detect objects that are not used but referenced (this is why the GC don't collect them). We think there are a lots and lost of objects in this situation: they are referenced but not used for a long time, only agains certain situations, etc. The idea is to detect subgraphs of unused objects and swap them to disk, and let proxies in the image that know how to load back them when necessary.

If you are interested, you may want to see the slides of my ESUG talk: http://www.slideshare.net/esug/swapping-esug2010
(the video is not yet available)

When do you consider an object to be "used"?

When it receives a message. This is why I changed #normalSend

Thanks in advance,

Mariano

Neither the class nor the compiled method "receive a message". Plus there are many ways an object "appears" to receive a message but isn't really. E.g. a Float won't receive #+ because that is short-circuited in the bytecodes. A super send does not use "normalSend". Etc.

Yes, that's one of my problems. I am interested in knowing all these "anormal" situations. And yes, I think the definition "an object is usage when it receives a message" is not clear enough. Because yes, I would like to mark the CompiledMethod, Float when sending #+, and super calls.

Judging from your test cases, if an objects is just passed as an argument, you do not consider it to be used.

Good question. If you pass an object as parameter, you will probably send a message to it. In such case, then yes, it will be marked.
Now...what happens if you pass an object by argument and you don't send any message? I guess it should be marked.

Anyway the mark has not to be THAT strict. I mean, I would like to be like that. But all these step of detecting unused objects is because I will try to swap them. Thus, the less they are usually used, the best. In the worst case, I will swap an object and will be used inmediatly -> overhead.

What if you send it the #class message? What if you identity-compare it (==)? Should that count as "used"?

Good question. I would guess that yes. Is there a way to know all these "anormal" sends? I know #class is in bytecode, etc. But I don't know all of them.

Maybe if you let us in on the bigger picture we could help you better.

Thanks Bert. Please let me know if there is something missing.

Mariano

- Bert -

On 03.10.2010, at 12:12, Mariano Martinez Peck wrote:

Hi. I have a related question once again with this topic. I've changed Interpreter >> normalSend to something like this:

normalSend
    "Send a message, starting lookup with the receiver's class."
    "Assume: messageSelector and argumentCount have been set, and that
    the receiver and arguments have been pushed onto the stack,"
    "Note: This method is inlined into the interpreter dispatch loop."
    | rcvr |
    self inline: true.
    self sharedCodeNamed: 'normalSend' inCase: 131.
    rcvr := self internalStackValue: argumentCount.
    ((self isIntegerObject: rcvr) not and: [hasToTrace])
        ifTrue: [
            self internalTurnOnUsedBit: rcvr.
            ].
    lkupClass := self fetchClassOf: rcvr.
    receiverClass := lkupClass.
    self commonSend.

So...if it is not a SmallInetger and if the flag is on, I turn on a bit.

The question is, if I send a normal message to a normal object. Example:

| anObject |
anObject := MyClass new.
anObject foo

Now...I am sure that "anObject" was marked with the bit. But what about:
a) the compiled method MyClass >> #foo
b) MyClass

should they be marked?

In other words:

self deny: (unUsed primitiveGetUsedBit: anObject).
    self deny: (unUsed primitiveGetUsedBit: anObject class).
    self deny: (unUsed primitiveGetUsedBit: (anObject class >> #foo)).

    anObject foo.

    self assert: (unUsed primitiveGetUsedBit: anObject).
    self assert: (unUsed primitiveGetUsedBit: anObject class).
    self assert: (unUsed primitiveGetUsedBit: (anObject class >> #foo)).


should all the asserts pass? I ask because I don't know how CompiledMethods are executed (they receive a normalSend like any other object?) nor how class are accessed.

Thanks in advance,

Mariano

On Tue, May 11, 2010 at 5:43 PM, Igor Stasenko <[hidden email]> wrote:

On 11 May 2010 17:40, Mariano Martinez Peck <[hidden email]> wrote:
>
>
>> >
>> > Thanks Igor. I could see #class does not the normal way. It was logic as it already has the pointer there ;)
>> >
>> > Now I wonder...to avoid those special cases, do you think it makes sense to intercept in commonSend rather than commonSend ? or it would be the same ?
>> >
>> err... commonSend or commonSend? i think it would be the same :)
>>
>
> hahahah sorry, I meant commonSend instead of normalSend.
>
>
>>
>> the other point, where you can try intercept a send is cache lookup.
>>
>
> internalFindNewMethod ?
>

I don't know, maybe :)

> Thanks
>
> Mariano
>

===========================================================================

John M. McIntosh <[hidden email]> Twitter: squeaker68882

Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com

===========================================================================

smime.p7s (5K) Download Attachment

ccrraaiigg

Re: normalSend, specialObjectsArray and VM

> Craig Latta has done all this work, talk to him.

Sure, I'd be happy to discuss it.

-C

--
Craig Latta
www.netjam.org
+ 31 020 894 6247
+ 1 415 287 3547