[squeak-dev] Lightweight Classes

2008/11/17 Hernán Morales Durand <[hidden email]>:

> Dear all,
>   In the attachment you will find an implementation of Lightweight Classes
> for Squeak that follows the original paper "Debugging Objects" of Hinkle et
> al. However I was unable to set up the #class method in the LightweightClass
> to answer the real class of the lightwighted object, is this possible
> currently in Squeak?.
>
> What I did:
> -Created the LightweightClass as in the old VisualWorks implementation. The
> only difference here was CompiledMethod's cannot be copied, so I used
> #clone.
> -Created and installed an UnstoredMethod (a CompiledMethod subclass) in the
> LightweightClass's methodDictionary. Since I want to store the source code
> but not through the changes log, I borrowed the MethodWrappers idea of
> storing the state (sourceCode) in a class instance variable, and compile
> without logging. The methods I borrowed from MW are:
>
> -#copyWithTrailerBytes: - I think the superimplementor in CompiledMethod
> could use "self class" to avoid reimplementors.
> -#tweakClassFormat - which set the format but I don't know why and cannot
> find documentation on this.
> -#objectAt: I suspected the class literal was stored in the literal frame
> everytime I accepted a method, but for the simple test below still answer
> the LightweightClass and not the real class.
>
> Here is the test:
>
> | aDate |
> aDate := Date today.
> aDate becomeLightweight.
> aDate dispatchingClass compile: 'day ^43' notifying: nil.
> aDate day. " 43 (works) "
> aDate perform: #class. " Date ------> (works) "
> aDate class. " {Date} ------> LightweightClass (wrong) "
>

The only cause of this can be compiler.
Instead of generating an instruction for sending #class message, it
generates a bytecode to fetch the class from receiver oop , instead of
sending real message.

> Any hint would be appreciated.
> Best regards.
>
> Hernán
>
> PS: Just to avoid duplicate efforts, I wrote a LightweightClasses Browser
> which will be available as soon as I find a solution to the problem above.
>

--
Best regards,
Igor Stasenko AKA sig.

Perhaps the compiler could ask the class (or meta) of the method being compiled as to whether to inline a selector. All after the fact until a full recompile of the class, of course...

Regards, Gary.

----- Original Message ----- From: "Igor Stasenko" <[hidden email]>
To: "The general-purpose Squeak developers list" <[hidden email]>
Sent: Wednesday, November 19, 2008 8:03 PM
Subject: Re: [squeak-dev] Lightweight Classes

2008/11/19 Eliot Miranda <[hidden email]>:

On Sun, Nov 16, 2008 at 7:11 PM, Igor Stasenko <[hidden email]> wrote:

2008/11/17 Hernán Morales Durand <[hidden email]>:
> Dear all,
>   In the attachment you will find an implementation of Lightweight
> Classes
> for Squeak that follows the original paper "Debugging Objects" of > Hinkle
> et
> al. However I was unable to set up the #class method in the
> LightweightClass
> to answer the real class of the lightwighted object, is this possible
> currently in Squeak?.
>
> What I did:
> -Created the LightweightClass as in the old VisualWorks > implementation.
> The
> only difference here was CompiledMethod's cannot be copied, so I used
> #clone.
> -Created and installed an UnstoredMethod (a CompiledMethod subclass) > in
> the
> LightweightClass's methodDictionary. Since I want to store the source
> code
> but not through the changes log, I borrowed the MethodWrappers idea of
> storing the state (sourceCode) in a class instance variable, and > compile
> without logging. The methods I borrowed from MW are:
>
> -#copyWithTrailerBytes: - I think the superimplementor in > CompiledMethod
> could use "self class" to avoid reimplementors.
> -#tweakClassFormat - which set the format but I don't know why and
> cannot
> find documentation on this.
> -#objectAt: I suspected the class literal was stored in the literal
> frame
> everytime I accepted a method, but for the simple test below still
> answer
> the LightweightClass and not the real class.
>
> Here is the test:
>
> | aDate |
> aDate := Date today.
> aDate becomeLightweight.
> aDate dispatchingClass compile: 'day ^43' notifying: nil.
> aDate day. " 43 (works) "
> aDate perform: #class. " Date ------> (works) "
> aDate class. " {Date} ------> LightweightClass (wrong) "
>

The only cause of this can be compiler.
Instead of generating an instruction for sending #class message, it
generates a bytecode to fetch the class from receiver oop , instead of
sending real message.

IMO the bug is in the VM.  The compiler generates bytecode 199, the special
selector class send.  But it is the VM that decides to short-cut this send
and implement it by accessing the receiver's class directly instead of
actually sending the message.  I doubt very much this short-circuiting has
any impact on performance (VisualWorks has never inlined this special
selector send), and it is very easy to fix in the VM.  A number of other
special selector bytecodes become ordinary sends (e.g. next nextPut: etc).
If class were important for performance one could implement something
analogous to the at cache that speeds up at:, at:put: and size for special
selectors.  The special selector bytecode would check that the receiver of
the class message had the standard implementation (primitive in Object) and
answer the class directly.  But this isn't going to save significant time.
To fix this change
bytecodePrimClass
| rcvr |
rcvr := self internalStackTop.
self internalPop: 1 thenPush: (self fetchClassOf: rcvr).
self fetchNextBytecode.
to
bytecodePrimClass
messageSelector := self specialSelector: 23.
argumentCount := 0.
self normalSend.

Agreed, it seem a VM feature, kind of :)
As  temporary workaround, one could modify compiler to generate a
normal send bytecode instead of using a short bytecode for special
selector which #class is. I'm not sure how easy/hard such modification
could be made.

And i agree, that such optimization buys a little. I didn't seen a
code which does a heavy numerical crunching, where speed is essential,
and at same time needs a #class sends. To my sense a #class appears in
places where we need to deal with polymorphism or doing something on
meta levels - but in such areas, a correctess is far more important
than speed.

> Any hint would be appreciated.
> Best regards.
>
> Hernán
>
> PS: Just to avoid duplicate efforts, I wrote a LightweightClasses
> Browser
> which will be available as soon as I find a solution to the problem
> above.
>

--
Best regards,
Igor Stasenko AKA sig.

--
Best regards,
Igor Stasenko AKA sig.

--------------------------------------------------------------------------------

The only cause of this can be compiler.
Instead of generating an instruction for sending #class message, it
generates a bytecode to fetch the class from receiver oop , instead of
sending real message.

On Mon, Nov 17, 2008 at 4:11 PM, Igor Stasenko <[hidden email]> wrote:

The only cause of this can be compiler.
Instead of generating an instruction for sending #class message, it
generates a bytecode to fetch the class from receiver oop , instead of
sending real message.

Yes, the Compiler and Smalltalk bytecode set is doing this. I need this change too at some stage, so I've spent some time on it.

See this: http://users.ipa.net/~dwighth/smalltalk/bluebook/bluebook_chapter28.html

The send bytecode in question is decimal 199. This pushes the receiver's class on the stack, no questions asked.

I've looked at modifying the compiler, but I'm finding it to be a daunting piece of software.

The send bytecode is written to the compiled method stream in SelectorNode>>emit:args:on:super: at the top of the method as a "short send".

The SelectorNode instance is initialised as a shared class variable called "StdSelectors" in VariableNode>>initialize. As you can see there, its values are retrieved from the SystemDictionary. Once initialised as a class variable, it is again copied in Encoder>>initScopeAndLiteralTables.

So the special selectors array lives in the SystemDictionary's special objects array in slot 24, meaning that the VM also refers to these selectors. I'm not sure why the VM wants them.

As with fixing this, I guess that replacing the value in the special selectors array and then running VariableNode>>initialize might do the trick. Or it might crash the VM.

To fix this change

bytecodePrimClass

| rcvr |

rcvr := self internalStackTop.

self internalPop: 1 thenPush: (self fetchClassOf: rcvr).

self fetchNextBytecode.

to

bytecodePrimClass

messageSelector := self specialSelector: 23.

argumentCount := 0.

self normalSend.

On Thu, Nov 20, 2008 at 11:33 AM, Michael van der Gulik <[hidden email]> wrote:

On Mon, Nov 17, 2008 at 4:11 PM, Igor Stasenko <[hidden email]> wrote:

The only cause of this can be compiler.
Instead of generating an instruction for sending #class message, it
generates a bytecode to fetch the class from receiver oop , instead of
sending real message.

Yes, the Compiler and Smalltalk bytecode set is doing this. I need this change too at some stage, so I've spent some time on it.

See this: http://users.ipa.net/~dwighth/smalltalk/bluebook/bluebook_chapter28.html

The send bytecode in question is decimal 199. This pushes the receiver's class on the stack, no questions asked.

I've looked at modifying the compiler, but I'm finding it to be a daunting piece of software.

The send bytecode is written to the compiled method stream in SelectorNode>>emit:args:on:super: at the top of the method as a "short send".

The SelectorNode instance is initialised as a shared class variable called "StdSelectors" in VariableNode>>initialize. As you can see there, its values are retrieved from the SystemDictionary. Once initialised as a class variable, it is again copied in Encoder>>initScopeAndLiteralTables.

So the special selectors array lives in the SystemDictionary's special objects array in slot 24, meaning that the VM also refers to these selectors. I'm not sure why the VM wants them.

As with fixing this, I guess that replacing the value in the special selectors array and then running VariableNode>>initialize might do the trick. Or it might crash the VM.

...which worked!

To make #class a normal message send, do this:

Inspect "Smalltalk specialObjectsArray".
Find item 24, inspect it. This should be the special selectors array, but it might not be.
Find item 47 in the special objects array. It should be "#class", but it might not be.
In the special objects array inspector, evaluate "self at: 47 put: #primitiveClass".
Evaluate "VariableNode initialize".

-- or --

Evaluate "(Smalltalk specialObjectsArray at: 24) at: 47 put: #primitiveClass"
(but that would be less fun)

and then recompile your code.

Why does the VM need access to this special selectors array?

Gulik.

--
http://people.squeakfoundation.org/person/mikevdg
http://gulik.pbwiki.com/

On Thu, Nov 20, 2008 at 7:29 AM, Eliot Miranda <[hidden email]> wrote:

To fix this change

bytecodePrimClass

| rcvr |

rcvr := self internalStackTop.

self internalPop: 1 thenPush: (self fetchClassOf: rcvr).

self fetchNextBytecode.

to

bytecodePrimClass

messageSelector := self specialSelector: 23.

argumentCount := 0.

self normalSend.

Eliot: have you submitted this to the VM guys (via bugs.squeak.org I guess)?

If not, let me know and I'll fill in a bug report.

Michael.

--
http://people.squeakfoundation.org/person/mikevdg
http://gulik.pbwiki.com/

On Wed, Nov 19, 2008 at 2:45 PM, Michael van der Gulik <[hidden email]> wrote:

On Thu, Nov 20, 2008 at 7:29 AM, Eliot Miranda <[hidden email]> wrote:

To fix this change

bytecodePrimClass

| rcvr |

rcvr := self internalStackTop.

self internalPop: 1 thenPush: (self fetchClassOf: rcvr).

self fetchNextBytecode.

to

bytecodePrimClass

messageSelector := self specialSelector: 23.

argumentCount := 0.

self normalSend.

Eliot: have you submitted this to the VM guys (via bugs.squeak.org I guess)?

If not, let me know and I'll fill in a bug report.

Arguably it's not a bug, but an obsolete optimization.  Feel free to submit the bug, but I'll probably change it in Cog anyway.  I'll talk it over with Andreas on Friday.

Hi Michael--

> I need this change too at some stage, so I've spent some time on it.

    FYI, the Spoon VM modifications[1] address all such messages, so that remote message-sending works transparently for all messages.

It works now.

I read the paper of Hinkle and al. Lightweight classes are useful to debug.
Using lightweight class assumes that a programmer is able to infer from which object his analyze should start from. I am wondering whether this is not a too strong assumption. From my experience, just guessing from which class to start from is a big step.

Cheers,
Alexandre

On 19 Nov 2008, at 11:56, Hernán Morales Durand wrote:

Hi Alexandre,
 Monticello store UnstoredMethod class definition type as #compiledMethod where should be #bytes in this particular case [*]. Try to load the attached .mcz or load the previous one using the Load option in the file list (you probably need to add the .mcz extension if was removed by your e-mail client).
Best regards.

Hernán

[*] To fix this in MC in #typeOfClass one should invert the type check order:

self instSpec = CompiledMethod instSpec ifTrue:[        ^ #compiledMethod]. "Very special!"
self isBytes ifTrue:[^#bytes].

to

self isBytes ifTrue:[^#bytes].
self instSpec = CompiledMethod instSpec ifTrue:[        ^ #compiledMethod]. "Very special!"

but maybe someone with good knowledge of MC should confirm if that's ok. (I'm using Monticello-pmm.318)

2008/11/19 Bergel, Alexandre <[hidden email]>
Hi Hernán,

I get an error when trying to load your file: '5 instvar names are required'
Strange.

Alexandre



On 16 Nov 2008, at 22:58, Hernán Morales Durand wrote:

Dear all,
In the attachment you will find an implementation of Lightweight Classes for Squeak that follows the original paper "Debugging Objects" of Hinkle et al. However I was unable to set up the #class method in the LightweightClass to answer the real class of the lightwighted object, is this possible currently in Squeak?.

What I did:
-Created the LightweightClass as in the old VisualWorks implementation. The only difference here was CompiledMethod's cannot be copied, so I used #clone.
-Created and installed an UnstoredMethod (a CompiledMethod subclass) in the LightweightClass's methodDictionary. Since I want to store the source code but not through the changes log, I borrowed the MethodWrappers idea of storing the state (sourceCode) in a class instance variable, and compile without logging. The methods I borrowed from MW are:

-#copyWithTrailerBytes: - I think the superimplementor in CompiledMethod could use "self class" to avoid reimplementors.
-#tweakClassFormat - which set the format but I don't know why and cannot find documentation on this.
-#objectAt: I suspected the class literal was stored in the literal frame everytime I accepted a method, but for the simple test below still answer the LightweightClass and not the real class.

Here is the test:

| aDate |
aDate := Date today.
aDate becomeLightweight.
aDate dispatchingClass compile: 'day ^43' notifying: nil.
aDate day. " 43 (works) "
aDate perform: #class. " Date ------> (works) "
aDate class. " {Date} ------> LightweightClass (wrong) "

Any hint would be appreciated.
Best regards.

Hernán

PS: Just to avoid duplicate efforts, I wrote a LightweightClasses Browser which will be available as soon as I find a solution to the problem above.
<LightweightClasses-hfm.1.mcz>

--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel  http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

<LightweightClasses-hfm.2.mcz>

--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel  http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.