[squeak-dev] A question about block closures.

Dear Squeakers,

I have a question for some of Squeak gurus on the list.  I've been porting a javascript project I did a couple months back to Squeak, and in the process of porting it, added an ad hoc object class that handled the prototype based inheritance I used in the javascript version.

One of the issues I encountered was that when I did a copy of my object, the block closures supplied for various properties required a bit of a hack so I have a method:

doesNotUnderstand: aMessage

"Generic catch all that performs all selectors"

(self can: aMessage selector) ifTrue: [ 

(self s: aMessage selector ) outerContext receiver: self. 

^( self s: aMessage selector )valueWithEnoughArguments: aMessage arguments ].

^ self s: aMessage selector.

Now this seems to do exactly what I wanted 99% of the time.  It mimics the behavior of using a block closure as a javascript function() {} object, and all other objects are returned as intended.  The #s: method returns nil if the property is not found in the object's dictionary as well, and #can: is just a fancy way to test if the property is a block closure.  By using valueWithEnoughArguments: it even gets padded out with nils like JS would.

I'm not entirely happy with how hackish this seems, patching the outerContext and then re-rolling the arguments array to one of the proper size just to copy into a MethodContext. 

My question is "Is there a better way?"

Dave

--
-=-=-=-=-=-=-=-=-=-=- http://blog.dloh.org/

You could do something analogous with messages and methods an d have the Vm associate the receiver with the activation in... a message send :)

If you had a dictionary that cached translations form short forms of selectors, e.g. myJSFunc:with: -> myJSFunc:with:with: then your code might look something like

doesNotUnderstand: aMessage

"Generic catch all that performs all selectors"

(self can: aMessage selector) ifTrue:

[actualSelector := self s: aMessage selector.

^self perform: actualSelector

withArguments: (aMessage arguments, (Array new: actualSelector numArgs - aMessage selector numArgs))]

Would this be better? 

On Wed, Jul 8, 2009 at 2:04 AM, Eliot Miranda <[hidden email]> wrote:

You could do something analogous with messages and methods an d have the Vm associate the receiver with the activation in... a message send :)

If you had a dictionary that cached translations form short forms of selectors, e.g. myJSFunc:with: -> myJSFunc:with:with: then your code might look something like

doesNotUnderstand: aMessage

"Generic catch all that performs all selectors"

(self can: aMessage selector) ifTrue:

[actualSelector := self s: aMessage selector.

^self perform: actualSelector

withArguments: (aMessage arguments, (Array new: actualSelector numArgs - aMessage selector numArgs))]

Would this be better? 

Well it it had a prayer of working :)  Here's a little backstory.  The way this object works is it inherits from IdentityDictionary has two accessor methods:

has: aProperty of: aValue

        " This corresponds to object.property = value and object['property'] = value"

        self at: aProperty put: aValue.

        ^ self.

and

s: aProperty

       " This corresponds to object.property and object['property']"

       ^ self at: aProperty ifAbsent: [^ nil ].

For a very limited sets of methods such as these, the ones that can not be overridden by end user code, these fields are accessed by doing a message

send to the object, and the dNU routine determines which route to return based on the method can:

can: aSymbol

     " This corresponds to the javascript:  Object.prototype.can = function(k) { return this.hasOwnProperty(k) && typeof(this[k]) == 'function' }"

     ^ ( self s: aSymbol ) isBlock.

What happens in most of the code is objects generate block closures which they then set as the values of other object's properties. The code ends up looking like this toy example:

----

(An object named: #Casper) has: #greets: of: [ :x | Transcript show: 'Hi ', x name , '! My name is ', self name; cr].

(An object named: #George) from: Casper.

Casper greets: George.

George greets: Casper.

Casper has: #greets: of: [ :x | Transcript show: 'Goodbye ', x name; cr]

Casper greets: George.

George greets: Casper.

----

And the Transcript then reads:

----

Hi George! My name is Casper

Hi Casper! My name is George

Goodbye George

Hi Casper! My name is George

----

The name and named: methods actually set key value and find key by value in the Smalltalk dictionary, so they're globals as in editing Javascript's window object.

But as you see, I'm passing a block closure to one object, and copying it to another object, and both objects are referencing the same block closure!  I'm overwriting the

original block closure's outerContext's receiver (which in this example is originally nil!), with whatever object is referencing it.  This is similar to what happens with Javascript 

method calls, where the this variable is set by hacking the scope object at call time.  But this lets these objects respond to any message that may come their way, and usually

do the right thing.  My original dNU used the valueWithEnoughArugments: method, because it was the only way I could find to neatly get all the block closures to work in both

the case of over and underflow, as their equivalent functions worked in the JS version.

If I compile an actual method send, and dNU is not called, I then need to always compile a method every time a property is set, even if it is never executed. 

In this application, objects inherit based on a shallow copy of object properties.  So you may inherit some methods using from: from one object, a bunch of others from another,

and then override some of each of those in a third.  (This scenario actually the norm not the exception btw!)  99% of the time, I want whatever the values the block closure 

derived from its original defining context, with the sole exception of the outerContext's receiver, which is almost always wrong for my purposes.    

I'm mostly wondering if there's a cleaner, more optimizeable way of messing around with the internals of either the resulting MethodContext or your closure objects :)

Dave

--
-=-=-=-=-=-=-=-=-=-=- http://blog.dloh.org/

Well the obvious way to clean up those closures is to pass the self reference as an argument, so

(An object named: #Casper) has: #greets: of: [ :theSelf :x | Transcript show: 'Hi ', x name , '! My name is ', theSelf name; cr].

doesNotUnderstand: aMessage

"Generic catch all that performs all selectors"

(self can: aMessage selector) ifTrue:

[^( self s: aMessage selector ) valueWithEffectiveReceiver: self andEnoughArguments: aMessage arguments ].

^ self s: aMessage selector.

As regards more generic property handling I don't know of a better way than the doesNotUnderstand: handler for handling properties held in per-instance dictionaries if the constraint is that you definitely don't want to compile accessors for the property and you are using the vanilla VM.

But methods are cheap and so I don't see the benefit of avoiding methods in favour of blocks.

  Vassili came up with the great idea of generating bytecode for slot accessors directly and minting a set of 256 accessors that traversed the parent chain in a loop.  Effectively each accessor method looked like

accessSlot0

    | object |

    object := self.

    [object slot0 ifNotNil: [:value| ^value].

     (object := object parentSlot) ~~ nil] whileTrue.

    ^#undefined

Both Claus Gittinger and Dan Ingalls have had the idea of creating an object that is an instance of itself.  One way, without VM mods (and ignoring issues with compact classes), is to do something like

prototype: aPrototype numNewSlots: numNewSlots

    | format object |

    object := (Array new: aPrototype numberOfSlots + numNewSlots + 3).

    object

         at: 1 "superclass" put: aPrototype;

         at: 2 "methods"    put: MethodDictionary new;

         at: 3 "format"       put: (self formatOf: aPrototype format withAdditionalSlots: numNewSlots).

   object changeClassTo: object.

   ^object

Combine one with the other and you have the potential for a lean and mean prototype-based system using the existing VM technology, which would mean an efficient JavaScript implementation within Smalltalk.

On Wed, Jul 8, 2009 at 12:55 PM, Eliot Miranda <[hidden email]> wrote:

Well the obvious way to clean up those closures is to pass the self reference as an argument, so

(An object named: #Casper) has: #greets: of: [ :theSelf :x | Transcript show: 'Hi ', x name , '! My name is ', theSelf name; cr].

That's an even worse hack. :)

 doesNotUnderstand: aMessage

"Generic catch all that performs all selectors"

(self can: aMessage selector) ifTrue:

[^( self s: aMessage selector ) valueWithEffectiveReceiver: self andEnoughArguments: aMessage arguments ].

^ self s: aMessage selector.

I take it you'd have to define #valueWithEffectiveReceiver:andEnoughArguments: as BlockClosure has no such beast.  If  the closure was promoted to a CompiledMethod, then I guess you could use #valueWithReceiver:arguments: and another patched Array.  Or did I miss something?

 As regards more generic property handling I don't know of a better way than the doesNotUnderstand: handler for handling properties held in per-instance dictionaries if the constraint is that you definitely don't want to compile accessors for the property and you are using the vanilla VM.

I figured the dNU approach was close enough to what Morphic did, that it was "good enough".  But I was surprised how 

weird the code got.

But methods are cheap and so I don't see the benefit of avoiding methods in favour of blocks.

It largely has to do with not being able to get methods to actually work :)  I originally was trying to patch in CompiledMethods but ran into too many of the blowing up in bizarre ways, and the semantics drifting too far

from the JS version of the API. 

  Vassili came up with the great idea of generating bytecode for slot accessors directly and minting a set of 256 accessors that traversed the parent chain in a loop.  Effectively each accessor method looked like

accessSlot0

    | object |

    object := self.

    [object slot0 ifNotNil: [:value| ^value].

     (object := object parentSlot) ~~ nil] whileTrue.

    ^#undefined

I have a OO Forth that does the exact same thing, except it rolls native x86 machine code :)

 Both Claus Gittinger and Dan Ingalls have had the idea of creating an object that is an instance of itself.  One way, without VM mods (and ignoring issues with compact classes), is to do something like

prototype: aPrototype numNewSlots: numNewSlots

    | format object |

    object := (Array new: aPrototype numberOfSlots + numNewSlots + 3).

    object

         at: 1 "superclass" put: aPrototype;

         at: 2 "methods"    put: MethodDictionary new;

         at: 3 "format"       put: (self formatOf: aPrototype format withAdditionalSlots: numNewSlots).

   object changeClassTo: object.

   ^object

I did something similar with another project, but I've found doing the deep copies on the MethodDictionary ends up 

failing in weird ways.  

 Combine one with the other and you have the potential for a lean and mean prototype-based system using the existing VM technology, which would mean an efficient JavaScript implementation within Smalltalk.

I guess I know what the next set of experiments I'm going to do are :)

Thanks Eliot for thinking through this with me.

The main reason I'm building this, is I've got this very fluid ad hoc Javascript editing environment I wrote that I'm trying to port to Squeak.  I've been writing a book on programming that runs the reader through the process of designing a simple 8bit CPU, that has been implemented as step by step drag and drop examples on top of it.  

The only problem is that the only JS + HTML5 implementation out there in the wild that can actually handle it is Safari, and I can't bundle that with the book :)

If I can get it ported to Squeak, I can add an additional appendix and give the code listings in both Smalltalk and Javascript.  But I want the code to have direct analogues, so the translations of concepts between languages are

clear.

Dave

--
-=-=-=-=-=-=-=-=-=-=- http://blog.dloh.org/

ur original message wasn't clear on the subject) that what you do when you want to reuse one of these closures is that you copy the closure and its enclosing MethodContext (remember this is a chain and blocks could be nested) and assign the new receiver in the copy.  That's not too bad but its still very heavyweight because you're not sharing code between instances.

But you don't need to copy the MethodDictionaries.  With the above approach they're shared/inherited.  So when a message is sent to a subinstance the lookup machinery will follow the prototype chain until it finds a method or triggers dNU.

 In the activation self is bound to the original receiver.  The issue is hence making sure that slot layouts in subinstances match their parents so that parent methods work correctly on subinstances.  But that's not a new problem, Smalltalk does exactly that :)

Please keep me in the loop.  This is interesting and I'm happy to review.