[GLASS] Compiling methods dinamically in Gemstone / Importing Pharo 3.0 classes

Hi, we're having some problems importing our Pharo 3.0 code into GemStone. More specifically, meta-programing code.

In Pharo 3.0, given a class you can ask for its compiler and compile methods for it easily. These methods can then be run dynamically for a recipient object and the arguments the method requires.

In GemStone, we couldn't find this functionality of just asking a class for its compiler. Instead we found out that there's a message #compile found in the class methods for class Behaviour. We tried this message out and it compiles code (strings) but it automatically assigns the resulting method to the method dictionary of the class of the recipient object of the message.

Examples:
In Gemstone,

    C compile: 'hello', String cr, '^4'

... running that collaboration would add a new method for message #hello in class C with the corresponding code. Instances of C now can answer to the message #hello

On the other hand, in Pharo 3.0
    code := 'hello', String cr, '^4'.
    aCompiler :=C compiler
                source: code;
                requestor: nil;
                category: '';
                failBlock:  [ ^nil ].
        aCompiledMethod := aCompiler translate generateWithSource.

...aCompiledMethod would be a compiled method "cooked" for instances of class C but it wouldn't be added to the method dictioary of Class C: instances of Class C don't know how to answer the message #hello. So, if we wanted to send the message "hello" to an instance X of class C then we would run that method with X as the recipient object and no arguments:
    aCompiledMethod valueWithReceiver: X arguments: #()

¿Is there a way to do something of the sort in GemStone? That is, compile methods dynamically without them auto-adding-themselves on the methods of a Class. In other words, to have standalone methods.

Everything we did on Pharo 3.0 is coupled to the way we compile methods. We would like it if we could import our code into GemStone without having to rewrite everything.

The best solution would be to be able to move the base classes that we are using in Pharo 3.0 to GemStone. That way, our developing enviroment (Pharo 3.0) would be independent of our persisting environment (GemStone). ¿Is there a way to do it? More generally, ¿Is there a way to move all of Pharo 3.0 to GemStone? I'm aware that Pharo has classes that wouldn't make any sense in GemStone, like the ones responsible for the GUI, but that shouldn't be a problem. They wouldn't make sense but they wouldn't hurt either.

Thanks.

On Dec 7, 2014, at 9:21 AM, AleDanos via Glass <[hidden email]> wrote:
>
> Hi, we're having some problems importing our *Pharo 3.0* code into GemStone.
> More specifically, meta-programing code.

Meta-programming is certainly an area where cross-dialect portability will be problematic.

> In Pharo 3.0, given a class you can ask for its compiler and compile methods
> for it easily. These methods can then be run dynamically for a recipient
> object and the arguments the method requires.
>
> In GemStone, we couldn't find this functionality of just asking a class for
> its compiler.

Right. In GemStone, the compiler is in C and always adds the compiled method to the class.

I don’t think GemStone has any direct facility to do this. You can create stand-alone methods as long as they don’t take any arguments (see String>>#’_compileInContext:symbolList:’), but not that take an argument. You could implement the #’valueWithReceiver:arguments:’ method to add the method to a subclass, use #’become:’ to switch ‘X’ to the subclass, and send the message, but this is pretty complicated.

> Everything we did on Pharo 3.0 is coupled to the way we compile methods. We
> would like it if we could import our code into GemStone without having to
> rewrite everything.

You will probably need a compatibility layer in each dialect that implements common methods.

> The best solution would be to be able to move *all of the pre-existant*
> classes that we are using in Pharo 3.0 to GemStone. That way, our developing
> enviroment (Pharo 3.0) would be independent of our persisting environment
> (GemStone). ¿Is there a way to do it? More generally, ¿Is there a way to
> move all of *Pharo 3.0* to GemStone? I'm aware that Pharo has classes that
> wouldn't make any sense in GemStone, like the ones responsible for the GUI,
> but that shouldn't be a problem. They wouldn't make sense but they wouldn't
> hurt either.

Actually, I’ve been interested in this idea for some time. As part of the Maglev/Ruby project, GemStone added method environments to the system so that Ruby code could be executed without any direct interference from the Smalltalk code. While it would be a substantial effort, it should be possible to put most of the Pharo methods in GemStone's environment 2 (environment 1 is reserved for Ruby) and then just run as if it were a Pharo image (with appropriate bridge methods to call VM primitives).

I think this would be too much effort to make it worthwhile for any one project (such as yours), but if it were done it would be useful to all. Of course, there are a lot of things in life for which that statement is true.

James

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

AleDanos wrote

Hi, we're having some problems importing our Pharo 3.0 code into GemStone. More specifically, meta-programing code.

In Pharo 3.0, given a class you can ask for its compiler and compile methods for it easily. These methods can then be run dynamically for a recipient object and the arguments the method requires.

In GemStone, we couldn't find this functionality of just asking a class for its compiler. Instead we found out that there's a message #compile found in the class methods for class Behaviour. We tried this message out and it compiles code (strings) but it automatically assigns the resulting method to the method dictionary of the class of the recipient object of the message.

Examples:
In Gemstone,

    C compile: 'hello', String cr, '^4'

... running that collaboration would add a new method for message #hello in class C with the corresponding code. Instances of C now can answer to the message #hello

On the other hand, in Pharo 3.0
    code := 'hello', String cr, '^4'.
    aCompiler :=C compiler
                source: code;
                requestor: nil;
                category: '';
                failBlock:  [ ^nil ].
        aCompiledMethod := aCompiler translate generateWithSource.

...aCompiledMethod would be a compiled method "cooked" for instances of class C but it wouldn't be added to the method dictioary of Class C: instances of Class C don't know how to answer the message #hello. So, if we wanted to send the message "hello" to an instance X of class C then we would run that method with X as the recipient object and no arguments:
    aCompiledMethod valueWithReceiver: X arguments: #()

¿Is there a way to do something of the sort in GemStone? That is, compile methods dynamically without them auto-adding-themselves on the methods of a Class. In other words, to have standalone methods.

Everything we did on Pharo 3.0 is coupled to the way we compile methods. We would like it if we could import our code into GemStone without having to rewrite everything.

The best solution would be to be able to move the base classes that we are using in Pharo 3.0 to GemStone. That way, our developing enviroment (Pharo 3.0) would be independent of our persisting environment (GemStone). ¿Is there a way to do it? More generally, ¿Is there a way to move all of Pharo 3.0 to GemStone? I'm aware that Pharo has classes that wouldn't make any sense in GemStone, like the ones responsible for the GUI, but that shouldn't be a problem. They wouldn't make sense but they wouldn't hurt either.

Thanks.

The Behavior method #compileMethod:dictionaries:category:intoMethodDict:intoCategories: might work for you. In this method, you provide the method and category dictionaries into which the method is inserted.

You should also look into session methods.


Ultimately, we can best recommend a GemStone solutino once we understand your use case. What problem is your Pharo solution addressing? Why don't you want your methods in the class?

Wow, they weren't kidding when they said that GemStone has a very active and helpful community. In just 2 hours 3 people came offering help.

@James: Ok, thanks for the help. Your workaround does sound like a hassle (the one involving the "became:" message) and I don't think we have the knowledge necessary to apply it in time (we are familiar with the "became:" message but we have no practice using it and neither with the rest of that workaround) . It does sound really interesting though. Thanks.

And, yeah. It'd be great if someone went through the trouble of making GemStone and Pharo compatible but, as you've said, "I think this would be too much effort to make it worthwhile for any one project". I, for one, don't have the experience nor the time needed. Maybe in a few months.

@Mariano
The "'hello ^4' _compileInContext: nil symbolList: SymbolList new oldLitVars: nil environmentId: 0" workaround raises an error when we try to run it in a Workspace ("Unexpected token" next to the word "hello" in the String) and when we try to debug it the debugger in turn raises another error (but the GemStone debugger never "worked" for us. It does show but for an error trying to show the debugger. Then we have to search for the method we are trying to debug in the Execution Stack of the Debugger).

The "strategy pattern" workaround  to delete the method sound a little "too much" but also possible. We could use it as a last resort if nothing else works.

More info on the problem: we are students at Univerdad de Buenos Aires (UBA), Argentina and we are working on an assignment to implement Self functionality in Smalltalk. That is, to have objects that behave like in Self Programming Language (prototyping instead of classes, slots, parents, etc) [1].
The assigment is divided in three stages:
Stage 1: implement the aforementioned Self Objects in Smalltalk. Our approach was to have a "SelfObject" class that knows how to answer to the minimum amount of messages as possible. That way, when we send any message to an instance of said class we could intercept it with a DNU (does not understand) method. We had a "slots" dictionary as an instance variable as well as a "parents" dictionary as an instance variable. We also implemented Mirrors [2], a Method Lookup resembling the Self Method Lookup, etc.

Stage 2: After finishing Stage 1 we had to design a website using Seaside that provided the functionality of creating objects, accesing them, cloning them, etc; from a Web Browser (it was a little much harder than it sounds. Not from the Seaside part but from adapting what we did in Stage 1 and expanding it to provide what our Seaside website expected).

Stage 3: The final stage would be to add GemStone to the mix. That is, persist the website with GLASS (and some other minor things).


@Richard Sargent
It sounds like that would cut it ( #compileMethod:dictionaries:category:intoMethodDict:intoCategories:). I'm out of time right now but i'll check it out tomorrow.

The problem with methods being automatically added is that then all of our Self Objects would know how to answer those messages. The idea was to compile a method and save it in the object's "slots" instance variable dictionary. Then, if someone sent the message "hello" to that SelfObject we would do the "method lookuping" and run the respective method for "hello". The set of methods implemented in the class should be kept to the minimum so all the instances can have different behaviour.

Thanks to all for your help.

[1] http://www.selflanguage.org/_static/published/self-power.pdf
[2] http://bracha.org/mirrors.pdf

Thanks for the comments and details about your project. While I assume that part of the assignment is to come up with a solution, I imagine that your professor is happy to have you come to the community for ideas.

One idea would be to use blocks and then send #’valueWithArguments:’ to the block. There would be a bit of work parsing the method name and body, but you could add the method to a class, obtain the selector, and then parse the source to leave out the selector. You could then create a block and save it instead of your method. I think this code would be more portable.

Another approach would be to have a separate class for each object. At first it seems like a lot of overhead, but a class is just another object and your implementation provides for dictionaries holding methods—essentially a class!

The unexpected token is the ‘hello’ since this method is expecting something equivalent to the internals of a zero-argument block.

This is used to implement session methods (that aren’t persisted as part of the class). The problem for you is that the method will still apply to all instances of the class in that session.

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

AleDanos wrote

Wow, they weren't kidding when they said that GemStone has a very active and helpful community. In just 2 hours 3 people came offering help.

@James: Ok, thanks for the help. Your workaround does sound like a hassle (the one involving the "became:" message) and I don't think we have the knowledge necessary to apply it in time (we are familiar with the "became:" message but we have no practice using it and neither with the rest of that workaround) . It does sound really interesting though. Thanks.

And, yeah. It'd be great if someone went through the trouble of making GemStone and Pharo compatible but, as you've said, "I think this would be too much effort to make it worthwhile for any one project". I, for one, don't have the experience nor the time needed. Maybe in a few months.

@Mariano
The "'hello ^4' _compileInContext: nil symbolList: SymbolList new oldLitVars: nil environmentId: 0" workaround raises an error when we try to run it in a Workspace ("Unexpected token" next to the word "hello" in the String) and when we try to debug it the debugger in turn raises another error (but the GemStone debugger never "worked" for us. It does show but for an error trying to show the debugger. Then we have to search for the method we are trying to debug in the Execution Stack of the Debugger).

The "strategy pattern" workaround  to delete the method sound a little "too much" but also possible. We could use it as a last resort if nothing else works.

More info on the problem: we are students at Univerdad de Buenos Aires (UBA), Argentina and we are working on an assignment to implement Self functionality in Smalltalk. That is, to have objects that behave like in Self Programming Language (prototyping instead of classes, slots, parents, etc) [1].
The assigment is divided in three stages:
Stage 1: implement the aforementioned Self Objects in Smalltalk. Our approach was to have a "SelfObject" class that knows how to answer to the minimum amount of messages as possible. That way, when we send any message to an instance of said class we could intercept it with a DNU (does not understand) method. We had a "slots" dictionary as an instance variable as well as a "parents" dictionary as an instance variable. We also implemented Mirrors [2], a Method Lookup resembling the Self Method Lookup, etc.

Stage 2: After finishing Stage 1 we had to design a website using Seaside that provided the functionality of creating objects, accesing them, cloning them, etc; from a Web Browser (it was a little much harder than it sounds. Not from the Seaside part but from adapting what we did in Stage 1 and expanding it to provide what our Seaside website expected).

Stage 3: The final stage would be to add GemStone to the mix. That is, persist the website with GLASS (and some other minor things).


@Richard Sargent
It sounds like that would cut it ( #compileMethod:dictionaries:category:intoMethodDict:intoCategories:). I'm out of time right now but i'll check it out tomorrow.

The problem with methods being automatically added is that then all of our Self Objects would know how to answer those messages. The idea was to compile a method and save it in the object's "slots" instance variable dictionary. Then, if someone sent the message "hello" to that SelfObject we would do the "method lookuping" and run the respective method for "hello". The set of methods implemented in the class should be kept to the minimum so all the instances can have different behaviour.

As James has mentioned, creating classes on the fly is probably your best bet for making Smalltalk emulate Self. Copy the method and category dictionaries from the prototype to the new class, compile the methods specific to the new class, and away you go.

This has the best chance for success, as it is viable in both Pharo and GemStone and it plays to Smalltalk's model.

As a general guideline, you want Smalltalk applications to play within the Smalltalk design. The further from this you go, the more work you have to do and the less your chances of success. As long as you can play within the model, you should be able to do so "easily". :-)

Thanks to all for your help.

[1] http://www.selflanguage.org/_static/published/self-power.pdf
[2] http://bracha.org/mirrors.pdf

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

@James and @Richard Sargent Yes, it was even his idea to ask for help in the community.
The "block" solution was our first try during Stage 1 but the "self" is lost. That is, "self" loses the "variable" in the "pseudovariable" name. It cannot be used in the code.

I think we also tried with the "one class per object" solution (or at least we thought about it), but I can't recall what was the problem with it (maybe the problem is no longer a problem now, It's late I'll have to test it out in a day or two as tomorrow I have an exam.)


@"The third commenter" (can't see your username on Nabble, no offense :) ) (it's probably you, Mariano)
Yes, we do need arguments but thanks for the work of testing it out. Any help is appreciated.


Ale

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

On Dec 8, 2014, at 8:15 PM, AleDanos via Glass <[hidden email]> wrote:
>
> @James and @Richard Sargent Yes, it was even his idea to ask for help in the
> community.
> The "block" solution was our first try during Stage 1 but the "self" is
> lost. That is, "self" loses the "variable" in the "pseudovariable" name. It
> cannot be used in the code.

Excellent point. I hadn’t thought of that.

> I think we also tried with the "one class per object" solution (or at least
> we thought about it), but I can't recall what was the problem with it (maybe
> the problem is no longer a problem now, It's late I'll have to test it out
> in a day or two as tomorrow I have an exam.)

I’ll be interested in reading more about this approach.

I have another idea to consider. Compile the method into your SelfObject, remove it from the method dictionary, then re-add it to the method dictionary with a unique key. Then in your DNU handler do a lookup from the selector and the object and resend the message with arguments to the unique key.

James

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass