Reify RBLiteralNode and RBBlockNode args/return with MetaLink?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Reify RBLiteralNode and RBBlockNode args/return with MetaLink?

Manuel Leuenberger
Hi,

I am currently playing with MetaLinks in Pharo 7 and stumbled upon some issues. For example, I found that instrumenting message sends may break compilation (see https://pharo.fogbugz.com/f/cases/22681/Metalink-on-message-may-break-compilation). Two other issues also arised:

i) How do I reify the value of an RB(Valuel|Literal|LiteralValue)Node? RFValueReification does not work, and there is no reification for literal, value, or literal value nodes.
ii) How do I reify arguments and results of blocks? I tried reify context for the block body #before and value for #after block "last", but I often run into the problem of no reification for literals.
iii) How are cascaded message sends reified? Am I supposed to instrument the cascade node or the individual message sends?

I am grateful for any pointers towards a solution. BTW, Marcus' lecture slides give a great intro to metalinks (http://marcusdenker.de/talks/18LectureMetaLinks/MetaLinks.pdf).

Cheers,
Manuel

Reply | Threaded
Open this post in threaded view
|

Re: Reify RBLiteralNode and RBBlockNode args/return with MetaLink?

Marcus Denker-4
Hi,

Thanks! I will check all of them… it might very well be that they are not implemented / there are bugs.

I will add them to the bug list.

(the problem is that each of them takes effort… write test, stare at byte-code… so it will take some time)

On 19 Nov 2018, at 09:30, Manuel Leuenberger <[hidden email]> wrote:

Hi,

I am currently playing with MetaLinks in Pharo 7 and stumbled upon some issues. For example, I found that instrumenting message sends may break compilation (see https://pharo.fogbugz.com/f/cases/22681/Metalink-on-message-may-break-compilation). Two other issues also arised:

i) How do I reify the value of an RB(Valuel|Literal|LiteralValue)Node? RFValueReification does not work, and there is no reification for literal, value, or literal value nodes.
ii) How do I reify arguments and results of blocks? I tried reify context for the block body #before and value for #after block "last", but I often run into the problem of no reification for literals.
iii) How are cascaded message sends reified? Am I supposed to instrument the cascade node or the individual message sends?

I am grateful for any pointers towards a solution. BTW, Marcus' lecture slides give a great intro to metalinks (http://marcusdenker.de/talks/18LectureMetaLinks/MetaLinks.pdf).

Cheers,
Manuel


Reply | Threaded
Open this post in threaded view
|

Re: Reify RBLiteralNode and RBBlockNode args/return with MetaLink?

Manuel Leuenberger
Thanks for picking this up into the backlog. I am not blocked if I don't have them, I was just curious. I am quite intrigued with the idea of reyfing the runtime semantics, this opens a whole lot of opportunities for tools based on dynamic data.

On Nov 19, 2018 17:40, Marcus Denker <[hidden email]> wrote:
Hi,

Thanks! I will check all of them… it might very well be that they are not implemented / there are bugs.

I will add them to the bug list.

(the problem is that each of them takes effort… write test, stare at byte-code… so it will take some time)

On 19 Nov 2018, at 09:30, Manuel Leuenberger <[hidden email]> wrote:

Hi,

I am currently playing with MetaLinks in Pharo 7 and stumbled upon some issues. For example, I found that instrumenting message sends may break compilation (see https://pharo.fogbugz.com/f/cases/22681/Metalink-on-message-may-break-compilation). Two other issues also arised:

i) How do I reify the value of an RB(Valuel|Literal|LiteralValue)Node? RFValueReification does not work, and there is no reification for literal, value, or literal value nodes.
ii) How do I reify arguments and results of blocks? I tried reify context for the block body #before and value for #after block "last", but I often run into the problem of no reification for literals.
iii) How are cascaded message sends reified? Am I supposed to instrument the cascade node or the individual message sends?

I am grateful for any pointers towards a solution. BTW, Marcus' lecture slides give a great intro to metalinks (http://marcusdenker.de/talks/18LectureMetaLinks/MetaLinks.pdf).

Cheers,
Manuel



Reply | Threaded
Open this post in threaded view
|

Re: Reify RBLiteralNode and RBBlockNode args/return with MetaLink?

Marcus Denker-4
I already have a fix for i) i) How do I reify the value of an RB(Valuel|Literal|LiteralValue)Node? 

I will commit it later today.

Marcus

On 19 Nov 2018, at 22:14, Manuel Leuenberger <[hidden email]> wrote:

Thanks for picking this up into the backlog. I am not blocked if I don't have them, I was just curious. I am quite intrigued with the idea of reyfing the runtime semantics, this opens a whole lot of opportunities for tools based on dynamic data.

On Nov 19, 2018 17:40, Marcus Denker <[hidden email]> wrote:
Hi,

Thanks! I will check all of them… it might very well be that they are not implemented / there are bugs.

I will add them to the bug list.

(the problem is that each of them takes effort… write test, stare at byte-code… so it will take some time)

On 19 Nov 2018, at 09:30, Manuel Leuenberger <[hidden email]> wrote:

Hi,

I am currently playing with MetaLinks in Pharo 7 and stumbled upon some issues. For example, I found that instrumenting message sends may break compilation (see https://pharo.fogbugz.com/f/cases/22681/Metalink-on-message-may-break-compilation). Two other issues also arised:

i) How do I reify the value of an RB(Valuel|Literal|LiteralValue)Node? RFValueReification does not work, and there is no reification for literal, value, or literal value nodes.
ii) How do I reify arguments and results of blocks? I tried reify context for the block body #before and value for #after block "last", but I often run into the problem of no reification for literals.
iii) How are cascaded message sends reified? Am I supposed to instrument the cascade node or the individual message sends?

I am grateful for any pointers towards a solution. BTW, Marcus' lecture slides give a great intro to metalinks (http://marcusdenker.de/talks/18LectureMetaLinks/MetaLinks.pdf).

Cheers,
Manuel




Reply | Threaded
Open this post in threaded view
|

Re: Reify RBLiteralNode and RBBlockNode args/return with MetaLink?

Marcus Denker-4
In reply to this post by Marcus Denker-4


i) How do I reify the value of an RB(Valuel|Literal|LiteralValue)Node? RFValueReification does not work, and there is no reification for literal, value, or literal value nodes.

This is now fixed. All subclasses of RBValueNode with the exception of Cascade now support #value. (Cascade: see below).

ii) How do I reify arguments and results of blocks? I tried reify context for the block body #before and value for #after block "last", but I often run into the problem of no reification for literals.

I added issue tracker entries:

The story is that we for now unified links on BlockNode to be the same as a link on the block body (as we have with RBMethodNode).

But this is wrong:  a link on a block (e.g. ) [1] should fire when the block *definition* is evaluated. Only a link on the body should fire on execution.

If we link on the body, it will be possible to reify the #value (which is the value of the last statement).

So this is some work (and a non-backward compatible change), but will be done till next week I hope.

iii) How are cascaded message sends reified? Am I supposed to instrument the cascade node or the individual message sends?


https://pharo.fogbugz.com/f/cases/22702/support-before-after-intead-on-RBCascadeNode

It turns out this is just not implemented. Single message sends *inside* should work (as it calls the same code as a normal send to emit code).
But even that needs tests.

Cascade itself needs some code for links, then it will be easy to add reifications for #receiver and #value (and #operation).

I will do that today I hope.

Marcus
Reply | Threaded
Open this post in threaded view
|

Re: Reify RBLiteralNode and RBBlockNode args/return with MetaLink?

Marcus Denker-4


iii) How are cascaded message sends reified? Am I supposed to instrument the cascade node or the individual message sends?


https://pharo.fogbugz.com/f/cases/22702/support-before-after-intead-on-RBCascadeNode

It turns out this is just not implemented. Single message sends *inside* should work (as it calls the same code as a normal send to emit code).
But even that needs tests.

Cascade itself needs some code for links, then it will be easy to add reifications for #receiver and #value (and #operation).


Done:



Marcus
Reply | Threaded
Open this post in threaded view
|

Re: Reify RBLiteralNode and RBBlockNode args/return with MetaLink?

Manuel Leuenberger
Awesome, thanks for the effort Marcus! 

I was wondering if it would be useful to use matrix-testing for MetaLinks. I imagine having example methods which employ all features of the Smalltalk syntax (syntax on a postcard?), for which all intermediate results of expressions are known, then apply MetaLinks in all possible combinations (arguments x control) and check the reifications and side-effects against the oracle. Then the tests would generate a matrix of which combinations work and which don't. This should cover most of the possible executions.

Cheers,
Manuel

On 28 Nov 2018, at 11:55, Marcus Denker <[hidden email]> wrote:



iii) How are cascaded message sends reified? Am I supposed to instrument the cascade node or the individual message sends?


https://pharo.fogbugz.com/f/cases/22702/support-before-after-intead-on-RBCascadeNode

It turns out this is just not implemented. Single message sends *inside* should work (as it calls the same code as a normal send to emit code).
But even that needs tests.

Cascade itself needs some code for links, then it will be easy to add reifications for #receiver and #value (and #operation).


Done:



Marcus

Reply | Threaded
Open this post in threaded view
|

Re: Reify RBLiteralNode and RBBlockNode args/return with MetaLink?

Marcus Denker-4


> On 28 Nov 2018, at 21:37, Manuel Leuenberger <[hidden email]> wrote:
>
> Awesome, thanks for the effort Marcus!
>
> I was wondering if it would be useful to use matrix-testing for MetaLinks. I imagine having example methods which employ all features of the Smalltalk syntax (syntax on a postcard?), for which all intermediate results of expressions are known, then apply MetaLinks in all possible combinations (arguments x control) and check the reifications and side-effects against the oracle. Then the tests would generate a matrix of which combinations work and which don't. This should cover most of the possible executions.
>

Yes, that would be nice.

        Marcus


Reply | Threaded
Open this post in threaded view
|

Re: Reify RBLiteralNode and RBBlockNode args/return with MetaLink?

Thierry Goubier
Hi Manuel,

would that extend to testing what happens if you add many, many
metalinks in a method?

I tried to rewrite the automated code tracer I had to use metalinks,
and I gave up once I discovered that going over a certain number of
metalinks in a method would stop working.

Thierry

Le jeu. 29 nov. 2018 à 09:03, Marcus Denker <[hidden email]> a écrit :

>
>
>
> > On 28 Nov 2018, at 21:37, Manuel Leuenberger <[hidden email]> wrote:
> >
> > Awesome, thanks for the effort Marcus!
> >
> > I was wondering if it would be useful to use matrix-testing for MetaLinks. I imagine having example methods which employ all features of the Smalltalk syntax (syntax on a postcard?), for which all intermediate results of expressions are known, then apply MetaLinks in all possible combinations (arguments x control) and check the reifications and side-effects against the oracle. Then the tests would generate a matrix of which combinations work and which don't. This should cover most of the possible executions.
> >
>
> Yes, that would be nice.
>
>         Marcus
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Reify RBLiteralNode and RBBlockNode args/return with MetaLink?

Marcus Denker-4


> On 29 Nov 2018, at 10:45, Thierry Goubier <[hidden email]> wrote:
>
> Hi Manuel,
>
> would that extend to testing what happens if you add many, many
> metalinks in a method?
>

yes, it would be nice to test that.


> I tried to rewrite the automated code tracer I had to use metalinks,
> and I gave up once I discovered that going over a certain number of
> metalinks in a method would stop working.
>


If you add a lot of code, you need to use the new byte code set as the old one is extremely limited for everything (number of temps, jump offset…)


        Marcus
Reply | Threaded
Open this post in threaded view
|

Re: Reify RBLiteralNode and RBBlockNode args/return with MetaLink?

Thierry Goubier
Le jeu. 29 nov. 2018 à 11:15, Marcus Denker <[hidden email]> a écrit :

>
>
>
> > On 29 Nov 2018, at 10:45, Thierry Goubier <[hidden email]> wrote:
> >
> > Hi Manuel,
> >
> > would that extend to testing what happens if you add many, many
> > metalinks in a method?
> >
>
> yes, it would be nice to test that.
>
>
> > I tried to rewrite the automated code tracer I had to use metalinks,
> > and I gave up once I discovered that going over a certain number of
> > metalinks in a method would stop working.
> >
>
>
> If you add a lot of code, you need to use the new byte code set as the old one is extremely limited for everything (number of temps, jump offset…)

Well, it failed before that limit; once the context was complex
enough, it would fail in various ways, such as being unable to bind to
the proper temporary, writing incorrect byte sequences, and so on

As anyway, it was injecting the same code that my tracer was already
doing, but in a less debuggable way, so I sort of gave up and kept
MetaLink use to simpler things, such as a type checker that inject
type checks on arguments based on the type inferred from arguments
names.

Now, when I see how you're improving metalinks in this thread, I'd be
interested in a higher view (source-level) of the transformations
you're doing for implementing metalinks: I think they then could be
reused in different contexts (i.e. refactorings, for example), or
generalised outside Smalltalk itself. It may probably already been
expressed that way, but I haven't looked into the metalink code
itself.

Thierry

>
>         Marcus

Reply | Threaded
Open this post in threaded view
|

Re: Reify RBLiteralNode and RBBlockNode args/return with MetaLink?

Marcus Denker-4


> On 29 Nov 2018, at 12:55, Thierry Goubier <[hidden email]> wrote:
>
> Le jeu. 29 nov. 2018 à 11:15, Marcus Denker <[hidden email]> a écrit :
>>
>>
>>
>>> On 29 Nov 2018, at 10:45, Thierry Goubier <[hidden email]> wrote:
>>>
>>> Hi Manuel,
>>>
>>> would that extend to testing what happens if you add many, many
>>> metalinks in a method?
>>>
>>
>> yes, it would be nice to test that.
>>
>>
>>> I tried to rewrite the automated code tracer I had to use metalinks,
>>> and I gave up once I discovered that going over a certain number of
>>> metalinks in a method would stop working.
>>>
>>
>>
>> If you add a lot of code, you need to use the new byte code set as the old one is extremely limited for everything (number of temps, jump offset…)
>
> Well, it failed before that limit; once the context was complex
> enough, it would fail in various ways, such as being unable to bind to
> the proper temporary, writing incorrect byte sequences, and so on
>

I think the changes we did will make that better…

> As anyway, it was injecting the same code that my tracer was already
> doing, but in a less debuggable way, so I sort of gave up and kept
> MetaLink use to simpler things, such as a type checker that inject
> type checks on arguments based on the type inferred from arguments
> names.
>
> Now, when I see how you're improving metalinks in this thread, I'd be
> interested in a higher view (source-level) of the transformations
> you're doing for implementing metalinks: I think they then could be
> reused in different contexts (i.e. refactorings, for example), or
> generalised outside Smalltalk itself. It may probably already been
> expressed that way, but I haven't looked into the metalink code
> itself.
>

For now it is implemented as a special compiler (with “self emitAfter: aNode”) added in overriden methods of the main compiler.

I made the mistake of doing some stuff (after links, for example) directly in the IR level, that did not work.

Now this is fixed for all but the links for Sequences (to be done later).

Steven did a version that is based on AST transformation only, we will explore that direction further in the future.

        Marcus
Reply | Threaded
Open this post in threaded view
|

Re: Reify RBLiteralNode and RBBlockNode args/return with MetaLink?

Thierry Goubier
Le ven. 30 nov. 2018 à 10:31, Marcus Denker <[hidden email]> a écrit :

>
>
>
> > On 29 Nov 2018, at 12:55, Thierry Goubier <[hidden email]> wrote:
> >
> > Le jeu. 29 nov. 2018 à 11:15, Marcus Denker <[hidden email]> a écrit :
> >>
> >>
> >>
> >>> On 29 Nov 2018, at 10:45, Thierry Goubier <[hidden email]> wrote:
> >>>
> >>> Hi Manuel,
> >>>
> >>> would that extend to testing what happens if you add many, many
> >>> metalinks in a method?
> >>>
> >>
> >> yes, it would be nice to test that.
> >>
> >>
> >>> I tried to rewrite the automated code tracer I had to use metalinks,
> >>> and I gave up once I discovered that going over a certain number of
> >>> metalinks in a method would stop working.
> >>>
> >>
> >>
> >> If you add a lot of code, you need to use the new byte code set as the old one is extremely limited for everything (number of temps, jump offset…)
> >
> > Well, it failed before that limit; once the context was complex
> > enough, it would fail in various ways, such as being unable to bind to
> > the proper temporary, writing incorrect byte sequences, and so on
> >
>
> I think the changes we did will make that better…
>
> > As anyway, it was injecting the same code that my tracer was already
> > doing, but in a less debuggable way, so I sort of gave up and kept
> > MetaLink use to simpler things, such as a type checker that inject
> > type checks on arguments based on the type inferred from arguments
> > names.
> >
> > Now, when I see how you're improving metalinks in this thread, I'd be
> > interested in a higher view (source-level) of the transformations
> > you're doing for implementing metalinks: I think they then could be
> > reused in different contexts (i.e. refactorings, for example), or
> > generalised outside Smalltalk itself. It may probably already been
> > expressed that way, but I haven't looked into the metalink code
> > itself.
> >
>
> For now it is implemented as a special compiler (with “self emitAfter: aNode”) added in overriden methods of the main compiler.
>
> I made the mistake of doing some stuff (after links, for example) directly in the IR level, that did not work.
>
> Now this is fixed for all but the links for Sequences (to be done later).
>
> Steven did a version that is based on AST transformation only, we will explore that direction further in the future.

Interesting, so this would be the one I'd like to see (because I
implemented with AST transforms). Would you have a link to his work?
Mine is on github, and the version I did at UBO was on the VisualWorks
AST.

Thierry

>         Marcus

Reply | Threaded
Open this post in threaded view
|

Re: Reify RBLiteralNode and RBBlockNode args/return with MetaLink?

Pharo Smalltalk Users mailing list


Steven did a version that is based on AST transformation only, we will explore that direction further in the future.

Interesting, so this would be the one I'd like to see (because I
implemented with AST transforms). Would you have a link to his work?
Mine is on github, and the version I did at UBO was on the VisualWorks
AST.



I want to look at that for real after finishing some things in the current version.

Marcus

Reply | Threaded
Open this post in threaded view
|

Re: Reify RBLiteralNode and RBBlockNode args/return with MetaLink?

Marcus Denker-4
In reply to this post by Marcus Denker-4


On 28 Nov 2018, at 09:38, Marcus Denker <[hidden email]> wrote:



i) How do I reify the value of an RB(Valuel|Literal|LiteralValue)Node? RFValueReification does not work, and there is no reification for literal, value, or literal value nodes.

This is now fixed. All subclasses of RBValueNode with the exception of Cascade now support #value. (Cascade: see below).

ii) How do I reify arguments and results of blocks? I tried reify context for the block body #before and value for #after block "last", but I often run into the problem of no reification for literals.

I added issue tracker entries:
...

I added a first fix and simple tests for this, it is in the latest version.

-> when putting a link on a block body, one can reify #arguments

TODO:
=> clean up the situation block definition vs block body
=> test in real world situations (might not yet work for more complex cases, e.g. nested blocks)
=> support #value in after links.


Marcus