Opal decompiler: I've been a big good ... I've been a bit bad ...

For whoever might care: I rewrote / fixed the Opal decompiler.

Sorry about me being a bit bad... my only test is this:

CompiledMethod allInstances collect: [ :m |
     (OCBytecodeDecompiler new decompile: m) compiledMethod ]

At least on the helvetia image that doesn't give any errors :)

On to hooking it in with my new classbuilder and MethodModification
model to apply bytecode rewriting for hyperfast class changes. I'll be
back in an hour ;P

cheers,
Toon

On 27 March 2011 19:14, Toon Verwaest <[hidden email]> wrote:
 I'll be back in an hour ;P

and bring a new compiler/decompiler for us this time :)

>
> cheers,
> Toon
>
>



--
Best regards,
Igor Stasenko AKA sig.

Does anybody know what I'm supposed to transform accesses to removed
instance variables into? Updating indexes of instance variables already
works.

And what data do I have to copy over to the new method so that it keeps
a pointer to the original source, the class pointer and the selector?

cheers,
Toon

On 03/27/2011 07:33 PM, Igor Stasenko wrote:

On 27 March 2011 19:46, Toon Verwaest <[hidden email]> wrote:
> Does anybody know what I'm supposed to transform accesses to removed
> instance variables into? Updating indexes of instance variables already
> works.
>
I guess  it would be cool to transform it to message like

'self undefinedVariableAccess: #varName'
and:
'self undefinedVariableAccess: #varName value: val'
for assignments.

Then later, when given method will run, it will trigger an error and
developer can fix it in debugger.

(Unless, of course you simply raise an error during (re)compilation
when given ivar removed from class).

> And what data do I have to copy over to the new method so that it keeps a
> pointer to the original source, the class pointer and the selector?
>

If you create a copy of method, copy the method's trailer.


trailer := oldMethod trailer.

newMethod := CompiledMethod newBytes: ... trailerBytes: ***trailer***
nArgs: ... nTemps: .. nStack: ... nLits: ... primitive: ...




--
Best regards,
Igor Stasenko AKA sig.

Hi toon

this is a good news. I would really like to see Opal taking off the ground.

Stef

On Mar 27, 2011, at 7:14 PM, Toon Verwaest wrote:

That's the plan yes. Niko is working on that part now. I guess for now
we might keep it on Eliot's proposal of just transforming to an
undefined field access; but we'll see.
One of the ideas is of course to collect the methods that get affected
by a change and report back to the user; just like with all other types
of errors that can occur during class modification. We have a pretty
cool model already so it's becoming increasingly easy to install such
hooks. I hate self error: 'your class is invalid because of something
somewhere', and we'll get that all fixed!
> If you create a copy of method, copy the method's trailer.
>
>
> trailer := oldMethod trailer.
>
> newMethod := CompiledMethod newBytes: ... trailerBytes: ***trailer***
> nArgs: ... nTemps: .. nStack: ... nLits: ... primitive: ...
It seems that the opal compiler has a compiledMethodWith: trailer. Useful ;)
I just had to set the methodClass: and selector: myself. Fair enough.

@Eliot thanks for the hints. I'll take it into account to make the
recompilation complete. If Niko doesn't finish it this evening I'll
probably do it tomorrow... I'm in the need of a small break :)

Ignoring the fact that it doesn't support removing instance variables
yet, it would very well already. Pretty cool to see methods updated on
bytecode level without slow recompilation. Aaah... rewriting the
classbuilder is starting to pay off :) And then on to stateful traits.

Just for credits: I'm working together with Camillo Bruni and Niko
Schwarz on this.

cheers,
Toon

>>
>>
>> (Unless, of course you simply raise an error during (re)compilation
>> when given ivar removed from class).
> That's the plan yes. Niko is working on that part now. I guess for now we might keep it on Eliot's proposal of just transforming to an undefined field access; but we'll see.
> One of the ideas is of course to collect the methods that get affected by a change and report back to the user; just like with all other types of errors that can occur during class modification. We have a pretty cool model already so it's becoming increasingly easy to install such hooks. I hate self error: 'your class is invalid because of something somewhere', and we'll get that all fixed!

I would love to have real object that contains compilation error (instead of Transcript show:) so that we can build tool to fix them :)

2011/3/28 Stéphane Ducasse <[hidden email]>:
>>>
>>>
>>> (Unless, of course you simply raise an error during (re)compilation
>>> when given ivar removed from class).
>> That's the plan yes. Niko is working on that part now. I guess for now we might keep it on Eliot's proposal of just transforming to an undefined field access; but we'll see.
>> One of the ideas is of course to collect the methods that get affected by a change and report back to the user; just like with all other types of errors that can occur during class modification. We have a pretty cool model already so it's becoming increasingly easy to install such hooks. I hate self error: 'your class is invalid because of something somewhere', and we'll get that all fixed!
>
> I would love to have real object that contains compilation error (instead of Transcript show:) so that we can build tool to fix them :)
>

Various errors can occur at different stages, hence be generated from
different classes in current (old) Compiler:
- errors at parsing (syntax errors)
- errors at name resolving (shadowing, undeclared ...)
- errors at code generation (byte code limits ...)

Several alternatives APi are possible for handling these errors:
- Compiler signal all warning/errors by raising Exceptions,
  Compiler invocations are wrapped with a handler
- Compiler signal all warnings/errors by sending a message to a
reified errorHandler inst. var.
  The errorHandler can be instanciated with different classes for
interactive  / non interactive
  The errorHandler knows itself how to interact with user thru
TextEditor, menus, or transcript...

There is already a mixture of these strategies in current (old) Compiler.
Except that compiler talks directly to a requestor and tries to handle
interaction by itself if interactive.

IMHO, it's better to write all your requirements before thinking of an
architecture.

Nicolas

To keep you up to date... I've already implemented bytecode rewriting
that inserts an error.

Basically when you have something like

instVarAt: 1

where 1 is now removed, this gets translated to:

pushConstant: (RemovedField new name: #old name)
send: invalidRead
pop
pushConstant: nil

and at:put: to

pushConstant: (RemovedField new originalSlot: oldSlotObject)
send: invalidWrite
pop

Then these messages will at runtime give you an error. Also since my
MethodFieldUpdater sees these methods as it is rewriting it, it can
collect them (doesn't do it yet though.. but that's 2 minutes of work)
and present them to the user.

Since we insert these magic constant object recompile time, the
decompiler can find them again when recompiling for the next class
update. When it finds them, it has all the meta-data it needs to figure
out what slot access it was. Now that the method is being recompiled it
can find out if there's a new slot that matches the old name; and put
the corresponding field-access back into the method.
At the moment the compiler already finds the slot accesses but doesn't
yet rewrite the IR back to a field-access. But that's semi-easy too. The
IR just doesn't provide an API yet for finding objects around a certain
IR node and removing them ... so I got bored :)

So in short, we have all the infrastructure to provide good runtime
errors, to collect methods at compiletime; and to reintroduce slots
after they were previously removed and reattach the accesses properly.
And it's all done outside of the decompiler using our classbuilder +
classmodification + field modification + method & instance modification
maps.

And so yes, I also want to have proper meta-info everywhere; proper
objects everywhere; so that we can build proper tooling to solve it. The
new classbuilder is fully built around this idea. It builds up
modification maps exactly so that it can find conflicts early, and
present them to the developer for fixing; BEFORE applying the
modification to the classes.

Once this fully works I can post an image somewhere that can give you a
sneak preview. At the moment it's still a bit intertwined with Helvetia
for the slot-related experiment; but that we'll extract after the paper
is written so that we can submit a clean model to Pharo, ... if Pharo
cares ;)

cheers,
Toon

On 03/28/2011 09:20 PM, Stéphane Ducasse wrote:

On Mar 29, 2011, at 12:24 AM, Toon Verwaest wrote:

oh yes it does...
we all want that :).
First class slots is a dream.