Fwd: More progress...

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Fwd: More progress...

Marcus Denker-4
… maybe others are interested. 

Begin forwarded message:

From: Marcus Denker <[hidden email]>
Subject: More progress...
Date: January 28, 2013 1:07:19 PM GMT+01:00

Hi,

Lots of progress…

-> BC to Text mapping (highlighting in the debugger)
    Instead of building the mapping in the code generator, we instead
            enhanced the AST and IR to have all the right information and needed
            mapping.

    This means: we can query the AST for "which statement generated the byte code at offset 24?"
    Or "which BlockNode is the one for this Context?".

    The AST combined with the  Scope/Var annotations and the IR contains all information needed,
            and in cases where something is wrong it is easy to debug.

-> Correct Temp Vars for Contexts (via the Scope/Var knowledge)
   For accessing temps in blocks in the debugger.
   Here we delegate queries to the annotated AST, this means the Debugger uses
           the same data structures as the compiler when it wants to know about e.g. local temps.

One of the nice effects is that the needed machinery is much simpler:

DebuggerMethodMap linesOfCode 164
DebuggerMethodMapOpal linesOfCode 62

it is actually so simple that in a second step, there is no need for DebuggerMethodMapOpal at all,
       the code can be moved directly to better places (e.g. MethodContext). Caching is handled by ASTCache.

-> Error Handling is working. All the error handling (Parsing and Semantic) are handled the same as
    the old compiler.

TODO
=> Pass over integration. Clean up the Compiler/Parser facades.
     (We will first have the whole thing to be compatible to the old from the Compiler/Parser POV,
              here more cleanups are following after we have the stuff in the image)

=> Pass over the decompiler. BC->IR is working, IR->AST needs to be checked.

=> Fix all showstoppers here: http://code.google.com/p/pharo-newcompiler/issues/list


Marcus and Clement


Reply | Threaded
Open this post in threaded view
|

Re: Fwd: More progress...

Sven Van Caekenberghe-2
Yes !

Looking forward to it…

On 28 Jan 2013, at 13:09, Marcus Denker <[hidden email]> wrote:

> … maybe others are interested.
>
> Begin forwarded message:
>
>> From: Marcus Denker <[hidden email]>
>> Subject: More progress...
>> Date: January 28, 2013 1:07:19 PM GMT+01:00
>> To: "[hidden email]" <[hidden email]>
>>
>> Hi,
>>
>> Lots of progress…
>>
>> -> BC to Text mapping (highlighting in the debugger)
>>     Instead of building the mapping in the code generator, we instead
>>             enhanced the AST and IR to have all the right information and needed
>>             mapping.
>>
>>     This means: we can query the AST for "which statement generated the byte code at offset 24?"
>>     Or "which BlockNode is the one for this Context?".
>>
>>     The AST combined with the  Scope/Var annotations and the IR contains all information needed,
>>             and in cases where something is wrong it is easy to debug.
>>
>> -> Correct Temp Vars for Contexts (via the Scope/Var knowledge)
>>    For accessing temps in blocks in the debugger.
>>    Here we delegate queries to the annotated AST, this means the Debugger uses
>>            the same data structures as the compiler when it wants to know about e.g. local temps.
>>
>> One of the nice effects is that the needed machinery is much simpler:
>>
>> DebuggerMethodMap linesOfCode 164
>> DebuggerMethodMapOpal linesOfCode 62
>>
>> it is actually so simple that in a second step, there is no need for DebuggerMethodMapOpal at all,
>>        the code can be moved directly to better places (e.g. MethodContext). Caching is handled by ASTCache.
>>
>> -> Error Handling is working. All the error handling (Parsing and Semantic) are handled the same as
>>     the old compiler.
>>
>> TODO
>> => Pass over integration. Clean up the Compiler/Parser facades.
>>      (We will first have the whole thing to be compatible to the old from the Compiler/Parser POV,
>>               here more cleanups are following after we have the stuff in the image)
>>
>> => Pass over the decompiler. BC->IR is working, IR->AST needs to be checked.
>>
>> => Fix all showstoppers here: http://code.google.com/p/pharo-newcompiler/issues/list
>>
>>
>> Marcus and Clement
>>
>


Reply | Threaded
Open this post in threaded view
|

Re: Fwd: More progress...

Fernando olivero-2
In reply to this post by Marcus Denker-4
Hi Marcus, this are great news!

For the less-knowledgable in OPAL, were can we read more about it?

Fernando


On Mon, Jan 28, 2013 at 1:15 PM, Sven Van Caekenberghe <[hidden email]> wrote:
Yes !

Looking forward to it…

On 28 Jan 2013, at 13:09, Marcus Denker <[hidden email]> wrote:

> … maybe others are interested.
>
> Begin forwarded message:
>
>> From: Marcus Denker <[hidden email]>
>> Subject: More progress...
>> Date: January 28, 2013 1:07:19 PM GMT+01:00
>> To: "[hidden email]" <[hidden email]>
>>
>> Hi,
>>
>> Lots of progress…
>>
>> -> BC to Text mapping (highlighting in the debugger)
>>           Instead of building the mapping in the code generator, we instead
>>             enhanced the AST and IR to have all the right information and needed
>>             mapping.
>>
>>           This means: we can query the AST for "which statement generated the byte code at offset 24?"
>>           Or "which BlockNode is the one for this Context?".
>>
>>           The AST combined with the  Scope/Var annotations and the IR contains all information needed,
>>             and in cases where something is wrong it is easy to debug.
>>
>> -> Correct Temp Vars for Contexts (via the Scope/Var knowledge)
>>          For accessing temps in blocks in the debugger.
>>          Here we delegate queries to the annotated AST, this means the Debugger uses
>>            the same data structures as the compiler when it wants to know about e.g. local temps.
>>
>>      One of the nice effects is that the needed machinery is much simpler:
>>
>>      DebuggerMethodMap linesOfCode 164
>>      DebuggerMethodMapOpal linesOfCode 62
>>
>>      it is actually so simple that in a second step, there is no need for DebuggerMethodMapOpal at all,
>>        the code can be moved directly to better places (e.g. MethodContext). Caching is handled by ASTCache.
>>
>> -> Error Handling is working. All the error handling (Parsing and Semantic) are handled the same as
>>     the old compiler.
>>
>> TODO
>>      => Pass over integration. Clean up the Compiler/Parser facades.
>>            (We will first have the whole thing to be compatible to the old from the Compiler/Parser POV,
>>               here more cleanups are following after we have the stuff in the image)
>>
>>      => Pass over the decompiler. BC->IR is working, IR->AST needs to be checked.
>>
>>      => Fix all showstoppers here: http://code.google.com/p/pharo-newcompiler/issues/list
>>
>>
>>      Marcus and Clement
>>
>



Reply | Threaded
Open this post in threaded view
|

Re: Fwd: More progress...

Nicolas Cellier
In reply to this post by Marcus Denker-4
But isn't BC->IR a trivial 1 to 1 mapping?
IR->AST is the difficult part because of inlining...
But in most cases this should be un-necessary no?

Indeed, if you get the source code + byte code.
then you can just check integrity like this:
    1) compile source->AST->(IR1->)BC1
    2) check BC1=BC
          ifTrue: [proceed with AST]
          ifFalse: [invoke decompiler BC->IR->AST]

Nicolas

2013/1/28 Marcus Denker <[hidden email]>:

> … maybe others are interested.
>
> Begin forwarded message:
>
> From: Marcus Denker <[hidden email]>
> Subject: More progress...
> Date: January 28, 2013 1:07:19 PM GMT+01:00
> To: "[hidden email]"
> <[hidden email]>
>
> Hi,
>
> Lots of progress…
>
> -> BC to Text mapping (highlighting in the debugger)
>     Instead of building the mapping in the code generator, we instead
>             enhanced the AST and IR to have all the right information and
> needed
>             mapping.
>
>     This means: we can query the AST for "which statement generated the byte
> code at offset 24?"
>     Or "which BlockNode is the one for this Context?".
>
>     The AST combined with the  Scope/Var annotations and the IR contains all
> information needed,
>             and in cases where something is wrong it is easy to debug.
>
> -> Correct Temp Vars for Contexts (via the Scope/Var knowledge)
>    For accessing temps in blocks in the debugger.
>    Here we delegate queries to the annotated AST, this means the Debugger
> uses
>            the same data structures as the compiler when it wants to know
> about e.g. local temps.
>
> One of the nice effects is that the needed machinery is much simpler:
>
> DebuggerMethodMap linesOfCode 164
> DebuggerMethodMapOpal linesOfCode 62
>
> it is actually so simple that in a second step, there is no need for
> DebuggerMethodMapOpal at all,
>        the code can be moved directly to better places (e.g. MethodContext).
> Caching is handled by ASTCache.
>
> -> Error Handling is working. All the error handling (Parsing and Semantic)
> are handled the same as
>     the old compiler.
>
> TODO
> => Pass over integration. Clean up the Compiler/Parser facades.
>      (We will first have the whole thing to be compatible to the old from
> the Compiler/Parser POV,
>               here more cleanups are following after we have the stuff in
> the image)
>
> => Pass over the decompiler. BC->IR is working, IR->AST needs to be checked.
>
> => Fix all showstoppers here:
> http://code.google.com/p/pharo-newcompiler/issues/list
>
>
> Marcus and Clement
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Fwd: More progress...

Clément Béra
Hello,

In fact IR->AST is useless but if you don't have the pharo sources aside from your vm.

You right, BC -> IR is kind of a trivial 1 to 1 mapping. But even if it is trivial, it had to be done :)

Adding to integration and decompiler, I think the doIt compilation might not be finished too. It is still on the pharo compiler bug tracker.

2013/1/28 Nicolas Cellier <[hidden email]>
But isn't BC->IR a trivial 1 to 1 mapping?
IR->AST is the difficult part because of inlining...
But in most cases this should be un-necessary no?

Indeed, if you get the source code + byte code.
then you can just check integrity like this:
    1) compile source->AST->(IR1->)BC1
    2) check BC1=BC
          ifTrue: [proceed with AST]
          ifFalse: [invoke decompiler BC->IR->AST]

Nicolas

2013/1/28 Marcus Denker <[hidden email]>:
> … maybe others are interested.
>
> Begin forwarded message:
>
> From: Marcus Denker <[hidden email]>
> Subject: More progress...
> Date: January 28, 2013 1:07:19 PM GMT+01:00
> To: "[hidden email]"
> <[hidden email]>
>
> Hi,
>
> Lots of progress…
>
> -> BC to Text mapping (highlighting in the debugger)
>     Instead of building the mapping in the code generator, we instead
>             enhanced the AST and IR to have all the right information and
> needed
>             mapping.
>
>     This means: we can query the AST for "which statement generated the byte
> code at offset 24?"
>     Or "which BlockNode is the one for this Context?".
>
>     The AST combined with the  Scope/Var annotations and the IR contains all
> information needed,
>             and in cases where something is wrong it is easy to debug.
>
> -> Correct Temp Vars for Contexts (via the Scope/Var knowledge)
>    For accessing temps in blocks in the debugger.
>    Here we delegate queries to the annotated AST, this means the Debugger
> uses
>            the same data structures as the compiler when it wants to know
> about e.g. local temps.
>
> One of the nice effects is that the needed machinery is much simpler:
>
> DebuggerMethodMap linesOfCode 164
> DebuggerMethodMapOpal linesOfCode 62
>
> it is actually so simple that in a second step, there is no need for
> DebuggerMethodMapOpal at all,
>        the code can be moved directly to better places (e.g. MethodContext).
> Caching is handled by ASTCache.
>
> -> Error Handling is working. All the error handling (Parsing and Semantic)
> are handled the same as
>     the old compiler.
>
> TODO
> => Pass over integration. Clean up the Compiler/Parser facades.
>      (We will first have the whole thing to be compatible to the old from
> the Compiler/Parser POV,
>               here more cleanups are following after we have the stuff in
> the image)
>
> => Pass over the decompiler. BC->IR is working, IR->AST needs to be checked.
>
> => Fix all showstoppers here:
> http://code.google.com/p/pharo-newcompiler/issues/list
>
>
> Marcus and Clement
>
>


Reply | Threaded
Open this post in threaded view
|

Re: More progress...

Marcus Denker-4

On Jan 28, 2013, at 2:53 PM, Clément Bera <[hidden email]> wrote:

> Hello,
>
> In fact IR->AST is useless but if you don't have the pharo sources aside from your vm.
>

I personally would like to not decompile at all… we don't have a X86-to-bytecode
decompiler either.

Instead of decompiling, we should store a high-level representation of the method
and use that.

We will do that eventually, but people always freak out when I explain that so
for now we will make the compiler a drop-in replacement.

        Marcus