BLOG: Block Translators - parsing magic

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
41 messages Options
123
Reply | Threaded
Open this post in threaded view
|

BLOG: Block Translators - parsing magic

Udo Schneider
All,

I just finished a blog entry. It shows how to use Smalltalk blocks as
parsers/translators. E.g. translating a Block

        [:customer | (customer joinDate year is: Date today year)]

into an SQL-like String

        (YEAR(customers.joinDate) = 2014)

The SQL stuff is just an example - you can create nearly any output.

Check out
http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html

Maybe that's old stuff for some of you - but I hope it's interesting for
some at least :-)

Comments and feedback appreciated.

CU,

Udo


Reply | Threaded
Open this post in threaded view
|

Re: BLOG: Block Translators - parsing magic

Ben Coman
Udo Schneider wrote:
All,

I just finished a blog entry. It shows how to use Smalltalk blocks as parsers/translators. E.g. translating a Block

    [:customer | (customer joinDate year is: Date today year)]

into an SQL-like String

    (YEAR(customers.joinDate) = 2014)

The SQL stuff is just an example - you can create nearly any output.

Check out http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html

Maybe that's old stuff for some of you - but I hope it's interesting for some at least :-)

Comments and feedback appreciated.

CU,

Udo

Very nice write up.  Fairly straight forward to follow but not something I could have done from scratch.  Nice to have a recipe for it. Thanks

One spelling mistake...
> This will give es enough clues
This will give us enough clues

cheers -ben
Reply | Threaded
Open this post in threaded view
|

Re: BLOG: Block Translators - parsing magic

Esteban A. Maringolo
In reply to this post by Udo Schneider
Excellent article.

I think GLORP uses a similar technique to setup its expressions, and
also have issues with #and:/#or: selectors due to inlining, so it uses
AND:/#OR: instead.

Regards!

Esteban A. Maringolo

pd: Your blog and it's choosen topic made me remember
http://use-the-index-luke.com/

2014-09-22 20:48 GMT-03:00 Udo Schneider <[hidden email]>:

>
> All,
>
> I just finished a blog entry. It shows how to use Smalltalk blocks as parsers/translators. E.g. translating a Block
>
>         [:customer | (customer joinDate year is: Date today year)]
>
> into an SQL-like String
>
>         (YEAR(customers.joinDate) = 2014)
>
> The SQL stuff is just an example - you can create nearly any output.
>
> Check out http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html
>
> Maybe that's old stuff for some of you - but I hope it's interesting for some at least :-)
>
> Comments and feedback appreciated.
>
> CU,
>
> Udo
>
>

Reply | Threaded
Open this post in threaded view
|

Re: BLOG: Block Translators - parsing magic

Udo Schneider
In reply to this post by Ben Coman


On 23.09.14 02:27, Ben Coman wrote:
> Very nice write up.  Fairly straight forward to follow but not something
> I could have done from scratch.  Nice to have a recipe for it. Thanks
>
> One spelling mistake...
>  > This will give es enough clues
> This will give us enough clues
Thanks for the feedback - the spelling mistake is fixed.

Thanks,

Udo



Reply | Threaded
Open this post in threaded view
|

Re: BLOG: Block Translators - parsing magic

Udo Schneider
In reply to this post by Esteban A. Maringolo
Hi Estaban,

I think the first time I saw this pattern was in ReStore on Dolphin
Smalltalk. I didn't understand it's implementation back then. I assume
that it's similar to what I described though. But having a Smalltalk
block automagically creating the equivalent SQL SELECT expression was
like black magic at that time :-)

CU,

Udo



On 23.09.14 04:15, Esteban A. Maringolo wrote:

> Excellent article.
>
> I think GLORP uses a similar technique to setup its expressions, and
> also have issues with #and:/#or: selectors due to inlining, so it uses
> AND:/#OR: instead.
>
> Regards!
>
> Esteban A. Maringolo
>
> pd: Your blog and it's choosen topic made me remember
> http://use-the-index-luke.com/
>
> 2014-09-22 20:48 GMT-03:00 Udo Schneider <[hidden email]>:
>>
>> All,
>>
>> I just finished a blog entry. It shows how to use Smalltalk blocks as parsers/translators. E.g. translating a Block
>>
>>          [:customer | (customer joinDate year is: Date today year)]
>>
>> into an SQL-like String
>>
>>          (YEAR(customers.joinDate) = 2014)
>>
>> The SQL stuff is just an example - you can create nearly any output.
>>
>> Check out http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html
>>
>> Maybe that's old stuff for some of you - but I hope it's interesting for some at least :-)
>>
>> Comments and feedback appreciated.
>>
>> CU,
>>
>> Udo
>>
>>
>
>



Reply | Threaded
Open this post in threaded view
|

Re: BLOG: Block Translators - parsing magic

Thierry Goubier
In reply to this post by Udo Schneider
Thanks Udo;

I reviewed a few techniques for implementing internal / embeddel DSLs in a host language and I didn't saw this one :)

Thierry

2014-09-23 1:48 GMT+02:00 Udo Schneider <[hidden email]>:
All,

I just finished a blog entry. It shows how to use Smalltalk blocks as parsers/translators. E.g. translating a Block

        [:customer | (customer joinDate year is: Date today year)]

into an SQL-like String

        (YEAR(customers.joinDate) = 2014)

The SQL stuff is just an example - you can create nearly any output.

Check out http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html

Maybe that's old stuff for some of you - but I hope it's interesting for some at least :-)

Comments and feedback appreciated.

CU,

Udo



Reply | Threaded
Open this post in threaded view
|

Re: BLOG: Block Translators - parsing magic

kilon.alios
In reply to this post by Udo Schneider
just as it is black magic for me now :D

At least I get the general feeling. I am new to parsing too, so far I have only played with regex parsing. Not the most smalltalkish way but it works well so far.

On Tue, Sep 23, 2014 at 9:39 AM, Udo Schneider <[hidden email]> wrote:
Hi Estaban,

I think the first time I saw this pattern was in ReStore on Dolphin Smalltalk. I didn't understand it's implementation back then. I assume that it's similar to what I described though. But having a Smalltalk block automagically creating the equivalent SQL SELECT expression was like black magic at that time :-)

CU,

Udo




On 23.09.14 04:15, Esteban A. Maringolo wrote:
Excellent article.

I think GLORP uses a similar technique to setup its expressions, and
also have issues with #and:/#or: selectors due to inlining, so it uses
AND:/#OR: instead.

Regards!

Esteban A. Maringolo

pd: Your blog and it's choosen topic made me remember
http://use-the-index-luke.com/

2014-09-22 20:48 GMT-03:00 Udo Schneider <[hidden email]>:

All,

I just finished a blog entry. It shows how to use Smalltalk blocks as parsers/translators. E.g. translating a Block

         [:customer | (customer joinDate year is: Date today year)]

into an SQL-like String

         (YEAR(customers.joinDate) = 2014)

The SQL stuff is just an example - you can create nearly any output.

Check out http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html

Maybe that's old stuff for some of you - but I hope it's interesting for some at least :-)

Comments and feedback appreciated.

CU,

Udo








Reply | Threaded
Open this post in threaded view
|

Re: BLOG: Block Translators - parsing magic

Udo Schneider
 > just as it is black magic for me now :D
The nice thing about this approach is the fact that it "just" piggybacks
the normal Smalltalk message sending. So you can step through it using
the Debugger - it's Smalltalk all the way down.

I still remember my first shock when (having no formal background on
parsing theory at that time) I saw the parsing tables generated by
T-Gen. Of course it was Smalltalk ... but understanding those tables was
a nightmare.

PetitParser is the gold standard here IMHO. It is able to parse
arbitrary input (compared to my block expressions only). And you can
still use the Debugger to step through the parsing process *and still
understand what's going on!*

CU,

Udo

On 23.09.14 09:54, kilon alios wrote:

> just as it is black magic for me now :D
>
> At least I get the general feeling. I am new to parsing too, so far I
> have only played with regex parsing. Not the most smalltalkish way but
> it works well so far.
>
> On Tue, Sep 23, 2014 at 9:39 AM, Udo Schneider
> <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi Estaban,
>
>     I think the first time I saw this pattern was in ReStore on Dolphin
>     Smalltalk. I didn't understand it's implementation back then. I
>     assume that it's similar to what I described though. But having a
>     Smalltalk block automagically creating the equivalent SQL SELECT
>     expression was like black magic at that time :-)
>
>     CU,
>
>     Udo
>
>
>
>
>     On 23.09.14 04:15, Esteban A. Maringolo wrote:
>
>         Excellent article.
>
>         I think GLORP uses a similar technique to setup its expressions, and
>         also have issues with #and:/#or: selectors due to inlining, so
>         it uses
>         AND:/#OR: instead.
>
>         Regards!
>
>         Esteban A. Maringolo
>
>         pd: Your blog and it's choosen topic made me remember
>         http://use-the-index-luke.com/
>
>         2014-09-22 20:48 GMT-03:00 Udo Schneider
>         <[hidden email]
>         <mailto:[hidden email]>>__:
>
>
>             All,
>
>             I just finished a blog entry. It shows how to use Smalltalk
>             blocks as parsers/translators. E.g. translating a Block
>
>                       [:customer | (customer joinDate year is: Date
>             today year)]
>
>             into an SQL-like String
>
>                       (YEAR(customers.joinDate) = 2014)
>
>             The SQL stuff is just an example - you can create nearly any
>             output.
>
>             Check out
>             http://readthesourceluke.__blogspot.de/2014/09/block-__translators-parsing-magic.html
>             <http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html>
>
>             Maybe that's old stuff for some of you - but I hope it's
>             interesting for some at least :-)
>
>             Comments and feedback appreciated.
>
>             CU,
>
>             Udo
>
>
>
>
>
>
>
>



Reply | Threaded
Open this post in threaded view
|

Re: BLOG: Block Translators - parsing magic

kilon.alios
it reminds a lot of Kent's Beck Smalltalk Practice Patterns where it removes all ifs and replaces them with regular unary messages . It is definitely an elegant way of coding making the code just flow.

I have not used PettitParser yet, looks powerful but I find it a bit weird in design. On the other hand regex is quite ugly and understanding complex regex a pain.  

On Tue, Sep 23, 2014 at 11:07 AM, Udo Schneider <[hidden email]> wrote:
> just as it is black magic for me now :D
The nice thing about this approach is the fact that it "just" piggybacks the normal Smalltalk message sending. So you can step through it using the Debugger - it's Smalltalk all the way down.

I still remember my first shock when (having no formal background on parsing theory at that time) I saw the parsing tables generated by T-Gen. Of course it was Smalltalk ... but understanding those tables was a nightmare.

PetitParser is the gold standard here IMHO. It is able to parse arbitrary input (compared to my block expressions only). And you can still use the Debugger to step through the parsing process *and still understand what's going on!*

CU,

Udo

On 23.09.14 09:54, kilon alios wrote:
just as it is black magic for me now :D

At least I get the general feeling. I am new to parsing too, so far I
have only played with regex parsing. Not the most smalltalkish way but
it works well so far.

On Tue, Sep 23, 2014 at 9:39 AM, Udo Schneider
<[hidden email]
<mailto:[hidden email]>> wrote:

    Hi Estaban,

    I think the first time I saw this pattern was in ReStore on Dolphin
    Smalltalk. I didn't understand it's implementation back then. I
    assume that it's similar to what I described though. But having a
    Smalltalk block automagically creating the equivalent SQL SELECT
    expression was like black magic at that time :-)

    CU,

    Udo




    On 23.09.14 04:15, Esteban A. Maringolo wrote:

        Excellent article.

        I think GLORP uses a similar technique to setup its expressions, and
        also have issues with #and:/#or: selectors due to inlining, so
        it uses
        AND:/#OR: instead.

        Regards!

        Esteban A. Maringolo

        pd: Your blog and it's choosen topic made me remember
        http://use-the-index-luke.com/

        2014-09-22 20:48 GMT-03:00 Udo Schneider
        <[hidden email]
        <mailto:[hidden email]>>__:


            All,

            I just finished a blog entry. It shows how to use Smalltalk
            blocks as parsers/translators. E.g. translating a Block

                      [:customer | (customer joinDate year is: Date
            today year)]

            into an SQL-like String

                      (YEAR(customers.joinDate) = 2014)

            The SQL stuff is just an example - you can create nearly any
            output.

            Check out
            http://readthesourceluke.__blogspot.de/2014/09/block-__translators-parsing-magic.html
            <http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html>

            Maybe that's old stuff for some of you - but I hope it's
            interesting for some at least :-)

            Comments and feedback appreciated.

            CU,

            Udo












Reply | Threaded
Open this post in threaded view
|

Re: BLOG: Block Translators - parsing magic

S Krish

MarkupBuilder in Groovy uses similar logic #doesNotUnderstand: ..  but not debug capable ..!..

On Tue, Sep 23, 2014 at 1:45 PM, kilon alios <[hidden email]> wrote:
it reminds a lot of Kent's Beck Smalltalk Practice Patterns where it removes all ifs and replaces them with regular unary messages . It is definitely an elegant way of coding making the code just flow.

I have not used PettitParser yet, looks powerful but I find it a bit weird in design. On the other hand regex is quite ugly and understanding complex regex a pain.  

On Tue, Sep 23, 2014 at 11:07 AM, Udo Schneider <[hidden email]> wrote:
> just as it is black magic for me now :D
The nice thing about this approach is the fact that it "just" piggybacks the normal Smalltalk message sending. So you can step through it using the Debugger - it's Smalltalk all the way down.

I still remember my first shock when (having no formal background on parsing theory at that time) I saw the parsing tables generated by T-Gen. Of course it was Smalltalk ... but understanding those tables was a nightmare.

PetitParser is the gold standard here IMHO. It is able to parse arbitrary input (compared to my block expressions only). And you can still use the Debugger to step through the parsing process *and still understand what's going on!*

CU,

Udo

On 23.09.14 09:54, kilon alios wrote:
just as it is black magic for me now :D

At least I get the general feeling. I am new to parsing too, so far I
have only played with regex parsing. Not the most smalltalkish way but
it works well so far.

On Tue, Sep 23, 2014 at 9:39 AM, Udo Schneider
<[hidden email]
<mailto:[hidden email]>> wrote:

    Hi Estaban,

    I think the first time I saw this pattern was in ReStore on Dolphin
    Smalltalk. I didn't understand it's implementation back then. I
    assume that it's similar to what I described though. But having a
    Smalltalk block automagically creating the equivalent SQL SELECT
    expression was like black magic at that time :-)

    CU,

    Udo




    On 23.09.14 04:15, Esteban A. Maringolo wrote:

        Excellent article.

        I think GLORP uses a similar technique to setup its expressions, and
        also have issues with #and:/#or: selectors due to inlining, so
        it uses
        AND:/#OR: instead.

        Regards!

        Esteban A. Maringolo

        pd: Your blog and it's choosen topic made me remember
        http://use-the-index-luke.com/

        2014-09-22 20:48 GMT-03:00 Udo Schneider
        <[hidden email]
        <mailto:[hidden email]>>__:


            All,

            I just finished a blog entry. It shows how to use Smalltalk
            blocks as parsers/translators. E.g. translating a Block

                      [:customer | (customer joinDate year is: Date
            today year)]

            into an SQL-like String

                      (YEAR(customers.joinDate) = 2014)

            The SQL stuff is just an example - you can create nearly any
            output.

            Check out
            http://readthesourceluke.__blogspot.de/2014/09/block-__translators-parsing-magic.html
            <http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html>

            Maybe that's old stuff for some of you - but I hope it's
            interesting for some at least :-)

            Comments and feedback appreciated.

            CU,

            Udo













Reply | Threaded
Open this post in threaded view
|

Re: BLOG: Block Translators - parsing magic

philippeback
In reply to this post by Udo Schneider
Cool article & technique indeed. Ah Smalltalk, where were you all those years ;-)

Speaking of PetitParser, which is excellent indeed, there is this #==> method in Boolean.

PetitParser uses that a lot. I can use the thing but do not really grasps how it works.

Now, the method comment says:

Boolean #==> aBlock
"The material conditional, also known as the material implication or truth functional conditional. Correspond to not ... or ... and does not correspond to the English if...then... construction.
known as:
b if a 
a implies b
if a then b
b is a consequence of a
a therefore b (but note: 'it is raining therefore it is cloudy' is implication; 'it is autumn therefore the leaves are falling' is equivalence).
Here is the truth table for material implication:
  p   |   q   |   p ==> q
-------|-------|-------------
  T   |   T   |      T
  T   |   F   |      F
  F   |   T   |      T
  F   |   F   |      T
"

^self not or: [aBlock value]


This is still a bit foggy to me.

Anyone caring to shed some light on that?
What are the typical ways to use that?

TIA
Phil




On Tue, Sep 23, 2014 at 10:07 AM, Udo Schneider <[hidden email]> wrote:
> just as it is black magic for me now :D
The nice thing about this approach is the fact that it "just" piggybacks the normal Smalltalk message sending. So you can step through it using the Debugger - it's Smalltalk all the way down.

I still remember my first shock when (having no formal background on parsing theory at that time) I saw the parsing tables generated by T-Gen. Of course it was Smalltalk ... but understanding those tables was a nightmare.

PetitParser is the gold standard here IMHO. It is able to parse arbitrary input (compared to my block expressions only). And you can still use the Debugger to step through the parsing process *and still understand what's going on!*

CU,

Udo

On 23.09.14 09:54, kilon alios wrote:
just as it is black magic for me now :D

At least I get the general feeling. I am new to parsing too, so far I
have only played with regex parsing. Not the most smalltalkish way but
it works well so far.

On Tue, Sep 23, 2014 at 9:39 AM, Udo Schneider
<[hidden email]
<mailto:[hidden email]>> wrote:

    Hi Estaban,

    I think the first time I saw this pattern was in ReStore on Dolphin
    Smalltalk. I didn't understand it's implementation back then. I
    assume that it's similar to what I described though. But having a
    Smalltalk block automagically creating the equivalent SQL SELECT
    expression was like black magic at that time :-)

    CU,

    Udo




    On 23.09.14 04:15, Esteban A. Maringolo wrote:

        Excellent article.

        I think GLORP uses a similar technique to setup its expressions, and
        also have issues with #and:/#or: selectors due to inlining, so
        it uses
        AND:/#OR: instead.

        Regards!

        Esteban A. Maringolo

        pd: Your blog and it's choosen topic made me remember
        http://use-the-index-luke.com/

        2014-09-22 20:48 GMT-03:00 Udo Schneider
        <[hidden email]
        <mailto:[hidden email]>>__:


            All,

            I just finished a blog entry. It shows how to use Smalltalk
            blocks as parsers/translators. E.g. translating a Block

                      [:customer | (customer joinDate year is: Date
            today year)]

            into an SQL-like String

                      (YEAR(customers.joinDate) = 2014)

            The SQL stuff is just an example - you can create nearly any
            output.

            Check out
            http://readthesourceluke.__blogspot.de/2014/09/block-__translators-parsing-magic.html
            <http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html>

            Maybe that's old stuff for some of you - but I hope it's
            interesting for some at least :-)

            Comments and feedback appreciated.

            CU,

            Udo













Reply | Threaded
Open this post in threaded view
|

Re: BLOG: Block Translators - parsing magic

philippeback
In reply to this post by kilon.alios
On Tue, Sep 23, 2014 at 10:15 AM, kilon alios <[hidden email]> wrote:
it reminds a lot of Kent's Beck Smalltalk Practice Patterns where it removes all ifs and replaces them with regular unary messages . It is definitely an elegant way of coding making the code just flow.

I have not used PettitParser yet, looks powerful but I find it a bit weird in design. On the other hand regex is quite ugly and understanding complex regex a pain.  

Well, I started writing Regexes for parsing some flat file content.
Quickly turned into a harder than needed way.
Moved to PP and things are now crystal clear. Give it a shot, it is really powerful and easy to write once you get the hang of it.

Phil 

On Tue, Sep 23, 2014 at 11:07 AM, Udo Schneider <[hidden email]> wrote:
> just as it is black magic for me now :D
The nice thing about this approach is the fact that it "just" piggybacks the normal Smalltalk message sending. So you can step through it using the Debugger - it's Smalltalk all the way down.

I still remember my first shock when (having no formal background on parsing theory at that time) I saw the parsing tables generated by T-Gen. Of course it was Smalltalk ... but understanding those tables was a nightmare.

PetitParser is the gold standard here IMHO. It is able to parse arbitrary input (compared to my block expressions only). And you can still use the Debugger to step through the parsing process *and still understand what's going on!*

CU,

Udo

On 23.09.14 09:54, kilon alios wrote:
just as it is black magic for me now :D

At least I get the general feeling. I am new to parsing too, so far I
have only played with regex parsing. Not the most smalltalkish way but
it works well so far.

On Tue, Sep 23, 2014 at 9:39 AM, Udo Schneider
<[hidden email]
<mailto:[hidden email]>> wrote:

    Hi Estaban,

    I think the first time I saw this pattern was in ReStore on Dolphin
    Smalltalk. I didn't understand it's implementation back then. I
    assume that it's similar to what I described though. But having a
    Smalltalk block automagically creating the equivalent SQL SELECT
    expression was like black magic at that time :-)

    CU,

    Udo




    On 23.09.14 04:15, Esteban A. Maringolo wrote:

        Excellent article.

        I think GLORP uses a similar technique to setup its expressions, and
        also have issues with #and:/#or: selectors due to inlining, so
        it uses
        AND:/#OR: instead.

        Regards!

        Esteban A. Maringolo

        pd: Your blog and it's choosen topic made me remember
        http://use-the-index-luke.com/

        2014-09-22 20:48 GMT-03:00 Udo Schneider
        <[hidden email]
        <mailto:[hidden email]>>__:


            All,

            I just finished a blog entry. It shows how to use Smalltalk
            blocks as parsers/translators. E.g. translating a Block

                      [:customer | (customer joinDate year is: Date
            today year)]

            into an SQL-like String

                      (YEAR(customers.joinDate) = 2014)

            The SQL stuff is just an example - you can create nearly any
            output.

            Check out
            http://readthesourceluke.__blogspot.de/2014/09/block-__translators-parsing-magic.html
            <http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html>

            Maybe that's old stuff for some of you - but I hope it's
            interesting for some at least :-)

            Comments and feedback appreciated.

            CU,

            Udo













Reply | Threaded
Open this post in threaded view
|

Re: BLOG: Block Translators - parsing magic

Udo Schneider
In reply to this post by kilon.alios
 > I have not used PettitParser yet, looks powerful but I find it a bit
 > weird in design. On the other hand regex is quite ugly and understanding
 > complex regex a pain.
I normally encounter two issues with RegExps:

1) The syntax between different apps/libs/frameworks differs sligtly.
Esp. for character classes or greediness. This drove me crazy more than
one time.
2) Often enough I realize (while developing the RegExp) that I need
something more capable than a Chomsky Type 3 parser (which RegExps are
in a way). So I have to use "a bigger gun" - e.g. PetitParser.

CU,

Udo


On 23.09.14 10:15, kilon alios wrote:

> it reminds a lot of Kent's Beck Smalltalk Practice Patterns where it
> removes all ifs and replaces them with regular unary messages . It is
> definitely an elegant way of coding making the code just flow.
>
> I have not used PettitParser yet, looks powerful but I find it a bit
> weird in design. On the other hand regex is quite ugly and understanding
> complex regex a pain.
>
> On Tue, Sep 23, 2014 at 11:07 AM, Udo Schneider
> <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     > just as it is black magic for me now :D
>     The nice thing about this approach is the fact that it "just"
>     piggybacks the normal Smalltalk message sending. So you can step
>     through it using the Debugger - it's Smalltalk all the way down.
>
>     I still remember my first shock when (having no formal background on
>     parsing theory at that time) I saw the parsing tables generated by
>     T-Gen. Of course it was Smalltalk ... but understanding those tables
>     was a nightmare.
>
>     PetitParser is the gold standard here IMHO. It is able to parse
>     arbitrary input (compared to my block expressions only). And you can
>     still use the Debugger to step through the parsing process *and
>     still understand what's going on!*
>
>     CU,
>
>     Udo
>
>     On 23.09.14 09:54, kilon alios wrote:
>
>         just as it is black magic for me now :D
>
>         At least I get the general feeling. I am new to parsing too, so
>         far I
>         have only played with regex parsing. Not the most smalltalkish
>         way but
>         it works well so far.
>
>         On Tue, Sep 23, 2014 at 9:39 AM, Udo Schneider
>         <[hidden email]
>         <mailto:[hidden email]>
>         <mailto:[hidden email]
>         <mailto:[hidden email]>>>
>         wrote:
>
>              Hi Estaban,
>
>              I think the first time I saw this pattern was in ReStore on
>         Dolphin
>              Smalltalk. I didn't understand it's implementation back then. I
>              assume that it's similar to what I described though. But
>         having a
>              Smalltalk block automagically creating the equivalent SQL
>         SELECT
>              expression was like black magic at that time :-)
>
>              CU,
>
>              Udo
>
>
>
>
>              On 23.09.14 04:15, Esteban A. Maringolo wrote:
>
>                  Excellent article.
>
>                  I think GLORP uses a similar technique to setup its
>         expressions, and
>                  also have issues with #and:/#or: selectors due to
>         inlining, so
>                  it uses
>                  AND:/#OR: instead.
>
>                  Regards!
>
>                  Esteban A. Maringolo
>
>                  pd: Your blog and it's choosen topic made me remember
>         http://use-the-index-luke.com/
>
>                  2014-09-22 20:48 GMT-03:00 Udo Schneider
>                  <[hidden email]
>         <mailto:[hidden email]>
>                  <mailto:[hidden email]
>         <mailto:[hidden email]>>>__:
>
>
>                      All,
>
>                      I just finished a blog entry. It shows how to use
>         Smalltalk
>                      blocks as parsers/translators. E.g. translating a Block
>
>                                [:customer | (customer joinDate year is: Date
>                      today year)]
>
>                      into an SQL-like String
>
>                                (YEAR(customers.joinDate) = 2014)
>
>                      The SQL stuff is just an example - you can create
>         nearly any
>                      output.
>
>                      Check out
>         http://readthesourceluke.__blo__gspot.de/2014/09/block-____translators-parsing-magic.html
>         <http://blogspot.de/2014/09/block-__translators-parsing-magic.html>
>
>         <http://readthesourceluke.__blogspot.de/2014/09/block-__translators-parsing-magic.html
>         <http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html>__>
>
>                      Maybe that's old stuff for some of you - but I hope
>         it's
>                      interesting for some at least :-)
>
>                      Comments and feedback appreciated.
>
>                      CU,
>
>                      Udo
>
>
>
>
>
>
>
>
>
>
>
>



Reply | Threaded
Open this post in threaded view
|

Re: BLOG: Block Translators - parsing magic

Udo Schneider
In reply to this post by philippeback
Phil,

I'd say that's an implementation of the "Material implication"[1]
operator from Propositional calculus.

You can write it as

P -> Q

and read it as "P implies Q" or (not 100% correct) "if P (is true) then
Q (is true)".

Let's take a look at the truth table:

  P | Q | P -> Q
---+---+-------
  t | t | t
---+---+-------

inserting it into the statement above yields:

"if true then true" which is true indeed.

The funny thing starts when we look at the case(s), where P is false.
The general (verbal) rule says that if the premise (P) is false the
truth value of the conclusion doesn't matter. Hence the complete term is
true:

  P | Q | P -> Q
---+---+-------
  f | t | t
---+---+-------
  f | f | t
---+---+-------

There is one case left: What happens if the premise is true but the
conclusion is false? That's in direct violation of the defintion which
states that "if P (is true) then Q (is true)". As P is true and Q is
false the truth value of the complete term is false!

  P | Q | P -> Q
---+---+-------
  t | f | f
---+---+-------

Or to put it in different words: An implication can only be false if the
premise is true but the conclusion is false.

So we end up with this truth table:

  P | Q | P -> Q
---+---+-------
  t | t | t
---+---+-------
  f | t | t
---+---+-------
  f | f | t
---+---+-------
  t | f | f
---+---+-------

We did not take a look yet, at how to implement this with basic boolean
operators. So we need to take a look at the table and find the
expression: Let's start with the exception (fourth case). We can express
this as:

!(P & !Q) -> "The term is false if P is true and Q is false"

I added the inner term "P & !Q" to the table to make it easier to follow:


  P | Q | P -> Q | P & !Q | !(P & !Q)
---+---+--------+--------+-----------
  t | t | t      | f      | t
---+---+--------+--------+-----------
  f | t | t      | t      | f
---+---+--------+--------+-----------
  f | f | t      | f      | t
---+---+--------+--------+-----------
  t | f | f      | f      | t
---+---+--------+--------+-----------

The "last" step is to simplify the term "!(P & !Q)" using one of de
Morgan's laws:
!(A & B) == !A | !B

Using this for our term gives

!(P & !Q) == !P | !!Q == !P | Q

  P | Q | P -> Q | P & !Q | !(P & !Q) | !P | Q
---+---+--------+--------+-----------+--------
  t | t | t      | f      | t         | t
---+---+--------+--------+-----------+--------
  f | t | t      | t      | f         | f
---+---+--------+--------+-----------+--------
  f | f | t      | f      | t         | t
---+---+--------+--------+-----------+--------
  t | f | f      | f      | t         | t
---+---+--------+--------+-----------+--------

So our final term is "correct" (proof in the truth table) and is
equivalent to the smalltalk term:

!P | Q  := self not or: [aBlock value]


I have to admit though that implications in Propositional calculus
really gave me a headache. You might simply want to accept that they are
defined this way. And maybe it doesn't help ... but there are even more
strange things lurking around the corner [4] :-)

Although not immidiatly obvious all the terms in Propositional calculus
do not neccessarly have a semantic meaning in context to each other.
They are totaly independent. The only thing that counts is the truth
value of its terms. E.g. something like this is
mathematically/syntacically valid but doesn't make any sense from a
semantic point of view:

P := I am 12y old
Q := It rains

The term "P -> Q" is perfectly fine mathematically/syntacically but
doesn't mean anything in terms of semantics.


Hope this helps.

CU,

Udo


[1] http://en.wikipedia.org/wiki/Material_implication_(rule_of_inference)
[2] http://en.wikipedia.org/wiki/Propositional_calculus
[3] http://en.wikipedia.org/wiki/De_Morgan's_laws
[4] http://en.wikipedia.org/wiki/Paradoxes_of_material_implication

On 23.09.14 10:49, [hidden email] wrote:

> Cool article & technique indeed. Ah Smalltalk, where were you all those
> years ;-)
>
> Speaking of PetitParser, which is excellent indeed, there is this #==>
> method in Boolean.
>
> PetitParser uses that a lot. I can use the thing but do not really
> grasps how it works.
>
> Now, the method comment says:
>
> Boolean #==> aBlock
> "The material conditional, also known as the material implication or
> truth functional conditional.Correspond to not ... or ... and does not
> correspond to the English if...then... construction.
> known as:
> b if a
> a implies b
> if a then b
> b is a consequence of a
> a therefore b (but note: 'it is raining therefore it is cloudy' is
> implication; 'it is autumn therefore the leaves are falling' is
> equivalence).
> Here is the truth table for material implication:
>    p   |   q   |   p ==> q
> -------|-------|-------------
>    T   |   T   |      T
>    T   |   F   |      F
>    F   |   T   |      T
>    F   |   F   |      T
> "
>
> ^self not or: [aBlock value]
>
>
> This is still a bit foggy to me.
>
> Anyone caring to shed some light on that?
> What are the typical ways to use that?
>
> TIA
> Phil
>
>
>
>
> On Tue, Sep 23, 2014 at 10:07 AM, Udo Schneider
> <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>      > just as it is black magic for me now :D
>     The nice thing about this approach is the fact that it "just"
>     piggybacks the normal Smalltalk message sending. So you can step
>     through it using the Debugger - it's Smalltalk all the way down.
>
>     I still remember my first shock when (having no formal background on
>     parsing theory at that time) I saw the parsing tables generated by
>     T-Gen. Of course it was Smalltalk ... but understanding those tables
>     was a nightmare.
>
>     PetitParser is the gold standard here IMHO. It is able to parse
>     arbitrary input (compared to my block expressions only). And you can
>     still use the Debugger to step through the parsing process *and
>     still understand what's going on!*
>
>     CU,
>
>     Udo
>
>     On 23.09.14 09:54, kilon alios wrote:
>
>         just as it is black magic for me now :D
>
>         At least I get the general feeling. I am new to parsing too, so
>         far I
>         have only played with regex parsing. Not the most smalltalkish
>         way but
>         it works well so far.
>
>         On Tue, Sep 23, 2014 at 9:39 AM, Udo Schneider
>         <[hidden email]
>         <mailto:[hidden email]>
>         <mailto:[hidden email]
>         <mailto:[hidden email]>>>
>         wrote:
>
>              Hi Estaban,
>
>              I think the first time I saw this pattern was in ReStore on
>         Dolphin
>              Smalltalk. I didn't understand it's implementation back then. I
>              assume that it's similar to what I described though. But
>         having a
>              Smalltalk block automagically creating the equivalent SQL
>         SELECT
>              expression was like black magic at that time :-)
>
>              CU,
>
>              Udo
>
>
>
>
>              On 23.09.14 04:15, Esteban A. Maringolo wrote:
>
>                  Excellent article.
>
>                  I think GLORP uses a similar technique to setup its
>         expressions, and
>                  also have issues with #and:/#or: selectors due to
>         inlining, so
>                  it uses
>                  AND:/#OR: instead.
>
>                  Regards!
>
>                  Esteban A. Maringolo
>
>                  pd: Your blog and it's choosen topic made me remember
>         http://use-the-index-luke.com/
>
>                  2014-09-22 20:48 GMT-03:00 Udo Schneider
>                  <[hidden email]
>         <mailto:[hidden email]>
>                  <mailto:[hidden email]
>         <mailto:[hidden email]>>>__:
>
>
>                      All,
>
>                      I just finished a blog entry. It shows how to use
>         Smalltalk
>                      blocks as parsers/translators. E.g. translating a Block
>
>                                [:customer | (customer joinDate year is: Date
>                      today year)]
>
>                      into an SQL-like String
>
>                                (YEAR(customers.joinDate) = 2014)
>
>                      The SQL stuff is just an example - you can create
>         nearly any
>                      output.
>
>                      Check out
>         http://readthesourceluke.__blo__gspot.de/2014/09/block-____translators-parsing-magic.html
>         <http://blogspot.de/2014/09/block-__translators-parsing-magic.html>
>
>         <http://readthesourceluke.__blogspot.de/2014/09/block-__translators-parsing-magic.html
>         <http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html>__>
>
>                      Maybe that's old stuff for some of you - but I hope
>         it's
>                      interesting for some at least :-)
>
>                      Comments and feedback appreciated.
>
>                      CU,
>
>                      Udo
>
>
>
>
>
>
>
>
>
>
>
>
>



Reply | Threaded
Open this post in threaded view
|

Re: BLOG: Block Translators - parsing magic

kilon.alios
In reply to this post by Udo Schneider
yeap noway I compare Regex with PettitParser. I will probably give a PettitParser a try, because I try to parse Pharo syntax to Python, I want now to parse Pharo classes to python class and that will be a nightmare with regex, so time to give PettitParser a serious try. 

On Tue, Sep 23, 2014 at 1:10 PM, Udo Schneider <[hidden email]> wrote:
> I have not used PettitParser yet, looks powerful but I find it a bit
> weird in design. On the other hand regex is quite ugly and understanding
> complex regex a pain.
I normally encounter two issues with RegExps:

1) The syntax between different apps/libs/frameworks differs sligtly. Esp. for character classes or greediness. This drove me crazy more than one time.
2) Often enough I realize (while developing the RegExp) that I need something more capable than a Chomsky Type 3 parser (which RegExps are in a way). So I have to use "a bigger gun" - e.g. PetitParser.

CU,

Udo


On 23.09.14 10:15, kilon alios wrote:
it reminds a lot of Kent's Beck Smalltalk Practice Patterns where it
removes all ifs and replaces them with regular unary messages . It is
definitely an elegant way of coding making the code just flow.

I have not used PettitParser yet, looks powerful but I find it a bit
weird in design. On the other hand regex is quite ugly and understanding
complex regex a pain.

On Tue, Sep 23, 2014 at 11:07 AM, Udo Schneider
<[hidden email]
<mailto:[hidden email]>> wrote:

    > just as it is black magic for me now :D
    The nice thing about this approach is the fact that it "just"
    piggybacks the normal Smalltalk message sending. So you can step
    through it using the Debugger - it's Smalltalk all the way down.

    I still remember my first shock when (having no formal background on
    parsing theory at that time) I saw the parsing tables generated by
    T-Gen. Of course it was Smalltalk ... but understanding those tables
    was a nightmare.

    PetitParser is the gold standard here IMHO. It is able to parse
    arbitrary input (compared to my block expressions only). And you can
    still use the Debugger to step through the parsing process *and
    still understand what's going on!*

    CU,

    Udo

    On 23.09.14 09:54, kilon alios wrote:

        just as it is black magic for me now :D

        At least I get the general feeling. I am new to parsing too, so
        far I
        have only played with regex parsing. Not the most smalltalkish
        way but
        it works well so far.

        On Tue, Sep 23, 2014 at 9:39 AM, Udo Schneider
        <[hidden email]
        <mailto:[hidden email]>
        <mailto:[hidden email]__homeaddress.de

        <mailto:[hidden email]>>>
        wrote:

             Hi Estaban,

             I think the first time I saw this pattern was in ReStore on
        Dolphin
             Smalltalk. I didn't understand it's implementation back then. I
             assume that it's similar to what I described though. But
        having a
             Smalltalk block automagically creating the equivalent SQL
        SELECT
             expression was like black magic at that time :-)

             CU,

             Udo




             On 23.09.14 04:15, Esteban A. Maringolo wrote:

                 Excellent article.

                 I think GLORP uses a similar technique to setup its
        expressions, and
                 also have issues with #and:/#or: selectors due to
        inlining, so
                 it uses
                 AND:/#OR: instead.

                 Regards!

                 Esteban A. Maringolo

                 pd: Your blog and it's choosen topic made me remember
        http://use-the-index-luke.com/

                 2014-09-22 20:48 GMT-03:00 Udo Schneider
                 <[hidden email]
        <mailto:[hidden email]>
                 <mailto:[hidden email]__homeaddress.de
        <mailto:[hidden email]>>>__:


                     All,

                     I just finished a blog entry. It shows how to use
        Smalltalk
                     blocks as parsers/translators. E.g. translating a Block

                               [:customer | (customer joinDate year is: Date
                     today year)]

                     into an SQL-like String

                               (YEAR(customers.joinDate) = 2014)

                     The SQL stuff is just an example - you can create
        nearly any
                     output.

                     Check out
        http://readthesourceluke.__blo__gspot.de/2014/09/block-____translators-parsing-magic.html
        <http://blogspot.de/2014/09/block-__translators-parsing-magic.html>

        <http://readthesourceluke.__blogspot.de/2014/09/block-__translators-parsing-magic.html
        <http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html>__>

                     Maybe that's old stuff for some of you - but I hope
        it's
                     interesting for some at least :-)

                     Comments and feedback appreciated.

                     CU,

                     Udo
















Reply | Threaded
Open this post in threaded view
|

Re: BLOG: Block Translators - parsing magic

Thierry Goubier


2014-09-23 13:35 GMT+02:00 kilon alios <[hidden email]>:
yeap noway I compare Regex with PettitParser. I will probably give a PettitParser a try, because I try to parse Pharo syntax to Python, I want now to parse Pharo classes to python class and that will be a nightmare with regex, so time to give PettitParser a serious try.


Kilon, just remember that you have a full Smalltalk parser already in your image, called RBParser.

For Python parsing, there is probably a PetitParser based parser somewhere and two implementations of a SmaCC-based Python parser (For Python 2.7.X).

Thierry
 

On Tue, Sep 23, 2014 at 1:10 PM, Udo Schneider <[hidden email]> wrote:
> I have not used PettitParser yet, looks powerful but I find it a bit
> weird in design. On the other hand regex is quite ugly and understanding
> complex regex a pain.
I normally encounter two issues with RegExps:

1) The syntax between different apps/libs/frameworks differs sligtly. Esp. for character classes or greediness. This drove me crazy more than one time.
2) Often enough I realize (while developing the RegExp) that I need something more capable than a Chomsky Type 3 parser (which RegExps are in a way). So I have to use "a bigger gun" - e.g. PetitParser.

CU,

Udo


On 23.09.14 10:15, kilon alios wrote:
it reminds a lot of Kent's Beck Smalltalk Practice Patterns where it
removes all ifs and replaces them with regular unary messages . It is
definitely an elegant way of coding making the code just flow.

I have not used PettitParser yet, looks powerful but I find it a bit
weird in design. On the other hand regex is quite ugly and understanding
complex regex a pain.

On Tue, Sep 23, 2014 at 11:07 AM, Udo Schneider
<[hidden email]
<mailto:[hidden email]>> wrote:

    > just as it is black magic for me now :D
    The nice thing about this approach is the fact that it "just"
    piggybacks the normal Smalltalk message sending. So you can step
    through it using the Debugger - it's Smalltalk all the way down.

    I still remember my first shock when (having no formal background on
    parsing theory at that time) I saw the parsing tables generated by
    T-Gen. Of course it was Smalltalk ... but understanding those tables
    was a nightmare.

    PetitParser is the gold standard here IMHO. It is able to parse
    arbitrary input (compared to my block expressions only). And you can
    still use the Debugger to step through the parsing process *and
    still understand what's going on!*

    CU,

    Udo

    On 23.09.14 09:54, kilon alios wrote:

        just as it is black magic for me now :D

        At least I get the general feeling. I am new to parsing too, so
        far I
        have only played with regex parsing. Not the most smalltalkish
        way but
        it works well so far.

        On Tue, Sep 23, 2014 at 9:39 AM, Udo Schneider
        <[hidden email]
        <mailto:[hidden email]>
        <mailto:[hidden email]__homeaddress.de

        <mailto:[hidden email]>>>
        wrote:

             Hi Estaban,

             I think the first time I saw this pattern was in ReStore on
        Dolphin
             Smalltalk. I didn't understand it's implementation back then. I
             assume that it's similar to what I described though. But
        having a
             Smalltalk block automagically creating the equivalent SQL
        SELECT
             expression was like black magic at that time :-)

             CU,

             Udo




             On 23.09.14 04:15, Esteban A. Maringolo wrote:

                 Excellent article.

                 I think GLORP uses a similar technique to setup its
        expressions, and
                 also have issues with #and:/#or: selectors due to
        inlining, so
                 it uses
                 AND:/#OR: instead.

                 Regards!

                 Esteban A. Maringolo

                 pd: Your blog and it's choosen topic made me remember
        http://use-the-index-luke.com/

                 2014-09-22 20:48 GMT-03:00 Udo Schneider
                 <[hidden email]
        <mailto:[hidden email]>
                 <mailto:[hidden email]__homeaddress.de
        <mailto:[hidden email]>>>__:


                     All,

                     I just finished a blog entry. It shows how to use
        Smalltalk
                     blocks as parsers/translators. E.g. translating a Block

                               [:customer | (customer joinDate year is: Date
                     today year)]

                     into an SQL-like String

                               (YEAR(customers.joinDate) = 2014)

                     The SQL stuff is just an example - you can create
        nearly any
                     output.

                     Check out
        http://readthesourceluke.__blo__gspot.de/2014/09/block-____translators-parsing-magic.html
        <http://blogspot.de/2014/09/block-__translators-parsing-magic.html>

        <http://readthesourceluke.__blogspot.de/2014/09/block-__translators-parsing-magic.html
        <http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html>__>

                     Maybe that's old stuff for some of you - but I hope
        it's
                     interesting for some at least :-)

                     Comments and feedback appreciated.

                     CU,

                     Udo

















Reply | Threaded
Open this post in threaded view
|

Re: BLOG: Block Translators - parsing magic

Udo Schneider
In reply to this post by kilon.alios
 > yeap noway I compare Regex with PettitParser. I will probably give a
 > PettitParser a try, because I try to parse Pharo syntax to Python, I
 > want now to parse Pharo classes to python class and that will be a
 > nightmare with regex, so time to give PettitParser a serious try.
Without wanting to go too deep into theory... Parsing Pharo or Python
would definitely need a Chromsky Type 2 language (Context free grammar).
So you'd be out of luck parsing Smalltalk or Python code with Regular
Expressions (Chromsky Type 3) only - except maybe for some very simple
expressions.

CU,

Udo


On 23.09.14 13:35, kilon alios wrote:

> yeap noway I compare Regex with PettitParser. I will probably give a
> PettitParser a try, because I try to parse Pharo syntax to Python, I
> want now to parse Pharo classes to python class and that will be a
> nightmare with regex, so time to give PettitParser a serious try.
>
> On Tue, Sep 23, 2014 at 1:10 PM, Udo Schneider
> <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     > I have not used PettitParser yet, looks powerful but I find it a bit
>     > weird in design. On the other hand regex is quite ugly and understanding
>     > complex regex a pain.
>     I normally encounter two issues with RegExps:
>
>     1) The syntax between different apps/libs/frameworks differs
>     sligtly. Esp. for character classes or greediness. This drove me
>     crazy more than one time.
>     2) Often enough I realize (while developing the RegExp) that I need
>     something more capable than a Chomsky Type 3 parser (which RegExps
>     are in a way). So I have to use "a bigger gun" - e.g. PetitParser.
>
>     CU,
>
>     Udo
>
>
>     On 23.09.14 10:15, kilon alios wrote:
>
>         it reminds a lot of Kent's Beck Smalltalk Practice Patterns where it
>         removes all ifs and replaces them with regular unary messages .
>         It is
>         definitely an elegant way of coding making the code just flow.
>
>         I have not used PettitParser yet, looks powerful but I find it a bit
>         weird in design. On the other hand regex is quite ugly and
>         understanding
>         complex regex a pain.
>
>         On Tue, Sep 23, 2014 at 11:07 AM, Udo Schneider
>         <[hidden email]
>         <mailto:[hidden email]>
>         <mailto:[hidden email]
>         <mailto:[hidden email]>>>
>         wrote:
>
>              > just as it is black magic for me now :D
>              The nice thing about this approach is the fact that it "just"
>              piggybacks the normal Smalltalk message sending. So you can
>         step
>              through it using the Debugger - it's Smalltalk all the way
>         down.
>
>              I still remember my first shock when (having no formal
>         background on
>              parsing theory at that time) I saw the parsing tables
>         generated by
>              T-Gen. Of course it was Smalltalk ... but understanding
>         those tables
>              was a nightmare.
>
>              PetitParser is the gold standard here IMHO. It is able to parse
>              arbitrary input (compared to my block expressions only).
>         And you can
>              still use the Debugger to step through the parsing process *and
>              still understand what's going on!*
>
>              CU,
>
>              Udo
>
>              On 23.09.14 09:54, kilon alios wrote:
>
>                  just as it is black magic for me now :D
>
>                  At least I get the general feeling. I am new to parsing
>         too, so
>                  far I
>                  have only played with regex parsing. Not the most
>         smalltalkish
>                  way but
>                  it works well so far.
>
>                  On Tue, Sep 23, 2014 at 9:39 AM, Udo Schneider
>                  <[hidden email]
>         <mailto:[hidden email]>
>                  <mailto:[hidden email]
>         <mailto:[hidden email]>>
>                  <mailto:udo.schneider@
>         <mailto:udo.schneider@>__homead__dress.de <http://homeaddress.de>
>
>                  <mailto:[hidden email]
>         <mailto:[hidden email]>>>>
>                  wrote:
>
>                       Hi Estaban,
>
>                       I think the first time I saw this pattern was in
>         ReStore on
>                  Dolphin
>                       Smalltalk. I didn't understand it's implementation
>         back then. I
>                       assume that it's similar to what I described
>         though. But
>                  having a
>                       Smalltalk block automagically creating the
>         equivalent SQL
>                  SELECT
>                       expression was like black magic at that time :-)
>
>                       CU,
>
>                       Udo
>
>
>
>
>                       On 23.09.14 04:15, Esteban A. Maringolo wrote:
>
>                           Excellent article.
>
>                           I think GLORP uses a similar technique to
>         setup its
>                  expressions, and
>                           also have issues with #and:/#or: selectors due to
>                  inlining, so
>                           it uses
>                           AND:/#OR: instead.
>
>                           Regards!
>
>                           Esteban A. Maringolo
>
>                           pd: Your blog and it's choosen topic made me
>         remember
>         http://use-the-index-luke.com/
>
>                           2014-09-22 20:48 GMT-03:00 Udo Schneider
>                           <[hidden email]
>         <mailto:[hidden email]>
>                  <mailto:[hidden email]
>         <mailto:[hidden email]>>
>                           <mailto:udo.schneider@
>         <mailto:udo.schneider@>__homead__dress.de <http://homeaddress.de>
>                  <mailto:[hidden email]
>         <mailto:[hidden email]>>>>__:
>
>
>                               All,
>
>                               I just finished a blog entry. It shows how
>         to use
>                  Smalltalk
>                               blocks as parsers/translators. E.g.
>         translating a Block
>
>                                         [:customer | (customer joinDate
>         year is: Date
>                               today year)]
>
>                               into an SQL-like String
>
>                                         (YEAR(customers.joinDate) = 2014)
>
>                               The SQL stuff is just an example - you can
>         create
>                  nearly any
>                               output.
>
>                               Check out
>         http://readthesourceluke.__blo____gspot.de/2014/09/block-______translators-parsing-magic.html
>         <http://blo__gspot.de/2014/09/block-____translators-parsing-magic.html>
>
>         <http://blogspot.de/2014/09/__block-__translators-parsing-__magic.html
>         <http://blogspot.de/2014/09/block-__translators-parsing-magic.html>>
>
>
>         <http://readthesourceluke.__bl__ogspot.de/2014/09/block-____translators-parsing-magic.html
>         <http://blogspot.de/2014/09/block-__translators-parsing-magic.html>
>
>         <http://readthesourceluke.__blogspot.de/2014/09/block-__translators-parsing-magic.html
>         <http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html>__>__>
>
>                               Maybe that's old stuff for some of you -
>         but I hope
>                  it's
>                               interesting for some at least :-)
>
>                               Comments and feedback appreciated.
>
>                               CU,
>
>                               Udo
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>



Reply | Threaded
Open this post in threaded view
|

Re: BLOG: Block Translators - parsing magic

philippeback
In reply to this post by Udo Schneider

Thx for this!

I'd say that the material is worthy of inclusion into a bool chapter.

Phil

Le 23 sept. 2014 12:51, "Udo Schneider" <[hidden email]> a écrit :
Phil,

I'd say that's an implementation of the "Material implication"[1] operator from Propositional calculus.

You can write it as

P -> Q

and read it as "P implies Q" or (not 100% correct) "if P (is true) then Q (is true)".

Let's take a look at the truth table:

 P | Q | P -> Q
---+---+-------
 t | t | t
---+---+-------

inserting it into the statement above yields:

"if true then true" which is true indeed.

The funny thing starts when we look at the case(s), where P is false. The general (verbal) rule says that if the premise (P) is false the truth value of the conclusion doesn't matter. Hence the complete term is true:

 P | Q | P -> Q
---+---+-------
 f | t | t
---+---+-------
 f | f | t
---+---+-------

There is one case left: What happens if the premise is true but the conclusion is false? That's in direct violation of the defintion which states that "if P (is true) then Q (is true)". As P is true and Q is false the truth value of the complete term is false!

 P | Q | P -> Q
---+---+-------
 t | f | f
---+---+-------

Or to put it in different words: An implication can only be false if the premise is true but the conclusion is false.

So we end up with this truth table:

 P | Q | P -> Q
---+---+-------
 t | t | t
---+---+-------
 f | t | t
---+---+-------
 f | f | t
---+---+-------
 t | f | f
---+---+-------

We did not take a look yet, at how to implement this with basic boolean operators. So we need to take a look at the table and find the expression: Let's start with the exception (fourth case). We can express this as:

!(P & !Q) -> "The term is false if P is true and Q is false"

I added the inner term "P & !Q" to the table to make it easier to follow:


 P | Q | P -> Q | P & !Q | !(P & !Q)
---+---+--------+--------+-----------
 t | t | t      | f      | t
---+---+--------+--------+-----------
 f | t | t      | t      | f
---+---+--------+--------+-----------
 f | f | t      | f      | t
---+---+--------+--------+-----------
 t | f | f      | f      | t
---+---+--------+--------+-----------

The "last" step is to simplify the term "!(P & !Q)" using one of de Morgan's laws:
!(A & B) == !A | !B

Using this for our term gives

!(P & !Q) == !P | !!Q == !P | Q

 P | Q | P -> Q | P & !Q | !(P & !Q) | !P | Q
---+---+--------+--------+-----------+--------
 t | t | t      | f      | t         | t
---+---+--------+--------+-----------+--------
 f | t | t      | t      | f         | f
---+---+--------+--------+-----------+--------
 f | f | t      | f      | t         | t
---+---+--------+--------+-----------+--------
 t | f | f      | f      | t         | t
---+---+--------+--------+-----------+--------

So our final term is "correct" (proof in the truth table) and is equivalent to the smalltalk term:

!P | Q  := self not or: [aBlock value]


I have to admit though that implications in Propositional calculus really gave me a headache. You might simply want to accept that they are defined this way. And maybe it doesn't help ... but there are even more strange things lurking around the corner [4] :-)

Although not immidiatly obvious all the terms in Propositional calculus do not neccessarly have a semantic meaning in context to each other. They are totaly independent. The only thing that counts is the truth value of its terms. E.g. something like this is mathematically/syntacically valid but doesn't make any sense from a semantic point of view:

P := I am 12y old
Q := It rains

The term "P -> Q" is perfectly fine mathematically/syntacically but doesn't mean anything in terms of semantics.


Hope this helps.

CU,

Udo


[1] http://en.wikipedia.org/wiki/Material_implication_(rule_of_inference)
[2] http://en.wikipedia.org/wiki/Propositional_calculus
[3] http://en.wikipedia.org/wiki/De_Morgan's_laws
[4] http://en.wikipedia.org/wiki/Paradoxes_of_material_implication

On 23.09.14 10:49, [hidden email] wrote:
Cool article & technique indeed. Ah Smalltalk, where were you all those
years ;-)

Speaking of PetitParser, which is excellent indeed, there is this #==>
method in Boolean.

PetitParser uses that a lot. I can use the thing but do not really
grasps how it works.

Now, the method comment says:

Boolean #==> aBlock
"The material conditional, also known as the material implication or
truth functional conditional.Correspond to not ... or ... and does not
correspond to the English if...then... construction.
known as:
b if a
a implies b
if a then b
b is a consequence of a
a therefore b (but note: 'it is raining therefore it is cloudy' is
implication; 'it is autumn therefore the leaves are falling' is
equivalence).
Here is the truth table for material implication:
   p   |   q   |   p ==> q
-------|-------|-------------
   T   |   T   |      T
   T   |   F   |      F
   F   |   T   |      T
   F   |   F   |      T
"

^self not or: [aBlock value]


This is still a bit foggy to me.

Anyone caring to shed some light on that?
What are the typical ways to use that?

TIA
Phil




On Tue, Sep 23, 2014 at 10:07 AM, Udo Schneider
<[hidden email]
<mailto:[hidden email]>> wrote:

     > just as it is black magic for me now :D
    The nice thing about this approach is the fact that it "just"
    piggybacks the normal Smalltalk message sending. So you can step
    through it using the Debugger - it's Smalltalk all the way down.

    I still remember my first shock when (having no formal background on
    parsing theory at that time) I saw the parsing tables generated by
    T-Gen. Of course it was Smalltalk ... but understanding those tables
    was a nightmare.

    PetitParser is the gold standard here IMHO. It is able to parse
    arbitrary input (compared to my block expressions only). And you can
    still use the Debugger to step through the parsing process *and
    still understand what's going on!*

    CU,

    Udo

    On 23.09.14 09:54, kilon alios wrote:

        just as it is black magic for me now :D

        At least I get the general feeling. I am new to parsing too, so
        far I
        have only played with regex parsing. Not the most smalltalkish
        way but
        it works well so far.

        On Tue, Sep 23, 2014 at 9:39 AM, Udo Schneider
        <[hidden email]
        <mailto:[hidden email]>
        <mailto:[hidden email]__homeaddress.de
        <mailto:[hidden email]>>>
        wrote:

             Hi Estaban,

             I think the first time I saw this pattern was in ReStore on
        Dolphin
             Smalltalk. I didn't understand it's implementation back then. I
             assume that it's similar to what I described though. But
        having a
             Smalltalk block automagically creating the equivalent SQL
        SELECT
             expression was like black magic at that time :-)

             CU,

             Udo




             On 23.09.14 04:15, Esteban A. Maringolo wrote:

                 Excellent article.

                 I think GLORP uses a similar technique to setup its
        expressions, and
                 also have issues with #and:/#or: selectors due to
        inlining, so
                 it uses
                 AND:/#OR: instead.

                 Regards!

                 Esteban A. Maringolo

                 pd: Your blog and it's choosen topic made me remember
        http://use-the-index-luke.com/

                 2014-09-22 20:48 GMT-03:00 Udo Schneider
                 <[hidden email]
        <mailto:[hidden email]>
                 <mailto:[hidden email]__homeaddress.de
        <mailto:[hidden email]>>>__:


                     All,

                     I just finished a blog entry. It shows how to use
        Smalltalk
                     blocks as parsers/translators. E.g. translating a Block

                               [:customer | (customer joinDate year is: Date
                     today year)]

                     into an SQL-like String

                               (YEAR(customers.joinDate) = 2014)

                     The SQL stuff is just an example - you can create
        nearly any
                     output.

                     Check out
        http://readthesourceluke.__blo__gspot.de/2014/09/block-____translators-parsing-magic.html
        <http://blogspot.de/2014/09/block-__translators-parsing-magic.html>

        <http://readthesourceluke.__blogspot.de/2014/09/block-__translators-parsing-magic.html
        <http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html>__>

                     Maybe that's old stuff for some of you - but I hope
        it's
                     interesting for some at least :-)

                     Comments and feedback appreciated.

                     CU,

                     Udo

















Reply | Threaded
Open this post in threaded view
|

Re: BLOG: Block Translators - parsing magic

Thierry Goubier
In reply to this post by Udo Schneider


2014-09-23 13:57 GMT+02:00 Udo Schneider <[hidden email]>:
> yeap noway I compare Regex with PettitParser. I will probably give a
> PettitParser a try, because I try to parse Pharo syntax to Python, I
> want now to parse Pharo classes to python class and that will be a
> nightmare with regex, so time to give PettitParser a serious try.
Without wanting to go too deep into theory... Parsing Pharo or Python would definitely need a Chromsky Type 2 language (Context free grammar). So you'd be out of luck parsing Smalltalk or Python code with Regular Expressions (Chromsky Type 3) only - except maybe for some very simple expressions.

Confirmed in even a better way: given how convoluted and hacky is writing a full Python Parser, it is probably not even a Context Free Grammar.

That is: the Python grammar is LALR which is a subset of Context Free Grammar (but already more than a type 3 Chomsky grammar), but the Lexer is a fair bit more than usual (dedent tokens come to mind; special handling of end of lines as well).

I think you need a fair bit of special magic in PetitParser to parse Python.

Luckily, Python is well documented in that regard.

Thierry
 

CU,

Udo


On 23.09.14 13:35, kilon alios wrote:
yeap noway I compare Regex with PettitParser. I will probably give a
PettitParser a try, because I try to parse Pharo syntax to Python, I
want now to parse Pharo classes to python class and that will be a
nightmare with regex, so time to give PettitParser a serious try.

On Tue, Sep 23, 2014 at 1:10 PM, Udo Schneider
<[hidden email]
<mailto:[hidden email]>> wrote:

    > I have not used PettitParser yet, looks powerful but I find it a bit
    > weird in design. On the other hand regex is quite ugly and understanding
    > complex regex a pain.
    I normally encounter two issues with RegExps:

    1) The syntax between different apps/libs/frameworks differs
    sligtly. Esp. for character classes or greediness. This drove me
    crazy more than one time.
    2) Often enough I realize (while developing the RegExp) that I need
    something more capable than a Chomsky Type 3 parser (which RegExps
    are in a way). So I have to use "a bigger gun" - e.g. PetitParser.

    CU,

    Udo


    On 23.09.14 10:15, kilon alios wrote:

        it reminds a lot of Kent's Beck Smalltalk Practice Patterns where it
        removes all ifs and replaces them with regular unary messages .
        It is
        definitely an elegant way of coding making the code just flow.

        I have not used PettitParser yet, looks powerful but I find it a bit
        weird in design. On the other hand regex is quite ugly and
        understanding
        complex regex a pain.

        On Tue, Sep 23, 2014 at 11:07 AM, Udo Schneider
        <[hidden email]
        <mailto:[hidden email]>
        <mailto:[hidden email]__homeaddress.de
        <mailto:[hidden email]>>>
        wrote:

             > just as it is black magic for me now :D
             The nice thing about this approach is the fact that it "just"
             piggybacks the normal Smalltalk message sending. So you can
        step
             through it using the Debugger - it's Smalltalk all the way
        down.

             I still remember my first shock when (having no formal
        background on
             parsing theory at that time) I saw the parsing tables
        generated by
             T-Gen. Of course it was Smalltalk ... but understanding
        those tables
             was a nightmare.

             PetitParser is the gold standard here IMHO. It is able to parse
             arbitrary input (compared to my block expressions only).
        And you can
             still use the Debugger to step through the parsing process *and
             still understand what's going on!*

             CU,

             Udo

             On 23.09.14 09:54, kilon alios wrote:

                 just as it is black magic for me now :D

                 At least I get the general feeling. I am new to parsing
        too, so
                 far I
                 have only played with regex parsing. Not the most
        smalltalkish
                 way but
                 it works well so far.

                 On Tue, Sep 23, 2014 at 9:39 AM, Udo Schneider
                 <[hidden email]
        <mailto:[hidden email]>
                 <mailto:[hidden email]__homeaddress.de
        <mailto:[hidden email]>>
                 <mailto:[hidden email]
        <mailto:[hidden email]>__homead__dress.de <http://homeaddress.de>


                 <mailto:[hidden email]__homeaddress.de
        <mailto:[hidden email]>>>>
                 wrote:

                      Hi Estaban,

                      I think the first time I saw this pattern was in
        ReStore on
                 Dolphin
                      Smalltalk. I didn't understand it's implementation
        back then. I
                      assume that it's similar to what I described
        though. But
                 having a
                      Smalltalk block automagically creating the
        equivalent SQL
                 SELECT
                      expression was like black magic at that time :-)

                      CU,

                      Udo




                      On 23.09.14 04:15, Esteban A. Maringolo wrote:

                          Excellent article.

                          I think GLORP uses a similar technique to
        setup its
                 expressions, and
                          also have issues with #and:/#or: selectors due to
                 inlining, so
                          it uses
                          AND:/#OR: instead.

                          Regards!

                          Esteban A. Maringolo

                          pd: Your blog and it's choosen topic made me
        remember
        http://use-the-index-luke.com/

                          2014-09-22 20:48 GMT-03:00 Udo Schneider
                          <[hidden email]
        <mailto:[hidden email]>
                 <mailto:[hidden email]__homeaddress.de
        <mailto:[hidden email]>>
                          <mailto:[hidden email]
        <mailto:[hidden email]>__homead__dress.de <http://homeaddress.de>
                 <mailto:[hidden email]__homeaddress.de
        <mailto:[hidden email]>>>>__:


                              All,

                              I just finished a blog entry. It shows how
        to use
                 Smalltalk
                              blocks as parsers/translators. E.g.
        translating a Block

                                        [:customer | (customer joinDate
        year is: Date
                              today year)]

                              into an SQL-like String

                                        (YEAR(customers.joinDate) = 2014)

                              The SQL stuff is just an example - you can
        create
                 nearly any
                              output.

                              Check out
        http://readthesourceluke.__blo____gspot.de/2014/09/block-______translators-parsing-magic.html
        <http://blo__gspot.de/2014/09/block-____translators-parsing-magic.html>

        <http://blogspot.de/2014/09/__block-__translators-parsing-__magic.html
        <http://blogspot.de/2014/09/block-__translators-parsing-magic.html>>


        <http://readthesourceluke.__bl__ogspot.de/2014/09/block-____translators-parsing-magic.html
        <http://blogspot.de/2014/09/block-__translators-parsing-magic.html>

        <http://readthesourceluke.__blogspot.de/2014/09/block-__translators-parsing-magic.html
        <http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html>__>__>

                              Maybe that's old stuff for some of you -
        but I hope
                 it's
                              interesting for some at least :-)

                              Comments and feedback appreciated.

                              CU,

                              Udo




















Reply | Threaded
Open this post in threaded view
|

Re: BLOG: Block Translators - parsing magic

kilon.alios
damn you guys speak in a language I am not aware of :D Looks like I have reading to do.

Anyway regex worked fin for converting pharo messages to python method calls and assignment / reading variables . Regex was actually quite simple to learn. 

Now for classes I dont know if RBParser would be an overkill or not, probably its good. I have asked before and I was shown how nodes works with AST but I have not used that information yet.  

I dont care about a full python parser, thats not my goal. Its not my goal to take pharo code and fully convert it to python code, only make it easy for pharoers to use python libraries. 

So I move in small steps , instead of trying to create a whole parser. 

Though a full python parser would not be a bad idea either. So I am definitely interested to what you saying. 

On Tue, Sep 23, 2014 at 3:20 PM, Thierry Goubier <[hidden email]> wrote:


2014-09-23 13:57 GMT+02:00 Udo Schneider <[hidden email]>:
> yeap noway I compare Regex with PettitParser. I will probably give a
> PettitParser a try, because I try to parse Pharo syntax to Python, I
> want now to parse Pharo classes to python class and that will be a
> nightmare with regex, so time to give PettitParser a serious try.
Without wanting to go too deep into theory... Parsing Pharo or Python would definitely need a Chromsky Type 2 language (Context free grammar). So you'd be out of luck parsing Smalltalk or Python code with Regular Expressions (Chromsky Type 3) only - except maybe for some very simple expressions.

Confirmed in even a better way: given how convoluted and hacky is writing a full Python Parser, it is probably not even a Context Free Grammar.

That is: the Python grammar is LALR which is a subset of Context Free Grammar (but already more than a type 3 Chomsky grammar), but the Lexer is a fair bit more than usual (dedent tokens come to mind; special handling of end of lines as well).

I think you need a fair bit of special magic in PetitParser to parse Python.

Luckily, Python is well documented in that regard.

Thierry
 

CU,

Udo


On 23.09.14 13:35, kilon alios wrote:
yeap noway I compare Regex with PettitParser. I will probably give a
PettitParser a try, because I try to parse Pharo syntax to Python, I
want now to parse Pharo classes to python class and that will be a
nightmare with regex, so time to give PettitParser a serious try.

On Tue, Sep 23, 2014 at 1:10 PM, Udo Schneider
<[hidden email]
<mailto:[hidden email]>> wrote:

    > I have not used PettitParser yet, looks powerful but I find it a bit
    > weird in design. On the other hand regex is quite ugly and understanding
    > complex regex a pain.
    I normally encounter two issues with RegExps:

    1) The syntax between different apps/libs/frameworks differs
    sligtly. Esp. for character classes or greediness. This drove me
    crazy more than one time.
    2) Often enough I realize (while developing the RegExp) that I need
    something more capable than a Chomsky Type 3 parser (which RegExps
    are in a way). So I have to use "a bigger gun" - e.g. PetitParser.

    CU,

    Udo


    On 23.09.14 10:15, kilon alios wrote:

        it reminds a lot of Kent's Beck Smalltalk Practice Patterns where it
        removes all ifs and replaces them with regular unary messages .
        It is
        definitely an elegant way of coding making the code just flow.

        I have not used PettitParser yet, looks powerful but I find it a bit
        weird in design. On the other hand regex is quite ugly and
        understanding
        complex regex a pain.

        On Tue, Sep 23, 2014 at 11:07 AM, Udo Schneider
        <[hidden email]
        <mailto:[hidden email]>
        <mailto:[hidden email]__homeaddress.de
        <mailto:[hidden email]>>>
        wrote:

             > just as it is black magic for me now :D
             The nice thing about this approach is the fact that it "just"
             piggybacks the normal Smalltalk message sending. So you can
        step
             through it using the Debugger - it's Smalltalk all the way
        down.

             I still remember my first shock when (having no formal
        background on
             parsing theory at that time) I saw the parsing tables
        generated by
             T-Gen. Of course it was Smalltalk ... but understanding
        those tables
             was a nightmare.

             PetitParser is the gold standard here IMHO. It is able to parse
             arbitrary input (compared to my block expressions only).
        And you can
             still use the Debugger to step through the parsing process *and
             still understand what's going on!*

             CU,

             Udo

             On 23.09.14 09:54, kilon alios wrote:

                 just as it is black magic for me now :D

                 At least I get the general feeling. I am new to parsing
        too, so
                 far I
                 have only played with regex parsing. Not the most
        smalltalkish
                 way but
                 it works well so far.

                 On Tue, Sep 23, 2014 at 9:39 AM, Udo Schneider
                 <[hidden email]
        <mailto:[hidden email]>
                 <mailto:[hidden email]__homeaddress.de
        <mailto:[hidden email]>>
                 <mailto:[hidden email]
        <mailto:[hidden email]>__homead__dress.de <http://homeaddress.de>


                 <mailto:[hidden email]__homeaddress.de
        <mailto:[hidden email]>>>>
                 wrote:

                      Hi Estaban,

                      I think the first time I saw this pattern was in
        ReStore on
                 Dolphin
                      Smalltalk. I didn't understand it's implementation
        back then. I
                      assume that it's similar to what I described
        though. But
                 having a
                      Smalltalk block automagically creating the
        equivalent SQL
                 SELECT
                      expression was like black magic at that time :-)

                      CU,

                      Udo




                      On 23.09.14 04:15, Esteban A. Maringolo wrote:

                          Excellent article.

                          I think GLORP uses a similar technique to
        setup its
                 expressions, and
                          also have issues with #and:/#or: selectors due to
                 inlining, so
                          it uses
                          AND:/#OR: instead.

                          Regards!

                          Esteban A. Maringolo

                          pd: Your blog and it's choosen topic made me
        remember
        http://use-the-index-luke.com/

                          2014-09-22 20:48 GMT-03:00 Udo Schneider
                          <[hidden email]
        <mailto:[hidden email]>
                 <mailto:[hidden email]__homeaddress.de
        <mailto:[hidden email]>>
                          <mailto:[hidden email]
        <mailto:[hidden email]>__homead__dress.de <http://homeaddress.de>
                 <mailto:[hidden email]__homeaddress.de
        <mailto:[hidden email]>>>>__:


                              All,

                              I just finished a blog entry. It shows how
        to use
                 Smalltalk
                              blocks as parsers/translators. E.g.
        translating a Block

                                        [:customer | (customer joinDate
        year is: Date
                              today year)]

                              into an SQL-like String

                                        (YEAR(customers.joinDate) = 2014)

                              The SQL stuff is just an example - you can
        create
                 nearly any
                              output.

                              Check out
        http://readthesourceluke.__blo____gspot.de/2014/09/block-______translators-parsing-magic.html
        <http://blo__gspot.de/2014/09/block-____translators-parsing-magic.html>

        <http://blogspot.de/2014/09/__block-__translators-parsing-__magic.html
        <http://blogspot.de/2014/09/block-__translators-parsing-magic.html>>


        <http://readthesourceluke.__bl__ogspot.de/2014/09/block-____translators-parsing-magic.html
        <http://blogspot.de/2014/09/block-__translators-parsing-magic.html>

        <http://readthesourceluke.__blogspot.de/2014/09/block-__translators-parsing-magic.html
        <http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html>__>__>

                              Maybe that's old stuff for some of you -
        but I hope
                 it's
                              interesting for some at least :-)

                              Comments and feedback appreciated.

                              CU,

                              Udo





















123