start of line parser

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

start of line parser

Tudor Girba-2
Hi,

Jan has implemented a nice parser that allows us to specify easily productions that depend on text having to be on a new line.

For example, in Pillar, a header is specified through an exclamation mark starting on a new line. Now, you can parse this as:

#startOfLine asParser , $! asParser , #newline asParser negate star

Pretty cool!


@Jan: I think we need to enhance a bit the implementation. The implementation looks like this right now:
isStartOfLine
(position = 0) ifTrue: [ ^ true ].
^ (collection at: position) = Character cr.

This will work when the line ending is either cr or lf+cr. However, it won't work if the line ending is just lf.

For example, in my parsers, the newline implementation looks like:
(#cr asParser , #lf asParser optional) / #lf asParser / #cr asParser


What do you think?

Cheers,
Doru

--

"Every thing has its own flow"

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev
Reply | Threaded
Open this post in threaded view
|

Re: start of line parser

Jan Kurš
@Doru: This might solve the problem:

isStartOfLine
(position = 0) ifTrue: [ ^ true ].
        ^ ((collection at: position) = Character cr) or: [ (collection at: position) = Character lf ].

Do you agree?

Cheers,
Jan


On 3 September 2014 14:46, Tudor Girba <[hidden email]> wrote:
Hi,

Jan has implemented a nice parser that allows us to specify easily productions that depend on text having to be on a new line.

For example, in Pillar, a header is specified through an exclamation mark starting on a new line. Now, you can parse this as:

#startOfLine asParser , $! asParser , #newline asParser negate star

Pretty cool!


@Jan: I think we need to enhance a bit the implementation. The implementation looks like this right now:
isStartOfLine
(position = 0) ifTrue: [ ^ true ].
^ (collection at: position) = Character cr.

This will work when the line ending is either cr or lf+cr. However, it won't work if the line ending is just lf.

For example, in my parsers, the newline implementation looks like:
(#cr asParser , #lf asParser optional) / #lf asParser / #cr asParser


What do you think?

Cheers,
Doru

--

"Every thing has its own flow"

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev



_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev
Reply | Threaded
Open this post in threaded view
|

Re: start of line parser

Tudor Girba-2
I think this is not enough, because when you have cr+lf, you do not want to be true on cr. I think we should go with the the same rule as in the parser I proposed, which means that we need to reason about two characters.

Doru


On Wed, Sep 3, 2014 at 3:38 PM, Jan Kurš <[hidden email]> wrote:
@Doru: This might solve the problem:


isStartOfLine
(position = 0) ifTrue: [ ^ true ].
        ^ ((collection at: position) = Character cr) or: [ (collection at: position) = Character lf ].

Do you agree?

Cheers,
Jan


On 3 September 2014 14:46, Tudor Girba <[hidden email]> wrote:
Hi,

Jan has implemented a nice parser that allows us to specify easily productions that depend on text having to be on a new line.

For example, in Pillar, a header is specified through an exclamation mark starting on a new line. Now, you can parse this as:

#startOfLine asParser , $! asParser , #newline asParser negate star

Pretty cool!


@Jan: I think we need to enhance a bit the implementation. The implementation looks like this right now:
isStartOfLine
(position = 0) ifTrue: [ ^ true ].
^ (collection at: position) = Character cr.

This will work when the line ending is either cr or lf+cr. However, it won't work if the line ending is just lf.

For example, in my parsers, the newline implementation looks like:
(#cr asParser , #lf asParser optional) / #lf asParser / #cr asParser


What do you think?

Cheers,
Doru

--

"Every thing has its own flow"

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev



_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev




--

"Every thing has its own flow"

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev
Reply | Threaded
Open this post in threaded view
|

Re: start of line parser

Jan Kurš
Ok, I ended up with this solution:

isStartOfLine
    (position = 0) ifTrue: [ ^ true ].

    self insideCRLF ifTrue: [ ^ false ].
   
    ^ (self peekBack = Character cr) or: [ self peekBack = Character lf].



The question that remains is, how to handle a situation when CR/LF/CRLF is at the end of the stream. Is it still a new line?

Cheers,
Jan


On 3 September 2014 15:52, Tudor Girba <[hidden email]> wrote:
I think this is not enough, because when you have cr+lf, you do not want to be true on cr. I think we should go with the the same rule as in the parser I proposed, which means that we need to reason about two characters.

Doru


On Wed, Sep 3, 2014 at 3:38 PM, Jan Kurš <[hidden email]> wrote:
@Doru: This might solve the problem:


isStartOfLine
(position = 0) ifTrue: [ ^ true ].
        ^ ((collection at: position) = Character cr) or: [ (collection at: position) = Character lf ].

Do you agree?

Cheers,
Jan


On 3 September 2014 14:46, Tudor Girba <[hidden email]> wrote:
Hi,

Jan has implemented a nice parser that allows us to specify easily productions that depend on text having to be on a new line.

For example, in Pillar, a header is specified through an exclamation mark starting on a new line. Now, you can parse this as:

#startOfLine asParser , $! asParser , #newline asParser negate star

Pretty cool!


@Jan: I think we need to enhance a bit the implementation. The implementation looks like this right now:
isStartOfLine
(position = 0) ifTrue: [ ^ true ].
^ (collection at: position) = Character cr.

This will work when the line ending is either cr or lf+cr. However, it won't work if the line ending is just lf.

For example, in my parsers, the newline implementation looks like:
(#cr asParser , #lf asParser optional) / #lf asParser / #cr asParser


What do you think?

Cheers,
Doru

--

"Every thing has its own flow"

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev



_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev




--

"Every thing has its own flow"

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev



_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev
Reply | Threaded
Open this post in threaded view
|

Re: start of line parser

Tudor Girba-2
This looks good!

I think that if we have a new line at the end of the stream, we should consider the last character a #startOfLine as well. This will have not much of an effect anyway, given that we will likely always use it in sequence with another parser.

Thanks,
Doru



On Wed, Sep 3, 2014 at 5:23 PM, Jan Kurš <[hidden email]> wrote:
Ok, I ended up with this solution:


isStartOfLine
    (position = 0) ifTrue: [ ^ true ].

    self insideCRLF ifTrue: [ ^ false ].
   
    ^ (self peekBack = Character cr) or: [ self peekBack = Character lf].



The question that remains is, how to handle a situation when CR/LF/CRLF is at the end of the stream. Is it still a new line?

Cheers,
Jan


On 3 September 2014 15:52, Tudor Girba <[hidden email]> wrote:
I think this is not enough, because when you have cr+lf, you do not want to be true on cr. I think we should go with the the same rule as in the parser I proposed, which means that we need to reason about two characters.

Doru


On Wed, Sep 3, 2014 at 3:38 PM, Jan Kurš <[hidden email]> wrote:
@Doru: This might solve the problem:


isStartOfLine
(position = 0) ifTrue: [ ^ true ].
        ^ ((collection at: position) = Character cr) or: [ (collection at: position) = Character lf ].

Do you agree?

Cheers,
Jan


On 3 September 2014 14:46, Tudor Girba <[hidden email]> wrote:
Hi,

Jan has implemented a nice parser that allows us to specify easily productions that depend on text having to be on a new line.

For example, in Pillar, a header is specified through an exclamation mark starting on a new line. Now, you can parse this as:

#startOfLine asParser , $! asParser , #newline asParser negate star

Pretty cool!


@Jan: I think we need to enhance a bit the implementation. The implementation looks like this right now:
isStartOfLine
(position = 0) ifTrue: [ ^ true ].
^ (collection at: position) = Character cr.

This will work when the line ending is either cr or lf+cr. However, it won't work if the line ending is just lf.

For example, in my parsers, the newline implementation looks like:
(#cr asParser , #lf asParser optional) / #lf asParser / #cr asParser


What do you think?

Cheers,
Doru

--

"Every thing has its own flow"

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev



_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev




--

"Every thing has its own flow"

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev



_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev




--

"Every thing has its own flow"

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev
Reply | Threaded
Open this post in threaded view
|

Re: start of line parser

Jan Kurš
OK, lets keep it this way, if it will be problem in a future, I can add the extra check for the last line.

Cheers,
Jan


On 4 September 2014 08:00, Tudor Girba <[hidden email]> wrote:
This looks good!

I think that if we have a new line at the end of the stream, we should consider the last character a #startOfLine as well. This will have not much of an effect anyway, given that we will likely always use it in sequence with another parser.

Thanks,
Doru



On Wed, Sep 3, 2014 at 5:23 PM, Jan Kurš <[hidden email]> wrote:
Ok, I ended up with this solution:


isStartOfLine
    (position = 0) ifTrue: [ ^ true ].

    self insideCRLF ifTrue: [ ^ false ].
   
    ^ (self peekBack = Character cr) or: [ self peekBack = Character lf].



The question that remains is, how to handle a situation when CR/LF/CRLF is at the end of the stream. Is it still a new line?

Cheers,
Jan


On 3 September 2014 15:52, Tudor Girba <[hidden email]> wrote:
I think this is not enough, because when you have cr+lf, you do not want to be true on cr. I think we should go with the the same rule as in the parser I proposed, which means that we need to reason about two characters.

Doru


On Wed, Sep 3, 2014 at 3:38 PM, Jan Kurš <[hidden email]> wrote:
@Doru: This might solve the problem:


isStartOfLine
(position = 0) ifTrue: [ ^ true ].
        ^ ((collection at: position) = Character cr) or: [ (collection at: position) = Character lf ].

Do you agree?

Cheers,
Jan


On 3 September 2014 14:46, Tudor Girba <[hidden email]> wrote:
Hi,

Jan has implemented a nice parser that allows us to specify easily productions that depend on text having to be on a new line.

For example, in Pillar, a header is specified through an exclamation mark starting on a new line. Now, you can parse this as:

#startOfLine asParser , $! asParser , #newline asParser negate star

Pretty cool!


@Jan: I think we need to enhance a bit the implementation. The implementation looks like this right now:
isStartOfLine
(position = 0) ifTrue: [ ^ true ].
^ (collection at: position) = Character cr.

This will work when the line ending is either cr or lf+cr. However, it won't work if the line ending is just lf.

For example, in my parsers, the newline implementation looks like:
(#cr asParser , #lf asParser optional) / #lf asParser / #cr asParser


What do you think?

Cheers,
Doru

--

"Every thing has its own flow"

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev



_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev




--

"Every thing has its own flow"

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev



_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev




--

"Every thing has its own flow"

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev



_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev