PPParser not returning file contents

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

PPParser not returning file contents

Usman Bhatti
Hello all,


We are using petitParser with a Stream by the parse method on a PPParser. This parse method calls asPetitStream on our stream which is supposed to return a stream with a valid format for PPParser. The problem is that this stream do not represent our test file but a lot of white spaces. this code: 

(StandardFileStream fileNamed: '/some/path/to/file/atextfile.txt' ) asPetitStream proves that.

Did someone else have a similar problem with petit parser?

thanx,

Usman & Guillaume 

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev
Reply | Threaded
Open this post in threaded view
|

Re: PPParser not returning file contents

Lukas Renggli
I do not understand what you are saying or asking.

Normally #asPetitStream should convert any non-memory stream to an
efficient in-memory representation. Maybe this does not work properly
in the Pharo version you are using? Or maybe your file is too big to
be loaded into memory?

While it is technically possible to use PetitParser on an external
stream (such as a file or socket stream) it is not advisable.
PetitParser requires highly efficient random access to the parsed data
and anything else than an indexed in-memory collection slows down the
parsing by magnitudes.

Lukas

On 13 February 2012 13:11, Usman Bhatti <[hidden email]> wrote:

> Hello all,
>
>
> We are using petitParser with a Stream by the parse method on a PPParser.
> This parse method calls asPetitStream on our stream which is supposed to
> return a stream with a valid format for PPParser. The problem is that this
> stream do not represent our test file but a lot of white spaces. this code:
>
> (StandardFileStream fileNamed: '/some/path/to/file/atextfile.txt' )
> asPetitStream proves that.
>
> Did someone else have a similar problem with petit parser?
>
> thanx,
>
> Usman & Guillaume
>
> _______________________________________________
> Moose-dev mailing list
> [hidden email]
> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>



--
Lukas Renggli
www.lukas-renggli.ch
_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev
Reply | Threaded
Open this post in threaded view
|

Re: PPParser not returning file contents

Usman Bhatti
After further analysis, it seems that the problem comes from asPetitStream in PositionnableStream. This method checks if the attribute collection is nil and uses the collection if it is not nil. In our case, the collection is not nil but does not contain our file data, it only contains whitespaces. The problem is about the behavior of our FileStream where before reading, collection is not nil. 

The question: does anyone have seen a similar problem with PP.

thanx

On Mon, Feb 13, 2012 at 1:40 PM, Lukas Renggli <[hidden email]> wrote:
I do not understand what you are saying or asking.

Normally #asPetitStream should convert any non-memory stream to an
efficient in-memory representation. Maybe this does not work properly
in the Pharo version you are using? Or maybe your file is too big to
be loaded into memory?

While it is technically possible to use PetitParser on an external
stream (such as a file or socket stream) it is not advisable.
PetitParser requires highly efficient random access to the parsed data
and anything else than an indexed in-memory collection slows down the
parsing by magnitudes.

Lukas

On 13 February 2012 13:11, Usman Bhatti <[hidden email]> wrote:
> Hello all,
>
>
> We are using petitParser with a Stream by the parse method on a PPParser.
> This parse method calls asPetitStream on our stream which is supposed to
> return a stream with a valid format for PPParser. The problem is that this
> stream do not represent our test file but a lot of white spaces. this code:
>
> (StandardFileStream fileNamed: '/some/path/to/file/atextfile.txt' )
> asPetitStream proves that.
>
> Did someone else have a similar problem with petit parser?
>
> thanx,
>
> Usman & Guillaume
>
> _______________________________________________
> Moose-dev mailing list
> [hidden email]
> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>



--
Lukas Renggli
www.lukas-renggli.ch
_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev


_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev
Reply | Threaded
Open this post in threaded view
|

Re: PPParser not returning file contents

Lukas Renggli
Ahh, of course as a temporary workaround you can use:

   aParser parse: (StandardFileStream fileNamed:
'/some/path/to/file/atextfile.txt') contents

Lukas

On 13 February 2012 15:33, Usman Bhatti <[hidden email]> wrote:

> After further analysis, it seems that the problem comes from asPetitStream
> in PositionnableStream. This method checks if the attribute collection is
> nil and uses the collection if it is not nil. In our case, the collection is
> not nil but does not contain our file data, it only contains whitespaces.
> The problem is about the behavior of our FileStream where before reading,
> collection is not nil.
>
> The question: does anyone have seen a similar problem with PP.
>
> thanx
>
> On Mon, Feb 13, 2012 at 1:40 PM, Lukas Renggli <[hidden email]> wrote:
>>
>> I do not understand what you are saying or asking.
>>
>> Normally #asPetitStream should convert any non-memory stream to an
>> efficient in-memory representation. Maybe this does not work properly
>> in the Pharo version you are using? Or maybe your file is too big to
>> be loaded into memory?
>>
>> While it is technically possible to use PetitParser on an external
>> stream (such as a file or socket stream) it is not advisable.
>> PetitParser requires highly efficient random access to the parsed data
>> and anything else than an indexed in-memory collection slows down the
>> parsing by magnitudes.
>>
>> Lukas
>>
>> On 13 February 2012 13:11, Usman Bhatti <[hidden email]> wrote:
>> > Hello all,
>> >
>> >
>> > We are using petitParser with a Stream by the parse method on a
>> > PPParser.
>> > This parse method calls asPetitStream on our stream which is supposed to
>> > return a stream with a valid format for PPParser. The problem is that
>> > this
>> > stream do not represent our test file but a lot of white spaces. this
>> > code:
>> >
>> > (StandardFileStream fileNamed: '/some/path/to/file/atextfile.txt' )
>> > asPetitStream proves that.
>> >
>> > Did someone else have a similar problem with petit parser?
>> >
>> > thanx,
>> >
>> > Usman & Guillaume
>> >
>> > _______________________________________________
>> > Moose-dev mailing list
>> > [hidden email]
>> > https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>> >
>>
>>
>>
>> --
>> Lukas Renggli
>> www.lukas-renggli.ch
>> _______________________________________________
>> Moose-dev mailing list
>> [hidden email]
>> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>
>
>
> _______________________________________________
> Moose-dev mailing list
> [hidden email]
> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>



--
Lukas Renggli
www.lukas-renggli.ch
_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev
Reply | Threaded
Open this post in threaded view
|

Re: PPParser not returning file contents

Lukas Renggli
Try loading

   Name: PetitParser-lr.227
   Author: lr
   Time: 13 February 2012, 8:27:11 pm
   UUID: 89e97023-d891-400e-b657-2ee63449acb3
   Ancestors: PetitParser-lr.226

   - fixed the ambiguous access to 'collection' in PositionableStream

I think that should fix your problem.

Lukas

On 13 February 2012 15:54, Lukas Renggli <[hidden email]> wrote:

> Ahh, of course as a temporary workaround you can use:
>
>   aParser parse: (StandardFileStream fileNamed:
> '/some/path/to/file/atextfile.txt') contents
>
> Lukas
>
> On 13 February 2012 15:33, Usman Bhatti <[hidden email]> wrote:
>> After further analysis, it seems that the problem comes from asPetitStream
>> in PositionnableStream. This method checks if the attribute collection is
>> nil and uses the collection if it is not nil. In our case, the collection is
>> not nil but does not contain our file data, it only contains whitespaces.
>> The problem is about the behavior of our FileStream where before reading,
>> collection is not nil.
>>
>> The question: does anyone have seen a similar problem with PP.
>>
>> thanx
>>
>> On Mon, Feb 13, 2012 at 1:40 PM, Lukas Renggli <[hidden email]> wrote:
>>>
>>> I do not understand what you are saying or asking.
>>>
>>> Normally #asPetitStream should convert any non-memory stream to an
>>> efficient in-memory representation. Maybe this does not work properly
>>> in the Pharo version you are using? Or maybe your file is too big to
>>> be loaded into memory?
>>>
>>> While it is technically possible to use PetitParser on an external
>>> stream (such as a file or socket stream) it is not advisable.
>>> PetitParser requires highly efficient random access to the parsed data
>>> and anything else than an indexed in-memory collection slows down the
>>> parsing by magnitudes.
>>>
>>> Lukas
>>>
>>> On 13 February 2012 13:11, Usman Bhatti <[hidden email]> wrote:
>>> > Hello all,
>>> >
>>> >
>>> > We are using petitParser with a Stream by the parse method on a
>>> > PPParser.
>>> > This parse method calls asPetitStream on our stream which is supposed to
>>> > return a stream with a valid format for PPParser. The problem is that
>>> > this
>>> > stream do not represent our test file but a lot of white spaces. this
>>> > code:
>>> >
>>> > (StandardFileStream fileNamed: '/some/path/to/file/atextfile.txt' )
>>> > asPetitStream proves that.
>>> >
>>> > Did someone else have a similar problem with petit parser?
>>> >
>>> > thanx,
>>> >
>>> > Usman & Guillaume
>>> >
>>> > _______________________________________________
>>> > Moose-dev mailing list
>>> > [hidden email]
>>> > https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>>> >
>>>
>>>
>>>
>>> --
>>> Lukas Renggli
>>> www.lukas-renggli.ch
>>> _______________________________________________
>>> Moose-dev mailing list
>>> [hidden email]
>>> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>>
>>
>>
>> _______________________________________________
>> Moose-dev mailing list
>> [hidden email]
>> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>>
>
>
>
> --
> Lukas Renggli
> www.lukas-renggli.ch



--
Lukas Renggli
www.lukas-renggli.ch

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev
Reply | Threaded
Open this post in threaded view
|

Re: PPParser not returning file contents

Stéphane Ducasse
thanks lukas
We are experimenting PP as a island grammar support.
So I guess that we will ask a lot of questions and learn.

Stef


On Feb 13, 2012, at 8:28 PM, Lukas Renggli wrote:

> Try loading
>
>   Name: PetitParser-lr.227
>   Author: lr
>   Time: 13 February 2012, 8:27:11 pm
>   UUID: 89e97023-d891-400e-b657-2ee63449acb3
>   Ancestors: PetitParser-lr.226
>
>   - fixed the ambiguous access to 'collection' in PositionableStream
>
> I think that should fix your problem.
>
> Lukas
>
> On 13 February 2012 15:54, Lukas Renggli <[hidden email]> wrote:
>> Ahh, of course as a temporary workaround you can use:
>>
>>   aParser parse: (StandardFileStream fileNamed:
>> '/some/path/to/file/atextfile.txt') contents
>>
>> Lukas
>>
>> On 13 February 2012 15:33, Usman Bhatti <[hidden email]> wrote:
>>> After further analysis, it seems that the problem comes from asPetitStream
>>> in PositionnableStream. This method checks if the attribute collection is
>>> nil and uses the collection if it is not nil. In our case, the collection is
>>> not nil but does not contain our file data, it only contains whitespaces.
>>> The problem is about the behavior of our FileStream where before reading,
>>> collection is not nil.
>>>
>>> The question: does anyone have seen a similar problem with PP.
>>>
>>> thanx
>>>
>>> On Mon, Feb 13, 2012 at 1:40 PM, Lukas Renggli <[hidden email]> wrote:
>>>>
>>>> I do not understand what you are saying or asking.
>>>>
>>>> Normally #asPetitStream should convert any non-memory stream to an
>>>> efficient in-memory representation. Maybe this does not work properly
>>>> in the Pharo version you are using? Or maybe your file is too big to
>>>> be loaded into memory?
>>>>
>>>> While it is technically possible to use PetitParser on an external
>>>> stream (such as a file or socket stream) it is not advisable.
>>>> PetitParser requires highly efficient random access to the parsed data
>>>> and anything else than an indexed in-memory collection slows down the
>>>> parsing by magnitudes.
>>>>
>>>> Lukas
>>>>
>>>> On 13 February 2012 13:11, Usman Bhatti <[hidden email]> wrote:
>>>>> Hello all,
>>>>>
>>>>>
>>>>> We are using petitParser with a Stream by the parse method on a
>>>>> PPParser.
>>>>> This parse method calls asPetitStream on our stream which is supposed to
>>>>> return a stream with a valid format for PPParser. The problem is that
>>>>> this
>>>>> stream do not represent our test file but a lot of white spaces. this
>>>>> code:
>>>>>
>>>>> (StandardFileStream fileNamed: '/some/path/to/file/atextfile.txt' )
>>>>> asPetitStream proves that.
>>>>>
>>>>> Did someone else have a similar problem with petit parser?
>>>>>
>>>>> thanx,
>>>>>
>>>>> Usman & Guillaume
>>>>>
>>>>> _______________________________________________
>>>>> Moose-dev mailing list
>>>>> [hidden email]
>>>>> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Lukas Renggli
>>>> www.lukas-renggli.ch
>>>> _______________________________________________
>>>> Moose-dev mailing list
>>>> [hidden email]
>>>> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> Moose-dev mailing list
>>> [hidden email]
>>> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>>>
>>
>>
>>
>> --
>> Lukas Renggli
>> www.lukas-renggli.ch
>
>
>
> --
> Lukas Renggli
> www.lukas-renggli.ch
>
> _______________________________________________
> Moose-dev mailing list
> [hidden email]
> https://www.iam.unibe.ch/mailman/listinfo/moose-dev


_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev
Reply | Threaded
Open this post in threaded view
|

Re: PPParser not returning file contents

Lukas Renggli
> We are experimenting PP as a island grammar support.
> So I guess that we will ask a lot of questions and learn.

That should be a relatively simple task, if you have your island well defined.

For example

   p :=  '/**' asParser , '*/' asParser negate star , '*/' asParser.
   p flatten matchesSkipIn: aLongJavaFile

should give you all JavaDoc.

If you want, we can have a look at more complicated examples this Saturday :-)

Lukas

--
Lukas Renggli
www.lukas-renggli.ch
_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev