PetitParser and external streams

Hello,

I am developing a parser with PetitParser and one of the options I would like to try is to get my parser to read from an external stream (file) and produce the output to another external stream (another file)

Is this possible? I found that Stream>>asPetitParser do this:
asPetitStream
        ^ self contents asPetitStream

Thanks! Federico

Federico.Balaguer wrote

Hello,

I am developing a parser with PetitParser and one of the options I would like to try is to get my parser to read from an external stream (file) and produce the output to another external stream (another file)

Is this possible? I found that Stream>>asPetitParser do this:
asPetitStream
        ^ self contents asPetitStream

Thanks! Federico

Am 12.11.2013 um 23:02 schrieb Sean P. DeNigris <[hidden email]>:
Reading from a stream is the normal way to use petit parser. On the output side it isn’t that easy. The parser builds a hierarchical structure with a lot of backtracking so writing sequentially to a stream needs eventually to be postponed until everything has been read.
Producing the output immediately often has the drawback that it is hard to alter the stream in between. So the usual case will be that you make an object representation of your input format. Then you have a hierarchical object structure that you can write on an output stream. You could use the #printOn: method on all your model objects. Your task would then be accomplished by doing

(MyParser parse: inputStream) printOn: outputStream

hope this helps,

Norbert

Thanks Norbert for your kind reply.

My only reminding question is about the input. PetitStream are made from the contents of another stream. So, if I have a readStream on a file and I convert it to PetitStream, it will end up loading the entire content of the file on memory.  

Could it be possible to use a "LazyPetitStream" that is getting the content from the original stream incrementally?

Thanks! Federico

Am 13.11.2013 um 13:15 schrieb Federico.Balaguer <[hidden email]>:
I don’t think so. What I said in my last mail is valid for the input stream as well. Parsing is about trying to match some content in multiple ways. This implies that you need to check if something matches and if it fails try matching something else. That also means you need to backtrack or better you need to get back and forth within a stream. This makes streaming less an option because you’ll always need to buffer and the buffer is likely to be as big as the whole content.
I don’t know what you are trying to achieve but probably there is another solution. Your mail sounds as you are concerned about memory consumption. Assuming further it is probably not a single entity you try to parse but a list of the same entity. In the case you could leave out the outermost star or plus parser and use the parser for a single entity until the end of the stream. So you would use the parser multiple times producing smaller chunks in memory.

Hope this helps,

Norbert

Norbert Hartl wrote

Producing the output immediately often has the drawback that it is hard to alter the stream in between...

I've had success outputting-as-I-go in the ==> block, instead of, or in addition to returning the token...

Can you provide an example? Because I can not imagine how you can determine that you can output something to stream before knowing if the combined expression levels higher matches at all.

norbert
>
> -----
> Cheers,
> Sean
> --
> View this message in context: http://forum.world.st/PetitParser-and-external-streams-tp4721440p4721814.html
> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>