Smalltalk › Pharo › Pharo Smalltalk Developers

ZnBufferedReadStream and #upToAll:

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

8 messages Options

darth-cheney

ZnBufferedReadStream and #upToAll:

Hi all,

Was there a conscious decision not to include the #upToAll: method onZnBufferedReadStream?. This method is really useful for parsing files -- finding sub-patterns of bytes, etc. Perhaps #upToAll: is not the idiomatic way to do this, but I found it very useful to use in the plain old ReadStream classes.

Sub-question: if I wanted to implement something like this and make a PR, would I be submitting to the Pharo dev repository or the Zinc repository?

Eric

ducasse

Re: ZnBufferedReadStream and #upToAll:

Hi eric

> Hi all,
>
> Was there a conscious decision not to include the #upToAll: method onZnBufferedReadStream?. This method is really useful for parsing files -- finding sub-patterns of bytes, etc. Perhaps #upToAll: is not the idiomatic way to do this, but I found it very useful to use in the plain old ReadStream classes.

I imagine that sven did not add it because on infinite stream it does not make sense.
Now it would be good to see how we can have useful extensions. But I let this to sven.
>
> Sub-question: if I wanted to implement something like this and make a PR, would I be submitting to the Pharo dev repository or the Zinc repository?

Right now to sven repo. In the future we want to have better tools to sync back from a project to its components.

Stef
>
> --
> Eric

darth-cheney

Re: ZnBufferedReadStream and #upToAll:

Thanks Stef,

I imagine that sven did not add it because on infinite stream it does not make sense.

Just to clarify my earlier email, I got here was by calling `'/path/to/some/file' asFileReference binaryReadStream` which responds with the ZnBufferedReadStream. I expected to be able to use something like #upToAll: to find binary-formatted headers etc within some encoding structure. It's entirely possible that this was the wrong expectation!

On Sun, Mar 17, 2019 at 10:38 AM ducasse <[hidden email]> wrote:

Hi eric

> Hi all,
>
> Was there a conscious decision not to include the #upToAll: method onZnBufferedReadStream?. This method is really useful for parsing files -- finding sub-patterns of bytes, etc. Perhaps #upToAll: is not the idiomatic way to do this, but I found it very useful to use in the plain old ReadStream classes.

I imagine that sven did not add it because on infinite stream it does not make sense.
Now it would be good to see how we can have useful extensions. But I let this to sven.
>
> Sub-question: if I wanted to implement something like this and make a PR, would I be submitting to the Pharo dev repository or the Zinc repository?

Right now to sven repo. In the future we want to have better tools to sync back from a project to its components.

Stef
>
> --
> Eric

Eric

Sven Van Caekenberghe-2

Re: ZnBufferedReadStream and #upToAll:

Eric,

Someone wrote a clean implementation of #upToAll: that does not use #position: - I forgot who - but it can be found in ZnCharacterReadStream>>#upToAll:

I guess this could be copied over to ZnBufferedReadStream.

The main reason that I was against #upToAll: was the implementation using #position: - but what Stef says it also very true: if you do not find what you are looking for, you will read up to end, which is not possible with network streams - you will hang. I also try to minimise the stream API in the newer streams, which is hard because the existing API is so broad.

Personally, I never needed #upToAll: during parsing (I also always limit lookahead to 1). I am really curious why you need it ?

Most users of #upToAll: search for CRLF, for which I would use a line reader class like ZnLineReader or ZnFastLineReader.

Another point is that, if you absolutely need everything that is in the classic ReadStream (which is much more than a stream as it holds a collection of all its elements), in most cases, you could read your content first (assuming it is of known size) and wrap a classic stream around it.

Sven

> On 17 Mar 2019, at 15:45, Eric Gade <[hidden email]> wrote:
>
> Thanks Stef,
>
> I imagine that sven did not add it because on infinite stream it does not make sense.
>
> Just to clarify my earlier email, I got here was by calling `'/path/to/some/file' asFileReference binaryReadStream` which responds with the ZnBufferedReadStream. I expected to be able to use something like #upToAll: to find binary-formatted headers etc within some encoding structure. It's entirely possible that this was the wrong expectation!
>
> On Sun, Mar 17, 2019 at 10:38 AM ducasse <[hidden email]> wrote:
> Hi eric
>
> > Hi all,
> >
> > Was there a conscious decision not to include the #upToAll: method onZnBufferedReadStream?. This method is really useful for parsing files -- finding sub-patterns of bytes, etc. Perhaps #upToAll: is not the idiomatic way to do this, but I found it very useful to use in the plain old ReadStream classes.
>
> I imagine that sven did not add it because on infinite stream it does not make sense.
> Now it would be good to see how we can have useful extensions. But I let this to sven.
> >
> > Sub-question: if I wanted to implement something like this and make a PR, would I be submitting to the Pharo dev repository or the Zinc repository?
>
> Right now to sven repo. In the future we want to have better tools to sync back from a project to its components.
>
> Stef
> >
> > --
> > Eric
>
>
>
>
>
> --
> Eric

Pharo Smalltalk Developers mailing list

Re: ZnBufferedReadStream and #upToAll:

In reply to this post by ducasse

If that was the case, then it makes no sense to implement #upTo: but not
#upToAll: , right ????

On 2019-03-17 10:37, ducasse wrote:
> Was there a conscious decision not to include the #upToAll: method onZnBufferedReadStream?. This method is really useful for parsing files -- finding sub-patterns of bytes, etc. Perhaps #upToAll: is not the idiomatic way to do this, but I found it very useful to use in the plain old ReadStream classes.
> I imagine that sven did not add it because on infinite stream it does not make sense.
> Now it would be good to see how we can have useful extensions. But I let this to sven.

ducasse

Re: ZnBufferedReadStream and #upToAll:

In reply to this post by ducasse

> On 17 Mar 2019, at 18:28, Benoit St-Jean <[hidden email]> wrote:
>
> If that was the case, then it makes no sense to implement #upTo: but not #upToAll: , right ????

Indeed :)

>
> On 2019-03-17 10:37, ducasse wrote:
>> Was there a conscious decision not to include the #upToAll: method onZnBufferedReadStream?. This method is really useful for parsing files -- finding sub-patterns of bytes, etc. Perhaps #upToAll: is not the idiomatic way to do this, but I found it very useful to use in the plain old ReadStream classes.
>> I imagine that sven did not add it because on infinite stream it does not make sense.
>> Now it would be good to see how we can have useful extensions. But I let this to sven.
>

Stephan Eggermont-3

Re: ZnBufferedReadStream and #upToAll:

In reply to this post by ducasse

ducasse <[hidden email]> wrote:
>
> I imagine that sven did not add it because on infinite stream it does not make sense.
> Now it would be good to see how we can have useful extensions. But I let this to sven.

That sounds like something that would be useful to make explicit:
isInfinite, with blocking, returning nil, returning null objects, or
returning errors as possible expected behaviors?

Stephan

darth-cheney

Re: ZnBufferedReadStream and #upToAll:

Hi Sven,

I am really curious why you need it ?

This came up for me when I was trying to parse out pieces of a PDF file. As you likely know, the PDF format has both text and binary information encoded within it. I had used #upToAll: in the past to find sequences of "magic bytes" that don't occur in the middle of a file. It seems that the buffered read stream is what you get if you send #binaryReadStream to a FileReference. But you're right — I could just convert streams and do it that way (if you have other techniques you use, please send along). #upToAll: would definitely be complicated to implement in the buffered version!

On Mon, Mar 18, 2019 at 5:11 AM Stephan Eggermont <[hidden email]> wrote:

ducasse <[hidden email]> wrote:
>
> I imagine that sven did not add it because on infinite stream it does not make sense.
> Now it would be good to see how we can have useful extensions. But I let this to sven.

That sounds like something that would be useful to make explicit:
isInfinite, with blocking, returning nil, returning null objects, or
returning errors as possible expected behaviors?

Stephan

Eric