Smalltalk › Pharo › Pharo Smalltalk Users

Stream API

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

31 messages Options

Evan Donahue

Stream API

I've heard mention once or twice on this list and in some release notes of what sounded like possible coming changes to the stream API. Could anyone point me to any concrete details about that? I haven't been able to dig anything up myself by searching. I'm about to write something that I'd like to be polymorphic with the stream API, but if that's about to change, I'd like to plan ahead.

Thanks,

Evan

Stephane Ducasse-3

Re: Stream API

Hi Evan

I think that we will use the ZnStreams.
If we use Xtreams we will transform their API because some messages
are not really good.
Stef

On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <[hidden email]> wrote:
> I've heard mention once or twice on this list and in some release notes of
> what sounded like possible coming changes to the stream API. Could anyone
> point me to any concrete details about that? I haven't been able to dig
> anything up myself by searching. I'm about to write something that I'd like
> to be polymorphic with the stream API, but if that's about to change, I'd
> like to plan ahead.
>
> Thanks,
> Evan

aglynn42

Re: Stream API

Xtreams has very good performance, but the API’s are messy. I haven’t compared the performance of ZnStreams, but it shouldn’t be radically different.

From: [hidden email]
Sent: Monday, November 13, 2017 1:59 PM
To: [hidden email]
Subject: Re: [Pharo-users] Stream API

Hi Evan

I think that we will use the ZnStreams.

If we use Xtreams we will transform their API because some messages

are not really good.

Stef

On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <[hidden email]> wrote:

> I've heard mention once or twice on this list and in some release notes of

> what sounded like possible coming changes to the stream API. Could anyone

> point me to any concrete details about that? I haven't been able to dig

> anything up myself by searching. I'm about to write something that I'd like

> to be polymorphic with the stream API, but if that's about to change, I'd

> like to plan ahead.

> Thanks,

> Evan

Sven Van Caekenberghe-2

Re: Stream API

In reply to this post by Stephane Ducasse-3

The idea is to have much simpler streams which can be composed to get more sophisticated behaviour.

The most primitive streams should be binary read or write streams, like a raw file or network connection.

To add a character encoding/decoding you wrap them in a ZnCharacterReadStream or ZnCharacterWriteStream (these use the newer, cleaner ZnCharacterEncoders).

If you want buffering, you wrap a ZnBufferedReadStream or ZnBufferedWriteStream around them.

And there are some other examples in the system too.

Have a look at BinaryFileStream and ZdcSocketStream.

Simply put, MultiByteFileStream and MultiByteBinaryOrTextStream must die, because they try to be everything at once and are impossible to change.

The contract of a stream should be much, much simpler than it is today.

For writing that means

#nextPut:
#nextPutAll:
#next:putAll:
#next:putAll:startingAt:

the 3 last ones can be written in terms of of the first one, but the last one is key because it can be the most efficient.
And maybe also

#flush
#close

Some helpers for character writing are

#space
#tab
#cr
#crlf
#lf

Maybe #newline

#<< is a handy method too.

For reading that means

#atEnd
#next
#next:
#next:into:
#next:into:startingAt:
#nextInto:
#peek
#skip:
#upToEnd
#upTo:
#readInto:startingAt:count:

Again, they can all be written in terms of #next, but #readInto:startingAt:count: is the core, efficient one.
Note that #peek allows a one character lookahead, which should be sufficient for almost all parsing needs.

#close is also a necessary operation, #peekFor: a handy one, #nextLine is popular too.

There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.

If you absolutely need these, read #upToEnd and use a regular ReadStream (over a fixed collection).

The collection based classic Streams should always remain in the system, they are too handy. But have you seen for example, #nextInt32 on PositionableStream ? Good luck with that when the the underlying collection is anything other than bytes.

All this being said, there is no one, single correct answer.

But if we all try to simplify what we expect of streams (use a more limited API), we'll be more nimble to make implementation changes later on.

Sven

> On 13 Nov 2017, at 19:58, Stephane Ducasse <[hidden email]> wrote:
>
> Hi Evan
>
> I think that we will use the ZnStreams.
> If we use Xtreams we will transform their API because some messages
> are not really good.
> Stef
>
> On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <[hidden email]> wrote:
>> I've heard mention once or twice on this list and in some release notes of
>> what sounded like possible coming changes to the stream API. Could anyone
>> point me to any concrete details about that? I haven't been able to dig
>> anything up myself by searching. I'm about to write something that I'd like
>> to be polymorphic with the stream API, but if that's about to change, I'd
>> like to plan ahead.
>>
>> Thanks,
>> Evan
>

Stephane Ducasse-3

Re: Stream API

On Mon, Nov 13, 2017 at 8:27 PM, Sven Van Caekenberghe <[hidden email]> wrote:
> The idea is to have much simpler streams which can be composed to get more sophisticated behaviour.
>
> The most primitive streams should be binary read or write streams, like a raw file or network connection.
>
> To add a character encoding/decoding you wrap them in a ZnCharacterReadStream or ZnCharacterWriteStream (these use the newer, cleaner ZnCharacterEncoders).

Yes really nice :)

And Guille started to use them and we are slowly rewriting all the
stream internal users to use Zn and after we will be free.

> If you want buffering, you wrap a ZnBufferedReadStream or ZnBufferedWriteStream around them.
>
> And there are some other examples in the system too.
>
> Have a look at BinaryFileStream and ZdcSocketStream.
>
> Simply put, MultiByteFileStream and MultiByteBinaryOrTextStream must die, because they try to be everything at once and are impossible to change.

YES YES YES and celebrate. I could never understand anything. My brain
is too limited for these kind of games :)

> The contract of a stream should be much, much simpler than it is today.

Fully agree.

>
> For writing that means
>
> #nextPut:
> #nextPutAll:
> #next:putAll:
> #next:putAll:startingAt:
>
> the 3 last ones can be written in terms of of the first one, but the last one is key because it can be the most efficient.
> And maybe also
>
> #flush
> #close
>
> Some helpers for character writing are
>
> #space
> #tab
> #cr
> #crlf
> #lf
>
> Maybe #newline

>
> #<< is a handy method too.
>
> For reading that means
>
> #atEnd
> #next
> #next:
> #next:into:
> #next:into:startingAt:
> #nextInto:
> #peek
> #skip:
> #upToEnd
> #upTo:
> #readInto:startingAt:count:
>
> Again, they can all be written in terms of #next, but #readInto:startingAt:count: is the core, efficient one.
> Note that #peek allows a one character lookahead, which should be sufficient for almost all parsing needs.
>
> #close is also a necessary operation, #peekFor: a handy one, #nextLine is popular too.
>
> There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.
>
> If you absolutely need these, read #upToEnd and use a regular ReadStream (over a fixed collection).
>
> The collection based classic Streams should always remain in the system, they are too handy. But have you seen for example, #nextInt32 on PositionableStream ? Good luck with that when the the underlying collection is anything other than bytes.
>
> All this being said, there is no one, single correct answer.
>
> But if we all try to simplify what we expect of streams (use a more limited API), we'll be more nimble to make implementation changes later on.
>
> Sven
>
>> On 13 Nov 2017, at 19:58, Stephane Ducasse <[hidden email]> wrote:
>>
>> Hi Evan
>>
>> I think that we will use the ZnStreams.
>> If we use Xtreams we will transform their API because some messages
>> are not really good.
>> Stef
>>
>> On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <[hidden email]> wrote:
>>> I've heard mention once or twice on this list and in some release notes of
>>> what sounded like possible coming changes to the stream API. Could anyone
>>> point me to any concrete details about that? I haven't been able to dig
>>> anything up myself by searching. I'm about to write something that I'd like
>>> to be polymorphic with the stream API, but if that's about to change, I'd
>>> like to plan ahead.
>>>
>>> Thanks,
>>> Evan
>>
>
>

Guillermo Polito

Re: Stream API

Thanks Sven, we should take that list of selectors and make a document or Lint rules on them.

On Mon, Nov 13, 2017 at 9:08 PM, Stephane Ducasse <[hidden email]> wrote:

On Mon, Nov 13, 2017 at 8:27 PM, Sven Van Caekenberghe <[hidden email]> wrote:
> The idea is to have much simpler streams which can be composed to get more sophisticated behaviour.
>
> The most primitive streams should be binary read or write streams, like a raw file or network connection.
>
> To add a character encoding/decoding you wrap them in a ZnCharacterReadStream or ZnCharacterWriteStream (these use the newer, cleaner ZnCharacterEncoders).

Yes really nice :)

And Guille started to use them and we are slowly rewriting all the
stream internal users to use Zn and after we will be free.

> If you want buffering, you wrap a ZnBufferedReadStream or ZnBufferedWriteStream around them.
>
> And there are some other examples in the system too.
>
> Have a look at BinaryFileStream and ZdcSocketStream.
>
> Simply put, MultiByteFileStream and MultiByteBinaryOrTextStream must die, because they try to be everything at once and are impossible to change.

YES YES YES and celebrate. I could never understand anything. My brain
is too limited for these kind of games :)

> The contract of a stream should be much, much simpler than it is today.

Fully agree.

>
> For writing that means
>
> #nextPut:
> #nextPutAll:
> #next:putAll:
> #next:putAll:startingAt:
>
> the 3 last ones can be written in terms of of the first one, but the last one is key because it can be the most efficient.
> And maybe also
>
> #flush
> #close
>
> Some helpers for character writing are
>
> #space
> #tab
> #cr
> #crlf
> #lf
>
> Maybe #newline

:)

>
> #<< is a handy method too.
>
> For reading that means
>
> #atEnd
> #next
> #next:
> #next:into:
> #next:into:startingAt:
> #nextInto:
> #peek
> #skip:
> #upToEnd
> #upTo:
> #readInto:startingAt:count:
>
> Again, they can all be written in terms of #next, but #readInto:startingAt:count: is the core, efficient one.
> Note that #peek allows a one character lookahead, which should be sufficient for almost all parsing needs.
>
> #close is also a necessary operation, #peekFor: a handy one, #nextLine is popular too.
>
> There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.
>
> If you absolutely need these, read #upToEnd and use a regular ReadStream (over a fixed collection).
>
> The collection based classic Streams should always remain in the system, they are too handy. But have you seen for example, #nextInt32 on PositionableStream ? Good luck with that when the the underlying collection is anything other than bytes.
>
> All this being said, there is no one, single correct answer.
>
> But if we all try to simplify what we expect of streams (use a more limited API), we'll be more nimble to make implementation changes later on.
>
> Sven
>
>> On 13 Nov 2017, at 19:58, Stephane Ducasse <[hidden email]> wrote:
>>
>> Hi Evan
>>
>> I think that we will use the ZnStreams.
>> If we use Xtreams we will transform their API because some messages
>> are not really good.
>> Stef
>>
>> On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <[hidden email]> wrote:
>>> I've heard mention once or twice on this list and in some release notes of
>>> what sounded like possible coming changes to the stream API. Could anyone
>>> point me to any concrete details about that? I haven't been able to dig
>>> anything up myself by searching. I'm about to write something that I'd like
>>> to be polymorphic with the stream API, but if that's about to change, I'd
>>> like to plan ahead.
>>>
>>> Thanks,
>>> Evan
>>
>
>

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: +33 06 52 70 66 13

NorbertHartl

Re: Stream API

In reply to this post by Stephane Ducasse-3

> Am 13.11.2017 um 21:08 schrieb Stephane Ducasse <[hidden email]>:
>
>> On Mon, Nov 13, 2017 at 8:27 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>> The idea is to have much simpler streams which can be composed to get more sophisticated behaviour.
>>
>> The most primitive streams should be binary read or write streams, like a raw file or network connection.
>>
>> To add a character encoding/decoding you wrap them in a ZnCharacterReadStream or ZnCharacterWriteStream (these use the newer, cleaner ZnCharacterEncoders).
>
> Yes really nice :)
>
> And Guille started to use them and we are slowly rewriting all the
> stream internal users to use Zn and after we will be free.
>
>

No, you will depend on zinc classes. How is that supposed to work in bootstrap?

Norbert

>> If you want buffering, you wrap a ZnBufferedReadStream or ZnBufferedWriteStream around them.
>>
>> And there are some other examples in the system too.
>>
>> Have a look at BinaryFileStream and ZdcSocketStream.
>>
>> Simply put, MultiByteFileStream and MultiByteBinaryOrTextStream must die, because they try to be everything at once and are impossible to change.
>
>
> YES YES YES and celebrate. I could never understand anything. My brain
> is too limited for these kind of games :)
>
>
>
>> The contract of a stream should be much, much simpler than it is today.
>
> Fully agree.
>
>>
>> For writing that means
>>
>> #nextPut:
>> #nextPutAll:
>> #next:putAll:
>> #next:putAll:startingAt:
>>
>> the 3 last ones can be written in terms of of the first one, but the last one is key because it can be the most efficient.
>> And maybe also
>>
>> #flush
>> #close
>>
>> Some helpers for character writing are
>>
>> #space
>> #tab
>> #cr
>> #crlf
>> #lf
>>
>> Maybe #newline
>
> :)
>
>
>>
>> #<< is a handy method too.
>>
>> For reading that means
>>
>> #atEnd
>> #next
>> #next:
>> #next:into:
>> #next:into:startingAt:
>> #nextInto:
>> #peek
>> #skip:
>> #upToEnd
>> #upTo:
>> #readInto:startingAt:count:
>>
>> Again, they can all be written in terms of #next, but #readInto:startingAt:count: is the core, efficient one.
>> Note that #peek allows a one character lookahead, which should be sufficient for almost all parsing needs.
>>
>> #close is also a necessary operation, #peekFor: a handy one, #nextLine is popular too.
>>
>> There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.
>>
>> If you absolutely need these, read #upToEnd and use a regular ReadStream (over a fixed collection).
>>
>> The collection based classic Streams should always remain in the system, they are too handy. But have you seen for example, #nextInt32 on PositionableStream ? Good luck with that when the the underlying collection is anything other than bytes.
>>
>> All this being said, there is no one, single correct answer.
>>
>> But if we all try to simplify what we expect of streams (use a more limited API), we'll be more nimble to make implementation changes later on.
>>
>> Sven
>>
>>> On 13 Nov 2017, at 19:58, Stephane Ducasse <[hidden email]> wrote:
>>>
>>> Hi Evan
>>>
>>> I think that we will use the ZnStreams.
>>> If we use Xtreams we will transform their API because some messages
>>> are not really good.
>>> Stef
>>>
>>>> On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <[hidden email]> wrote:
>>>> I've heard mention once or twice on this list and in some release notes of
>>>> what sounded like possible coming changes to the stream API. Could anyone
>>>> point me to any concrete details about that? I haven't been able to dig
>>>> anything up myself by searching. I'm about to write something that I'd like
>>>> to be polymorphic with the stream API, but if that's about to change, I'd
>>>> like to plan ahead.
>>>>
>>>> Thanks,
>>>> Evan
>>>
>>
>>

Pavel Krivanek-3

Re: Stream API

2017-11-13 23:51 GMT+01:00 Norbert Hartl <[hidden email]>:

> Am 13.11.2017 um 21:08 schrieb Stephane Ducasse <[hidden email]>:
>
>> On Mon, Nov 13, 2017 at 8:27 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>> The idea is to have much simpler streams which can be composed to get more sophisticated behaviour.
>>
>> The most primitive streams should be binary read or write streams, like a raw file or network connection.
>>
>> To add a character encoding/decoding you wrap them in a ZnCharacterReadStream or ZnCharacterWriteStream (these use the newer, cleaner ZnCharacterEncoders).
>
> Yes really nice :)
>
> And Guille started to use them and we are slowly rewriting all the
> stream internal users to use Zn and after we will be free.
>
>
No, you will depend on zinc classes. How is that supposed to work in bootstrap?

In bootstrap we already use the package 'Zinc-Character-Encoding-Core'. It is standalone so there is no dependency issue.

-- Pavel

Norbert

>> If you want buffering, you wrap a ZnBufferedReadStream or ZnBufferedWriteStream around them.
>>
>> And there are some other examples in the system too.
>>
>> Have a look at BinaryFileStream and ZdcSocketStream.
>>
>> Simply put, MultiByteFileStream and MultiByteBinaryOrTextStream must die, because they try to be everything at once and are impossible to change.
>
>
> YES YES YES and celebrate. I could never understand anything. My brain
> is too limited for these kind of games :)
>
>
>
>> The contract of a stream should be much, much simpler than it is today.
>
> Fully agree.
>
>>
>> For writing that means
>>
>> #nextPut:
>> #nextPutAll:
>> #next:putAll:
>> #next:putAll:startingAt:
>>
>> the 3 last ones can be written in terms of of the first one, but the last one is key because it can be the most efficient.
>> And maybe also
>>
>> #flush
>> #close
>>
>> Some helpers for character writing are
>>
>> #space
>> #tab
>> #cr
>> #crlf
>> #lf
>>
>> Maybe #newline
>
> :)
>
>
>>
>> #<< is a handy method too.
>>
>> For reading that means
>>
>> #atEnd
>> #next
>> #next:
>> #next:into:
>> #next:into:startingAt:
>> #nextInto:
>> #peek
>> #skip:
>> #upToEnd
>> #upTo:
>> #readInto:startingAt:count:
>>
>> Again, they can all be written in terms of #next, but #readInto:startingAt:count: is the core, efficient one.
>> Note that #peek allows a one character lookahead, which should be sufficient for almost all parsing needs.
>>
>> #close is also a necessary operation, #peekFor: a handy one, #nextLine is popular too.
>>
>> There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.
>>
>> If you absolutely need these, read #upToEnd and use a regular ReadStream (over a fixed collection).
>>
>> The collection based classic Streams should always remain in the system, they are too handy. But have you seen for example, #nextInt32 on PositionableStream ? Good luck with that when the the underlying collection is anything other than bytes.
>>
>> All this being said, there is no one, single correct answer.
>>
>> But if we all try to simplify what we expect of streams (use a more limited API), we'll be more nimble to make implementation changes later on.
>>
>> Sven
>>
>>> On 13 Nov 2017, at 19:58, Stephane Ducasse <[hidden email]> wrote:
>>>
>>> Hi Evan
>>>
>>> I think that we will use the ZnStreams.
>>> If we use Xtreams we will transform their API because some messages
>>> are not really good.
>>> Stef
>>>
>>>> On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <[hidden email]> wrote:
>>>> I've heard mention once or twice on this list and in some release notes of
>>>> what sounded like possible coming changes to the stream API. Could anyone
>>>> point me to any concrete details about that? I haven't been able to dig
>>>> anything up myself by searching. I'm about to write something that I'd like
>>>> to be polymorphic with the stream API, but if that's about to change, I'd
>>>> like to plan ahead.
>>>>
>>>> Thanks,
>>>> Evan
>>>
>>
>>

Denis Kudriashov

Re: Stream API

In reply to this post by NorbertHartl

What about contributing to zinc streams? Imaging that I will create block based streams, collecting:/selecting streams like in XSteam. Where I should put them?

2017-11-13 23:51 GMT+01:00 Norbert Hartl <[hidden email]>:

> Am 13.11.2017 um 21:08 schrieb Stephane Ducasse <[hidden email]>:
>
>> On Mon, Nov 13, 2017 at 8:27 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>> The idea is to have much simpler streams which can be composed to get more sophisticated behaviour.
>>
>> The most primitive streams should be binary read or write streams, like a raw file or network connection.
>>
>> To add a character encoding/decoding you wrap them in a ZnCharacterReadStream or ZnCharacterWriteStream (these use the newer, cleaner ZnCharacterEncoders).
>
> Yes really nice :)
>
> And Guille started to use them and we are slowly rewriting all the
> stream internal users to use Zn and after we will be free.
>
>
No, you will depend on zinc classes. How is that supposed to work in bootstrap?

Norbert

>> If you want buffering, you wrap a ZnBufferedReadStream or ZnBufferedWriteStream around them.
>>
>> And there are some other examples in the system too.
>>
>> Have a look at BinaryFileStream and ZdcSocketStream.
>>
>> Simply put, MultiByteFileStream and MultiByteBinaryOrTextStream must die, because they try to be everything at once and are impossible to change.
>
>
> YES YES YES and celebrate. I could never understand anything. My brain
> is too limited for these kind of games :)
>
>
>
>> The contract of a stream should be much, much simpler than it is today.
>
> Fully agree.
>
>>
>> For writing that means
>>
>> #nextPut:
>> #nextPutAll:
>> #next:putAll:
>> #next:putAll:startingAt:
>>
>> the 3 last ones can be written in terms of of the first one, but the last one is key because it can be the most efficient.
>> And maybe also
>>
>> #flush
>> #close
>>
>> Some helpers for character writing are
>>
>> #space
>> #tab
>> #cr
>> #crlf
>> #lf
>>
>> Maybe #newline
>
> :)
>
>
>>
>> #<< is a handy method too.
>>
>> For reading that means
>>
>> #atEnd
>> #next
>> #next:
>> #next:into:
>> #next:into:startingAt:
>> #nextInto:
>> #peek
>> #skip:
>> #upToEnd
>> #upTo:
>> #readInto:startingAt:count:
>>
>> Again, they can all be written in terms of #next, but #readInto:startingAt:count: is the core, efficient one.
>> Note that #peek allows a one character lookahead, which should be sufficient for almost all parsing needs.
>>
>> #close is also a necessary operation, #peekFor: a handy one, #nextLine is popular too.
>>
>> There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.
>>
>> If you absolutely need these, read #upToEnd and use a regular ReadStream (over a fixed collection).
>>
>> The collection based classic Streams should always remain in the system, they are too handy. But have you seen for example, #nextInt32 on PositionableStream ? Good luck with that when the the underlying collection is anything other than bytes.
>>
>> All this being said, there is no one, single correct answer.
>>
>> But if we all try to simplify what we expect of streams (use a more limited API), we'll be more nimble to make implementation changes later on.
>>
>> Sven
>>
>>> On 13 Nov 2017, at 19:58, Stephane Ducasse <[hidden email]> wrote:
>>>
>>> Hi Evan
>>>
>>> I think that we will use the ZnStreams.
>>> If we use Xtreams we will transform their API because some messages
>>> are not really good.
>>> Stef
>>>
>>>> On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <[hidden email]> wrote:
>>>> I've heard mention once or twice on this list and in some release notes of
>>>> what sounded like possible coming changes to the stream API. Could anyone
>>>> point me to any concrete details about that? I haven't been able to dig
>>>> anything up myself by searching. I'm about to write something that I'd like
>>>> to be polymorphic with the stream API, but if that's about to change, I'd
>>>> like to plan ahead.
>>>>
>>>> Thanks,
>>>> Evan
>>>
>>
>>

Guillermo Polito

Re: Stream API

To a package next to block?

On Tue, Nov 14, 2017 at 9:16 AM, Denis Kudriashov <[hidden email]> wrote:

What about contributing to zinc streams? Imaging that I will create block based streams, collecting:/selecting streams like in XSteam. Where I should put them?

2017-11-13 23:51 GMT+01:00 Norbert Hartl <[hidden email]>:

> Am 13.11.2017 um 21:08 schrieb Stephane Ducasse <[hidden email]>:
>
>> On Mon, Nov 13, 2017 at 8:27 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>> The idea is to have much simpler streams which can be composed to get more sophisticated behaviour.
>>
>> The most primitive streams should be binary read or write streams, like a raw file or network connection.
>>
>> To add a character encoding/decoding you wrap them in a ZnCharacterReadStream or ZnCharacterWriteStream (these use the newer, cleaner ZnCharacterEncoders).
>
> Yes really nice :)
>
> And Guille started to use them and we are slowly rewriting all the
> stream internal users to use Zn and after we will be free.
>
>
No, you will depend on zinc classes. How is that supposed to work in bootstrap?

Norbert

>> If you want buffering, you wrap a ZnBufferedReadStream or ZnBufferedWriteStream around them.
>>
>> And there are some other examples in the system too.
>>
>> Have a look at BinaryFileStream and ZdcSocketStream.
>>
>> Simply put, MultiByteFileStream and MultiByteBinaryOrTextStream must die, because they try to be everything at once and are impossible to change.
>
>
> YES YES YES and celebrate. I could never understand anything. My brain
> is too limited for these kind of games :)
>
>
>
>> The contract of a stream should be much, much simpler than it is today.
>
> Fully agree.
>
>>
>> For writing that means
>>
>> #nextPut:
>> #nextPutAll:
>> #next:putAll:
>> #next:putAll:startingAt:
>>
>> the 3 last ones can be written in terms of of the first one, but the last one is key because it can be the most efficient.
>> And maybe also
>>
>> #flush
>> #close
>>
>> Some helpers for character writing are
>>
>> #space
>> #tab
>> #cr
>> #crlf
>> #lf
>>
>> Maybe #newline
>
> :)
>
>
>>
>> #<< is a handy method too.
>>
>> For reading that means
>>
>> #atEnd
>> #next
>> #next:
>> #next:into:
>> #next:into:startingAt:
>> #nextInto:
>> #peek
>> #skip:
>> #upToEnd
>> #upTo:
>> #readInto:startingAt:count:
>>
>> Again, they can all be written in terms of #next, but #readInto:startingAt:count: is the core, efficient one.
>> Note that #peek allows a one character lookahead, which should be sufficient for almost all parsing needs.
>>
>> #close is also a necessary operation, #peekFor: a handy one, #nextLine is popular too.
>>
>> There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.
>>
>> If you absolutely need these, read #upToEnd and use a regular ReadStream (over a fixed collection).
>>
>> The collection based classic Streams should always remain in the system, they are too handy. But have you seen for example, #nextInt32 on PositionableStream ? Good luck with that when the the underlying collection is anything other than bytes.
>>
>> All this being said, there is no one, single correct answer.
>>
>> But if we all try to simplify what we expect of streams (use a more limited API), we'll be more nimble to make implementation changes later on.
>>
>> Sven
>>
>>> On 13 Nov 2017, at 19:58, Stephane Ducasse <[hidden email]> wrote:
>>>
>>> Hi Evan
>>>
>>> I think that we will use the ZnStreams.
>>> If we use Xtreams we will transform their API because some messages
>>> are not really good.
>>> Stef
>>>
>>>> On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <[hidden email]> wrote:
>>>> I've heard mention once or twice on this list and in some release notes of
>>>> what sounded like possible coming changes to the stream API. Could anyone
>>>> point me to any concrete details about that? I haven't been able to dig
>>>> anything up myself by searching. I'm about to write something that I'd like
>>>> to be polymorphic with the stream API, but if that's about to change, I'd
>>>> like to plan ahead.
>>>>
>>>> Thanks,
>>>> Evan
>>>
>>
>>

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: +33 06 52 70 66 13

Denis Kudriashov

Re: Stream API

I look at the code, So Zinc provides only binary/character streams. Right?

About contribution: it is in external repository of Sven. Can we contribute with normal process, create pull request into Pharo repo?

2017-11-14 9:36 GMT+01:00 Guillermo Polito <[hidden email]>:

To a package next to block?

On Tue, Nov 14, 2017 at 9:16 AM, Denis Kudriashov <[hidden email]> wrote:
What about contributing to zinc streams? Imaging that I will create block based streams, collecting:/selecting streams like in XSteam. Where I should put them?

2017-11-13 23:51 GMT+01:00 Norbert Hartl <[hidden email]>:

> Am 13.11.2017 um 21:08 schrieb Stephane Ducasse <[hidden email]>:
>
>> On Mon, Nov 13, 2017 at 8:27 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>> The idea is to have much simpler streams which can be composed to get more sophisticated behaviour.
>>
>> The most primitive streams should be binary read or write streams, like a raw file or network connection.
>>
>> To add a character encoding/decoding you wrap them in a ZnCharacterReadStream or ZnCharacterWriteStream (these use the newer, cleaner ZnCharacterEncoders).
>
> Yes really nice :)
>
> And Guille started to use them and we are slowly rewriting all the
> stream internal users to use Zn and after we will be free.
>
>
No, you will depend on zinc classes. How is that supposed to work in bootstrap?

Norbert

>> If you want buffering, you wrap a ZnBufferedReadStream or ZnBufferedWriteStream around them.
>>
>> And there are some other examples in the system too.
>>
>> Have a look at BinaryFileStream and ZdcSocketStream.
>>
>> Simply put, MultiByteFileStream and MultiByteBinaryOrTextStream must die, because they try to be everything at once and are impossible to change.
>
>
> YES YES YES and celebrate. I could never understand anything. My brain
> is too limited for these kind of games :)
>
>
>
>> The contract of a stream should be much, much simpler than it is today.
>
> Fully agree.
>
>>
>> For writing that means
>>
>> #nextPut:
>> #nextPutAll:
>> #next:putAll:
>> #next:putAll:startingAt:
>>
>> the 3 last ones can be written in terms of of the first one, but the last one is key because it can be the most efficient.
>> And maybe also
>>
>> #flush
>> #close
>>
>> Some helpers for character writing are
>>
>> #space
>> #tab
>> #cr
>> #crlf
>> #lf
>>
>> Maybe #newline
>
> :)
>
>
>>
>> #<< is a handy method too.
>>
>> For reading that means
>>
>> #atEnd
>> #next
>> #next:
>> #next:into:
>> #next:into:startingAt:
>> #nextInto:
>> #peek
>> #skip:
>> #upToEnd
>> #upTo:
>> #readInto:startingAt:count:
>>
>> Again, they can all be written in terms of #next, but #readInto:startingAt:count: is the core, efficient one.
>> Note that #peek allows a one character lookahead, which should be sufficient for almost all parsing needs.
>>
>> #close is also a necessary operation, #peekFor: a handy one, #nextLine is popular too.
>>
>> There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.
>>
>> If you absolutely need these, read #upToEnd and use a regular ReadStream (over a fixed collection).
>>
>> The collection based classic Streams should always remain in the system, they are too handy. But have you seen for example, #nextInt32 on PositionableStream ? Good luck with that when the the underlying collection is anything other than bytes.
>>
>> All this being said, there is no one, single correct answer.
>>
>> But if we all try to simplify what we expect of streams (use a more limited API), we'll be more nimble to make implementation changes later on.
>>
>> Sven
>>
>>> On 13 Nov 2017, at 19:58, Stephane Ducasse <[hidden email]> wrote:
>>>
>>> Hi Evan
>>>
>>> I think that we will use the ZnStreams.
>>> If we use Xtreams we will transform their API because some messages
>>> are not really good.
>>> Stef
>>>
>>>> On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <[hidden email]> wrote:
>>>> I've heard mention once or twice on this list and in some release notes of
>>>> what sounded like possible coming changes to the stream API. Could anyone
>>>> point me to any concrete details about that? I haven't been able to dig
>>>> anything up myself by searching. I'm about to write something that I'd like
>>>> to be polymorphic with the stream API, but if that's about to change, I'd
>>>> like to plan ahead.
>>>>
>>>> Thanks,
>>>> Evan
>>>
>>
>>

--

Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13

Pavel Krivanek-3

Re: Stream API

2017-11-14 9:53 GMT+01:00 Denis Kudriashov <[hidden email]>:

I look at the code, So Zinc provides only binary/character streams. Right?

About contribution: it is in external repository of Sven. Can we contribute with normal process, create pull request into Pharo repo?

yes, time to time we make sync with Zinc upstream.

-- Pavel

2017-11-14 9:36 GMT+01:00 Guillermo Polito <[hidden email]>:
To a package next to block?

On Tue, Nov 14, 2017 at 9:16 AM, Denis Kudriashov <[hidden email]> wrote:
What about contributing to zinc streams? Imaging that I will create block based streams, collecting:/selecting streams like in XSteam. Where I should put them?

2017-11-13 23:51 GMT+01:00 Norbert Hartl <[hidden email]>:

> Am 13.11.2017 um 21:08 schrieb Stephane Ducasse <[hidden email]>:
>
>> On Mon, Nov 13, 2017 at 8:27 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>> The idea is to have much simpler streams which can be composed to get more sophisticated behaviour.
>>
>> The most primitive streams should be binary read or write streams, like a raw file or network connection.
>>
>> To add a character encoding/decoding you wrap them in a ZnCharacterReadStream or ZnCharacterWriteStream (these use the newer, cleaner ZnCharacterEncoders).
>
> Yes really nice :)
>
> And Guille started to use them and we are slowly rewriting all the
> stream internal users to use Zn and after we will be free.
>
>
No, you will depend on zinc classes. How is that supposed to work in bootstrap?

Norbert

>> If you want buffering, you wrap a ZnBufferedReadStream or ZnBufferedWriteStream around them.
>>
>> And there are some other examples in the system too.
>>
>> Have a look at BinaryFileStream and ZdcSocketStream.
>>
>> Simply put, MultiByteFileStream and MultiByteBinaryOrTextStream must die, because they try to be everything at once and are impossible to change.
>
>
> YES YES YES and celebrate. I could never understand anything. My brain
> is too limited for these kind of games :)
>
>
>
>> The contract of a stream should be much, much simpler than it is today.
>
> Fully agree.
>
>>
>> For writing that means
>>
>> #nextPut:
>> #nextPutAll:
>> #next:putAll:
>> #next:putAll:startingAt:
>>
>> the 3 last ones can be written in terms of of the first one, but the last one is key because it can be the most efficient.
>> And maybe also
>>
>> #flush
>> #close
>>
>> Some helpers for character writing are
>>
>> #space
>> #tab
>> #cr
>> #crlf
>> #lf
>>
>> Maybe #newline
>
> :)
>
>
>>
>> #<< is a handy method too.
>>
>> For reading that means
>>
>> #atEnd
>> #next
>> #next:
>> #next:into:
>> #next:into:startingAt:
>> #nextInto:
>> #peek
>> #skip:
>> #upToEnd
>> #upTo:
>> #readInto:startingAt:count:
>>
>> Again, they can all be written in terms of #next, but #readInto:startingAt:count: is the core, efficient one.
>> Note that #peek allows a one character lookahead, which should be sufficient for almost all parsing needs.
>>
>> #close is also a necessary operation, #peekFor: a handy one, #nextLine is popular too.
>>
>> There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.
>>
>> If you absolutely need these, read #upToEnd and use a regular ReadStream (over a fixed collection).
>>
>> The collection based classic Streams should always remain in the system, they are too handy. But have you seen for example, #nextInt32 on PositionableStream ? Good luck with that when the the underlying collection is anything other than bytes.
>>
>> All this being said, there is no one, single correct answer.
>>
>> But if we all try to simplify what we expect of streams (use a more limited API), we'll be more nimble to make implementation changes later on.
>>
>> Sven
>>
>>> On 13 Nov 2017, at 19:58, Stephane Ducasse <[hidden email]> wrote:
>>>
>>> Hi Evan
>>>
>>> I think that we will use the ZnStreams.
>>> If we use Xtreams we will transform their API because some messages
>>> are not really good.
>>> Stef
>>>
>>>> On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <[hidden email]> wrote:
>>>> I've heard mention once or twice on this list and in some release notes of
>>>> what sounded like possible coming changes to the stream API. Could anyone
>>>> point me to any concrete details about that? I haven't been able to dig
>>>> anything up myself by searching. I'm about to write something that I'd like
>>>> to be polymorphic with the stream API, but if that's about to change, I'd
>>>> like to plan ahead.
>>>>
>>>> Thanks,
>>>> Evan
>>>
>>
>>

--

Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13

Prof. Andrew P. Black

Re: Stream API

In reply to this post by Sven Van Caekenberghe-2

> On 13 Nov 2017, at 20:27 , Sven Van Caekenberghe <[hidden email]> wrote:
>
> There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.

I agree with that. But I think that general streams can, and should support:

# saveState “answers an opaque cookie”
# restoreState: aCookieObtainedFromTheSameStream

This enables pone to read ahead, and then reset the state to the way that it was. The implementation of the cookie will depend on the implementation of the stream. In the simplest case it may be an integer position (sufficient for an in-core stream); in the more general case it may include a copy of the stream’s buffer, and the cookie obtained by saving the state of any wrapped stream.

Andrew

Sven Van Caekenberghe-2

Re: Stream API

In reply to this post by Denis Kudriashov

> On 14 Nov 2017, at 09:53, Denis Kudriashov <[hidden email]> wrote:
>
> I look at the code, So Zinc provides only binary/character streams. Right?

Yes, Zn streams focus on classic binary(byte) / character streams.

Streaming over arbitrary data is very cool and well covered by the old ones.

> About contribution: it is in external repository of Sven. Can we contribute with normal process, create pull request into Pharo repo?
>
> 2017-11-14 9:36 GMT+01:00 Guillermo Polito <[hidden email]>:
> To a package next to block?
>
> On Tue, Nov 14, 2017 at 9:16 AM, Denis Kudriashov <[hidden email]> wrote:
> What about contributing to zinc streams? Imaging that I will create block based streams, collecting:/selecting streams like in XSteam. Where I should put them?
>
>
> 2017-11-13 23:51 GMT+01:00 Norbert Hartl <[hidden email]>:
>
>
> > Am 13.11.2017 um 21:08 schrieb Stephane Ducasse <[hidden email]>:
> >
> >> On Mon, Nov 13, 2017 at 8:27 PM, Sven Van Caekenberghe <[hidden email]> wrote:
> >> The idea is to have much simpler streams which can be composed to get more sophisticated behaviour.
> >>
> >> The most primitive streams should be binary read or write streams, like a raw file or network connection.
> >>
> >> To add a character encoding/decoding you wrap them in a ZnCharacterReadStream or ZnCharacterWriteStream (these use the newer, cleaner ZnCharacterEncoders).
> >
> > Yes really nice :)
> >
> > And Guille started to use them and we are slowly rewriting all the
> > stream internal users to use Zn and after we will be free.
> >
> >
> No, you will depend on zinc classes. How is that supposed to work in bootstrap?
>
> Norbert
> >> If you want buffering, you wrap a ZnBufferedReadStream or ZnBufferedWriteStream around them.
> >>
> >> And there are some other examples in the system too.
> >>
> >> Have a look at BinaryFileStream and ZdcSocketStream.
> >>
> >> Simply put, MultiByteFileStream and MultiByteBinaryOrTextStream must die, because they try to be everything at once and are impossible to change.
> >
> >
> > YES YES YES and celebrate. I could never understand anything. My brain
> > is too limited for these kind of games :)
> >
> >
> >
> >> The contract of a stream should be much, much simpler than it is today.
> >
> > Fully agree.
> >
> >>
> >> For writing that means
> >>
> >> #nextPut:
> >> #nextPutAll:
> >> #next:putAll:
> >> #next:putAll:startingAt:
> >>
> >> the 3 last ones can be written in terms of of the first one, but the last one is key because it can be the most efficient.
> >> And maybe also
> >>
> >> #flush
> >> #close
> >>
> >> Some helpers for character writing are
> >>
> >> #space
> >> #tab
> >> #cr
> >> #crlf
> >> #lf
> >>
> >> Maybe #newline
> >
> > :)
> >
> >
> >>
> >> #<< is a handy method too.
> >>
> >> For reading that means
> >>
> >> #atEnd
> >> #next
> >> #next:
> >> #next:into:
> >> #next:into:startingAt:
> >> #nextInto:
> >> #peek
> >> #skip:
> >> #upToEnd
> >> #upTo:
> >> #readInto:startingAt:count:
> >>
> >> Again, they can all be written in terms of #next, but #readInto:startingAt:count: is the core, efficient one.
> >> Note that #peek allows a one character lookahead, which should be sufficient for almost all parsing needs.
> >>
> >> #close is also a necessary operation, #peekFor: a handy one, #nextLine is popular too.
> >>
> >> There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.
> >>
> >> If you absolutely need these, read #upToEnd and use a regular ReadStream (over a fixed collection).
> >>
> >> The collection based classic Streams should always remain in the system, they are too handy. But have you seen for example, #nextInt32 on PositionableStream ? Good luck with that when the the underlying collection is anything other than bytes.
> >>
> >> All this being said, there is no one, single correct answer.
> >>
> >> But if we all try to simplify what we expect of streams (use a more limited API), we'll be more nimble to make implementation changes later on.
> >>
> >> Sven
> >>
> >>> On 13 Nov 2017, at 19:58, Stephane Ducasse <[hidden email]> wrote:
> >>>
> >>> Hi Evan
> >>>
> >>> I think that we will use the ZnStreams.
> >>> If we use Xtreams we will transform their API because some messages
> >>> are not really good.
> >>> Stef
> >>>
> >>>> On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <[hidden email]> wrote:
> >>>> I've heard mention once or twice on this list and in some release notes of
> >>>> what sounded like possible coming changes to the stream API. Could anyone
> >>>> point me to any concrete details about that? I haven't been able to dig
> >>>> anything up myself by searching. I'm about to write something that I'd like
> >>>> to be polymorphic with the stream API, but if that's about to change, I'd
> >>>> like to plan ahead.
> >>>>
> >>>> Thanks,
> >>>> Evan
> >>>
> >>
> >>
>
>
>
>
>
> --
>
> Guille Polito
> Research Engineer
>
> Centre de Recherche en Informatique, Signal et Automatique de Lille
> CRIStAL - UMR 9189
> French National Center for Scientific Research - http://www.cnrs.fr
>
> Web: http://guillep.github.io
> Phone: +33 06 52 70 66 13
>

Sven Van Caekenberghe-2

Re: Stream API

In reply to this post by Prof. Andrew P. Black

> On 14 Nov 2017, at 10:20, Prof. Andrew P. Black <[hidden email]> wrote:
>
>
>> On 13 Nov 2017, at 20:27 , Sven Van Caekenberghe <[hidden email]> wrote:
>>
>> There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.
>
> I agree with that. But I think that general streams can, and should support:
>
> # saveState “answers an opaque cookie”
> # restoreState: aCookieObtainedFromTheSameStream
>
> This enables pone to read ahead, and then reset the state to the way that it was. The implementation of the cookie will depend on the implementation of the stream. In the simplest case it may be an integer position (sufficient for an in-core stream); in the more general case it may include a copy of the stream’s buffer, and the cookie obtained by saving the state of any wrapped stream.
>
> Andrew

Yes and no: #saveState / #restoreState: need a possibly unlimited memory/buffer, similar to #position / #position:

They cannot be offered in general (a socket stream is infinite, has no begin/end, in most cases).

But I do think that a wrapper stream could add such feature, we only have to implement one and agree on the API (and the buffer size should probably be limited in size).

I am curious though, why do you need it ?

I mean, I got away (and try hard) to only use 1-element peeking and managed to implement STON, NeoJSON, NeoCSV, Stomp, PostgresSQL, Redis, and other protocols.

A syntax that really requires multi element peeking is not so common.

Sven

Denis Kudriashov

Re: Stream API

In reply to this post by Sven Van Caekenberghe-2

2017-11-14 14:00 GMT+01:00 Sven Van Caekenberghe <[hidden email]>:

> On 14 Nov 2017, at 09:53, Denis Kudriashov <[hidden email]> wrote:
>
> I look at the code, So Zinc provides only binary/character streams. Right?

Yes, Zn streams focus on classic binary(byte) / character streams.

Streaming over arbitrary data is very cool and well covered by the old ones.

While I really like traditional streams I can not agree here. Sometimes it reminds me poor java collections which force writing loops all the time.

For example the most common task in my experience was writing contents of read stream into write stream. And the only possibility now is loop.

From this point of view XStreams really pushes streams to the level of smalltalk collections.

> About contribution: it is in external repository of Sven. Can we contribute with normal process, create pull request into Pharo repo?
>
> 2017-11-14 9:36 GMT+01:00 Guillermo Polito <[hidden email]>:
> To a package next to block?
>
> On Tue, Nov 14, 2017 at 9:16 AM, Denis Kudriashov <[hidden email]> wrote:
> What about contributing to zinc streams? Imaging that I will create block based streams, collecting:/selecting streams like in XSteam. Where I should put them?
>
>
> 2017-11-13 23:51 GMT+01:00 Norbert Hartl <[hidden email]>:
>
>
> > Am 13.11.2017 um 21:08 schrieb Stephane Ducasse <[hidden email]>:
> >
> >> On Mon, Nov 13, 2017 at 8:27 PM, Sven Van Caekenberghe <[hidden email]> wrote:
> >> The idea is to have much simpler streams which can be composed to get more sophisticated behaviour.
> >>
> >> The most primitive streams should be binary read or write streams, like a raw file or network connection.
> >>
> >> To add a character encoding/decoding you wrap them in a ZnCharacterReadStream or ZnCharacterWriteStream (these use the newer, cleaner ZnCharacterEncoders).
> >
> > Yes really nice :)
> >
> > And Guille started to use them and we are slowly rewriting all the
> > stream internal users to use Zn and after we will be free.
> >
> >
> No, you will depend on zinc classes. How is that supposed to work in bootstrap?
>
> Norbert
> >> If you want buffering, you wrap a ZnBufferedReadStream or ZnBufferedWriteStream around them.
> >>
> >> And there are some other examples in the system too.
> >>
> >> Have a look at BinaryFileStream and ZdcSocketStream.
> >>
> >> Simply put, MultiByteFileStream and MultiByteBinaryOrTextStream must die, because they try to be everything at once and are impossible to change.
> >
> >
> > YES YES YES and celebrate. I could never understand anything. My brain
> > is too limited for these kind of games :)
> >
> >
> >
> >> The contract of a stream should be much, much simpler than it is today.
> >
> > Fully agree.
> >
> >>
> >> For writing that means
> >>
> >> #nextPut:
> >> #nextPutAll:
> >> #next:putAll:
> >> #next:putAll:startingAt:
> >>
> >> the 3 last ones can be written in terms of of the first one, but the last one is key because it can be the most efficient.
> >> And maybe also
> >>
> >> #flush
> >> #close
> >>
> >> Some helpers for character writing are
> >>
> >> #space
> >> #tab
> >> #cr
> >> #crlf
> >> #lf
> >>
> >> Maybe #newline
> >
> > :)
> >
> >
> >>
> >> #<< is a handy method too.
> >>
> >> For reading that means
> >>
> >> #atEnd
> >> #next
> >> #next:
> >> #next:into:
> >> #next:into:startingAt:
> >> #nextInto:
> >> #peek
> >> #skip:
> >> #upToEnd
> >> #upTo:
> >> #readInto:startingAt:count:
> >>
> >> Again, they can all be written in terms of #next, but #readInto:startingAt:count: is the core, efficient one.
> >> Note that #peek allows a one character lookahead, which should be sufficient for almost all parsing needs.
> >>
> >> #close is also a necessary operation, #peekFor: a handy one, #nextLine is popular too.
> >>
> >> There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.
> >>
> >> If you absolutely need these, read #upToEnd and use a regular ReadStream (over a fixed collection).
> >>
> >> The collection based classic Streams should always remain in the system, they are too handy. But have you seen for example, #nextInt32 on PositionableStream ? Good luck with that when the the underlying collection is anything other than bytes.
> >>
> >> All this being said, there is no one, single correct answer.
> >>
> >> But if we all try to simplify what we expect of streams (use a more limited API), we'll be more nimble to make implementation changes later on.
> >>
> >> Sven
> >>
> >>> On 13 Nov 2017, at 19:58, Stephane Ducasse <[hidden email]> wrote:
> >>>
> >>> Hi Evan
> >>>
> >>> I think that we will use the ZnStreams.
> >>> If we use Xtreams we will transform their API because some messages
> >>> are not really good.
> >>> Stef
> >>>
> >>>> On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <[hidden email]> wrote:
> >>>> I've heard mention once or twice on this list and in some release notes of
> >>>> what sounded like possible coming changes to the stream API. Could anyone
> >>>> point me to any concrete details about that? I haven't been able to dig
> >>>> anything up myself by searching. I'm about to write something that I'd like
> >>>> to be polymorphic with the stream API, but if that's about to change, I'd
> >>>> like to plan ahead.
> >>>>
> >>>> Thanks,
> >>>> Evan
> >>>
> >>
> >>
>
>
>
>
>
> --
>

> Guille Polito
> Research Engineer
>
> Centre de Recherche en Informatique, Signal et Automatique de Lille
> CRIStAL - UMR 9189
> French National Center for Scientific Research - http://www.cnrs.fr
>
> Web: http://guillep.github.io
> Phone: <a href="tel:%2B33%2006%2052%2070%2066%2013" value="+33652706613">+33 06 52 70 66 13
>

Sven Van Caekenberghe-2

Re: Stream API

> On 14 Nov 2017, at 14:18, Denis Kudriashov <[hidden email]> wrote:
>
>
> 2017-11-14 14:00 GMT+01:00 Sven Van Caekenberghe <[hidden email]>:
>
>
> > On 14 Nov 2017, at 09:53, Denis Kudriashov <[hidden email]> wrote:
> >
> > I look at the code, So Zinc provides only binary/character streams. Right?
>
> Yes, Zn streams focus on classic binary(byte) / character streams.
>
> Streaming over arbitrary data is very cool and well covered by the old ones.
>
> While I really like traditional streams I can not agree here. Sometimes it reminds me poor java collections which force writing loops all the time.
> For example the most common task in my experience was writing contents of read stream into write stream. And the only possibility now is loop.
> From this point of view XStreams really pushes streams to the level of smalltalk collections.

Yes, I agree, Xtreams is much better (but steep learning curve).

I just wanted to point out that my contributions in Zn streams focus and better/simpler byte/character IO.

Improvement on the classic streams are, of course, welcome.

> > About contribution: it is in external repository of Sven. Can we contribute with normal process, create pull request into Pharo repo?
> >
> > 2017-11-14 9:36 GMT+01:00 Guillermo Polito <[hidden email]>:
> > To a package next to block?
> >
> > On Tue, Nov 14, 2017 at 9:16 AM, Denis Kudriashov <[hidden email]> wrote:
> > What about contributing to zinc streams? Imaging that I will create block based streams, collecting:/selecting streams like in XSteam. Where I should put them?
> >
> >
> > 2017-11-13 23:51 GMT+01:00 Norbert Hartl <[hidden email]>:
> >
> >
> > > Am 13.11.2017 um 21:08 schrieb Stephane Ducasse <[hidden email]>:
> > >
> > >> On Mon, Nov 13, 2017 at 8:27 PM, Sven Van Caekenberghe <[hidden email]> wrote:
> > >> The idea is to have much simpler streams which can be composed to get more sophisticated behaviour.
> > >>
> > >> The most primitive streams should be binary read or write streams, like a raw file or network connection.
> > >>
> > >> To add a character encoding/decoding you wrap them in a ZnCharacterReadStream or ZnCharacterWriteStream (these use the newer, cleaner ZnCharacterEncoders).
> > >
> > > Yes really nice :)
> > >
> > > And Guille started to use them and we are slowly rewriting all the
> > > stream internal users to use Zn and after we will be free.
> > >
> > >
> > No, you will depend on zinc classes. How is that supposed to work in bootstrap?
> >
> > Norbert
> > >> If you want buffering, you wrap a ZnBufferedReadStream or ZnBufferedWriteStream around them.
> > >>
> > >> And there are some other examples in the system too.
> > >>
> > >> Have a look at BinaryFileStream and ZdcSocketStream.
> > >>
> > >> Simply put, MultiByteFileStream and MultiByteBinaryOrTextStream must die, because they try to be everything at once and are impossible to change.
> > >
> > >
> > > YES YES YES and celebrate. I could never understand anything. My brain
> > > is too limited for these kind of games :)
> > >
> > >
> > >
> > >> The contract of a stream should be much, much simpler than it is today.
> > >
> > > Fully agree.
> > >
> > >>
> > >> For writing that means
> > >>
> > >> #nextPut:
> > >> #nextPutAll:
> > >> #next:putAll:
> > >> #next:putAll:startingAt:
> > >>
> > >> the 3 last ones can be written in terms of of the first one, but the last one is key because it can be the most efficient.
> > >> And maybe also
> > >>
> > >> #flush
> > >> #close
> > >>
> > >> Some helpers for character writing are
> > >>
> > >> #space
> > >> #tab
> > >> #cr
> > >> #crlf
> > >> #lf
> > >>
> > >> Maybe #newline
> > >
> > > :)
> > >
> > >
> > >>
> > >> #<< is a handy method too.
> > >>
> > >> For reading that means
> > >>
> > >> #atEnd
> > >> #next
> > >> #next:
> > >> #next:into:
> > >> #next:into:startingAt:
> > >> #nextInto:
> > >> #peek
> > >> #skip:
> > >> #upToEnd
> > >> #upTo:
> > >> #readInto:startingAt:count:
> > >>
> > >> Again, they can all be written in terms of #next, but #readInto:startingAt:count: is the core, efficient one.
> > >> Note that #peek allows a one character lookahead, which should be sufficient for almost all parsing needs.
> > >>
> > >> #close is also a necessary operation, #peekFor: a handy one, #nextLine is popular too.
> > >>
> > >> There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.
> > >>
> > >> If you absolutely need these, read #upToEnd and use a regular ReadStream (over a fixed collection).
> > >>
> > >> The collection based classic Streams should always remain in the system, they are too handy. But have you seen for example, #nextInt32 on PositionableStream ? Good luck with that when the the underlying collection is anything other than bytes.
> > >>
> > >> All this being said, there is no one, single correct answer.
> > >>
> > >> But if we all try to simplify what we expect of streams (use a more limited API), we'll be more nimble to make implementation changes later on.
> > >>
> > >> Sven
> > >>
> > >>> On 13 Nov 2017, at 19:58, Stephane Ducasse <[hidden email]> wrote:
> > >>>
> > >>> Hi Evan
> > >>>
> > >>> I think that we will use the ZnStreams.
> > >>> If we use Xtreams we will transform their API because some messages
> > >>> are not really good.
> > >>> Stef
> > >>>
> > >>>> On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <[hidden email]> wrote:
> > >>>> I've heard mention once or twice on this list and in some release notes of
> > >>>> what sounded like possible coming changes to the stream API. Could anyone
> > >>>> point me to any concrete details about that? I haven't been able to dig
> > >>>> anything up myself by searching. I'm about to write something that I'd like
> > >>>> to be polymorphic with the stream API, but if that's about to change, I'd
> > >>>> like to plan ahead.
> > >>>>
> > >>>> Thanks,
> > >>>> Evan
> > >>>
> > >>
> > >>
> >
> >
> >
> >
> >
> > --
> >
> > Guille Polito
> > Research Engineer
> >
> > Centre de Recherche en Informatique, Signal et Automatique de Lille
> > CRIStAL - UMR 9189
> > French National Center for Scientific Research - http://www.cnrs.fr
> >
> > Web: http://guillep.github.io
> > Phone: +33 06 52 70 66 13
> >
>
>
>

Denis Kudriashov

Re: Stream API

2017-11-14 14:25 GMT+01:00 Sven Van Caekenberghe <[hidden email]>:

> Yes, Zn streams focus on classic binary(byte) / character streams.
>
> Streaming over arbitrary data is very cool and well covered by the old ones.
>
> While I really like traditional streams I can not agree here. Sometimes it reminds me poor java collections which force writing loops all the time.
> For example the most common task in my experience was writing contents of read stream into write stream. And the only possibility now is loop.
> From this point of view XStreams really pushes streams to the level of smalltalk collections.

Yes, I agree, Xtreams is much better (but steep learning curve).

I just wanted to point out that my contributions in Zn streams focus and better/simpler byte/character IO.

Yes, and it is really nice.

Interesting how many users we have in system for general streams? (created on arbitrary collections).

From first look the main users are #printOn: methods. But they use string based, so they are not general. What others we have?

Improvement on the classic streams are, of course, welcome.

> > About contribution: it is in external repository of Sven. Can we contribute with normal process, create pull request into Pharo repo?
> >
> > 2017-11-14 9:36 GMT+01:00 Guillermo Polito <[hidden email]>:
> > To a package next to block?
> >
> > On Tue, Nov 14, 2017 at 9:16 AM, Denis Kudriashov <[hidden email]> wrote:
> > What about contributing to zinc streams? Imaging that I will create block based streams, collecting:/selecting streams like in XSteam. Where I should put them?
> >
> >
> > 2017-11-13 23:51 GMT+01:00 Norbert Hartl <[hidden email]>:
> >
> >
> > > Am 13.11.2017 um 21:08 schrieb Stephane Ducasse <[hidden email]>:
> > >
> > >> On Mon, Nov 13, 2017 at 8:27 PM, Sven Van Caekenberghe <[hidden email]> wrote:
> > >> The idea is to have much simpler streams which can be composed to get more sophisticated behaviour.
> > >>
> > >> The most primitive streams should be binary read or write streams, like a raw file or network connection.
> > >>
> > >> To add a character encoding/decoding you wrap them in a ZnCharacterReadStream or ZnCharacterWriteStream (these use the newer, cleaner ZnCharacterEncoders).
> > >
> > > Yes really nice :)
> > >
> > > And Guille started to use them and we are slowly rewriting all the
> > > stream internal users to use Zn and after we will be free.
> > >
> > >
> > No, you will depend on zinc classes. How is that supposed to work in bootstrap?
> >
> > Norbert
> > >> If you want buffering, you wrap a ZnBufferedReadStream or ZnBufferedWriteStream around them.
> > >>
> > >> And there are some other examples in the system too.
> > >>
> > >> Have a look at BinaryFileStream and ZdcSocketStream.
> > >>
> > >> Simply put, MultiByteFileStream and MultiByteBinaryOrTextStream must die, because they try to be everything at once and are impossible to change.
> > >
> > >
> > > YES YES YES and celebrate. I could never understand anything. My brain
> > > is too limited for these kind of games :)
> > >
> > >
> > >
> > >> The contract of a stream should be much, much simpler than it is today.
> > >
> > > Fully agree.
> > >
> > >>
> > >> For writing that means
> > >>
> > >> #nextPut:
> > >> #nextPutAll:
> > >> #next:putAll:
> > >> #next:putAll:startingAt:
> > >>
> > >> the 3 last ones can be written in terms of of the first one, but the last one is key because it can be the most efficient.
> > >> And maybe also
> > >>
> > >> #flush
> > >> #close
> > >>
> > >> Some helpers for character writing are
> > >>
> > >> #space
> > >> #tab
> > >> #cr
> > >> #crlf
> > >> #lf
> > >>
> > >> Maybe #newline
> > >
> > > :)
> > >
> > >
> > >>
> > >> #<< is a handy method too.
> > >>
> > >> For reading that means
> > >>
> > >> #atEnd
> > >> #next
> > >> #next:
> > >> #next:into:
> > >> #next:into:startingAt:
> > >> #nextInto:
> > >> #peek
> > >> #skip:
> > >> #upToEnd
> > >> #upTo:
> > >> #readInto:startingAt:count:
> > >>
> > >> Again, they can all be written in terms of #next, but #readInto:startingAt:count: is the core, efficient one.
> > >> Note that #peek allows a one character lookahead, which should be sufficient for almost all parsing needs.
> > >>
> > >> #close is also a necessary operation, #peekFor: a handy one, #nextLine is popular too.
> > >>
> > >> There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.
> > >>
> > >> If you absolutely need these, read #upToEnd and use a regular ReadStream (over a fixed collection).
> > >>
> > >> The collection based classic Streams should always remain in the system, they are too handy. But have you seen for example, #nextInt32 on PositionableStream ? Good luck with that when the the underlying collection is anything other than bytes.
> > >>
> > >> All this being said, there is no one, single correct answer.
> > >>
> > >> But if we all try to simplify what we expect of streams (use a more limited API), we'll be more nimble to make implementation changes later on.
> > >>
> > >> Sven
> > >>
> > >>> On 13 Nov 2017, at 19:58, Stephane Ducasse <[hidden email]> wrote:
> > >>>
> > >>> Hi Evan
> > >>>
> > >>> I think that we will use the ZnStreams.
> > >>> If we use Xtreams we will transform their API because some messages
> > >>> are not really good.
> > >>> Stef
> > >>>
> > >>>> On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <[hidden email]> wrote:
> > >>>> I've heard mention once or twice on this list and in some release notes of
> > >>>> what sounded like possible coming changes to the stream API. Could anyone
> > >>>> point me to any concrete details about that? I haven't been able to dig
> > >>>> anything up myself by searching. I'm about to write something that I'd like
> > >>>> to be polymorphic with the stream API, but if that's about to change, I'd
> > >>>> like to plan ahead.
> > >>>>
> > >>>> Thanks,
> > >>>> Evan
> > >>>
> > >>
> > >>
> >
> >
> >
> >
> >
> > --
> >
> > Guille Polito
> > Research Engineer
> >
> > Centre de Recherche en Informatique, Signal et Automatique de Lille
> > CRIStAL - UMR 9189
> > French National Center for Scientific Research - http://www.cnrs.fr
> >
> > Web: http://guillep.github.io
> > Phone: <a href="tel:%2B33%2006%2052%2070%2066%2013" value="+33652706613">+33 06 52 70 66 13
> >
>
>
>

Steffen Märcker

Re: Stream API

Hi!

>> Yes, I agree, Xtreams is much better (but steep learning curve).
>>
>> I just wanted to point out that my contributions in Zn streams focus and
>> better/simpler byte/character IO.
>
> Yes, and it is really nice.
> Interesting how many users we have in system for general streams?
> (created on arbitrary collections).

I really think streams (in general) should focus on what they are best at.
Namely, (stepwise) reading and writing from and to various sources, and
buffering for efficiency, too. XStreams does an excellent job here.
However, higher level operations - like collecting, selecting, splitting
(map, filter, partition) and such - should be addressed by other means.
Those operations apply to streams, collections, generators and other data
structures. They can efficiently be implemented independent from the data
structure. By doing so, code duplication can be avoided and the API of
streams, etc. can be kept simple.

Although I won't have time to contribute code, before finishing my thesis,
I'd like to point out, that transducers are here to address exactly this.
The package already works with collections, streams and xstreams.

Best,
Steffen

Sven Van Caekenberghe-2

Re: Stream API

> On 14 Nov 2017, at 15:33, Steffen Märcker <[hidden email]> wrote:
>
> Hi!
>
>>> Yes, I agree, Xtreams is much better (but steep learning curve).
>>>
>>> I just wanted to point out that my contributions in Zn streams focus and
>>> better/simpler byte/character IO.
>>
>> Yes, and it is really nice.
>> Interesting how many users we have in system for general streams? (created on arbitrary collections).
>
> I really think streams (in general) should focus on what they are best at. Namely, (stepwise) reading and writing from and to various sources, and buffering for efficiency, too. XStreams does an excellent job here. However, higher level operations - like collecting, selecting, splitting (map, filter, partition) and such - should be addressed by other means. Those operations apply to streams, collections, generators and other data structures. They can efficiently be implemented independent from the data structure. By doing so, code duplication can be avoided and the API of streams, etc. can be kept simple.
>
> Although I won't have time to contribute code, before finishing my thesis, I'd like to point out, that transducers are here to address exactly this. The package already works with collections, streams and xstreams.

Are transducers the subject of your thesis ?
Any pointers to more information ?

> Best,
> Steffen
>