Re: [Pharo-dev] Changed #atEnd primitive - #atEnd vs #next returning nil

Hi Denis, Hi Sven,

On Apr 4, 2018, at 3:16 AM, Denis Kudriashov <[hidden email]> wrote:

Hi.

#next returning nil is definitely looks bad because we will always have case when nil value will not means the end:

#(1 nil 2) readSteam next; next; atEnd

But I understand that in many cases analysing #next for nil is much more suitable than #atEnd. I am sure that it can be always avoided but it can be not easy to do.

Alternatively we can reify real EndOfStream object which will be used as result of reading operation when stream is at end.

It can provide nice API like #ifEndOfStream:. which will reflect intention in much cleaner way than ifNil: checks. 

There is a much better scheme which is to allow streams to have an end-of-file element in an inst var, "endOfStream".  It is initialised with the stream itself (nil cannot be used as we want it as a value).  If the end of the stream is reached and endOfStream == self an EndIfStream exception is raised.  Otherwise the value of endOfStream is answered.  We can't answer nil because in some applications (streams of objects) nil is a valid element to read from the steam and we need to manufacture a unique sentinel.

We made this change in VisualWorks and it works well.  One can avoid using atEnd and lion for the sentinel instead, which is much cleaner and faster and less problematic than atEnd.

2018-04-04 12:00 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:

> On 4 Apr 2018, at 11:38, Nicolas Cellier <[hidden email]> wrote:
>
> Hi Sven,
> See also discussion at https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/232

Thanks Nicolas, I had already seen parts of it.

Now, I still want image level changes to be based on clear semantic definitions of the abstract stream API, we're not just talking about file or unix streams, the general Smalltalk concept.

For reading, they are IMHO,

#next
#readInto:startingAt:count:
#peek
#atEnd
#upToEnd (can be derived but still the semantics are important in relation to #atEnd)

For writing, we have

#nextPut:
#next:putAll:startingAt:
#flush

For both, we have

#atEnd
#close
#closed (new)

So, I know, #next returning nil exists, but is it universally/officially defined as such ? Where is that documented ?

Positioning, sizing are not universal, IMHO, but should be clearly defined as well.

> 2018-04-04 11:32 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:
> Somehow, somewhere there was a change to the implementation of the primitive called by some streams' #atEnd.
>
> IIRC, someone said it is implemented as 'remaining size being zero' and some virtual unix files like /dev/random are zero sized.
>
> Now, all kinds of changes are being done image size to work around this.
>
> I am a strong believer in simple, real (i.e. infinite) streams, but I am not sure we are doing the right thing here.
>
> Point is, I am not sure #next returning nil is official and universal.
>
> Consider the comments:
>
> Stream>>#next
>   "Answer the next object accessible by the receiver."
>
> ReadStream>>#next
>   "Primitive. Answer the next object in the Stream represented by the
>   receiver. Fail if the collection of this stream is not an Array or a String.
>   Fail if the stream is positioned at its end, or if the position is out of
>   bounds in the collection. Optional. See Object documentation
>   whatIsAPrimitive."
>
> Note how there is no talk about returning nil !
>
> I think we should discuss about this first.
>
> Was the low level change really correct and the right thing to do ?
>
> Note also that a Guille introduced something new, #closed which is related to the difference between having no more elements (maybe right now, like an open network stream) and never ever being able to produce more data.
>
> Sven
>
>
>
>

Hi Denis, Hi Sven,

On Apr 4, 2018, at 3:16 AM, Denis Kudriashov <[hidden email]> wrote:

Hi.

#next returning nil is definitely looks bad because we will always have case when nil value will not means the end:

#(1 nil 2) readSteam next; next; atEnd

But I understand that in many cases analysing #next for nil is much more suitable than #atEnd. I am sure that it can be always avoided but it can be not easy to do.

Alternatively we can reify real EndOfStream object which will be used as result of reading operation when stream is at end.

It can provide nice API like #ifEndOfStream:. which will reflect intention in much cleaner way than ifNil: checks. 

There is a much better scheme which is to allow streams to have an end-of-file element in an inst var, "endOfStream".  It is initialised with the stream itself (nil cannot be used as we want it as a value).  If the end of the stream is reached and endOfStream == self an EndIfStream exception is raised.  Otherwise the value of endOfStream is answered.  We can't answer nil because in some applications (streams of objects) nil is a valid element to read from the steam and we need to manufacture a unique sentinel.

We made this change in VisualWorks and it works well.  One can avoid using atEnd and lion for the sentinel instead, which is much cleaner and faster and less problematic than atEnd.

2018-04-04 12:00 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:

> On 4 Apr 2018, at 11:38, Nicolas Cellier <[hidden email]> wrote:
>
> Hi Sven,
> See also discussion at https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/232

Thanks Nicolas, I had already seen parts of it.

Now, I still want image level changes to be based on clear semantic definitions of the abstract stream API, we're not just talking about file or unix streams, the general Smalltalk concept.

For reading, they are IMHO,

#next
#readInto:startingAt:count:
#peek
#atEnd
#upToEnd (can be derived but still the semantics are important in relation to #atEnd)

For writing, we have

#nextPut:
#next:putAll:startingAt:
#flush

For both, we have

#atEnd
#close
#closed (new)

So, I know, #next returning nil exists, but is it universally/officially defined as such ? Where is that documented ?

Positioning, sizing are not universal, IMHO, but should be clearly defined as well.

> 2018-04-04 11:32 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:
> Somehow, somewhere there was a change to the implementation of the primitive called by some streams' #atEnd.
>
> IIRC, someone said it is implemented as 'remaining size being zero' and some virtual unix files like /dev/random are zero sized.
>
> Now, all kinds of changes are being done image size to work around this.
>
> I am a strong believer in simple, real (i.e. infinite) streams, but I am not sure we are doing the right thing here.
>
> Point is, I am not sure #next returning nil is official and universal.
>
> Consider the comments:
>
> Stream>>#next
>   "Answer the next object accessible by the receiver."
>
> ReadStream>>#next
>   "Primitive. Answer the next object in the Stream represented by the
>   receiver. Fail if the collection of this stream is not an Array or a String.
>   Fail if the stream is positioned at its end, or if the position is out of
>   bounds in the collection. Optional. See Object documentation
>   whatIsAPrimitive."
>
> Note how there is no talk about returning nil !
>
> I think we should discuss about this first.
>
> Was the low level change really correct and the right thing to do ?
>
> Note also that a Guille introduced something new, #closed which is related to the difference between having no more elements (maybe right now, like an open network stream) and never ever being able to produce more data.
>
> Sven
>
>
>
>

2018-04-04 16:12 GMT+02:00 Eliot Miranda <[hidden email]>:

Hi Denis, Hi Sven,

On Apr 4, 2018, at 3:16 AM, Denis Kudriashov <[hidden email]> wrote:

Hi.

#next returning nil is definitely looks bad because we will always have case when nil value will not means the end:

#(1 nil 2) readSteam next; next; atEnd

But I understand that in many cases analysing #next for nil is much more suitable than #atEnd. I am sure that it can be always avoided but it can be not easy to do.

Alternatively we can reify real EndOfStream object which will be used as result of reading operation when stream is at end.

It can provide nice API like #ifEndOfStream:. which will reflect intention in much cleaner way than ifNil: checks. 

There is a much better scheme which is to allow streams to have an end-of-file element in an inst var, "endOfStream".  It is initialised with the stream itself (nil cannot be used as we want it as a value).  If the end of the stream is reached and endOfStream == self an EndIfStream exception is raised.  Otherwise the value of endOfStream is answered.  We can't answer nil because in some applications (streams of objects) nil is a valid element to read from the steam and we need to manufacture a unique sentinel.

We made this change in VisualWorks and it works well.  One can avoid using atEnd and lion for the sentinel instead, which is much cleaner and faster and less problematic than atEnd.

Hi Eliot,

Using self is a nice hack for saving an inst. var. :)

If we want to afford an inst.var. then we can also reify an endOfStreamAction inst.var. like I did in ExtendedStream

    ^endOfStreamAction value

We are then free to use Notification/Error, default value (reified EndOfStream object, nil, ...), block with block return...

The problem I had is when I started using stream wrappers like in Xtreams.

Each wrapper has it's own action, and it's not obvious how we should compose these.

Nicolas

Sorry to double post, I should have respond-to-all as default action...
 

2018-04-04 12:00 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:

> On 4 Apr 2018, at 11:38, Nicolas Cellier <[hidden email]> wrote:
>
> Hi Sven,
> See also discussion at https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/232

Thanks Nicolas, I had already seen parts of it.

Now, I still want image level changes to be based on clear semantic definitions of the abstract stream API, we're not just talking about file or unix streams, the general Smalltalk concept.

For reading, they are IMHO,

#next
#readInto:startingAt:count:
#peek
#atEnd
#upToEnd (can be derived but still the semantics are important in relation to #atEnd)

For writing, we have

#nextPut:
#next:putAll:startingAt:
#flush

For both, we have

#atEnd
#close
#closed (new)

So, I know, #next returning nil exists, but is it universally/officially defined as such ? Where is that documented ?

Positioning, sizing are not universal, IMHO, but should be clearly defined as well.

> 2018-04-04 11:32 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:
> Somehow, somewhere there was a change to the implementation of the primitive called by some streams' #atEnd.
>
> IIRC, someone said it is implemented as 'remaining size being zero' and some virtual unix files like /dev/random are zero sized.
>
> Now, all kinds of changes are being done image size to work around this.
>
> I am a strong believer in simple, real (i.e. infinite) streams, but I am not sure we are doing the right thing here.
>
> Point is, I am not sure #next returning nil is official and universal.
>
> Consider the comments:
>
> Stream>>#next
>   "Answer the next object accessible by the receiver."
>
> ReadStream>>#next
>   "Primitive. Answer the next object in the Stream represented by the
>   receiver. Fail if the collection of this stream is not an Array or a String.
>   Fail if the stream is positioned at its end, or if the position is out of
>   bounds in the collection. Optional. See Object documentation
>   whatIsAPrimitive."
>
> Note how there is no talk about returning nil !
>
> I think we should discuss about this first.
>
> Was the low level change really correct and the right thing to do ?
>
> Note also that a Guille introduced something new, #closed which is related to the difference between having no more elements (maybe right now, like an open network stream) and never ever being able to produce more data.
>
> Sven
>
>
>
>