Smalltalk › Squeak › Squeak - Dev

Generators on Streams on Generators on Collections on ...

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

28 messages Options

marcel.taeumel

Generators on Streams on Generators on Collections on ...

Hi, there!

Please try the attached change set: stream-chains.cs

It adds the possibility to attach transformations to streams using generators.

Example:

foo := (#(1 2 3 4 5 6) readStream
select: [:ea | ea even])
collect: [:ea | 2 * ea].
foo upToNil. "#(4 8 12)"

It is tested. :-)

Thanks Patrick for the idea.

Best,
Marcel

Chris Muller-3

Re: Generators on Streams on Generators on Collections on ...

This is so Smalltalk. Very cool.
We should consider it for trunk.

On Mon, Apr 11, 2016 at 10:57 AM, marcel.taeumel <[hidden email]> wrote:

> Hi, there!
>
> Please try the attached change set: stream-chains.cs
> <http://forum.world.st/file/n4889389/stream-chains.cs>
>
> It adds the possibility to attach transformations to streams using
> generators.
>
> Example:
>
> foo := (#(1 2 3 4 5 6) readStream
> select: [:ea | ea even])
> collect: [:ea | 2 * ea].
> foo upToNil. "#(4 8 12)"
>
> It is tested. :-)
>
> Thanks Patrick for the idea.
>
> Best,
> Marcel
>
>
>
> --
> View this message in context: http://forum.world.st/Generators-on-Streams-on-Generators-on-Collections-on-tp4889389.html
> Sent from the Squeak - Dev mailing list archive at Nabble.com.
>

Tobias Pape

Re: Generators on Streams on Generators on Collections on ...

In reply to this post by marcel.taeumel

On 11.04.2016, at 17:57, marcel.taeumel <[hidden email]> wrote:

upToNil -> upToEnd? :D

Btw: I really like this api, can we do such things with Xtreams, too?

Best
-Tobias
>
> It is tested. :-)
>
> Thanks Patrick for the idea.
>
> Best,
> Marcel

Eliot Miranda-2

Re: Generators on Streams on Generators on Collections on ...

> On Apr 11, 2016, at 10:59 AM, Tobias Pape <[hidden email]> wrote:
>
>
>> On 11.04.2016, at 17:57, marcel.taeumel <[hidden email]> wrote:
>>
>> Hi, there!
>>
>> Please try the attached change set: stream-chains.cs
>> <http://forum.world.st/file/n4889389/stream-chains.cs>
>>
>> It adds the possibility to attach transformations to streams using
>> generators.
>>
>> Example:
>>
>> foo := (#(1 2 3 4 5 6) readStream
>> select: [:ea | ea even])
>> collect: [:ea | 2 * ea].
>> foo upToNil. "#(4 8 12)"
>
> upToNil -> upToEnd? :D

>
> Btw: I really like this api, can we do such things with Xtreams, too?
>
> Best
> -Tobias
>>
>> It is tested. :-)
>>
>> Thanks Patrick for the idea.
>>
>> Best,
>> Marcel
>
>
>
>

Frank Shearar-3

Re: Generators on Streams on Generators on Collections on ...

In reply to this post by Tobias Pape

On 11 April 2016 at 10:59, Tobias Pape <[hidden email]> wrote:

On 11.04.2016, at 17:57, marcel.taeumel <[hidden email]> wrote:

> Hi, there!
>
> Please try the attached change set: stream-chains.cs
> <http://forum.world.st/file/n4889389/stream-chains.cs>
>
> It adds the possibility to attach transformations to streams using
> generators.
>
> Example:
>
> foo := (#(1 2 3 4 5 6) readStream
> select: [:ea | ea even])
> collect: [:ea | 2 * ea].
> foo upToNil. "#(4 8 12)"

upToNil -> upToEnd? :D

Btw: I really like this api, can we do such things with Xtreams, too?

Yes. I wrote up an intro a few years back, after Michael presented Xtreams at UKSTUG: http://www.lshift.net/blog/2010/11/29/xtreams-framework-for-squeak/

frank

Best
-Tobias

>
> It is tested. :-)
>
> Thanks Patrick for the idea.
>
> Best,
> Marcel

marcel.taeumel

Re: Generators on Streams on Generators on Collections on ...

In reply to this post by Tobias Pape

Hi Tobias,

this version is made for endless streams, not the ones the always operate on a collection but sockets etc. So, the generator never ends and hence #atEnd is never true. I thought that #upToNil might be sufficient and also practical for other streams.

Best,
Marcel

Tobias Pape

Re: Generators on Streams on Generators on Collections on ...

On 12.04.2016, at 08:38, marcel.taeumel <[hidden email]> wrote:

> Hi Tobias,
>
> this version is made for endless streams, not the ones the always operate on
> a collection but sockets etc. So, the generator never ends and hence #atEnd
> is never true. I thought that #upToNil might be sufficient and also
> practical for other streams.
>
> Best,
> Marcel

Sounds reasonable :)
Best regards
-Tobias

Francisco Garau-2

Re: Generators on Streams on Generators on Collections on ...

In reply to this post by marcel.taeumel

I think #all would read much nicer than #upToNil

Cheers
- Francisco

> On 12 Apr 2016, at 07:38, marcel.taeumel <[hidden email]> wrote:
>
> Hi Tobias,
>
> this version is made for endless streams, not the ones the always operate on
> a collection but sockets etc. So, the generator never ends and hence #atEnd
> is never true. I thought that #upToNil might be sufficient and also
> practical for other streams.
>
> Best,
> Marcel
>
>
>
> --
> View this message in context: http://forum.world.st/Generators-on-Streams-on-Generators-on-Collections-on-tp4889389p4889453.html
> Sent from the Squeak - Dev mailing list archive at Nabble.com.
>

marcel.taeumel

Re: Generators on Streams on Generators on Collections on ...

I want it to sound like #upToEnd. :-)

Best,
Marcel

Bert Freudenberg

Re: Generators on Streams on Generators on Collections on ...

In reply to this post by Tobias Pape

> On 12.04.2016, at 11:10, Tobias Pape <[hidden email]> wrote:
>
>
> On 12.04.2016, at 08:38, marcel.taeumel <[hidden email]> wrote:
>
>> Hi Tobias,
>>
>> this version is made for endless streams, not the ones the always operate on
>> a collection but sockets etc. So, the generator never ends and hence #atEnd
>> is never true. I thought that #upToNil might be sufficient and also
>> practical for other streams.
>>
>> Best,
>> Marcel
>
>
> Sounds reasonable :)

Actually, no, it does not ;)

If the stream hasn’t ended, the generator should continue. If the stream is at end, the generator should stop.

We don’t need #upToNil, you just need to stop the generator when the stream ends:

select: block
^ Generator on: [:g |
[self atEnd] whileFalse: [
| object |
object := self next.
(block value: object) ifTrue: [g yield: object]]]

... and similarly for the other methods. With that change, #upToEnd works just fine on the generator.

Agreed? :)

- Bert -

smime.p7s (5K) Download Attachment

Nicolas Cellier

Re: Generators on Streams on Generators on Collections on ...

2016-04-12 18:35 GMT+02:00 Bert Freudenberg <[hidden email]>:

> On 12.04.2016, at 11:10, Tobias Pape <[hidden email]> wrote:
>
>
> On 12.04.2016, at 08:38, marcel.taeumel <[hidden email]> wrote:
>
>> Hi Tobias,
>>
>> this version is made for endless streams, not the ones the always operate on
>> a collection but sockets etc. So, the generator never ends and hence #atEnd
>> is never true. I thought that #upToNil might be sufficient and also
>> practical for other streams.
>>
>> Best,
>> Marcel
>
>
> Sounds reasonable :)

Actually, no, it does not ;)

If the stream hasn’t ended, the generator should continue. If the stream is at end, the generator should stop.

We don’t need #upToNil, you just need to stop the generator when the stream ends:

select: block
^ Generator on: [:g |
[self atEnd] whileFalse: [
| object |
object := self next.
(block value: object) ifTrue: [g yield: object]]]

... and similarly for the other methods. With that change, #upToEnd works just fine on the generator.

Agreed? :)

- Bert -

+1,
what's so special about nil?
For me upToNil is just upTo: nil

Nicolas

marcel.taeumel

Re: Generators on Streams on Generators on Collections on ...

In reply to this post by Bert Freudenberg

Hi Bert,

yes. I made that so.

It remains a question about

#collectInfinitely:
#selectInfinitely:
#gatherInfinitely:
#rejectInfinitely:
#upToNil

If you wrap this stuff around SocketStream, you can re-used even after a time window without data.

So, should we remove it or keep it? It already is in trunk, if you don't know what I refer to.

Best,
Marcel

Nicolas Cellier

Re: Generators on Streams on Generators on Collections on ...

Hi Marcel,
So what are you trying to achieve?

Is it something like (self nextAvailable: infinitelyMany)?

-- fetch as much as possible from the stream, but don't block (wait) --

2016-04-12 19:43 GMT+02:00 marcel.taeumel <[hidden email]>:

Hi Bert,

yes. I made that so.

It remains a question about

#collectInfinitely:
#selectInfinitely:
#gatherInfinitely:
#rejectInfinitely:
#upToNil

If you wrap this stuff around SocketStream, you can re-used even after a
time window without data.

So, should we remove it or keep it? It already is in trunk, if you don't
know what I refer to.

Best,
Marcel

--
View this message in context: http://forum.world.st/Generators-on-Streams-on-Generators-on-Collections-on-tp4889389p4889611.html

Sent from the Squeak - Dev mailing list archive at Nabble.com.

marcel.taeumel

Re: Generators on Streams on Generators on Collections on ...

Hi Nicolas,

if you construct something like:

| someStream |
someStream := mySocketStream select: [:ea | ... ] thenCollect: [:ea | ... ].
...
someStream next.
...
someStream next: 5.
...

You can only use someStream as long as there is data available while it is connected. Now, it can happen that socket streams are temporarily "at end" while no data is waiting in the buffer. Then, you would have to reconstruct someStream to continue. And know about it.

Best,
Marcel

Robert Withers

Re: Generators on Streams on Generators on Collections on ...

In reply to this post by Nicolas Cellier

In my case I think of this as draining what has collected. How about #drainAvailable?

---

robert

On Apr 12, 2016, at 15:44, Nicolas Cellier <[hidden email]> wrote:

Hi Marcel,
So what are you trying to achieve?
Is it something like (self nextAvailable: infinitelyMany)?
-- fetch as much as possible from the stream, but don't block (wait) --

2016-04-12 19:43 GMT+02:00 marcel.taeumel <[hidden email]>:
Hi Bert,

yes. I made that so.

It remains a question about

#collectInfinitely:
#selectInfinitely:
#gatherInfinitely:
#rejectInfinitely:
#upToNil

If you wrap this stuff around SocketStream, you can re-used even after a
time window without data.

So, should we remove it or keep it? It already is in trunk, if you don't
know what I refer to.

Best,
Marcel

--
View this message in context: http://forum.world.st/Generators-on-Streams-on-Generators-on-Collections-on-tp4889389p4889611.html

Sent from the Squeak - Dev mailing list archive at Nabble.com.

Bert Freudenberg

Re: Generators on Streams on Generators on Collections on ...

In reply to this post by marcel.taeumel

On 12.04.2016, at 21:33, marcel.taeumel <[hidden email]> wrote:

>
> Hi Nicolas,
>
> if you construct something like:
>
> | someStream |
> someStream := mySocketStream select: [:ea | ... ] thenCollect: [:ea | ... ].
> ...
> someStream next.
> ...
> someStream next: 5.
> ...
>
> You can only use someStream as long as there is data available while it is
> connected. Now, it can happen that socket streams are temporarily "at end"
> while no data is waiting in the buffer. Then, you would have to reconstruct
> someStream to continue. And know about it.

AFAIK, “at end” means there will not be any more data:

SocketStream>>atEnd
"There is nothing more to read when
there is no more data in our inBuffer, the socket
is disconnected and there is none available on the socket.
Note that we need to check isConnected before isDataAvailable,
otherwise data may sneak in in the meantime. But we check the
buffer first, because it is faster.”

self isInBufferEmpty ifFalse: [^false].
^self isConnected not
and: [self isDataAvailable not]

- Bert -

smime.p7s (5K) Download Attachment

David T. Lewis

Re: Generators on Streams on Generators on Collections on ...

FWIW, In OSProcess I used #upToEnd to mean "all of the data that is
currently available in the stream" and #upToEndOfFile to mean "all of that
data that will ever be available". Separately, the steam may be in
blocking or non-blocking mode, where blocking means that #upToEndOfFile
will block until EOF is detected.

Dave

> On 12.04.2016, at 21:33, marcel.taeumel <[hidden email]> wrote:
>>
>> Hi Nicolas,
>>
>> if you construct something like:
>>
>> | someStream |
>> someStream := mySocketStream select: [:ea | ... ] thenCollect: [:ea |
>> ... ].
>> ...
>> someStream next.
>> ...
>> someStream next: 5.
>> ...
>>
>> You can only use someStream as long as there is data available while it
>> is
>> connected. Now, it can happen that socket streams are temporarily "at
>> end"
>> while no data is waiting in the buffer. Then, you would have to
>> reconstruct
>> someStream to continue. And know about it.
>
> AFAIK, âat endâ means there will not be any more data:
>
> SocketStream>>atEnd
> "There is nothing more to read when
> there is no more data in our inBuffer, the socket
> is disconnected and there is none available on the socket.
> Note that we need to check isConnected before isDataAvailable,
> otherwise data may sneak in in the meantime. But we check the
> buffer first, because it is faster.â
>
> self isInBufferEmpty ifFalse: [^false].
> ^self isConnected not
> and: [self isDataAvailable not]
>
>
> - Bert -
>
>
>

Craig Latta

re: Generators on Streams on Generators on Collections on ...

In reply to this post by Nicolas Cellier

Hi--

Nicolas writes:

> So what [is Marcel] trying to achieve? Is it something like (self
> nextAvailable: infinitelyMany)? -- fetch as much as possible from the
> stream, but don't block (wait)

Otherwise known simply as "nextAvailable", yes?

-C

--
Craig Latta
Black Page Digital
Amsterdam
[hidden email]
+31 6 2757 7177 (SMS ok)
+ 1 415 287 3547 (no SMS)

Eliot Miranda-2

Re: Generators on Streams on Generators on Collections on ...

In reply to this post by marcel.taeumel

> On Apr 11, 2016, at 11:38 PM, marcel.taeumel <[hidden email]> wrote:
>
> Hi Tobias,
>
> this version is made for endless streams, not the ones the always operate on
> a collection but sockets etc. So, the generator never ends and hence #atEnd
> is never true. I thought that #upToNil might be sufficient and also
> practical for other streams.

I disagree. In general it is wrong to assume nil is the end-of-stream value. In VW it is only a default, and one can set the end-of-stream value to something else, a facility we could use also. In some circumstances nil can be a valid element in the stream (#(nil true false) readStream). upToEnd is the right message to provide.

_,,,^..^,,,_ (phone)

>
> Best,
> Marcel
>
>
>
> --
> View this message in context: http://forum.world.st/Generators-on-Streams-on-Generators-on-Collections-on-tp4889389p4889453.html
> Sent from the Squeak - Dev mailing list archive at Nabble.com.
>

Eliot Miranda-2

Re: Generators on Streams on Generators on Collections on ...

In reply to this post by marcel.taeumel

> On Apr 12, 2016, at 10:43 AM, marcel.taeumel <[hidden email]> wrote:
>
> Hi Bert,
>
> yes. I made that so.
>
> It remains a question about
>
> #collectInfinitely:
> #selectInfinitely:
> #gatherInfinitely:
> #rejectInfinitely:
> #upToNil
>
> If you wrap this stuff around SocketStream, you can re-used even after a
> time window without data.
>
> So, should we remove it or keep it? It already is in trunk, if you don't
> know what I refer to.

Remove upToNil. It is a mistake.

>
> Best,
> Marcel
>
>
>
> --
> View this message in context: http://forum.world.st/Generators-on-Streams-on-Generators-on-Collections-on-tp4889389p4889611.html
> Sent from the Squeak - Dev mailing list archive at Nabble.com.
>