Smalltalk › Pharo › Pharo Smalltalk Users

Stream API

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

31 messages Options

Steffen Märcker

Re: Stream API

Hi,

> Are transducers the subject of your thesis ?

No. Transducers is my side project. I've implemented a package for
VisualWorks. Unfortunately, I did not finish the port to Pharo yet, simply
due to a lack of time. Originally, transducers evolved in the clojure
community. I figured, the concept a good fit for Smalltalk and adapted it
accordingly. (My thesis is on conditional probabilities in model-checking
of probabilistic systems.)

> Any pointers to more information ?

http://www.cincomsmalltalk.com/publicRepository/Transducers.html
https://clojure.org/reference/transducers
Plus some mails on this list and more on the vwnc list. Feel free to ask;
I understand, that the package comment has lots of potential for
improvement to help understanding.

Best, Steffen

Sven Van Caekenberghe-2

Re: Stream API

> On 14 Nov 2017, at 16:00, Steffen Märcker <[hidden email]> wrote:
>
> Hi,
>
>> Are transducers the subject of your thesis ?
>
> No. Transducers is my side project. I've implemented a package for VisualWorks. Unfortunately, I did not finish the port to Pharo yet, simply due to a lack of time. Originally, transducers evolved in the clojure community. I figured, the concept a good fit for Smalltalk and adapted it accordingly. (My thesis is on conditional probabilities in model-checking of probabilistic systems.)
>
>> Any pointers to more information ?
>
> http://www.cincomsmalltalk.com/publicRepository/Transducers.html
> https://clojure.org/reference/transducers
> Plus some mails on this list and more on the vwnc list. Feel free to ask; I understand, that the package comment has lots of potential for improvement to help understanding.
>
> Best, Steffen

Some code seems to be here: http://smalltalkhub.com/#!/~cdlm/Experiments

Not sure if it is complete or what the relation is, or the difference between transducers and reducers ...

Steffen Märcker

Re: Stream API

I forgot to mention, that the most recent code for Pharo is already on
Github: https://github.com/Pharophile/Transducers

Reducers was the name of the first very first implementation.

(In fact, I was originally inspired by clojures Reducers lib. After
implementing it in Smalltalk, I developed the concept further. Later I
found out, that the clojure guys did the same in parallel and ended up
with the same abstraction but a differnt name. Hence I decided change the
name in order to make the relation clear.)

Am .11.2017, 16:18 Uhr, schrieb Sven Van Caekenberghe <[hidden email]>:

>
>
>> On 14 Nov 2017, at 16:00, Steffen Märcker <[hidden email]> wrote:
>>
>> Hi,
>>
<---Schnitt--->
>>
>> No. Transducers is my side project. I've implemented a package for
>> VisualWorks. Unfortunately, I did not finish the port to Pharo yet,
>> simply due to a lack of time. Originally, transducers evolved in the
>> clojure community. I figured, the concept a good fit for Smalltalk and
>> adapted it accordingly. (My thesis is on conditional probabilities in
>> model-checking of probabilistic systems.)
>>
<---Schnitt--->

>>
>> http://www.cincomsmalltalk.com/publicRepository/Transducers.html
>> https://clojure.org/reference/transducers
>> Plus some mails on this list and more on the vwnc list. Feel free to
>> ask; I understand, that the package comment has lots of potential for
>> improvement to help understanding.
>>
>> Best, Steffen
>
> Some code seems to be here: http://smalltalkhub.com/#!/~cdlm/Experiments
>
> Not sure if it is complete or what the relation is, or the difference
> between transducers and reducers ...
>
>

Denis Kudriashov

Re: Stream API

2017-11-14 16:30 GMT+01:00 Steffen Märcker <[hidden email]>:

I forgot to mention, that the most recent code for Pharo is already on Github: https://github.com/Pharophile/Transducers

Reducers was the name of the first very first implementation.

(In fact, I was originally inspired by clojures Reducers lib. After implementing it in Smalltalk, I developed the concept further. Later I found out, that the clojure guys did the same in parallel and ended up with the same abstraction but a differnt name. Hence I decided change the name in order to make the relation clear.)

I like abstraction. But I think names and order of computation should be changed to be more Smalltalk friendly. Because now it looks like Haskell with right to left order:

squares := Set <~ 1000 take <~ #squared map <~ (1 to: 1000).
fileOut writeStream <~ #isSeparator filter <~ fileIn readStream.

Is there any reason to not change it? I would like to start expressions with source of data:

squares := (1 to: 1000) ~> #squared map ~> 1000 take ~> Set.
fileIn readStream ~> #isSeparator filter ~> fileOut writeStream.

Am .11.2017, 16:18 Uhr, schrieb Sven Van Caekenberghe <[hidden email]>:

On 14 Nov 2017, at 16:00, Steffen Märcker <[hidden email]> wrote:

Hi,

<---Schnitt--->

No. Transducers is my side project. I've implemented a package for VisualWorks. Unfortunately, I did not finish the port to Pharo yet, simply due to a lack of time. Originally, transducers evolved in the clojure community. I figured, the concept a good fit for Smalltalk and adapted it accordingly. (My thesis is on conditional probabilities in model-checking of probabilistic systems.)

<---Schnitt--->

http://www.cincomsmalltalk.com/publicRepository/Transducers.html
https://clojure.org/reference/transducers
Plus some mails on this list and more on the vwnc list. Feel free to ask; I understand, that the package comment has lots of potential for improvement to help understanding.

Best, Steffen

Some code seems to be here: http://smalltalkhub.com/#!/~cdlm/Experiments

Not sure if it is complete or what the relation is, or the difference between transducers and reducers ...

Stephane Ducasse-3

Re: Stream API

Denis I agree. I do not like to code in reverse order.

On Tue, Nov 14, 2017 at 4:49 PM, Denis Kudriashov <[hidden email]> wrote:

> 2017-11-14 16:30 GMT+01:00 Steffen Märcker <[hidden email]>:
>>
>> I forgot to mention, that the most recent code for Pharo is already on
>> Github: https://github.com/Pharophile/Transducers
>>
>> Reducers was the name of the first very first implementation.
>>
>> (In fact, I was originally inspired by clojures Reducers lib. After
>> implementing it in Smalltalk, I developed the concept further. Later I found
>> out, that the clojure guys did the same in parallel and ended up with the
>> same abstraction but a differnt name. Hence I decided change the name in
>> order to make the relation clear.)
>
>
> I like abstraction. But I think names and order of computation should be
> changed to be more Smalltalk friendly. Because now it looks like Haskell
> with right to left order:
>
> squares := Set <~ 1000 take <~ #squared map <~ (1 to: 1000).
> fileOut writeStream <~ #isSeparator filter <~ fileIn readStream.
>
>
> Is there any reason to not change it? I would like to start expressions with
> source of data:
>
> squares := (1 to: 1000) ~> #squared map ~> 1000 take ~> Set.
> fileIn readStream ~> #isSeparator filter ~> fileOut writeStream.
>
>
>
>>
>> Am .11.2017, 16:18 Uhr, schrieb Sven Van Caekenberghe <[hidden email]>:
>>
>>>
>>>
>>>> On 14 Nov 2017, at 16:00, Steffen Märcker <[hidden email]> wrote:
>>>>
>>>> Hi,
>>>>
>> <---Schnitt--->
>>>>
>>>>
>>>> No. Transducers is my side project. I've implemented a package for
>>>> VisualWorks. Unfortunately, I did not finish the port to Pharo yet, simply
>>>> due to a lack of time. Originally, transducers evolved in the clojure
>>>> community. I figured, the concept a good fit for Smalltalk and adapted it
>>>> accordingly. (My thesis is on conditional probabilities in model-checking of
>>>> probabilistic systems.)
>>>>
>> <---Schnitt--->
>>
>>>>
>>>> http://www.cincomsmalltalk.com/publicRepository/Transducers.html
>>>> https://clojure.org/reference/transducers
>>>> Plus some mails on this list and more on the vwnc list. Feel free to
>>>> ask; I understand, that the package comment has lots of potential for
>>>> improvement to help understanding.
>>>>
>>>> Best, Steffen
>>>
>>>
>>> Some code seems to be here: http://smalltalkhub.com/#!/~cdlm/Experiments
>>>
>>> Not sure if it is complete or what the relation is, or the difference
>>> between transducers and reducers ...
>>>
>>>
>>
>>
>>
>

Stephane Ducasse-3

Re: Stream API

Sven

From the thread discussion here my take.

- I do not think that we want to reuse any of the old streams.
- So first Zn to replace the binary and MultiUglyStream and after we
will have to check
if we should port the design of Xtream (but not the API) to Pharo.

Stef

On Tue, Nov 14, 2017 at 5:59 PM, Stephane Ducasse
<[hidden email]> wrote:

> Denis I agree. I do not like to code in reverse order.
>
>
> On Tue, Nov 14, 2017 at 4:49 PM, Denis Kudriashov <[hidden email]> wrote:
>> 2017-11-14 16:30 GMT+01:00 Steffen Märcker <[hidden email]>:
>>>
>>> I forgot to mention, that the most recent code for Pharo is already on
>>> Github: https://github.com/Pharophile/Transducers
>>>
>>> Reducers was the name of the first very first implementation.
>>>
>>> (In fact, I was originally inspired by clojures Reducers lib. After
>>> implementing it in Smalltalk, I developed the concept further. Later I found
>>> out, that the clojure guys did the same in parallel and ended up with the
>>> same abstraction but a differnt name. Hence I decided change the name in
>>> order to make the relation clear.)
>>
>>
>> I like abstraction. But I think names and order of computation should be
>> changed to be more Smalltalk friendly. Because now it looks like Haskell
>> with right to left order:
>>
>> squares := Set <~ 1000 take <~ #squared map <~ (1 to: 1000).
>> fileOut writeStream <~ #isSeparator filter <~ fileIn readStream.
>>
>>
>> Is there any reason to not change it? I would like to start expressions with
>> source of data:
>>
>> squares := (1 to: 1000) ~> #squared map ~> 1000 take ~> Set.
>> fileIn readStream ~> #isSeparator filter ~> fileOut writeStream.
>>
>>
>>
>>>
>>> Am .11.2017, 16:18 Uhr, schrieb Sven Van Caekenberghe <[hidden email]>:
>>>
>>>>
>>>>
>>>>> On 14 Nov 2017, at 16:00, Steffen Märcker <[hidden email]> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>> <---Schnitt--->
>>>>>
>>>>>
>>>>> No. Transducers is my side project. I've implemented a package for
>>>>> VisualWorks. Unfortunately, I did not finish the port to Pharo yet, simply
>>>>> due to a lack of time. Originally, transducers evolved in the clojure
>>>>> community. I figured, the concept a good fit for Smalltalk and adapted it
>>>>> accordingly. (My thesis is on conditional probabilities in model-checking of
>>>>> probabilistic systems.)
>>>>>
>>> <---Schnitt--->
>>>
>>>>>
>>>>> http://www.cincomsmalltalk.com/publicRepository/Transducers.html
>>>>> https://clojure.org/reference/transducers
>>>>> Plus some mails on this list and more on the vwnc list. Feel free to
>>>>> ask; I understand, that the package comment has lots of potential for
>>>>> improvement to help understanding.
>>>>>
>>>>> Best, Steffen
>>>>
>>>>
>>>> Some code seems to be here: http://smalltalkhub.com/#!/~cdlm/Experiments
>>>>
>>>> Not sure if it is complete or what the relation is, or the difference
>>>> between transducers and reducers ...
>>>>
>>>>
>>>
>>>
>>>
>>

Steffen Märcker

Re: Stream API

In reply to this post by Stephane Ducasse-3

Short: No. =)

Actually, the <~ notation was originally proposed by another Smalltalker
on the vwnc list. The idea was that <~ visualizes the flow into the
variable, which I find nice (see first expression). However, I do
understand that this alienates others. Hence, I am non-religious about the
matter; I could even live with both messages coexisting.

> Denis I agree. I do not like to code in reverse order.
>
>> I like abstraction. But I think names and order of computation should be
>> changed to be more Smalltalk friendly. Because now it looks like Haskell
>> with right to left order:
>>
>> squares := Set <~ 1000 take <~ #squared map <~ (1 to: 1000).
>> fileOut writeStream <~ #isSeparator filter <~ fileIn readStream.
>>
>> Is there any reason to not change it?

Best, Steffen

Steffen Märcker

Re: Stream API

In reply to this post by Steffen Märcker

Please note, the port on GitHub is not the most recent version. I had
trouble porting it to Pharo and code exchange in general. For now it's
unfinished due to a lack of time.

Am .11.2017, 16:30 Uhr, schrieb Steffen Märcker <[hidden email]>:

> I forgot to mention, that the most recent code for Pharo is already on
> Github: https://github.com/Pharophile/Transducers
>
> Reducers was the name of the first very first implementation.
>
> (In fact, I was originally inspired by clojures Reducers lib. After
> implementing it in Smalltalk, I developed the concept further. Later I
> found out, that the clojure guys did the same in parallel and ended up
> with the same abstraction but a differnt name. Hence I decided change
> the name in order to make the relation clear.)
>
> Am .11.2017, 16:18 Uhr, schrieb Sven Van Caekenberghe <[hidden email]>:
>
>>
>>
>>> On 14 Nov 2017, at 16:00, Steffen Märcker <[hidden email]> wrote:
>>>
>>> Hi,
>>>
> <---Schnitt--->
>>>
>>> No. Transducers is my side project. I've implemented a package for
>>> VisualWorks. Unfortunately, I did not finish the port to Pharo yet,
>>> simply due to a lack of time. Originally, transducers evolved in the
>>> clojure community. I figured, the concept a good fit for Smalltalk and
>>> adapted it accordingly. (My thesis is on conditional probabilities in
>>> model-checking of probabilistic systems.)
>>>
> <---Schnitt--->
>>>
>>> http://www.cincomsmalltalk.com/publicRepository/Transducers.html
>>> https://clojure.org/reference/transducers
>>> Plus some mails on this list and more on the vwnc list. Feel free to
>>> ask; I understand, that the package comment has lots of potential for
>>> improvement to help understanding.
>>>
>>> Best, Steffen
>>
>> Some code seems to be here: http://smalltalkhub.com/#!/~cdlm/Experiments
>>
>> Not sure if it is complete or what the relation is, or the difference
>> between transducers and reducers ...
>>
>>
>
>

Guillermo Polito

Re: Stream API

In reply to this post by Steffen Märcker

I have a meta-comment here.

As a community I feel sometimes that we are trying to make people converge. "Somebody already wrote a transducers library, I cannot write another transducers library, we have to merge". Now, I understand that sometimes we don't have the "manpower" to solve all the problems in the world, but let's not coerce people to "converge" just for converging.

Most of the times, different libraries are implemented with different tradeoffs and different use case scenarios, and even with different aesthetic preferences. I think I'm saying nothing strange if I say that there is no silver bullet :).

Some people may implement their own library for fun:

- I don't want to prevent people from having fun

- I don't want to prevent myself from having fun

So Steffen, please deliver something nice, with nice docs, and have fun and thanks :)

On Wed, Nov 15, 2017 at 11:12 AM, Steffen Märcker <[hidden email]> wrote:

Short: No. =)

Actually, the <~ notation was originally proposed by another Smalltalker on the vwnc list. The idea was that <~ visualizes the flow into the variable, which I find nice (see first expression). However, I do understand that this alienates others. Hence, I am non-religious about the matter; I could even live with both messages coexisting.

Denis I agree. I do not like to code in reverse order.

I like abstraction. But I think names and order of computation should be
changed to be more Smalltalk friendly. Because now it looks like Haskell
with right to left order:

squares := Set <~ 1000 take <~ #squared map <~ (1 to: 1000).
fileOut writeStream <~ #isSeparator filter <~ fileIn readStream.

Is there any reason to not change it?

Best, Steffen

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: +33 06 52 70 66 13

Prof. Andrew P. Black

Re: Stream API

In reply to this post by Sven Van Caekenberghe-2

> On 14 Nov 2017, at 14:07 , Sven Van Caekenberghe <[hidden email]> wrote:
>
> I am curious though, why do you need it ?
>
> I mean, I got away (and try hard) to only use 1-element peeking and managed to implement STON, NeoJSON, NeoCSV, Stomp, PostgresSQL, Redis, and other protocols.
>
> A syntax that really requires multi element peeking is not so common.

I need it in my present situation because I’m inside a framework that thinks that it owns the Stream (viz, inside SmaCC). I can write a custom action in the scanner for, in my case, <newline> tokens, that deals with indentation. Sometimes that action need to look ahead to do the right thing. For example, if the current line is completely blank, it can’t be an indentation error, and does not re-set the current indentation to zero. I need to make such determinations from inside the newline method, and then reset the state of the stream so that the rest of the SmaCC-generated parser is undisturbed.

In a framework like a PEG, you get a similar situation. A parser in a PEG should either succeed, and consume input, or fail, and leave the input where it was. This is true not just for simple parsers that read one character, but also for compound parsers that may perform arbitrary actions before they fail. This usually works by saving and resetting the stream state (but also maintaining a cache in the case of a Packrat Parser).

In general, this is a modularity issue. Although globally, one may be able to limit lookahead, locally, it’s not so easy.

An alternative to saving a restoring state is to allow arbitrary putback, as in OS6 streams. But I think that state-saving is more useful, even if it is less general.

Andrew

Prof. Andrew P. Black

Re: Stream API

In reply to this post by Denis Kudriashov

> On 14 Nov 2017, at 16:49 , Denis Kudriashov <[hidden email]> wrote:
>
>
> squares := (1 to: 1000) ~> #squared map ~> 1000 take ~> Set.
> fileIn readStream ~> #isSeparator filter ~> fileOut writeStream.

I’ve actually done something very similar for collections in Grace, except that I use >> as the operator symbol.

My original idea was when implementing Grace’s filter and map methods, which correspond to select: and collect: in Smalltalk. I wanted to avoid species and it’s spawn. So instead I made filter and map produce streams of values, which can then be >>’d into the container of the programmer’s choice.

In Grace we can write now

(1..100).filter { each → each.isEven } >> set.empty

I’ve been vacillating over whether the final sink should be a collection factory (like Set) or a collection instance (which might be empty or might already contain some elements). We could also make >> accept a block as its argument, and eliminate either the map or filter word, but not both.

Andrew