Reducers API style question

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

Reducers API style question

Steffen Märcker
Hi,

I am about to simplify the API a bit and I am really eager to hear your  
opinion. Suppose we want to gather squares of odds numbers. Previously, we  
could write:

    (Reducer map: #squared) <> (Reducer filter: #odd)
       from: #(1 2 3 4 5 6).

The repetition of 'Reducer' makes this pretty verbose. I intend to change  
this to

    #squared mapping <> #odd filtering
       from: #(1 2 3 4 5 6).

However, flattening a nested collection requires no parameter (like  
mapping or filtering). Thus it does not quite fit this style (flattening  
is done by a singleton object):

    #squared mapping <> #odd filtering <> ?_flattening_?
       from: #((1 2 3) (4 5 6)).

How would you implement flattening in this API style? So far, I see only  
two options:

1) Either extending the class side of Flattening to support the  
composition protocol:
    #squared mapping <> #odd filtering <> Flattening

2) Or making the composition obvious by creating a flattening reducer  
object:
    #squared mapping <> #odd filtering <> Flattening reducer

Which option would your prefer? Do you have alternative ideas? Or would  
you even reject this API style completely?

Kind regards,
Steffen
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Reducers API style question

Karsten Kusche
Why don’t you turn the API around like:

( 1 to: 10) reduce:[:reducer | 
reducer map: #squared;
filter: #odd;
flatten]

i think that’s easier than remembering which new binary operators need to be used.

Kind Regards
Karsten


-- 
Karsten Kusche - Dipl. Inf. (FH) - [hidden email]
Georg Heeg eK - Köthen
Handelsregister: Amtsgericht Dortmund A 12812 

Am Dienstag, 6. Mai 2014 um 08:27 schrieb Steffen Märcker:

Hi,

I am about to simplify the API a bit and I am really eager to hear your
opinion. Suppose we want to gather squares of odds numbers. Previously, we
could write:

(Reducer map: #squared) <> (Reducer filter: #odd)
from: #(1 2 3 4 5 6).

The repetition of 'Reducer' makes this pretty verbose. I intend to change
this to

#squared mapping <> #odd filtering
from: #(1 2 3 4 5 6).

However, flattening a nested collection requires no parameter (like
mapping or filtering). Thus it does not quite fit this style (flattening
is done by a singleton object):

#squared mapping <> #odd filtering <> ?_flattening_?
from: #((1 2 3) (4 5 6)).

How would you implement flattening in this API style? So far, I see only
two options:

1) Either extending the class side of Flattening to support the
composition protocol:
#squared mapping <> #odd filtering <> Flattening

2) Or making the composition obvious by creating a flattening reducer
object:
#squared mapping <> #odd filtering <> Flattening reducer

Which option would your prefer? Do you have alternative ideas? Or would
you even reject this API style completely?

Kind regards,
Steffen
_______________________________________________
vwnc mailing list


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Reducers API style question

Michael Lucas-Smith-2
In reply to this post by Steffen Märcker

On 6 May 2014, at 4:27 pm, Steffen Märcker <[hidden email]> wrote:

> Hi,
>
> I am about to simplify the API a bit and I am really eager to hear your opinion. Suppose we want to gather squares of odds numbers. Previously, we could write:
>
>   (Reducer map: #squared) <> (Reducer filter: #odd)
>      from: #(1 2 3 4 5 6).
>
> The repetition of 'Reducer' makes this pretty verbose. I intend to change this to
>
>   #squared mapping <> #odd filtering
>      from: #(1 2 3 4 5 6).

I like this. The ‘reducer’ terminology disappears and is no longer a question of what the code is doing.

>
> However, flattening a nested collection requires no parameter (like mapping or filtering). Thus it does not quite fit this style (flattening is done by a singleton object):
>
>   #squared mapping <> #odd filtering <> ?_flattening_?
>      from: #((1 2 3) (4 5 6)).
>
> How would you implement flattening in this API style? So far, I see only two options:
>
> 1) Either extending the class side of Flattening to support the composition protocol:
>   #squared mapping <> #odd filtering <> Flattening

This would be my preference. I’d feel comfortable if a ‘reducer’ could be a Reducer, class, or finally a closure for the ultimate in flexibility.
The other part of this API that doesn’t sit quite right with me is the ‘from’ method. The reduces let you predefine a program that can execute.. so you’d be applying it to a collection, or running it with a collection, e.g.:

(1 to: 15) reduce: (#squared mapping <> #odd filtering <> [:each | each + 1])

(#squared mapping <> #odd filtering <> [:each | each + 1]) reduce: (1 to: 15)

Not sure which direction reads better. Collection>>reduce: fits in with all the other collection protocols, while Reducer>>reduce: is much more verb-like and owned by the subsystem (reducers) that you’re using (similar to doing BlockClosure>>value: in that respect, since a BlockClosure is a program and so is a Reducer).

Cheers,
Michael
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Reducers API style question

Steffen Märcker
In reply to this post by Karsten Kusche
Hi Karsten,

a style similar to the classic collection API is supported as well, e.g.,

    (((1 to: 10) reducer flatten map: #squared) filter: #odd) into: Set

The advantage of using <> is that the computation, i.e. mapping filtering,  
can be defined independently of the source and drain collections/objects.  
For example, consider

    squaredOdds := #squared mapping <> #odd filtering.
    squaredOdds from: (1 to: 10) into: Set.
   (squaredOdds from: #(1 2 3 4 5 6)) reduce: #+ init: 0.
    squaredOdds <> Flattening reducer from: #(1 (2 3 (4 5 6))) into:  
OrderedCollection.

Or did I missunderstand you?

Cheers, Steffen

Am 06.05.2014, 08:55 Uhr, schrieb Karsten Kusche <[hidden email]>:

> Why don’t you turn the API around like:
>
> ( 1 to: 10) reduce:[:reducer |
> reducer map: #squared;
> filter: #odd;
> flatten]
>
> i think that’s easier than remembering which new binary operators need  
> to be used.
>
> Kind Regards
> Karsten
>
>
> --
> Karsten Kusche - Dipl. Inf. (FH) - [hidden email]
> Georg Heeg eK - Köthen
> Handelsregister: Amtsgericht Dortmund A 12812
>
>
> Am Dienstag, 6. Mai 2014 um 08:27 schrieb Steffen Märcker:
>
>> Hi,
>>
>> I am about to simplify the API a bit and I am really eager to hear your
>> opinion. Suppose we want to gather squares of odds numbers. Previously,  
>> we
>> could write:
>>
>> (Reducer map: #squared) <> (Reducer filter: #odd)
>> from: #(1 2 3 4 5 6).
>>
>> The repetition of 'Reducer' makes this pretty verbose. I intend to  
>> change
>> this to
>>
>> #squared mapping <> #odd filtering
>> from: #(1 2 3 4 5 6).
>>
>> However, flattening a nested collection requires no parameter (like
>> mapping or filtering). Thus it does not quite fit this style (flattening
>> is done by a singleton object):
>>
>> #squared mapping <> #odd filtering <> ?_flattening_?
>> from: #((1 2 3) (4 5 6)).
>>
>> How would you implement flattening in this API style? So far, I see only
>> two options:
>>
>> 1) Either extending the class side of Flattening to support the
>> composition protocol:
>> #squared mapping <> #odd filtering <> Flattening
>>
>> 2) Or making the composition obvious by creating a flattening reducer
>> object:
>> #squared mapping <> #odd filtering <> Flattening reducer
>>
>> Which option would your prefer? Do you have alternative ideas? Or would
>> you even reject this API style completely?
>>
>> Kind regards,
>> Steffen
>> _______________________________________________
>> vwnc mailing list
>> [hidden email] (mailto:[hidden email])
>> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
>>
>>
>
>


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Reducers API style question

Steffen Märcker
In reply to this post by Michael Lucas-Smith-2
Hi Michael,

thanks for your thoughts. I agree, that the reducer terminology should  
fade into the background. =)

Considering your examples, I am afraid that I failed to make clear what  
#reduce:init: does and what a reducer object is. Sorry, here's my second  
attempt:

1) "reduce: block init: value" is #inject:into: ; it serves as an  
interface on top of which all other enumerations (map, filter, etc.) can  
be implemented.

2) A reducer is a wrapper on a collection (an object that understands  
#reduce:init:) providing a 'transformed view' on a collection. For example,

   successors := [:x|x+1] mapping from: (1 to: 10).

creates a Reducers wrapping the interval. When sending #reduce:init: to  
that reducer, the block is evaluated with the successors, i.e., (2 to: 11).
Reducers can be curried and combined independently of a collection. The  
message #from: is only used to finally wrap a collection. The actual  
computation is performed by invoking #reduce:init: (#into: calls  
#reduce:init:).

    view := #squared mapping <> #odd filtering <> [:x|x+1] mapping.
    wrapped_1 := view from: (1 to: 10).
    sum := wrapped_1 reduce: #+ init: 0. "computation is performed here"
    wrapped_2 := view from: #(1 2 3).
    set := wrapped_2 into: Set. "computation is performed here"

Does this make more sense to you? Or did I completely miss your point?

Best,
Steffen


Am 06.05.2014, 09:02 Uhr, schrieb Michael Lucas-Smith  
<[hidden email]>:

>
> On 6 May 2014, at 4:27 pm, Steffen Märcker <[hidden email]> wrote:
>
>> Hi,
>>
>> I am about to simplify the API a bit and I am really eager to hear your  
>> opinion. Suppose we want to gather squares of odds numbers. Previously,  
>> we could write:
>>
>>   (Reducer map: #squared) <> (Reducer filter: #odd)
>>      from: #(1 2 3 4 5 6).
>>
>> The repetition of 'Reducer' makes this pretty verbose. I intend to  
>> change this to
>>
>>   #squared mapping <> #odd filtering
>>      from: #(1 2 3 4 5 6).
>
> I like this. The ‘reducer’ terminology disappears and is no longer a  
> question of what the code is doing.
>
>>
>> However, flattening a nested collection requires no parameter (like  
>> mapping or filtering). Thus it does not quite fit this style  
>> (flattening is done by a singleton object):
>>
>>   #squared mapping <> #odd filtering <> ?_flattening_?
>>      from: #((1 2 3) (4 5 6)).
>>
>> How would you implement flattening in this API style? So far, I see  
>> only two options:
>>
>> 1) Either extending the class side of Flattening to support the  
>> composition protocol:
>>   #squared mapping <> #odd filtering <> Flattening
>
> This would be my preference. I’d feel comfortable if a ‘reducer’ could  
> be a Reducer, class, or finally a closure for the ultimate in  
> flexibility.
> The other part of this API that doesn’t sit quite right with me is the  
> ‘from’ method. The reduces let you predefine a program that can  
> execute.. so you’d be applying it to a collection, or running it with a  
> collection, e.g.:
>
> (1 to: 15) reduce: (#squared mapping <> #odd filtering <> [:each | each  
> + 1])
>
> (#squared mapping <> #odd filtering <> [:each | each + 1]) reduce: (1  
> to: 15)
>
> Not sure which direction reads better. Collection>>reduce: fits in with  
> all the other collection protocols, while Reducer>>reduce: is much more  
> verb-like and owned by the subsystem (reducers) that you’re using  
> (similar to doing BlockClosure>>value: in that respect, since a  
> BlockClosure is a program and so is a Reducer).
>
> Cheers,
> Michael

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Reducers API style question

Michael Lucas-Smith-2
You didn't miss my point. Thanks for the explanation. I guess what I'm saying is that the 'wrapped' bit is unnecessary, since once you've reduced a collection and have a result, the only thing you're likely to reuse is the reduction objects, not the wrapper object. I'm proposing that your API reflect the action of using a reducer on a collection, "hiding" the wrapping step.

transformation := #squared mapping <> #odd filtering <> Set.
(1 to: 3) collect: transformation    -> Set(1 9)

In many respects, if the last thing you specify is the collection to operate on, then the description of the transformation should be right-to-left, eg:

transformation := Set <> #odd filtering <> #squared mapping <> (1 to: 3).

But if you still like the idea of wrapping, then it might make more sense to flip it back around like this:

transformation := (1 to: 3) <> #squared mapping <> #odd filtering <> Set.

As you can see in both these scenarios, there's no verb to indicate that you want to do it. Unless it acts as a virtual collection (which it doesn't seem to do) you'd want the verb, which is why above I was suggesting you give the reducer object(s) to the collection as an action.

transformation := #squared mapping <> #odd filtering <> #+ reducing.
sum := (1 to: 3) inject: 0 into: transformation.

Here the transformation object takes the place of the traditional block, as above with the collect: example. The idea is that the collection protocol need not have to know the difference between a block closure or a reducer and therefore the same APIs we currently use can be reused. At least, some of them anyway.

I also used the collection itself as part of the reducer binding so that it would unique the items. One thing that is a little confusing for me is the use of <> which suggests directionality is not important - but it seems to me that it is somewhat important - unless I'm mistaken, there's no obvious way to 'undo' a transformation, especially if you've done a reduction like inject:into:.

You'd have to know the opposite of +, the opposite of #squared, the opposite of #odd, etc.. all of which tells me that directionality is important. Given you're dealing with reducer objects (and perhaps a couple of others like classes) it might be better to use a more asymmetrical selector.

Off the top of my head, it could be something like ~> to indicate a direction that data will flow through the objects.
transformation := #squared mapping ~> #odd filtering ~> Set.
theAnswer := (1 to: 3) collect: transformation.

I'm keen on the composability of the reducers, just as closures are. The whole thing is very bimadic (as opposed to monadic).

Cheers,
Michael

On 6 May 2014 18:37, Steffen Märcker <[hidden email]> wrote:
Hi Michael,

thanks for your thoughts. I agree, that the reducer terminology should fade into the background. =)

Considering your examples, I am afraid that I failed to make clear what #reduce:init: does and what a reducer object is. Sorry, here's my second attempt:

1) "reduce: block init: value" is #inject:into: ; it serves as an interface on top of which all other enumerations (map, filter, etc.) can be implemented.

2) A reducer is a wrapper on a collection (an object that understands #reduce:init:) providing a 'transformed view' on a collection. For example,

  successors := [:x|x+1] mapping from: (1 to: 10).

creates a Reducers wrapping the interval. When sending #reduce:init: to that reducer, the block is evaluated with the successors, i.e., (2 to: 11).
Reducers can be curried and combined independently of a collection. The message #from: is only used to finally wrap a collection. The actual computation is performed by invoking #reduce:init: (#into: calls #reduce:init:).

   view := #squared mapping <> #odd filtering <> [:x|x+1] mapping.
   wrapped_1 := view from: (1 to: 10).
   sum := wrapped_1 reduce: #+ init: 0. "computation is performed here"
   wrapped_2 := view from: #(1 2 3).
   set := wrapped_2 into: Set. "computation is performed here"

Does this make more sense to you? Or did I completely miss your point?

Best,
Steffen


Am 06.05.2014, 09:02 Uhr, schrieb Michael Lucas-Smith <[hidden email]>:



On 6 May 2014, at 4:27 pm, Steffen Märcker <[hidden email]> wrote:

Hi,

I am about to simplify the API a bit and I am really eager to hear your opinion. Suppose we want to gather squares of odds numbers. Previously, we could write:

  (Reducer map: #squared) <> (Reducer filter: #odd)
     from: #(1 2 3 4 5 6).

The repetition of 'Reducer' makes this pretty verbose. I intend to change this to

  #squared mapping <> #odd filtering
     from: #(1 2 3 4 5 6).

I like this. The ‘reducer’ terminology disappears and is no longer a question of what the code is doing.


However, flattening a nested collection requires no parameter (like mapping or filtering). Thus it does not quite fit this style (flattening is done by a singleton object):

  #squared mapping <> #odd filtering <> ?_flattening_?
     from: #((1 2 3) (4 5 6)).

How would you implement flattening in this API style? So far, I see only two options:

1) Either extending the class side of Flattening to support the composition protocol:
  #squared mapping <> #odd filtering <> Flattening

This would be my preference. I’d feel comfortable if a ‘reducer’ could be a Reducer, class, or finally a closure for the ultimate in flexibility.
The other part of this API that doesn’t sit quite right with me is the ‘from’ method. The reduces let you predefine a program that can execute.. so you’d be applying it to a collection, or running it with a collection, e.g.:

(1 to: 15) reduce: (#squared mapping <> #odd filtering <> [:each | each + 1])

(#squared mapping <> #odd filtering <> [:each | each + 1]) reduce: (1 to: 15)

Not sure which direction reads better. Collection>>reduce: fits in with all the other collection protocols, while Reducer>>reduce: is much more verb-like and owned by the subsystem (reducers) that you’re using (similar to doing BlockClosure>>value: in that respect, since a BlockClosure is a program and so is a Reducer).

Cheers,
Michael


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Reducers API style question

Michael Lucas-Smith-2
Oh and just one more thought - if operations such as injecting are involved, then perhaps the collection API is never needed, since the last reducer 'connection' would always be a source and would swap from being a constructor to an operator.

theAnswer := Set <~ #odd filtering <~ #squared mapping <~ (1 to: 10).
theSum := (#+ injecting: 0) <~ #odd filtering <~ #squared mapping <~ (1 to: 10).

In this scenario the direction has been swapped around so that it points from the source on the right in to the answer on the left.

Cheers,
Michael


On 6 May 2014 20:07, Michael Lucas-Smith <[hidden email]> wrote:
You didn't miss my point. Thanks for the explanation. I guess what I'm saying is that the 'wrapped' bit is unnecessary, since once you've reduced a collection and have a result, the only thing you're likely to reuse is the reduction objects, not the wrapper object. I'm proposing that your API reflect the action of using a reducer on a collection, "hiding" the wrapping step.

transformation := #squared mapping <> #odd filtering <> Set.
(1 to: 3) collect: transformation    -> Set(1 9)

In many respects, if the last thing you specify is the collection to operate on, then the description of the transformation should be right-to-left, eg:

transformation := Set <> #odd filtering <> #squared mapping <> (1 to: 3).

But if you still like the idea of wrapping, then it might make more sense to flip it back around like this:

transformation := (1 to: 3) <> #squared mapping <> #odd filtering <> Set.

As you can see in both these scenarios, there's no verb to indicate that you want to do it. Unless it acts as a virtual collection (which it doesn't seem to do) you'd want the verb, which is why above I was suggesting you give the reducer object(s) to the collection as an action.

transformation := #squared mapping <> #odd filtering <> #+ reducing.
sum := (1 to: 3) inject: 0 into: transformation.

Here the transformation object takes the place of the traditional block, as above with the collect: example. The idea is that the collection protocol need not have to know the difference between a block closure or a reducer and therefore the same APIs we currently use can be reused. At least, some of them anyway.

I also used the collection itself as part of the reducer binding so that it would unique the items. One thing that is a little confusing for me is the use of <> which suggests directionality is not important - but it seems to me that it is somewhat important - unless I'm mistaken, there's no obvious way to 'undo' a transformation, especially if you've done a reduction like inject:into:.

You'd have to know the opposite of +, the opposite of #squared, the opposite of #odd, etc.. all of which tells me that directionality is important. Given you're dealing with reducer objects (and perhaps a couple of others like classes) it might be better to use a more asymmetrical selector.

Off the top of my head, it could be something like ~> to indicate a direction that data will flow through the objects.
transformation := #squared mapping ~> #odd filtering ~> Set.
theAnswer := (1 to: 3) collect: transformation.

I'm keen on the composability of the reducers, just as closures are. The whole thing is very bimadic (as opposed to monadic).

Cheers,
Michael

On 6 May 2014 18:37, Steffen Märcker <[hidden email]> wrote:
Hi Michael,

thanks for your thoughts. I agree, that the reducer terminology should fade into the background. =)

Considering your examples, I am afraid that I failed to make clear what #reduce:init: does and what a reducer object is. Sorry, here's my second attempt:

1) "reduce: block init: value" is #inject:into: ; it serves as an interface on top of which all other enumerations (map, filter, etc.) can be implemented.

2) A reducer is a wrapper on a collection (an object that understands #reduce:init:) providing a 'transformed view' on a collection. For example,

  successors := [:x|x+1] mapping from: (1 to: 10).

creates a Reducers wrapping the interval. When sending #reduce:init: to that reducer, the block is evaluated with the successors, i.e., (2 to: 11).
Reducers can be curried and combined independently of a collection. The message #from: is only used to finally wrap a collection. The actual computation is performed by invoking #reduce:init: (#into: calls #reduce:init:).

   view := #squared mapping <> #odd filtering <> [:x|x+1] mapping.
   wrapped_1 := view from: (1 to: 10).
   sum := wrapped_1 reduce: #+ init: 0. "computation is performed here"
   wrapped_2 := view from: #(1 2 3).
   set := wrapped_2 into: Set. "computation is performed here"

Does this make more sense to you? Or did I completely miss your point?

Best,
Steffen


Am 06.05.2014, 09:02 Uhr, schrieb Michael Lucas-Smith <[hidden email]>:



On 6 May 2014, at 4:27 pm, Steffen Märcker <[hidden email]> wrote:

Hi,

I am about to simplify the API a bit and I am really eager to hear your opinion. Suppose we want to gather squares of odds numbers. Previously, we could write:

  (Reducer map: #squared) <> (Reducer filter: #odd)
     from: #(1 2 3 4 5 6).

The repetition of 'Reducer' makes this pretty verbose. I intend to change this to

  #squared mapping <> #odd filtering
     from: #(1 2 3 4 5 6).

I like this. The ‘reducer’ terminology disappears and is no longer a question of what the code is doing.


However, flattening a nested collection requires no parameter (like mapping or filtering). Thus it does not quite fit this style (flattening is done by a singleton object):

  #squared mapping <> #odd filtering <> ?_flattening_?
     from: #((1 2 3) (4 5 6)).

How would you implement flattening in this API style? So far, I see only two options:

1) Either extending the class side of Flattening to support the composition protocol:
  #squared mapping <> #odd filtering <> Flattening

This would be my preference. I’d feel comfortable if a ‘reducer’ could be a Reducer, class, or finally a closure for the ultimate in flexibility.
The other part of this API that doesn’t sit quite right with me is the ‘from’ method. The reduces let you predefine a program that can execute.. so you’d be applying it to a collection, or running it with a collection, e.g.:

(1 to: 15) reduce: (#squared mapping <> #odd filtering <> [:each | each + 1])

(#squared mapping <> #odd filtering <> [:each | each + 1]) reduce: (1 to: 15)

Not sure which direction reads better. Collection>>reduce: fits in with all the other collection protocols, while Reducer>>reduce: is much more verb-like and owned by the subsystem (reducers) that you’re using (similar to doing BlockClosure>>value: in that respect, since a BlockClosure is a program and so is a Reducer).

Cheers,
Michael



_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Reducers API style question

Reinout Heeck-2
In reply to this post by Steffen Märcker
I think the English your code uses is both strained and and in reverse
order.

In the following I try to move away from the reducers as the central
paradigm, and more into a pragmatic imperative style to 'get the
reducing done'.

It seems to gravitate towards using only unary and binary selectors
(#>>, #<<, #asFilter and #asMap) to compose / create reducers and
keyword selectors to execute the reduction, which seems to make quite
readable code :-)


#odd asFilter >> #squared asMapping  reduce: #(1 2 3 4 5 6). "an array"

#squared asMapping << #odd asFilter reduce:  #(1 2 3 4 5 6).
"same array, for those insisting on backwards composition"

Flattner >> #odd asFilter >> #squared asMapping  reduce:  #((1 2 3) (4 5 6)).

#((1 2 3) (4 5 6)) reducedBy: Flattner >> #odd asFilter >> #squared asMapping.

squaredOdds := #odd asFilter >> #squared asMapping.

squaredOdds reduce: (1 to: 10) as: Set.

squaredOdds >> Set reduce: (1 to: 10).

(squaredOdds >>  #(1 2 3 4 5 6)) inject: 0 into: #+. "an array, do we need #injecting:into: to create a reducer?"

Flattner >> squaredOdds reduce: #(1 (2 3 (4 5 6))) as: OrderedCollection. "a collection"

Flattner >> squaredOdds << #(1 (2 3 (4 5 6))) >> OrderedCollection "a reducer, do we need #reduced or perhaps #value here?"

(1 to: 15) reducedBy: [:each | each + 1] >> squaredOdds .

[:x|x+1] reduce: (1 to: 10). "a collection"

[:x|x+1] << (1 to: 10) "a reducer"

view := [:x|x+1] >> #odd asFilter >> #squared asMapping .

wrapped_1 := view << (1 to: 10).

sum := wrapped_1 inject: 0 into: #+ . "computation is performed here"

wrapped_2 := view << #(1 2 3).

set := wrapped_2 as: Set. "computation is performed here"


Cheers,

Reinout
-------

On 5/6/2014 8:27 AM, Steffen Märcker wrote:

> Hi,
>
> I am about to simplify the API a bit and I am really eager to hear
> your opinion. Suppose we want to gather squares of odds numbers.
> Previously, we could write:
>
>    (Reducer map: #squared) <> (Reducer filter: #odd)
>       from: #(1 2 3 4 5 6).
>
> The repetition of 'Reducer' makes this pretty verbose. I intend to
> change this to
>
>    #squared mapping <> #odd filtering
>       from: #(1 2 3 4 5 6).
>
> However, flattening a nested collection requires no parameter (like
> mapping or filtering). Thus it does not quite fit this style
> (flattening is done by a singleton object):
>
>    #squared mapping <> #odd filtering <> ?_flattening_?
>       from: #((1 2 3) (4 5 6)).
>
> How would you implement flattening in this API style? So far, I see
> only two options:
>
> 1) Either extending the class side of Flattening to support the
> composition protocol:
>    #squared mapping <> #odd filtering <> Flattening
>
> 2) Or making the composition obvious by creating a flattening reducer
> object:
>    #squared mapping <> #odd filtering <> Flattening reducer
>
> Which option would your prefer? Do you have alternative ideas? Or
> would you even reject this API style completely?
>
> Kind regards,
> Steffen
> _______________________________________________
> vwnc mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
>
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Reducers API style question

Steffen Märcker
Hi Reinout,

thanks for your input. May I ask some questions for clarification?

> I think the English your code uses is both strained [...]

Since I'm not a native speaker, can you please elaborate in which way it  
is (feels?) strained?

> #odd asFilter >> #squared asMapping reduce: #(1 2 3 4 5 6).
> #squared asMapping << #odd asFilter reduce: #(1 2 3 4 5 6).

Interesting, similarly to Micheal, you'd let the selectors (#<< and #>>)  
indicate the direction of the composition.

However, your #reduce: message behaves very different because it takes a  
source collection and produces the result directly. Currently, Reducers  
builds on the idea that a collection has to implement just #reduce:init:  
to enable all other collection operations. The semantics of (#reduce:  
block int: val) is (inject: val into: block). A Reducer object is a  
virtual collection that wraps a collection and a transformation (map,  
filter, etc.) that is applied when #reduce:init: is called.

> Flattner >> #odd asFilter >> #squared asMapping  reduce:  #((1 2 3) (4 5  
> 6)).

You prefer to extend the class Flattener in order to be used directly. Is  
there a particular reason not to create an instance explicitly, e.g.,  
Flattener transform >> #odd asFilter ?

> squaredOdds := #odd asFilter >> #squared asMapping.
> squaredOdds >> Set reduce: (1 to: 10). "(1)"
> (squaredOdds >>  #(1 2 3 4 5 6)) inject: 0 into: #+. "(2)"

What is the proposed semantics of #>> here? It seems to 'bind' (1) a drain  
type as well as (2) source collection. What if the drain is an existing  
collection?

> Flattner >> squaredOdds << #(1 (2 3 (4 5 6))) >> OrderedCollection "a  
> reducer, do we need #reduced or perhaps #value here?"

What do you mean by a reducer here? I'd read the code to produce an  
OrderedCollection where the part

Flattner >> squaredOdds << #(1 (2 3 (4 5 6)))

produces an intermediate reducer object (virtual collection).

> (1 to: 15) reducedBy: [:each | each + 1] >> squaredOdds .
> [:x|x+1] reduce: (1 to: 10). "a collection"
> [:x|x+1] << (1 to: 10) "a reducer"

It seems ambiguous to me what the unary block does here. It doesn't tell  
whether it defines a mapping and a filter etc.


Kind regards,
Steffen
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Reducers API style question

Reinout Heeck-2
On 5/12/2014 2:54 PM, Steffen Märcker wrote:
>
>
>> I think the English your code uses is both strained [...]
>
> Since I'm not a native speaker, can you please elaborate in which way
> it is (feels?) strained?

Consider a snippet like
   #squared mapping <> #odd filtering

I sense it as strained in that it seems more like technical words
stacked on top of each other rather than a more abstract 'what am I
doing' language.
To me it feels like I have to envisage what the implementation will be
doing instead of 'what I want'. Maybe it has to do with Smalltalk idiom
too, me having been steeped in it for more than 15 years.
The strain seems to go away with a simple transformation:
    #odd asFilter >- #squared asMapping


>
>> #odd asFilter >> #squared asMapping reduce: #(1 2 3 4 5 6).
>> #squared asMapping << #odd asFilter reduce: #(1 2 3 4 5 6).
>
> Interesting, similarly to Micheal, you'd let the selectors (#<< and
> #>>) indicate the direction of the composition.

Yes, that seems to work very well :-)
Unfortunately Symbol (a collection)  implements both of these, let's try
#>- and #-< instead ;-)



>
> However, your #reduce: message behaves very different because it takes
> a source collection and produces the result directly.
Indeed (intentionally).

> Currently, Reducers builds on the idea that a collection has to
> implement just #reduce:init: to enable all other collection
> operations. The semantics of (#reduce: block int: val) is (inject: val
> into: block).
I find the second keyword 'init:' jarring.
The solution I used in my examples was to use #inject:into: instead
(because Smalltalkers already know how that works) but I can imagine you
want to use your domain's vocabulary. So perhaps #reduce:startingWith: ?

> A Reducer object is a virtual collection that wraps a collection and a
> transformation (map, filter, etc.) that is applied when #reduce:init:
> is called.

I'll try to discern between transformation and reducer, I lumped them
together in my previous post. However my language proposal also seems to
need a  (transformation+result kind) class or is that still a
transformation in your framework?


>
>> Flattner >> #odd asFilter >> #squared asMapping  reduce:  #((1 2 3)
>> (4 5 6)).
>
> You prefer to extend the class Flattener in order to be used directly.
> Is there a particular reason not to create an instance explicitly,
> e.g., Flattener transform >> #odd asFilter ?
No :-)
I propose Flattner is a global variable that holds a singleton instance
of the flatting transformation.

Flattener transform >> #odd asFilter
   sounds to me like the Flattner wil transform the rest of the
statement, which would work if there were no composing '>>' in there.
I guess I read the  'transform' as assembly code shining through the
composition I want to express.


>
>> squaredOdds := #odd asFilter >> #squared asMapping.
>> squaredOdds >> Set reduce: (1 to: 10). "(1)"
>> (squaredOdds >>  #(1 2 3 4 5 6)) inject: 0 into: #+. "(2)"
>
> What is the proposed semantics of #>> here? It seems to 'bind' (1) a
> drain type as well as (2) source collection. What if the drain is an
> existing collection?

I don't know the details of your framework, hence some hand waving:

Double dispatch is your friend.
It can do 'the right thing' and where what I tell it to do is wrong it
can fail fast.
The nice thing is that all those different type combinations can do the
right thing using a single composing symbol #>-.

So if I specify a drain type and use a drain instance (I don't know what
that does, add to an existing collection?) the code could complain (or
perhaps complain selectively if drain type and collection are incompatible).

Hope this answers the question :-)

>
>
>> Flattner >> squaredOdds << #(1 (2 3 (4 5 6))) >> OrderedCollection "a
>> reducer, do we need #reduced or perhaps #value here?"
>
> What do you mean by a reducer here? I'd read the code to produce an
> OrderedCollection where the part
>
> Flattner >> squaredOdds << #(1 (2 3 (4 5 6)))
>
> produces an intermediate reducer object (virtual collection).

That seems to be the (transformation+result kind) class I mentioned above.


But don't sweat it, if this concept doesn't exist in your framework then
throw an exception when I try to create that particular composition :-)
We are talking about the language here, not the semantics.

The important part of my proposal is that there are easily recognized
_separate_ languages for composition (using the same words to compose
transformations as well as reducers) vs running the reduction.

>
>> (1 to: 15) reducedBy: [:each | each + 1] >> squaredOdds .
>> [:x|x+1] reduce: (1 to: 10). "a collection"
>> [:x|x+1] << (1 to: 10) "a reducer"
>
> It seems ambiguous to me what the unary block does here. It doesn't
> tell whether it defines a mapping and a filter etc.

Interesting accident!

It suggests an asymmetry in that I automatically assumed mapping. You
could experiment with exploiting that, with #asMapping being implied
default:
   #odd asFilter >- #squared  "squared odds transformation"
   #squared >- #odd asFilter "odd squares transformation"

   #odd asFilter reduce: #(1 to: 10) "five integers"
   #odd reduce: #(1 to: 10) "ten booleans"

I'm not sure I like it though...




Enjoy!

Reinout
-------

>
>
> Kind regards,
> Steffen
> _______________________________________________
> vwnc mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
>
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Reducers API style question

Steffen Märcker
In reply to this post by Steffen Märcker
First of all, thanks a lot for you inspiring answers! =)

I'd like to focus on one idea and to discuss it a bit further.
Michael and Reinout suggested to indicate the direction of the data flow.  
Lets, consider a right-to-left variant:

   #squared mapping <~ #odd filtering <~ (1 to: 10).
   "\_ transf. __/     \_ transf. _/             /"
   " \______ transformation ______/             /"
   "  \__________ virtual collection __________/"

This creates a virtual collection that squares odd numbers from (1  
to:10).  It appears to be convenient to use #<~ for result aggregation,  
too. But we want to aggregate either in parallel or not. This  
functionality is implemented in two basic methods: (parallel) #fold:init:  
and #reduce:init:. Both have similar semantics to #inject:into: and thus,  
require a block/symbol and an initial value.

How would you express this using #<~ ?
Currently I can think of the following ways:

1) (pragmatic) Stick to the basic methods.

   (#squared mapping <~ #odd filtering <~ (1 to: 10)) reduce: #+ init: 0.

2) Let Symbol/Block reduce a (virtual) collection.

   #+ init: 0 reduce: (#odd filtering <~ #squared mapping <~ (1 to: 10)).

3) Create a 'Reduction' object.

   #+ <~ #odd filtering <~ #squared mapping <~ (1 to: 10).
   "\    \_ transf. _/      \_ transf. __/             /"
   " \_ reduction __/                   /             /"
   "  \__________ reduction ___________/             /"
   "   \__________________ result __________________/"

But How to inject the initial value? Maybe using a curried Reduction?

   (#+ init: 0) <~ #odd filtering <~ #squared mapping <~ (1 to: 10).
   "\curr. red/    \_ transf. _/      \_ transf. __/             /"
   " \___ curried reduction ____/                 /             /"
   "  \___________ curried reduction ____________/             /"
   "   \_______________________ result _______________________/"


I'd really appreciate your comments and suggestions on this API questions.  
Since the Reducers package aims to address (efficient) parallel collection  
processing, a usable API matters a lot.

Kind regards,
Steffen


PS: The classical API style is still in place to support easy code  
migration:

   (((1 to: 10) reducer map: #squared) filter: #odd) reduce: #+ init: 0..
   "\__ virt. col. __/              /             /"
   " \____ virtual collection _____/             /"
   "  \__________ virtual collection ___________/"
 
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Reducers API style question

Steffen Märcker
In reply to this post by Steffen Märcker
Hi!

I've just published a new version of Reducers to the public repository.  
This version simplifies the creation of transformers besides many  
improvements under the hood.

     (Reducer filter: #odd) <> (Reducer map: squared) <> (Reducer flatten)

is now written as

     #odd filtering <> #squared mapping <> Flattening transformer

However, the new version does not (yet) include the work towards a more  
data-flow like API using #<~ instead of #<>.

I also overhauled the package comment. I'd be happy to hear whether it  
makes thing clearer now. Therefore, I'll attach the comment below. =)

Best, Steffen



Reducers is a library for efficient collection processing. It emphasizes  
composition and lays a foundation for parallel collection processing.  
Support for custom collection-like types can be implemented easily.

Overview
## Source Collections
A source collection can be any object that implements #reduce:init: (also  
called a Reducible). Reducers provides the necessary extensions for  
collections, streams, random number generators and block closures that are  
used to compute a sequence of elements. The abstract class #Reducible  
provides additional convenient methods. For example, #cat: concatenates  
two collection lazily, i.e., without building a new collection.

## Transformations
Transformations, e.g., mapping and filtering, are defined independently of  
the type of the source collections using transformers. Applying a  
transformer to a collection creates a virtual collection (also called  
Reducer) that provides a transformed view on the source.

## Result Aggregation
After applying transformers, the result can be either aggregated in a  
collection or computed by calling #reduce:init: on a (virtual) collection.

Usage
Reducers provides two API styles for transformations: compositional and  
classic. The latter is similar to the ST collection API to facilitate easy  
code adaption. See the reducing protocol in Reducer.

## Introductory Example
The example uses symbols instead of blocks for the sake of brevity.

        | squares sum set |
        "squared odd numbers, virtual collection"
        squares := (#squared mapping <> #odd filtering) from: (1 to: 10).

        "calculating the sum"
        sum := squares reduce: #+ init: 0.

        "aggregating the numbers in a set"
        set := squares into: Set.

Using the classical transformation API, the first statement can be written  
as:

        squares := ((1 to: 10) reducing filter: #odd) map: #squared.

## Result Aggregation
The principal method to aggregate results is #reduce:init:. The semantic  
is similar to #inject:into: but differs in two points:
* Invoked with a ternary block or symbol (called reducing function), it  
will evaluate it with the running value and the current key and value.
* If a ReducedNotification is raised, #reduce:init: stops and returns the  
current running value.

        (1 to: 10) reduce: #+ init: 0.
        #(one two three)
                reduce: [:dict :k :v | dict at: k put: v; yourself]
                init: Dictionary new.

To aggregate results in a collection more easily, Reducers  provides  
#into: which takes either a collection or a collection class as argument.

        | col dict |
        col := OrderedCollection with: 0.
        (1 to: 10) into: col.
        dict := #(one two three) into: Dictionary

The method #into: sends #addAllFromReducible: to its argument. This method  
can be implemented by custom types to support easy result adding via  
#into:.

## Transformations
Reducers provides many common collection transformations, e.g., filtering,  
mapping and flattening. For a complete list see the subclasses of  
ReducingFunctionTransformer.
To create a transformer, send a corresponding message to either a block, a  
symbol or number.

        [:x | x < 0.01] filtering.
        #squared mapping.
        10 taking.

However, there is one exception:

        Flattening transformer.

Transformers can be composed and applied to (virtual) collections.

        | squares set |
        "composed transformer"
        squares := #squared mapping <> #odd filtering.

        "apply transformer and aggregate the numbers in a set"
        set := squares from: (1 to: 10) into: Set.
        (squares <> Flattening transformer) from: #(11 (12 (13))) into: set.

Implementing a new transformation is done by subclassing  
ReducingFunctionTransformer and adding constructor methods to either  
BlockClosure and Symbol or Integer. To support the classical API style, a  
similar method can be added to Reducer.

## Concatenating Collections
Collections (reducibles) can be concatenated using #cat:

        | cat |
        cat := (-1 to: -10) cat: (1 to: 10).

To support concatenation, custom types should implement #cat: to create a  
ConcatenatedReducible.

## Infinite Collections
Reducers can handle infinite reducibles, e.g., random numbers or  
generating blocks.

        | random small |
        random := Random new.
        "10 small random numbers"
        small := (10 taking <> [:x | x < 0.01] filtering) from: random.

        | n naturals odds |
        n := 0.
        naturals := [n := n+1].
        "first 10 odd naturals"
        odds := (10 taking <> #odd filtering) from: naturals.

## Inspiration
This package is based on Clojure's Reducers library. See:
http://clojure.com/blog/2012/05/08/reducers-a-library-and-model-for-collection-processing.html
http://clojure.com/blog/2012/05/15/anatomy-of-reducer.html
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Reducers API style question

Michael Lucas-Smith-2
Have you thought about when you might want to contact Niall about including Reducers as a goodie? I’m guessing it’d be after your API changes.

Michael

On 3 Jun 2014, at 5:26 pm, Steffen Märcker <[hidden email]> wrote:

> Hi!
>
> I've just published a new version of Reducers to the public repository. This version simplifies the creation of transformers besides many improvements under the hood.
>
>    (Reducer filter: #odd) <> (Reducer map: squared) <> (Reducer flatten)
>
> is now written as
>
>    #odd filtering <> #squared mapping <> Flattening transformer
>
> However, the new version does not (yet) include the work towards a more data-flow like API using #<~ instead of #<>.
>
> I also overhauled the package comment. I'd be happy to hear whether it makes thing clearer now. Therefore, I'll attach the comment below. =)
>
> Best, Steffen
>
>
>
> Reducers is a library for efficient collection processing. It emphasizes composition and lays a foundation for parallel collection processing. Support for custom collection-like types can be implemented easily.
>
> Overview
> ## Source Collections
> A source collection can be any object that implements #reduce:init: (also called a Reducible). Reducers provides the necessary extensions for collections, streams, random number generators and block closures that are used to compute a sequence of elements. The abstract class #Reducible provides additional convenient methods. For example, #cat: concatenates two collection lazily, i.e., without building a new collection.
>
> ## Transformations
> Transformations, e.g., mapping and filtering, are defined independently of the type of the source collections using transformers. Applying a transformer to a collection creates a virtual collection (also called Reducer) that provides a transformed view on the source.
>
> ## Result Aggregation
> After applying transformers, the result can be either aggregated in a collection or computed by calling #reduce:init: on a (virtual) collection.
>
> Usage
> Reducers provides two API styles for transformations: compositional and classic. The latter is similar to the ST collection API to facilitate easy code adaption. See the reducing protocol in Reducer.
>
> ## Introductory Example
> The example uses symbols instead of blocks for the sake of brevity.
>
> | squares sum set |
> "squared odd numbers, virtual collection"
> squares := (#squared mapping <> #odd filtering) from: (1 to: 10).
>
> "calculating the sum"
> sum := squares reduce: #+ init: 0.
>
> "aggregating the numbers in a set"
> set := squares into: Set.
>
> Using the classical transformation API, the first statement can be written as:
>
> squares := ((1 to: 10) reducing filter: #odd) map: #squared.
>
> ## Result Aggregation
> The principal method to aggregate results is #reduce:init:. The semantic is similar to #inject:into: but differs in two points:
> * Invoked with a ternary block or symbol (called reducing function), it will evaluate it with the running value and the current key and value.
> * If a ReducedNotification is raised, #reduce:init: stops and returns the current running value.
>
> (1 to: 10) reduce: #+ init: 0.
> #(one two three)
> reduce: [:dict :k :v | dict at: k put: v; yourself]
> init: Dictionary new.
>
> To aggregate results in a collection more easily, Reducers  provides #into: which takes either a collection or a collection class as argument.
>
> | col dict |
> col := OrderedCollection with: 0.
> (1 to: 10) into: col.
> dict := #(one two three) into: Dictionary
>
> The method #into: sends #addAllFromReducible: to its argument. This method can be implemented by custom types to support easy result adding via #into:.
>
> ## Transformations
> Reducers provides many common collection transformations, e.g., filtering, mapping and flattening. For a complete list see the subclasses of ReducingFunctionTransformer.
> To create a transformer, send a corresponding message to either a block, a symbol or number.
>
> [:x | x < 0.01] filtering.
> #squared mapping.
> 10 taking.
>
> However, there is one exception:
>
> Flattening transformer.
>
> Transformers can be composed and applied to (virtual) collections.
>
> | squares set |
> "composed transformer"
> squares := #squared mapping <> #odd filtering.
>
> "apply transformer and aggregate the numbers in a set"
> set := squares from: (1 to: 10) into: Set.
> (squares <> Flattening transformer) from: #(11 (12 (13))) into: set.
>
> Implementing a new transformation is done by subclassing ReducingFunctionTransformer and adding constructor methods to either BlockClosure and Symbol or Integer. To support the classical API style, a similar method can be added to Reducer.
>
> ## Concatenating Collections
> Collections (reducibles) can be concatenated using #cat:
>
> | cat |
> cat := (-1 to: -10) cat: (1 to: 10).
>
> To support concatenation, custom types should implement #cat: to create a ConcatenatedReducible.
>
> ## Infinite Collections
> Reducers can handle infinite reducibles, e.g., random numbers or generating blocks.
>
> | random small |
> random := Random new.
> "10 small random numbers"
> small := (10 taking <> [:x | x < 0.01] filtering) from: random.
>
> | n naturals odds |
> n := 0.
> naturals := [n := n+1].
> "first 10 odd naturals"
> odds := (10 taking <> #odd filtering) from: naturals.
>
> ## Inspiration
> This package is based on Clojure's Reducers library. See:
> http://clojure.com/blog/2012/05/08/reducers-a-library-and-model-for-collection-processing.html
> http://clojure.com/blog/2012/05/15/anatomy-of-reducer.html
> _______________________________________________
> vwnc mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Reducers API style question

Steffen Märcker
Hi Michael,

I didn't thought about providing Reducers as a goodie yet. Now, as you  
mention it, I'll contact Niall as soon as the last pieces dropped into  
place. Do you know the closing date for 8.0?
Actually, I'd be happy to get some more feedback on my the last API draft  
(based on <~) I posted in May. I'll attach the relevant parts below.

Best wishes,
Steffen


[...]
Currently I can think of the following ways:

1) (pragmatic) Stick to the basic methods.

   (#squared mapping <~ #odd filtering <~ (1 to: 10)) reduce: #+ init: 0.

2) Let Symbol/Block reduce a (virtual) collection.

   #+ init: 0 reduce: (#odd filtering <~ #squared mapping <~ (1 to: 10)).

3) Create a 'Reduction' object.

   #+ <~ #odd filtering <~ #squared mapping <~ (1 to: 10).
   "\    \_ transf. _/      \_ transf. __/             /"
   " \_ reduction __/                   /             /"
   "  \__________ reduction ___________/             /"
   "   \__________________ result __________________/"

But How to inject the initial value? Maybe using a curried Reduction?

   (#+ init: 0) <~ #odd filtering <~ #squared mapping <~ (1 to: 10).
   "\curr. red/    \_ transf. _/      \_ transf. __/             /"
   " \__ curried reduction ___/                   /             /"
   "  \___________ curried reduction ____________/             /"
   "   \_______________________ result _______________________/"


At the moment I am pretty keen on the last one. But I am not quite sure of  
the proper wording:
    (#+ init: 0)
or (#+ reducing: 0)
or (#+ reducingWith: 0) ?
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Reducers bugfix release Was: Reducers API style question

Steffen Märcker
In reply to this post by Steffen Märcker
Hi,

yesterday I became aware of a hidden bug that prevents stopping a  
reduction with a ReducedNotification. The bug is triggered by nested  
reductions (e.g. using flatten and mapcat) and has an observable impact  
only if the underlying collection is infinite, e.g., a stream of random  
numbers. For example:

     10 taking <> [:x | Array with: x with: x negated] mapcatting
         from: Random new into: Set.

I published a new revision fixing that bug and - as a side-effect -  
improves concatenation of reducibles.

Regards,
Steffen


Am .06.2014, 09:26 Uhr, schrieb Steffen Märcker <[hidden email]>:

> Hi!
>
> I've just published a new version of Reducers to the public repository.  
> This version simplifies the creation of transformers besides many  
> improvements under the hood.
>
>      (Reducer filter: #odd) <> (Reducer map: squared) <> (Reducer  
> flatten)
>
> is now written as
>
>      #odd filtering <> #squared mapping <> Flattening transformer
>
> However, the new version does not (yet) include the work towards a more  
> data-flow like API using #<~ instead of #<>.
>
> I also overhauled the package comment. I'd be happy to hear whether it  
> makes thing clearer now. Therefore, I'll attach the comment below. =)
>
> Best, Steffen
>
>
>
> Reducers is a library for efficient collection processing. It emphasizes  
> composition and lays a foundation for parallel collection processing.  
> Support for custom collection-like types can be implemented easily.
>
> Overview
> ## Source Collections
> A source collection can be any object that implements #reduce:init:  
> (also called a Reducible). Reducers provides the necessary extensions  
> for collections, streams, random number generators and block closures  
> that are used to compute a sequence of elements. The abstract class  
> #Reducible provides additional convenient methods. For example, #cat:  
> concatenates two collection lazily, i.e., without building a new  
> collection.
>
> ## Transformations
> Transformations, e.g., mapping and filtering, are defined independently  
> of the type of the source collections using transformers. Applying a  
> transformer to a collection creates a virtual collection (also called  
> Reducer) that provides a transformed view on the source.
>
> ## Result Aggregation
> After applying transformers, the result can be either aggregated in a  
> collection or computed by calling #reduce:init: on a (virtual)  
> collection.
>
> Usage
> Reducers provides two API styles for transformations: compositional and  
> classic. The latter is similar to the ST collection API to facilitate  
> easy code adaption. See the reducing protocol in Reducer.
>
> ## Introductory Example
> The example uses symbols instead of blocks for the sake of brevity.
>
> | squares sum set |
> "squared odd numbers, virtual collection"
> squares := (#squared mapping <> #odd filtering) from: (1 to: 10).
>
> "calculating the sum"
> sum := squares reduce: #+ init: 0.
>
> "aggregating the numbers in a set"
> set := squares into: Set.
>
> Using the classical transformation API, the first statement can be  
> written as:
>
> squares := ((1 to: 10) reducing filter: #odd) map: #squared.
>
> ## Result Aggregation
> The principal method to aggregate results is #reduce:init:. The semantic  
> is similar to #inject:into: but differs in two points:
> * Invoked with a ternary block or symbol (called reducing function), it  
> will evaluate it with the running value and the current key and value.
> * If a ReducedNotification is raised, #reduce:init: stops and returns  
> the current running value.
>
> (1 to: 10) reduce: #+ init: 0.
> #(one two three)
> reduce: [:dict :k :v | dict at: k put: v; yourself]
> init: Dictionary new.
>
> To aggregate results in a collection more easily, Reducers  provides  
> #into: which takes either a collection or a collection class as argument.
>
> | col dict |
> col := OrderedCollection with: 0.
> (1 to: 10) into: col.
> dict := #(one two three) into: Dictionary
>
> The method #into: sends #addAllFromReducible: to its argument. This  
> method can be implemented by custom types to support easy result adding  
> via #into:.
>
> ## Transformations
> Reducers provides many common collection transformations, e.g.,  
> filtering, mapping and flattening. For a complete list see the  
> subclasses of ReducingFunctionTransformer.
> To create a transformer, send a corresponding message to either a block,  
> a symbol or number.
>
> [:x | x < 0.01] filtering.
> #squared mapping.
> 10 taking.
>
> However, there is one exception:
>
> Flattening transformer.
>
> Transformers can be composed and applied to (virtual) collections.
>
> | squares set |
> "composed transformer"
> squares := #squared mapping <> #odd filtering.
>
> "apply transformer and aggregate the numbers in a set"
> set := squares from: (1 to: 10) into: Set.
> (squares <> Flattening transformer) from: #(11 (12 (13))) into: set.
>
> Implementing a new transformation is done by subclassing  
> ReducingFunctionTransformer and adding constructor methods to either  
> BlockClosure and Symbol or Integer. To support the classical API style,  
> a similar method can be added to Reducer.
>
> ## Concatenating Collections
> Collections (reducibles) can be concatenated using #cat:
>
> | cat |
> cat := (-1 to: -10) cat: (1 to: 10).
>
> To support concatenation, custom types should implement #cat: to create  
> a ConcatenatedReducible.
>
> ## Infinite Collections
> Reducers can handle infinite reducibles, e.g., random numbers or  
> generating blocks.
>
> | random small |
> random := Random new.
> "10 small random numbers"
> small := (10 taking <> [:x | x < 0.01] filtering) from: random.
>
> | n naturals odds |
> n := 0.
> naturals := [n := n+1].
> "first 10 odd naturals"
> odds := (10 taking <> #odd filtering) from: naturals.
>
> ## Inspiration
> This package is based on Clojure's Reducers library. See:
> http://clojure.com/blog/2012/05/08/reducers-a-library-and-model-for-collection-processing.html
> http://clojure.com/blog/2012/05/15/anatomy-of-reducer.html
> _______________________________________________
> vwnc mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Reducers API style question

Steffen Märcker
In reply to this post by Steffen Märcker
Hi!

I published a new Reducers version that alters the composition API to  
emphasize the data flow using #<~. This major change renders #<>, #from:  
and #into: obsolete.

     #even filtering <~ Flattening <~ #(1 2 (3 4 (5 6))).
     Set <~ #squared mapping <~ #(-10 to: 10).
     (#+ reduction: 0) <~ #odd filtering <~ (1 to: 10)

The last statement introduces a new type: Reduction = function + initial  
value. You can think of a reduction as a reusable deferred computation in  
terms of #reduce:init:. It can be combined with a transformer, too.  
However, I am not quite happy with the selector #reduction: yet. I am  
eager to hear your opinion on alternatives before I'll fix this part of  
the API. Maybe

     #+ init: 0.
     #+ reducingWith: 0.
     #+ reductionWith: 0.

...?

Best regards,
Steffen

Am .05.2014, 08:27 Uhr, schrieb Steffen Märcker <[hidden email]>:

> Hi,
>
> I am about to simplify the API a bit and I am really eager to hear your  
> opinion. Suppose we want to gather squares of odds numbers. Previously,  
> we could write:
>
>     (Reducer map: #squared) <> (Reducer filter: #odd)
>        from: #(1 2 3 4 5 6).
>
> The repetition of 'Reducer' makes this pretty verbose. I intend to  
> change this to
>
>     #squared mapping <> #odd filtering
>        from: #(1 2 3 4 5 6).
>
> However, flattening a nested collection requires no parameter (like  
> mapping or filtering). Thus it does not quite fit this style (flattening  
> is done by a singleton object):
>
>     #squared mapping <> #odd filtering <> ?_flattening_?
>        from: #((1 2 3) (4 5 6)).
>
> How would you implement flattening in this API style? So far, I see only  
> two options:
>
> 1) Either extending the class side of Flattening to support the  
> composition protocol:
>     #squared mapping <> #odd filtering <> Flattening
>
> 2) Or making the composition obvious by creating a flattening reducer  
> object:
>     #squared mapping <> #odd filtering <> Flattening reducer
>
> Which option would your prefer? Do you have alternative ideas? Or would  
> you even reject this API style completely?
>
> Kind regards,
> Steffen
> _______________________________________________
> vwnc mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Reducers API style question

Paul Baumann
>> I am not quite happy with the selector #reduction: yet. I am eager to hear your opinion on alternatives before I'll fix this part of the API.

If I understand what you are proposing then:

     (#+ reduction: 0) <~ #odd filtering <~ (1 to: 10)

would answer 25 just as:

    ((1 to: 10) select: [:ea | ea odd ])
         inject: 0 into: [:sum :ea | sum + ea ]

The argument of #reduction: is the initial value, correct? Call it #inject: so that it is familiar.

Paul Baumann


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Steffen Märcker
Sent: Monday, June 16, 2014 12:43
To: [hidden email]
Subject: Re: [vwnc] Reducers API style question

Hi!

I published a new Reducers version that alters the composition API to emphasize the data flow using #<~. This major change renders #<>, #from:
and #into: obsolete.

     #even filtering <~ Flattening <~ #(1 2 (3 4 (5 6))).
     Set <~ #squared mapping <~ #(-10 to: 10).
     (#+ reduction: 0) <~ #odd filtering <~ (1 to: 10)

The last statement introduces a new type: Reduction = function + initial value. You can think of a reduction as a reusable deferred computation in terms of #reduce:init:. It can be combined with a transformer, too.
However, I am not quite happy with the selector #reduction: yet. I am eager to hear your opinion on alternatives before I'll fix this part of the API. Maybe

     #+ init: 0.
     #+ reducingWith: 0.
     #+ reductionWith: 0.

...?

Best regards,
Steffen

Am .05.2014, 08:27 Uhr, schrieb Steffen Märcker <[hidden email]>:

> Hi,
>
> I am about to simplify the API a bit and I am really eager to hear
> your opinion. Suppose we want to gather squares of odds numbers.
> Previously, we could write:
>
>     (Reducer map: #squared) <> (Reducer filter: #odd)
>        from: #(1 2 3 4 5 6).
>
> The repetition of 'Reducer' makes this pretty verbose. I intend to
> change this to
>
>     #squared mapping <> #odd filtering
>        from: #(1 2 3 4 5 6).
>
> However, flattening a nested collection requires no parameter (like
> mapping or filtering). Thus it does not quite fit this style
> (flattening is done by a singleton object):
>
>     #squared mapping <> #odd filtering <> ?_flattening_?
>        from: #((1 2 3) (4 5 6)).
>
> How would you implement flattening in this API style? So far, I see
> only two options:
>
> 1) Either extending the class side of Flattening to support the
> composition protocol:
>     #squared mapping <> #odd filtering <> Flattening
>
> 2) Or making the composition obvious by creating a flattening reducer
> object:
>     #squared mapping <> #odd filtering <> Flattening reducer
>
> Which option would your prefer? Do you have alternative ideas? Or
> would you even reject this API style completely?
>
> Kind regards,
> Steffen
> _______________________________________________
> vwnc mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Reducers API style question

Steffen Märcker
Hi Paul,

#inject: reads very reasonable. =) Suppose you have a second, very similar  
message, e.g. #squash:inject: and want to create a corresponding Squash  
object. How would deal with both cases? I tend to go for

     (#+ inject: 0) "Reduction"
     (#+ inject: 0) squashing "Squash"

since something

    (#+ reduceInject: 0)
    (#+ squashInject: 0)

reads rather clumsy.

Cheers,
Steffen


Am .06.2014, 16:09 Uhr, schrieb Paul Baumann <[hidden email]>:

<---Schnitt--->

>
> If I understand what you are proposing then:
>
>      (#+ reduction: 0) <~ #odd filtering <~ (1 to: 10)
>
> would answer 25 just as:
>
>     ((1 to: 10) select: [:ea | ea odd ])
>          inject: 0 into: [:sum :ea | sum + ea ]
>
> The argument of #reduction: is the initial value, correct? Call it  
> #inject: so that it is familiar.
>
> Paul Baumann
>
>
> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]] On  
> Behalf Of Steffen Märcker
> Sent: Monday, June 16, 2014 12:43
> To: [hidden email]
> Subject: Re: [vwnc] Reducers API style question
>
> Hi!
>
> I published a new Reducers version that alters the composition API to  
> emphasize the data flow using #<~. This major change renders #<>, #from:
> and #into: obsolete.
>
>      #even filtering <~ Flattening <~ #(1 2 (3 4 (5 6))).
>      Set <~ #squared mapping <~ #(-10 to: 10).
>      (#+ reduction: 0) <~ #odd filtering <~ (1 to: 10)
>
> The last statement introduces a new type: Reduction = function + initial  
> value. You can think of a reduction as a reusable deferred computation  
> in terms of #reduce:init:. It can be combined with a transformer, too.
> However, I am not quite happy with the selector #reduction: yet. I am  
> eager to hear your opinion on alternatives before I'll fix this part of  
> the API. Maybe
>
>      #+ init: 0.
>      #+ reducingWith: 0.
>      #+ reductionWith: 0.
>
> ...?
>
> Best regards,
> Steffen
>
> Am .05.2014, 08:27 Uhr, schrieb Steffen Märcker <[hidden email]>:
>
>> Hi,
>>
>> I am about to simplify the API a bit and I am really eager to hear
>> your opinion. Suppose we want to gather squares of odds numbers.
>> Previously, we could write:
>>
>>     (Reducer map: #squared) <> (Reducer filter: #odd)
>>        from: #(1 2 3 4 5 6).
>>
>> The repetition of 'Reducer' makes this pretty verbose. I intend to
>> change this to
>>
>>     #squared mapping <> #odd filtering
>>        from: #(1 2 3 4 5 6).
>>
>> However, flattening a nested collection requires no parameter (like
>> mapping or filtering). Thus it does not quite fit this style
>> (flattening is done by a singleton object):
>>
>>     #squared mapping <> #odd filtering <> ?_flattening_?
>>        from: #((1 2 3) (4 5 6)).
>>
>> How would you implement flattening in this API style? So far, I see
>> only two options:
>>
>> 1) Either extending the class side of Flattening to support the
>> composition protocol:
>>     #squared mapping <> #odd filtering <> Flattening
>>
>> 2) Or making the composition obvious by creating a flattening reducer
>> object:
>>     #squared mapping <> #odd filtering <> Flattening reducer
>>
>> Which option would your prefer? Do you have alternative ideas? Or
>> would you even reject this API style completely?
>>
>> Kind regards,
>> Steffen
>> _______________________________________________
>> vwnc mailing list
>> [hidden email]
>> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
>
> _______________________________________________
> vwnc mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
>
> This message may contain confidential information and is intended for  
> specific recipients unless explicitly noted otherwise. If you have  
> reason to believe you are not an intended recipient of this message,  
> please delete it and notify the sender. This message may not represent  
> the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or  
> affiliates, and does not constitute a contract or guarantee. Unencrypted  
> electronic mail is not secure and the recipient of this message is  
> expected to provide safeguards from viruses and pursue alternate means  
> of communication where privacy or a binding message is desired.


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Reducers API style question

Paul Baumann
Steffen,

I don't know how you have the reduction stuff implemented, but I had been thinking along the lines of  using #inject: to configure a reduction. Something like this:

        (#+ reduction inject: 0) <~ #odd filtering <~ (1 to: 10)

The same would likely work for whatever #squashing does. Allowing the result of #reduction and #squashing to be configured would reduce the number of  extensions you need to maintain.

These are style issues, do it however you believe makes sense.

Paul Baumann


-----Original Message-----
From: Steffen Märcker [mailto:[hidden email]]
Sent: Wednesday, June 18, 2014 05:05
To: [hidden email]; Paul Baumann
Subject: Re: [vwnc] Reducers API style question

Hi Paul,

#inject: reads very reasonable. =) Suppose you have a second, very similar message, e.g. #squash:inject: and want to create a corresponding Squash object. How would deal with both cases? I tend to go for

     (#+ inject: 0) "Reduction"
     (#+ inject: 0) squashing "Squash"

since something

    (#+ reduceInject: 0)
    (#+ squashInject: 0)

reads rather clumsy.

Cheers,
Steffen


Am .06.2014, 16:09 Uhr, schrieb Paul Baumann <[hidden email]>:

<---Schnitt--->

>
> If I understand what you are proposing then:
>
>      (#+ reduction: 0) <~ #odd filtering <~ (1 to: 10)
>
> would answer 25 just as:
>
>     ((1 to: 10) select: [:ea | ea odd ])
>          inject: 0 into: [:sum :ea | sum + ea ]
>
> The argument of #reduction: is the initial value, correct? Call it
> #inject: so that it is familiar.
>
> Paul Baumann
>
>
> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]] On
> Behalf Of Steffen Märcker
> Sent: Monday, June 16, 2014 12:43
> To: [hidden email]
> Subject: Re: [vwnc] Reducers API style question
>
> Hi!
>
> I published a new Reducers version that alters the composition API to
> emphasize the data flow using #<~. This major change renders #<>, #from:
> and #into: obsolete.
>
>      #even filtering <~ Flattening <~ #(1 2 (3 4 (5 6))).
>      Set <~ #squared mapping <~ #(-10 to: 10).
>      (#+ reduction: 0) <~ #odd filtering <~ (1 to: 10)
>
> The last statement introduces a new type: Reduction = function +
> initial value. You can think of a reduction as a reusable deferred
> computation in terms of #reduce:init:. It can be combined with a transformer, too.
> However, I am not quite happy with the selector #reduction: yet. I am
> eager to hear your opinion on alternatives before I'll fix this part
> of the API. Maybe
>
>      #+ init: 0.
>      #+ reducingWith: 0.
>      #+ reductionWith: 0.
>
> ...?
>
> Best regards,
> Steffen
>
> Am .05.2014, 08:27 Uhr, schrieb Steffen Märcker <[hidden email]>:
>
>> Hi,
>>
>> I am about to simplify the API a bit and I am really eager to hear
>> your opinion. Suppose we want to gather squares of odds numbers.
>> Previously, we could write:
>>
>>     (Reducer map: #squared) <> (Reducer filter: #odd)
>>        from: #(1 2 3 4 5 6).
>>
>> The repetition of 'Reducer' makes this pretty verbose. I intend to
>> change this to
>>
>>     #squared mapping <> #odd filtering
>>        from: #(1 2 3 4 5 6).
>>
>> However, flattening a nested collection requires no parameter (like
>> mapping or filtering). Thus it does not quite fit this style
>> (flattening is done by a singleton object):
>>
>>     #squared mapping <> #odd filtering <> ?_flattening_?
>>        from: #((1 2 3) (4 5 6)).
>>
>> How would you implement flattening in this API style? So far, I see
>> only two options:
>>
>> 1) Either extending the class side of Flattening to support the
>> composition protocol:
>>     #squared mapping <> #odd filtering <> Flattening
>>
>> 2) Or making the composition obvious by creating a flattening reducer
>> object:
>>     #squared mapping <> #odd filtering <> Flattening reducer
>>
>> Which option would your prefer? Do you have alternative ideas? Or
>> would you even reject this API style completely?
>>
>> Kind regards,
>> Steffen
>> _______________________________________________
>> vwnc mailing list
>> [hidden email]
>> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
>
> _______________________________________________
> vwnc mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
>
> This message may contain confidential information and is intended for
> specific recipients unless explicitly noted otherwise. If you have
> reason to believe you are not an intended recipient of this message,
> please delete it and notify the sender. This message may not represent
> the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries
> or affiliates, and does not constitute a contract or guarantee.
> Unencrypted electronic mail is not secure and the recipient of this
> message is expected to provide safeguards from viruses and pursue
> alternate means of communication where privacy or a binding message is desired.



This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc