Pipe syntax and the current methods

Would any of the "pipe" advocates mind taking a stab at the *current* source
methods, and rewrite the method showing how pipe syntax would have simplified
or clarified the method?

I ask this because I suspect that if you're following good practices (such as
those adovocated in Beck's "Smalltalk Best Practice Patterns"), you won't
actually *need* a pipe syntax, because your code would never have gotten that
complicated.

So, instead of writing Smalltalk with a bias for your previous programming
language where pipe makes more sense, how about taking some *native* Smalltalk
to show how pipe would have helped?

--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<[hidden email]> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

Hi Randal,

Thanks for the suggestion.  

To be clear, even though I support using the symbol ';;' as pipe operator IF
we put it in, I doubt I would actually ever use it.  I prefer parenthesis,
and agree that if you are many levels nested you probably need to ether
create more objects to model your complicated behavior, or you need to add
temps to make it easier to read your code.

On the other hand I find the cascade extremely useful, especially for
creation methods.  I find chaining more tolerable when the object doesn't
change.  It just makes more sense.  For the same reason I usually look at
long chained methods as an indication that the code is probably written on
the wrong class and needs to be cleaned up.

Ron

It's not like you even need a new syntax to avoid the paranthesis - just
new kinds of objects, eg:

nonHermsOlderThanTensMothersNames := lotsOfPeople selecting: [:each |
each age > 10]; rejecting: [:each | each sex = #unknown]; collecting:
[:each | each mother]; selecting: [:each | each notNil]; collecting:
[:each | each name]; asOrderedCollection

No new syntax, just new objects. Has this already been covered in the
thread? if so, sorry for bringing it up again.

Michael

Ron Teitelbaum wrote:

I haven't seen it, good catch.

On 8/27/07, Michael Lucas-Smith <[hidden email]> wrote:

I agree (also with what Randal said).  I view myself as someone who
uses functional paradigms quite often, but Smalltalk is not a
functional language.  It's a beautiful blend of pure OO and higher
order programming.  Just as a solution would look different in Haskell
then e.g. Lisp, it will look different in Smalltalk too.

On 8/27/07, Ron Teitelbaum <[hidden email]> wrote:

Michael Lucas-Smith wrote: But it's not fully equivalent.

Adding new methods that return results (and not self) must be done at
least once per selector--and requires a naming convention that
associates the "return receiver" with the "return result" version of
each selector (and there would be many cases where the distinction
wouldn't matter.)

Adding a new "Pipe" class (e.g., as in Bert's "Squeak asPipe" example)
results in an implementation that isn't thread safe (if the Pipe object
isn't private to a single thread.)  And it's not as efficient as
syntax-based piping.

The pipe operator is a single, always thread-safe, efficient change. But
that doesn't mean I endorse it.  I am still considering the matter.

"Randal L. Schwartz" <[hidden email]> wrote in message
news:[hidden email]...
>
> Would any of the "pipe" advocates mind taking a stab at the *current*
> source
> methods, and rewrite the method showing how pipe syntax would have
> simplified
> or clarified the method?

I wouldn't describe myself as an 'advocate', but I don't mind exploring the
concept.
So here is an example..

Collection has a helper method #select:thenCollect:  .
Looking at the senders of that, I found ChangeSet
class>>highestNumberedChangeSet
(I have reformatted the source to make things clearer)

   highestNumberedChangeSet
      "ChangeSorter highestNumberedChangeSet"
     | aList |
     aList := self allChangeSetNames
        select: [:aString | aString startsWithDigit]
        thenCollect: [:aString | aString initialIntegerOrNil].
     ^aList size > 0
        ifTrue: [aList max]
        ifFalse: [nil]

With pipes, this could be written as

    highestNumberedChangeSet
        "ChangeSorter highestNumberedChangeSet"
        ^self allChangeSetNames
            select:[:aString | aString startsWithDigit]  ;;
            collect:[:aString | aString initialIntegerOrNil] ;;
            ifNotEmpty:[:list | list max]

The problem with these kind of helper methods is that there many other
permutations (collect:thenDo:, collect:thenSelect: select:thenDo:
select:thenCollect:thenDetect: etc.).
Either, methods are created for all permutations, and most of them never
actually used. (Collection>>collect:thenDo: has no senders).
Or, only those with senders are added, and the question is asked, for
exanple, :-  Why is there a #select:thenDo: but no #collect:thenDo: ?

With the pipe you get all permutations, without needing to create methods
for them all.

Here is another example; a snippet from Morph>>renameTo:

  classes := (self systemNavigation allCallsOn: assoc)
    collect: [:each | each classSymbol].
  classes asSet
    do: [:clsName | (Smalltalk at: clsName) replaceSilently: oldName to:
aName].

which, with pipes, could be rewritten as...

  self systemNavigation
    allCallsOn: assoc ;;
    collect: [:each | each classSymbol] ;;
    asSet ;;
    do: [:clsName | (Smalltalk at: clsName) replaceSilently: oldName to:
aName].

Cheers,
Andy

> With pipes, this could be written as
>
>    highestNumberedChangeSet
>        "ChangeSorter highestNumberedChangeSet"
>        ^self allChangeSetNames
>            select:[:aString | aString startsWithDigit]  ;;
>            collect:[:aString | aString initialIntegerOrNil] ;;
>            ifNotEmpty:[:list | list max]
>
With pipe objects using standard smalltalk syntax, this could be written as:

highestNumberedChangeSet
     "ChangeSorter highestNumberedChangeSet"
       ^self allChangeSetNames asPipe
           selecting: [:aString | aString startsWithDigit];
           collecting: [:aString | aString initialIntegerOrNil];
           ifNotEmpty: [:list | list max]
> which, with pipes, could be rewritten as...
>
>  self systemNavigation
>    allCallsOn: assoc ;;
>    collect: [:each | each classSymbol] ;;
>    asSet ;;
>    do: [:clsName | (Smalltalk at: clsName) replaceSilently: oldName
> to: aName].
>
And again:

(self systemNavigation allCallsOn: assoc) asPipe
   collecting: [:each | each classSymbol];
   selectingAsUniqueSet;
   do: [:clsName | (Smalltalk at: clsName) replaceSilently: oldName to:
aName]

I guess I should point out that work like this has been done in
VisualWorks from two fronts - StreamWrappers and ComputingStreams. Both
packages are available in public store.

Pipes are a great idea - streams talking to streams is the only way to
do efficient large-data-set programming (eg: google's map+reduce technique).

I wish more of Smalltalk were written with this approach in mind, it'd
scale without effort then.. and programmers wouldn't accidently create
memory explosion bottlenecks without trying. Multiple select:, collect:,
reject: calls on large data sets will bring any image to its knees in
seconds if more than one concurrent user invokes the same sort of
operation at once.

The speed issue comes not from the time it takes to do one call of this
- but what happens when multiple processes try to do the same thing (eg:
multiple users hitting your server at once). And the speed issue comes
in not from computing CPU cycles, but from memory allocation and garbage
collection.

If you start with a collection of 100,000 things, you do 4 operations on
it - three collect:'s and a reject:.. you'll probably be allocating 4
arrays of 100,000 slots. That's 1.2mb's of data you've just allocated.
Now get 10 users using the same function at the same time and you've
just made 12mb's of data. Scale it up a little more elaborate chains of
functions or more users and you have serious scalability issues.

Now to put the point home - if you are generating web pages from the
server. You start with a parse of a node tree which concatenates dozens
of little strings together to produce the page - which pushes it through
a zip library - which pushes it through a chunks stream - perhaps
there's a utf8 encoder in there too. Unless all those streams are using
a cyclic buffer, streams to streams, they're going to be generating LOTS
of small and big strings as they build up their own internal streams and
buffers.

Anyway.. just food for thought. At Wizard we spent a considerably amount
of time optimizing our Http/1.1 server to deal with exactly this sort of
thing. We also found we could use the same code for database operations
too (we were using BerkeleyDB as our database, so querying was done by us).

Cheers,
Michael

On Aug 27, 2007, at 13:38 , Michael Lucas-Smith wrote:

With the generic pipe object from my change-set in the original  
thread this gets you both - no need to define new methods:

    highestNumberedChangeSet
        "ChangeSorter highestNumberedChangeSet"
        ^self allChangeSetNames asPipe
            select:[:aString | aString startsWithDigit];
            collect:[:aString | aString initialIntegerOrNil];
            ifNotEmpty:[:list | list max]

and again, too:

self systemNavigation asPipe
    allCallsOn: assoc;
    collect: [:each | each classSymbol];
    asSet;
    do: [:clsName | (Smalltalk at: clsName) replaceSilently: oldName  
to: aName].

- Bert -

OK, now I'm confused. <s>

I thought if you had:

a msg | msg1 | msg2

msg1 went to the object (we'll call it "b") returned by "a msg", and msg2  
went to the object created by "b msg1". In other words, the equivalent of:

b := a msg.
c := b msg1.
c msg2.

But this to me:

>     highestNumberedChangeSet
>         "ChangeSorter highestNumberedChangeSet"
>         ^self allChangeSetNames asPipe
>             select:[:aString | aString startsWithDigit];
>             collect:[:aString | aString initialIntegerOrNil];
>             ifNotEmpty:[:list | list max]

seems more like:

b := a msg.
b msg1.
b msg2.

        Do I misunderstand? (And at what point does the potential for confusion  
and ambiguity outweigh the feature?)

        ===Blake===

Blake wrote:
> b := a msg.
> b msg1.
> b msg2.
>
>     Do I misunderstand? (And at what point does the potential for
> confusion and ambiguity outweigh the feature?)
>
Yes.

The message sent to a is #asPipe, not msg:

p := a asPipe.
p msg.
p msg1.
p msg2

Or alternatively:

a asPipe
    msg;
    msg1;
    msg2

The Pipe arranges for each message to be sent to the right object.

<snip>
> With the generic pipe object from my change-set in the original  thread
> this gets you both - no need to define new methods:
>
>    highestNumberedChangeSet
>        "ChangeSorter highestNumberedChangeSet"
>        ^self allChangeSetNames asPipe
>            select:[:aString | aString startsWithDigit];
>            collect:[:aString | aString initialIntegerOrNil];
>            ifNotEmpty:[:list | list max]

Yes. I am coming to the conclusion that there is no need for additional
syntax; your solution works really well.
I would suggest, however, that Pipe is made a subclass of ProtoObject rather
than Object. This will avoid oddities such as...

    (1 asPipe + 1; + 1; = 3) evaluates to false

but with Pipe a subclass of ProtoObject, it evaluates to true, as expected.

Cheers,
Andy

On Aug 27, 2007, at 16:08 , Alan Lovejoy wrote:


and it does so by retaining the previous value inside the Pipe  
object, so this is equivalent to

p := a asPipe. "p value: a"
p msg.         "p value: p value msg"
p msg1.        "p value: p value msg1"
p msg2.        "p value: p value msg2"


>>   (And at what point does the potential for confusion and  
>> ambiguity outweigh the feature?)

At the point where this code would leave my machine ... as I said,  
I'd not use that in production. In that sense, adding new syntax  
actually provides clarity.

Which reminds me - actually spelling this out using a running value  
is what I would use in production:

val := a.
val := val msg.
val := val msg1.
val := val msg2.
^val

I use this code pattern sometimes. You couldn't do this in a  
statically typed language, but in Smalltalk

- Bert -

On Aug 27, 2007, at 17:32 , Andrew Tween wrote:

Sure. It just immensely improves debuggability if you start with an  
Object subclass first ;)  Also, the reasonable use cases would  
usually involve only methods that are not in Object. Provided someone  
would actually want to play these meta games ;)

- Bert -

>>>>> "Bert" == Bert Freudenberg <[hidden email]> writes:

Bert> Which reminds me - actually spelling this out using a running value  is what I
Bert> would use in production:

Bert> val := a.
Bert> val := val msg.
Bert> val := val msg1.
Bert> val := val msg2.
Bert> ^val

Yeah, and this is single-steppable, and "self halt"-able.
These pipe schemes seem more like pipe dreams. :)

--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<[hidden email]> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

I wouldn't say that, while the above may be simple, it's not very concise,
and it's no more debuggable.  Once in the dubugger you could easily
highlight and inspect the value of any inner expression without the need to
shove it into a temporary first.

Burt's pipe hack is quite nice and with the addition I posted the meta stuff
is optimized out very quickly and you end up with a pipe object doing
ordinary message sends.  It's also far more concise than the above without
being at all cryptic.

I'll take this any day

    ^val asPipe
        msg;
        msg1;
        msg2

Ramon Leon
http://onsmalltalk.com

Fabio FIlasieno

On Aug 28, 2007, at 1:58 , Fabio Filasieno wrote:
But also, newcomers to Smalltalk need to understand that to  
Smalltalkers, the *language* part is actually not all that important.  
This is very different from other communities like the ones you list,  
which are in fact centered around their language. For us Smalltalkers  
it is the environment that counts most, not the syntax.

- Bert -

On 28-Aug-07, at 2:41 PM, Bert Freudenberg wrote:

>
> But also, newcomers to Smalltalk need to understand that to  
> Smalltalkers, the *language* part is actually not all that  
> important. This is very different from other communities like the  
> ones you list, which are in fact centered around their language.  
> For us Smalltalkers it is the environment that counts most, not the  
> syntax.
Well yes, but.

The environment is only possible because we have a rather good  
semantic model serviced via a good, clean, regular, intelligible  
syntax. And some smart people using it, of course.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Useful random insult:- Hypnotized as a child and couldn't be woken.