doInParallel:

Dolphin experts,

I would like to process the elements of an ordered collection in parallel.  After lots
of exploring the image, trail, and error, the following example solution emerged:

   | outerSemaphore innerSemaphore array |
   array := #(1 2 3 4 5) copy.
   outerSemaphore := Semaphore new
      primSetSignals: 1 - array size;
      yourself.
   innerSemaphore := Semaphore new.
   array do: [ :index |
      [
         | tmpIndex |
         tmpIndex := index.
         "(Delay forSeconds: 1) wait."
         innerSemaphore signal.
         "(Delay forSeconds: 1) wait."
         array at: tmpIndex put: tmpIndex + 1.
         outerSemaphore signal.
      ] fork.
      innerSemaphore wait.
   ].
   outerSemaphore wait.
   ^array.

The inner semaphore is used to protect access to the block variable.  The outer one is
to make the whole thing block until the entire array is processed.

There are a couple of problems with this.  If the second delay line is uncommented, the
answer is incorrect.  Waiting there allows the block to rerun and tmpIndex gets
overwritten before it is used.  The second problem is that I can't get a more general
solution to work when it is added to OrderedCollection or some of its superclasses so
that I can do

#(1 2 3) doInParallel: [ :each | each something ].

Besides, if no delays are allowed in the block, this isn't a very handy construction.

Both of these problems may have to do with the not quite block closures that Dolphin
5 is reported to have.  Does anyone know if that's the case?  Also, will this work
better in Dolphin 6: without the inner semaphore or with allowing delays inside the
block?  If someone has a better solution, please let me know.  Thanks.

Keith

You wrote in message news:[hidden email]...
> Dolphin experts,
>
> I would like to process the elements of an ordered collection in parallel.
After lots
> of exploring the image, trail, and error, the following example solution
emerged:
> ...[snip]...
> There are a couple of problems with this.  If the second delay line is
uncommented, the
> answer is incorrect.  Waiting there allows the block to rerun and tmpIndex
gets
> overwritten before it is used.  The second problem is that I can't get a
more general
> solution to work when it is added to OrderedCollection or some of its
superclasses so
> that I can do
>
> #(1 2 3) doInParallel: [ :each | each something ].
>
> Besides, if no delays are allowed in the block, this isn't a very handy
construction.
>
> Both of these problems may have to do with the not quite block closures
that Dolphin
> 5 is reported to have.  Does anyone know if that's the case?

Yes, you are certainly running into the limitations of Smalltalk-80 style
blocks there. Remember that all the temps in the blocks (including any block
arguments) are allocated in a shared slot in a context associated with the
method. Therefore if you create a sequence of blocks in a loop inside the
same method, then all will share the same temp slots. Consequently each of
the forked blocks is using the same slot for 'tmpIndex'.

A general pattern to overcome this issue is to extract the block creation
into a separate method invocation, i.e.
...
array do: [:index | (self innerBlock: index) fork]
...

You can't do this for a higher-order parallel enumerator, because you aren't
in control of the creation of the operation block. However a workaround is
to explicitly create a separate home context for each block, e.g. for your
original example:

    | counter array r |
    array := #(1 2 3 4 5) copy.
    counter := Semaphore new.
    r := Random new.
    array do:
      [:index |
      block :=
        [Processor sleep: r next * 1000.
        array at: index put: index + 1.
        counter signal].
      block outer: block outer copy.
      block fork].
    array size timesRepeat: [counter wait].
    ^array

If the block is passed in to a method, then you'll obviously have to copy
that too to create unique instances.

>...Also, will this work
> better in Dolphin 6: without the inner semaphore or with allowing delays
inside the
> block?

Yes, it will (does) work as expected in D6. You don't (in D6) need the
'tmpIndex' variable or 'innerSemaphore', since the 'index' argument is
copied into each of the forked blocks. Also the excessSignal count of a
Semaphore must be >= 0, not negative. Here's how I might write a general
parallel transform for D6:

collectInParallel: transformBlock
 | finished answer size |
 size := self size.
 answer := self copyLikeOfSize: size.
 finished := Semaphore new.
 self keysAndValuesDo:
   [:i :each |

   [answer at: i put: (transformBlock value: each).
   finished signal] fork].
 size timesRepeat: [finished wait].
 ^answer

>...If someone has a better solution, please let me know.  Thanks.

I've run out of time, so I must leave it as an exercise for the reader to
write a #collectInParallel: for D5.

Another way would might be to use DeferredValues.

Regards

Blair