Generator (resumable) methods

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Generator (resumable) methods

Steffen Märcker
Hi,

I wonder if there is a simple way to write generator methods known from  
python. A generator method does a (possible) computation and  
returns/yields intermediate results. If the method is called again, it  
resumes at the last return and continues until the next return/yield. For  
example (made up):

Integer>>naturals
   | n |
   n := 0.
   [n := n+1.
    self yield: n "return n and resume here on next invocation"] repeat.

Best regards,
Steffen
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Generator (resumable) methods

Andres Valloud-4
Raise a resumable exception such as a notification?  I did something
like that for prime factoring and it worked out ok... in that case, the
notification carries whatever new factor was found.

On 6/18/14 1:49 , Steffen Märcker wrote:

> Hi,
>
> I wonder if there is a simple way to write generator methods known from
> python. A generator method does a (possible) computation and
> returns/yields intermediate results. If the method is called again, it
> resumes at the last return and continues until the next return/yield. For
> example (made up):
>
> Integer>>naturals
>     | n |
>     n := 0.
>     [n := n+1.
>      self yield: n "return n and resume here on next invocation"] repeat.
>
> Best regards,
> Steffen
> _______________________________________________
> vwnc mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
>
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Generator (resumable) methods

Mark Plas
Something like this?

| n b |
n := 0.
b := [n := n + 1].
b value.
b value

Each time you send #value to b you will get the next value...

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Andres Valloud
Sent: woensdag 18 juni 2014 11:34
To: Visualworks Mailing List
Subject: Re: [vwnc] Generator (resumable) methods

Raise a resumable exception such as a notification?  I did something like that for prime factoring and it worked out ok... in that case, the notification carries whatever new factor was found.

On 6/18/14 1:49 , Steffen Märcker wrote:

> Hi,
>
> I wonder if there is a simple way to write generator methods known
> from python. A generator method does a (possible) computation and
> returns/yields intermediate results. If the method is called again, it
> resumes at the last return and continues until the next return/yield.
> For example (made up):
>
> Integer>>naturals
>     | n |
>     n := 0.
>     [n := n+1.
>      self yield: n "return n and resume here on next invocation"] repeat.
>
> Best regards,
> Steffen
> _______________________________________________
> vwnc mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
>
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Generator (resumable) methods

Michael Lucas-Smith-2
In reply to this post by Steffen Märcker
The cool thing about the python generators is you can do something like this:

n := 0.
[self yield: n.
 n := n + 1.
 self yield: n.
 n := n + 2.
 self yield n] repeat.

So to simulate that we’d need to actually do a yield, but also provide a ‘current value’. We could either run it with two processes or we could use a closure as callback. First the simpler version:

generator := [:yield |
    n := 0.
    [yield value: n.
    n := n + 1.
    yield value: n.
    n := n + 2.
    yield value: n] repeat
].
generator value: [:n | Transcript cr; print: n]

Of course this version takes control away from you and gives control to the generator, so the more ‘true to its nature’ version would use a second process which would actually yield in the process sense, rather than just as a callback:

n := nil.
ready := Semaphore new.
generator := [
   n := 0.
   [ready signal.  generator yield.
   n := n + 1.
   ready signal.   generator yield.
   n := n + 2.
   ready signal.   generator yield]
      repeat] newProcess.

100 timesRepeat: [
   generator resume.
   ready wait.
   Transcript cr; print: n].
generator terminate.

In this version we have control over the generator. We can pull from it as much as we want and we can terminate it when we’re done.
This is pretty verbose though. I bet we could make a simpler version with a bit of API design.

Generator (process value lock)
Generator class>>on: aBlockClosure
   ^self new initialise: aBlockClsoure

Generator>>initialize: aBlockClosure
   lock := Semaphore new.
   process := [[[aBlockClosure value: self] repeat] ensure: [process := nil. lock signal]] newProcess.

Generator>>yield: aValue
   value := aValue.
   lock signal.
   process yield.

Generator>>close
   process terminate.

Generator>>next
   process == nil ifTrue: [self error: ‘Generator has concluded its funstuffs’].
   process resume.
   lock wait.
   process == nil ifTrue: [self error: ‘Generator was terminated’].
   ^value

BlockClosure>>newGenerator
   ^Generator on: self

generator := [:control |
   n := 0.
   [control yield: n.
   n := n + 1.
   control yield: n.
   n := n + 2.
   control yield: n]
     repeat] newGenerator.

100 timesRepeat: [
   Transcript cr; print: generator next].
generator close.

Now it looks a lot like a stream and it only generates what you need.


Have fun!,
Michael


On 18 Jun 2014, at 6:49 pm, Steffen Märcker <[hidden email]> wrote:

> Hi,
>
> I wonder if there is a simple way to write generator methods known from python. A generator method does a (possible) computation and returns/yields intermediate results. If the method is called again, it resumes at the last return and continues until the next return/yield. For example (made up):
>
> Integer>>naturals
>  | n |
>  n := 0.
>  [n := n+1.
>   self yield: n "return n and resume here on next invocation"] repeat.
>
> Best regards,
> Steffen
> _______________________________________________
> vwnc mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Generator (resumable) methods

Steffen Märcker
In reply to this post by Andres Valloud-4
Hi Andres,

I've tried to do come up with something using stored resumable exceptions.  
But this failed due to non-local returns. However, I'd be whether your  
approach applies nevertheless. The goal might be to build a stream-like  
object from a computation. For example:

Generator>>next

     someObject do: [:each | self yield: each].

gen := Generator on: hugeCollection.
10 timesRepeat: [
     Transcript
     show: gen next printString;
     cr].

Btw, Micheal L.-S. just posted another nice solution using threads.

Am .06.2014, 11:34 Uhr, schrieb Andres Valloud  
<[hidden email]>:

> Raise a resumable exception such as a notification?  I did something  
> like that for prime factoring and it worked out ok... in that case, the  
> notification carries whatever new factor was found.
>
> On 6/18/14 1:49 , Steffen Märcker wrote:
>> Hi,
>>
>> I wonder if there is a simple way to write generator methods known from
>> python. A generator method does a (possible) computation and
>> returns/yields intermediate results. If the method is called again, it
>> resumes at the last return and continues until the next return/yield.  
>> For
>> example (made up):
>>
>> Integer>>naturals
>>     | n |
>>     n := 0.
>>     [n := n+1.
>>      self yield: n "return n and resume here on next invocation"]  
>> repeat.
>>
>> Best regards,
>> Steffen
>> _______________________________________________
>> vwnc mailing list
>> [hidden email]
>> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
>>
> _______________________________________________
> vwnc mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Generator (resumable) methods

Steffen Märcker
In reply to this post by Michael Lucas-Smith-2
Hi Michael,

using processes is a nice solution! =) However, I suspect the context  
switches are expensive. Compare the times in this toy example:

Time millisecondsToRun: [ | generator |
     generator := Generator on:
             [:control | | random |
             random := Random new.
             [control yield: random next] repeat].
     (10**6) timesRepeat: [generator next].
     generator close]. "~500ms"

Time millisecondsToRun: [ | random |
     random := Random new.
     (10**6) timesRepeat: [random next]]. "~60ms"

The first code runs about a magnitude slower than the second. I wonder  
whether we can do something about this penalty or not?

Btw, I think it has to be:

> Generator>>yield: aValue
>    value := aValue.
>    lock signal.
>    process suspend. "instead of yield"

Right?

Cheers,
Steffen


Am .06.2014, 13:35 Uhr, schrieb Michael Lucas-Smith  
<[hidden email]>:

> The cool thing about the python generators is you can do something like  
> this:
>
> n := 0.
> [self yield: n.
>  n := n + 1.
>  self yield: n.
>  n := n + 2.
>  self yield n] repeat.
>
> So to simulate that we’d need to actually do a yield, but also provide a  
> ‘current value’. We could either run it with two processes or we could  
> use a closure as callback. First the simpler version:
>
> generator := [:yield |
>     n := 0.
>     [yield value: n.
>     n := n + 1.
>     yield value: n.
>     n := n + 2.
>     yield value: n] repeat
> ].
> generator value: [:n | Transcript cr; print: n]
>
> Of course this version takes control away from you and gives control to  
> the generator, so the more ‘true to its nature’ version would use a  
> second process which would actually yield in the process sense, rather  
> than just as a callback:
>
> n := nil.
> ready := Semaphore new.
> generator := [
>    n := 0.
>    [ready signal.  generator yield.
>    n := n + 1.
>    ready signal.   generator yield.
>    n := n + 2.
>    ready signal.   generator yield]
>       repeat] newProcess.
>
> 100 timesRepeat: [
>    generator resume.
>    ready wait.
>    Transcript cr; print: n].
> generator terminate.
>
> In this version we have control over the generator. We can pull from it  
> as much as we want and we can terminate it when we’re done.
> This is pretty verbose though. I bet we could make a simpler version  
> with a bit of API design.
>
> Generator (process value lock)
> Generator class>>on: aBlockClosure
>    ^self new initialise: aBlockClsoure
>
> Generator>>initialize: aBlockClosure
>    lock := Semaphore new.
>    process := [[[aBlockClosure value: self] repeat] ensure: [process :=  
> nil. lock signal]] newProcess.
>
> Generator>>yield: aValue
>    value := aValue.
>    lock signal.
>    process yield.
>
> Generator>>close
>    process terminate.
>
> Generator>>next
>    process == nil ifTrue: [self error: ‘Generator has concluded its  
> funstuffs’].
>    process resume.
>    lock wait.
>    process == nil ifTrue: [self error: ‘Generator was terminated’].
>    ^value
>
> BlockClosure>>newGenerator
>    ^Generator on: self
>
> generator := [:control |
>    n := 0.
>    [control yield: n.
>    n := n + 1.
>    control yield: n.
>    n := n + 2.
>    control yield: n]
>      repeat] newGenerator.
>
> 100 timesRepeat: [
>    Transcript cr; print: generator next].
> generator close.
>
> Now it looks a lot like a stream and it only generates what you need.
>
>
> Have fun!,
> Michael
>
>
> On 18 Jun 2014, at 6:49 pm, Steffen Märcker <[hidden email]> wrote:
>
>> Hi,
>>
>> I wonder if there is a simple way to write generator methods known from  
>> python. A generator method does a (possible) computation and  
>> returns/yields intermediate results. If the method is called again, it  
>> resumes at the last return and continues until the next return/yield.  
>> For example (made up):
>>
>> Integer>>naturals
>>  | n |
>>  n := 0.
>>  [n := n+1.
>>   self yield: n "return n and resume here on next invocation"] repeat.
>>
>> Best regards,
>> Steffen
>> _______________________________________________
>> vwnc mailing list
>> [hidden email]
>> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: Generator (resumable) methods

Andres Valloud-4
In reply to this post by Steffen Märcker
I had something along the lines of

PrimeFactorFinder>>nextFactor

   [
     "do work, update internal state along the way"
     nextFactorFound
       ifTrue: [NewPrimeFactor raiseRequestWith: nextFactor]
   ] whileThereIsWorkToDo


There is a hierarchy of such finders, those are pluggable with something
else that manages factoring.  So...

IntegerFactorizer>>factorize

   "blah blah choose finders for the integer to factor, blah blah"
   [finder doMoreWork]
     on: NewPrimeFactor
     do:
       [:ex |
         self updateFactorizationWith: ex foundFactor.
         ex return
       ].
   self thereIsStillWorkToDo ifTrue: [self goBackAndFactorizeSomeMore]


I'm writing off the top of my head, I'm sure you get the idea of what's
going on.

On 6/18/14 6:50 , Steffen Märcker wrote:

> Hi Andres,
>
> I've tried to do come up with something using stored resumable exceptions.
> But this failed due to non-local returns. However, I'd be whether your
> approach applies nevertheless. The goal might be to build a stream-like
> object from a computation. For example:
>
> Generator>>next
>
>       someObject do: [:each | self yield: each].
>
> gen := Generator on: hugeCollection.
> 10 timesRepeat: [
>       Transcript
>       show: gen next printString;
>       cr].
>
> Btw, Micheal L.-S. just posted another nice solution using threads.
>
> Am .06.2014, 11:34 Uhr, schrieb Andres Valloud
> <[hidden email]>:
>
>> Raise a resumable exception such as a notification?  I did something
>> like that for prime factoring and it worked out ok... in that case, the
>> notification carries whatever new factor was found.
>>
>> On 6/18/14 1:49 , Steffen Märcker wrote:
>>> Hi,
>>>
>>> I wonder if there is a simple way to write generator methods known from
>>> python. A generator method does a (possible) computation and
>>> returns/yields intermediate results. If the method is called again, it
>>> resumes at the last return and continues until the next return/yield.
>>> For
>>> example (made up):
>>>
>>> Integer>>naturals
>>>      | n |
>>>      n := 0.
>>>      [n := n+1.
>>>       self yield: n "return n and resume here on next invocation"]
>>> repeat.
>>>
>>> Best regards,
>>> Steffen
>>> _______________________________________________
>>> vwnc mailing list
>>> [hidden email]
>>> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
>>>
>> _______________________________________________
>> vwnc mailing list
>> [hidden email]
>> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
>
>
> _______________________________________________
> vwnc mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
> .
>
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc