Timed processes

I want to have a timed save on a document, say every 10 minutes.  As an
experiment I have modified Notepad with the addition of these two
methods:

Notepad>>saveDocument
        super saveDocument.
        self startTimer.

Notepad>>startTimer
        timedProcess isNil ifFalse: [timedProcess terminate].
        timedProcess :=
                        [(Delay forSeconds: 10) wait.
                        self saveDocument] forkAt: Processor userBackgroundPriority

timedProcess is an ivar in Notepad.

This works -- the first time that saveDocument is called -- but then I
would expect the timedProcess to kick in every 10seconds. But it
doesn't.   I read a similiar discussion (
http://groups.google.com.au/group/comp.lang.smalltalk.dolphin/browse_thread/thread/917d71c86fa5c87f)
but I still don't understand ...

Edward,

You are missing a step.  The block is forked and run as a separate
process.  When the delay finishes, and the saveDocument is performed,
the block ends and, as there is no more code to evaluate, the process
terminates.  You just need to tell process to keep repeating the
block. Try enclosing it in another block and using the #repeat
message...


Notepad>>startTimer
        timedProcess isNil ifFalse: [timedProcess terminate].
        timedProcess :=
                        [[(Delay forSeconds: 10) wait.
                        self saveDocument] repeat]  forkAt: Processor
userBackgroundPriority

.
You will also want to include a way of terminating the timedProcess
when the Notepad instance is closed.  You can put that in the
#onViewClosed method but I would also recommend having some sort of
finalization.

onViewOpened
        [blah]
        self beFinalizable

onViewClosed
        [blah]
        self endTimedProcess

finalize
        self endTimedProcess

endTimedProcess
        timedProcess
                ifNotNil: [
                        timedProcess terminate.
                        timedProcess := nil]
--
Ian

The From address is valid - for the moment

Ian, Edward,
>
> You are missing a step.  The block is forked and run as a separate
> process.  When the delay finishes, and the saveDocument is performed,
> the block ends and, as there is no more code to evaluate, the process
> terminates.  You just need to tell process to keep repeating the
> block. Try enclosing it in another block and using the #repeat
> message...

I also recommend having the background thread queue the actual save as a
deferred action, so it gets done at the next convenience of the GUI
rather than possibly at odds with it.  Toward that end, you might want
to read up on Steve's View Queue goodie, which allows deferred actions
to run even when Windows goes into extended modal states (menus popped
and not closed, etc.).

Have a good one,

Bill


--
Wilhelm K. Schwab, Ph.D.
[hidden email]

Ian thanks ...

Your suggestion has worked ...

But as my implementation of #saveDocument contained a call to
#startTimer I expected that a new timed process would be started in
recursive dance between the two methods.

I also found that forking the self saveDocument works as expected:

Notepad>>startTimer
  timedProcess isNil ifFalse: [timedProcess terminate].
        timedProcess :=
                        [(Delay forSeconds: 30) wait.
                        [self saveDocument] fork] forkAt: Processor userBackgroundPriority

Can anybody explain why this works as expected and my original
implementation does not?

Ian B wrote:

Edwards,

Maybe it's because your earlier implementation calls startTimer
recursively which will terminate timedProcess which is process of its
own. So it will not reach the next #fork stantence before its
termination.

But your latest implementation forks the recursive startTimer, which
process is a separate thread from the running timedProcess. Therefore
terminating timedProcess no longer mean terminating itself as it did in
the earlier one.



Periodic event is highly needed for many applications, but implementing
it correctly is not easy as it looks in the first sight. I hope some de
facto standard class for doing it. I've made my own Clock class with
some helps of Chris. The class is highly reused in my image.

If your model object is always expected to stick together with its View
than, simple, easy and safest way of creating a periodic timer event
would be to use #setTimer: of the View class

onCreated: anEvent

        self when: #timerTick: send: #onTick to: self.
        self setTimer: 12321 interval: 50.
        ^super onCreated: anEvent

The down side of this method is that priority of thread cannot be
modified as you want. (higher / lower)


Have a good one
Howard J Oh

Edward Stow wrote:

> Notepad>>startTimer
> timedProcess isNil ifFalse: [timedProcess terminate].
> timedProcess :=
> [(Delay forSeconds: 10) wait.
> self saveDocument] forkAt: Processor userBackgroundPriority

I think that Howard's explanation of why this fails is correct.  I just wanted
to add another warning to the collection you've already received ;-)

Suppose we re-write the code a little so that it works correctly, but still has
the call to:

    timedProcess isNil ifFalse: [timedProcess terminate].

which is invoked whenever the file is saved.  You are now in a dangerous
situation.  If the user happens to save the file explicitly just before the
timer is due to go off, then that code will save the file, then -- if you are
unlucky -- the delay will expire at just the wrong moment, and the background
Process will start to overwrite the file.  And then it will be #terminate-ed
while it is in the middle of doing so.  Not good.

I advise against /ever/ using #terminate to kill Processes unless:

    You are just playing around or debugging in a development image,
    and don't mind too much if things get screwed up.
or:
    You have been able to prove that a mistimed #terminate cannot leave the
    system (your code or Dolphin's) in an inconsistent state.
or:
    You are already in such a badly screwed-up state that a little more damage
    doesn't matter.

(BTW, most of the uses of #terminate in the image fall into one of those
categories, but there is at least one -- in BlockingCallMontitor -- which does
not.)

The technique I use is for the forked Process to check to see whether it is
still wanted before doing anything, and if not then it just quietly exists.  If
it is waiting on a Semaphore then I wake it up explicitly, but if not (i.e. it
is always actively running, or it alternates running and sleeping) then I just
set the flag to say "stop now" and leave it up to the Process to see the flag
before it next does any work.

In point of fact, my favourite way of expressing that is for the Process to
check that it is still the value of the backgroundProcess variable, and if not
then it assumes that either the program doesn't want a background process any
more, or that another Process has taken over the job.  In either case it just
silently dies.

As an example of how nasty #terminate is (unless it has guarantees built in
that I don't know about), the following pattern of code is used a lot in the
Dolphin image (and I use it a lot too).

    aResource := ... code to acquire some resource...
    [self doSomethingWith: aResource]
        ensure: [... code to release the resource...]

Even if the bits of code which acquire, use, and release the resource are
themselves #terminate-safe (which is unlikely), the pattern itself isn't
#terminate-safe since it's possible for the #terminate to happen after the
resource is acquired, but before the #ensure: block is entered.  It's possible
to reduce the size of the window when that can happen:

    [aResource := ... code to acquire some resource...
    self doSomethingWith: aResource]
        ensure: [aResource ifNotNil: [:it | ... code to release "it"...]]

But: (a) that's a pain in the arse, and (b) it still isn't safe -- it just
reduces the chance of problems.

    -- chris

Chris,

Thank you for this very useful message and advice!

> [aResource := ... code to acquire some resource...
> self doSomethingWith: aResource]
> ensure: [aResource ifNotNil: [:it | ... code to release "it"...]]
> But: (a) that's a pain in the arse, and (b) it still isn't safe --
> it just reduces the chance of problems.

One concrete example, I guess the most common, would be opening a file.
I have always worried about File>>open ... often wondering if I was
being too paranoid, but avoiding opening files in background processes.

I bookmarked a message by Blair
<http://groups.google.com.au/group/comp.lang.smalltalk.dolphin/msg/6bda0462d8402a6d>
 which says in part:

  In Dolphin a process switch can theoretically occur
  between bytecodes, however in practice interrupts are
  not recognised except on method activations and
  unconditional jumps (as part of the same mechanism
  used to poll the message queue). Any processor
  synchronisation primitive such as signalling a Semaphore
  could also cause a process switch.

Does this mean that File>>open could get interrupted (therefore
terminated) between the external CreateFile call and the assignment of
the temporary to handle? (I am not sure if an unconditional jump is
involved .... certainly it reads like the send of #beFinalizable could
be interrupted).

If it can be interrupted in practice, I think it would be an example of
what you point out as being not #terminate-safe no matter what you do
from the sending code. If you add into the above mix opening files with
limited FILE_SHARE flags, you have a nasty bug just waiting to happen.

Bottom line, Chris ... you have convinced me. I am adding a new rule to
my use of background processes ... terminate with extreme caution. Your
simple rule should eliminate a whole class of potential bugs, and stop
me having to think about whether methods like File>>open can get
interrupted  at a nasty spot .... Thanks!

Steve
 --
[hidden email]

Chris, Fantastic advice ...
Chris Uppal wrote:
.. many lines deleted ... Would this implementation be ok ...  I have also added a mutex on the
saveDocument method in Notepad class.

(Note this is not tested D6 code -- I only have Squeak at my current
desktop)

Notepad>>intialize
    super initialize.
    "Other init stuff ..."
    mutex := Semaphore forMutualExclusion.

Notepad>>saveDocument
    self stopTimer.
    mutex critical: [super saveDocument].
    self startTimer.

Notepad>>stopTimer
    timedProcess := nil.

Notepad>>startTimer
    timedProcess :=
        [(Delay forSeconds: 5*60) wait.
        (timedProcess = Processor activeProcess)
            ifTrue:
               [[self saveDocument] fork]]
                   forkAt: Processor userBackgroundPriority

I have also trialled Howard Oh's suggestion of #setTimer:interval:

Thanks to the excellent discussion.

Edward,

As an aside, a Squeak semaphore set for mutual exclusion is not the same
as a mutex, or it wasn't when I tested it.  The extra tests that protect
a thread from deadlocking itself are well worth the effort.  In general,
it is better to overprotect, at least until it starts showing up in
profiles.


> Notepad>>saveDocument
>     self stopTimer.
>     mutex critical: [super saveDocument].
>     self startTimer.

With proper protection, you should not need to stop/start the background
thread.  At most, you might want to use a wait/signal trick that Blair
described here some time ago.  The idea is to have the background thread
send #wait and then #signal to a semaphore.  Send #signal to said
semaphore to release the thread, and send #wait to the semaphore to
block it.  Done correctly, there is no concern about blocking yourself
when trying to block the background thread; the semaphore spends most of
its time with one excess signal, and you will simply be consuming it.

When interacting with a model of a GUI, I would post the save as a
deferred action.  Your background thread then acts much as the user
would if only they remembered to hit the save button every so often.
But see Steve's ViewQueue; don't learn the hard way about unpopped menus.

I also think it is a good idea to make a copy (under a different name)
of the previous backup, and then save.  Otherwise, one risks
crashing/hanging in the middle of the save, corrupting the only good
copy.  However you do it, you should have at least two representations
of the user's work, as it is "impossible" to crash while writing both.
Note that you could still get into trouble with a write-back cache though.


> Notepad>>stopTimer
>     timedProcess := nil.

With respect to Chris' [*] arguments against it, I would explicitly
terminate the thread.  I do most cleanup in an #ensure: block that is
part of any such thread, so it runs whether the tread quits on its own
or is clobbered.  With some critical sections, I have had good results
for quite a few machine-years.  I do not suggest that it is "easy" but
it is possible, and I think threads are well worth the effort.

[*] Any time I disagree with Chris, I spend some time wondering whether
I am in the wrong.  That will happen here too.

Have a good one,

Bill


--
Wilhelm K. Schwab, Ph.D.
[hidden email]

Edward,

> Notepad>>intialize
>     super initialize.
>     "Other init stuff ..."
>     mutex := Semaphore forMutualExclusion.

As Bill has said, a instance of Mutex would be preferable here (the equivalent
of VW's RecursionLocks, I'm not sure whether there is an equivalent in Squeak
at all).  But see below:


> Notepad>>startTimer
>     timedProcess :=
>         [(Delay forSeconds: 5*60) wait.
>         (timedProcess = Processor activeProcess)
>             ifTrue:
>                [[self saveDocument] fork]]
>                    forkAt: Processor userBackgroundPriority

As something of an aside (since it won't affect you) there's a theoretical
race-condition here.  If the background process starts up fast enough, and if
its main loop is such that it tests to see if it should die more-or-less
immediately (which your main loop doesn't, unless something goes wrong with
Delay), then it could in principle check the "timedProcess" variable before the
original Process has had a chance to assign it.  For that reason, I usually
start my background Processes in a suspended state (BlockClosure>>newProcess),
and only later send #resume to it to start it running.  (Which may be pedantic,
but I prefer to be correct when it doesn't cost me too much -- especially when
multi-threading is involved.)

Another aside: both you and I are assuming that reading and writing an instvar
(or other variable) is an atomic operation.  In principle, I suppose, we should
use a Mutex to protect access to that variable -- but that's /too/ pedantic,
even for me[*] (too much effort for zero actual gain...)

As Bill has said (twice now ;-) you would be better off using a deferred action
to cause the actual save.  Window's tends not to like "things" happening to its
display off the main (UI) thread (such as changing the caption to remove a
"changed-but-unsaved" marker).  Also users tend not to like it if they are
typing into the document while it is saved ;-)   To defer the action (which
means arrange for it to run on the UI thread) replace:

    [self saveDocument] fork

by:

    [self saveDocument] postToInputQueue.

One extra benefit of that is that since saves will now always happen on the UI
thread (whether explicit or implicit) you don't need the Mutex (or Semaphore).


> I have also trialled Howard Oh's suggestion of #setTimer:interval:

That's also a good way of doing this kind of thing if it fits in with your
overall architecture (sometimes it does, sometimes it doesn't).  It has the
same benefit of ensuring that all the action happens on the UI thread.

    -- chris

([*] In Java, on the other hand, I /would/ use that kind of protection -- but
then Java has real multiprocessing, and the protection is actually necessary
there...)

Steve,

> Does this mean that File>>open could get interrupted (therefore
> terminated) between the external CreateFile call and the assignment of
> the temporary to handle? (I am not sure if an unconditional jump is
> involved .... certainly it reads like the send of #beFinalizable could
> be interrupted).

I'm not sure either.  My /guess/ is that the current implementation can
actually be interrupted there, but really I don't care too much (for these
purposes, I am interested for other reasons) since I'd rather not depend on
subtle interactions between the VM implementation and the concrete
implementation in File.  I hope that doesn't sound priggish...

Thinking about it, I doubt if my final example does have an open window where
the VM will actually interupt between ...allocate resource... and the
assignment to the variable -- but (by the above logic) I'd rather not rely on
it unless I have to.

    -- chris

Bill,

> With respect to Chris' [*] arguments against it, I would explicitly
> terminate the thread.  I do most cleanup in an #ensure: block that is
> part of any such thread, so it runs whether the tread quits on its own
> or is clobbered.

Well, here's a (good natured) challenge for you -- which was actually inspired
by a thread of your own from a year or two back about how to block a Process
until several conditions all became true at the same time.

Given that we have instvars:

    semaphore := semaphore new.
    symbols := IdentitySet new.
    mutex := Mutex new.

We want to add symbols to the Set, and "signals" to the semaphore, such that
there is exactly one signal for each symbol.  We are (by hypothesis) in a
multi-threaded situation, so we obviously have to use the Mutex:

add: aSymbol
    mutex critical:
        [(symbols includes: aSymbol) ifFalse:
            [symbols add: aSymbol.
            semaphore signal]].

In point of fact, the test isn't important to the challenge, so we'll simplify
it:

add: aSymbol
    mutex critical:
        [symbols add: aSymbol.
         semaphore signal].

I cannot think of any way to make that #terminate-safe (unless, perhaps, by
using private API's and undocumented knowledge/guesses/assumptions about the
behaviour of the VM.  Can you ?

Or anyone ?

    -- chris


P.S.  I count BlockClosure>>critical: as cheating ;-)  But, OTOH, I won't
insist that you have to make Set>>add: and Semaphore>>signal #terminate-safe
themselves...

> But see Steve's ViewQueue; don't learn the hard way about unpopped menus.

I am unable to find it.
It seems that
http://www.stevewaring.net/blog/articles/viewQueue/SWViewQueue_5.0.0.zip
no longer works.

Could someone post it or a link?

Thanks,
Dan

Dan,

>> But see Steve's ViewQueue; don't learn the hard way about unpopped menus.
>
> I am unable to find it.
> It seems that
> http://www.stevewaring.net/blog/articles/viewQueue/SWViewQueue_5.0.0.zip
> no longer works.
>
> Could someone post it or a link?

I certainly can with Steve's permission, but better to get it from him
in case he has made improvements.

You will not regret using it.  I had an app ultimately freeze (!!!)
because somebody popped a menu and walked away :(

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]

Bill,

> You will not regret using it.  I had an app ultimately freeze (!!!)
> because somebody popped a menu and walked away :(

/My/ users have more sense...

;-)


But seriously, I don't think that's an issue for the more commons kinds of
application[*] -- the ones where the apps job is /just/ to provide a service to
the user, rather than being some sort of monitoring or control interface to a
process which should continue whether the user is watching or not.  Or have I
misunderstood the issue (I remember thinking "I don't understand this issue"
when the topic first cropped up ;-)

("more common" for a mostly desktop-oriented programming product like Dolphin)

    -- chris

Chris,

>> With respect to Chris' [*] arguments against it, I would explicitly
>> terminate the thread.  I do most cleanup in an #ensure: block that is
>> part of any such thread, so it runs whether the tread quits on its own
>> or is clobbered.
>
> Well, here's a (good natured) challenge for you
                   ^^^^^^^^^^^^
Of course.  I will have to read over all of this and give it some real
thought.  My stuff isn't crashing, so the priority won't be terribly
high, but it _will_ happen.

When I posed the multi-object wait question, I expected something along
the lines of a thread that loops to wait on semaphores from a shared
queue, but I either had not reached that point, or saw the hole in it
that I am now missing =:0


> P.S.  I count BlockClosure>>critical: as cheating ;-)  

I tend to agree.  Mutex>>critical: is of course fair game.


 > But, OTOH, I won't
> insist that you have to make Set>>add: and Semaphore>>signal #terminate-safe
> themselves...

This might be part of our potential disagreement: I tend to assume that
the data structures belong to the thread, and if they are corrupted,
that's ok, because I didn't want them anymore anyway.  I have learned
the hard way to never open a file stream without an #ensure: block to
close it, etc.

Have a good one,

Bill


--
Wilhelm K. Schwab, Ph.D.
[hidden email]

Chris,

>> You will not regret using it.  I had an app ultimately freeze (!!!)
>> because somebody popped a menu and walked away :(
>
> /My/ users have more sense...
>
> ;-)

Hmmmmm.  If you pay up regularly, I won't tell him you said that :)


> But seriously, I don't think that's an issue for the more commons kinds of
> application[*] -- the ones where the apps job is /just/ to provide a service to
> the user, rather than being some sort of monitoring or control interface to a
> process which should continue whether the user is watching or not.  Or have I
> misunderstood the issue (I remember thinking "I don't understand this issue"
> when the topic first cropped up ;-)
>
> ("more common" for a mostly desktop-oriented programming product like Dolphin)

While I will admit that medical education is almost a requirement for
being inclined to open a menu and then ignore the machine for a while,
but the fact is that robust software should be able to handle it.  Put
another way: the defect is very real, and might or might not hurt your
users without Steve's workaround.

As an extreme example, one project that predates me employed a teenager
and a video taping system to test their release candidate.  The idea was
that the kid would eventually clobber it, and then they could trace his
steps.  They didn't need the camera - he killed it in less than five
seconds by simply clicking in a strange spot.  Further mayhem ensued, of
course.

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]

Bill,

> When I posed the multi-object wait question, I expected something along
> the lines of a thread that loops to wait on semaphores from a shared
> queue, but I either had not reached that point, or saw the hole in it
> that I am now missing =:0

If I remember correctly, your solution doesn't suffer from this problem, at
least not at this level.  That's to say: I couldn't make my approach
#terminate-safe /even/ if the components it was built from were OK themselves.
Your approach (iirc) was safe in that sense.

Personally, I stick with the idea that the problem is not my implementation of
the challenge problem, but that #terminate should never be used (in production
code) on a Process which is not in an exactly known, safe, state. (E.g.
Processor activeProcess terminate would normally be OK).


> This might be part of our potential disagreement: I tend to assume that
> the data structures belong to the thread, and if they are corrupted,
> that's ok, because I didn't want them anymore anyway.

Well, yes. #terminate won't give problems to structures that /are/ local in
that
sense.  But thing like external files (or other externally visible resources),
and locks, queues, and other IPC primitives, are not usually local to the domed
Process.

    -- chris

Bill,

> While I will admit that medical education is almost a requirement for
> being inclined to open a menu and then ignore the machine for a while,

<grin/>

> but the fact is that robust software should be able to handle it.  Put
> another way: the defect is very real, and might or might not hurt your
> users without Steve's workaround.

Yes, I suppose this is the same kind of logic as I am using about
thread-safety.  Better to start in a position where you know it'll work than
have to work to discover whether the "might or might not" turns out to be "not"
in each specific application.

I shall have to revisit this.  Thanks.


> As an extreme example, one project that predates me employed a teenager
> and a video taping system to test their release candidate.  The idea was
> that the kid would eventually clobber it, and then they could trace his
> steps.  They didn't need the camera - he killed it in less than five
> seconds by simply clicking in a strange spot.

/Lovely/ story!

    -- chris

Hi Dan,

The D6 version is on the front page of <http://www.dolphinharbor.org>

The direct link is:

<http://www.dolphinharbor.org/dh/projects/goodies/SWViewActionQueue.zip>

Thanks,
Steve
 --
[hidden email]