OS Threads and external callbacks

Hi,

I suspect this is one for Blair, but insight from anyone would be welcome...

I'm seeing some behaviour that isn't consistent with what I *thought* was the
way that external callbacks work when they are issued from non-Dolphin threads.

What I thought was that when the VM found itself being invoked from a thread
that it didn't recognise, instead of directly handling the callback, it would
block the "foreign" thread while it passed the callback arguments across to the
Dolphin thread, and then wait until it had been handled there.  The Dolphin
thread (running all the Dolphin Processes) would handle the callback when it
realised it was pending.  Presumably it does some sort of polling when it
switches Processes, such as when an external call is made (or returns?), or a
method call is made?, or in some sort of idle processing.  The VM would then
co-opt whatever Process happened to be executing when it realised there was an
external callback to handle, and handle it from that Process.  Hence it is
arbitrary what Process will be co-opted, but the callback will always be
handled provided that *some* process is executing normally, or they are all
idle.

That's what I *thought* happened, but apparently not; at least it doesn't fit
with what I'm seeing.

I'm trying to write regression tests (I decline to call them unit tests,
because they're not, even though I'm using the SUnit framework) for some
external callback code.  The way I want the test to work is that the test
issues a call, say #start, then it sleeps for 5 seconds.  The implementation of
#start makes an external call to some code that starts a new OS thread and then
returns immediately.  The new thread issues a fixed set of 3 callbacks to
Dolphin at intervals of 1 second.  Each external callback is recorded as part
of the test object.  My unit^H^H^H^H regression test wakes up after 5 seconds
and checks that all the callbacks have happened.  The test methodology is
hardly ideal, but it'll do.

The problem is that if my test sleeps by doing:
    self start.
    (Delay forMilliseconds: 5000) wait.
    self checkResults.
then none of the callbacks happen until after the Delay has finished, they then
happen duly at 1 second intervals.  So it looks as if Dolphin doesn't handle
the callbacks if the main thread is in a Delay (which is only a wait on a
semaphore).

I had thought that maybe the sleep meant that all the threads were in a state
where they wouldn't "let" the VM poll for pending callbacks, so I tried
breaking up the delay with a loop:
    self start.
    50 timesRepeat: [(Delay forMilliseconds: 100) wait].
    self checkResults.
But that had no effect.

I also tried forking a background Process that would wake up periodically, with
the idea that it would be "available" for the VM to co-opt for the callback:
    self start.
    [50 timesRepeat: [(Delay forMilliseconds: 100) wait]] fork.
    (Delay forMilliseconds: 5000) wait.
    self checkResults.
But that had no effect either.

The only thing that does work is to use #forkMainIfMain
    Processor forkMainIfMain.
    self start.
    (Delay forMilliseconds: 5000) wait.
    self checkResults.
But I don't understand why that is necessary.  It suggests that the polling for
pending callbacks happens *only* on the "main" Process.  Is that the case ?
Why ?  And even if it is, why didn't breaking up the Delay
into lots of little ones allow the VM to see the callback ?

BTW, my stuff seems to works well enough in normal use (with no need for
#forkMainIfMain), and its only the artificial context of a test that shows a
problem.  I'm not bothered by the tests needing "odd" code, but I would like to
confirm or correct my understanding of how callbacks work across threads, and
how the VM realises that there's one pending.  My app's architecture depends on
it rather crucially, so the unexpected way Dolphin seems to be working is
worrying.

    -- chris

"Chris Uppal" <[hidden email]> wrote in message
news:3e5a08eb$1$9709$[hidden email]...
> Hi,
>
> I suspect this is one for Blair, but insight from anyone would be
welcome...
>
> I'm seeing some behaviour that isn't consistent with what I *thought* was
the
> way that external callbacks work when they are issued from non-Dolphin
threads.
>
> What I thought was that when the VM found itself being invoked from a
thread
> that it didn't recognise, instead of directly handling the callback, it
would
> block the "foreign" thread while it passed the callback arguments across
to the
> Dolphin thread, and then wait until it had been handled there.  The
Dolphin
> thread (running all the Dolphin Processes) would handle the callback when
it
> realised it was pending.  Presumably it does some sort of polling when it
> switches Processes, such as when an external call is made (or returns?),
or a
> method call is made?, or in some sort of idle processing.  The VM would
then
> co-opt whatever Process happened to be executing when it realised there
was an
> external callback to handle, and handle it from that Process.  Hence it is
> arbitrary what Process will be co-opted, but the callback will always be
> handled provided that *some* process is executing normally, or they are
all
> idle.

No, that's not quite right. There is no polling involved. The incoming
callbacks are forwarded to the main Dolphin thread via the message queue,
and this means (effectively) that they must usually be dispatched by the
main UI process, and so if that is blocked the calls will not get serviced.

>
> ...
> The problem is that if my test sleeps by doing:
>     self start.
>     (Delay forMilliseconds: 5000) wait.
>     self checkResults.
> then none of the callbacks happen until after the Delay has finished, they
then
> happen duly at 1 second intervals.  So it looks as if Dolphin doesn't
handle
> the callbacks if the main thread is in a Delay (which is only a wait on a
> semaphore).

That would be (as I'm sure you now realise) because the main UI process is
blocked.

>...

Regards

Blair

Blair

> No, that's not quite right. There is no polling involved. The incoming
> callbacks are forwarded to the main Dolphin thread via the message
> queue, and this means (effectively) that they must usually be
> dispatched by the main UI process, and so if that is blocked the
> calls will not get serviced.

Thank you for the explanation.

Now I shall have to ponder what it means for my architecture -- first guess is
that it's beneficial :-))

Just a quick further question: what does it mean for "console" deployment ?  I
seem to remember that Dolphin console apps don't have/service the Windows
message queue; does that mean that a console app can't handle off-thread
callbacks ?

    -- chris

"Chris Uppal" <[hidden email]> wrote in message
news:3e5b71fc$0$59862$[hidden email]... guess is
> that it's beneficial :-))
>
> Just a quick further question: what does it mean for "console" deployment
?  I
> seem to remember that Dolphin console apps don't have/service the Windows
> message queue; does that mean that a console app can't handle off-thread
> callbacks ?

They can if you fork off a separate process to run your mainline processing
(or start a new main UI process with #forkMainIfMain). This will relinquish
control of the main UI process, and allow it to service the message queue,
still available even in a console app. Of course you'll also need to take
steps to manage the lifetime of the application since it will normally
terminate when the ConsoleSessionManager>>main method returns, in other
words you have to override #keepAlive to do nothing, and then explicitly
terminate the session some other way.

Regards

Blair