overlap and virtual

I am calling a free threaded AX library to control a machine. The call is
<virtual stdcall: hresult 30>

However, the call returns in 10 mins so I would like to operlap it.

The problem is that I'am not able to compile the next code <overlap
virtual stdcall: hresult 30>

"umur" <[hidden email]> wrote in message
news:[hidden email]...
>I am calling a free threaded AX library to control a machine. The call is
> <virtual stdcall: hresult 30>
>
> However, the call returns in 10 mins so I would like to operlap it.
>
> The problem is that I'am not able to compile the next code <overlap
> virtual stdcall: hresult 30>
>
>

Virtual calls cannot be overlapped in Dolphin 5.1, sorry. They are not
supported by the compiler and, more importantly, there is no VM primitive
for performing asynchronous virtual calls.
The reasons for this are:
1) Dolphin 5.1 allocates threads for overlapped calls from a pool to which
the thread is returned as soon as the call finishes*. This means that it is
far from guaranteed that successive calls will occur on the same thread.
Most COM objects have thread affinity, and should not be called from an
unmarshalled (raw) interface pointer from other than the thread which
created them. Of course this does not apply to free-threaded objects, but it
would result in an unstable system for most experimenting with overlapping
COM calls.
2) Any interface pointers passed out from Dolphin via an overlapped call
would be unmarshalled, and therefore invalid for calling from other threads
unless they originated from free-threaded COM objects themselves. This means
that any COM interfaces implemented by Dolphin itself could not be passed
out via an overlapped COM call (since Dolphin itself only implements the
apartment threaded model) unless they were explicity marshalled into that
thread. Since you don't know which thread you are going to get, you can't do
this marshalling. In any case it is quite complex to get right.

It is possible (but unsupported) to perform virtual calls using the normal
asynchronous call primitive by defining a non-virtual call taking an extra
first parameter (the 'this' pointer, or the COM interface pointer itself in
the case of a COM interface), and by patching up the "proc address" (which
is stored in the first 32-bits of the external call descriptor) to point at
the address of the virtual function. The virtual function address it up in
the virtual function table.You need to know a little low-level COM/C++ stuff
to work this out, but it is documented in the COM specs and in books such as
the venerable "Inside OLE2" by Brockschmidt (sp?). You'll almost certainly
find it easier to build an external DLL in C and deal with the threading in
that.  Actually if the COM component doesn't return from the call in 10
minutes, it sounds to me like a candidate for a service running in a
separate process that you communicate with over sockets. You could do this
directly, or put the component behind a web server and use Steve Waring's
HTTP client to communicate with it. He has also done SOAP stuff if you want
to build it into a proper web service. See http://www.dolphinharbor.org

Regards

Blair
--------------
*This will be different in Dolphin 6, which permanently associates an
overlapped call thread with a Smalltalk Process the first time it performs
any overlapped call.

Blair,

> Virtual calls cannot be overlapped in Dolphin 5.1, sorry. They are not
> supported by the compiler and, more importantly, there is no VM primitive
> for performing asynchronous virtual calls.
> The reasons for this are: [snipped]

I only understood a little of your explanation, but it did seem to me that you
are equating C++ virtual functions with COM.  There *are* DLLs that use C++
semantics independently of COM.  Admittedly the vtable layout (or even the
existence of a vtable) is implementation dependent, but on the Windows platform
it's not unreasonable to specialise on the MSVC layout (as Dolphin does).

What I'm getting at is that while the ability to call virtual functions is
necessary to support COM, that's not /all/ that the feature is useful for.

The example that's closest to my heart is the interface with Java's JNI in my
JNIPort.  JNIPort is too general to be able to make use of overlapped external
calls (too much overhead in the common case where overlapping isn't required),
but JNI itself is an example of:

    -- C++ virtual call semantics.
    -- nothing to do with COM.
    -- calls can take arbitrarily long to complete.
    -- no other interfaces are available

In such circumstances it seems (to me) /obvious/ that it should be possible to
overlap calls to virtual functions.

> *This will be different in Dolphin 6, which permanently associates an
> overlapped call thread with a Smalltalk Process the first time it performs
> any overlapped call.

Sounds good.  Will that apply to all external calls, or only to those that
aren't made from the "main Process" ?

    -- chris

"Chris Uppal" <[hidden email]> wrote in message
news:[hidden email]...
Sorry if I gave that impression. Originally the virtual call capability was
designed to interface to C++ objects - the dominant language at the time if
one ignores VB (and we certainly tried to:-)). If you look at the relevant
external interfacing documentation, this is apparent. However there are a
couple of issues that made it rather less useful than we originally thought:
1) Many C++ member functions (probably most) are not virtual. In order to
call these externally, they have to be exported from the DLL and by default
they use a different calling convention that we didn't support (and didn't
think worth adding in the end).
2) We don't support the full range of virtual function table layouts needed
to support C++'s complex multiple inheritance model.
3) Defining classes to represent external C++ classes by hand is too time
consuming and too error prone, and parsing C++ to do the job automatically
is just too hard. This is not to say that constructing the type library
analyser was not hard, but it is at least a manageable problem.

Yes, perhaps, and it wouldn't have been too difficult to put in if you'd
asked before :-).

>> *This will be different in Dolphin 6, which permanently associates an
>> overlapped call thread with a Smalltalk Process the first time it
>> performs
>> any overlapped call.
>
> Sounds good.  Will that apply to all external calls, or only to those that
> aren't made from the "main Process" ?

I'm not sure quite what you mean. It certainly won't be the case that all
external calls will be made through the overlapped thread. That would be too
expensive, and would lead to deadlocks when the VM had to map callbacks to
the main VM thread if the callout itself was made from the main UI process.
On the other hand all overlapped calls from the same process will use the
same thread, so the same applies to the main UI process. The difference here
of course is that the lifetime of the main UI process can be somewhat short,
since actions such as popping dialogs require that a new main thread be
started. Its possible that we might do something to reassign the thread over
to the main UI process, but I'm not sure if this can be done safely at
present. If not then obviously the change will not help much towards the use
of COM (or C++) objects with thread affinity from the main UI process.

Regards

Blair

Blair,

> > > *This will be different in Dolphin 6, which permanently associates an
> > > overlapped call thread with a Smalltalk Process the first time it
> > > performs
> > > any overlapped call.
> >
> > Sounds good.  Will that apply to all external calls, or only to those
> > that aren't made from the "main Process" ?
>
> I'm not sure quite what you mean.

To be honest, /I'm/ not sure what I meant either -- the question that I
/wanted/ to ask is whether the new feature will help any with a generic problem
I have with the native interfacing.

The problem is that the choice between overlapped and direct calls happens at
the very bottom of the software stack, but that -- almost by definition -- has
the least information about whether direct or overlapped is the appropriate
choice.

For example, in printing, the GDILibrary>>startDoc:lpdi: method is not
overlapped, but if I want to print in the background (as I do) then either I
have to change the base system methods to make them overlapped, or duplicate
several layers of printing software (in this particular case, there aren't
/that/ many layers, so that's what I've done).

I don't have a solution to the problem.  But if the system were able to ignore
the overlapped flag for calls issued from the main Process, then we/you could
be a great deal more aggressive about making external calls overlapped since
the extra cost of an overlapped call would only be paid in the case when the
call was issued in the background (where not blocking the whole system is
presumably more important than outright speed).  I realise that that would
cause problems in the case where the main thread performed a slow external
operation at the same time as some background Process was trying to do some
useful work, but given that well-designed apps will prefer to issue such slow
calls from background Processes anyway, it might be that the gain is greater
than the loss.

If you squint a little, then you may be able to see some slight connection
between that and my original question...

    -- chris

"Chris Uppal" <[hidden email]> wrote in message
news:[hidden email]...
I know what you mean, and I have pondered this problem on occassion before,
but generally come to the conclusion that the complexity and potentially
greater instability outweighed the benefit. As it currently stands the VM is
not directly aware of any process having special status, and I don't think
it should be. I think one would still want to screen external calls and make
a decision as to whether they were overlappable. If the VM was main UI
process aware (or if the processes themselves were appropriately marked) it
could make use of an 'overlappable' attribute that was a hint rather than a
directive. On the other hand one could draw a tighter relationship between
Dolphin's green threads and the system threads by overlapping all calls on
background threads, disallowing overlapped calls on the main UI process,
effectively associating the main interpreter thread with the main UI
process. Nothing seems entirely satisfactory, and one begins to wonder if
this isn't taking a relatively simple idea to allow the use of blocking
calls from green threads in most cases, beyond its design limit.

One approach I thought of (which is by no means perfect, but which at least
remains with in the image and is therefore workable now) is to actually use
separate ExternalLibrary classes (and singleton instances), selecting
between them in the #default class method based on whatever criteria you
feel is appropriate. You can then add a subclass which overrides certain
external methods just to overlap them. You might not want to pay the cost of
slowing down every #default access, so another method could be defined on
the class.

Regards

Blair

Blair,

> You can then add a subclass which overrides certain
> external methods just to overlap them. You might not want to pay the cost of
> slowing down every #default access, so another method could be defined on
> the class.

If I'm following you, I think that another method would be preferred -
that way one could as for

    UserLibrary dareDevil someRiskyThingOrAnother.

making the bleeding edge calls easier to find.  Reasonable??

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]

Blair,

> I know what you mean, and I have pondered this problem on occassion
> before, but generally come to the conclusion that the complexity and
> potentially greater instability outweighed the benefit.
[snip].
> Nothing seems entirely
> satisfactory, and one begins to wonder if this isn't taking a relatively
> simple idea to allow the use of blocking calls from green threads in most
> cases, beyond its design limit.

Indeed.  In an ideal world Dolphin would be able to talk the lingua franca of
multi-processing by working directly with OS-level threads.  But I can see that
that would be -- to put it mildly -- difficult.

Here's an approach that you have probably already thought of and dismissed,
but...

If you turned the problem the other way around then some of these difficulties
would go away.

Create an OS thread for each Smalltalk Process (so that all the processing done
by that Process was happening on the thread).  Protect the "interpreter" by a
mutex (or similar) so that only one thread/Process can ever be executing at
once, and so that switching between them happens at essentially the same
controlled times as it does at present.

In that case the 'overlapped' flag would be implemented by a thread/Process
releasing the interpreter mutex while it was executing the external call (and
re-acquiring it if it needed to handle a callback), which should be fast enough
that only a few very fast calls should not be overlapped.

I don't know nearly enough about the Dolphin VM to know whether that would
involve a radical upheaval, or just relatively simple modifications to the
Process scheduling code; nor to tell whether the impact on the efficiency of
scheduling would be acceptable (and, course there would be other effects, such
as adding some complexity to the GC with more than one stack to scan).

One could add niceties like lightweight Smalltalk Processes that weren't
implemented with extra threads (i.e. the same as they are now); or using a
thread pool to ease the case where there is considerable "churn" in Processes.
But that's just code ;-)

All just speculation...

    -- chris

"Chris Uppal" <[hidden email]> wrote in message
news:[hidden email]...
Hmmm, what an interesting idea. It might just work (and rather well),
although a couple of issues do spring to mind:
1) How to correctly manage the existing process synchronisation objects.
These would have to be replaced by Win32 equivalents, which would be a good
thing, but it needs some thought to consider all the implications.
2) How to maintain the flexiblity of the ephemeral nature of the existing
main UI process, which allows for multiple modal dialogs. Windows (small w)
in Windows (big W) have an affinity to the thread that created them.
Probably the only way to correctly manage multiple top-level Windows in this
situation (at least if one is to allow dialogs that are modal to a shell and
not the entire application) is to associate a thread with each top-level
window. That arrangement would allow each window to dispatch messages
independently from its own queue (although of course single-threading
through the Smalltalk code), and would eliminate a number of unsatifactory
aspects of the existing implementation.
3) There might be problems supporting Win9x. I'm no longer concerned about
Win95, but at the moment we still intend to support Win98 in D6, even though
we'd really rather not.
4) Per process overhead would increase. The memory usage is no longer an
issue. The context switch time would significantly increase - probably by an
order of magnitude at least, since trapping into the OS to effect a context
switch is very very expensive in comparison to the code the Dolphin VM needs
to execute to achieve the same effect for its own "threads". Also in most
Smalltalk systems one hasn't had to worry about "thread pooling" because the
time to create and tear down threads is very little different to any other
object (i.e. very fast). Heavier weight threads might invalidate some
existing application architectures.

> In that case the 'overlapped' flag would be implemented by a
> thread/Process
> releasing the interpreter mutex while it was executing the external call
> (and
> re-acquiring it if it needed to handle a callback), which should be fast
> enough
> that only a few very fast calls should not be overlapped.
>

It would be tempting to make it the default, but that might prove rather
expensive.

> I don't know nearly enough about the Dolphin VM to know whether that would
> involve a radical upheaval, or just relatively simple modifications to the
> Process scheduling code; nor to tell whether the impact on the efficiency
> of
> scheduling would be acceptable (and, course there would be other effects,
> such
> as adding some complexity to the GC with more than one stack to scan).
>

Oddly enough I think the effects might be felt rather more in the image.
Because we push as much into the image as possible, the infrastructure to
support windows message dispatching, quiescing at idle time, etc, is all
there. The existing idle process, for example, would no longer be needed
since one could just call the MsgWaitForMultipleObjects API directly from
each of the user interface processes. Multiple UI threads would mean that
more process synchronisation would probably be required in the development
system, and any application with multiple top-level windows. If anything I
think the VM would be considerably simplified by this idea.

> One could add niceties like lightweight Smalltalk Processes that weren't
> implemented with extra threads (i.e. the same as they are now); or using a
> thread pool to ease the case where there is considerable "churn" in
> Processes.
> But that's just code ;-)
>

It think it would complicate matters excessively to support lightweight
processes along the lines of the existing Process objects alongside real
Win32 threads. Win32 does have its own lightweight threads, however, called
Fibres (or Fibers in US spelling), which could possibly be used. In fact if
these had been around when the original VM design took place we'd probably
have used them to implement Dolphin's lighweight threads.

Anyway, thanks for the idea. I'll mull it over next week.

Regards

Blair

Blair

> [...] a couple of issues do spring to mind:
> 1) How to correctly manage the existing process synchronisation objects.
> These would have to be replaced by Win32 equivalents, which would be a
> good thing, but it needs some thought to consider all the implications.

And don't the Windows synchronisation primitives have a higher overhead on MP
systems ?


> 2) How to maintain the flexiblity of the ephemeral nature of the existing
> main UI process, which allows for multiple modal dialogs.

And the debugger/walkbacks, I guess ?  But then I don't really understand the
interaction between Windows event dispatch, modal windows, and either real
threads, or the lower levels of Dolphin...


> 4) Per process overhead would increase.
> [...]
> Also in most Smalltalk systems one hasn't had to worry about
> "thread pooling" because the time to create and tear down threads is very
> little different to any other object (i.e. very fast). Heavier weight
> threads might invalidate some existing application architectures.

That's why I thought that a thread pool (in the VM) could be used to reduce the
impact of high turnover of Processes.  Of course that wouldn't help the apps
(if there are any ?) that assume it's cheap to create a few thousand Processes.


    -- chris

Chris, Blair,

> > [...] a couple of issues do spring to mind:
> > 1) How to correctly manage the existing process synchronisation objects.
> > These would have to be replaced by Win32 equivalents, which would be a
> > good thing, but it needs some thought to consider all the implications.
>
> And don't the Windows synchronisation primitives have a higher overhead on
MP
> systems ?

I share this concern.  In fact, I doubt it's limited to MP systems.  If I
understand correctly, the closer one gets to Win32 synch objects and
threads, the closer one gets to a multi-threaded VM and system, with all of
the costs.  Or, can one truly have it both ways?


> That's why I thought that a thread pool (in the VM) could be used to
reduce the
> impact of high turnover of Processes.  Of course that wouldn't help the
apps
> (if there are any ?) that assume it's cheap to create a few thousand
Processes.

I don't use thousands of threads at one time, though it is quite simple to
get "more than ten" going as a baseline, with many short-lived threads for
timeouts etc.  I was surpised to discover that Andreas Raab was very
familiar with Squeak's performance in the case of a thousand or more
Processes, so it apparently can arise.

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]

Bill,

> > That's why I thought that a thread pool (in the VM) could be used to
> > reduce the impact of high turnover of Processes.  Of course that
> > wouldn't help the apps (if there are any ?) that assume it's cheap to
> > create a few thousand Processes.
>
> I don't use thousands of threads at one time, though it is quite simple to
> get "more than ten" going as a baseline, with many short-lived threads for
> timeouts etc.

It's that kind of use of short-lived threads that seems (to me) to call for a
pool of Window's threads (assuming the general technique we're speculating
about).


> I was surpised to discover that Andreas Raab was very
> familiar with Squeak's performance in the case of a thousand or more
> Processes, so it apparently can arise.

It'd be interesting to know what he was using them for.  Off the top of my
head, I can't think of anything except simulation that'd naturally give rise to
so many independent flows-of-control.

    -- chris

"Chris Uppal" <[hidden email]> wrote in message
news:[hidden email]...
> Blair
>
> > [...] a couple of issues do spring to mind:
> > 1) How to correctly manage the existing process synchronisation objects.
> > These would have to be replaced by Win32 equivalents, which would be a
> > good thing, but it needs some thought to consider all the implications.
>
> And don't the Windows synchronisation primitives have a higher overhead on
MP
> systems ?
>

Yes, in the sense that special machine instructions with a lock prefix have
to ensure that each processors caches are kept in sync. Of course micro
performance differences like this may be less significant than higher level
design. Although not exactly "high-level" a critical section behaves
differently on single CPU and multiple CPU Windows kernels. On a single CPU
system a test and set operation is used to determine if the critical section
has been acquired, and to acquire it if not. This uses the aforementioned
interlocked instructions. The cost of this is very low if, as commonly the
common case, no other thread is in the critical section. However if a thread
is in a critical section then a transition into the kernel and a context
switch is needed, which is relatively very expensive. There's not much that
can be done about this because obviously the thread must be blocked until
the critical section can be aquired, and this means a context switch is
necessary. In contrast on a multiple CPU system the critical section employs
a spin lock to allow a short while for the critical section to be released.
The processor spins in a loop for a short while attempting the test and set
operation, and consequently may avoid the expensive context switch if the
critical section is short and is released by a thread running on another
process in time. This means you might incur less context switching overhead
on a multi-processor system where there is a short serialisation bottleneck.

>
> > 2) How to maintain the flexiblity of the ephemeral nature of the
existing
> > main UI process, which allows for multiple modal dialogs.
>
> And the debugger/walkbacks, I guess ?  But then I don't really understand
the
> interaction between Windows event dispatch, modal windows, and either real
> threads, or the lower levels of Dolphin...
>

I think I only understand it momentarily myself :-). Actually the complexity
of this (particularly the means by which multiple C entry points have to be
supported on a single thread stack with serialisation of the returns) is a
strong reason to move to a native thread model.

>
> > 4) Per process overhead would increase.
> > [...]
> > Also in most Smalltalk systems one hasn't had to worry about
> > "thread pooling" because the time to create and tear down threads is
very
> > little different to any other object (i.e. very fast). Heavier weight
> > threads might invalidate some existing application architectures.
>
> That's why I thought that a thread pool (in the VM) could be used to
reduce the
> impact of high turnover of Processes.

Possibly, although why do it in the VM? I like to keep the VM as simple as
possible and push everything I can up into the image.

>...Of course that wouldn't help the
apps
> (if there are any ?) that assume it's cheap to create a few thousand
Processes.

No, but I'm not sure I care much about that anyway. Whoops...

Regards

Blair

Blair,

> Yes, in the sense that special machine instructions with a lock prefix
> have to ensure that each processors caches are kept in sync. [...]

Thanks for the cogent explanation.


> > That's why I thought that a thread pool (in the VM) could be used to
> > reduce the impact of high turnover of Processes.
>
> Possibly, although why do it in the VM? I like to keep the VM as simple as
> possible and push everything I can up into the image.

You mean so that BlockClosuer>>fork would become a higher-level operation
layered over a lower-level pool of Proceses ?


    -- chris

Chris

Yes. When one sends a #terminate message to a Process it causes an exception
to be raised in that Process which is caught at the bottom of the stack.
This has the useful property of running all the unwind/ensure blocks when a
process is terminated. After handling the termination exception the Process
drops out by calling a primitive. At this point the Process could be
returned to a pool rather than invoking a VM primitive to kill it off. The
only reason for the primitive is to make the final termination of the
process an atomic operation that switches away to a new process without
requeuing the dying process (a process has to be running when it finally
dies off in order to unwind the stack). The VMs involvement in process
initiation is even now entirely in the image, with the obvious exception of
the allocation of the memory for the object/stack. In other words the very
little, if any, VM magic is involved in initiating/terminating processes.
This would mean that a different policy for Process allocation could be
employed at the image level.

Regards

Blair

Chris Uppal wrote:


> It'd be interesting to know what he was using them for.  Off the top of my
> head, I can't think of anything except simulation that'd naturally give rise to
> so many independent flows-of-control.

Erlang (http://www.erlang.org) style programming comes to mind.

Instead of catching and handling exceptions a piece-of-work runs in its
own process. When it encounters an error it stops, they say "let it
crash".  You may like to read chapter 4 of "Making reliable distributed
systems in presence of software errors" by Joe Armstrong for a muuuch
beter explanation.


  - Pieter

http://www.sics.se/~joe/thesis/armstrong_thesis_2003.pdf

Pieter Emmelot wrote:

> > It'd be interesting to know what he was using them for.  Off the top of
> > my head, I can't think of anything except simulation that'd naturally
> > give rise to so many independent flows-of-control.
>
> Erlang (http://www.erlang.org) style programming comes to mind.
> http://www.sics.se/~joe/thesis/armstrong_thesis_2003.pdf

Thanks for the links.  I have heard a little about Erlang, but hadn't realised
it was now publically available.  Armstrong's thesis looks interesting, but may
take a while to read...

    -- chris

Pieter Emmelot wrote:
> Chris Uppal wrote:
>
>
>> It'd be interesting to know what he was using them for.  Off the top
>> of my
>> head, I can't think of anything except simulation that'd naturally
>> give rise to
>> so many independent flows-of-control.

Assuming that you are referring to Chris' reference to "thousands or
processes", I too am curious.  However, I wonder whether one really
needs to use thousands of concurrent processes to care about their
overhead.  I use processes to handle things that need to happen in a
given amount of time, or can/must be assumed to have gone wrong.  I
really should instrument it and get some data, but my hunch is that
quite a few of these things get created.  Given more expensive
threading, I could probably come up with a way to queue or otherwise
pool the evaluations myself.  Still, I think there is value in having
fast and efficient threading.


> Erlang (http://www.erlang.org) style programming comes to mind.
>
> Instead of catching and handling exceptions a piece-of-work runs     in
> its own process. When it encounters an error it stops, they     say "let
> it crash".  You may like to read chapter 4 of "Making reliable
> distributed systems in presence of software errors" by Joe Armstrong for
> a muuuch beter explanation.

Thanks for the link.  Of course, one wonders what happens when the
processes do not crash, but instead spew out incorrect data.  Still, it
is an interesting idea.


Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]