https://forum.world.st/Alien-and-callbacks-from-foreign-threads-tp3175989p3176374.html
On Wed, Jan 5, 2011 at 11:58 AM, Igor Stasenko
<[hidden email]> wrote:
On 5 January 2011 20:07, Eliot Miranda <
[hidden email]> wrote:
>
>
> On Wed, Jan 5, 2011 at 10:36 AM, Igor Stasenko <
[hidden email]> wrote:
>>
>> I think any interoperability should have a reasonable limitations.
>> Otherwise, there is no way how to guarantee the system stability.
>>
>> Squeak and Cog virtual machines can't run multiple native threads
>> interpreting smalltalk code, which means that
>> any callback(s) will be forced to be handled in single (VM) thread.
>
> No. One can arrange to shae the VM between native threads just like Python,
> which is what I've done in my prototype.
>
much for it.
What do you mean "too much for it"? It costs too much?
>>
>> Apparently callbacks from foreign threads require a special checks to
>> be added to detect it and then serialize the call to vm thread.
>
> No. One can arrange for the current thread to cede ownership to the foreign
> thread and allow it to run. With a two-evel scheduler one can still arrange
> to preserve Smalltalk process priorities and scheduling semantics.
> Essentially one assigns a priority to foreign callbacks and they get to run
> as soon as their priority is high enough to displace the priority of the
> current process (in whatever thread it may be).
> The scheme you're describing is one I implemented in VisualWorks and it is
> slow and clumsy. The scheme I'm now using is due to David Simmons and by
> comparison is lightweight and fast. But neither are simple.
>
Sharing VM among multiple threads.. Okay.. but this is same.
It doesn't eliminates a need of synchronizing with thread which
currently 'owns' VM to obtain a control,
which means you can't start handling callback immediately, that's why
i told that it kills an idea of using multithreading.
That doesn't follow. he fact you can't release the VM immediately doesn't mean it can't happen very quickly, certainly quickly enough. Nothing happens instantaneously in a computer. Computations take time. Thread switches take time. But computers are fast. So I don't see that the idea of using multithreading is killed at all.
Again, i hope you realizing that there is a lot of situations, where
this won't go.
Suppose that i using some library which using a thread-local storage.
How i could guarantee that all calls to this library will be made in
same thread,
in situation, when VM could run interpreter in any random thread,
resulted because of need to handle callback
from other library, not related to one that used by code related to my library?
Because my system allows one to bind a process to a specific thread so that you can guarantee which thread a computation will run in by reserving a process for that computation and binding it to the relevant thread. The process scheduler (remember this is now a two-level scheduler) takes care of switching threads when one switches processes.
I don't see how it can be made 'automagically' without introducing an
additional discipline in code, and without any
notion of native threads at language side, where developer could
enforce some rules, known by him. Otherwise an interoperability scheme
which you introducing is mine field.
If one has to interoperate with code that has threading constraints (such as your example above) then one needs to code to those constraints. To be able to code to those constraints there has to be some relevant mechanism. That relevant mechanism is implementable (I have implemented it). But I don't see what your argument is. The current non-threaded system doesn't provide any way of interacting with foreign threads directly, right? So interacting requires programming. Introducing threading into the VM makes things easier, not more difficult, no?
Also, now you must check every single primitive which currently used
by VM is thread safe, i.e.
it can be invoked from any 'random' thread without leading to unwanted
effect(s). Which is a good thing (oh and i did that in Hydra btw).
Why?
I know very well that some OS(es) and some libraries don't like when
you using their functions either
from non-main process thread, or from multiple threads. (Btw, Johm
told scary story about GUI and multithreading on Macs multiple times
;) )
So, before introducing such kind of sharing , you should take care
about every single
primitive which using OS-specific function(s) or using some external
library, that it is safe to make a calls
from random thread.
You certainly need to be able to control threading, yes. But you're being a bit of a Cassandra. Most primitives are the VM's own and they work fine whatever thread they're in. GUI activities can be restricted to the GUI thread. FFI calls can be marked as threaded or non-threaded. Porcesses can be bound to threads or free to run on any thread. So there's lots of control. The system doesn't collapse into chaos as soon as one introduces a thread.
So, the question remains open: is it worth spending so much efforts
for such small outcome? :)
Small outcome? The ability to interact with multi-threaded libraries is a small outcome? The ability to perform non-blocking calls is a small outcome? We see things differently :)
best
Eliot
>>
>> And therefore, handling callback in different thread will require a
>> synchronization with it, which kills an idea of
>> invoking callback from different thread in a first place.
>
> Nope.
>
>>
>> So, even if Alien will implement support of callbacks in non-VM
>> thread(s), the value of such feature are questionable,
>> because it simply kills the attempt(s) of foreign language to gain
>> performance benefits through exploiting multithreading.
>
> I disagree.
>
access to VM run-time are obviosly synchronous. And as long as it true,
there is very small reason to use multiple threads.
A simple example. Suppose you using a 3rd party library for setting up
a socket server, which is a single call, accepting a pointer to
function
which will handle incoming connections. The external library takes
care about rest itself: it using thread pool(s) for incoming
connections
and in this way distributing load among multiple native threads.
Now, what will happen if you pass a smalltalk callback to it?
I can elaborate that, but i think you know it yourself.
>>
>> Allowing to interpret smalltalk using multiple threads is another
>> story - see RoarVM :)
>
> That's not the only way to go. Python (and my Cog prototype) shares the VM
> amongst native threads and it is quite a lot simpler than implementing a
> concurrently-threaded VM.
sure its simpler. but kills multithreading benefits :)
>>
>> So, unless you running RoarVM, i don't see that it is reasonable to
>> support cross-thread callbacks.
>
> Not at all. It's quite a reasonable thing to want to do, and is not beyond
> us.
Yeah, it is painful to turn once non-thread aware VM into be thread
aware and play nicely with other players on [mine] field :)
> best
> Eliot
>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>
>
--
Best regards,
Igor Stasenko AKA sig.