[squeak-dev] Suspending process fix

On 4/29/09, Eliot Miranda <[hidden email]> wrote:

> Hi Andreas,
>
> On Tue, Apr 28, 2009 at 8:33 PM, Andreas Raab <[hidden email]> wrote:

>> This is actually along similar lines of thought that I had when I was
>> thinking of how to get rid of the builtin VM scheduling behavior. The main
>> thought that I had was that the VM may have a "special" process - the
>> scheduler process (duh!) which it runs when it doesn't know what else to
>> do.
>> The VM would then not directly schedule processes after semaphore signals
>> but rather put them onto a "ready" queue that can be read by the scheduler
>> process and switch to the scheduler process. The scheduler process decides
>> what to run next and resumes the process via a primitive. Whenever an
>> external signal comes in, the VM automatically activates the scheduler
>> process and the scheduler process then decides whether to resume the
>> previously running process or to switch to a different process.
>>
>> In a way this folds the timer process into the scheduler (which makes good
>> sense from my perspective because much of the work in the timer is stuff
>> that could be more effectively take place in the scheduler). The
>> implementation should be relatively straightforward - just add a scheduler
>> process and a ready list to the special objects, and wherever the VM would
>> normally process switch you just switch to the scheduler. Voila, there is
>> your user-manipulable scheduler ;-) And obviously, anything that is run
>> out
>> of the scheduler process is by definition non-interruptable because there
>> is
>> simply nothing to switch to!
>
>
> How would you generalise this to a natively multi-threaded VM?  Obviously
> one simple way is to stop the other processors at a suspension point before
> letting the scheduler process proceed, but is there anything cleverer that
> doesn't halt all processors until the singleton scheduler has made its mind
> up?

Every OS thread would have it's own scheduler process. Each scheduler
process has it's own data structure (collection of process lists in
runlevels / balancing tree / whatever) which it has complete ownership
over meaning that it doesn't need to synchonise access to it. Each
process has affinity with a particular scheduler so that the
interrupt-like mechanism Igor mentioned would hold only for that
particular scheduler instance and it's associated list/tree/whatever
of processes.

Each scheduler process would then communicate with other scheduler
processes using something like... umm... a shared queue of messages
that it would poll on... or something. I haven't thought out all the
details here yet. The communication would be necessary for (at least:)

* moving processes between schedulers. This would happen when a
scheduler goes idle so that load balancing happens.

* signalling semaphores where the waiting process belongs to another scheduler.

* and other stuff I haven't thought of yet.

Gulik.

--
http://gulik.pbwiki.com/

On 4/29/09, Andreas Raab <[hidden email]> wrote:

> Eliot Miranda wrote:
> Note that
> you could even run GC only from the scheduler (i.e., treat GC as an
> external signal that requests a GC from the scheduler) which solves the
> concurrent GC problem even with the current VM design.

...or even implement the GC in bytecodes. You could provide primitives
which gives lower level access to the image - raw memory maybe, or
perhaps objects indexed by oop.

It would be a bit tricky managing the garbage the GC itself would
generate. Maybe you could just leave it lying around, or maybe
deallocate it manually? And it would be a bit slow.

Gulik.

--
http://gulik.pbwiki.com/

2009/5/7 Joshua Gargus <[hidden email]>:
> Igor Stasenko wrote:
>
> 2009/5/7 Joshua Gargus <[hidden email]>:
>
>
> Igor Stasenko wrote:
>
> 2009/5/7 Eliot Miranda <[hidden email]>:
>
>
> On Tue, Apr 28, 2009 at 9:26 PM, Michael van der Gulik <[hidden email]>
> wrote:
>
>
> On 4/29/09, Andreas Raab <[hidden email]> wrote:
>
>
> Eliot Miranda wrote:
> Note that
> you could even run GC only from the scheduler (i.e., treat GC as an
> external signal that requests a GC from the scheduler) which solves the
> concurrent GC problem even with the current VM design.
>
>
> ...or even implement the GC in bytecodes. You could provide primitives
> which gives lower level access to the image - raw memory maybe, or
> perhaps objects indexed by oop.
>
> It would be a bit tricky managing the garbage the GC itself would
> generate. Maybe you could just leave it lying around, or maybe
> deallocate it manually? And it would be a bit slow.
>
>
> See Expert to Expert - Erik Meijer and Lars Bak: Inside V8.  Lars Bak thinks
> this is a bad idea, and I agree with him :)
>
>
> Then make GC which can't generate the garbage. Or, in other words, it
> should consume a predictable preallocated limited amount of memory to
> do garbage collection. Since most of GC working in that way, i don't
> see how a bytecode (or better to say - the 'in-language') GC
> implementation could be different. Of course you need to allow a
> direct memory access, and need to be careful with code/objects
> relocations which is used by GC itself during running GC, and of
> course it would be better to run it in native code - for speed reasons
> :)
> This is not something , which is impossible to do.
>
>
> It would be easier to use Hydra instead of jumping though hoops to run the
> GC code from the same object-memory that you're GCing.  And, the effort put
> into making Hydra support this would actually be useful, instead of doing
> something extra-complicated just to show that you can ;-)
>
>
>
> Heh, no this couldn't help. Who will sweep the garbage in object
> memory which used to collect garbage in another object memory? 3rd
> one? :)
>
>
> Sure, why not?   "It's turtles all the way down", and to hell with the
> finite hardware!
>
> Of course you don't need an infinite chain of images.  An image could
> publish a list of services that it provides, one of them being
> garbage-collection.  When an image requires garbage collection, it can send
> a request to one of the images that publish that service.  Or more simply,
> just have 2 GC images that take care of each other's needs (in a massively
> multicore scenario, you'd want more than one image performing GCs anyway).
>

Consider we're talking about self-sustaining system. In this case, you
have a number of heaps where one of them having permission to access
other(s). But in this scenario you will need to have a 'heap of heaps'
then, to rule them out. And who will control that 'heap of heaps'
then? Chicken & egg problem :)

> I think that to make an 'in-language' GC possible, it should use an
> object/classes mirrors, which providing methods required by GC in
> functional style i.e.:
> to visit all refs of a single oop, GC would call following method:
> oop mirror iterateReferencesFor: oop iterationCallback: closure.
> instead of:
> oop allReferences do: [:each| GC mark: each ].
>
>
> Pretty!  The point, I gather, is that you would avoid generating garbage
> during the iteration?
>

Yup.

> i started prototyping these things in Moebius. Currently i'm using
> them to bootstrap an object memory. I could instantiate such mirrors
> in Squeak and then using them to fill the oops in newly created object
> memory. It is possible because a functional/declarative code behavior
> is invariant in any environment.
>
>
>
> (BTW, are you still working on Hydra much?  Waiting for Cog?)
>
>
> I'd prefer help finishing Cog first, then we could get back to Hydra sooner
> :)
>
>
> :-)  Hopefully Eliot's "Stack VM" will be released soon; it should be a good
> point to merge the Hydra changes before "Cog" makes it out.
>
> Mainly, what i don't like in Squeak VM building process is the
> involvement of C compiler.
> I keep advocating the idea that by having a native code generator, we
> don't need the C compiler anymore.
>
>
> That would be swell.  Eliot met Slava Pestov at PyCon, and was quite
> impressed by the code-generator in Factor
> (http://www.mirandabanda.org/cogblog/2008/06/06/cog/#comment-4027).
>

thanks for pointer. i'll read about it now.

> Cheers,
> Josh
>
>
>
> Cheers,
> Josh
>
>
>
>
>
> Gulik.
>
> --
> http://gulik.pbwiki.com/
>

--
Best regards,
Igor Stasenko AKA sig.

2009/5/7 Joshua Gargus <[hidden email]>:

> Igor Stasenko wrote:
>

Mainly, what i don't like in Squeak VM building process is the
involvement of C compiler.
I keep advocating the idea that by having a native code generator, we
don't need the C compiler anymore.