An event driven Squeak VM

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
31 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: An event driven Squeak VM

David T. Lewis
 
On Wed, Nov 11, 2009 at 09:16:44PM +0200, Igor Stasenko wrote:
>
> Instead, why not expose the new VM abilities to language, so one could
> tell that specified callout should use a specified thread. Something
> like:
>
> threadHandle := Smalltalk currentThreadId.

It is not hard to make these things accessible in the image, for example:

  OSProcess thisOSProcess pthread ==> a ByteArray(240 166 184 224 255 255 255 255)

(this on a 64-bit platform, hence the 8-byte handle)

Dave

Reply | Threaded
Open this post in threaded view
|

Re: An event driven Squeak VM

Igor Stasenko

2009/11/12 David T. Lewis <[hidden email]>:

>
> On Wed, Nov 11, 2009 at 09:16:44PM +0200, Igor Stasenko wrote:
>>
>> Instead, why not expose the new VM abilities to language, so one could
>> tell that specified callout should use a specified thread. Something
>> like:
>>
>> threadHandle := Smalltalk currentThreadId.
>
> It is not hard to make these things accessible in the image, for example:
>
>  OSProcess thisOSProcess pthread ==> a ByteArray(240 166 184 224 255 255 255 255)
>
> (this on a 64-bit platform, hence the 8-byte handle)
>

its not really matters what object represents the thread handle (it
could be simple smallint).
And surely, its not that hard to expose some thread-aware API to
language level, given that VM _is_ thread-aware.
But by not exposing it, we losing another lever of control, using
which we could write a sophisticated software,
purely in smalltalk without getting down at C level.
That's what i ment to say.

> Dave
>
>



--
Best regards,
Igor Stasenko AKA sig.
Reply | Threaded
Open this post in threaded view
|

Re: An event driven Squeak VM

David T. Lewis
 
On Thu, Nov 12, 2009 at 05:48:52AM +0200, Igor Stasenko wrote:

>
> 2009/11/12 David T. Lewis <[hidden email]>:
> >
> > On Wed, Nov 11, 2009 at 09:16:44PM +0200, Igor Stasenko wrote:
> >>
> >> Instead, why not expose the new VM abilities to language, so one could
> >> tell that specified callout should use a specified thread. Something
> >> like:
> >>
> >> threadHandle := Smalltalk currentThreadId.
> >
> > It is not hard to make these things accessible in the image, for example:
> >
> > ??OSProcess thisOSProcess pthread ==> a ByteArray(240 166 184 224 255 255 255 255)
> >
> > (this on a 64-bit platform, hence the 8-byte handle)
> >
>
> its not really matters what object represents the thread handle (it
> could be simple smallint).
> And surely, its not that hard to expose some thread-aware API to
> language level, given that VM _is_ thread-aware.
> But by not exposing it, we losing another lever of control, using
> which we could write a sophisticated software,
> purely in smalltalk without getting down at C level.
> That's what i ment to say.

Yes I think that I understand your meaning. I was trying to agree
with you ;-)

Unix, Windows, and Mac have similar (but not identical) thread models,
and these can be made available to the image as you suggest. Getting
the cross-platform semantics right may be difficult but I am sure that
it can be done. And yes, the VM must be aware of the threads in order
to expose them to the image in a useful way.

Dave

Reply | Threaded
Open this post in threaded view
|

Re: An event driven Squeak VM

Bert Freudenberg
In reply to this post by Andreas.Raab

On 10.11.2009, at 09:00, Andreas Raab wrote:

> ... make the Squeak VM be "truly event driven" that is invoke it in response to OS or other events instead of having the VM poll.

Don't have time to really get involved but I like the idea :)

- Bert -


Reply | Threaded
Open this post in threaded view
|

Re: An event driven Squeak VM

Jecel Assumpcao Jr
 
Bert Freudenberg wrote:
> On 10.11.2009, at 09:00, Andreas Raab wrote:
>
> > ... make the Squeak VM be "truly event driven" that is invoke it in response to OS or other events instead of having the VM poll.
>
> Don't have time to really get involved but I like the idea :)

I am interested as well. On a related issue, another programmer is
working on the "announcements for i/o events" idea we briefly discussed
in the beginning of the year.

About having the scheduler in the image rather than in the VM, that is
how Self does it. A friend of mine started his PhD thesis using Squeak
but switched to Self because of this feature even though I showed him
that you could use a few tricks to change the behavior of the Squeak
scheduler in interesting ways (I have implemented real-time scheduling
with stuff like first-deadline-first, rate monotonic and so on using a
higher priority thread waiting on the timer as the actual scheduler).

About to pros and cons of having platform dependent code in the image,
there is an interesting discussion in the "Dorado" chapter of the green
book. My proposal to the SqueakNOS group was that we could have two
images: a small headless one with all the SqueakNOS drivers and
primitives and a normal one with the user application. The second image
would send messages to the first one for all its i/o so no code would
directly access the hardware. In practice, you have the same protection
as if there low level stuff were in the VM (or some underlying OS) and
the image stays clean of any platform dependency. But if you really know
what you are doing, you can call up remote debuggers, inspectors and
browser on the first image and it will feel like all this stuff is
implemented in the image instead of the VM.

-- Jecel

Reply | Threaded
Open this post in threaded view
|

Re: An event driven Squeak VM

Stephen Pair
In reply to this post by Bert Freudenberg
 
On Thu, Nov 12, 2009 at 6:56 AM, Bert Freudenberg <[hidden email]> wrote:

On 10.11.2009, at 09:00, Andreas Raab wrote:

> ... make the Squeak VM be "truly event driven" that is invoke it in response to OS or other events instead of having the VM poll.

Don't have time to really get involved but I like the idea :)

- Bert -

Same here (I've only skimmed this thread)...I think making the VM event driven is a great idea (generally speaking).  One benefit is that it lends itself well to wrapping an image/vm as a shared library with efficient inbound calls.  It's also very similar to what you'd need for running on bare hardware where input is eventful (via interrupts).  I would just say that the code listening for and processing the events should be done in Smalltalk as much as is practical at this time.

In the case of running on bare hardware, an interrupt might actually be interrupting interpret(), so you would want to handle that interrupt quickly and safely (from a concurrent access perspective)...for example, you might want to queue an event in some safely shared structure and exit.  To allow for this and similar circumstances involving shared libraries...I could imagine two interfaces...queueEvent() and interpret().

Just some thoughts.

- Stephen
Reply | Threaded
Open this post in threaded view
|

VMMaker MCM config and update

David T. Lewis
In reply to this post by Bert Freudenberg
 
I believe that we now have a working configuration map for VMMaker and sundry plugins:

  MCMcmUpdater updateFromRepositories: #('http://squeaksource.com/VMMaker' )

Once loaded, updates can be requested with VMMaker class>>updateFromServer.

So far, the MCM consists of Andreas recipe, plus a few of my favorites. We'll
need to add references for other plugins needed for Mac OS, SqueakLand, etc.

Dave

Reply | Threaded
Open this post in threaded view
|

Re: VMMaker MCM config and update

Andreas.Raab
 
Cool. Seems to work great. The two "unusual" things I noticed where
MemoryAccess and SlangBrowser. How do I use those?

Cheers,
   - Andreas

David T. Lewis wrote:

>  
> I believe that we now have a working configuration map for VMMaker and sundry plugins:
>
>   MCMcmUpdater updateFromRepositories: #('http://squeaksource.com/VMMaker' )
>
> Once loaded, updates can be requested with VMMaker class>>updateFromServer.
>
> So far, the MCM consists of Andreas recipe, plus a few of my favorites. We'll
> need to add references for other plugins needed for Mac OS, SqueakLand, etc.
>
> Dave
>
Reply | Threaded
Open this post in threaded view
|

Re: VMMaker MCM config and update

David T. Lewis
 
On Tue, Nov 17, 2009 at 09:22:44PM -0800, Andreas Raab wrote:
>
> Cool. Seems to work great. The two "unusual" things I noticed where
> MemoryAccess and SlangBrowser. How do I use those?

MemoryAccess is a version of the sqMemoryAccess.h macros implemented in
Slang. To my surprise it seems to perform just as well as the C macros,
and it is handy for inspecting the generated C code if you are looking
for type declaration problems and improper casts. It is described here:

  http://wiki.squeak.org/squeak/6081

SlangBrowser permits you to browse the generated C code interactively
in the image. It is described here:

  http://wiki.squeak.org/squeak/5916

There are some hacks required to get SlangBrowser integrated into the
browsers (it's different for Squeak 3.8 and 3.9+). I have not yet tried
getting this working for Squeak trunk, and I'll probably be needing some
advice on that as I'm almost certainly not doing it in a good way.

Dave
 
Reply | Threaded
Open this post in threaded view
|

Re: An event driven Squeak VM

Ang BeePeng
In reply to this post by Andreas.Raab
Hi,

I'd like to make my interpreter run until there's no more work to do, and return. I'm not good enough to put things that you explained into coding. Can you show or lead me to get it work?

Or, maybe, can you share the source of event-driven Squeak VM, so that I can learn from the code?

Thanks.

Ang Beepeng

Andreas Raab wrote
 
having interpret() run until "there is
no more work to do" and return from interpret() when it's all said and
done. The trick is that instead of running the idle loop, the VM would
determine that it has no more work to do when there is no runnable
process, so when it finds that there is no runnable process it would
return from interpret saying "my work's done here, there is no more code
to run at this point, ask me again when an external event comes in".

The changes would be fairly straight forward: First, nuke the idle loop
and allow wakeHighestPriority to return nil when there's no runnable
process. Second, have transferTo: do a longjmp to the registered
vmExitBuf to leave interpret(). Third, have interpret register the
vmExitBuf and wake up the highest priorty process like here:

interpret
     "install jmpbuf for main interpreter"
     (self setjmp: vmExitBuf) == 0 ifTrue:[
         self checkForInterrupts. "timers etc"
         "transferTo: longjmps if arg is nil so no need to check"
         self transferTo: self wakeHighestPriority.

         "this is the current interpret() implementation"
         self internalizeIPandSP.
        self fetchNextBytecode.
        [true] whileTrue: [self dispatchOn: currentBytecode in: BytecodeTable].

     ].

At this point we can write a client loop that effectively looks like:

   /* run the interpreter */
   while(!done) {
     /* check for new events */
     ioProcessEvents();
     /* run processes resulting from the events */
     interpret();
   }

Now, obviously this is inefficient, we'd want to replace the
ioProcessEvents() call with something more elaborate that reacts to the
incoming OS events, takes the next scheduled delay into account, checks
for socket handles etc. But I'm sure you're getting the idea. Instead of
wasting our time in the idleProcess, we just return when there's no more
work to do and it's up to the caller to run interpret() as often or as
rarely as desired.

I also think that this scheme could be made backwards compatible by
ensuring that we never call interpret() recursively. In this case an
"old" image with the idle process would run the way it does today, and a
"new" image without the idle process would live in the shiny new event
driven world and return as needed.

What do you think? Any reasons why this wouldn't work?

Cheers,
   - Andreas
Reply | Threaded
Open this post in threaded view
|

Re: An event driven Squeak VM

Andreas.Raab
 
Hi Ang -

I haven't completed the work. At this point I've been able to run the
VM, execute a blocking FFI call (MessageBox, OpenFile) have the VM run
happily in the background until I hit OK in the dialog. When the VM
should return from the callback, it violently crashes :( There is some
state that isn't preserved correctly but it's hard to find these things.
I'll keep folks informed how this unfolds.

Cheers,
   - Andreas


Ang BeePeng wrote:

>  
>
> Hi,
>
> I'd like to make my interpreter run until there's no more work to do, and
> return. I'm not good enough to put things that you explained into coding.
> Can you show or lead me to get it work?
>
> Or, maybe, can you share the source of event-driven Squeak VM, so that I can
> learn from the code?
>
> Thanks.
>
> Ang Beepeng
>
>
> Andreas Raab wrote:
>>  
>> having interpret() run until "there is
>> no more work to do" and return from interpret() when it's all said and
>> done. The trick is that instead of running the idle loop, the VM would
>> determine that it has no more work to do when there is no runnable
>> process, so when it finds that there is no runnable process it would
>> return from interpret saying "my work's done here, there is no more code
>> to run at this point, ask me again when an external event comes in".
>>
>> The changes would be fairly straight forward: First, nuke the idle loop
>> and allow wakeHighestPriority to return nil when there's no runnable
>> process. Second, have transferTo: do a longjmp to the registered
>> vmExitBuf to leave interpret(). Third, have interpret register the
>> vmExitBuf and wake up the highest priorty process like here:
>>
>> interpret
>>      "install jmpbuf for main interpreter"
>>      (self setjmp: vmExitBuf) == 0 ifTrue:[
>>          self checkForInterrupts. "timers etc"
>>          "transferTo: longjmps if arg is nil so no need to check"
>>          self transferTo: self wakeHighestPriority.
>>
>>          "this is the current interpret() implementation"
>>          self internalizeIPandSP.
>> self fetchNextBytecode.
>> [true] whileTrue: [self dispatchOn: currentBytecode in: BytecodeTable].
>>
>>      ].
>>
>> At this point we can write a client loop that effectively looks like:
>>
>>    /* run the interpreter */
>>    while(!done) {
>>      /* check for new events */
>>      ioProcessEvents();
>>      /* run processes resulting from the events */
>>      interpret();
>>    }
>>
>> Now, obviously this is inefficient, we'd want to replace the
>> ioProcessEvents() call with something more elaborate that reacts to the
>> incoming OS events, takes the next scheduled delay into account, checks
>> for socket handles etc. But I'm sure you're getting the idea. Instead of
>> wasting our time in the idleProcess, we just return when there's no more
>> work to do and it's up to the caller to run interpret() as often or as
>> rarely as desired.
>>
>> I also think that this scheme could be made backwards compatible by
>> ensuring that we never call interpret() recursively. In this case an
>> "old" image with the idle process would run the way it does today, and a
>> "new" image without the idle process would live in the shiny new event
>> driven world and return as needed.
>>
>> What do you think? Any reasons why this wouldn't work?
>>
>> Cheers,
>>    - Andreas
>>
>>
>>
>
12