[VM][ENH] LowSpaceAndInterruptHandler-dtl

These change sets are updates to the VM and to the low space watcher that
fix low space interrupt handling and user interrupt handling in Morphic.

Under certain conditions the low space watcher was unable to determine the
correct process to suspend following a low space signal. These changes permit
the VM to remember the identity of the process that caused the low space
condition, and to report it to the image through a primitive.

The low space watcher changes are backward compatible for use with VMs
that do not yet have the new primitive (i.e. broken the same way as was
the case in Squeak 3.7).

Caveats:
- This is done with VMMaker 3-8b2, hence is not updated for 64 bit VM.
- I added a numbered primitive, which is probably a bad thing (but I'm not
  sure where best to put it, possibly MiscPrimitivePlugin?).
- There may be a simpler way to accomplish this, but at least this is a
  working implementation.
- I have not tested the interpreter simulator, still need to make sure I did
  not break it.

Cross posted to vm-dev and squeak-dev. Follow-ups should go to vm-dev
(unless someone can think of a way to handle this properly without
touching the VM).

Dave

Thank you for your report.  I have transferred your report to Squeak's
Mantis Database and you can followup on the issue if desired by going to
http://bugs.impara.de/view.php?id=1041 .

In the future please report new issues on Squeak's Mantis Database at
http://bugs.impara.de/ .

Thanks!

On Sun, 2005-04-03 at 17:15 -0400, David T. Lewis wrote:

Hi David -

Thanks for the code but let's not get too fast with this - I am still
trying to understand why you could make the crash happen reliably... in
some ways this doesn't make sense (since the failure should be triggered
statistically) and I'd rather see that we either have situations in
which we do not fail or that we understand why exactly we can repeat
this behavior with 100% reliability (my explanation does not explain that).


Cheers,
   - Andreas

David T. Lewis wrote:

On Mon, Apr 04, 2005 at 09:50:13AM -0700, Andreas Raab wrote:
> Thanks for the code but let's not get too fast with this - I am still
> trying to understand why you could make the crash happen reliably... in
> some ways this doesn't make sense (since the failure should be triggered
> statistically) and I'd rather see that we either have situations in
> which we do not fail or that we understand why exactly we can repeat
> this behavior with 100% reliability (my explanation does not explain that).

It depends on how much time elapses between the low space condition
and the arrival of the signal to the semaphore. If more than 500ms is
consistently being used for garbage collection, then your explanation
holds up very well. (Note: I am using a relatively slow 850MHz Pentium).

Dave

p.s. Speaking of going to fast, there is now a Mantis report
corresponding to this at http://bugs.impara.de/view.php?id=1041.
My posting was intended only as a draft proposal; I should not
have put the [ENH] tag on the message.

On Mon, Apr 04, 2005 at 02:14:06PM -0400, David T. Lewis wrote:
I ran some additional tests this evening and confirmed that, on my machine,
the GC at the time of low space detection is consistently taking more than
500ms. Under these conditions, we should see a 100% failure rate, which
is what I am observing. This is with a 9M fixed memory size, an 850MHz
Pentium, and Unix VM.

The debugging output from five repeated tests is attached, showing console
output produced with #createStackOverflow. The numbers to the left of each
line are millisecond clock values, the first three of them done with printf
in the VM, and the fourth with an OSProcess trace from the image. They
show about 600ms elapsed time between entering allocateChunk() and signaling
the low space semaphore from checkForInterrupts().

Dave

====
28230 allocateChunk() entered
28823 allocateChunk() detected low memory condition
28824 checkForInterrupts() about to signal the low space semaphore
28825:2390:3864:SystemDictionary>>lowSpaceWatcher:low space semaphore signal received, preempted process is
 [] in EventSensor>>eventTickler {[delay wait.  delta := Time millisecondClockValue - lastEventPoll.  (delta <...]}
 BlockContext>>on:do:
 [] in EventSensor>>installEventTickler {[self eventTickler]}
 [] in BlockContext>>newProcess {[self value.  Processor terminateActive]}
====
27584 allocateChunk() entered
28176 allocateChunk() detected low memory condition
28177 checkForInterrupts() about to signal the low space semaphore
28178:1185:3864:SystemDictionary>>lowSpaceWatcher:low space semaphore signal received, preempted process is
 [] in EventSensor>>eventTickler {[delay wait.  delta := Time millisecondClockValue - lastEventPoll.  (delta <...]}
 BlockContext>>on:do:
 [] in EventSensor>>installEventTickler {[self eventTickler]}
 [] in BlockContext>>newProcess {[self value.  Processor terminateActive]}
====
73783 allocateChunk() entered
74374 allocateChunk() detected low memory condition
74375 checkForInterrupts() about to signal the low space semaphore
74377:3662:3864:SystemDictionary>>lowSpaceWatcher:low space semaphore signal received, preempted process is
 [] in EventSensor>>eventTickler {[delay wait.  delta := Time millisecondClockValue - lastEventPoll.  (delta <...]}
 BlockContext>>on:do:
 [] in EventSensor>>installEventTickler {[self eventTickler]}
 [] in BlockContext>>newProcess {[self value.  Processor terminateActive]}
====
121860 allocateChunk() entered
122451 allocateChunk() detected low memory condition
122452 checkForInterrupts() about to signal the low space semaphore
122453:792:3864:SystemDictionary>>lowSpaceWatcher:low space semaphore signal received, preempted process is
 [] in EventSensor>>eventTickler {[delay wait.  delta := Time millisecondClockValue - lastEventPoll.  (delta <...]}
 BlockContext>>on:do:
 [] in EventSensor>>installEventTickler {[self eventTickler]}
 [] in BlockContext>>newProcess {[self value.  Processor terminateActive]}
====
166147 allocateChunk() entered
166743 allocateChunk() detected low memory condition
166744 checkForInterrupts() about to signal the low space semaphore
166745:3674:3864:SystemDictionary>>lowSpaceWatcher:low space semaphore signal received, preempted process is
 [] in EventSensor>>eventTickler {[delay wait.  delta := Time millisecondClockValue - lastEventPoll.  (delta <...]}
 BlockContext>>on:do:
 [] in EventSensor>>installEventTickler {[self eventTickler]}
 [] in BlockContext>>newProcess {[self value.  Processor terminateActive]}
====

This is an update for MVC.

LowSpaceAndInterruptHandlerMVC-dtl.cs.gz adds low space and interrupt key
handling for MVC. Load this after LowSpaceAnddInterruptHandler-dtl, and
load VMMLowSpaceAndInterruptHander-dtl for the VM changes.

With these change sets loaded (and with the VM rebuilt), Squeak MVC passes
the four Andreas tests for interruptability:

  * "[true] whileTrue"
  * "[[true] whileTrue] forkAt: Processor userSchedulingPriority + 1"
  * "Smalltalk createStackOverflow"
  * "[Smalltalk createStackOverflow] forkAt: Processor userSchedulingPriority + 1"

These changes also correct the long-standing (unreported) problem of not being
able to interrupt a high priority background process in MVC (test 2 above).
That problem goes back to at least Squeak 3.6.

(I've attached all three change sets so folks don't have to dig through
the mail to find them, but LowSpaceAndInterruptHandlerMVC-dtl.cs.gz is
the only thing that is new.)

Dave

On Sun, Apr 03, 2005 at 05:15:54PM -0400, David T. Lewis wrote:

On Sun, Apr 03, 2005 at 05:15:54PM -0400, David T. Lewis wrote: Tim suggested (on Mantis) that the process causing the low space condition
could be passed back to the image in the special objects array, rather
than passing the identity hash via a primitive. The implemention is
attached.

LowSpaceAndInterruptHandler-2-dtl - image side patches for Morphic and MVC
VMMLowSpaceAndInterruptHandler-2-dtl - VM patches

Dave