2008/7/2 Eliot Miranda <
[hidden email]>:
>
>
> On Wed, Jul 2, 2008 at 3:32 AM, Igor Stasenko <
[hidden email]> wrote:
>>
>> 2008/7/2 John M McIntosh <
[hidden email]>:
>>
>> [ big snip.... ]
>>
>> Few thoughts, about how VM could determine what process(es) is memory
>> hungry to kill them w/o mercy, leaving rest intact and avoid dying:
>>
>> add 'memory policy' slot to Process instance, which can tell VM, is
>> given process can be killed w/o damaging most critical parts of image.
>
> There is no need for this to be in the VM. Instead the process that runs
> once the LowSpaceSemaphore is signalled should be of a very high priority
> (probably above finalization but below timing priority). It then enumerates
> the runnable processes (processes waiting on semaphores are resumably not
> the ones chewing up memory). It then suspends any processes on the runnable
> queues that meet certain criteria.
>
Well, its more flexible but less stable approach. You need to be sure,
that there is someone who still listening for this semaphore, and
where is guarantees that process which waits on this semaphore will
care to clean anything?
Of course, in same way, there is no guarantees that process with
memory policy == 0 will not eat up all available space :)
So, you are right, we can put decision point into image rather in VM.
> Something that *could* be in the VM is accounting of how much space a
> process consumes. Add a slot to each process known to the VM called e.g.
> slotsAllocated. The VM computes the slots allocated between context
> switches. This is cheap to compute because it can compute how much space
> was allocated since the last garbage collection or process switch simply by
> subtracting the allocation pointer at the end of the previous GC or process
> switch from the current allocation pointer. The slots allocated since the
> last process switch is added to the slot in the old process and the slots
> allocated count zeroed on each context switch. We then have an accurate
> measure of how much space each process has allocated.
>
> Wen the low space process runs it simply examines the runnable processes,
> checking their space allocation. It can either maintain a per-process
> allocation rate by computing the amount allocated since the last low space
> signal, or it can zero the per-process allocation count at each low space
> signal.
>
> This is much more flexible and less arbitrary than having the VM do it.
>
> Similarly a VM that does context-to-stack mapping should be able to cheaply
> maintain a stack size count per process since it only has to increase or
> decrease the current process's stack size on each stack page
> overfow/underflow and context switch. Traversing a stack page's frames to
> get an accurate count of the number of "contexts" on each page is pretty
> quick (just a walk of the frame pointer->caller frame pointer chain).
> Getting an approximation by dividing the used portion of a stack page by
> the minimum frame size is even quicker.
>
> You could then provide the image with either an accurate stack depth, or a
> good approximation thereof. That gives the low space process an easy job to
> identify a potentially infinitely recursive process.
>
> In general it is good design to put mechanism in the VM and keep policy up
> in the image.
>
This is what i wanted to avoid: let developer decide what process(es)
should stay and what can die instead of stupid machine :)
Killing processes based on measuring how much space it allocates is
bad practice, because you really don't have any clues why given
process allocated so much space and where its used/involved, and is it
safe to kill such process or not. A 'memory policy' i described is
much better approach for solving such problems. By finding process
with highest possible memory policy slot value and killing it, not the
process which consuming memory a lot. So, the developers can control
in order of preference, what can be killed in case of problem, and
what should stay at any circumstances.
Collecting a memory allocation statistics per running process is
useful (for measuring hard limits in running system, for example), but
not in situation when you reached memory space limits. Because it
can't give answer what process should be killed, because process which
allocated most of memory is just process which allocated most of
memory. This is not means that something wrong with it. Some other
process could covertly sit near, allocating few bytes between each GC
cycle , step by step increasing allocated memory with useless stuff,
and after weeks of stable work you will reach low memory issue. And in
such situation, killing process which allocated most in between full
GCs will buy you nothing. Thats why i think that counting on memory
usage is bad criteria for making decision on what to kill or not.
--
Best regards,
Igor Stasenko AKA sig.
Locked
