react on low space

Hi guys

SmalltalkImage isSpaceLow

ifTrue: [ <suspend my process> ]

Max,

#isSpaceLow will be of value only if it is called on some type of event (from the gc?) or polled in some way.  When my Dolphin images have crashed due to low space, it has invariably been due to one of two things: (1) early on, improper clean up of ODBC statements; (2) loops over very large numbers of objects serialized from disk without proper attention to #release.  Object Arts often insisted that finalization water marks would save the day in such situations; experience tells me otherwise.

As a general rule, I am in favor of having the system leave behind information of how it failed.  In the case of low space, one *might* have to settle for knowing when a low-space condition arose; a complete set of callstacks might be helpful, assuming the system can write it under such cirumstances.  The computation that goes over the cliff could easily be unrelated to the resource hog that caused the problem.

Bill


________________________________________
From: [hidden email] [[hidden email]] On Behalf Of Max Leske [[hidden email]]
Sent: Tuesday, February 01, 2011 6:00 AM
To: [hidden email]
Subject: Re: [Pharo-project] react on low space

I'm bumping this message. I think it's an important question and could help other developers who might have the same problem one day. The low space watcher is cool but if you don't get the chance to actually look at the stack it's pretty useless.

Max

On 30.01.2011, at 13:10, Max Leske wrote:

Hi guys

Is there an easy way to react on low space? Something like:

SmalltalkImage isSpaceLow
ifTrue: [ <suspend my process> ]

The reason I'm asking is that I have a background process running that causes a low space condition but I never get around to opening the debugger and the image crashes on me. So for debugging purposes I'd like to periodically check the space that is left or alternatively be notified if low space is signalled.

Any ideas?

Thanks,
Max

I think you can catch the OutOfMemory exception:

[ x :=  Array new: 1000.
   1000000 timesRepeat: [ x at: 1 put: (Array new: 10000). x := x at: 1 ]
] on: OutOfMemory do: [:ex |
 ....
]

On 1 February 2011 12:00, Max Leske <[hidden email]> wrote:


--
Best regards,
Igor Stasenko AKA sig.

On 1 February 2011 14:11, Igor Stasenko <[hidden email]> wrote:
> I think you can catch the OutOfMemory exception:
>
Oops. example was a bit wrong. something like that:

 [ | y x | y := x :=  Array new: 1000.
   1000000 timesRepeat: [ x at: 1 put: (Array new: 10000). x := x at: 1 ]
 ] on: OutOfMemory do: [:ex |
  ....
 ]




--
Best regards,
Igor Stasenko AKA sig.

Bill,

I agree that it is important to at least know that there has been a low space condition. But then again, if there is a low space watcher anyway, shouldn't there be a way to register as a listener or something?



Igor,

I haven't run your code yet but it looks exactly like what I was looking for! Thanks.

Max


On 01.02.2011, at 14:13, Igor Stasenko wrote:

On 2 February 2011 09:16, Max Leske <[hidden email]> wrote:
> Bill,
>
> I agree that it is important to at least know that there has been a low space condition. But then again, if there is a low space watcher anyway, shouldn't there be a way to register as a listener or something?
>
>

Well, take a look at OutOfMemory defaultAction,
it signals the lowSpace semaphore.
So, there a process which waiting on this semaphore, and once it
receives this signal it awakens and doing something.

And you are right, there is no way to hook into low space watcher to
include some custom handling.
So, this is a work to do.

Check the SmalltalkImage>>lowSpaceWatcher

There you can extend it by adding the LowSpaceHandler class variable,
then add an appropriate method to register handler,
 and once signal is received, send message to it, like:

LowSpaceHandler ifNotNil: [ LowSpaceHandler handleLowSpaceSignal ].

or something like that.

Also, there are a memory hogs, which almost same thing as a handler,
but a bit wrong terminology.
So, you can register some object as a memory hog:
Smalltalk memoryHogs add: myHog.

Then once signal is arrives, your memory hog will receive
#freeSomeSpace message, but it won't prevent the default UI message
from popping up, informing user that
something there is something which eats too much memory.

Also, i think that mechanism which suspends the process which made the
last memory allocation attempt is useless,  because
there is no any guarantees that exactly given process causing memory
overflow due to some runaway allocation, instead of some other one
which runs in parallel.

Btw, memoryHogs interface is BAD, because by itself it could provoke a
memory leaks.
This container should reference all hogs weakly, so an application
only need to register hogs, but don't unregister it manually,
because you can forget unregistering it, or in case of exception the
code which doing unregister can be not invoked, leaving hog hanging in
memory,
and referenced only by memoryHogs list and not by your application.

I filed the issue for this:
http://code.google.com/p/pharo/issues/detail?id=3646

>
> Igor,
>
> I haven't run your code yet but it looks exactly like what I was looking for! Thanks.
>
> Max
>
>>



--
Best regards,
Igor Stasenko AKA sig.

Max,

I'm on your side, really :)  Silent failures are a big problem and should not be allowed to persist.  What I don't know is whether the remedy for low space is simply to log it and then proceed to crash, or if a handler can do something useful.  The handler probably would need to be registered; otherwise, the low space condition might show up in a thread other than the one hogging all of the memory, I think.

If a handler had to bail me out, I would probably want that logged, but if it is my handler, I should be able to do the logging.

Bill


________________________________________
From: [hidden email] [[hidden email]] On Behalf Of Max Leske [[hidden email]]
Sent: Wednesday, February 02, 2011 3:16 AM
To: [hidden email]
Subject: Re: [Pharo-project] react on low space

Bill,

I agree that it is important to at least know that there has been a low space condition. But then again, if there is a low space watcher anyway, shouldn't there be a way to register as a listener or something?



Igor,

I haven't run your code yet but it looks exactly like what I was looking for! Thanks.

Max


On 01.02.2011, at 14:13, Igor Stasenko wrote: