Memory management, OOM, Exceptions

Dear Paolo,

I have some questions regarding the memory management of GST. There
are two unrelated questions below.

1.) 'reducing' memory usage. Is there an easy way to reduce the
memory footprint (e.g. less objects, less oldspace.. etc)? Would it
make sense to introduce a 'small' profile for gst and gst-remote?
Maybe another profile for a lot of memory as well.



2.) For my fakebts (debian packages here[1]) I write code like this..

gst> 1 to: 10000 do: [:unused | [test requireAnyChannel] fork ].

now this can easily generate an OOM and cause an abortion..

gst> 1 to: 10000 do: [:unused |
        [test requireAnyChannel. ObjectMemory compact] fork]

is 'working' on the other hand. I wonder if there could be a better
solution? In Pharo I think there is a OOM watcher task, I wonder if
one could reserve like three contexts.. and triger a cleaner task
once the memory is running low?


holger





[1] http://download.opensuse.org/repositories/home:/zecke23/Debian_6.0/

_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk

Il 27/10/2012 21:00, Holger Hans Peter Freyther ha scritto:
> Dear Paolo,
>
> I have some questions regarding the memory management of GST. There
> are two unrelated questions below.
>
> 1.) 'reducing' memory usage. Is there an easy way to reduce the
> memory footprint (e.g. less objects, less oldspace.. etc)? Would it
> make sense to introduce a 'small' profile for gst and gst-remote?
> Maybe another profile for a lot of memory as well.

Did you try these?

ObjectMemory>>#growThresholdPercent:
ObjectMemory>>#bigObjectThreshold:
ObjectMemory>>#spaceGrowRate:

OOM is a bad problem indeed.  3 contexts are definitely not enough; a
cleaner can allocate memory for BlockClosures too.  There is also the
mark stack, which is needed to actually make some room after running the
cleaner.

Can you do some plots of the ObjectMemory statistics after each of the
10000 iterations?

Paolo

_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk

On Mon, Oct 29, 2012 at 08:56:08AM +0100, Paolo Bonzini wrote:
> ObjectMemory>>#growThresholdPercent:
> ObjectMemory>>#bigObjectThreshold:
> ObjectMemory>>#spaceGrowRate:

I need to try that.. is there another way to shrink that after an image
is running? For one of my images (running for a while)

                size    used            ratio
edenSize 3145728 57240 0.01819610595703
survSpaceSize 131072 114588 0.87423706054688
oldSpaceSize 5452595 774144 0.14197716866923
fixedSpaceSize 5452595 0 0

so my vspace could be a lot smaller (right now it is 337m), resident
space is looking reasonable though.

>
> Can you do some plots of the ObjectMemory statistics after each of the
> 10000 iterations?

I will plot it.

_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk

Il 14/11/2012 17:17, Holger Hans Peter Freyther ha scritto:
vspace includes the total amount allocated to the ObjectMemory.  Most of
it is mapped MAP_NORESERVE, and doesn't really matter.

Looks like only 4.5 MB of memory is wasted (edenSize is very volatile,
do not count it).

Paolo

>
>>
>> Can you do some plots of the ObjectMemory statistics after each of the
>> 10000 iterations?
>
> I will plot it.
>


_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk

On Wed, Nov 14, 2012 at 05:19:13PM +0100, Paolo Bonzini wrote:
> > so my vspace could be a lot smaller (right now it is 337m), resident
> > space is looking reasonable though.
>
> vspace includes the total amount allocated to the ObjectMemory.  Most of
> it is mapped MAP_NORESERVE, and doesn't really matter.

hehe, even if no pages are assigned why is so much allocated? To make sure
all OOPs/Objects will remain in the same area to ease GC?

holger

_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk

Il 14/11/2012 18:56, Holger Hans Peter Freyther ha scritto:
>>> > > so my vspace could be a lot smaller (right now it is 337m), resident
>>> > > space is looking reasonable though.
>> >
>> > vspace includes the total amount allocated to the ObjectMemory.  Most of
>> > it is mapped MAP_NORESERVE, and doesn't really matter.
> hehe, even if no pages are assigned why is so much allocated? To make sure
> all OOPs/Objects will remain in the same area to ease GC?

Yes, that is the reason.  It avoids fragmentation of the heap.

Paolo

_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk

On Wed, Nov 14, 2012 at 10:13:46PM +0100, Paolo Bonzini wrote:
>
> Yes, that is the reason.  It avoids fragmentation of the heap.

Hi,

it makes sense. I am looking at it from the side of running gst on
a 128MB RAM machine and no possible with no overcommit. This would
bring me back to the question if one could introduce a switch to
have fewer possible OOPs/Objects?

holger

_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk