Smalltalk › Cincom › VisualWorks

ObjectMemory and finalization

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

2 messages Options

Ladislav Lenart

ObjectMemory and finalization

Hello,

we run into trouble with ObjectMemory and finalization mechanism.
We were profiling loading of large project from file to our application
using something like this (on VW7.4 running on Linux/Debian 3GHz Intel
Pentium IV, 1GB RAM):

| x app |
x := 1.
app := Application start.
x timesRepeat: [
app loadProjectaFrom: 'filename' asFilename.
"This loads project from given file but not assign
it anywhere so the project can be GC'ed."
].

One project loads without problems and it seems that it occuppies cca 50MB
in memory (including SubscriptionRegistries and our XML machinery that loads
it), because GC reclaims about this much memory afterwards. However when we
run the load of project several times in a row we often get primitive 469
failed error in:

WeakArray class>>growFinalizationQueueTo: newSize
"Attempt to grow the finalization queue to the requested number of
slots. Fail if the argument is not a positive SmallInteger or if the
Finalization Queue cannot be grown."

<primitive: 469 errorCode: error>
^self primitiveFailed

We got thir error regularly in workspace for 20 projects and almost always
in time or allocations profiler for 10 (and even for 2) projects.

The newSize argument is something about 100000 when this occurs. The problem,
it seems, is that unreferenced projects are not Gc'ed fast enough so they start
to fill in memory which eventually leads to the failure. Though I still don't
understnad it much because overall system memory is fine. Next interesting thing
is, when I load one project then I have to perform 2 global GCs to actually
free the memory (#globalGarbageCollect).

I was reading AppDevGuide but there is nothing regarding this issue...

So my questions are:
1. How can one retrieve exact size of an object (something like sizeof operator
from C would be enough)?
2. Given the fact that 100000 is a SmallInteger what is/can be the cause of
the primitive failure - why Finalization Queue cannot be grown? Does this
have something to do with MemoryPolicy settings because the overall OS
memory was fine?
3. Why I have to run 2 global compacting Gc's (#globalGarbageCollect) in a
row in order to actually garbage the garbage in this case? Or should I
use different method?
4. Links to documentation where I can read more about this (or about setting
up my own MemoryPolicy)?

Thank you for any hints,

Ladislav Lenart

Terry Raymond

RE: ObjectMemory and finalization

Ladislav

I would suspect that the IGC is not doing its job. Furthermore,
tuning the IGC so it works properly ia a very difficult task.
When you have a process that runs continually for a long time
and generates a fair amount of garbage it is the job of the
IGC to collect the garbage. It is supposed to keep up with
the garbage generation rate, but it generally fails to do so.
When the system encounters a low space condition it then goes
though a series of attempts to grow memory or garbage collection
in an attempt to provide enough memory at the lowest cost.
Because of various techniques available one of them usually
succeeds in some manner. If the IGC operated correctly then
the system would not have to do a full gc or data compaction
as often to free up memory.

However, sometimes increasing the size of new space will
compensate for a poorly running IGC and reduce the occurance
of full gcs and data compacts.

If you have not tried already I would begin by increasing
Eden and SurvivorSpace by a factor of 10.

Terry

===========================================================
Terry Raymond Smalltalk Professional Debug Package
Crafted Smalltalk
80 Lazywood Ln.
Tiverton, RI 02878
(401) 624-4517 [hidden email]
<http://www.craftedsmalltalk.com>
===========================================================

> -----Original Message-----
> From: Ladislav Lenart [mailto:[hidden email]]
> Sent: Thursday, August 17, 2006 7:24 AM
> To: vwnc
> Subject: ObjectMemory and finalization
>
> Hello,
>
> we run into trouble with ObjectMemory and finalization mechanism.
> We were profiling loading of large project from file to our application
> using something like this (on VW7.4 running on Linux/Debian 3GHz Intel
> Pentium IV, 1GB RAM):
>
> | x app |
> x := 1.
> app := Application start.
> x timesRepeat: [
> app loadProjectaFrom: 'filename' asFilename.
> "This loads project from given file but not assign
> it anywhere so the project can be GC'ed."
> ].
>
> One project loads without problems and it seems that it occuppies cca 50MB
> in memory (including SubscriptionRegistries and our XML machinery that
> loads
> it), because GC reclaims about this much memory afterwards. However when
> we
> run the load of project several times in a row we often get primitive 469
> failed error in:
>
> WeakArray class>>growFinalizationQueueTo: newSize
> "Attempt to grow the finalization queue to the requested number of
> slots. Fail if the argument is not a positive SmallInteger or if the
> Finalization Queue cannot be grown."
>
> <primitive: 469 errorCode: error>
> ^self primitiveFailed
>
> We got thir error regularly in workspace for 20 projects and almost always
> in time or allocations profiler for 10 (and even for 2) projects.
>
> The newSize argument is something about 100000 when this occurs. The
> problem,
> it seems, is that unreferenced projects are not Gc'ed fast enough so they
> start
> to fill in memory which eventually leads to the failure. Though I still
> don't
> understnad it much because overall system memory is fine. Next interesting
> thing
> is, when I load one project then I have to perform 2 global GCs to
> actually
> free the memory (#globalGarbageCollect).
>
> I was reading AppDevGuide but there is nothing regarding this issue...
>
> So my questions are:
> 1. How can one retrieve exact size of an object (something like sizeof
> operator
> from C would be enough)?
> 2. Given the fact that 100000 is a SmallInteger what is/can be the cause
> of
> the primitive failure - why Finalization Queue cannot be grown? Does
> this
> have something to do with MemoryPolicy settings because the overall OS
> memory was fine?
> 3. Why I have to run 2 global compacting Gc's (#globalGarbageCollect) in a
> row in order to actually garbage the garbage in this case? Or should I
> use different method?
> 4. Links to documentation where I can read more about this (or about
> setting
> up my own MemoryPolicy)?
>
> Thank you for any hints,
>
> Ladislav Lenart