Smalltalk › Gemtalk › GLASS

wait for reclaim

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

2 messages Options

GLASS mailing list

wait for reclaim

Hi,

We're trying to control repository growth when changing large numbers
of objects. We have 1 extent file.

We've had this discussion on the list before, but our solution prior
to upgrading to 3.2.6 does not seem to work anymore.

We call "waitForReclaim" periodically to give the reclaim gem a chance
to work if there are lots to reclaim.

waitForReclaim
self pagesNeedReclaimSize > 2000
ifTrue: [
self log: 'Waiting for reclaim of ' , self pagesNeedReclaimSize
printString , ' pages'.
[ self pagesNeedReclaimSize > 250 ]
whileTrue: [
(Delay forMilliseconds: 100) wait.
System abortTransaction ].
self log: 'Reclaimed'.
System startCheckpointAsync "The checkpoint frees up the space
allocated by the pages. This is because GS need to keep the pages
between checkpoints to recover from a stone crash." ]

Where pagesNeedReclaimSize is:
^ (System cacheStatistics: 1) at: (System cacheStatisticsDescription
indexOf: 'PagesNeedReclaimSize')

Did something perhaps change quite a bit between 3.1.0.5 and 3.2.6?
Does this strategy still hold?

We get this kind of output in the reclaim gem's log, indicating that
it reclaimed a huge number of pages:

--- 06/22/2015 17:58:05.198 SAST Reclaim Summary
2530 reclaims 975978 pagesReclaimed 859450 singleObjPages
28324822 processedObjs 28324822 liveObjs 0 deadObjs 29.0 avgObjsPerPage

Thanks
Otto
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

GLASS mailing list

Re: wait for reclaim

Otto,

I think I need a little more information ... We would probably need a
statmon file to tell what is actually going on and some information on
how you are "changing large numbers of objects" and where the
#waitForReclaim is called in relation to the changes you are made:
perhaps some session stats to annotate when a waitForReclaim is called
and then number of times you loop waiting for pagesNeedReclaimSize to
drop...

After a bit of internal discussion, we *have* made changes
(improvements) to multi-threaded reclaim in 3.2.x and if I were to make
wild guess, I guess that the the reclaim threads are spinning away fast
enough that the PagesNeedReclaimSize stat never reaches 2000.... But
that is a guess ... a statmon would make it possible for us to pinpoint
the "problem" ...

Mariano, this is a case where the growth is happening on disk, so it
would no be directly related to the memory footprint of seaside gems ...

Dale

On 06/22/2015 09:24 AM, Otto Behrens via Glass wrote:

> Hi,
>
> We're trying to control repository growth when changing large numbers
> of objects. We have 1 extent file.
>
> We've had this discussion on the list before, but our solution prior
> to upgrading to 3.2.6 does not seem to work anymore.
>
> We call "waitForReclaim" periodically to give the reclaim gem a chance
> to work if there are lots to reclaim.
>
> waitForReclaim
> self pagesNeedReclaimSize > 2000
> ifTrue: [
> self log: 'Waiting for reclaim of ' , self pagesNeedReclaimSize
> printString , ' pages'.
> [ self pagesNeedReclaimSize > 250 ]
> whileTrue: [
> (Delay forMilliseconds: 100) wait.
> System abortTransaction ].
> self log: 'Reclaimed'.
> System startCheckpointAsync "The checkpoint frees up the space
> allocated by the pages. This is because GS need to keep the pages
> between checkpoints to recover from a stone crash." ]
>
> Where pagesNeedReclaimSize is:
> ^ (System cacheStatistics: 1) at: (System cacheStatisticsDescription
> indexOf: 'PagesNeedReclaimSize')
>
> Did something perhaps change quite a bit between 3.1.0.5 and 3.2.6?
> Does this strategy still hold?
>
> We get this kind of output in the reclaim gem's log, indicating that
> it reclaimed a huge number of pages:
>
> --- 06/22/2015 17:58:05.198 SAST Reclaim Summary
> 2530 reclaims 975978 pagesReclaimed 859450 singleObjPages
> 28324822 processedObjs 28324822 liveObjs 0 deadObjs 29.0 avgObjsPerPage
>
> Thanks
> Otto
> _______________________________________________
> Glass mailing list
> [hidden email]
> http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass