[Glass] gc settings

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[Glass] gc settings

otto
Hi,

Do you have a set of GcUser settings that we can use for aggressive
garbage collection during a migration process? We would like to do the
garbage collection (epoch and reclaim) while the migration process is
generating a lot of dead stuff. We don't mind if the migration takes a
bit longer. We do mind if the migration generates a lot of free space
in the extent.

Thanks
Otto
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: [Glass] gc settings

Dale Henrichs-3
Otto,

I talked this over with one of the guys here and he suggests that the trick is to not create shadow pages faster than the gc gem can handle them ... the gcgems are pretty aggreessive by default ... 

He suggests that you  commit frequently and the  monitor the PagesNeedsReclaiming stat in your migration loop and when it gets above a certain threshold (a couple thousand pages?) stop migrating and go into an "idle loop" where you abort (to avoid CR backlog and allow pages to run through their normal life cycle) and check for the stat to drop below another threshold (250 pages? once it's below your threshold get going again ...

regarding the epoch gc. the become operation does not itself generate a lot of dead if you dow a become between a persistent object and a temp object and commit, the temp object will be written as the persist object and no garbage will be produced ... you will however create shadowpages as the new versions of the objects are copied to new pages (page reclaim is why the gcgems get revved up) ... now if you are creating short-lived persist objects as some other part of your  migration process then an epoch will help keep those dead boys in check ...

Dale


On Fri, Feb 14, 2014 at 4:07 AM, Otto Behrens <[hidden email]> wrote:
Hi,

Do you have a set of GcUser settings that we can use for aggressive
garbage collection during a migration process? We would like to do the
garbage collection (epoch and reclaim) while the migration process is
generating a lot of dead stuff. We don't mind if the migration takes a
bit longer. We do mind if the migration generates a lot of free space
in the extent.

Thanks
Otto
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: [Glass] gc settings

otto
Thanks a lot. Trying it out.

On Fri, Feb 14, 2014 at 9:44 PM, Dale Henrichs
<[hidden email]> wrote:

> Otto,
>
> I talked this over with one of the guys here and he suggests that the trick
> is to not create shadow pages faster than the gc gem can handle them ... the
> gcgems are pretty aggreessive by default ...
>
> He suggests that you  commit frequently and the  monitor the
> PagesNeedsReclaiming stat in your migration loop and when it gets above a
> certain threshold (a couple thousand pages?) stop migrating and go into an
> "idle loop" where you abort (to avoid CR backlog and allow pages to run
> through their normal life cycle) and check for the stat to drop below
> another threshold (250 pages? once it's below your threshold get going again
> ...
>
> regarding the epoch gc. the become operation does not itself generate a lot
> of dead if you dow a become between a persistent object and a temp object
> and commit, the temp object will be written as the persist object and no
> garbage will be produced ... you will however create shadowpages as the new
> versions of the objects are copied to new pages (page reclaim is why the
> gcgems get revved up) ... now if you are creating short-lived persist
> objects as some other part of your  migration process then an epoch will
> help keep those dead boys in check ...
>
> Dale
>
>
> On Fri, Feb 14, 2014 at 4:07 AM, Otto Behrens <[hidden email]> wrote:
>>
>> Hi,
>>
>> Do you have a set of GcUser settings that we can use for aggressive
>> garbage collection during a migration process? We would like to do the
>> garbage collection (epoch and reclaim) while the migration process is
>> generating a lot of dead stuff. We don't mind if the migration takes a
>> bit longer. We do mind if the migration generates a lot of free space
>> in the extent.
>>
>> Thanks
>> Otto
>> _______________________________________________
>> Glass mailing list
>> [hidden email]
>> http://lists.gemtalksystems.com/mailman/listinfo/glass
>
>
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: [Glass] gc settings

Jon Paynter-2
Another option is to allocate extra time for extra aggressive cleanup of your shadow pages.  Then let the migration run all out for as long as it needs.  When its done, kick everyone out of the stone and spin up a bunch of reclaim gems.  Like Dale said, they quite fast.

Im not sure where the "couple thousand" number came from -- but we will only pause a migration for cleanup if the shadowpages get above 4million. 


On Fri, Feb 14, 2014 at 1:35 PM, Otto Behrens <[hidden email]> wrote:
Thanks a lot. Trying it out.

On Fri, Feb 14, 2014 at 9:44 PM, Dale Henrichs
<[hidden email]> wrote:
> Otto,
>
> I talked this over with one of the guys here and he suggests that the trick
> is to not create shadow pages faster than the gc gem can handle them ... the
> gcgems are pretty aggreessive by default ...
>
> He suggests that you  commit frequently and the  monitor the
> PagesNeedsReclaiming stat in your migration loop and when it gets above a
> certain threshold (a couple thousand pages?) stop migrating and go into an
> "idle loop" where you abort (to avoid CR backlog and allow pages to run
> through their normal life cycle) and check for the stat to drop below
> another threshold (250 pages? once it's below your threshold get going again
> ...
>
> regarding the epoch gc. the become operation does not itself generate a lot
> of dead if you dow a become between a persistent object and a temp object
> and commit, the temp object will be written as the persist object and no
> garbage will be produced ... you will however create shadowpages as the new
> versions of the objects are copied to new pages (page reclaim is why the
> gcgems get revved up) ... now if you are creating short-lived persist
> objects as some other part of your  migration process then an epoch will
> help keep those dead boys in check ...
>
> Dale
>
>
> On Fri, Feb 14, 2014 at 4:07 AM, Otto Behrens <[hidden email]> wrote:
>>
>> Hi,
>>
>> Do you have a set of GcUser settings that we can use for aggressive
>> garbage collection during a migration process? We would like to do the
>> garbage collection (epoch and reclaim) while the migration process is
>> generating a lot of dead stuff. We don't mind if the migration takes a
>> bit longer. We do mind if the migration generates a lot of free space
>> in the extent.
>>
>> Thanks
>> Otto
>> _______________________________________________
>> Glass mailing list
>> [hidden email]
>> http://lists.gemtalksystems.com/mailman/listinfo/glass
>
>
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: [Glass] gc settings

Dale Henrichs-3
Yeah a couple thousand was a very conservative figure :) ... depends upon how much extra space you can afford in your repo ... at 4M (16k) pages, that's 64G which might be bigger than a lot of GLASS data bases (not all, though:)

Dale


On Fri, Feb 14, 2014 at 2:32 PM, Jon Paynter <[hidden email]> wrote:
Another option is to allocate extra time for extra aggressive cleanup of your shadow pages.  Then let the migration run all out for as long as it needs.  When its done, kick everyone out of the stone and spin up a bunch of reclaim gems.  Like Dale said, they quite fast.

Im not sure where the "couple thousand" number came from -- but we will only pause a migration for cleanup if the shadowpages get above 4million. 


On Fri, Feb 14, 2014 at 1:35 PM, Otto Behrens <[hidden email]> wrote:
Thanks a lot. Trying it out.

On Fri, Feb 14, 2014 at 9:44 PM, Dale Henrichs
<[hidden email]> wrote:
> Otto,
>
> I talked this over with one of the guys here and he suggests that the trick
> is to not create shadow pages faster than the gc gem can handle them ... the
> gcgems are pretty aggreessive by default ...
>
> He suggests that you  commit frequently and the  monitor the
> PagesNeedsReclaiming stat in your migration loop and when it gets above a
> certain threshold (a couple thousand pages?) stop migrating and go into an
> "idle loop" where you abort (to avoid CR backlog and allow pages to run
> through their normal life cycle) and check for the stat to drop below
> another threshold (250 pages? once it's below your threshold get going again
> ...
>
> regarding the epoch gc. the become operation does not itself generate a lot
> of dead if you dow a become between a persistent object and a temp object
> and commit, the temp object will be written as the persist object and no
> garbage will be produced ... you will however create shadowpages as the new
> versions of the objects are copied to new pages (page reclaim is why the
> gcgems get revved up) ... now if you are creating short-lived persist
> objects as some other part of your  migration process then an epoch will
> help keep those dead boys in check ...
>
> Dale
>
>
> On Fri, Feb 14, 2014 at 4:07 AM, Otto Behrens <[hidden email]> wrote:
>>
>> Hi,
>>
>> Do you have a set of GcUser settings that we can use for aggressive
>> garbage collection during a migration process? We would like to do the
>> garbage collection (epoch and reclaim) while the migration process is
>> generating a lot of dead stuff. We don't mind if the migration takes a
>> bit longer. We do mind if the migration generates a lot of free space
>> in the extent.
>>
>> Thanks
>> Otto
>> _______________________________________________
>> Glass mailing list
>> [hidden email]
>> http://lists.gemtalksystems.com/mailman/listinfo/glass
>
>
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass



_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: [Glass] gc settings

otto
Thanks. This is seem to help. A few methods work together here. There
is some room for improvement, but perhaps this helps someone. When
writing scripts or migrating, we use this method:

SequenceableCollection | committingInBatchesOf: aNumber do: aBlock
  self
    doWithIndex: [ :instance :index |
      aBlock value: instance.
      index \\ aNumber = 0
        ifTrue: [ instance commitOrFail; waitForReclaim ] ];
    commitOrFail;
    waitForReclaim

commitOrFail is a method that we use to raise an exception if a commit failed.

waitForReclaim
  self pagesNeedReclaimSize > 2000
    ifTrue: [
      Transcript show: 'Waiting for reclaim of ' , self
pagesNeedReclaimSize printString , ' pages'.
      [ self pagesNeedReclaimSize > 250 ]
        whileTrue: [
          (Delay forSeconds: 1) wait.
          System abortTransaction ].
      Transcript show: 'Reclaimed' ]

pagesNeedReclaimSize
^ (System cacheStatistics: 1) at: (System cacheStatisticsDescription
indexOf: 'PagesNeedReclaimSize')
www.FinWorks.biz
+27 82 809 2375


On Sat, Feb 15, 2014 at 12:37 AM, Dale Henrichs
<[hidden email]> wrote:

> Yeah a couple thousand was a very conservative figure :) ... depends upon
> how much extra space you can afford in your repo ... at 4M (16k) pages,
> that's 64G which might be bigger than a lot of GLASS data bases (not all,
> though:)
>
> Dale
>
>
> On Fri, Feb 14, 2014 at 2:32 PM, Jon Paynter <[hidden email]> wrote:
>>
>> Another option is to allocate extra time for extra aggressive cleanup of
>> your shadow pages.  Then let the migration run all out for as long as it
>> needs.  When its done, kick everyone out of the stone and spin up a bunch of
>> reclaim gems.  Like Dale said, they quite fast.
>>
>> Im not sure where the "couple thousand" number came from -- but we will
>> only pause a migration for cleanup if the shadowpages get above 4million.
>>
>>
>> On Fri, Feb 14, 2014 at 1:35 PM, Otto Behrens <[hidden email]> wrote:
>>>
>>> Thanks a lot. Trying it out.
>>>
>>> On Fri, Feb 14, 2014 at 9:44 PM, Dale Henrichs
>>> <[hidden email]> wrote:
>>> > Otto,
>>> >
>>> > I talked this over with one of the guys here and he suggests that the
>>> > trick
>>> > is to not create shadow pages faster than the gc gem can handle them
>>> > ... the
>>> > gcgems are pretty aggreessive by default ...
>>> >
>>> > He suggests that you  commit frequently and the  monitor the
>>> > PagesNeedsReclaiming stat in your migration loop and when it gets above
>>> > a
>>> > certain threshold (a couple thousand pages?) stop migrating and go into
>>> > an
>>> > "idle loop" where you abort (to avoid CR backlog and allow pages to run
>>> > through their normal life cycle) and check for the stat to drop below
>>> > another threshold (250 pages? once it's below your threshold get going
>>> > again
>>> > ...
>>> >
>>> > regarding the epoch gc. the become operation does not itself generate a
>>> > lot
>>> > of dead if you dow a become between a persistent object and a temp
>>> > object
>>> > and commit, the temp object will be written as the persist object and
>>> > no
>>> > garbage will be produced ... you will however create shadowpages as the
>>> > new
>>> > versions of the objects are copied to new pages (page reclaim is why
>>> > the
>>> > gcgems get revved up) ... now if you are creating short-lived persist
>>> > objects as some other part of your  migration process then an epoch
>>> > will
>>> > help keep those dead boys in check ...
>>> >
>>> > Dale
>>> >
>>> >
>>> > On Fri, Feb 14, 2014 at 4:07 AM, Otto Behrens <[hidden email]>
>>> > wrote:
>>> >>
>>> >> Hi,
>>> >>
>>> >> Do you have a set of GcUser settings that we can use for aggressive
>>> >> garbage collection during a migration process? We would like to do the
>>> >> garbage collection (epoch and reclaim) while the migration process is
>>> >> generating a lot of dead stuff. We don't mind if the migration takes a
>>> >> bit longer. We do mind if the migration generates a lot of free space
>>> >> in the extent.
>>> >>
>>> >> Thanks
>>> >> Otto
>>> >> _______________________________________________
>>> >> Glass mailing list
>>> >> [hidden email]
>>> >> http://lists.gemtalksystems.com/mailman/listinfo/glass
>>> >
>>> >
>>> _______________________________________________
>>> Glass mailing list
>>> [hidden email]
>>> http://lists.gemtalksystems.com/mailman/listinfo/glass
>>
>>
>>
>> _______________________________________________
>> Glass mailing list
>> [hidden email]
>> http://lists.gemtalksystems.com/mailman/listinfo/glass
>>
>
>
> _______________________________________________
> Glass mailing list
> [hidden email]
> http://lists.gemtalksystems.com/mailman/listinfo/glass
>
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass