Why leave Admin GcGem and Reclaim GcGem running at all time?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Why leave Admin GcGem and Reclaim GcGem running at all time?

GLASS mailing list
Hi guys,

I am reading the sys admin manual and it says:
  • We recommend that you leave the Admin GcGem running at all times, although this GcGem is required only following a markForCollection or markGcCandidatesFromFile:, or epoch garbage collection operation. (Subsequent sections of this chapter describe these operations in detail.) If the Admin GcGem is not running following one of these operations, the garbage collection process cannot complete, and garbage can build up in your system.

  • We recommend that you have at least one Reclaim GcGem running at all times, to reclaim shadow objects. 



Why do you recommend running both at all time if they only happen to be needed for MFC and #reclaimAll? (if I do not use epoch GC, of course)
In other words...what are the drawbacks if I only start/stop them as part of my GC scripts?

Thanks in advance,

--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: Why leave Admin GcGem and Reclaim GcGem running at all time?

GLASS mailing list


On Fri, Jun 19, 2015 at 7:34 PM, James Foster <[hidden email]> wrote:
Hi Mariano,

The repository-wide GC process is somewhat complex. In addition to the Sys Admin guide, I recommend videos 19-22 in https://programminggems.wordpress.com/2010/02/05/scaling-objects-videos/.

In any case, the Reclaim GcGem is needed at all times (as opposed to the Admin GcGem). One interesting characteristic of GemStone is that objects are NEVER modified in place. When you modify an object the new version of the object is written to a new page and the object table associated with that commit points to the new page while the object table associated with earlier commits point to the old page. This is what allows transaction consistency and repeatable reads (an important attribute of any database system). 

This design means that you can end up with the same object existing on many pages in the repository (each a different version associated with a different transaction). One of the last stages in the transaction processing happens when a particular view of the database (associated with a commit record) is no longer visible to any session. At that point the old versions of committed objects ("shadowed objects”) are taking up space on “shadowed” pages, and this space is essentially wasted. Note, however, that there may be other objects on these pages that are still visible. 

One of the tasks of the Reclaim GcGem is to take the live objects from shadowed pages and copy them onto new pages (without the shadowed objects). This temporarily increases the number of pages in use (since the same objects are on two pages), but eventually the shadowed page can be reclaimed. For this reason it is important to keep the Reclaim GcGem running all the time; otherwise your repository will fill with shadowed objects that waste space. It seems that leaving the Admin GcGem running should be fine as well.

Does that help?


Thanks James, crystal clear!!! 
 
I was just trying to see how I can minimize the memory consume while the stone/gems are not being too much used. 

Thanks,

James Foster
Director of Operations
GemTalk Systems LLC
<a href="tel:%2B1%20503%20766%204714" value="+15037664714" target="_blank" class="vt-p">+1 503 766 4714 (voice & fax)
[hidden email]

On Jun 19, 2015, at 3:15 PM, Mariano Martinez Peck via Glass <[hidden email]> wrote:

Hi guys,

I am reading the sys admin manual and it says:
  • We recommend that you leave the Admin GcGem running at all times, although this GcGem is required only following a markForCollection or markGcCandidatesFromFile:, or epoch garbage collection operation. (Subsequent sections of this chapter describe these operations in detail.) If the Admin GcGem is not running following one of these operations, the garbage collection process cannot complete, and garbage can build up in your system.

  • We recommend that you have at least one Reclaim GcGem running at all times, to reclaim shadow objects. 



Why do you recommend running both at all time if they only happen to be needed for MFC and #reclaimAll? (if I do not use epoch GC, of course)
In other words...what are the drawbacks if I only start/stop them as part of my GC scripts?

Thanks in advance,

--
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass




--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: Why leave Admin GcGem and Reclaim GcGem running at all time?

GLASS mailing list
> On Jun 19, 2015, at 3:47 PM, Mariano Martinez Peck <[hidden email]> wrote:
>
> I was just trying to see how I can minimize the memory consume while the stone/gems are not being too much used.

I assume that the OS will swap out inactive processes, so that shouldn’t be much of a concern.

James

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: Why leave Admin GcGem and Reclaim GcGem running at all time?

GLASS mailing list


On Fri, Jun 19, 2015 at 7:53 PM, James Foster <[hidden email]> wrote:
> On Jun 19, 2015, at 3:47 PM, Mariano Martinez Peck <[hidden email]> wrote:
>
> I was just trying to see how I can minimize the memory consume while the stone/gems are not being too much used.

I assume that the OS will swap out inactive processes, so that shouldn’t be much of a concern.

I was also thinking about that. Do you know is seaside gems ever get to the idle state in which the OS would swap it's memory?
On the other hand, I have the seaside maintenance vm running which expires sessions every 1 hour...so maybe that task makes all seaside gems none-idle ?  mmmmm


--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: Why leave Admin GcGem and Reclaim GcGem running at all time?

GLASS mailing list


On 06/19/2015 04:01 PM, Mariano Martinez Peck via Glass wrote:


On Fri, Jun 19, 2015 at 7:53 PM, James Foster <[hidden email]> wrote:
> On Jun 19, 2015, at 3:47 PM, Mariano Martinez Peck <[hidden email]> wrote:
>
> I was just trying to see how I can minimize the memory consume while the stone/gems are not being too much used.

I assume that the OS will swap out inactive processes, so that shouldn’t be much of a concern.

I was also thinking about that. Do you know is seaside gems ever get to the idle state in which the OS would swap it's memory?
On the other hand, I have the seaside maintenance vm running which expires sessions every 1 hour...so maybe that task makes all seaside gems none-idle ?  mmmmm

I am not a  student of Linux swapping algorithms, but I'm pretty sure that the maintenance vm will not cause other seaside gems to "seem non-idle".

The mainenance vm, will activate it's own memory pages and those pages in the SPC that are needed as well, but the private memory of other gems will not be affected ...

We really try to make sure that idle gems stay idle and will only wake up when needed ... but there is at least one restless process that wakes up every 30 seconds to facilitate the handling of SigAborts ... the process itself doesn't touch much memory, but if a SigAbort is required then the gem will also do an abort and page in a bit more memory, but an abort is pretty inexpensive for a previously idle process ...

BTW, this little "sigabort process" would be a spot where you could add the logic for running the periodic scavenge and mark sweep:)

Dale

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass