My extent0.dbf also grows ...

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

My extent0.dbf also grows ...

GLASS mailing list
Hey,

I've also problems with the size of my extent0.dbf. Within several hours
(running our tests) the extents grows to more than 20 GB in size.

I'm not using Seaside, but just Zinc as REST server. The tests are doing
no big real data creation, but more or less only movements of objects
around (the database world).

The ObjectLog has only 3000 entries (created by me) - therefore not
really much data. The database seems to be pretty empty:

SystemRepository fileSize      21.298.675.712
SystemRepository freeSpace     20.171.669.504

We are producing lots of HTTP requests and lots of transactions with our
test tools.

I've not investigated much in this area - and I start the stone via
normal startStone script - nothing else.

Any idea ?


Marten

--
Marten Feldtmann
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: My extent0.dbf also grows ...

GLASS mailing list

Tell us a bit more about your gc configuration. Do you run seaside maintenance vm? How frequently do you run mfc? Do you reclaim as well?

BTW was it there a bit import operation during those hours?

On Aug 23, 2015 3:51 AM, "[hidden email] via Glass" <[hidden email]> wrote:
Hey,

I've also problems with the size of my extent0.dbf. Within several hours
(running our tests) the extents grows to more than 20 GB in size.

I'm not using Seaside, but just Zinc as REST server. The tests are doing
no big real data creation, but more or less only movements of objects
around (the database world).

The ObjectLog has only 3000 entries (created by me) - therefore not
really much data. The database seems to be pretty empty:

SystemRepository fileSize      21.298.675.712
SystemRepository freeSpace     20.171.669.504

We are producing lots of HTTP requests and lots of transactions with our
test tools.

I've not investigated much in this area - and I start the stone via
normal startStone script - nothing else.

Any idea ?


Marten

--
Marten Feldtmann
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: My extent0.dbf also grows ...

GLASS mailing list
Actually I only do a startStone command and nothing more special and I
thought, that this would help for the most cases.

That's the process structure I have (without the http server processes):

>  2760 ?        Ss     0:03          \_ /home/glass/gsDevKitHome/gemstone/products/GemStone64Bit3.2.7-x86_64.Linux/sys/netldid webcati2_ldi -g -aglass -l
>  3760 ?        Ssl    0:25          \_ /home/glass/gsDevKitHome/gemstone/products/GemStone64Bit3.2.7-x86_64.Linux/sys/stoned webcati2 -N
>  3762 ?        Sl     0:25              \_ /home/glass/gsDevKitHome/gemstone/stones/webcati2/product/sys/shrpcmonitor 'webcati2~53f740ed88ff7716' setloc
>  3769 ?        S      0:03              \_ /home/glass/gsDevKitHome/gemstone/stones/webcati2/product/sys/pgsvrmain webcati2~53f740ed88ff7716 0 1 -1 TCP
>  3772 ?        S     44:01              \_ /home/glass/gsDevKitHome/gemstone/stones/webcati2/product/sys/pgsvrmain TCP 13 90
>  3785 ?        Sl     0:02              \_ /home/glass/gsDevKitHome/gemstone/stones/webcati2/product/sys/gem reclaimgcgem 'webcati2' 1 -T 5000
>  3789 ?        Sl     0:19              \_ /home/glass/gsDevKitHome/gemstone/stones/webcati2/product/sys/gem admingcgem 'webcati2' -T 5000

Because of the numbers I posted I thought, that the database is 90%
empty - therefore the gc seems to work.


Am 23.08.2015 um 16:06 schrieb Mariano Martinez Peck:
> Tell us a bit more about your gc configuration. Do you run seaside
> maintenance vm? How frequently do you run mfc? Do you reclaim as well?
>

?


--
Marten Feldtmann
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: My extent0.dbf also grows ...

GLASS mailing list
In reply to this post by GLASS mailing list
Hi Marten,

The best way of analyzing these problems is by looking at logs and statistics. Are you running statmonitor? If not, then you should! If so, then we can do some analysis. Absent statistics (and even with them), we can often figure out things based on a description of the symptoms.

The extent should not grow unless there is no space in the extent. So, even though now you see a lot of free space, at the time of the growth there was no space available. While this could be due to application data (including the ObjectLog), those would tend to remain and you would not have free space now.

The most common cause of unexpected repository growth that is resolved when you restart the database is a “commit record backlog” that comes from having a logged-in session that does not commit while other sessions are committing.

While you are running in Jade, go to the ‘All Sessions’ tab on the Transcript, click Update, and then look for a session that has an old view age. Depending on your version, there might also be something in the ‘Backlog’ column that shows the commit record backlog. If that is more than a dozen or so, you should track down the oldest session and have it do an abort or commit.

James

P.S. Note also that as a paying customer (thank you!) you may file a help request with tech support (though I’m happy to offer comments here as well).


> On Aug 22, 2015, at 11:51 PM, [hidden email] via Glass <[hidden email]> wrote:
>
> Hey,
>
> I've also problems with the size of my extent0.dbf. Within several hours
> (running our tests) the extents grows to more than 20 GB in size.
>
> I'm not using Seaside, but just Zinc as REST server. The tests are doing
> no big real data creation, but more or less only movements of objects
> around (the database world).
>
> The ObjectLog has only 3000 entries (created by me) - therefore not
> really much data. The database seems to be pretty empty:
>
> SystemRepository fileSize      21.298.675.712
> SystemRepository freeSpace     20.171.669.504
>
> We are producing lots of HTTP requests and lots of transactions with our
> test tools.
>
> I've not investigated much in this area - and I start the stone via
> normal startStone script - nothing else.
>
> Any idea ?
>
>
> Marten
>
> --
> Marten Feldtmann
> _______________________________________________
> Glass mailing list
> [hidden email]
> http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: My extent0.dbf also grows ...

GLASS mailing list


On Sun, Aug 23, 2015 at 3:49 PM, James Foster via Glass <[hidden email]> wrote:
Hi Marten,

The best way of analyzing these problems is by looking at logs and statistics. Are you running statmonitor? If not, then you should! If so, then we can do some analysis. Absent statistics (and even with them), we can often figure out things based on a description of the symptoms.

The extent should not grow unless there is no space in the extent. So, even though now you see a lot of free space, at the time of the growth there was no space available. While this could be due to application data (including the ObjectLog), those would tend to remain and you would not have free space now.

The most common cause of unexpected repository growth that is resolved when you restart the database is a “commit record backlog” that comes from having a logged-in session that does not commit while other sessions are committing.

While you are running in Jade, go to the ‘All Sessions’ tab on the Transcript, click Update, and then look for a session that has an old view age. Depending on your version, there might also be something in the ‘Backlog’ column that shows the commit record backlog. If that is more than a dozen or so, you should track down the oldest session and have it do an abort or commit.


James, cannot it be the same gem of Jade itself (I have never used Jade so I don't know)? You know, like when using GemTools and you do have dirty objects and do not commit, you forget about it and you let it open (happened to me many times).  

 
James

P.S. Note also that as a paying customer (thank you!) you may file a help request with tech support (though I’m happy to offer comments here as well).


> On Aug 22, 2015, at 11:51 PM, [hidden email] via Glass <[hidden email]> wrote:
>
> Hey,
>
> I've also problems with the size of my extent0.dbf. Within several hours
> (running our tests) the extents grows to more than 20 GB in size.
>
> I'm not using Seaside, but just Zinc as REST server. The tests are doing
> no big real data creation, but more or less only movements of objects
> around (the database world).
>
> The ObjectLog has only 3000 entries (created by me) - therefore not
> really much data. The database seems to be pretty empty:
>
> SystemRepository fileSize      21.298.675.712
> SystemRepository freeSpace     20.171.669.504
>
> We are producing lots of HTTP requests and lots of transactions with our
> test tools.
>
> I've not investigated much in this area - and I start the stone via
> normal startStone script - nothing else.
>
> Any idea ?
>
>
> Marten
>
> --
> Marten Feldtmann
> _______________________________________________
> Glass mailing list
> [hidden email]
> http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass



--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: My extent0.dbf also grows ...

GLASS mailing list

On Aug 24, 2015, at 7:41 AM, Mariano Martinez Peck <[hidden email]> wrote:



On Sun, Aug 23, 2015 at 3:49 PM, James Foster via Glass <[hidden email]> wrote:
Hi Marten,

The best way of analyzing these problems is by looking at logs and statistics. Are you running statmonitor? If not, then you should! If so, then we can do some analysis. Absent statistics (and even with them), we can often figure out things based on a description of the symptoms.

The extent should not grow unless there is no space in the extent. So, even though now you see a lot of free space, at the time of the growth there was no space available. While this could be due to application data (including the ObjectLog), those would tend to remain and you would not have free space now.

The most common cause of unexpected repository growth that is resolved when you restart the database is a “commit record backlog” that comes from having a logged-in session that does not commit while other sessions are committing.

While you are running in Jade, go to the ‘All Sessions’ tab on the Transcript, click Update, and then look for a session that has an old view age. Depending on your version, there might also be something in the ‘Backlog’ column that shows the commit record backlog. If that is more than a dozen or so, you should track down the oldest session and have it do an abort or commit.


James, cannot it be the same gem of Jade itself (I have never used Jade so I don't know)? You know, like when using GemTools and you do have dirty objects and do not commit, you forget about it and you let it open (happened to me many times).  

Any gem (Jade or otherwise) can hold a commit record and thus create a backlog. However, the dirty objects in the gem would not contribute significantly to repository size since GEM_TEMPOBJ_CACHE_SIZE is typically much smaller than available disk space.


 
James

P.S. Note also that as a paying customer (thank you!) you may file a help request with tech support (though I’m happy to offer comments here as well).


> On Aug 22, 2015, at 11:51 PM, [hidden email] via Glass <[hidden email]> wrote:
>
> Hey,
>
> I've also problems with the size of my extent0.dbf. Within several hours
> (running our tests) the extents grows to more than 20 GB in size.
>
> I'm not using Seaside, but just Zinc as REST server. The tests are doing
> no big real data creation, but more or less only movements of objects
> around (the database world).
>
> The ObjectLog has only 3000 entries (created by me) - therefore not
> really much data. The database seems to be pretty empty:
>
> SystemRepository fileSize      21.298.675.712
> SystemRepository freeSpace     20.171.669.504
>
> We are producing lots of HTTP requests and lots of transactions with our
> test tools.
>
> I've not investigated much in this area - and I start the stone via
> normal startStone script - nothing else.
>
> Any idea ?
>
>
> Marten
>
> --
> Marten Feldtmann
> _______________________________________________
> Glass mailing list
> [hidden email]
> http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass



--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: My extent0.dbf also grows ...

GLASS mailing list
But Jade - when doing nothing - produces backlogs ... even when updating
the session infos ... I had to manually do abort or commit to put the
backlogs to 0.

Now I would like to view the bahviour of the Zinc server (without having
to answer a http request over a long time).

Pretty interesting stuff (also the Rc* class difficulties I was also not
aware of).

Marten


--
Marten Feldtmann
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass