Is it possible to disable tranlogs for some gems?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Is it possible to disable tranlogs for some gems?

GLASS mailing list
Hi guys,

I almost sure the answer is no. But I would like to comment my scenario since there could be other workarounds. I have some daily cron jobs that run at night that do quite large commits. Basically, they read huge files and do a kind of bulk load into GemStone. 

Once of the things I notice is that for every day, I have like 4 log files of 1GB each. This makes much more space usage on disk (which in my case is importante) and also, I can imagine there is quite a performance downgrade while having to write logs. 

So since this jobs run at night, with little or none user connect, I am fine with not writing logs while these jobs run. "I am fine" in the sense that if the system crashes while doing so, I am ok to recover from previous checkpoint and lost a few transactions if any. 

However, it seems it is not possible to disable tranlogs for particular gems, right? In addition, I did not found an easy way to directly enable/disable logs of the stone. To disable, the admin guide says I must run a full backup, then change file, then stop stone, start again etc...So it seems quite complicated...

Is there anything that could help me here? 

I am using GemStone 3.1.0.6

Thanks in advance, 

--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to disable tranlogs for some gems?

GLASS mailing list
OK the admin guide proposes to like:

STN_TRAN_LOG_DIRECTORIES = /dev/null, /dev/null; 
STN_TRAN_FULL_LOGGING = TRUE;

But that still implies files modification and stone stop/start right? At least I do not need to make a full backup? mmm sounds like....they forgot to say that there? 

 

On Mon, Aug 31, 2015 at 12:04 PM, Mariano Martinez Peck <[hidden email]> wrote:
Hi guys,

I almost sure the answer is no. But I would like to comment my scenario since there could be other workarounds. I have some daily cron jobs that run at night that do quite large commits. Basically, they read huge files and do a kind of bulk load into GemStone. 

Once of the things I notice is that for every day, I have like 4 log files of 1GB each. This makes much more space usage on disk (which in my case is importante) and also, I can imagine there is quite a performance downgrade while having to write logs. 

So since this jobs run at night, with little or none user connect, I am fine with not writing logs while these jobs run. "I am fine" in the sense that if the system crashes while doing so, I am ok to recover from previous checkpoint and lost a few transactions if any. 

However, it seems it is not possible to disable tranlogs for particular gems, right? In addition, I did not found an easy way to directly enable/disable logs of the stone. To disable, the admin guide says I must run a full backup, then change file, then stop stone, start again etc...So it seems quite complicated...

Is there anything that could help me here? 

I am using GemStone 3.1.0.6

Thanks in advance, 

--



--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to disable tranlogs for some gems?

GLASS mailing list
Yeah, you have to stop the stone, change configs, start stone, do bulk load, stop stone, reset configs and the restart stones ... should disable customer access during that time as well ... there is a not-tranlogged data feature but that is a case where persistent data is not tranlogged and is not recoverable on a crash, but in this case you lose the ability to recover the persistent state altogether on a crash ... which is probably not appropriate in your case .... I am hoping to get the not-tranlogged feature hooked up for Seaside3.2 and GemStone3.3, but that is a pipe dream at this point:)

Dale

On 8/31/15 8:13 AM, Mariano Martinez Peck via Glass wrote:
OK the admin guide proposes to like:

STN_TRAN_LOG_DIRECTORIES = /dev/null, /dev/null; 
STN_TRAN_FULL_LOGGING = TRUE;

But that still implies files modification and stone stop/start right? At least I do not need to make a full backup? mmm sounds like....they forgot to say that there? 

 

On Mon, Aug 31, 2015 at 12:04 PM, Mariano Martinez Peck <[hidden email]> wrote:
Hi guys,

I almost sure the answer is no. But I would like to comment my scenario since there could be other workarounds. I have some daily cron jobs that run at night that do quite large commits. Basically, they read huge files and do a kind of bulk load into GemStone. 

Once of the things I notice is that for every day, I have like 4 log files of 1GB each. This makes much more space usage on disk (which in my case is importante) and also, I can imagine there is quite a performance downgrade while having to write logs. 

So since this jobs run at night, with little or none user connect, I am fine with not writing logs while these jobs run. "I am fine" in the sense that if the system crashes while doing so, I am ok to recover from previous checkpoint and lost a few transactions if any. 

However, it seems it is not possible to disable tranlogs for particular gems, right? In addition, I did not found an easy way to directly enable/disable logs of the stone. To disable, the admin guide says I must run a full backup, then change file, then stop stone, start again etc...So it seems quite complicated...

Is there anything that could help me here? 

I am using GemStone 3.1.0.6

Thanks in advance, 

--



--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to disable tranlogs for some gems?

GLASS mailing list
For Seaside it would save on tranlog state if the seaside session state were not tranlogged and and you may be able to afford to lose the seaside session state on a system recovery after a crashed stone ...

Dale

On 8/31/15 8:32 AM, Dale Henrichs wrote:
Yeah, you have to stop the stone, change configs, start stone, do bulk load, stop stone, reset configs and the restart stones ... should disable customer access during that time as well ... there is a not-tranlogged data feature but that is a case where persistent data is not tranlogged and is not recoverable on a crash, but in this case you lose the ability to recover the persistent state altogether on a crash ... which is probably not appropriate in your case .... I am hoping to get the not-tranlogged feature hooked up for Seaside3.2 and GemStone3.3, but that is a pipe dream at this point:)

Dale

On 8/31/15 8:13 AM, Mariano Martinez Peck via Glass wrote:
OK the admin guide proposes to like:

STN_TRAN_LOG_DIRECTORIES = /dev/null, /dev/null; 
STN_TRAN_FULL_LOGGING = TRUE;

But that still implies files modification and stone stop/start right? At least I do not need to make a full backup? mmm sounds like....they forgot to say that there? 

 

On Mon, Aug 31, 2015 at 12:04 PM, Mariano Martinez Peck <[hidden email]> wrote:
Hi guys,

I almost sure the answer is no. But I would like to comment my scenario since there could be other workarounds. I have some daily cron jobs that run at night that do quite large commits. Basically, they read huge files and do a kind of bulk load into GemStone. 

Once of the things I notice is that for every day, I have like 4 log files of 1GB each. This makes much more space usage on disk (which in my case is importante) and also, I can imagine there is quite a performance downgrade while having to write logs. 

So since this jobs run at night, with little or none user connect, I am fine with not writing logs while these jobs run. "I am fine" in the sense that if the system crashes while doing so, I am ok to recover from previous checkpoint and lost a few transactions if any. 

However, it seems it is not possible to disable tranlogs for particular gems, right? In addition, I did not found an easy way to directly enable/disable logs of the stone. To disable, the admin guide says I must run a full backup, then change file, then stop stone, start again etc...So it seems quite complicated...

Is there anything that could help me here? 

I am using GemStone 3.1.0.6

Thanks in advance, 

--



--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass



_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to disable tranlogs for some gems?

GLASS mailing list
In reply to this post by GLASS mailing list

On Mon, Aug 31, 2015 at 12:32 PM, Dale Henrichs via Glass <[hidden email]> wrote:
Yeah, you have to stop the stone, change configs, start stone, do bulk load, stop stone, reset configs and the restart stones ... should disable customer access during that time as well ...

Yeah, indeed. Seems messy.... Probably not worth. 
 
there is a not-tranlogged data feature but that is a case where persistent data is not tranlogged and is not recoverable on a crash, but in this case you lose the ability to recover the persistent state altogether on a crash ... which is probably not appropriate in your case ....

Well, I think this IS my case. I mean, I do not care if a crash happens while I am not tran-logging the persistent objects from these bulk processes since these can be re-executed without problem. I mean these processes take for example huge csv files, parse them and persist. So if there is a system crash and I lost the data I persisted in those jobs (I do not loose data I persisted via seaside gems right?), then no problem, I can easily run the jobs again once the stone is restored :)
 
I am hoping to get the not-tranlogged feature hooked up for Seaside3.2 and GemStone3.3, but that is a pipe dream at this point:)


hehehhe I understand. 

Haven't you ever have the need of a not-tranlogged feature at a gem level? Say...everything from this gem, do not need to be logged? I know... I can imagine that at the repository/stone level and tranlogs level you know nothing about the gems right?

Anyway..thanks for the answer!
 
Dale


On 8/31/15 8:13 AM, Mariano Martinez Peck via Glass wrote:
OK the admin guide proposes to like:

STN_TRAN_LOG_DIRECTORIES = /dev/null, /dev/null; 
STN_TRAN_FULL_LOGGING = TRUE;

But that still implies files modification and stone stop/start right? At least I do not need to make a full backup? mmm sounds like....they forgot to say that there? 

 

On Mon, Aug 31, 2015 at 12:04 PM, Mariano Martinez Peck <[hidden email][hidden email]> wrote:
Hi guys,

I almost sure the answer is no. But I would like to comment my scenario since there could be other workarounds. I have some daily cron jobs that run at night that do quite large commits. Basically, they read huge files and do a kind of bulk load into GemStone. 

Once of the things I notice is that for every day, I have like 4 log files of 1GB each. This makes much more space usage on disk (which in my case is importante) and also, I can imagine there is quite a performance downgrade while having to write logs. 

So since this jobs run at night, with little or none user connect, I am fine with not writing logs while these jobs run. "I am fine" in the sense that if the system crashes while doing so, I am ok to recover from previous checkpoint and lost a few transactions if any. 

However, it seems it is not possible to disable tranlogs for particular gems, right? In addition, I did not found an easy way to directly enable/disable logs of the stone. To disable, the admin guide says I must run a full backup, then change file, then stop stone, start again etc...So it seems quite complicated...

Is there anything that could help me here? 

I am using GemStone 3.1.0.6

Thanks in advance, 

--



--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass




--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to disable tranlogs for some gems?

GLASS mailing list
In reply to this post by GLASS mailing list


On Mon, Aug 31, 2015 at 12:32 PM, Dale Henrichs via Glass <[hidden email]> wrote:
Yeah, you have to stop the stone, change configs, start stone, do bulk load, stop stone, reset configs and the restart stones ... should disable customer access during that time as well ... there is a not-tranlogged data feature but that is a case where persistent data is not tranlogged and is not recoverable on a crash, but in this case you lose the ability to recover the persistent state altogether on a crash ... which is probably not appropriate in your case .... I am hoping to get the not-tranlogged feature hooked up for Seaside3.2 and GemStone3.3, but that is a pipe dream at this point:)


Hi Dale,

What is the status of this not-tranlogged data? My question is because I have some cron jobs that run at night, some processing text files and filling some data. These processes are generating 13GB of tranglos every day.. 

Thanks!
 

Dale


On 8/31/15 8:13 AM, Mariano Martinez Peck via Glass wrote:
OK the admin guide proposes to like:

STN_TRAN_LOG_DIRECTORIES = /dev/null, /dev/null; 
STN_TRAN_FULL_LOGGING = TRUE;

But that still implies files modification and stone stop/start right? At least I do not need to make a full backup? mmm sounds like....they forgot to say that there? 

 

On Mon, Aug 31, 2015 at 12:04 PM, Mariano Martinez Peck <[hidden email][hidden email]> wrote:
Hi guys,

I almost sure the answer is no. But I would like to comment my scenario since there could be other workarounds. I have some daily cron jobs that run at night that do quite large commits. Basically, they read huge files and do a kind of bulk load into GemStone. 

Once of the things I notice is that for every day, I have like 4 log files of 1GB each. This makes much more space usage on disk (which in my case is importante) and also, I can imagine there is quite a performance downgrade while having to write logs. 

So since this jobs run at night, with little or none user connect, I am fine with not writing logs while these jobs run. "I am fine" in the sense that if the system crashes while doing so, I am ok to recover from previous checkpoint and lost a few transactions if any. 

However, it seems it is not possible to disable tranlogs for particular gems, right? In addition, I did not found an easy way to directly enable/disable logs of the stone. To disable, the admin guide says I must run a full backup, then change file, then stop stone, start again etc...So it seems quite complicated...

Is there anything that could help me here? 

I am using GemStone 3.1.0.6

Thanks in advance, 

--



--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass




--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to disable tranlogs for some gems?

GLASS mailing list



On 02/09/2017 04:27 AM, Mariano Martinez Peck wrote:


On Mon, Aug 31, 2015 at 12:32 PM, Dale Henrichs via Glass <[hidden email]> wrote:
Yeah, you have to stop the stone, change configs, start stone, do bulk load, stop stone, reset configs and the restart stones ... should disable customer access during that time as well ... there is a not-tranlogged data feature but that is a case where persistent data is not tranlogged and is not recoverable on a crash, but in this case you lose the ability to recover the persistent state altogether on a crash ... which is probably not appropriate in your case .... I am hoping to get the not-tranlogged feature hooked up for Seaside3.2 and GemStone3.3, but that is a pipe dream at this point:)


Hi Dale,

What is the status of this not-tranlogged data? My question is because I have some cron jobs that run at night, some processing text files and filling some data. These processes are generating 13GB of tranglos every day.. 

Thanks!
 


I haven't done anymore work on not-tranlogged data since that message ... Whether or not not-tranlogged data can reduce the size of your tranlogs depends upon the nature of your cron jobs ...

If you are using Seaside for the batch jobs and the bulk of the data dumped into the tranlogs is seaside session state, then the work that I've done previously might be able to reduce the load.

If you are not using Seaside, you still might be able to reduce the size of your tranlogs with not-tranlogged _if_ you happen to be persisting a lot of intermediate results that do not survive in the final result (large strings from files for example or large XML data structures, or ???)... If the stone crashes while doing batch processing and your strategy is to rerun the batch jobs after a crash, then it might make sense to use not-tranlogged data for your intermediate results ...

It is relatively straightforward to start using not-tranlogged data, you basically root the objects in NotTranloggedGlobals and follow the rules: not-tranlogged objects can reference tranlogged objects, but tranlogged objects cannot reference not-tranlogged objects. If you have a high volume of intermediate objects that do not survive after the batch job then you should see a reduction in tranlog size ...

Dale



_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to disable tranlogs for some gems?

GLASS mailing list

As a further thought ... if you are thinking of doing something with not-tranlogged data you should know that there are a couple of utility methods that you may want to use: Object>>beTranlogged and Object>>isTranlogged ... 

Dale


On 02/09/2017 08:21 AM, Dale Henrichs wrote:



On 02/09/2017 04:27 AM, Mariano Martinez Peck wrote:


On Mon, Aug 31, 2015 at 12:32 PM, Dale Henrichs via Glass <[hidden email]> wrote:
Yeah, you have to stop the stone, change configs, start stone, do bulk load, stop stone, reset configs and the restart stones ... should disable customer access during that time as well ... there is a not-tranlogged data feature but that is a case where persistent data is not tranlogged and is not recoverable on a crash, but in this case you lose the ability to recover the persistent state altogether on a crash ... which is probably not appropriate in your case .... I am hoping to get the not-tranlogged feature hooked up for Seaside3.2 and GemStone3.3, but that is a pipe dream at this point:)


Hi Dale,

What is the status of this not-tranlogged data? My question is because I have some cron jobs that run at night, some processing text files and filling some data. These processes are generating 13GB of tranglos every day.. 

Thanks!
 


I haven't done anymore work on not-tranlogged data since that message ... Whether or not not-tranlogged data can reduce the size of your tranlogs depends upon the nature of your cron jobs ...

If you are using Seaside for the batch jobs and the bulk of the data dumped into the tranlogs is seaside session state, then the work that I've done previously might be able to reduce the load.

If you are not using Seaside, you still might be able to reduce the size of your tranlogs with not-tranlogged _if_ you happen to be persisting a lot of intermediate results that do not survive in the final result (large strings from files for example or large XML data structures, or ???)... If the stone crashes while doing batch processing and your strategy is to rerun the batch jobs after a crash, then it might make sense to use not-tranlogged data for your intermediate results ...

It is relatively straightforward to start using not-tranlogged data, you basically root the objects in NotTranloggedGlobals and follow the rules: not-tranlogged objects can reference tranlogged objects, but tranlogged objects cannot reference not-tranlogged objects. If you have a high volume of intermediate objects that do not survive after the batch job then you should see a reduction in tranlog size ...

Dale




_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to disable tranlogs for some gems?

Paul DeBruicker
In reply to this post by GLASS mailing list
As part of your bulk loading process can you add a step/check to clean up the unneeded tranlogs?

e.g. use

SystemRepository oldestLogFileIdForRecovery


To determine which can be discarded after each CSV file is loaded and then delete the outdated ones.  


Or tidy up at the end?  Other than spending then discarding the effort of making the tranlogs in the first place are there undesirable consequences to that?



GLASS mailing list wrote
On Mon, Aug 31, 2015 at 12:32 PM, Dale Henrichs via Glass <
[hidden email]> wrote:

> Yeah, you have to stop the stone, change configs, start stone, do bulk
> load, stop stone, reset configs and the restart stones ... should disable
> customer access during that time as well ... there is a not-tranlogged data
> feature but that is a case where persistent data is not tranlogged and is
> not recoverable on a crash, but in this case you lose the ability to
> recover the persistent state altogether on a crash ... which is probably
> not appropriate in your case .... I am hoping to get the not-tranlogged
> feature hooked up for Seaside3.2 and GemStone3.3, but that is a pipe dream
> at this point:)
>


Hi Dale,

What is the status of this not-tranlogged data? My question is because I
have some cron jobs that run at night, some processing text files and
filling some data. These processes are generating 13GB of tranglos every
day..

Thanks!


>
> Dale
>
>
> On 8/31/15 8:13 AM, Mariano Martinez Peck via Glass wrote:
>
> OK the admin guide proposes to like:
>
> STN_TRAN_LOG_DIRECTORIES = /dev/null, /dev/null;
> STN_TRAN_FULL_LOGGING = TRUE;
>
> But that still implies files modification and stone stop/start right? At
> least I do not need to make a full backup? mmm sounds like....they forgot
> to say that there?
>
>
>
> On Mon, Aug 31, 2015 at 12:04 PM, Mariano Martinez Peck <
> <[hidden email]>[hidden email]> wrote:
>
>> Hi guys,
>>
>> I almost sure the answer is no. But I would like to comment my scenario
>> since there could be other workarounds. I have some daily cron jobs that
>> run at night that do quite large commits. Basically, they read huge files
>> and do a kind of bulk load into GemStone.
>>
>> Once of the things I notice is that for every day, I have like 4 log
>> files of 1GB each. This makes much more space usage on disk (which in my
>> case is importante) and also, I can imagine there is quite a performance
>> downgrade while having to write logs.
>>
>> So since this jobs run at night, with little or none user connect, I am
>> fine with not writing logs while these jobs run. "I am fine" in the sense
>> that if the system crashes while doing so, I am ok to recover from previous
>> checkpoint and lost a few transactions if any.
>>
>> However, it seems it is not possible to disable tranlogs for particular
>> gems, right? In addition, I did not found an easy way to directly
>> enable/disable logs of the stone. To disable, the admin guide says I must
>> run a full backup, then change file, then stop stone, start again etc...So
>> it seems quite complicated...
>>
>> Is there anything that could help me here?
>>
>> I am using GemStone 3.1.0.6
>>
>> Thanks in advance,
>>
>> --
>> Mariano
>> http://marianopeck.wordpress.com
>>
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
>
> _______________________________________________
> Glass mailing [hidden email]://lists.gemtalksystems.com/mailman/listinfo/glass
>
>
>
> _______________________________________________
> Glass mailing list
> [hidden email]
> http://lists.gemtalksystems.com/mailman/listinfo/glass
>
>


--
Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to disable tranlogs for some gems?

GLASS mailing list


On Fri, Feb 10, 2017 at 2:26 AM, Paul DeBruicker via Glass <[hidden email]> wrote:
As part of your bulk loading process can you add a step/check to clean up the
unneeded tranlogs?

e.g. use

SystemRepository oldestLogFileIdForRecovery


To determine which can be discarded after each CSV file is loaded and then
delete the outdated ones.



I thought about that too. But, in my case, I have a N number of backups of the stone, and I keep all necessary tranlogs for the oldest backup (via LAST_NEEDED_TRANLOG=`$GEMSTONE/bin/copydbf -i $file 2>&1 | grep tranlog | cut -d ' ' -f 13` ) so that I am sure I can reapply logs to any of the N backups.  I guess this has a price. 

To do what you say, I should always remove all logs up to #oldestLogFileIdForRecovery rather than oldest backup. Right?

 

Or tidy up at the end?  Other than spending then discarding the effort of
making the tranlogs in the first place are there undesirable consequences to
that?


The fact is that I hate spending GB on tranlogs (and probably writing the tranlogs impact performance of these bulk load cron jobs) which are mostly generated from data which I don't care to recover. If the stone would crash, I would simply re-run the background jobs and load the data again. 


 



GLASS mailing list wrote
> On Mon, Aug 31, 2015 at 12:32 PM, Dale Henrichs via Glass <

> glass@.gemtalksystems

>> wrote:
>
>> Yeah, you have to stop the stone, change configs, start stone, do bulk
>> load, stop stone, reset configs and the restart stones ... should disable
>> customer access during that time as well ... there is a not-tranlogged
>> data
>> feature but that is a case where persistent data is not tranlogged and is
>> not recoverable on a crash, but in this case you lose the ability to
>> recover the persistent state altogether on a crash ... which is probably
>> not appropriate in your case .... I am hoping to get the not-tranlogged
>> feature hooked up for Seaside3.2 and GemStone3.3, but that is a pipe
>> dream
>> at this point:)
>>
>
>
> Hi Dale,
>
> What is the status of this not-tranlogged data? My question is because I
> have some cron jobs that run at night, some processing text files and
> filling some data. These processes are generating 13GB of tranglos every
> day..
>
> Thanks!
>
>
>>
>> Dale
>>
>>
>> On 8/31/15 8:13 AM, Mariano Martinez Peck via Glass wrote:
>>
>> OK the admin guide proposes to like:
>>
>> STN_TRAN_LOG_DIRECTORIES = /dev/null, /dev/null;
>> STN_TRAN_FULL_LOGGING = TRUE;
>>
>> But that still implies files modification and stone stop/start right? At
>> least I do not need to make a full backup? mmm sounds like....they forgot
>> to say that there?
>>
>>
>>
>> On Mon, Aug 31, 2015 at 12:04 PM, Mariano Martinez Peck <
>> &lt;

> marianopeck@

> &gt;

> marianopeck@

>> wrote:
>>
>>> Hi guys,
>>>
>>> I almost sure the answer is no. But I would like to comment my scenario
>>> since there could be other workarounds. I have some daily cron jobs that
>>> run at night that do quite large commits. Basically, they read huge
>>> files
>>> and do a kind of bulk load into GemStone.
>>>
>>> Once of the things I notice is that for every day, I have like 4 log
>>> files of 1GB each. This makes much more space usage on disk (which in my
>>> case is importante) and also, I can imagine there is quite a performance
>>> downgrade while having to write logs.
>>>
>>> So since this jobs run at night, with little or none user connect, I am
>>> fine with not writing logs while these jobs run. "I am fine" in the
>>> sense
>>> that if the system crashes while doing so, I am ok to recover from
>>> previous
>>> checkpoint and lost a few transactions if any.
>>>
>>> However, it seems it is not possible to disable tranlogs for particular
>>> gems, right? In addition, I did not found an easy way to directly
>>> enable/disable logs of the stone. To disable, the admin guide says I
>>> must
>>> run a full backup, then change file, then stop stone, start again
>>> etc...So
>>> it seems quite complicated...
>>>
>>> Is there anything that could help me here?
>>>
>>> I am using GemStone 3.1.0.6
>>>
>>> Thanks in advance,
>>>
>>> --
>>> Mariano
>>> http://marianopeck.wordpress.com
>>>
>>
>>
>>
>> --
>> Mariano
>> http://marianopeck.wordpress.com
>>
>>
>> _______________________________________________
>> Glass mailing

> listGlass@.gemtalksystems

> ://lists.gemtalksystems.com/mailman/listinfo/glass
>>
>>
>>
>> _______________________________________________
>> Glass mailing list
>>

> Glass@.gemtalksystems

>> http://lists.gemtalksystems.com/mailman/listinfo/glass
>>
>>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
> _______________________________________________
> Glass mailing list

> Glass@.gemtalksystems

> http://lists.gemtalksystems.com/mailman/listinfo/glass





--
View this message in context: http://forum.world.st/Is-it-possible-to-disable-tranlogs-for-some-gems-tp4847185p4933756.html
Sent from the GLASS mailing list archive at Nabble.com.
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass



--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to disable tranlogs for some gems?

GLASS mailing list
In reply to this post by GLASS mailing list


On Thu, Feb 9, 2017 at 1:21 PM, Dale Henrichs <[hidden email]> wrote:



On 02/09/2017 04:27 AM, Mariano Martinez Peck wrote:


On Mon, Aug 31, 2015 at 12:32 PM, Dale Henrichs via Glass <[hidden email]> wrote:
Yeah, you have to stop the stone, change configs, start stone, do bulk load, stop stone, reset configs and the restart stones ... should disable customer access during that time as well ... there is a not-tranlogged data feature but that is a case where persistent data is not tranlogged and is not recoverable on a crash, but in this case you lose the ability to recover the persistent state altogether on a crash ... which is probably not appropriate in your case .... I am hoping to get the not-tranlogged feature hooked up for Seaside3.2 and GemStone3.3, but that is a pipe dream at this point:)


Hi Dale,

What is the status of this not-tranlogged data? My question is because I have some cron jobs that run at night, some processing text files and filling some data. These processes are generating 13GB of tranglos every day.. 

Thanks!
 


I haven't done anymore work on not-tranlogged data since that message ... Whether or not not-tranlogged data can reduce the size of your tranlogs depends upon the nature of your cron jobs ...

If you are using Seaside for the batch jobs and the bulk of the data dumped into the tranlogs is seaside session state, then the work that I've done previously might be able to reduce the load.


No, no Seaside sessions involved at all.
 

If you are not using Seaside, you still might be able to reduce the size of your tranlogs with not-tranlogged _if_ you happen to be persisting a lot of intermediate results that do not survive in the final result (large strings from files for example or large XML data structures, or ???)... If the stone crashes while doing batch processing and your strategy is to rerun the batch jobs after a crash, then it might make sense to use not-tranlogged data for your intermediate results ...

Yes, exactly, if the stone crashes, I don't care to recover these data. I would simply re-run the job.
 

It is relatively straightforward to start using not-tranlogged data, you basically root the objects in NotTranloggedGlobals and follow the rules: not-tranlogged objects can reference tranlogged objects, but tranlogged objects cannot reference not-tranlogged objects. If you have a high volume of intermediate objects that do not survive after the batch job then you should see a reduction in tranlog size ...



OK, thanks for the explanation. I will see if I can play a bit with this.



--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass