Is it possible to disable tranlogs for some gems?

OK the admin guide proposes to like:

STN_TRAN_LOG_DIRECTORIES = /dev/null, /dev/null; 

STN_TRAN_FULL_LOGGING = TRUE;

But that still implies files modification and stone stop/start right? At least I do not need to make a full backup? mmm sounds like....they forgot to say that there? 

 

On Mon, Aug 31, 2015 at 12:04 PM, Mariano Martinez Peck <[hidden email]> wrote:

Hi guys,

I almost sure the answer is no. But I would like to comment my scenario since there could be other workarounds. I have some daily cron jobs that run at night that do quite large commits. Basically, they read huge files and do a kind of bulk load into GemStone. 

Once of the things I notice is that for every day, I have like 4 log files of 1GB each. This makes much more space usage on disk (which in my case is importante) and also, I can imagine there is quite a performance downgrade while having to write logs. 

So since this jobs run at night, with little or none user connect, I am fine with not writing logs while these jobs run. "I am fine" in the sense that if the system crashes while doing so, I am ok to recover from previous checkpoint and lost a few transactions if any. 

However, it seems it is not possible to disable tranlogs for particular gems, right? In addition, I did not found an easy way to directly enable/disable logs of the stone. To disable, the admin guide says I must run a full backup, then change file, then stop stone, start again etc...So it seems quite complicated...

Is there anything that could help me here? 

I am using GemStone 3.1.0.6

Thanks in advance, 

--

Mariano
http://marianopeck.wordpress.com

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Yeah, you have to stop the stone, change configs, start stone, do bulk load, stop stone, reset configs and the restart stones ... should disable customer access during that time as well ... there is a not-tranlogged data feature but that is a case where persistent data is not tranlogged and is not recoverable on a crash, but in this case you lose the ability to recover the persistent state altogether on a crash ... which is probably not appropriate in your case .... I am hoping to get the not-tranlogged feature hooked up for Seaside3.2 and GemStone3.3, but that is a pipe dream at this point:)

Dale

On 8/31/15 8:13 AM, Mariano Martinez Peck via Glass wrote:

OK the admin guide proposes to like:

STN_TRAN_LOG_DIRECTORIES = /dev/null, /dev/null; 

STN_TRAN_FULL_LOGGING = TRUE;

But that still implies files modification and stone stop/start right? At least I do not need to make a full backup? mmm sounds like....they forgot to say that there? 

 

On Mon, Aug 31, 2015 at 12:04 PM, Mariano Martinez Peck <[hidden email]> wrote:

Hi guys,

I almost sure the answer is no. But I would like to comment my scenario since there could be other workarounds. I have some daily cron jobs that run at night that do quite large commits. Basically, they read huge files and do a kind of bulk load into GemStone. 

Once of the things I notice is that for every day, I have like 4 log files of 1GB each. This makes much more space usage on disk (which in my case is importante) and also, I can imagine there is quite a performance downgrade while having to write logs. 

So since this jobs run at night, with little or none user connect, I am fine with not writing logs while these jobs run. "I am fine" in the sense that if the system crashes while doing so, I am ok to recover from previous checkpoint and lost a few transactions if any. 

However, it seems it is not possible to disable tranlogs for particular gems, right? In addition, I did not found an easy way to directly enable/disable logs of the stone. To disable, the admin guide says I must run a full backup, then change file, then stop stone, start again etc...So it seems quite complicated...

Is there anything that could help me here? 

I am using GemStone 3.1.0.6

Thanks in advance, 

--

Mariano
http://marianopeck.wordpress.com

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

On Mon, Aug 31, 2015 at 12:32 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Yeah, you have to stop the stone, change configs, start stone, do bulk load, stop stone, reset configs and the restart stones ... should disable customer access during that time as well ... there is a not-tranlogged data feature but that is a case where persistent data is not tranlogged and is not recoverable on a crash, but in this case you lose the ability to recover the persistent state altogether on a crash ... which is probably not appropriate in your case .... I am hoping to get the not-tranlogged feature hooked up for Seaside3.2 and GemStone3.3, but that is a pipe dream at this point:)

Hi Dale,

What is the status of this not-tranlogged data? My question is because I have some cron jobs that run at night, some processing text files and filling some data. These processes are generating 13GB of tranglos every day.. 

Thanks!

 

As a further thought ... if you are thinking of doing something with not-tranlogged data you should know that there are a couple of utility methods that you may want to use: Object>>beTranlogged and Object>>isTranlogged ... 

Dale

On 02/09/2017 04:27 AM, Mariano Martinez Peck wrote:

On Mon, Aug 31, 2015 at 12:32 PM, Dale Henrichs via Glass <[hidden email]> wrote:

Yeah, you have to stop the stone, change configs, start stone, do bulk load, stop stone, reset configs and the restart stones ... should disable customer access during that time as well ... there is a not-tranlogged data feature but that is a case where persistent data is not tranlogged and is not recoverable on a crash, but in this case you lose the ability to recover the persistent state altogether on a crash ... which is probably not appropriate in your case .... I am hoping to get the not-tranlogged feature hooked up for Seaside3.2 and GemStone3.3, but that is a pipe dream at this point:)

Hi Dale,

What is the status of this not-tranlogged data? My question is because I have some cron jobs that run at night, some processing text files and filling some data. These processes are generating 13GB of tranglos every day.. 

Thanks!

 

I haven't done anymore work on not-tranlogged data since that message ... Whether or not not-tranlogged data can reduce the size of your tranlogs depends upon the nature of your cron jobs ...

If you are using Seaside for the batch jobs and the bulk of the data dumped into the tranlogs is seaside session state, then the work that I've done previously might be able to reduce the load.

If you are not using Seaside, you still might be able to reduce the size of your tranlogs with not-tranlogged _if_ you happen to be persisting a lot of intermediate results that do not survive in the final result (large strings from files for example or large XML data structures, or ???)... If the stone crashes while doing batch processing and your strategy is to rerun the batch jobs after a crash, then it might make sense to use not-tranlogged data for your intermediate results ...

It is relatively straightforward to start using not-tranlogged data, you basically root the objects in NotTranloggedGlobals and follow the rules: not-tranlogged objects can reference tranlogged objects, but tranlogged objects cannot reference not-tranlogged objects. If you have a high volume of intermediate objects that do not survive after the batch job then you should see a reduction in tranlog size ...

Dale

GLASS mailing list wrote

On Mon, Aug 31, 2015 at 12:32 PM, Dale Henrichs via Glass <
[hidden email]> wrote:

> Yeah, you have to stop the stone, change configs, start stone, do bulk
> load, stop stone, reset configs and the restart stones ... should disable
> customer access during that time as well ... there is a not-tranlogged data
> feature but that is a case where persistent data is not tranlogged and is
> not recoverable on a crash, but in this case you lose the ability to
> recover the persistent state altogether on a crash ... which is probably
> not appropriate in your case .... I am hoping to get the not-tranlogged
> feature hooked up for Seaside3.2 and GemStone3.3, but that is a pipe dream
> at this point:)
>


Hi Dale,

What is the status of this not-tranlogged data? My question is because I
have some cron jobs that run at night, some processing text files and
filling some data. These processes are generating 13GB of tranglos every
day..

Thanks!


>
> Dale
>
>
> On 8/31/15 8:13 AM, Mariano Martinez Peck via Glass wrote:
>
> OK the admin guide proposes to like:
>
> STN_TRAN_LOG_DIRECTORIES = /dev/null, /dev/null;
> STN_TRAN_FULL_LOGGING = TRUE;
>
> But that still implies files modification and stone stop/start right? At
> least I do not need to make a full backup? mmm sounds like....they forgot
> to say that there?
>
>
>
> On Mon, Aug 31, 2015 at 12:04 PM, Mariano Martinez Peck <
> <[hidden email]>[hidden email]> wrote:
>
>> Hi guys,
>>
>> I almost sure the answer is no. But I would like to comment my scenario
>> since there could be other workarounds. I have some daily cron jobs that
>> run at night that do quite large commits. Basically, they read huge files
>> and do a kind of bulk load into GemStone.
>>
>> Once of the things I notice is that for every day, I have like 4 log
>> files of 1GB each. This makes much more space usage on disk (which in my
>> case is importante) and also, I can imagine there is quite a performance
>> downgrade while having to write logs.
>>
>> So since this jobs run at night, with little or none user connect, I am
>> fine with not writing logs while these jobs run. "I am fine" in the sense
>> that if the system crashes while doing so, I am ok to recover from previous
>> checkpoint and lost a few transactions if any.
>>
>> However, it seems it is not possible to disable tranlogs for particular
>> gems, right? In addition, I did not found an easy way to directly
>> enable/disable logs of the stone. To disable, the admin guide says I must
>> run a full backup, then change file, then stop stone, start again etc...So
>> it seems quite complicated...
>>
>> Is there anything that could help me here?
>>
>> I am using GemStone 3.1.0.6
>>
>> Thanks in advance,
>>
>> --
>> Mariano
>> http://marianopeck.wordpress.com
>>
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
>
> _______________________________________________
> Glass mailing [hidden email]://lists.gemtalksystems.com/mailman/listinfo/glass
>
>
>
> _______________________________________________
> Glass mailing list
> [hidden email]
> http://lists.gemtalksystems.com/mailman/listinfo/glass
>
>


--
Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass