" Waiting for vote to complete..." and the famous dev gem alive

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

" Waiting for vote to complete..." and the famous dev gem alive

GLASS mailing list
Hi, 

Remember that famous problem of letting a dev gem (from GemTools, topaz, etc) connected to the stone and making all kind of problems? 

Well, I was making a backup with tODE like this:

bu backup --wait --commit betaTesting1_333_backup_20171115_033542.dbf.gz

BTW, note the --wait. 

My question is ... I thought tODE fixed this issue (by hocking in TransactionBacklog defaultHandler??) but it seems it's still happening to me. 

Of course, as soon as I closed the opened tODE image, the backup proceeded. 

Thanks in advance, 

--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: " Waiting for vote to complete..." and the famous dev gem alive

GLASS mailing list


On 11/15/17 5:23 AM, Mariano Martinez Peck via Glass wrote:

> Hi,
>
> Remember that famous problem of letting a dev gem (from GemTools,
> topaz, etc) connected to the stone and making all kind of problems?
> Well, I was making a backup with tODE like this:
>
> bu backup --wait --commit betaTesting1_333_backup_20171115_033542.dbf.gz
>
> BTW, note the --wait.
>
> My question is ... I thought tODE fixed this issue (by hocking in
> TransactionBacklog defaultHandler??) but it seems it's still happening
> to me.
Hmmmm, the tODE code itself doesn't have any references to the
TransactionBacklog class and I don't see any references
TransactionBacklog in the tODE issues (opened or closed), so I'm not
sure where you got that idea?

TransactionBacklog is involved but on the other end. The `--wait` flag
was added for those cases where you have done an MFC right before doing
a backup and you want the backup to wait for the vote following the mfc
to complete instead of failing the backup (which was the old behavior).

If w look at the implementation of
TDGemStoneTool>>buBackup:waitForVotingToComplete:safely:uncompressed:validate:
and the block that is invoked when the `--wait` flag is used

   waitForVotingToComplete
     ifTrue: [
       Transcript
         cr;
         show: 'Waiting for vote to complete...'.
       self waitForVoteStateIdle.
       self
         checkGcLock: [ :sessionHoldingGcLock |
           self
             error:
               'System has completed voting, but the gc lock is still
being held by session with id '
                 , sessionHoldingGcLock printString , '.' ].
       Transcript show: 'Vote complete.' ]

we see that #waitForVoteStateIdle is what `--wait` is waiting for... and
if you follow the chain to System class>>voteState this is the list of
vote states involved:

   0 IDLE, 1 HIDING_SYMBOLS, 2 VOTING, 3 DONE_VOTING, 4 IN_PDWSU_SWEEP,
5 PDWSU_DONE

and I think that vote states 4 and 5 prevent a backup from starting (and
were causing errors previous to the introduction of `--wait`) ... I also
think that immediately following an MFC the system goes into state 2
.... and this is the point where TransactionBacklog comes into play ...

The `--wait` code waits for voting to be completed ... as I've already
said the original purpose of the --wait flag was to allow MFC to finish
NORMALLY before attempting to run the backup, however idle (or active)
sessions will not vote until they have crossed a transaction boundary.

The TransactionBacklog notification is something that gems can implement
a handler for to avoid running into commit record backlog issues --- the
TransactionBacklog notification is signalled when a certain commit
record backlog threshold is crossed --- we do not signal the
TransactionBacklog to force a vote ... and now that I think about it,
you have likely installed TransactionBacklog in your application so that
idle gems won't cause commit record backlogs ...

This of course raises the question about whether or not the
TransactionBacklog notivication can/should be used to inform idle
sessions that they should vote --- so that the combo MFC then BACKUP can
proceed without "waiting for the cr backlog to climb" ...

Does this make sense?

I will submit an internal bug, if you think this is worth doing...

Dale

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: " Waiting for vote to complete..." and the famous dev gem alive

GLASS mailing list
If only idle tODE sessions are your concern, we could add a
TransactionBacklog handler to tODE --- which is probably not a bad idea,
but we'd still be open the issue where the TransactionBacklog only
protects agains commit record backlogs -- I think.

Dale


On 11/15/17 12:21 PM, Dale Henrichs wrote:

>
>
> On 11/15/17 5:23 AM, Mariano Martinez Peck via Glass wrote:
>> Hi,
>>
>> Remember that famous problem of letting a dev gem (from GemTools,
>> topaz, etc) connected to the stone and making all kind of problems?
>> Well, I was making a backup with tODE like this:
>>
>> bu backup --wait --commit betaTesting1_333_backup_20171115_033542.dbf.gz
>>
>> BTW, note the --wait.
>>
>> My question is ... I thought tODE fixed this issue (by hocking in
>> TransactionBacklog defaultHandler??) but it seems it's still
>> happening to me.
> Hmmmm, the tODE code itself doesn't have any references to the
> TransactionBacklog class and I don't see any references
> TransactionBacklog in the tODE issues (opened or closed), so I'm not
> sure where you got that idea?
>
> TransactionBacklog is involved but on the other end. The `--wait` flag
> was added for those cases where you have done an MFC right before
> doing a backup and you want the backup to wait for the vote following
> the mfc to complete instead of failing the backup (which was the old
> behavior).
>
> If w look at the implementation of
> TDGemStoneTool>>buBackup:waitForVotingToComplete:safely:uncompressed:validate:
> and the block that is invoked when the `--wait` flag is used
>
>   waitForVotingToComplete
>     ifTrue: [
>       Transcript
>         cr;
>         show: 'Waiting for vote to complete...'.
>       self waitForVoteStateIdle.
>       self
>         checkGcLock: [ :sessionHoldingGcLock |
>           self
>             error:
>               'System has completed voting, but the gc lock is still
> being held by session with id '
>                 , sessionHoldingGcLock printString , '.' ].
>       Transcript show: 'Vote complete.' ]
>
> we see that #waitForVoteStateIdle is what `--wait` is waiting for...
> and if you follow the chain to System class>>voteState this is the
> list of vote states involved:
>
>   0 IDLE, 1 HIDING_SYMBOLS, 2 VOTING, 3 DONE_VOTING, 4 IN_PDWSU_SWEEP,
> 5 PDWSU_DONE
>
> and I think that vote states 4 and 5 prevent a backup from starting
> (and were causing errors previous to the introduction of `--wait`) ...
> I also think that immediately following an MFC the system goes into
> state 2 .... and this is the point where TransactionBacklog comes into
> play ...
>
> The `--wait` code waits for voting to be completed ... as I've already
> said the original purpose of the --wait flag was to allow MFC to
> finish NORMALLY before attempting to run the backup, however idle (or
> active) sessions will not vote until they have crossed a transaction
> boundary.
>
> The TransactionBacklog notification is something that gems can
> implement a handler for to avoid running into commit record backlog
> issues --- the TransactionBacklog notification is signalled when a
> certain commit record backlog threshold is crossed --- we do not
> signal the TransactionBacklog to force a vote ... and now that I think
> about it, you have likely installed TransactionBacklog in your
> application so that idle gems won't cause commit record backlogs ...
>
> This of course raises the question about whether or not the
> TransactionBacklog notivication can/should be used to inform idle
> sessions that they should vote --- so that the combo MFC then BACKUP
> can proceed without "waiting for the cr backlog to climb" ...
>
> Does this make sense?
>
> I will submit an internal bug, if you think this is worth doing...
>
> Dale
>

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: " Waiting for vote to complete..." and the famous dev gem alive

GLASS mailing list
In reply to this post by GLASS mailing list


On Wed, Nov 15, 2017 at 5:21 PM, Dale Henrichs via Glass <[hidden email]> wrote:


On 11/15/17 5:23 AM, Mariano Martinez Peck via Glass wrote:
Hi,

Remember that famous problem of letting a dev gem (from GemTools, topaz, etc) connected to the stone and making all kind of problems?
Well, I was making a backup with tODE like this:

bu backup --wait --commit betaTesting1_333_backup_20171115_033542.dbf.gz

BTW, note the --wait.

My question is ... I thought tODE fixed this issue (by hocking in TransactionBacklog defaultHandler??) but it seems it's still happening to me.
Hmmmm, the tODE code itself doesn't have any references to the TransactionBacklog class and I don't see any references TransactionBacklog in the tODE issues (opened or closed), so I'm not sure where you got that idea?

From some emails (or conversations?) from the past, I thought that letting an idle tODE wouldn't harm GC as it would a GemTools client. But I might be wrong. And from where did I get the TransactionBacklog? From start seaside scripts. 



TransactionBacklog is involved but on the other end. The `--wait` flag was added for those cases where you have done an MFC right before doing a backup and you want the backup to wait for the vote following the mfc to complete instead of failing the backup (which was the old behavior).



Exactly, and I very much appreciate the `--wait` and that's why I think it makes sense and I use it. 

 
If w look at the implementation of TDGemStoneTool>>buBackup:waitForVotingToComplete:safely:uncompressed:validate: and the block that is invoked when the `--wait` flag is used

  waitForVotingToComplete
    ifTrue: [
      Transcript
        cr;
        show: 'Waiting for vote to complete...'.
      self waitForVoteStateIdle.
      self
        checkGcLock: [ :sessionHoldingGcLock |
          self
            error:
              'System has completed voting, but the gc lock is still being held by session with id '
                , sessionHoldingGcLock printString , '.' ].
      Transcript show: 'Vote complete.' ]

we see that #waitForVoteStateIdle is what `--wait` is waiting for... and if you follow the chain to System class>>voteState this is the list of vote states involved:

  0 IDLE, 1 HIDING_SYMBOLS, 2 VOTING, 3 DONE_VOTING, 4 IN_PDWSU_SWEEP, 5 PDWSU_DONE

and I think that vote states 4 and 5 prevent a backup from starting (and were causing errors previous to the introduction of `--wait`) ... I also think that immediately following an MFC the system goes into state 2 .... and this is the point where TransactionBacklog comes into play ...

The `--wait` code waits for voting to be completed ... as I've already said the original purpose of the --wait flag was to allow MFC to finish NORMALLY before attempting to run the backup, however idle (or active) sessions will not vote until they have crossed a transaction boundary.


Exactly!   So i was assumming/thinking that tODE had an out of the box solution in case it was "idle" so that it could vote.
 
The TransactionBacklog notification is something that gems can implement a handler for to avoid running into commit record backlog issues --- the TransactionBacklog notification is signalled when a certain commit record backlog threshold is crossed --- we do not signal the TransactionBacklog to force a vote ... and now that I think about it, you have likely installed TransactionBacklog in your application so that idle gems won't cause commit record backlogs ...

Exactly, that's  where I get the idea (from the start seaside gems script). I do understand that solving the "grown" of commit record backlog is not 100% the same as causing a transaction boundary in order to make the gem to vote. I do understand I would need to raise some treshold so that I get that exception signaled. 
 

This of course raises the question about whether or not the TransactionBacklog notivication can/should be used to inform idle sessions that they should vote --- so that the combo MFC then BACKUP can proceed without "waiting for the cr backlog to climb" ...


Well, ASIDE from using TransactionBacklog for voting, I think we both agree that it would be nice if tODE wouldn't generate commit record backlogs and hence a handler (similar to the one of Seaside gems) is a good idea, right?
Then, as for solving the vote itself, then we have 2 choices if I understand correct:
1) signaling TransactionBacklog too (and re-use same handler)
2) can't we use SigAbort handler here too? 

The goal is clear: we do not want to have idle dev sessions (from tODE) causing either growth of commit record backlog neither preventing the vote and hence GC.

 
Does this make sense?

I will submit an internal bug, if you think this is worth doing...


Well, I think that being able to do my above bold line is important.  


Thanks, 

--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass