write-write conflicts

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

write-write conflicts

SeanTAllen
We are getting write-write conflicts from time to time and I'm really
not sure where they are coming from.
I can see them in the topaz logs for each swazoo but it doesn't have
any details.

What is the best way to capture information about write-write
conflicts when they occur so I can diagnose the
issues?
Reply | Threaded
Open this post in threaded view
|

Re: write-write conflicts

Dale
If I'm not mistaken the info you need should already be in the logs when you get a conflict the oops of the objects involved in the conflict are listed in the log ...

If you send the snippet of the log file (+-20 lines) where the conflict is reported, I can interpret for you and tell you the next steps ...

Dale
----- "Sean Allen" <[hidden email]> wrote:

| We are getting write-write conflicts from time to time and I'm really
| not sure where they are coming from.
| I can see them in the topaz logs for each swazoo but it doesn't have
| any details.
|
| What is the best way to capture information about write-write
| conflicts when they occur so I can diagnose the
| issues?
Reply | Threaded
Open this post in threaded view
|

Re: write-write conflicts

SeanTAllen
I'll send next time we get one.

I went to go retrieve any of the usually 2 to 3 every 2 days ones and
realized that one both production machines,
I had cleared out the server logs and we hadn't had any in the last
couple days ( cant complain about that ).


On Thu, Mar 25, 2010 at 3:49 PM, Dale Henrichs
<[hidden email]> wrote:

> If I'm not mistaken the info you need should already be in the logs when you get a conflict the oops of the objects involved in the conflict are listed in the log ...
>
> If you send the snippet of the log file (+-20 lines) where the conflict is reported, I can interpret for you and tell you the next steps ...
>
> Dale
> ----- "Sean Allen" <[hidden email]> wrote:
>
> | We are getting write-write conflicts from time to time and I'm really
> | not sure where they are coming from.
> | I can see them in the topaz logs for each swazoo but it doesn't have
> | any details.
> |
> | What is the best way to capture information about write-write
> | conflicts when they occur so I can diagnose the
> | issues?
>
Reply | Threaded
Open this post in threaded view
|

Re: write-write conflicts

SeanTAllen
This is what we get in the logs:

 _____________________________________________________________________________
|             GemStone/S64 Object-Oriented Data Management System             |
|               Copyright (C) GemStone Systems, Inc. 1986-2009                |
|                            All rights reserved.                             |
|     covered by Patent Number 6,567,905 Generational Garbage Collector.      |
+-----------------------------------------------------------------------------+
|    PROGRAM: topaz, Linear GemStone Interface (Linked Session)               |
|    VERSION: 2.4.0, Mon Sep 14 11:37:14 2009                                 |
|      BUILD: gss64_2_3_x_branch-22259                                        |
|  BUILT FOR: x86-64 (Linux)                                                  |
|       MODE: 64 bit                                                          |
| RUNNING ON: 8-CPU cicada.insect-dreams.com x86_64 (Linux 2.6.26-1-amd64 #1  |
| SMP Fri Mar 13 17:46:45 UTC 2009) 8007MB                                    |
| PROCESS ID: 26098     DATE: 03/25/2010 08:02:43 PM GMT                      |
|   USER IDS: REAL=ah (1000) EFFECTIVE=ah (1000)                              |
|_____________________________________________________________________________|
[Info]: LNK client/gem GCI levels = 830/830
topaz 1> topaz 1> Warning: GemStone is clearing the previous GemStone password.
topaz 1> topaz 1> topaz 1> topaz 1> topaz 1>
-----------------------------------------------------
GemStone: Error         Nonfatal
The current GCI session is already logged in.
Error Category: 231169 [GemStone] Number: 2147 Arg Count: 0 Context : 20

Now executing the following command saved from "iferr 1":
   where
Stack is not active
topaz 1> topaz 1> [284014849 sz:3 cls: 114945 GsFile] a GsFile
  isClient        [2 sz:0 cls: 74241 SmallInteger] 0
  pathName        [284044801 sz:68 cls: 74753 String]
/opt/gemstone/product/seaside/data/WAGsSwazooAdaptor_server-9001.pid
  mode            [9207553 sz:1 cls: 74753 String] w

topaz 1> topaz 1> WAGsSwazooAdaptor Server started on port 9001
retryFailure
Read-Write Conflicts...
Write-Write Conflicts...
Write-Dependency Conflicts...
Write-ReadLock Conflicts...
Write-WriteLock Conflicts...
Rc-Write-Write Conflicts...
    184905217
Synchronized-Commit Conflicts...
----------- Commit failure - retrying LOG ENTRY: aSymbolDictionary-----------


On Fri, Mar 26, 2010 at 1:20 AM, Sean Allen <[hidden email]> wrote:

> I'll send next time we get one.
>
> I went to go retrieve any of the usually 2 to 3 every 2 days ones and
> realized that one both production machines,
> I had cleared out the server logs and we hadn't had any in the last
> couple days ( cant complain about that ).
>
>
> On Thu, Mar 25, 2010 at 3:49 PM, Dale Henrichs
> <[hidden email]> wrote:
>> If I'm not mistaken the info you need should already be in the logs when you get a conflict the oops of the objects involved in the conflict are listed in the log ...
>>
>> If you send the snippet of the log file (+-20 lines) where the conflict is reported, I can interpret for you and tell you the next steps ...
>>
>> Dale
>> ----- "Sean Allen" <[hidden email]> wrote:
>>
>> | We are getting write-write conflicts from time to time and I'm really
>> | not sure where they are coming from.
>> | I can see them in the topaz logs for each swazoo but it doesn't have
>> | any details.
>> |
>> | What is the best way to capture information about write-write
>> | conflicts when they occur so I can diagnose the
>> | issues?
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: write-write conflicts

Dale
In reply to this post by SeanTAllen
Sean,

The method System class>>_commitPrintingDiagnostics is used to do the commit, so you can look at the code there to see how the info is printed out. System class>>transactionConflicts has some more documentation as well.

here's the interpretation of the log entry:

  retryFailure
  Read-Write Conflicts...
  Write-Write Conflicts...
  Write-Dependency Conflicts...
  Write-ReadLock Conflicts...
  Write-WriteLock Conflicts...
  Rc-Write-Write Conflicts...
      184905217
  Synchronized-Commit Conflicts...

The commit failure was a retryFailure which means that an rc retry attempt failed. The number after the Rc-Write-Write Conflicts... is the oop of the object that had the conflict.

You can inspect the object by evaluating the following expression:

  Object _objectForOop: 184905217

If it isn't immediately obvious which object this is, then you can use

  SystemRepository findReferencePathToObject: (Object _objectForOop: 184905217)

to start tracking it down ...

It will end up being an RC structure that you are using or one that is in the framework ... if you don't recognize the structure or you find that it _is_ rooted in the Seaside3.0 structure, send me the output from a topaz sesson where you do something like the following:

display oops
limit oops 1000
level 4
run
SystemRepository findReferencePathToObject: (Object _objectForOop: 184905217)
%

this just might give me enough detail to recognize the structure.

Dale
----- "Sean Allen" <[hidden email]> wrote:

| This is what we get in the logs:
|
|
| _____________________________________________________________________________
| |             GemStone/S64 Object-Oriented Data Management System    
|        |
| |               Copyright (C) GemStone Systems, Inc. 1986-2009        
|        |
| |                            All rights reserved.                    
|        |
| |     covered by Patent Number 6,567,905 Generational Garbage
| Collector.      |
| +-----------------------------------------------------------------------------+
| |    PROGRAM: topaz, Linear GemStone Interface (Linked Session)      
|        |
| |    VERSION: 2.4.0, Mon Sep 14 11:37:14 2009                        
|        |
| |      BUILD: gss64_2_3_x_branch-22259                                
|        |
| |  BUILT FOR: x86-64 (Linux)                                          
|        |
| |       MODE: 64 bit                                                  
|        |
| | RUNNING ON: 8-CPU cicada.insect-dreams.com x86_64 (Linux
| 2.6.26-1-amd64 #1  |
| | SMP Fri Mar 13 17:46:45 UTC 2009) 8007MB                            
|        |
| | PROCESS ID: 26098     DATE: 03/25/2010 08:02:43 PM GMT              
|        |
| |   USER IDS: REAL=ah (1000) EFFECTIVE=ah (1000)                      
|        |
| |_____________________________________________________________________________|
| [Info]: LNK client/gem GCI levels = 830/830
| topaz 1> topaz 1> Warning: GemStone is clearing the previous GemStone
| password.
| topaz 1> topaz 1> topaz 1> topaz 1> topaz 1>
| -----------------------------------------------------
| GemStone: Error         Nonfatal
| The current GCI session is already logged in.
| Error Category: 231169 [GemStone] Number: 2147 Arg Count: 0 Context :
| 20
|
| Now executing the following command saved from "iferr 1":
|    where
| Stack is not active
| topaz 1> topaz 1> [284014849 sz:3 cls: 114945 GsFile] a GsFile
|   isClient        [2 sz:0 cls: 74241 SmallInteger] 0
|   pathName        [284044801 sz:68 cls: 74753 String]
| /opt/gemstone/product/seaside/data/WAGsSwazooAdaptor_server-9001.pid
|   mode            [9207553 sz:1 cls: 74753 String] w
|
| topaz 1> topaz 1> WAGsSwazooAdaptor Server started on port 9001
| retryFailure
| Read-Write Conflicts...
| Write-Write Conflicts...
| Write-Dependency Conflicts...
| Write-ReadLock Conflicts...
| Write-WriteLock Conflicts...
| Rc-Write-Write Conflicts...
|     184905217
| Synchronized-Commit Conflicts...
| ----------- Commit failure - retrying LOG ENTRY:
| aSymbolDictionary-----------
|
|
| On Fri, Mar 26, 2010 at 1:20 AM, Sean Allen
| <[hidden email]> wrote:
| > I'll send next time we get one.
| >
| > I went to go retrieve any of the usually 2 to 3 every 2 days ones
| and
| > realized that one both production machines,
| > I had cleared out the server logs and we hadn't had any in the last
| > couple days ( cant complain about that ).
| >
| >
| > On Thu, Mar 25, 2010 at 3:49 PM, Dale Henrichs
| > <[hidden email]> wrote:
| >> If I'm not mistaken the info you need should already be in the logs
| when you get a conflict the oops of the objects involved in the
| conflict are listed in the log ...
| >>
| >> If you send the snippet of the log file (+-20 lines) where the
| conflict is reported, I can interpret for you and tell you the next
| steps ...
| >>
| >> Dale
| >> ----- "Sean Allen" <[hidden email]> wrote:
| >>
| >> | We are getting write-write conflicts from time to time and I'm
| really
| >> | not sure where they are coming from.
| >> | I can see them in the topaz logs for each swazoo but it doesn't
| have
| >> | any details.
| >> |
| >> | What is the best way to capture information about write-write
| >> | conflicts when they occur so I can diagnose the
| >> | issues?
| >>
| >
Reply | Threaded
Open this post in threaded view
|

Re: write-write conflicts

SeanTAllen
> You can inspect the object by evaluating the following expression:
>
>  Object _objectForOop: 184905217
>
> If it isn't immediately obvious which object this is, then you can use
>
>  SystemRepository findReferencePathToObject: (Object _objectForOop: 184905217)

This gets me the same result as Object _objectForOop: 184905217

So I must be doing something not right with that.

I tried both 'print it' and 'inspect it' but its the exactly the same
as printing or insepect just the object.

I can see the error is happening with a collection that contains
instances of our seaside session classes.

aRcKeyValueDictionary( anArdisSystemWebSession->'FMLqrB7eg_8TLlVJ',
anArdisSystemWebSession->'Dblm6MPYXG9OUC8M',
anArdisSystemWebSession->'TkpKAO0UHsPeJl-G',
anArdisSystemWebSession->'C5GgrjiwMEWBHqAE',
anArdisSystemWebSession->'lAVU5hpV02ew3Su4',
aWADocumentHandler->'b9_VmTqFP7y3hTuZ',
anArdisSystemWebSession->'E2G_1TgMrOB-GqIL',
anArdisSystemWebSession->'dnH2oKqxnF2TuMqF',
anArdisSystemWebSession->'U3VZEfQ_EC3JyLU1',
anArdisSystemWebSession->'v_0xAEbMmT2tn25u',
anArdisSystemWebSession->'p-70HgPg0ztXg4Jl',
anArdisSystemWebSession->'I56If31cg_fJjz2j',
anArdisSystemWebSession->'9AY_gGjEGRgI98mK',
anArdisSystemWebSession->'eliDX1FRkv4MOgD8',
anArdisSystemWebSession->'WBMk1jc2uXsrlGAB',...

That would be a seaside level issue yes?
Reply | Threaded
Open this post in threaded view
|

Re: write-write conflicts

Dale
In reply to this post by SeanTAllen
Sean,

The oop is the RcKeyValueDictionary? ... I would guess that the commit conflict is happening when two "equal" keys are added by separate gemstone sessons. Unless #= was defined in ArdisSystemWebSession, I can't imagine how two different gemstone sessions could create and add "equal" sessions...

I am more suspicious about the document handler ... there is potential there for two seaside siessions adding the same document handler ... if not the document handler, then some other guy ...

To find out more, we need to log more information when the commit conflict occurs...You'd need to modify the GRGemStonePlatform>>seasideProcessRequestWithRetry:resultBlock: in the false block of the doCommitTransaction call, you could add the following line:

  Transcript cr; show: System redoLog printString.

You _could_ put the redoLog into the object log, but we just need a printString to see the interesting bits...

The redoLog contains all of the rc operations that were performed during the transaction, so we will probably be able to get a good idea of exactly which object is causing the problem (I hope:)...

If you don't want to put this extra logging into production, then the second best thing would be to reproduce the problem in your testing environment ...

BTW, have you happened to have introduced the "commit while processing http request" code? If you have I would be suspicious but so far we haven't found a smoking gun.

Dale

----- "Sean Allen" <[hidden email]> wrote:

| > You can inspect the object by evaluating the following expression:
| >
| >  Object _objectForOop: 184905217
| >
| > If it isn't immediately obvious which object this is, then you can
| use
| >
| >  SystemRepository findReferencePathToObject: (Object _objectForOop:
| 184905217)
|
| This gets me the same result as Object _objectForOop: 184905217
|
| So I must be doing something not right with that.
|
| I tried both 'print it' and 'inspect it' but its the exactly the same
| as printing or insepect just the object.
|
| I can see the error is happening with a collection that contains
| instances of our seaside session classes.
|
| aRcKeyValueDictionary( anArdisSystemWebSession->'FMLqrB7eg_8TLlVJ',
| anArdisSystemWebSession->'Dblm6MPYXG9OUC8M',
| anArdisSystemWebSession->'TkpKAO0UHsPeJl-G',
| anArdisSystemWebSession->'C5GgrjiwMEWBHqAE',
| anArdisSystemWebSession->'lAVU5hpV02ew3Su4',
| aWADocumentHandler->'b9_VmTqFP7y3hTuZ',
| anArdisSystemWebSession->'E2G_1TgMrOB-GqIL',
| anArdisSystemWebSession->'dnH2oKqxnF2TuMqF',
| anArdisSystemWebSession->'U3VZEfQ_EC3JyLU1',
| anArdisSystemWebSession->'v_0xAEbMmT2tn25u',
| anArdisSystemWebSession->'p-70HgPg0ztXg4Jl',
| anArdisSystemWebSession->'I56If31cg_fJjz2j',
| anArdisSystemWebSession->'9AY_gGjEGRgI98mK',
| anArdisSystemWebSession->'eliDX1FRkv4MOgD8',
| anArdisSystemWebSession->'WBMk1jc2uXsrlGAB',...
|
| That would be a seaside level issue yes?
Reply | Threaded
Open this post in threaded view
|

Re: write-write conflicts

SeanTAllen
On Fri, Mar 26, 2010 at 3:22 PM, Dale Henrichs
<[hidden email]> wrote:
> Sean,
>
> The oop is the RcKeyValueDictionary? ... I would guess that the commit conflict is happening when two "equal" keys are added by separate gemstone sessons. Unless #= was defined in ArdisSystemWebSession, I can't imagine how two different gemstone sessions could create and add "equal" sessions...

#= isnt defined.

>
> I am more suspicious about the document handler ... there is potential there for two seaside siessions adding the same document handler ... if not the document handler, then some other guy ...
>
> To find out more, we need to log more information when the commit conflict occurs...You'd need to modify the GRGemStonePlatform>>seasideProcessRequestWithRetry:resultBlock: in the false block of the doCommitTransaction call, you could add the following line:
>
>  Transcript cr; show: System redoLog printString.

Done. I added before the ^ nil.

Is that an ok place to add?

>
> BTW, have you happened to have introduced the "commit while processing http request" code? If you have I would be suspicious but so far we haven't found a smoking gun.
>

Nope.
Reply | Threaded
Open this post in threaded view
|

Re: write-write conflicts

SeanTAllen
In reply to this post by Dale
I have a feeling this isnt going to be very helpful:

[Info]: LNK client/gem GCI levels = 830/830
topaz 1> topaz 1> Warning: GemStone is clearing the previous GemStone password.
topaz 1> topaz 1> topaz 1> topaz 1> topaz 1>
-----------------------------------------------------
GemStone: Error         Nonfatal
The current GCI session is already logged in.
Error Category: 231169 [GemStone] Number: 2147 Arg Count: 0 Context : 20

Now executing the following command saved from "iferr 1":
   where
Stack is not active
topaz 1> topaz 1> [403218177 sz:3 cls: 114945 GsFile] a GsFile
  isClient        [2 sz:0 cls: 74241 SmallInteger] 0
  pathName        [403218689 sz:68 cls: 74753 String]
/opt/gemstone/product/seaside/data/WAGsSwazooAdaptor_server-9001.pid
  mode            [9207553 sz:1 cls: 74753 String] w

topaz 1> topaz 1> WAGsSwazooAdaptor Server started on port 9001
retryFailure
Read-Write Conflicts...
Write-Write Conflicts...
Write-Dependency Conflicts...
Write-ReadLock Conflicts...
Write-WriteLock Conflicts...
Rc-Write-Write Conflicts...
    184905217
Synchronized-Commit Conflicts...
----------- Commit failure - retrying LOG ENTRY: aSymbolDictionary-----------
--transcript--'Conflicts (
)
Redo (
)'


On Fri, Mar 26, 2010 at 1:24 PM, Dale Henrichs
<[hidden email]> wrote:

> Sean,
>
> The method System class>>_commitPrintingDiagnostics is used to do the commit, so you can look at the code there to see how the info is printed out. System class>>transactionConflicts has some more documentation as well.
>
> here's the interpretation of the log entry:
>
>  retryFailure
>  Read-Write Conflicts...
>  Write-Write Conflicts...
>  Write-Dependency Conflicts...
>  Write-ReadLock Conflicts...
>  Write-WriteLock Conflicts...
>  Rc-Write-Write Conflicts...
>      184905217
>  Synchronized-Commit Conflicts...
>
> The commit failure was a retryFailure which means that an rc retry attempt failed. The number after the Rc-Write-Write Conflicts... is the oop of the object that had the conflict.
>
> You can inspect the object by evaluating the following expression:
>
>  Object _objectForOop: 184905217
>
> If it isn't immediately obvious which object this is, then you can use
>
>  SystemRepository findReferencePathToObject: (Object _objectForOop: 184905217)
>
> to start tracking it down ...
>
> It will end up being an RC structure that you are using or one that is in the framework ... if you don't recognize the structure or you find that it _is_ rooted in the Seaside3.0 structure, send me the output from a topaz sesson where you do something like the following:
>
> display oops
> limit oops 1000
> level 4
> run
> SystemRepository findReferencePathToObject: (Object _objectForOop: 184905217)
> %
>
> this just might give me enough detail to recognize the structure.
>
> Dale
> ----- "Sean Allen" <[hidden email]> wrote:
>
> | This is what we get in the logs:
> |
> |
> | _____________________________________________________________________________
> | |             GemStone/S64 Object-Oriented Data Management System
> |        |
> | |               Copyright (C) GemStone Systems, Inc. 1986-2009
> |        |
> | |                            All rights reserved.
> |        |
> | |     covered by Patent Number 6,567,905 Generational Garbage
> | Collector.      |
> | +-----------------------------------------------------------------------------+
> | |    PROGRAM: topaz, Linear GemStone Interface (Linked Session)
> |        |
> | |    VERSION: 2.4.0, Mon Sep 14 11:37:14 2009
> |        |
> | |      BUILD: gss64_2_3_x_branch-22259
> |        |
> | |  BUILT FOR: x86-64 (Linux)
> |        |
> | |       MODE: 64 bit
> |        |
> | | RUNNING ON: 8-CPU cicada.insect-dreams.com x86_64 (Linux
> | 2.6.26-1-amd64 #1  |
> | | SMP Fri Mar 13 17:46:45 UTC 2009) 8007MB
> |        |
> | | PROCESS ID: 26098     DATE: 03/25/2010 08:02:43 PM GMT
> |        |
> | |   USER IDS: REAL=ah (1000) EFFECTIVE=ah (1000)
> |        |
> | |_____________________________________________________________________________|
> | [Info]: LNK client/gem GCI levels = 830/830
> | topaz 1> topaz 1> Warning: GemStone is clearing the previous GemStone
> | password.
> | topaz 1> topaz 1> topaz 1> topaz 1> topaz 1>
> | -----------------------------------------------------
> | GemStone: Error         Nonfatal
> | The current GCI session is already logged in.
> | Error Category: 231169 [GemStone] Number: 2147 Arg Count: 0 Context :
> | 20
> |
> | Now executing the following command saved from "iferr 1":
> |    where
> | Stack is not active
> | topaz 1> topaz 1> [284014849 sz:3 cls: 114945 GsFile] a GsFile
> |   isClient        [2 sz:0 cls: 74241 SmallInteger] 0
> |   pathName        [284044801 sz:68 cls: 74753 String]
> | /opt/gemstone/product/seaside/data/WAGsSwazooAdaptor_server-9001.pid
> |   mode            [9207553 sz:1 cls: 74753 String] w
> |
> | topaz 1> topaz 1> WAGsSwazooAdaptor Server started on port 9001
> | retryFailure
> | Read-Write Conflicts...
> | Write-Write Conflicts...
> | Write-Dependency Conflicts...
> | Write-ReadLock Conflicts...
> | Write-WriteLock Conflicts...
> | Rc-Write-Write Conflicts...
> |     184905217
> | Synchronized-Commit Conflicts...
> | ----------- Commit failure - retrying LOG ENTRY:
> | aSymbolDictionary-----------
> |
> |
> | On Fri, Mar 26, 2010 at 1:20 AM, Sean Allen
> | <[hidden email]> wrote:
> | > I'll send next time we get one.
> | >
> | > I went to go retrieve any of the usually 2 to 3 every 2 days ones
> | and
> | > realized that one both production machines,
> | > I had cleared out the server logs and we hadn't had any in the last
> | > couple days ( cant complain about that ).
> | >
> | >
> | > On Thu, Mar 25, 2010 at 3:49 PM, Dale Henrichs
> | > <[hidden email]> wrote:
> | >> If I'm not mistaken the info you need should already be in the logs
> | when you get a conflict the oops of the objects involved in the
> | conflict are listed in the log ...
> | >>
> | >> If you send the snippet of the log file (+-20 lines) where the
> | conflict is reported, I can interpret for you and tell you the next
> | steps ...
> | >>
> | >> Dale
> | >> ----- "Sean Allen" <[hidden email]> wrote:
> | >>
> | >> | We are getting write-write conflicts from time to time and I'm
> | really
> | >> | not sure where they are coming from.
> | >> | I can see them in the topaz logs for each swazoo but it doesn't
> | have
> | >> | any details.
> | >> |
> | >> | What is the best way to capture information about write-write
> | >> | conflicts when they occur so I can diagnose the
> | >> | issues?
> | >>
> | >
>
Reply | Threaded
Open this post in threaded view
|

Re: write-write conflicts

Dale
In reply to this post by SeanTAllen
Sean,

Every bit of information is helpful:)

This just means that I will have to start reading code and thinking up ways to reproduce the problem:)

keep your logging code in place and if you see different information showing up send mail ..

I've got some other tasks that are consuming me right now, so unless the occasional commit failures are causing problems, I'll take a look at this problem when I get back to work on Seaside 3.0.

Tracking this problem with Issue 80: http://code.google.com/p/glassdb/issues/detail?id=80

Dale
----- "Sean Allen" <[hidden email]> wrote:

| I have a feeling this isnt going to be very helpful:
|
| [Info]: LNK client/gem GCI levels = 830/830
| topaz 1> topaz 1> Warning: GemStone is clearing the previous GemStone
| password.
| topaz 1> topaz 1> topaz 1> topaz 1> topaz 1>
| -----------------------------------------------------
| GemStone: Error         Nonfatal
| The current GCI session is already logged in.
| Error Category: 231169 [GemStone] Number: 2147 Arg Count: 0 Context :
| 20
|
| Now executing the following command saved from "iferr 1":
|    where
| Stack is not active
| topaz 1> topaz 1> [403218177 sz:3 cls: 114945 GsFile] a GsFile
|   isClient        [2 sz:0 cls: 74241 SmallInteger] 0
|   pathName        [403218689 sz:68 cls: 74753 String]
| /opt/gemstone/product/seaside/data/WAGsSwazooAdaptor_server-9001.pid
|   mode            [9207553 sz:1 cls: 74753 String] w
|
| topaz 1> topaz 1> WAGsSwazooAdaptor Server started on port 9001
| retryFailure
| Read-Write Conflicts...
| Write-Write Conflicts...
| Write-Dependency Conflicts...
| Write-ReadLock Conflicts...
| Write-WriteLock Conflicts...
| Rc-Write-Write Conflicts...
|     184905217
| Synchronized-Commit Conflicts...
| ----------- Commit failure - retrying LOG ENTRY:
| aSymbolDictionary-----------
| --transcript--'Conflicts (
| )
| Redo (
| )'
|
|
| On Fri, Mar 26, 2010 at 1:24 PM, Dale Henrichs
| <[hidden email]> wrote:
| > Sean,
| >
| > The method System class>>_commitPrintingDiagnostics is used to do
| the commit, so you can look at the code there to see how the info is
| printed out. System class>>transactionConflicts has some more
| documentation as well.
| >
| > here's the interpretation of the log entry:
| >
| >  retryFailure
| >  Read-Write Conflicts...
| >  Write-Write Conflicts...
| >  Write-Dependency Conflicts...
| >  Write-ReadLock Conflicts...
| >  Write-WriteLock Conflicts...
| >  Rc-Write-Write Conflicts...
| >      184905217
| >  Synchronized-Commit Conflicts...
| >
| > The commit failure was a retryFailure which means that an rc retry
| attempt failed. The number after the Rc-Write-Write Conflicts... is
| the oop of the object that had the conflict.
| >
| > You can inspect the object by evaluating the following expression:
| >
| >  Object _objectForOop: 184905217
| >
| > If it isn't immediately obvious which object this is, then you can
| use
| >
| >  SystemRepository findReferencePathToObject: (Object _objectForOop:
| 184905217)
| >
| > to start tracking it down ...
| >
| > It will end up being an RC structure that you are using or one that
| is in the framework ... if you don't recognize the structure or you
| find that it _is_ rooted in the Seaside3.0 structure, send me the
| output from a topaz sesson where you do something like the following:
| >
| > display oops
| > limit oops 1000
| > level 4
| > run
| > SystemRepository findReferencePathToObject: (Object _objectForOop:
| 184905217)
| > %
| >
| > this just might give me enough detail to recognize the structure.
| >
| > Dale
| > ----- "Sean Allen" <[hidden email]> wrote:
| >
| > | This is what we get in the logs:
| > |
| > |
| > |
| _____________________________________________________________________________
| > | |             GemStone/S64 Object-Oriented Data Management System
| > |        |
| > | |               Copyright (C) GemStone Systems, Inc. 1986-2009
| > |        |
| > | |                            All rights reserved.
| > |        |
| > | |     covered by Patent Number 6,567,905 Generational Garbage
| > | Collector.      |
| > |
| +-----------------------------------------------------------------------------+
| > | |    PROGRAM: topaz, Linear GemStone Interface (Linked Session)
| > |        |
| > | |    VERSION: 2.4.0, Mon Sep 14 11:37:14 2009
| > |        |
| > | |      BUILD: gss64_2_3_x_branch-22259
| > |        |
| > | |  BUILT FOR: x86-64 (Linux)
| > |        |
| > | |       MODE: 64 bit
| > |        |
| > | | RUNNING ON: 8-CPU cicada.insect-dreams.com x86_64 (Linux
| > | 2.6.26-1-amd64 #1  |
| > | | SMP Fri Mar 13 17:46:45 UTC 2009) 8007MB
| > |        |
| > | | PROCESS ID: 26098     DATE: 03/25/2010 08:02:43 PM GMT
| > |        |
| > | |   USER IDS: REAL=ah (1000) EFFECTIVE=ah (1000)
| > |        |
| > |
| |_____________________________________________________________________________|
| > | [Info]: LNK client/gem GCI levels = 830/830
| > | topaz 1> topaz 1> Warning: GemStone is clearing the previous
| GemStone
| > | password.
| > | topaz 1> topaz 1> topaz 1> topaz 1> topaz 1>
| > | -----------------------------------------------------
| > | GemStone: Error         Nonfatal
| > | The current GCI session is already logged in.
| > | Error Category: 231169 [GemStone] Number: 2147 Arg Count: 0
| Context :
| > | 20
| > |
| > | Now executing the following command saved from "iferr 1":
| > |    where
| > | Stack is not active
| > | topaz 1> topaz 1> [284014849 sz:3 cls: 114945 GsFile] a GsFile
| > |   isClient        [2 sz:0 cls: 74241 SmallInteger] 0
| > |   pathName        [284044801 sz:68 cls: 74753 String]
| > |
| /opt/gemstone/product/seaside/data/WAGsSwazooAdaptor_server-9001.pid
| > |   mode            [9207553 sz:1 cls: 74753 String] w
| > |
| > | topaz 1> topaz 1> WAGsSwazooAdaptor Server started on port 9001
| > | retryFailure
| > | Read-Write Conflicts...
| > | Write-Write Conflicts...
| > | Write-Dependency Conflicts...
| > | Write-ReadLock Conflicts...
| > | Write-WriteLock Conflicts...
| > | Rc-Write-Write Conflicts...
| > |     184905217
| > | Synchronized-Commit Conflicts...
| > | ----------- Commit failure - retrying LOG ENTRY:
| > | aSymbolDictionary-----------
| > |
| > |
| > | On Fri, Mar 26, 2010 at 1:20 AM, Sean Allen
| > | <[hidden email]> wrote:
| > | > I'll send next time we get one.
| > | >
| > | > I went to go retrieve any of the usually 2 to 3 every 2 days
| ones
| > | and
| > | > realized that one both production machines,
| > | > I had cleared out the server logs and we hadn't had any in the
| last
| > | > couple days ( cant complain about that ).
| > | >
| > | >
| > | > On Thu, Mar 25, 2010 at 3:49 PM, Dale Henrichs
| > | > <[hidden email]> wrote:
| > | >> If I'm not mistaken the info you need should already be in the
| logs
| > | when you get a conflict the oops of the objects involved in the
| > | conflict are listed in the log ...
| > | >>
| > | >> If you send the snippet of the log file (+-20 lines) where the
| > | conflict is reported, I can interpret for you and tell you the
| next
| > | steps ...
| > | >>
| > | >> Dale
| > | >> ----- "Sean Allen" <[hidden email]> wrote:
| > | >>
| > | >> | We are getting write-write conflicts from time to time and
| I'm
| > | really
| > | >> | not sure where they are coming from.
| > | >> | I can see them in the topaz logs for each swazoo but it
| doesn't
| > | have
| > | >> | any details.
| > | >> |
| > | >> | What is the best way to capture information about
| write-write
| > | >> | conflicts when they occur so I can diagnose the
| > | >> | issues?
| > | >>
| > | >
| >
Reply | Threaded
Open this post in threaded view
|

Re: write-write conflicts

Dale
In reply to this post by SeanTAllen
Sean,

Try putting the System class>>redoLog print statement _before_ the log entry that logs the Commit failure ... it probably does an abort which clears the redoLog...

Dale
----- "Sean Allen" <[hidden email]> wrote:

| I have a feeling this isnt going to be very helpful:
|
| [Info]: LNK client/gem GCI levels = 830/830
| topaz 1> topaz 1> Warning: GemStone is clearing the previous GemStone
| password.
| topaz 1> topaz 1> topaz 1> topaz 1> topaz 1>
| -----------------------------------------------------
| GemStone: Error         Nonfatal
| The current GCI session is already logged in.
| Error Category: 231169 [GemStone] Number: 2147 Arg Count: 0 Context :
| 20
|
| Now executing the following command saved from "iferr 1":
|    where
| Stack is not active
| topaz 1> topaz 1> [403218177 sz:3 cls: 114945 GsFile] a GsFile
|   isClient        [2 sz:0 cls: 74241 SmallInteger] 0
|   pathName        [403218689 sz:68 cls: 74753 String]
| /opt/gemstone/product/seaside/data/WAGsSwazooAdaptor_server-9001.pid
|   mode            [9207553 sz:1 cls: 74753 String] w
|
| topaz 1> topaz 1> WAGsSwazooAdaptor Server started on port 9001
| retryFailure
| Read-Write Conflicts...
| Write-Write Conflicts...
| Write-Dependency Conflicts...
| Write-ReadLock Conflicts...
| Write-WriteLock Conflicts...
| Rc-Write-Write Conflicts...
|     184905217
| Synchronized-Commit Conflicts...
| ----------- Commit failure - retrying LOG ENTRY:
| aSymbolDictionary-----------
| --transcript--'Conflicts (
| )
| Redo (
| )'
|
|
| On Fri, Mar 26, 2010 at 1:24 PM, Dale Henrichs
| <[hidden email]> wrote:
| > Sean,
| >
| > The method System class>>_commitPrintingDiagnostics is used to do
| the commit, so you can look at the code there to see how the info is
| printed out. System class>>transactionConflicts has some more
| documentation as well.
| >
| > here's the interpretation of the log entry:
| >
| >  retryFailure
| >  Read-Write Conflicts...
| >  Write-Write Conflicts...
| >  Write-Dependency Conflicts...
| >  Write-ReadLock Conflicts...
| >  Write-WriteLock Conflicts...
| >  Rc-Write-Write Conflicts...
| >      184905217
| >  Synchronized-Commit Conflicts...
| >
| > The commit failure was a retryFailure which means that an rc retry
| attempt failed. The number after the Rc-Write-Write Conflicts... is
| the oop of the object that had the conflict.
| >
| > You can inspect the object by evaluating the following expression:
| >
| >  Object _objectForOop: 184905217
| >
| > If it isn't immediately obvious which object this is, then you can
| use
| >
| >  SystemRepository findReferencePathToObject: (Object _objectForOop:
| 184905217)
| >
| > to start tracking it down ...
| >
| > It will end up being an RC structure that you are using or one that
| is in the framework ... if you don't recognize the structure or you
| find that it _is_ rooted in the Seaside3.0 structure, send me the
| output from a topaz sesson where you do something like the following:
| >
| > display oops
| > limit oops 1000
| > level 4
| > run
| > SystemRepository findReferencePathToObject: (Object _objectForOop:
| 184905217)
| > %
| >
| > this just might give me enough detail to recognize the structure.
| >
| > Dale
| > ----- "Sean Allen" <[hidden email]> wrote:
| >
| > | This is what we get in the logs:
| > |
| > |
| > |
| _____________________________________________________________________________
| > | |             GemStone/S64 Object-Oriented Data Management System
| > |        |
| > | |               Copyright (C) GemStone Systems, Inc. 1986-2009
| > |        |
| > | |                            All rights reserved.
| > |        |
| > | |     covered by Patent Number 6,567,905 Generational Garbage
| > | Collector.      |
| > |
| +-----------------------------------------------------------------------------+
| > | |    PROGRAM: topaz, Linear GemStone Interface (Linked Session)
| > |        |
| > | |    VERSION: 2.4.0, Mon Sep 14 11:37:14 2009
| > |        |
| > | |      BUILD: gss64_2_3_x_branch-22259
| > |        |
| > | |  BUILT FOR: x86-64 (Linux)
| > |        |
| > | |       MODE: 64 bit
| > |        |
| > | | RUNNING ON: 8-CPU cicada.insect-dreams.com x86_64 (Linux
| > | 2.6.26-1-amd64 #1  |
| > | | SMP Fri Mar 13 17:46:45 UTC 2009) 8007MB
| > |        |
| > | | PROCESS ID: 26098     DATE: 03/25/2010 08:02:43 PM GMT
| > |        |
| > | |   USER IDS: REAL=ah (1000) EFFECTIVE=ah (1000)
| > |        |
| > |
| |_____________________________________________________________________________|
| > | [Info]: LNK client/gem GCI levels = 830/830
| > | topaz 1> topaz 1> Warning: GemStone is clearing the previous
| GemStone
| > | password.
| > | topaz 1> topaz 1> topaz 1> topaz 1> topaz 1>
| > | -----------------------------------------------------
| > | GemStone: Error         Nonfatal
| > | The current GCI session is already logged in.
| > | Error Category: 231169 [GemStone] Number: 2147 Arg Count: 0
| Context :
| > | 20
| > |
| > | Now executing the following command saved from "iferr 1":
| > |    where
| > | Stack is not active
| > | topaz 1> topaz 1> [284014849 sz:3 cls: 114945 GsFile] a GsFile
| > |   isClient        [2 sz:0 cls: 74241 SmallInteger] 0
| > |   pathName        [284044801 sz:68 cls: 74753 String]
| > |
| /opt/gemstone/product/seaside/data/WAGsSwazooAdaptor_server-9001.pid
| > |   mode            [9207553 sz:1 cls: 74753 String] w
| > |
| > | topaz 1> topaz 1> WAGsSwazooAdaptor Server started on port 9001
| > | retryFailure
| > | Read-Write Conflicts...
| > | Write-Write Conflicts...
| > | Write-Dependency Conflicts...
| > | Write-ReadLock Conflicts...
| > | Write-WriteLock Conflicts...
| > | Rc-Write-Write Conflicts...
| > |     184905217
| > | Synchronized-Commit Conflicts...
| > | ----------- Commit failure - retrying LOG ENTRY:
| > | aSymbolDictionary-----------
| > |
| > |
| > | On Fri, Mar 26, 2010 at 1:20 AM, Sean Allen
| > | <[hidden email]> wrote:
| > | > I'll send next time we get one.
| > | >
| > | > I went to go retrieve any of the usually 2 to 3 every 2 days
| ones
| > | and
| > | > realized that one both production machines,
| > | > I had cleared out the server logs and we hadn't had any in the
| last
| > | > couple days ( cant complain about that ).
| > | >
| > | >
| > | > On Thu, Mar 25, 2010 at 3:49 PM, Dale Henrichs
| > | > <[hidden email]> wrote:
| > | >> If I'm not mistaken the info you need should already be in the
| logs
| > | when you get a conflict the oops of the objects involved in the
| > | conflict are listed in the log ...
| > | >>
| > | >> If you send the snippet of the log file (+-20 lines) where the
| > | conflict is reported, I can interpret for you and tell you the
| next
| > | steps ...
| > | >>
| > | >> Dale
| > | >> ----- "Sean Allen" <[hidden email]> wrote:
| > | >>
| > | >> | We are getting write-write conflicts from time to time and
| I'm
| > | really
| > | >> | not sure where they are coming from.
| > | >> | I can see them in the topaz logs for each swazoo but it
| doesn't
| > | have
| > | >> | any details.
| > | >> |
| > | >> | What is the best way to capture information about
| write-write
| > | >> | conflicts when they occur so I can diagnose the
| > | >> | issues?
| > | >>
| > | >
| >