[Glass] Further commits have been disabled for this session because: 'CorruptObj error'. This session must logout.
Locked
[Glass] Further commits have been disabled for this session because: 'CorruptObj error'. This session must logout.
 
|
Hi,
Today, we had a security audit on a Seaside 3.0.10 application running in a GS 3.1.0.5 stone with FastCGI behind nginx. I have no idea what exactly they did to obtain this, but the system went unresponsive after the following error until I restarted the gems. - a InternalError occurred (error 2261), The object with object ID 'Hannes_Alfvén' is corrupt. Reason: 'carrysize > 0 at end of utf8 decode' - a TransactionError occurred (error 2249), Further commits have been disabled for this session because: 'CorruptObj error'. This session must logout. - repeat last one The stacks from the gem log are attached. I am trying to trace what happened. Any clues? Johan ----------- Internal FASTCGI ERROR Encountered: 2014-05-26T13:37:41.31612992286682+02:00 a InternalError occurred (error 2261), The object with object ID 'Hannes_Alfvén' is corrupt. Reason: 'carrysize > 0 at end of utf8 decode' 1 GRGemStonePlatform >> logError:title:shouldCommit: (envId 0) @2 line 4 [GsNMethod 7469480705] 2 GRGemStonePlatform >> logError:title: (envId 0) @2 line 3 [GsNMethod 7469494785] 3 WAFastCGIAdaptor >> internalServerErrorMessage: (envId 0) @20 line 14 [GsNMethod 9828254465] 4 [] in GRGemStonePlatform >> seasideProcessRequest:adaptor:resultBlock: (envId 0) @4 line 26 [GsNMethod 10506396161] 5 AbstractException >> _executeHandler: (envId 0) @3 line 8 [GsNMethod 1394121473] 6 AbstractException >> _signalFromPrimitive: (envId 0) @1 line 1 [GsNMethod 1360964097] 7 String >> decodeFromUTF8 (envId 0) @1 line 1 [GsNMethod 1064104705] 8 UTF8PrimitiveEncoding >> decode: (envId 0) @2 line 4 [GsNMethod 7470277121] 9 GRUtf8GemStoneCodec >> decode: (envId 0) @3 line 3 [GsNMethod 7468617985] 10 [] in WAUrl >> decodedWith: (envId 0) @3 line 17 [GsNMethod 9230396417] 11 SequenceableCollection >> collect: (envId 0) @9 line 16 [GsNMethod 1064183041] 12 WAUrl >> decodedWith: (envId 0) @22 line 17 [GsNMethod 8789933313] 13 WAFastCGIRequestConverter >> requestUrlFor: (envId 0) @6 line 4 [GsNMethod 9828231425] 14 WAServerAdaptor >> requestFor: (envId 0) @3 line 6 [GsNMethod 8790261761] 15 WAFastCGIRequestConverter >> requestFor: (envId 0) @12 line 7 [GsNMethod 9828215809] 16 WAFastCGIAdaptor >> requestFor: (envId 0) @2 line 4 [GsNMethod 9828250881] 17 WAServerAdaptor >> contextFor: (envId 0) @2 line 5 [GsNMethod 8790264577] 18 WAServerAdaptor >> process: (envId 0) @2 line 5 [GsNMethod 8790258433] 19 [] in WAFastCGIAdaptor >> process: (envId 0) @2 line 6 [GsNMethod 8794996737] 20 [] in GRGemStonePlatform >> seasideProcessRequestWithRetry:resultBlock: (envId 0) @2 line 13 [GsNMethod 10501067521] 21 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457] 22 [] in GRGemStonePlatform >> seasideProcessRequestWithRetry:resultBlock: (envId 0) @14 line 14 [GsNMethod 10506395905] 23 ExecBlock >> ensure: (envId 0) @2 line 12 [GsNMethod 1064640769] 24 TransientRecursionLock >> critical: (envId 0) @11 line 12 [GsNMethod 6527748609] 25 GRGemStonePlatform >> seasideProcessRequestWithRetry:resultBlock: (envId 0) @3 line 5 [GsNMethod 10509761025] 26 [] in GRGemStonePlatform >> seasideProcessRequest:adaptor:resultBlock: (envId 0) @6 line 15 [GsNMethod 10506396417] 27 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457] 28 GRGemStonePlatform >> seasideProcessRequest:adaptor:resultBlock: (envId 0) @3 line 23 [GsNMethod 10509761281] 29 WAFastCGIAdaptor >> process: (envId 0) @3 line 4 [GsNMethod 9828418049] 30 [] in WAFastCGIAdaptor >> answerResponderRole: (envId 0) @2 line 4 [GsNMethod 8795113729] 31 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457] 32 WAFastCGIAdaptor >> answerResponderRole: (envId 0) @2 line 5 [GsNMethod 9828248321] 33 FSResponderRole >> answer (envId 0) @3 line 4 [GsNMethod 8854227457] 34 FSRole >> handleConnection (envId 0) @3 line 5 [GsNMethod 8854243329] 35 FSConnection >> unsafeServe (envId 0) @5 line 8 [GsNMethod 8853951745] 36 [] in FSConnection >> safeServe (envId 0) @2 line 8 [GsNMethod 9557561601] 37 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457] 38 [] in FSConnection >> safeServe (envId 0) @2 line 9 [GsNMethod 9322044673] 39 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457] 40 [] in FSConnection >> safeServe (envId 0) @2 line 12 [GsNMethod 9258731777] 41 ExecBlock >> ensure: (envId 0) @2 line 12 [GsNMethod 1064640769] 42 FSConnection >> safeServe (envId 0) @2 line 15 [GsNMethod 8853958913] 43 FSConnection >> serve (envId 0) @2 line 4 [GsNMethod 8853957889] 44 [] in FSSocketServer >> listen: (envId 0) @3 line 15 [GsNMethod 9261209601] 45 GsProcess >> _start (envId 0) @7 line 16 [GsNMethod 1403422977] 46 <Reenter marker> ----------- ----------- Internal FASTCGI LOG ENTRY: anArray----------- ----------- Internal FASTCGI ERROR Encountered: 2014-05-26T13:37:41.37823009490967+02:00 a TransactionError occurred (error 2249), Further commits have been disabled for this session because: 'CorruptObj error'. This session must logout. 1 GRGemStonePlatform >> logError:title:shouldCommit: (envId 0) @2 line 4 [GsNMethod 7469480705] 2 GRGemStonePlatform >> logError:title: (envId 0) @2 line 3 [GsNMethod 7469494785] 3 WAFastCGIAdaptor >> internalServerErrorMessage: (envId 0) @20 line 14 [GsNMethod 9828254465] 4 [] in GRGemStonePlatform >> seasideProcessRequest:adaptor:resultBlock: (envId 0) @4 line 26 [GsNMethod 10506396161] 5 AbstractException >> _executeOuterHandler: (envId 0) @3 line 7 [GsNMethod 1394117633] 6 AbstractException >> _pass:with: (envId 0) @4 line 13 [GsNMethod 1393936385] 7 AbstractException >> pass (envId 0) @2 line 14 [GsNMethod 1393935361] 8 [] in System class >> _localCommit: (envId 0) @2 line 34 [GsNMethod 5944183041] 9 AbstractException >> _executeHandler: (envId 0) @3 line 8 [GsNMethod 1394121473] 10 AbstractException >> _signalFromPrimitive: (envId 0) @1 line 1 [GsNMethod 1360964097] 11 System class >> _primitiveCommit: (envId 0) @1 line 1 [GsNMethod 1789434881] 12 System class >> __commit: (envId 0) @2 line 8 [GsNMethod 1789345025] 13 [] in System class >> _localCommit: (envId 0) @2 line 30 [GsNMethod 5944183297] 14 ExecBlock >> onException:do: (envId 0) @2 line 66 [GsNMethod 1064628225] 15 System class >> _localCommit: (envId 0) @8 line 31 [GsNMethod 1789345281] 16 TransactionBoundaryDefaultPolicy >> commit: (envId 0) @2 line 3 [GsNMethod 5986577665] 17 System class >> _commit: (envId 0) @7 line 16 [GsNMethod 1789345537] 18 System class >> commitTransaction (envId 0) @5 line 7 [GsNMethod 1789402113] 19 System class >> _commitPrintingDiagnostics (envId 0) @2 line 8 [GsNMethod 1700522241] 20 SystemCommitTransaction >> defaultAction (envId 0) @2 line 3 [GsNMethod 7468825857] 21 AbstractException >> _signalWith: (envId 0) @5 line 25 [GsNMethod 1394122241] 22 AbstractException class >> signal (envId 0) @3 line 5 [GsNMethod 1172775681] 23 GRGemStonePlatform >> doCommitTransaction (envId 0) @4 line 3 [GsNMethod 7469481473] 24 [] in GRGemStonePlatform >> seasideProcessRequestWithRetry:resultBlock: (envId 0) @47 line 54 [GsNMethod 10506395905] 25 ExecBlock >> ensure: (envId 0) @2 line 12 [GsNMethod 1064640769] 26 TransientRecursionLock >> critical: (envId 0) @11 line 12 [GsNMethod 6527748609] 27 GRGemStonePlatform >> seasideProcessRequestWithRetry:resultBlock: (envId 0) @3 line 5 [GsNMethod 10509761025] 28 [] in GRGemStonePlatform >> seasideProcessRequest:adaptor:resultBlock: (envId 0) @6 line 15 [GsNMethod 10506396417] 29 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457] 30 GRGemStonePlatform >> seasideProcessRequest:adaptor:resultBlock: (envId 0) @3 line 23 [GsNMethod 10509761281] 31 WAFastCGIAdaptor >> process: (envId 0) @3 line 4 [GsNMethod 9828418049] 32 [] in WAFastCGIAdaptor >> answerResponderRole: (envId 0) @2 line 4 [GsNMethod 8795113729] 33 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457] 34 WAFastCGIAdaptor >> answerResponderRole: (envId 0) @2 line 5 [GsNMethod 9828248321] 35 FSResponderRole >> answer (envId 0) @3 line 4 [GsNMethod 8854227457] 36 FSRole >> handleConnection (envId 0) @3 line 5 [GsNMethod 8854243329] 37 FSConnection >> unsafeServe (envId 0) @5 line 8 [GsNMethod 8853951745] 38 [] in FSConnection >> safeServe (envId 0) @2 line 8 [GsNMethod 9557561601] 39 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457] 40 [] in FSConnection >> safeServe (envId 0) @2 line 9 [GsNMethod 9322044673] 41 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457] 42 [] in FSConnection >> safeServe (envId 0) @2 line 12 [GsNMethod 9258731777] 43 ExecBlock >> ensure: (envId 0) @2 line 12 [GsNMethod 1064640769] 44 FSConnection >> safeServe (envId 0) @2 line 15 [GsNMethod 8853958913] 45 FSConnection >> serve (envId 0) @2 line 4 [GsNMethod 8853957889] 46 [] in FSSocketServer >> listen: (envId 0) @3 line 15 [GsNMethod 9261209601] 47 GsProcess >> _start (envId 0) @7 line 16 [GsNMethod 1403422977] 48 <Reenter marker> ----------- _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] Further commits have been disabled for this session because: 'CorruptObj error'. This session must logout.
|
|
Johan, I think that a corruptObj error in this case may be a bit extreme. I have asked engineering if there is a "session safe" method for finding the obviously invalid utf8 ... Today is a holiday, so I may not get a response until tomorrow. I hope that crashed vms is an acceptable outcome of the security audit? The corruptObj error is our response to the error thrown by ICU ... the commit prohibiting error normally is thrown when we think that "memory stomping" has occurred and we want to avoid persisting potentially corrupt objects ... presumably the security audit folks knew that mishandling this particular ICU error condition could lead to a security breach:)
Dale On Mon, May 26, 2014 at 9:44 AM, Johan Brichau <[hidden email]> wrote: Hi, _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] Further commits have been disabled for this session because: 'CorruptObj error'. This session must logout.
|
|
Okay, I've talked to engineering and they agree that the corruptObj error in this case is too extreme. We are in the final stages of a 3.2.1 release, so it may be too late to include a fix in 3.2.1, but it will most likely be included in a 3.2.2, but we don't have a schedule for that. Until then I think that the following patch to GRGemStonePlatform>>seasideProcessRequest:adaptor:resultBlock: is the best we can do: ----- Method: GRGemStonePlatform>>seasideProcessRequest:adaptor:resultBlock: (in category '*seaside-gemstone-core') -----seasideProcessRequest: aNativeRequest adaptor: adaptor resultBlock: resultBlock
| result | [ | retryCount result count | retryCount := self retryCount. count := 1. retryCount timesRepeat: [ (result := self
seasideProcessRequestWithRetry: aNativeRequest resultBlock: resultBlock) ~~ nil ifTrue: [ ^ result ]. System _sessionCacheStatAt: 2 put: (System _sessionCacheStatAt: 2) + 1. "requests retried"
(Delay forMilliseconds: self retryDelay * count) wait. count := count * 10 ]. "exceeded retry limit" ^ adaptor
internalServerErrorMessage: 'Too many retries: ' , (retryCount + 1) printString ] on: Error do: [ :ex | self doAbortTransaction.
+ result := adaptor internalServerErrorMessage: ex description. + ex gsNumber == 2261 + ifTrue: + [ + "see discussion in: http://forum.world.st/Glass-Further-commits-have-been-disabled-for-this-session-because-CorruptObj-error-This-session-must-td4760409.html
+ " + Transcript + cr; + show: + 'Session terminating due to CorruptObj error ... session must logout'.
+ System logout ] + ex + return: nil "Do an explicit return. Because of the abort above, the default action for an exception (resume) is set. see bug39246." ].
- result := adaptor internalServerErrorMessage: ex description. "Do an explicit return. Because of the abort above, the default action for an exception (resume) is set. see bug39246."
- ex return: nil ]. ^ result If you use this patch, make sure that you have a system in place for automatically restarting gems that get fatal errors. You should be using monit or daaemontools in production since gems may crash occasionally and they need to be restarted. Other than the extra time/cpu/disk for restarting a gem, stopping/restarting gems is not a fatal problem as no persistent data is lost or corrupted ... see my post on "GLASS 101: Disposable Gems, Durable Data"[1].
I'm on the fence about including this patch in the standard release of Seaside for GemStone ... on the one hand if you get this particular error (or any other "commit prohibiting error" the gem is effectively out of service and needs to be restarted...
An alternative to logout would be to update a field in SessionTemps that indicates the the gem needs to be restarted ... that session state can be queried by a monit http request (presumably) ... so at the end of the day we should probably invent some sort of gem termination policy that can be controlled at the application level...
Dale On Mon, May 26, 2014 at 10:33 AM, Dale Henrichs <[hidden email]> wrote:
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] Further commits have been disabled for this session because: 'CorruptObj error'. This session must logout.
|
|
I forgot to mention that internally we are tracking this as bug 44291... Dale On Tue, May 27, 2014 at 12:16 PM, Dale Henrichs <[hidden email]> wrote:
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] Further commits have been disabled for this session because: 'CorruptObj error'. This session must logout.
|
|
... and opened a GemStone Seaside3.1 issue[1]... On Tue, May 27, 2014 at 12:18 PM, Dale Henrichs <[hidden email]> wrote:
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] Further commits have been disabled for this session because: 'CorruptObj error'. This session must logout.
|
|
Thanks Dale. This is really appreciated.
I'm a bit swamped until thursday but it's on my stack of things to test and I get back to that issue as well. Johan On 27 May 2014, at 21:38, Dale Henrichs <[hidden email]> wrote: > ... and opened a GemStone Seaside3.1 issue[1]... > > Dale > > [1] https://github.com/glassdb/Seaside31/issues/20 > > > On Tue, May 27, 2014 at 12:18 PM, Dale Henrichs <[hidden email]> wrote: > I forgot to mention that internally we are tracking this as bug 44291... > > Dale > > > On Tue, May 27, 2014 at 12:16 PM, Dale Henrichs <[hidden email]> wrote: > Okay, I've talked to engineering and they agree that the corruptObj error in this case is too extreme. We are in the final stages of a 3.2.1 release, so it may be too late to include a fix in 3.2.1, but it will most likely be included in a 3.2.2, but we don't have a schedule for that. > > Until then I think that the following patch to GRGemStonePlatform>>seasideProcessRequest:adaptor:resultBlock: is the best we can do: > > > ----- Method: GRGemStonePlatform>>seasideProcessRequest:adaptor:resultBlock: (in category '*seaside-gemstone-core') -----seasideProcessRequest: aNativeRequest adaptor: adaptor resultBlock: resultBlock > | result | > [ > | retryCount result count | > retryCount := self retryCount. > count := 1. > retryCount > timesRepeat: [ > (result := self > seasideProcessRequestWithRetry: aNativeRequest > resultBlock: resultBlock) ~~ nil > ifTrue: [ ^ result ]. > System _sessionCacheStatAt: 2 put: (System _sessionCacheStatAt: 2) + 1. "requests retried" > (Delay forMilliseconds: self retryDelay * count) wait. > count := count * 10 ]. "exceeded retry limit" > ^ adaptor > internalServerErrorMessage: > 'Too many retries: ' , (retryCount + 1) printString ] > on: Error > do: [ :ex | > self doAbortTransaction. > + result := adaptor internalServerErrorMessage: ex description. > + ex gsNumber == 2261 > + ifTrue: > + [ > + "see discussion in: http://forum.world.st/Glass-Further-commits-have-been-disabled-for-this-session-because-CorruptObj-error-This-session-must-td4760409.html > + " > + Transcript > + cr; > + show: > + 'Session terminating due to CorruptObj error ... session must logout'. > + System logout ] > + ex > + return: nil "Do an explicit return. Because of the abort above, the default action for an exception (resume) is set. see bug39246." ]. > - result := adaptor internalServerErrorMessage: ex description. "Do an explicit return. Because of the abort above, the default action for an exception (resume) is set. see bug39246." > - ex return: nil ]. > ^ result > > If you use this patch, make sure that you have a system in place for automatically restarting gems that get fatal errors. You should be using monit or daaemontools in production since gems may crash occasionally and they need to be restarted. Other than the extra time/cpu/disk for restarting a gem, stopping/restarting gems is not a fatal problem as no persistent data is lost or corrupted ... see my post on "GLASS 101: Disposable Gems, Durable Data"[1]. > > I'm on the fence about including this patch in the standard release of Seaside for GemStone ... on the one hand if you get this particular error (or any other "commit prohibiting error" the gem is effectively out of service and needs to be restarted... > > An alternative to logout would be to update a field in SessionTemps that indicates the the gem needs to be restarted ... that session state can be queried by a monit http request (presumably) ... so at the end of the day we should probably invent some sort of gem termination policy that can be controlled at the application level... > > Dale > > > [1] http://gemstonesoup.wordpress.com/2008/03/08/glass-101-disposable-gems-durable-data/ > > > > On Mon, May 26, 2014 at 10:33 AM, Dale Henrichs <[hidden email]> wrote: > Johan, > > I think that a corruptObj error in this case may be a bit extreme. I have asked engineering if there is a "session safe" method for finding the obviously invalid utf8 ... Today is a holiday, so I may not get a response until tomorrow. > > I hope that crashed vms is an acceptable outcome of the security audit? > > The corruptObj error is our response to the error thrown by ICU ... the commit prohibiting error normally is thrown when we think that "memory stomping" has occurred and we want to avoid persisting potentially corrupt objects ... presumably the security audit folks knew that mishandling this particular ICU error condition could lead to a security breach:) > > Dale > > > On Mon, May 26, 2014 at 9:44 AM, Johan Brichau <[hidden email]> wrote: > Hi, > > Today, we had a security audit on a Seaside 3.0.10 application running in a GS 3.1.0.5 stone with FastCGI behind nginx. > > I have no idea what exactly they did to obtain this, but the system went unresponsive after the following error until I restarted the gems. > - a InternalError occurred (error 2261), The object with object ID 'Hannes_Alfvén' is corrupt. Reason: 'carrysize > 0 at end of utf8 decode' > - a TransactionError occurred (error 2249), Further commits have been disabled for this session because: 'CorruptObj error'. This session must logout. > - repeat last one > > The stacks from the gem log are attached. > > I am trying to trace what happened. Any clues? > > Johan > > ----------- Internal FASTCGI ERROR Encountered: 2014-05-26T13:37:41.31612992286682+02:00 > a InternalError occurred (error 2261), The object with object ID 'Hannes_Alfvén' is corrupt. Reason: 'carrysize > 0 at end of utf8 decode' > 1 GRGemStonePlatform >> logError:title:shouldCommit: (envId 0) @2 line 4 [GsNMethod 7469480705] > 2 GRGemStonePlatform >> logError:title: (envId 0) @2 line 3 [GsNMethod 7469494785] > 3 WAFastCGIAdaptor >> internalServerErrorMessage: (envId 0) @20 line 14 [GsNMethod 9828254465] > 4 [] in GRGemStonePlatform >> seasideProcessRequest:adaptor:resultBlock: (envId 0) @4 line 26 [GsNMethod 10506396161] > 5 AbstractException >> _executeHandler: (envId 0) @3 line 8 [GsNMethod 1394121473] > 6 AbstractException >> _signalFromPrimitive: (envId 0) @1 line 1 [GsNMethod 1360964097] > 7 String >> decodeFromUTF8 (envId 0) @1 line 1 [GsNMethod 1064104705] > 8 UTF8PrimitiveEncoding >> decode: (envId 0) @2 line 4 [GsNMethod 7470277121] > 9 GRUtf8GemStoneCodec >> decode: (envId 0) @3 line 3 [GsNMethod 7468617985] > 10 [] in WAUrl >> decodedWith: (envId 0) @3 line 17 [GsNMethod 9230396417] > 11 SequenceableCollection >> collect: (envId 0) @9 line 16 [GsNMethod 1064183041] > 12 WAUrl >> decodedWith: (envId 0) @22 line 17 [GsNMethod 8789933313] > 13 WAFastCGIRequestConverter >> requestUrlFor: (envId 0) @6 line 4 [GsNMethod 9828231425] > 14 WAServerAdaptor >> requestFor: (envId 0) @3 line 6 [GsNMethod 8790261761] > 15 WAFastCGIRequestConverter >> requestFor: (envId 0) @12 line 7 [GsNMethod 9828215809] > 16 WAFastCGIAdaptor >> requestFor: (envId 0) @2 line 4 [GsNMethod 9828250881] > 17 WAServerAdaptor >> contextFor: (envId 0) @2 line 5 [GsNMethod 8790264577] > 18 WAServerAdaptor >> process: (envId 0) @2 line 5 [GsNMethod 8790258433] > 19 [] in WAFastCGIAdaptor >> process: (envId 0) @2 line 6 [GsNMethod 8794996737] > 20 [] in GRGemStonePlatform >> seasideProcessRequestWithRetry:resultBlock: (envId 0) @2 line 13 [GsNMethod 10501067521] > 21 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457] > 22 [] in GRGemStonePlatform >> seasideProcessRequestWithRetry:resultBlock: (envId 0) @14 line 14 [GsNMethod 10506395905] > 23 ExecBlock >> ensure: (envId 0) @2 line 12 [GsNMethod 1064640769] > 24 TransientRecursionLock >> critical: (envId 0) @11 line 12 [GsNMethod 6527748609] > 25 GRGemStonePlatform >> seasideProcessRequestWithRetry:resultBlock: (envId 0) @3 line 5 [GsNMethod 10509761025] > 26 [] in GRGemStonePlatform >> seasideProcessRequest:adaptor:resultBlock: (envId 0) @6 line 15 [GsNMethod 10506396417] > 27 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457] > 28 GRGemStonePlatform >> seasideProcessRequest:adaptor:resultBlock: (envId 0) @3 line 23 [GsNMethod 10509761281] > 29 WAFastCGIAdaptor >> process: (envId 0) @3 line 4 [GsNMethod 9828418049] > 30 [] in WAFastCGIAdaptor >> answerResponderRole: (envId 0) @2 line 4 [GsNMethod 8795113729] > 31 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457] > 32 WAFastCGIAdaptor >> answerResponderRole: (envId 0) @2 line 5 [GsNMethod 9828248321] > 33 FSResponderRole >> answer (envId 0) @3 line 4 [GsNMethod 8854227457] > 34 FSRole >> handleConnection (envId 0) @3 line 5 [GsNMethod 8854243329] > 35 FSConnection >> unsafeServe (envId 0) @5 line 8 [GsNMethod 8853951745] > 36 [] in FSConnection >> safeServe (envId 0) @2 line 8 [GsNMethod 9557561601] > 37 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457] > 38 [] in FSConnection >> safeServe (envId 0) @2 line 9 [GsNMethod 9322044673] > 39 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457] > 40 [] in FSConnection >> safeServe (envId 0) @2 line 12 [GsNMethod 9258731777] > 41 ExecBlock >> ensure: (envId 0) @2 line 12 [GsNMethod 1064640769] > 42 FSConnection >> safeServe (envId 0) @2 line 15 [GsNMethod 8853958913] > 43 FSConnection >> serve (envId 0) @2 line 4 [GsNMethod 8853957889] > 44 [] in FSSocketServer >> listen: (envId 0) @3 line 15 [GsNMethod 9261209601] > 45 GsProcess >> _start (envId 0) @7 line 16 [GsNMethod 1403422977] > 46 <Reenter marker> > ----------- > ----------- Internal FASTCGI LOG ENTRY: anArray----------- > ----------- Internal FASTCGI ERROR Encountered: 2014-05-26T13:37:41.37823009490967+02:00 > a TransactionError occurred (error 2249), Further commits have been disabled for this session because: 'CorruptObj error'. This session must logout. > 1 GRGemStonePlatform >> logError:title:shouldCommit: (envId 0) @2 line 4 [GsNMethod 7469480705] > 2 GRGemStonePlatform >> logError:title: (envId 0) @2 line 3 [GsNMethod 7469494785] > 3 WAFastCGIAdaptor >> internalServerErrorMessage: (envId 0) @20 line 14 [GsNMethod 9828254465] > 4 [] in GRGemStonePlatform >> seasideProcessRequest:adaptor:resultBlock: (envId 0) @4 line 26 [GsNMethod 10506396161] > 5 AbstractException >> _executeOuterHandler: (envId 0) @3 line 7 [GsNMethod 1394117633] > 6 AbstractException >> _pass:with: (envId 0) @4 line 13 [GsNMethod 1393936385] > 7 AbstractException >> pass (envId 0) @2 line 14 [GsNMethod 1393935361] > 8 [] in System class >> _localCommit: (envId 0) @2 line 34 [GsNMethod 5944183041] > 9 AbstractException >> _executeHandler: (envId 0) @3 line 8 [GsNMethod 1394121473] > 10 AbstractException >> _signalFromPrimitive: (envId 0) @1 line 1 [GsNMethod 1360964097] > 11 System class >> _primitiveCommit: (envId 0) @1 line 1 [GsNMethod 1789434881] > 12 System class >> __commit: (envId 0) @2 line 8 [GsNMethod 1789345025] > 13 [] in System class >> _localCommit: (envId 0) @2 line 30 [GsNMethod 5944183297] > 14 ExecBlock >> onException:do: (envId 0) @2 line 66 [GsNMethod 1064628225] > 15 System class >> _localCommit: (envId 0) @8 line 31 [GsNMethod 1789345281] > 16 TransactionBoundaryDefaultPolicy >> commit: (envId 0) @2 line 3 [GsNMethod 5986577665] > 17 System class >> _commit: (envId 0) @7 line 16 [GsNMethod 1789345537] > 18 System class >> commitTransaction (envId 0) @5 line 7 [GsNMethod 1789402113] > 19 System class >> _commitPrintingDiagnostics (envId 0) @2 line 8 [GsNMethod 1700522241] > 20 SystemCommitTransaction >> defaultAction (envId 0) @2 line 3 [GsNMethod 7468825857] > 21 AbstractException >> _signalWith: (envId 0) @5 line 25 [GsNMethod 1394122241] > 22 AbstractException class >> signal (envId 0) @3 line 5 [GsNMethod 1172775681] > 23 GRGemStonePlatform >> doCommitTransaction (envId 0) @4 line 3 [GsNMethod 7469481473] > 24 [] in GRGemStonePlatform >> seasideProcessRequestWithRetry:resultBlock: (envId 0) @47 line 54 [GsNMethod 10506395905] > 25 ExecBlock >> ensure: (envId 0) @2 line 12 [GsNMethod 1064640769] > 26 TransientRecursionLock >> critical: (envId 0) @11 line 12 [GsNMethod 6527748609] > 27 GRGemStonePlatform >> seasideProcessRequestWithRetry:resultBlock: (envId 0) @3 line 5 [GsNMethod 10509761025] > 28 [] in GRGemStonePlatform >> seasideProcessRequest:adaptor:resultBlock: (envId 0) @6 line 15 [GsNMethod 10506396417] > 29 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457] > 30 GRGemStonePlatform >> seasideProcessRequest:adaptor:resultBlock: (envId 0) @3 line 23 [GsNMethod 10509761281] > 31 WAFastCGIAdaptor >> process: (envId 0) @3 line 4 [GsNMethod 9828418049] > 32 [] in WAFastCGIAdaptor >> answerResponderRole: (envId 0) @2 line 4 [GsNMethod 8795113729] > 33 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457] > 34 WAFastCGIAdaptor >> answerResponderRole: (envId 0) @2 line 5 [GsNMethod 9828248321] > 35 FSResponderRole >> answer (envId 0) @3 line 4 [GsNMethod 8854227457] > 36 FSRole >> handleConnection (envId 0) @3 line 5 [GsNMethod 8854243329] > 37 FSConnection >> unsafeServe (envId 0) @5 line 8 [GsNMethod 8853951745] > 38 [] in FSConnection >> safeServe (envId 0) @2 line 8 [GsNMethod 9557561601] > 39 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457] > 40 [] in FSConnection >> safeServe (envId 0) @2 line 9 [GsNMethod 9322044673] > 41 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457] > 42 [] in FSConnection >> safeServe (envId 0) @2 line 12 [GsNMethod 9258731777] > 43 ExecBlock >> ensure: (envId 0) @2 line 12 [GsNMethod 1064640769] > 44 FSConnection >> safeServe (envId 0) @2 line 15 [GsNMethod 8853958913] > 45 FSConnection >> serve (envId 0) @2 line 4 [GsNMethod 8853957889] > 46 [] in FSSocketServer >> listen: (envId 0) @3 line 15 [GsNMethod 9261209601] > 47 GsProcess >> _start (envId 0) @7 line 16 [GsNMethod 1403422977] > 48 <Reenter marker> > ----------- > _______________________________________________ > Glass mailing list > [hidden email] > http://lists.gemtalksystems.com/mailman/listinfo/glass > > > > _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] Further commits have been disabled for this session because: 'CorruptObj error'. This session must logout.
|
|
In reply to this post by Dale Henrichs-3
... and it turns that the particular case that was causing the corruptObj error in 3.1.0.5 produces an ArgumentError in 3.2 ... there are some paths in the 3.2 code base that could lead to corruptObj errors and THOSE will be fixed in 3.2.x. If we do a 3.1.0.7 release, the bug will be fixed there. Right now there is no schedule for a 3.1.0.7 release. Dale On Tue, May 27, 2014 at 12:38 PM, Dale Henrichs <[hidden email]> wrote:
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: Further commits have been disabled for this session because: 'CorruptObj error'. This session must logout.
|
|
Just for the record in case someone finds a similar problem. I was also getting the error "a TransactionError occurred (error 2249), Further commits have been disabled for this session because: 'CorruptObj error'. This session must logout." I got this error in a stone that I just migrated frlom 3.1.0.6 to 3.2.9. The stack was not showing the real error, but something like this: ----------- Internal FASTCGI ERROR Encountered: 2015-10-28T09:05:03.8873279094696-04:00 a TransactionError occurred (error 2249), Further commits have been disabled for this session because: 'CorruptObj error'. This session must logout. 1 GRGemStonePlatform >> logError:title:shouldCommit: @2 line 4 [GsNMethod 32628056833] 2 GRGemStonePlatform >> logError:title: @2 line 3 [GsNMethod 32628060929] 3 WAFastCGIAdaptor >> internalServerErrorMessage: @20 line 14 [GsNMethod 35365869057] 4 [] in ExecBlock1 (GRGemStonePlatform) >> seasideProcessRequest:adaptor:resultBlock: @4 line 15 [GsNMethod 34982101505] 5 TransactionError (AbstractException) >> _executeOuterHandler: @3 line 7 [GsNMethod 26844481025] 6 TransactionError (AbstractException) >> _pass:with: @4 line 13 [GsNMethod 26844477953] 7 TransactionError (AbstractException) >> pass @2 line 14 [GsNMethod 26844477185] 8 [] in ExecBlock1 (System class) >> _localCommit: @2 line 34 [GsNMethod 27208227073] 9 TransactionError (AbstractException) >> _executeHandler: @3 line 8 [GsNMethod 26844481537] 10 TransactionError (AbstractException) >> _signalFromPrimitive: @1 line 1 [GsNMethod 26844471553] 11 System class >> _primitiveCommit: @1 line 1 [GsNMethod 26844868097] 12 System class >> __commit: @2 line 8 [GsNMethod 26844854785] 13 [] in ExecBlock0 (System class) >> _localCommit: @2 line 30 [GsNMethod 27208227329] 14 ExecBlock0 (ExecBlock) >> onException:do: @2 line 66 [GsNMethod 26843884289] 15 System class >> _localCommit: @8 line 31 [GsNMethod 26844855041] 16 SessionMethodTransactionBoundaryPolicy (TransactionBoundaryDefaultPolicy) >> commit: @2 line 3 [GsNMethod 27996333825] 17 System class >> _commit: @7 line 16 [GsNMethod 26844855297] 18 System class >> commitTransaction @5 line 7 [GsNMethod 26844864257] 19 System class >> _commitPrintingDiagnostics @2 line 8 [GsNMethod 26844773377] 20 SystemCommitTransaction >> defaultAction @2 line 3 [GsNMethod 32628044801] 21 SystemCommitTransaction (AbstractException) >> _signalWith: @5 line 25 [GsNMethod 26844482049] 22 SystemCommitTransaction class (AbstractException class) >> signal @3 line 5 [GsNMethod 26844463873] 23 GRGemStonePlatform >> doCommitTransaction @4 line 3 [GsNMethod 32628057857] 24 [] in ExecBlock0 (GRGemStonePlatform) >> seasideProcessRequestWithRetry:resultBlock: @24 line 32 [GsNMethod 34982100993] 25 ExecBlock0 (ExecBlock) >> ensure: @2 line 12 [GsNMethod 26843897345] 26 TransientRecursionLock >> critical: @11 line 12 [GsNMethod 34929957889] 27 GRGemStonePlatform >> seasideProcessRequestWithRetry:resultBlock: @3 line 6 [GsNMethod 34965516033] 28 [] in ExecBlock1 (GRGemStonePlatform) >> seasideProcessRequest:adaptor:resultBlock: @2 line 6 [GsNMethod 34997412097] 29 Array (Collection) >> do: @5 line 10 [GsNMethod 26844013569] 30 [] in ExecBlock0 (GRGemStonePlatform) >> seasideProcessRequest:adaptor:resultBlock: @3 line 5 [GsNMethod 34982101761] 31 ExecBlock0 (ExecBlock) >> on:do: @3 line 42 [GsNMethod 26843883521] 32 GRGemStonePlatform >> seasideProcessRequest:adaptor:resultBlock: @2 line 12 [GsNMethod 34965516289] 33 WAFastCGIAdaptor >> process: @3 line 4 [GsNMethod 35365868545] 34 [] in ExecBlock0 (WAFastCGIAdaptor) >> answerResponderRole: @2 line 4 [GsNMethod 35391972609] 35 ExecBlock0 (ExecBlock) >> on:do: @3 line 42 [GsNMethod 26843883521] 36 WAFastCGIAdaptor >> answerResponderRole: @2 line 5 [GsNMethod 35365870337] 37 FSResponderRole >> answer @3 line 4 [GsNMethod 35366788609] 38 FSResponderRole (FSRole) >> handleConnection @3 line 5 [GsNMethod 35366808065] 39 FSConnection >> unsafeServe @5 line 8 [GsNMethod 35366100481] 40 [] in ExecBlock0 (FSConnection) >> safeServe @2 line 8 [GsNMethod 35409050625] 41 ExecBlock0 (ExecBlock) >> on:do: @3 line 42 [GsNMethod 26843883521] 42 [] in ExecBlock0 (FSConnection) >> safeServe @2 line 9 [GsNMethod 35403513089] 43 ExecBlock0 (ExecBlock) >> on:do: @3 line 42 [GsNMethod 26843883521] 44 [] in ExecBlock0 (FSConnection) >> safeServe @2 line 12 [GsNMethod 35392137217] 45 ExecBlock0 (ExecBlock) >> ensure: @2 line 12 [GsNMethod 26843897345] 46 FSConnection >> safeServe @2 line 15 [GsNMethod 35366102529] 47 FSConnection >> serve @2 line 4 [GsNMethod 35366102017] 48 [] in ExecBlock (FSSocketServer) >> listen: @3 line 15 [GsNMethod 35392254721] 49 GsProcess >> _start @7 line 16 [GsNMethod 26844511489] 50 <Reenter marker> The corrupt thingy made me think in running an audit in the indexes since I knew they changed in 3.2.9. I executed this code that I normally run to check indexes health: | collectionsWithBadIndexes | IndexManager current removeAllIncompleteIndexes. collectionsWithBadIndexes := IndexManager current nscsWithBadIndexes. Transcript show: 'There are ' , collectionsWithBadIndexes size asString, ' collections with bad indexes.'; cr. Transcript show: 'The following are the OOPs of such collections: '. collectionsWithBadIndexes do: [ :each | Transcript show: ' - ' , each asOop asString. ]. Transcript cr. And that indeed triggered an error saying something about Unicode and collector etc. So it was clearly the problem that I had an index mixing plain strings and instances of Unicode* classes. So the fix was to rebuild the indexes (first removing all of them via `IndexManager current removeAllTracking.`) and specifying a Unicode* as the `pathElementClass` or using the new API for creating indexes and using the message for unicode ones. Best, On Tue, May 27, 2014 at 4:54 PM, Dale Henrichs <[hidden email]> wrote:
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: Further commits have been disabled for this session because: 'CorruptObj error'. This session must logout.
|
|
Mariano,
I would like to understand this a bit more - the corruptObj error in this original message should not be the source of the corruptObj in this case, because we believe that we have fixed that bug I'd like to get to the bottom of this (perhaps a different email thread, since at the moment I don't think they share the same root cause ... Dale On 10/28/2015 06:30 AM, Mariano
Martinez Peck wrote:
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: Further commits have been disabled for this session because: 'CorruptObj error'. This session must logout.
|
|
Sure, I can start a new thread. But what else could I provided you in order to get to the bottom of this? The only thing I did not mention in my email is the exact unicode error message when checking the health of the indexes. What else could help? On Wed, Oct 28, 2015 at 1:34 PM, Dale Henrichs <[hidden email]> wrote:
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: Further commits have been disabled for this session because: 'CorruptObj error'. This session must logout.
|
|
I guess the exact unicode error message would be a starting point
...
This whole thread started because we had a bug that would throw a corruptObj message when invalid unicode was fed to a unicode primitive ... that bug was fixed in 3.2.2 or so ... So this corruptObj message is occuring for a different reason altogether ... In general you should not have to rebuild indexes across upgrade boundaries (unless we document that you have to) .... While it is good to know that a rebuild of indexes fixed you problem,. I am still interested in understanding what caused the error in the first place, since it may actually be another bug ... Dale On 10/28/2015 09:41 AM, Mariano
Martinez Peck wrote:
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
«
Return to GLASS
|
1 view|%1 views
| Free forum by Nabble | Edit this page |