TransactionBacklog Exception

Hey,

another experience from load testing.

I had some simple file upload API calls in my software and Gemstone should just receive the data and store it in the local server filesystem.

When testing it against a non-stressed database everything went ok - no problems at all: I uploaded a 4 MB file from my applications, stored it into the file system, executed an external shell command to expand the data and then I did an abort (because no database data was changed) and everthing was fine. It worked from the UI and it worked from my Python application and yes it seems to be a long running task.

Then I did heavy load testing and wanted to test this upload under heavy traffic ... and I always got a TransactionBacklog Exeption. I did NOT get it with smaller files.

So I noticed from the documentation that such an exception has to been enabled to be thrown in an image, but what are the initiating reason, that such an exception is thrown ? It is because the file upload did took so long ? I had much longer  (pure internal database) tasks running in the database and it never got such an exeption. I think, that I should simply ignore such an exeption ...

And it happens always at the same location

[] in MSKRESTApiServer >> executeOneRequestResponseOn: @94 line 23 [GsNMethod 3863550465]
2 TransactionBacklog (AbstractException) >> _executeHandler: @8 line 11 [GsNMethod 5454593]
3 TransactionBacklog (AbstractException) >> _signalWith: @1 line 1 [GsNMethod 4438273]
4 [] in AbstractException >> _signalAsync @11 line 19 [GsNMethod 8260865]
5 ExecBlock0 (ExecBlock) >> ensure: @2 line 12 [GsNMethod 4357889]
6 TransactionBacklog (AbstractException) >> _signalAsync @7 line 21 [GsNMethod 5453313]
7 AnsiReadStream (PositionableStream) >> readInto:startingAt:count: @12 line 6 [GsNMethod 163697921]
8 ZnUtils class >> readUpToEnd:limit: @17 line 9 [GsNMethod 165463809]
9 ZnByteArrayEntity >> readFrom: @7 line 4 [GsNMethod 162900993]

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Marten

There's a pretty good description of the circumstances surrounding TransactionBacklog in the manual[1]

And the method IndexManagerAutoCommitPolicy>>evaluate:for: is an example of an TransactionBacklog handler that does a commit when the commit record backlog exceeds the defined threshold ...

So ... if you have a long running operation and an active system (lots of commits) you may choose to either increase the STN_CR_BACKLOG_THRESHOLD so the TransactionBacklogs are not signalled or you can add a handler for the exception (recommended) that does commits when the threshold is exceeded ... if you do not want your primary data updated until the load is complete, then I suggest that you create a temporary persistent root and hook your data into that root until all of the objects have made it into the db (committing along the way to keep the CR backlog in check) and then you hook up your data strcutre to the primary system once the long-running is complete ..

Dale

[1] https://downloads.gemtalksystems.com/docs/GemStone64/3.4.x/GS64-SysAdminGuide-3.4/9-RepositorySpace.htm#pgfId-1140400

Hey,

another experience from load testing.

I had some simple file upload API calls in my software and Gemstone should just receive the data and store it in the local server filesystem.

When testing it against a non-stressed database everything went ok - no problems at all: I uploaded a 4 MB file from my applications, stored it into the file system, executed an external shell command to expand the data and then I did an abort (because no database data was changed) and everthing was fine. It worked from the UI and it worked from my Python application and yes it seems to be a long running task.

Then I did heavy load testing and wanted to test this upload under heavy traffic ... and I always got a TransactionBacklog Exeption. I did NOT get it with smaller files.

So I noticed from the documentation that such an exception has to been enabled to be thrown in an image, but what are the initiating reason, that such an exception is thrown ? It is because the file upload did took so long ? I had much longer  (pure internal database) tasks running in the database and it never got such an exeption. I think, that I should simply ignore such an exeption ...

And it happens always at the same location

[] in MSKRESTApiServer >> executeOneRequestResponseOn: @94 line 23 [GsNMethod 3863550465]
2 TransactionBacklog (AbstractException) >> _executeHandler: @8 line 11 [GsNMethod 5454593]
3 TransactionBacklog (AbstractException) >> _signalWith: @1 line 1 [GsNMethod 4438273]
4 [] in AbstractException >> _signalAsync @11 line 19 [GsNMethod 8260865]
5 ExecBlock0 (ExecBlock) >> ensure: @2 line 12 [GsNMethod 4357889]
6 TransactionBacklog (AbstractException) >> _signalAsync @7 line 21 [GsNMethod 5453313]
7 AnsiReadStream (PositionableStream) >> readInto:startingAt:count: @12 line 6 [GsNMethod 163697921]
8 ZnUtils class >> readUpToEnd:limit: @17 line 9 [GsNMethod 165463809]
9 ZnByteArrayEntity >> readFrom: @7 line 4 [GsNMethod 162900993]

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass