Smalltalk › Gemtalk › Gemstone/S

A question about #transactionless mode

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

4 messages Options

Kevin Szabo-2

A question about #transactionless mode

Hello again GemStone folks,

I am supporting a system that has a number of reporting TOPAZ scripts kicked off by CRON. Many of these scripts were written a decade or so ago, and of course, still work perfectly because that is just what GemStone does.

Some of the scripts are using the default transaction Automatic (since I see their state as #autoBegin). This was the time before #transactionless mode.

I am considering moving them to #transactionless since they do read-only reports on the database. However, I need someone to help me with the interpretation of the manual on this. When I run #transactionless all appears to work correctly, but according to some documents I should not do this because I have an unstable view on the database? Is this correct? Should I set to transactionless, do a beginTransaction (to get a stable view), run my report, and then abort the transaction?

Some of the reports generated don't have to be completely accurate. That is, if I am measuring the size of a work queue I don't mind if there is some fuzziness in the length reported. I just care if the queue is 10, 100, or 10,000. I don't need the exact length, because it has changed by the time the report finished anyway. Also, I would like to run this report fairly frequently (to update a web display of system health) so I prefer fast and fuzzy rather than slow and exact.

Thanks for any thoughts on this.

Regards,
Kevin Szabo.

P.S. I still enjoy amazing up-times and tremendous reliability.

_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk

jgfoster

Re: A question about #transactionless mode

Hi Kevin,

Transactionless mode is discussed on pages 144-46 of the Programming Guide (found at http://gemtalksystems.com/index.php/community/gss-support/documentation/gs64/).

In general, the system keeps a consistent view of the database and you control when the view changes with an abort or commit. Because there is overhead in keeping a unique view, it is costly to the system if you stay on an old view after everyone else has moved ahead. The “commit record backlog” is one of the first things we look at when people have problems with their database.

Selecting ‘transactionless’ mode is a way to tell the system that you are willing to let the system perform an abort anytime you are on the oldest commit record (to move you to the newest one). As described in the documentation, this would typically be reasonable for an idle session.

The risk of an “unstable” view of the database is that if you perform an operation that looks at data across multiple transactions there can be an inconsistency. The typical example is with bank accounts where if you looked at the balance in my savings account, then (in a single transaction) I transferred funds from savings to checking, and then (in a later view) you looked at the balance in my checking account, you would get an incorrect idea of how much money I have in total. It is in this way that the database is “unstable.” That is, the values might not be consistent in a logical sense; there is nothing “unstable” in the sense that you could do physical harm to the database.

Depending on the operation, this “instability” might not be a problem. If you wait till after midnight and then do an analysis of all the transactions that took place yesterday, then subsequent transactions will never (logically) affect what you see and your analysis will always be the same as if you were in a single transaction.

Transactionless mode only has meaning when you are not in a transaction. If you begin a transaction (whether in auto, manual, or transactionless mode), you are in a transaction and will stay in that exact transaction until you commit or abort. Thus, you lose the “benefit” of telling the system that it may advance your view if you are behind the rest.

Do you have an idea of how long the report takes to run and how many other transaction are typically done during that time? If it takes a minute to run and there are 20 other transactions, then you shouldn’t worry about a commit record backlog. If it takes 30 minutes to run and there are 20000 other transactions, then you should be (very) worried.

James

On Oct 29, 2013, at 3:27 AM, Kevin Szabo <[hidden email]> wrote:

Hello again GemStone folks,

I am supporting a system that has a number of reporting TOPAZ scripts kicked off by CRON. Many of these scripts were written a decade or so ago, and of course, still work perfectly because that is just what GemStone does.

Some of the scripts are using the default transaction Automatic (since I see their state as #autoBegin). This was the time before #transactionless mode.

I am considering moving them to #transactionless since they do read-only reports on the database. However, I need someone to help me with the interpretation of the manual on this. When I run #transactionless all appears to work correctly, but according to some documents I should not do this because I have an unstable view on the database? Is this correct? Should I set to transactionless, do a beginTransaction (to get a stable view), run my report, and then abort the transaction?

Some of the reports generated don't have to be completely accurate. That is, if I am measuring the size of a work queue I don't mind if there is some fuzziness in the length reported. I just care if the queue is 10, 100, or 10,000. I don't need the exact length, because it has changed by the time the report finished anyway. Also, I would like to run this report fairly frequently (to update a web display of system health) so I prefer fast and fuzzy rather than slow and exact.

Thanks for any thoughts on this.

Regards,
Kevin Szabo.

P.S. I still enjoy amazing up-times and tremendous reliability.
_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk

_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk

Paul Baumann

Re: A question about #transactionless mode

In reply to this post by Kevin Szabo-2

If you go transactionless (and without aborts) then your reports would show the same data until enough CR backlog accumulates for the gem's view to be updated. Your reports would not be as current and you'd be allowing CR accumulation and then causing the server to have to manage views though a longer series of changes. If you go transactionless (with aborts) then your reports would show timely information and would allow for data to be refreshed when a report is taking too long to generate. If there is a long gap between report runs then being transactionless automatically avoids holding the oldest CR, but it still allows CR accumulation which is slow to resolve through. Whether in transaction or not, server performance is improved from minimizing the number of change records the server needs to resolve though; it means doing periodic aborts before the server demands them and according to your own control.

It sounds like your reports are currently run in automatic transaction mode, are accurate, and have never caused a problem from years of running that way. If the reporting gem is not involved in holding a CR backlog then I don't see an advantage to making it transactionless. If the report had a bug that allowed infinite recursion then running transactionless would allow it to trudge on consuming CPU without ever being terminated for holding the oldest CR too long. Not a strong reason for avoiding transactionless mode, but not a good argument for it either. An argument against using transactionless can be that some enumerations would fail if the size unexpectedly changes during the enumeration. It is better to have control over when objects change.

Paul Baumann

From: [hidden email] [mailto:[hidden email]] On Behalf Of Kevin Szabo
Sent: Tuesday, October 29, 2013 02:28
To: [hidden email]
Subject: [GemStone-Smalltalk] A question about #transactionless mode

Hello again GemStone folks,

Thanks for any thoughts on this.

Regards,
Kevin Szabo.

P.S. I still enjoy amazing up-times and tremendous reliability.

This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.

_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk

James Foster-9

Re: A question about #transactionless mode

Paul make a good point about iterating over a collection. Experienced Smalltalkers know that you shouldn’t modify a collection while iterating over it, but GemStone presents the risk that someone else might modify a collection while you are iterating over it.

If you are iterating over a collection, you should do it in one transaction or make a private copy and iterate over the copy. Then the collection won’t change (though the properties of the objects in the collection could change if you commit or abort).

James

On Oct 29, 2013, at 3:19 PM, Paul Baumann <[hidden email]> wrote:

If you go transactionless (and without aborts) then your reports would show the same data until enough CR backlog accumulates for the gem's view to be updated. Your reports would not be as current and you'd be allowing CR accumulation and then causing the server to have to manage views though a longer series of changes. If you go transactionless (with aborts) then your reports would show timely information and would allow for data to be refreshed when a report is taking too long to generate. If there is a long gap between report runs then being transactionless automatically avoids holding the oldest CR, but it still allows CR accumulation which is slow to resolve through. Whether in transaction or not, server performance is improved from minimizing the number of change records the server needs to resolve though; it means doing periodic aborts before the server demands them and according to your own control.

It sounds like your reports are currently run in automatic transaction mode, are accurate, and have never caused a problem from years of running that way. If the reporting gem is not involved in holding a CR backlog then I don't see an advantage to making it transactionless. If the report had a bug that allowed infinite recursion then running transactionless would allow it to trudge on consuming CPU without ever being terminated for holding the oldest CR too long. Not a strong reason for avoiding transactionless mode, but not a good argument for it either. An argument against using transactionless can be that some enumerations would fail if the size unexpectedly changes during the enumeration. It is better to have control over when objects change.

Paul Baumann

From: [hidden email] [[hidden email]] On Behalf Of Kevin Szabo
Sent: Tuesday, October 29, 2013 02:28
To: [hidden email]
Subject: [GemStone-Smalltalk] A question about #transactionless mode

Hello again GemStone folks,

I am supporting a system that has a number of reporting TOPAZ scripts kicked off by CRON. Many of these scripts were written a decade or so ago, and of course, still work perfectly because that is just what GemStone does.

Some of the scripts are using the default transaction Automatic (since I see their state as #autoBegin). This was the time before #transactionless mode.

I am considering moving them to #transactionless since they do read-only reports on the database. However, I need someone to help me with the interpretation of the manual on this. When I run #transactionless all appears to work correctly, but according to some documents I should not do this because I have an unstable view on the database? Is this correct? Should I set to transactionless, do a beginTransaction (to get a stable view), run my report, and then abort the transaction?

Some of the reports generated don't have to be completely accurate. That is, if I am measuring the size of a work queue I don't mind if there is some fuzziness in the length reported. I just care if the queue is 10, 100, or 10,000. I don't need the exact length, because it has changed by the time the report finished anyway. Also, I would like to run this report fairly frequently (to update a web display of system health) so I prefer fast and fuzzy rather than slow and exact.

Thanks for any thoughts on this.

Regards,
Kevin Szabo.

P.S. I still enjoy amazing up-times and tremendous reliability.

This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.

_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk

_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk