Smalltalk › Frameworks & Tools › GLORP

Caching of objects (how to disable for certain classes)

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

11 messages Options

Esteban A. Maringolo

Caching of objects (how to disable for certain classes)

I'm having a case I didn't have before. And I'm sure it is because I'm using GLORP wrong. This is Pharo 3.

tl;dr

I have two sessions, each one with its own connection, accessing the same database.

When I change the object in one session, they are stored okay in the database but the other session doesn't get the changes after querying the database again.

Long version:

I have a business transaction class named TsOrderNotice, stored in the table BUSINESSTRANSACTION.

TsOrderNotice is a subclass of TsBusinessTransaction and it is using a filtered type resolver.

Session #1:

In one session I mark one session as approved by doing:

glorp inUnitOfWorkDo: [:session |

session register: anOrder.

session beApproved ].

And that's it. Everything is fine, the transaction is flagged as approved and stored as such in the DB (the status is a simple column).

Once approved, the other actor sees this new approved transaction, and then can mark it as shipped:

Session #2:

glorp inUnitOfWorkDo: [:session |

session register: anOrder.

session beShipped ].

The database is happy, and the order has the correct shipped status indicator in the status column.

Now going back to the session #1, it keeps seeing the order as Approved, and never loads the new status.

It doesn't matter if I query that single record or a large data set that includes it.

In an inspector of GlorpSession #1 I evaluated the following to test whether I was nuts or what was going on. The result is between quotes.

(self accessor basicExecuteSQLString: 'SELECT ID, STATUS FROM BUSINESSTRANSACTION WHERE ID=5182') upToEnd last. "#(5182 'Shipped') OK"

I query the session and got the "old" version:

(self readOneOf: TsOrderNotice where: [ :e | e id = 5182 ]) status. "Approved"

"I check in the cache, and it is there"

(self cacheAt: 5182 forClass: TsOrderNotice ifNone: [ ]) status. "Approved"

"I read many instances, without any filter/expression whatsoever."

((self readManyOf: TsOrderNotice) detect: [ :one | one id = 5182 ] ) status. "Approved"

"I force the expiration of the glorp cache"

(self instVarNamed: 'cache') expireAll.

"Check again, it is still there"

(self cacheAt: 5182 forClass: TsOrderNotice ifNone: [ ]) status. "Approved"

How can I be sure I'm reading the latest data from the database? Can this be enforced? Something like noCacheDo: aBlock...

For certain objects I'm okay with keeping instances in cache, because they seldom change, but for something that is transactional, this is an issue.

I'm reading all the objects through the glorp session and performing any modifications always inside a UOW. So I guess this time I'm playing by the rules.

Edit: after looking at the default CachePolicy the #expire: method is a no op, so no expiration can be forced.

Regards,

Esteban.

--
You received this message because you are subscribed to the Google Groups "glorp-group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/glorp-group.
For more options, visit https://groups.google.com/d/optout.

jtuchel

Re: Caching of objects (how to disable for certain classes)

Esteban.

I guess what you need to do is refresh the objects you need to be sure have not changed or reread the changed version. see GlorpSession>>#refresh: This will re-read an object and register its latest copy with the current session.

Are you using optimistic locking with a version counter or something? This would at least tell you that you are trying to update an object that has been changed in another session and you could refresh on demand by handling that locking exception.

I don't use inUnitOfWorkDo:, and I am not sure this is a good way to work with Glorp. But maybe there are people more knowledgable on Glorp usage than me who can comment on this.

I hope these sparkles help

Joachim

Am Freitag, 30. Mai 2014 00:18:59 UTC+2 schrieb Esteban A. Maringolo:

I'm having a case I didn't have before. And I'm sure it is because I'm using GLORP wrong. This is Pharo 3.

tl;dr
I have two sessions, each one with its own connection, accessing the same database.
When I change the object in one session, they are stored okay in the database but the other session doesn't get the changes after querying the database again.

Long version:
I have a business transaction class named TsOrderNotice, stored in the table BUSINESSTRANSACTION.
TsOrderNotice is a subclass of TsBusinessTransaction and it is using a filtered type resolver.

Session #1:
In one session I mark one session as approved by doing:
glorp inUnitOfWorkDo: [:session |
session register: anOrder.
session beApproved ].

And that's it. Everything is fine, the transaction is flagged as approved and stored as such in the DB (the status is a simple column).

Once approved, the other actor sees this new approved transaction, and then can mark it as shipped:
Session #2:
glorp inUnitOfWorkDo: [:session |
session register: anOrder.
session beShipped ].

The database is happy, and the order has the correct shipped status indicator in the status column.

Now going back to the session #1, it keeps seeing the order as Approved, and never loads the new status.
It doesn't matter if I query that single record or a large data set that includes it.

In an inspector of GlorpSession #1 I evaluated the following to test whether I was nuts or what was going on. The result is between quotes.

(self accessor basicExecuteSQLString: 'SELECT ID, STATUS FROM BUSINESSTRANSACTION WHERE ID=5182') upToEnd last. "#(5182 'Shipped') OK"
I query the session and got the "old" version:
(self readOneOf: TsOrderNotice where: [ :e | e id = 5182 ]) status. "Approved"

"I check in the cache, and it is there"
(self cacheAt: 5182 forClass: TsOrderNotice ifNone: [ ]) status. "Approved"

"I read many instances, without any filter/expression whatsoever."
((self readManyOf: TsOrderNotice) detect: [ :one | one id = 5182 ] ) status. "Approved"

"I force the expiration of the glorp cache"
(self instVarNamed: 'cache') expireAll.

"Check again, it is still there"
(self cacheAt: 5182 forClass: TsOrderNotice ifNone: [ ]) status. "Approved"

How can I be sure I'm reading the latest data from the database? Can this be enforced? Something like noCacheDo: aBlock...

For certain objects I'm okay with keeping instances in cache, because they seldom change, but for something that is transactional, this is an issue.
I'm reading all the objects through the glorp session and performing any modifications always inside a UOW. So I guess this time I'm playing by the rules.

Edit: after looking at the default CachePolicy the #expire: method is a no op, so no expiration can be forced.

Regards,

--
Esteban.

Maarten Mostert

Re: Caching of objects (how to disable for certain classes)

Think Joachim is right.

The sessions each have their own cache so as long as the cache policy doesn't decide to refresh the Object it will be happy with the older version.

Another way would be removing the Object from the secondary session to be sure once required it will look to refresh. If you do it like that you maintain identity, as the Glorp will redefine it through the proxy. If you refresh however you basically get a copy of the Object. Which is fine only up to the moment you're going to write an update and Glorp will find two different things wanting to write to the same database row, resulting in an primary key error.

My experience is that if you start refreshing you end up refreshing and refreshing everything in order to maintain integrity, which will give for lots of extra queries and a dead end.

So my rule of thumb is.

- Read only the root Object of your model and make accessor methods to go and look thought your hierarchy to aces other Object through the proxies.

- Always use parameters to pass Objects from method to method or put them in an Array.

Avoid to do something like

rootTask := self getGlorpSession readOneOf: Task

where: [:each | each key = 1].

taskB:=rootTask. <== this will give you trouble as taskB will not be in the cache only rootTask is.

taskB:= Array with: rootTask. <=== this will make sure you have the same Object.

However if you are sure to use the secondary session only for reading you can implement the following within GlorpSession that will do just that.

readRefreshedOneOf: aClass where: aBlock

^self execute: ((Query returningOneOf: aClass where: aBlock)

shouldRefresh: true;

yourself)

Regards,

@+Maaren,

> "jtuchel" <[hidden email]> |

Esteban.

I don't use inUnitOfWorkDo:, and I am not sure this is a good way to work with Glorp. But maybe there are people more knowledgable on Glorp usage than me who can comment on this.

I hope these sparkles help

Joachim

Am Freitag, 30. Mai 2014 00:18:59 UTC+2 schrieb Esteban A. Maringolo:

I'm having a case I didn't have before. And I'm sure it is because I'm using GLORP wrong. This is Pharo 3.
tl;dr

I have two sessions, each one with its own connection, accessing the same database.

When I change the object in one session, they are stored okay in the database but the other session doesn't get the changes after querying the database again.

Long version:

I have a business transaction class named TsOrderNotice, stored in the table BUSINESSTRANSACTION.

TsOrderNotice is a subclass of TsBusinessTransaction and it is using a filtered type resolver.

Session #1:

In one session I mark one session as approved by doing:

glorp inUnitOfWorkDo: [:session |

session register: anOrder.

session beApproved ].

And that's it. Everything is fine, the transaction is flagged as approved and stored as such in the DB (the status is a simple column).

Once approved, the other actor sees this new approved transaction, and then can mark it as shipped:

Session #2:

glorp inUnitOfWorkDo: [:session |

session register: anOrder.

session beShipped ].

The database is happy, and the order has the correct shipped status indicator in the status column.

Now going back to the session #1, it keeps seeing the order as Approved, and never loads the new status.

It doesn't matter if I query that single record or a large data set that includes it.

In an inspector of GlorpSession #1 I evaluated the following to test whether I was nuts or what was going on. The result is between quotes.

(self accessor basicExecuteSQLString: 'SELECT ID, STATUS FROM BUSINESSTRANSACTION WHERE ID=5182') upToEnd last. "#(5182 'Shipped') OK"

I query the session and got the "old" version:

(self readOneOf: TsOrderNotice where: [ :e | e id = 5182 ]) status. "Approved"

"I check in the cache, and it is there"

(self cacheAt: 5182 forClass: TsOrderNotice ifNone: [ ]) status. "Approved"

"I read many instances, without any filter/expression whatsoever."

((self readManyOf: TsOrderNotice) detect: [ :one | one id = 5182 ] ) status. "Approved"

"I force the expiration of the glorp cache"

(self instVarNamed: 'cache') expireAll.

"Check again, it is still there"

(self cacheAt: 5182 forClass: TsOrderNotice ifNone: [ ]) status. "Approved"

How can I be sure I'm reading the latest data from the database? Can this be enforced? Something like noCacheDo: aBlock...

For certain objects I'm okay with keeping instances in cache, because they seldom change, but for something that is transactional, this is an issue.

I'm reading all the objects through the glorp session and performing any modifications always inside a UOW. So I guess this time I'm playing by the rules.

Edit: after looking at the default CachePolicy the #expire: method is a no op, so no expiration can be forced.

Regards,

--

Esteban.

Esteban A. Maringolo

Re: Caching of objects (how to disable for certain classes)

In reply to this post by jtuchel

El viernes, 30 de mayo de 2014 02:15:55 UTC-3, jtuchel escribió:

Esteban.

I guess what you need to do is refresh the objects you need to be sure have not changed or reread the changed version. see GlorpSession>>#refresh: This will re-read an object and register its latest copy with the current session.

That would work for a single object, but not for a whole list.

Are you using optimistic locking with a version counter or something? This would at least tell you that you are trying to update an object that has been changed in another session and you could refresh on demand by handling that locking exception.

The problem is when reading the object back. No write conflicts (because they're handled by app logic so far).

I don't use inUnitOfWorkDo:, and I am not sure this is a good way to work with Glorp. But maybe there are people more knowledgable on Glorp usage than me who can comment on this.

How do you modify objects if it's not inside a unit of work?

Regards!

Esteban A. Maringolo

Re: Caching of objects (how to disable for certain classes)

In reply to this post by Maarten Mostert

Hi,

El viernes, 30 de mayo de 2014 03:55:53 UTC-3, [hidden email] escribió:

Think Joachim is right.

The sessions each have their own cache so as long as the cache policy doesn't decide to refresh the Object it will be happy with the older version.

Another way would be removing the Object from the secondary session to be sure once required it will look to refresh. If you do it like that you maintain identity, as the Glorp will redefine it through the proxy. If you refresh however you basically get a copy of the Object. Which is fine only up to the moment you're going to write an update and Glorp will find two different things wanting to write to the same database row, resulting in an primary key error.

My experience is that if you start refreshing you end up refreshing and refreshing everything in order to maintain integrity, which will give for lots of extra queries and a dead end.

If I need to get the latest version of an object this can't be avoided.

I don't want those objects (Orders) to be cached, and I'm willing to pay the extra I/O time in order to always have the latest version.

As said before, some kind of objects can be kept in cache for the lifetime of the session, and that'd be fine because they seldom change.

But "dated objects" shouldn't.

So my rule of thumb is.

- Read only the root Object of your model and make accessor methods to go and look thought your hierarchy to aces other Object through the proxies.

If I read 1000 instances of, let's say, Order, which have related objects such as Customers, and Providers, and many other classes, I will end up querying the database only time for the Orders, and then lots of times for each referenced object (if the cardinality is high this can be huge).

- Always use parameters to pass Objects from method to method or put them in an Array.

Avoid to do something like

rootTask := self getGlorpSession readOneOf: Task

where: [:each | each key = 1].

taskB:=rootTask. <== this will give you trouble as taskB will not be in the cache only rootTask is.

taskB:= Array with: rootTask. <=== this will make sure you have the same Object.

Sorry, I didn't get this.

However if you are sure to use the secondary session only for reading you can implement the following within GlorpSession that will do just that.

^self execute: ((Query returningOneOf: aClass where: aBlock)

shouldRefresh: true;

yourself)

The #shouldRefresh: parameter of the query is what I was looking for.

Though it refreshes everything related with the object, and not just aClass.

E.g.:

Suppose I have this object graph:

anOrder *

\_ aCustomer

\_ aProvider

\_ aCustomReferencedObject

\_ n (many) _orderItems *

\_ n aProduct

\_ n Items (if the product is composite)

I only want to get the latest version of anOrder and its items. Everything marked with asterisk, the other classes can be pulled from the cache safely.

It is... I want to specify a per-class caching policy, having a "never cache" policy, or at least being able to force the expiration, something like "aGlorpSession resetCacheFor: aClass".

I know there must be a way, or I'll have to make one. :)

Best regards!

Alan Knight

Re: Caching of objects (how to disable for certain classes)

In reply to this post by Esteban A. Maringolo

There are lots of options, but it's a semantically complicated issue. Any time you have caching, you have the issue of when to update the cache. http://martinfowler.com/bliki/TwoHardThings.html

Mechanisms you have in Glorp include

- refresh: you can ask the session to refresh an object, but it's also an option you can set on any query, so you can refresh a large set of objects

- timestamps / version numbers on objects. So you can e.g. query for objects you have but whose timestamp or version number is different from what you have and refresh those. But with an object graph this always gets semantically complicated. So, suppose the order has changed. That may mean that anything it contains has changed and needs to be refreshed. But what constitutes containment.

- Cache policies, which can be set on a per-class basis, and you can also control the caching on a query.

- Removing items from the cache, but that's quite dangerous. Better to have a weak cache policy and then if nothing refers to an object it can disappear. If you remove it manually and anything in your program still has a reference to it, including an active stack frame, then you can have two copies that appear the same but the system thinks they're different. Duplicate primary key errors and other problems arise.

The refresh option on the query should, I think, only refresh the object in question, but will reset proxies to the other objects it contains. If those objects are still in cache, then when they're accessed they should be retrieved without a query. But that only applies to single objects, a collection would always require a query. And a weak cache might throw away the related objects before you used them again.

I also note that I always use inUnitOfWorkDo: or similar mechanisms (e.g. transact:) if at all possible, i.e. when the thing I'm doing is a computation that fits inside a block, not some long-running thing.

P.S. Glad you like alsoFetch: And if you set refresh on the query, it will also apply to the objects also being fetched.

On Fri May 30 2014 at 7:30:56 AM, Esteban A. Maringolo <[hidden email]> wrote:

El viernes, 30 de mayo de 2014 02:15:55 UTC-3, jtuchel escribió:

Esteban.

I guess what you need to do is refresh the objects you need to be sure have not changed or reread the changed version. see GlorpSession>>#refresh: This will re-read an object and register its latest copy with the current session.

That would work for a single object, but not for a whole list.

Are you using optimistic locking with a version counter or something? This would at least tell you that you are trying to update an object that has been changed in another session and you could refresh on demand by handling that locking exception.

The problem is when reading the object back. No write conflicts (because they're handled by app logic so far).

I don't use inUnitOfWorkDo:, and I am not sure this is a good way to work with Glorp. But maybe there are people more knowledgable on Glorp usage than me who can comment on this.

How do you modify objects if it's not inside a unit of work?

Regards!

--
You received this message because you are subscribed to the Google Groups "glorp-group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/glorp-group.
For more options, visit https://groups.google.com/d/optout.

Esteban A. Maringolo

Re: Caching of objects (how to disable for certain classes)

El viernes, 30 de mayo de 2014 12:47:22 UTC-3, alan.knight escribió:

There are lots of options, but it's a semantically complicated issue. Any time you have caching, you have the issue of when to update the cache. <a href="http://martinfowler.com/bliki/TwoHardThings.html" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fmartinfowler.com%2Fbliki%2FTwoHardThings.html\46sa\75D\46sntz\0751\46usg\75AFQjCNG39OLEOLIbg7Dv1UvJdz9GG31gzA';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fmartinfowler.com%2Fbliki%2FTwoHardThings.html\46sa\75D\46sntz\0751\46usg\75AFQjCNG39OLEOLIbg7Dv1UvJdz9GG31gzA';return true;">http://martinfowler.com/bliki/TwoHardThings.html

Knew that one. One of my favorites too. :)

Mechanisms you have in Glorp include
- refresh: you can ask the session to refresh an object, but it's also an option you can set on any query, so you can refresh a large set of objects

- timestamps / version numbers on objects. So you can e.g. query for objects you have but whose timestamp or version number is different from what you have and refresh those. But with an object graph this always gets semantically complicated. So, suppose the order has changed. That may mean that anything it contains has changed and needs to be refreshed. But what constitutes containment.

I would have to have a "last query" ts, though it requires me to have a "state machine" where I'm performing such queries.

- Cache policies, which can be set on a per-class basis, and you can also control the caching on a query.

- Removing items from the cache, but that's quite dangerous.

Yes it is.

Better to have a weak cache policy and then if nothing refers to an object it can disappear. If you remove it manually and anything in your program still has a reference to it, including an active stack frame, then you can have two copies that appear the same but the system thinks they're different. Duplicate primary key errors and other problems arise.

As long as the read instance is only used for read only operations, the risk of having PK errors and similar is reduced, if not eliminated completely.

Of course there would be a case of a cache invalidation happening while a modification is running, but it is also unlikely, unless you have long running Unit of works (which I don't).

The refresh option on the query should, I think, only refresh the object in question, but will reset proxies to the other objects it contains. If those objects are still in cache, then when they're accessed they should be retrieved without a query. But that only applies to single objects, a collection would always require a query. And a weak cache might throw away the related objects before you used them again.

The refresh option works okay when I'm showing the details of a single object, I perform the query enforcing the refresh and It gets the right version.

But if I have to list a bunch of objects, I don't know whether they changed or not, and keeping track of the last time i queried the database or similar would be a waste of resources.

I also note that I always use inUnitOfWorkDo: or similar mechanisms (e.g. transact:) if at all possible, i.e. when the thing I'm doing is a computation that fits inside a block, not some long-running thing.

I avoid long-running things, but I perform reads just by building a Query and executing it in the context of a session.

As far as I understand the Unit of Work is meant to be used to modify objects, because if perform the query inside the UOW it will traverse all the read objects to detect which one was modified. It doubles the time it takes to read.

P.S. Glad you like alsoFetch: And if you set refresh on the query, it will also apply to the objects also being fetched

I'd like to fully understand how caching works in GLORP, I've debugged it a little, but more is required :)

In the meantime I implemented a clearCacheFor: aClass which basically releases a previous cache (if exists) in the CacheManager and replaces it with a new one. It's a sharp move, that could cause bleeding, but so far it hasn't, because I'm using it only at one point.

Thank you!

Alan Knight

Re: Caching of objects (how to disable for certain classes)

On Fri May 30 2014 at 10:45:49 AM, Esteban A. Maringolo <[hidden email]> wrote:

El viernes, 30 de mayo de 2014 12:47:22 UTC-3, alan.knight escribió:
There are lots of options, but it's a semantically complicated issue. Any time you have caching, you have the issue of when to update the cache. http://martinfowler.com/bliki/TwoHardThings.html

Knew that one. One of my favorites too. :)

Mechanisms you have in Glorp include
- refresh: you can ask the session to refresh an object, but it's also an option you can set on any query, so you can refresh a large set of objects

- timestamps / version numbers on objects. So you can e.g. query for objects you have but whose timestamp or version number is different from what you have and refresh those. But with an object graph this always gets semantically complicated. So, suppose the order has changed. That may mean that anything it contains has changed and needs to be refreshed. But what constitutes containment.

I would have to have a "last query" ts, though it requires me to have a "state machine" where I'm performing such queries.

If your rows have a lock column defined, then Glorp knows what the last timestamp/version number is and you can ask for it from the cache. It shouldn't be too difficult to write a query to ask

for all objects of class X, list the ones that I have in cache, or some other set, and whose timestamp is newer than the timestamp I have for them.

- Cache policies, which can be set on a per-class basis, and you can also control the caching on a query.

- Removing items from the cache, but that's quite dangerous.

Yes it is.

Better to have a weak cache policy and then if nothing refers to an object it can disappear. If you remove it manually and anything in your program still has a reference to it, including an active stack frame, then you can have two copies that appear the same but the system thinks they're different. Duplicate primary key errors and other problems arise.

As long as the read instance is only used for read only operations, the risk of having PK errors and similar is reduced, if not eliminated completely.

Of course there would be a case of a cache invalidation happening while a modification is running, but it is also unlikely, unless you have long running Unit of works (which I don't).

The refresh option on the query should, I think, only refresh the object in question, but will reset proxies to the other objects it contains. If those objects are still in cache, then when they're accessed they should be retrieved without a query. But that only applies to single objects, a collection would always require a query. And a weak cache might throw away the related objects before you used them again.

The refresh option works okay when I'm showing the details of a single object, I perform the query enforcing the refresh and It gets the right version.
But if I have to list a bunch of objects, I don't know whether they changed or not, and keeping track of the last time i queried the database or similar would be a waste of resources.

Another thing you can do if you're listing a bunch of objects in the UI is to do a query that doesn't read the objects at all, just the attributes you want to display. I forget the syntax now, but it was something like read:, but instead of specifying other objects, you just specify attributes. And if you don't ever read the main object, you just get it

aQuery

read: [:order | order orderNumber];

read: [:order | order amount ];

read: [:order | order status];

then your results aren't objects, they are arrays of three values. And if you want to look at the details of one, you can refresh it before opening a viewer.

I also note that I always use inUnitOfWorkDo: or similar mechanisms (e.g. transact:) if at all possible, i.e. when the thing I'm doing is a computation that fits inside a block, not some long-running thing.

I avoid long-running things, but I perform reads just by building a Query and executing it in the context of a session.
As far as I understand the Unit of Work is meant to be used to modify objects, because if perform the query inside the UOW it will traverse all the read objects to detect which one was modified. It doubles the time it takes to read.

Yes, I meant that I use it if I'm going to modify something. If you read inside a unit of work it will make copies of the objects that are read so that it can tell later which ones are modified by comparing to the objects. If you're not modifying them, there's no point. You just need to remember to register: the objects that you might modify when you do start doing modifications.

P.S. Glad you like alsoFetch: And if you set refresh on the query, it will also apply to the objects also being fetched

I'd like to fully understand how caching works in GLORP, I've debugged it a little, but more is required :)

In the meantime I implemented a clearCacheFor: aClass which basically releases a previous cache (if exists) in the CacheManager and replaces it with a new one. It's a sharp move, that could cause bleeding, but so far it hasn't, because I'm using it only at one point.

I'm pretty sure that the ability to do that is there somewhere already. But it may not be easy to find. I think maybe #reset on the cache object?

Thank you!

--
You received this message because you are subscribed to the Google Groups "glorp-group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/glorp-group.
For more options, visit https://groups.google.com/d/optout.

Esteban A. Maringolo

Re: Caching of objects (how to disable for certain classes)

El viernes, 30 de mayo de 2014 15:15:55 UTC-3, alan.knight escribió:

On Fri May 30 2014 at 10:45:49 AM, Esteban A. Maringolo <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="TbAh2moK5ecJ" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">emari...@...> wrote:

El viernes, 30 de mayo de 2014 12:47:22 UTC-3, alan.knight escribió:
There are lots of options, but it's a semantically complicated issue. Any time you have caching, you have the issue of when to update the cache. <a href="http://martinfowler.com/bliki/TwoHardThings.html" target="_blank" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fmartinfowler.com%2Fbliki%2FTwoHardThings.html\46sa\75D\46sntz\0751\46usg\75AFQjCNG39OLEOLIbg7Dv1UvJdz9GG31gzA';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fmartinfowler.com%2Fbliki%2FTwoHardThings.html\46sa\75D\46sntz\0751\46usg\75AFQjCNG39OLEOLIbg7Dv1UvJdz9GG31gzA';return true;">http://martinfowler.com/bliki/TwoHardThings.html

Knew that one. One of my favorites too. :)

Mechanisms you have in Glorp include
- refresh: you can ask the session to refresh an object, but it's also an option you can set on any query, so you can refresh a large set of objects

- timestamps / version numbers on objects. So you can e.g. query for objects you have but whose timestamp or version number is different from what you have and refresh those. But with an object graph this always gets semantically complicated. So, suppose the order has changed. That may mean that anything it contains has changed and needs to be refreshed. But what constitutes containment.

I would have to have a "last query" ts, though it requires me to have a "state machine" where I'm performing such queries.

If your rows have a lock column defined, then Glorp knows what the last timestamp/version number is and you can ask for it from the cache. It shouldn't be too difficult to write a query to ask

for all objects of class X, list the ones that I have in cache, or some other set, and whose timestamp is newer than the timestamp I have for them.

I don't know what exactly is a "lock column", it does have version fields for creation/modification ts.

I could use those.

But how would the query work?

Because I still have to query for the whole range of results.

The refresh option works okay when I'm showing the details of a single object, I perform the query enforcing the refresh and It gets the right version.
But if I have to list a bunch of objects, I don't know whether they changed or not, and keeping track of the last time i queried the database or similar would be a waste of resources.

Another thing you can do if you're listing a bunch of objects in the UI is to do a query that doesn't read the objects at all, just the attributes you want to display. I forget the syntax now, but it was something like read:, but instead of specifying other objects, you just specify attributes. And if you don't ever read the main object, you just get it

aQuery
read: [:order | order orderNumber];
read: [:order | order amount ];
read: [:order | order status];

then your results aren't objects, they are arrays of three values. And if you want to look at the details of one, you can refresh it before opening a viewer.

Yes, I know I can do that, but before that, I'm better implementing server side paging, it's not going to be blazing fast as client side (I'm using http://datatables.net/). But will certainly reduce the instantiation cost.

I also note that I always use inUnitOfWorkDo: or similar mechanisms (e.g. transact:) if at all possible, i.e. when the thing I'm doing is a computation that fits inside a block, not some long-running thing.

I avoid long-running things, but I perform reads just by building a Query and executing it in the context of a session.
As far as I understand the Unit of Work is meant to be used to modify objects, because if perform the query inside the UOW it will traverse all the read objects to detect which one was modified. It doubles the time it takes to read.

Yes, I meant that I use it if I'm going to modify something. If you read inside a unit of work it will make copies of the objects that are read so that it can tell later which ones are modified by comparing to the objects. If you're not modifying them, there's no point. You just need to remember to register: the objects that you might modify when you do start doing modifications.

P.S. Glad you like alsoFetch: And if you set refresh on the query, it will also apply to the objects also being fetched

I'd like to fully understand how caching works in GLORP, I've debugged it a little, but more is required :)

In the meantime I implemented a clearCacheFor: aClass which basically releases a previous cache (if exists) in the CacheManager and replaces it with a new one. It's a sharp move, that could cause bleeding, but so far it hasn't, because I'm using it only at one point.

I'm pretty sure that the ability to do that is there somewhere already. But it may not be easy to find. I think maybe #reset on the cache object?

The GLORP version in Pharo doesn't have a #reset method, it does have a #release one though, which delegates to the CachePolicy to release the cache. But the default policy does nothing, which turns #release into a double dispatch no-op. :)

Cache>>#release

policy release: self.

extraReferences := nil.

Thanks for your support.

Regards!

Alan Knight

Re: Caching of objects (how to disable for certain classes)

On Fri May 30 2014 at 4:43:01 PM, Esteban A. Maringolo <[hidden email]> wrote:

El viernes, 30 de mayo de 2014 15:15:55 UTC-3, alan.knight escribió:

On Fri May 30 2014 at 10:45:49 AM, Esteban A. Maringolo <[hidden email]> wrote:

El viernes, 30 de mayo de 2014 12:47:22 UTC-3, alan.knight escribió:
There are lots of options, but it's a semantically complicated issue. Any time you have caching, you have the issue of when to update the cache. http://martinfowler.com/bliki/TwoHardThings.html

Knew that one. One of my favorites too. :)

Mechanisms you have in Glorp include
- refresh: you can ask the session to refresh an object, but it's also an option you can set on any query, so you can refresh a large set of objects

- timestamps / version numbers on objects. So you can e.g. query for objects you have but whose timestamp or version number is different from what you have and refresh those. But with an object graph this always gets semantically complicated. So, suppose the order has changed. That may mean that anything it contains has changed and needs to be refreshed. But what constitutes containment.

I would have to have a "last query" ts, though it requires me to have a "state machine" where I'm performing such queries.

If your rows have a lock column defined, then Glorp knows what the last timestamp/version number is and you can ask for it from the cache. It shouldn't be too difficult to write a query to ask

for all objects of class X, list the ones that I have in cache, or some other set, and whose timestamp is newer than the timestamp I have for them.

I don't know what exactly is a "lock column", it does have version fields for creation/modification ts.

I could use those.

But how would the query work?
Because I still have to query for the whole range of results.

If you send beLockKey to a field when adding it to a table, then update queries will append the WHERE MY_VERSION_FIELD = expectedValue and check the rowcount. If it's not found, it throws an exception.

So you could write a query that knows the things you have in memory and queries for the things that are newer. Or you could write a query that retrieves the whole set but also reads the version field and checks for ones that are newer than what you have if you have the object in cache already. That would involve a little more diving into the guts. But for the first query, you'd just need to know enough to get the version field for the objects that you have in the cache.

The refresh option works okay when I'm showing the details of a single object, I perform the query enforcing the refresh and It gets the right version.

But if I have to list a bunch of objects, I don't know whether they changed or not, and keeping track of the last time i queried the database or similar would be a waste of resources.

Another thing you can do if you're listing a bunch of objects in the UI is to do a query that doesn't read the objects at all, just the attributes you want to display. I forget the syntax now, but it was something like read:, but instead of specifying other objects, you just specify attributes. And if you don't ever read the main object, you just get it

aQuery
read: [:order | order orderNumber];
read: [:order | order amount ];
read: [:order | order status];

then your results aren't objects, they are arrays of three values. And if you want to look at the details of one, you can refresh it before opening a viewer.

Yes, I know I can do that, but before that, I'm better implementing server side paging, it's not going to be blazing fast as client side (I'm using http://datatables.net/). But will certainly reduce the instantiation cost.

I also note that I always use inUnitOfWorkDo: or similar mechanisms (e.g. transact:) if at all possible, i.e. when the thing I'm doing is a computation that fits inside a block, not some long-running thing.

I avoid long-running things, but I perform reads just by building a Query and executing it in the context of a session.
As far as I understand the Unit of Work is meant to be used to modify objects, because if perform the query inside the UOW it will traverse all the read objects to detect which one was modified. It doubles the time it takes to read.

Yes, I meant that I use it if I'm going to modify something. If you read inside a unit of work it will make copies of the objects that are read so that it can tell later which ones are modified by comparing to the objects. If you're not modifying them, there's no point. You just need to remember to register: the objects that you might modify when you do start doing modifications.

P.S. Glad you like alsoFetch: And if you set refresh on the query, it will also apply to the objects also being fetched

I'd like to fully understand how caching works in GLORP, I've debugged it a little, but more is required :)

In the meantime I implemented a clearCacheFor: aClass which basically releases a previous cache (if exists) in the CacheManager and replaces it with a new one. It's a sharp move, that could cause bleeding, but so far it hasn't, because I'm using it only at one point.

I'm pretty sure that the ability to do that is there somewhere already. But it may not be easy to find. I think maybe #reset on the cache object?

The GLORP version in Pharo doesn't have a #reset method, it does have a #release one though, which delegates to the CachePolicy to release the cache. But the default policy does nothing, which turns #release into a double dispatch no-op. :)

Cache>>#release
policy release: self.
extraReferences := nil.

Yes, #reset is for the whole session, not for an individual cache. You might be able to send #initializeCache, but I'm not sure exactly what that would do. And I'm not sure exactly what the Pharo port did. I wish it had made more of an effort to be more compatible, or to contribute changes back to the dialect-independent code.

Thanks for your support.

Regards!

--
You received this message because you are subscribed to the Google Groups "glorp-group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/glorp-group.
For more options, visit https://groups.google.com/d/optout.

Esteban A. Maringolo

Re: Caching of objects (how to disable for certain classes)

El domingo, 1 de junio de 2014 02:34:05 UTC-3, alan.knight escribió:

I don't know what exactly is a "lock column", it does have version fields for creation/modification ts.

I could use those.

But how would the query work?
Because I still have to query for the whole range of results.

If you send beLockKey to a field when adding it to a table, then update queries will append the WHERE MY_VERSION_FIELD = expectedValue and check the rowcount. If it's not found, it throws an exception.

How did I miss this? :D

There is so much value in GLORP... :)

Yes, #reset is for the whole session, not for an individual cache. You might be able to send #initializeCache, but I'm not sure exactly what that would do. And I'm not sure exactly what the Pharo port did. I wish it had made more of an effort to be more compatible, or to contribute changes back to the dialect-independent code.

I don't know the whole story behind its porting, but I know it wasn't easy either. So probably some compromises where made.

We're planning to create a new port someday not too long in the future. But it would be good to use it and understand it as much as possible before performing any fileout-filein.

Regards!