DuplicatePrimaryKeyException

Hi,

The Task object I map to Glorp is a subclass off AssociationTreeWithParent

 

 

 

Smalltalk defineClass: #Task

superclass: #{UI.AssociationTreeWithParent}

 

As Tasks are linked together with various relations ships both parent child and different end to start relationships they and up with an almost unlimited level of proxied relationships.

 

Now when Glorp checks whether the Cache contains a Task Object with the method

 

 

 

includesKey: key as: anObject

"Return true if we include the object, and it matches the given object. If we include a different object with the same key, raise an exception. Don't listen to any expiry policy"

| item value |

item := self basicAt: key ifAbsent: [^false].

value := policy contentsOf: item.

value == anObject ifFalse: [

(DuplicatePrimaryKeyException new: anObject existing: value) signal].

^true.

 

The comparaison value == anObject seem to return false because of the lower level proxy differences. (and hash values are sometimes different to ?!)

 

 

This has given me lots of problems as most of the time I have to use refeshed reads in order not to have duplicate errors. Meaning creating a seperate Object with a refreshed read, updating it and then refreshing the original one. In the end instead of doing a single update I refesh a whole number of related things, then I update and then I reread all these things again, ending up very inefficient in database trafic having way to much code.

 

No if I read all the comments in "isNew: anObject " I potentially have the impression that I might have hit something unfinished here ?!?

 

isNew: anObject

"When registering, do we need to add this object to the collection of new objects? New objects are treated specially when computing what needs to be written, since we don't have their previous state"

 

| key descriptor |

(currentUnitOfWork notNil and: [currentUnitOfWork isRegistered: anObject]) ifTrue: [^false].

descriptor := self descriptorFor: anObject.

descriptor isNil ifTrue: [^false].

"For embedded values we assume that they are not new. This appears to work. I can't really justify it."

self needsWork: 'cross your fingers'.

descriptor mapsPrimaryKeys ifFalse: [^false].

 

key := descriptor primaryKeyFor: anObject.

 

key isNil ifTrue: [super halt.   ^true].

"If the cache contains the object, but the existing entry is due to be deleted, then count this entry as a new one being added with the same primary key (ick) as the old one"

^[(self cacheContainsObject: anObject key: key) not]

on: DuplicatePrimaryKeyException

do: [:ex |

(currentUnitOfWork notNil and: [currentUnitOfWork willDelete: ex existingObject])

ifTrue: [

self cacheRemoveObject: ex existingObject.

ex return: true]

ifFalse: [ex pass]].

 

I am now experimenting with = comparaisons instead of == comparisons, not sure this is the way to doit though.

Any hints ?

 

Regards,

 

@+Maarten,

 

 

 

Well the problem is what Tom indicated, I tried the hash thing but that doesn't give any improvement. I modified the cache policy and that seems to help a bit. I also simplified some of my descriptor systems and passed some operations over to a separate (simpler) mapping using the same table.

I don't really have the problem of not knowing my primary keys (as Anthony indicated), I can image this within a kind of web application where you might do a lot of things before actually registering but in my case I am on a distant desktop were other users can do annoying things at the same time.

All I want to do is to reduce traffic as much as possible while remaining as close as possible in sync with the db.

 

The thing is that I don't really know how this cache operates. Why is it that I can have 10 different versions of the same Object in my cache. ? Why is it that isRegistered: is a useless method ? What is the difference between realObject, registeredObject ExpiredObject etc.

 

In the next snippet I add newAv to a collection within aTask. aTask has just been updated and committed so totally up to date. However if I don't refresh: aTask before adding newAv (or crate another Task with the same key) it will provoke a duplicate error, but only sometimes ??.

 

newAv := Avancement

registerdate: Date today

percentage: aTask task_pct_comp

delta: aTask task_pct_comp / 100 * aTask task_work

projectKey: aTask proj_id

wbs_chain: aTask task_wbs_chain.

 

self getGlorpSession inUnitOfWorkDo:

[self getGlorpSession register: newAv.

self getGlorpSession register: (self getGlorpSession refresh: aTask).

aTask avancementcol add: newAv].

 

I suspect this to happen because of the multiple examples of the same Object that can reside within the cache. The lookup method will return whatever Object in the cache with the same key. If the object returned has the same pointer Glorp is happy. But if it is not the same pointer I will get an duplicate error … In order to avoid troubles I read object and all of its associated stuff which cost me maybe 30 milliseconds. Knowing that all this information is ready available within my system annoys me however, because I do this at hundreds of places.

 

The natural behavior for me would be that register: does an insert only when the primary key is nil. If the primary key is not nil it provokes an update if the object is available in the cache and otherwise checks the database of the key (row) actually exists.  If the object knows it primary key, is not in the cache and does not exists on the database it does an insert, all other cases its an update.

 

What do you think ??

 

Regards,

 

@+Maarten,

 

 

> "Tom Robinson" [hidden email] |

On 4/27/13 7:30 AM, [hidden email] wrote:

Hi,

The Task object I map to Glorp is a subclass off AssociationTreeWithParent

 

 

 

Smalltalk defineClass: #Task

superclass: #{UI.AssociationTreeWithParent}

 

As Tasks are linked together with various relations ships both parent child and different end to start relationships they and up with an almost unlimited level of proxied relationships.

 

Now when Glorp checks whether the Cache contains a Task Object with the method

 

 

 

includesKey: key as: anObject

"Return true if we include the object, and it matches the given object. If we include a different object with the same key, raise an exception. Don't listen to any expiry policy"

| item value |

item := self basicAt: key ifAbsent: [^false].

value := policy contentsOf: item.

value == anObject ifFalse: [

(DuplicatePrimaryKeyException new: anObject existing: value) signal].

^true.

 

The comparaison value == anObject seem to return false because of the lower level proxy differences. (and hash values are sometimes different to ?!)

If you remove a child, do you want the Task with the child to be = to the Task without the child? If you change the parent or the predecessor or successor, do you want the Task before the change to be equal to the one after the change? I would suggest, assuming that there is a bunch of Task related info inside the Task object, that you want it to be equal to itself even if the relationships change. That suggests that you may want to look at implementing = and hash on your Task class such that both exclude the parent(s), children, predecessor, successor, etc from their calculations.  This may mean that you need other methods that do comparisions between 2 tasks that are =, but have different relationships, but that depends on your application.

It seems to me that the relationships *can't* be included in = or hash because what the Cache is trying to figure out is "Does this object represent a row in the database that I have loaded and turned into an object?". The only values that can be included in the = or hash calculation are ones that, if changed, would require that you write a new row to the database and delete the old one, I think.

 

 

This has given me lots of problems as most of the time I have to use refeshed reads in order not to have duplicate errors. Meaning creating a seperate Object with a refreshed read, updating it and then refreshing the original one. In the end instead of doing a single update I refesh a whole number of related things, then I update and then I reread all these things again, ending up very inefficient in database trafic having way to much code.

 

No if I read all the comments in "isNew: anObject " I potentially have the impression that I might have hit something unfinished here ?!?

Probably the only person who could speak to that question definitively is Alan Knight...

 

isNew: anObject

"When registering, do we need to add this object to the collection of new objects? New objects are treated specially when computing what needs to be written, since we don't have their previous state"

 

| key descriptor |

(currentUnitOfWork notNil and: [currentUnitOfWork isRegistered: anObject]) ifTrue: [^false].

descriptor := self descriptorFor: anObject.

descriptor isNil ifTrue: [^false].

"For embedded values we assume that they are not new. This appears to work. I can't really justify it."

self needsWork: 'cross your fingers'.

descriptor mapsPrimaryKeys ifFalse: [^false].

 

key := descriptor primaryKeyFor: anObject.

 

key isNil ifTrue: [super halt.   ^true].

"If the cache contains the object, but the existing entry is due to be deleted, then count this entry as a new one being added with the same primary key (ick) as the old one"

^[(self cacheContainsObject: anObject key: key) not]

on: DuplicatePrimaryKeyException

do: [:ex |

(currentUnitOfWork notNil and: [currentUnitOfWork willDelete: ex existingObject])

ifTrue: [

self cacheRemoveObject: ex existingObject.

ex return: true]

ifFalse: [ex pass]].

 

I am now experimenting with = comparaisons instead of == comparisons, not sure this is the way to doit though.

Any hints ?

 

Regards,

 

@+Maarten,

 

 

 

Well the problem is what Tom indicated, I tried the hash thing but that doesn't give any improvement. I modified the cache policy and that seems to help a bit. I also simplified some of my descriptor systems and passed some operations over to a separate (simpler) mapping using the same table.

I don't really have the problem of not knowing my primary keys (as Anthony indicated), I can image this within a kind of web application where you might do a lot of things before actually registering but in my case I am on a distant desktop were other users can do annoying things at the same time.

All I want to do is to reduce traffic as much as possible while remaining as close as possible in sync with the db.

 

The thing is that I don't really know how this cache operates. Why is it that I can have 10 different versions of the same Object in my cache. ? Why is it that isRegistered: is a useless method ? What is the difference between realObject, registeredObject ExpiredObject etc.

 

In the next snippet I add newAv to a collection within aTask. aTask has just been updated and committed so totally up to date. However if I don't refresh: aTask before adding newAv (or crate another Task with the same key) it will provoke a duplicate error, but only sometimes ??.

 

newAv := Avancement

registerdate: Date today

percentage: aTask task_pct_comp

delta: aTask task_pct_comp / 100 * aTask task_work

projectKey: aTask proj_id

wbs_chain: aTask task_wbs_chain.

 

self getGlorpSession inUnitOfWorkDo:

[self getGlorpSession register: newAv.

self getGlorpSession register: (self getGlorpSession refresh: aTask).

aTask avancementcol add: newAv].

 

I suspect this to happen because of the multiple examples of the same Object that can reside within the cache. The lookup method will return whatever Object in the cache with the same key. If the object returned has the same pointer Glorp is happy. But if it is not the same pointer I will get an duplicate error … In order to avoid troubles I read object and all of its associated stuff which cost me maybe 30 milliseconds. Knowing that all this information is ready available within my system annoys me however, because I do this at hundreds of places.

 

The natural behavior for me would be that register: does an insert only when the primary key is nil. If the primary key is not nil it provokes an update if the object is available in the cache and otherwise checks the database of the key (row) actually exists.  If the object knows it primary key, is not in the cache and does not exists on the database it does an insert, all other cases its an update.

 

What do you think ??

 

Regards,

 

@+Maarten,

 

 

> "Tom Robinson" [hidden email] |

On 4/27/13 7:30 AM, [hidden email] wrote:

Hi,

The Task object I map to Glorp is a subclass off AssociationTreeWithParent

 

 

 

Smalltalk defineClass: #Task

superclass: #{UI.AssociationTreeWithParent}

 

As Tasks are linked together with various relations ships both parent child and different end to start relationships they and up with an almost unlimited level of proxied relationships.

 

Now when Glorp checks whether the Cache contains a Task Object with the method

 

 

 

includesKey: key as: anObject

"Return true if we include the object, and it matches the given object. If we include a different object with the same key, raise an exception. Don't listen to any expiry policy"

| item value |

item := self basicAt: key ifAbsent: [^false].

value := policy contentsOf: item.

value == anObject ifFalse: [

(DuplicatePrimaryKeyException new: anObject existing: value) signal].

^true.

 

The comparaison value == anObject seem to return false because of the lower level proxy differences. (and hash values are sometimes different to ?!)

If you remove a child, do you want the Task with the child to be = to the Task without the child? If you change the parent or the predecessor or successor, do you want the Task before the change to be equal to the one after the change? I would suggest, assuming that there is a bunch of Task related info inside the Task object, that you want it to be equal to itself even if the relationships change. That suggests that you may want to look at implementing = and hash on your Task class such that both exclude the parent(s), children, predecessor, successor, etc from their calculations.  This may mean that you need other methods that do comparisions between 2 tasks that are =, but have different relationships, but that depends on your application.

It seems to me that the relationships *can't* be included in = or hash because what the Cache is trying to figure out is "Does this object represent a row in the database that I have loaded and turned into an object?". The only values that can be included in the = or hash calculation are ones that, if changed, would require that you write a new row to the database and delete the old one, I think.

 

 

This has given me lots of problems as most of the time I have to use refeshed reads in order not to have duplicate errors. Meaning creating a seperate Object with a refreshed read, updating it and then refreshing the original one. In the end instead of doing a single update I refesh a whole number of related things, then I update and then I reread all these things again, ending up very inefficient in database trafic having way to much code.

 

No if I read all the comments in "isNew: anObject " I potentially have the impression that I might have hit something unfinished here ?!?

Probably the only person who could speak to that question definitively is Alan Knight...

 

isNew: anObject

"When registering, do we need to add this object to the collection of new objects? New objects are treated specially when computing what needs to be written, since we don't have their previous state"

 

| key descriptor |

(currentUnitOfWork notNil and: [currentUnitOfWork isRegistered: anObject]) ifTrue: [^false].

descriptor := self descriptorFor: anObject.

descriptor isNil ifTrue: [^false].

"For embedded values we assume that they are not new. This appears to work. I can't really justify it."

self needsWork: 'cross your fingers'.

descriptor mapsPrimaryKeys ifFalse: [^false].

 

key := descriptor primaryKeyFor: anObject.

 

key isNil ifTrue: [super halt.   ^true].

"If the cache contains the object, but the existing entry is due to be deleted, then count this entry as a new one being added with the same primary key (ick) as the old one"

^[(self cacheContainsObject: anObject key: key) not]

on: DuplicatePrimaryKeyException

do: [:ex |

(currentUnitOfWork notNil and: [currentUnitOfWork willDelete: ex existingObject])

ifTrue: [

self cacheRemoveObject: ex existingObject.

ex return: true]

ifFalse: [ex pass]].

 

I am now experimenting with = comparaisons instead of == comparisons, not sure this is the way to doit though.

Any hints ?

 

Regards,

 

@+Maarten,