Smalltalk - Re: Large collection and common practice

Smalltalk › Gemtalk › GLASS

Re: Large collection and common practice

Posted by GLASS mailing list on Jan 03, 2017; 8:21pm
URL: https://forum.world.st/Large-collection-and-common-practice-tp4928607p4928622.html

Hi Bruno,

Multiple sessions can feed an RC collection with reduced commit conflicts, but you don't always want to keep the objects in the RC collection. One common technique is to have a manager session dedicated to moving objects from RC collections into collections that can be accessed more efficiently. Design so that the manager is the only session that will be updating the collections (so that commit conflicts will not happen). The manager session can do polling for new items and you can add gem-to-gem signaling to wake the manager for more timely responses. The challenges with this kind of design are related to update timing between sessions. The process involves a commit to add to the RC collection, an abort for the manager session to see the objects, a commit by the manager to update the root collection (with RC collection removal, RcQueues are usually used BTW), and an abort by the original session if it needs to see the indexed item was added to the root collection. If there is timing sensitivity with this data then you'll likely resort to searching first in your indexeded collection and then also reviewing objects still in the queue waiting for the manager to process them.

A variation of the manager session technique is to send data to the manager session without doing a commit, this might be through communication between gems or by using session-specific file updates that the manager gem reads. Gem-to-gem signaling can be added to this approach later too if you need to improve timing. This variation can avoid the intermediate commit, but you'd still may need to #continueTransaction to see what the manager session updated.

I wonder what kind of indexing you would need besides ID. If you don't need to query for anything other than ID then a dictionary would be fine with the ID as key. A dictionary can even use a key that is a custom object that redefines equality and hash from attributes of what is searched for. Merkle tree hashes might also be used as a way to test if some attribute is contained, but that is a bit advanced to go into. Another advanced item that I once implemented was a custom Dictionary where the key was derived from the value by behavior (it was more efficient because it avoided the cost of Association creation). So many cool tricks, I loved working with GS/S.

Paul Baumann

On Tue, Jan 3, 2017 at 2:39 PM, Mariano Martinez Peck via Glass <[hidden email]> wrote:

On Tue, Jan 3, 2017 at 3:14 PM, BrunoBB via Glass <[hidden email]> wrote:
Hi All,

I have a lot RcKeyValueDictionary where the key is the id of the object and
the value is the object itself.
This id once assigned it does NOT change, so far so good :)

The RcKeyValueDictionary is used intensively to add and remove objects (in
my case OrbeonFormInstance). The dictionary is very useful because the key
is always given as parameter.

Also there are searchs by specific inst var of OrbeonFormInstance class
(like username,group, createdTime and so on).

My problem is that i can NOT create an index on aRcKeyValueDictionary.
So which is the commom practice in these cases:
1- Change the RcKeyValueDictionary to be an UnorderedCollection ?
2- Add a new instance variable to the class that holds the
RcKeyValueDictionary and this new variable to be anUnorderedCollection ?

1) This will complicate my direct searchs using the ID.
2) Extra computation when adding and removing objects (now there 2
collections to maintain)

The general question will be something like:
When Dictionaries are very suitable to store large quantity of objects but
indexes are also needed which solution should be implemented ?

Assuming you do need or get benefits from the RC flavor (else it brings unnecessary overhead), then quickly analyzing the situation (until GemStone have indexed and rc-flavor Dictionary impl), I think I would use a RcIdentityBag. I would create a identity index for #id , and yes, you will have to modify your code that access the dict, to know detect on the collection using the identity index of ID.

But...I am sure someone more experienced will come with a better approach!

Cheers,

regards
bruno

--
View this message in context: http://forum.world.st/Large-collection-and-common-practice-tp4928607.html
Sent from the GLASS mailing list archive at Nabble.com.
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--
Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass