Identity hashes bigger than maximum
Locked
 
|
In VisualWorks 7.8, there are two identityHash methods that return
values larger than ObjectMemory maximumIdentityHashValue. Store.Glorp.StoreBlob >> identityHash Store.Glorp.StoreObject >> identityHash They combine the identityHash with a regular hash to produce a result that's the size of a SmallInteger. Is there a reason for this? David _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: Identity hashes bigger than maximum
|
|
The reason is to be able
to store hashed collections with more than 14 bits worth of keys in
them. In fact, when I wrote that I wasn't even aware that there was a
maximumIdentityHashValue method. Regardless, it seems to me that it's
fine, as the maximumIdentityHashValue really serves to identify the
maximum identity hash that you would get from the object's header.
Returning a larger value doesn't hurt anything, as long as you obey the
important rule that the identity hash shouldn't change or any hashed
collections using that information will no longer find the object. In
fact, when that issue was previously raised, Andres made the comment
that the identityHash didn't even particularly have to be an integer.
Is there a problem you're running into because of this?
_______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: Identity hashes bigger than maximum
|
|
In reply to this post by davidbuck
We need a larger identity hash. You get too many collisions in
IdentityDictionary with the standard identity hash when you put tens of thousands of objects into the dictionary. Terry =========================================================== Terry Raymond Crafted Smalltalk 80 Lazywood Ln. Tiverton, RI 02878 (401) 624-4517 [hidden email] =========================================================== > -----Original Message----- > From: [hidden email] [mailto:[hidden email]] On > Behalf Of David Buck > Sent: Friday, December 16, 2011 9:43 AM > To: 'VWNC' > Subject: [vwnc] Identity hashes bigger than maximum > > In VisualWorks 7.8, there are two identityHash methods that return values > larger than ObjectMemory maximumIdentityHashValue. > > Store.Glorp.StoreBlob >> identityHash > Store.Glorp.StoreObject >> identityHash > > They combine the identityHash with a regular hash to produce a result > the size of a SmallInteger. Is there a reason for this? > > David > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
In reply to this post by Alan Knight-2
You would think that this mechanism only affects GemStone objects (which Store records obviously aren't), but GBS overrides some Trippy functionality to always check to see if the objects being inspected are in the GemStone cache and to use a special wrapper for them if they are. The upshot of this is that when you inspect a Store object, you get an index out of bounds exception in the GemStone cache. My solution is to change the GemBuilder cache to calculate key identityHash \\ self basicSize + 1 when looking up buckets rather than using the raw identity hash. The 64 bit version of the cache works differently and isn't susceptible to this problem. I'll have to submit this problem to GemStone for them to fix officially since it's outside the scope of GemKit itself. Thanks David Alan Knight wrote: The reason is to be able to store hashed collections with more than 14 bits worth of keys in them. In fact, when I wrote that I wasn't even aware that there was a maximumIdentityHashValue method. Regardless, it seems to me that it's fine, as the maximumIdentityHashValue really serves to identify the maximum identity hash that you would get from the object's header. Returning a larger value doesn't hurt anything, as long as you obey the important rule that the identity hash shouldn't change or any hashed collections using that information will no longer find the object. In fact, when that issue was previously raised, Andres made the comment that the identityHash didn't even particularly have to be an integer.* _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: Identity hashes bigger than maximum
|
|
In reply to this post by Terry Raymond
Terry Raymond wrote:
> We need a larger identity hash. You get too many collisions in
> IdentityDictionary with the standard identity hash when you > put tens of thousands of objects into the dictionary.
Yes. To quote myself from 2006: In the 64-bit versions of VW, the identity hash value is increased from 14 bits to 20 bits: 16,384 to 1,048,576.
http://www.cincomsmalltalk.com/blog/blogView?entry=3281185978 The older hash discussion in the UIUC Wiki is also interesting: http://web.archive.org/web/20050420081733/http://wiki.cs.uiuc.edu/VisualWorks/Hashing+methods+of+VW and if you enjoyed that, the full 2001 discussion from comp.lang.smalltalk is here: (I came over all nostalgic when reading that!)
Of course much VW hashing has improved since then (thanks Andres!), but identityHash remains problematic.
Steve _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: Identity hashes bigger than maximum
|
|
In reply to this post by davidbuck
We did that for better performance in some cases. Hopefully you have
not run into a problem with it. Also, note that this is hardly the first example of objects that answer values greater than ObjectMemory maximumIdentityHashValue when sent identityHash, e.g.: SmallInteger. On 12/16/2011 6:42 AM, David Buck wrote: > In VisualWorks 7.8, there are two identityHash methods that return > values larger than ObjectMemory maximumIdentityHashValue. > > Store.Glorp.StoreBlob>> identityHash > Store.Glorp.StoreObject>> identityHash > > They combine the identityHash with a regular hash to produce a result > that's the size of a SmallInteger. Is there a reason for this? > > David > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: Identity hashes bigger than maximum
|
|
In reply to this post by Steven Kelly
Generally speaking, it's not so much of a problem that identityHash is
comparatively limited. It's worse that Set / IdentitySet try to guess the origin of an identityHash value and make decisions accordingly. Sometimes the results are rather bad. This is something we wanted to fix for 7.9, but it has been pushed to a subsequent release because of other important work in the scavenger (to make the story short, the scavenger has been rewritten). On 12/16/2011 3:48 PM, Steven Kelly wrote: > Terry Raymond wrote: > > We need a larger identity hash. You get too many collisions in > > IdentityDictionary with the standard identity hash when you > > put tens of thousands of objects into the dictionary. > Yes. To quote myself from 2006: In the 64-bit versions of VW, the > identity hash value is increased from 14 bits to 20 bits: 16,384 to > 1,048,576. > http://www.cincomsmalltalk.com/blog/blogView?entry=3281185978 > > The older hash discussion in the UIUC Wiki is also interesting: > http://web.archive.org/web/20050420081733/http://wiki.cs.uiuc.edu/VisualWorks/Hashing+methods+of+VW > and if you enjoyed that, the full 2001 discussion from > comp.lang.smalltalk is here: > http://groups.google.com/group/comp.lang.smalltalk/browse_thread/thread/da42af48c7cc934f/60521b7e06c99c6a > (I came over all nostalgic when reading that!) > Of course much VW hashing has improved since then (thanks Andres!), but > identityHash remains problematic. > Steve > > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: Identity hashes bigger than maximum
|
|
In reply to this post by davidbuck
On 12/16/2011 10:13 AM, David Buck wrote:
> GemBuilder/S for VisualWorks maintains a cache of client objects in a > class called GbxClientObjDictionary32. The problem is that it caches > objects into buckets that are stored in 16383 indexed instance variables > in the object. It uses the identityHash to locate the bucket then > searches within the bucket to find the instance. Hi David, Thanks for the report. I've filed a bug on this; GBS shouldn't be relying on #identityHash only being a 14-bit value for all non-special Objects. [...] > > My solution is to change the GemBuilder cache to calculate > > key identityHash \\ self basicSize + 1 That will work, but slows down the common case in order to handle the uncommon case. Our fix will probably define Object>>gbxIdentityHash, guaranteed to answer a 14-bit value, and use that for the 32-bit cache. Regards, -Martin _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
«
Return to VisualWorks
|
1 view|%1 views
| Free forum by Nabble | Edit this page |