About MagmaPreallocatedDictionary

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

About MagmaPreallocatedDictionary

Chris Muller-3
It is important to understand the nuances of using a
MagmaPreallocatedDictionary.

I must admit I intended it only for myself at the time I wrote it
under a tight time-constraint; and so did not have time to develop
what I really wanted..

Note the adjective, "Preallocated", it means all the hash-slots of the
Dictionary are pre-allocated.  This dictionary will not grow anymore.
So this means there is a cap to how large it will scale before
collisions are inevitable, which would rapidly degrade performance.

Thankfully, the size of the support file on disk is actually only as
large as needs to be to accommodate the *highest-used* hash value.
Still, you will want to assume the highest-possible hash value could
be used.

It's 6-bytes per object-pointer so the file is

  6 * [highest-used-hash-value]

bytes in size.  This makes the file on the server a many megabytes
right out of the gate, but it won't grow again until a higher hash is
used.

It is absolutely _essential_ that you do not allow any identityHash to
be part of your hash-calculation.  Otherwise, it won't work;
identityHash'es are unique between image sessions.

Ultimately, MagmaPreallocatedDictionary will be deprecated.  I would
like to replace it with a new-and-improved MagmaDictionary based on
the same concept (employing an underlying MagmaArray), but with a
better hashing algorithm that allows expansion without rehashing the
elements.

 - Chris
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma
Reply | Threaded
Open this post in threaded view
|

Re: About MagmaPreallocatedDictionary

fvozzi
Hi Chris,
thanks for your advice. I'll replace it for Dictionary or MagmaDictionary, It isn't going to have  more than 20000 keys for now.

Do you have a roadmap for the next magma version?

See you,
Facu

On Thu, Dec 16, 2010 at 7:42 PM, Chris Muller <[hidden email]> wrote:
It is important to understand the nuances of using a
MagmaPreallocatedDictionary.

I must admit I intended it only for myself at the time I wrote it
under a tight time-constraint; and so did not have time to develop
what I really wanted..

Note the adjective, "Preallocated", it means all the hash-slots of the
Dictionary are pre-allocated.  This dictionary will not grow anymore.
So this means there is a cap to how large it will scale before
collisions are inevitable, which would rapidly degrade performance.

Thankfully, the size of the support file on disk is actually only as
large as needs to be to accommodate the *highest-used* hash value.
Still, you will want to assume the highest-possible hash value could
be used.

It's 6-bytes per object-pointer so the file is

 6 * [highest-used-hash-value]

bytes in size.  This makes the file on the server a many megabytes
right out of the gate, but it won't grow again until a higher hash is
used.

It is absolutely _essential_ that you do not allow any identityHash to
be part of your hash-calculation.  Otherwise, it won't work;
identityHash'es are unique between image sessions.

Ultimately, MagmaPreallocatedDictionary will be deprecated.  I would
like to replace it with a new-and-improved MagmaDictionary based on
the same concept (employing an underlying MagmaArray), but with a
better hashing algorithm that allows expansion without rehashing the
elements.

 - Chris
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma


_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma