difference between MaDictionary and MagmaDictionary

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

difference between MaDictionary and MagmaDictionary

EstebanLM
Hi,
I wonder what's the difference between MaDictionary and MagmaDictionary... and which should I use... is indistinct?

Cheers,
Esteban_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma
Reply | Threaded
Open this post in threaded view
|

Re: difference between MaDictionary and MagmaDictionary

Igor Stasenko
On 14 December 2010 01:40, Esteban Lorenzano <[hidden email]> wrote:
> Hi,
> I wonder what's the difference between MaDictionary and MagmaDictionary... and which should I use... is indistinct?
>

I think you better do not use these dicts. They are mainly for
internal use only.
They are optimized for speed not for space. And taking part in keeping
correspondence between
objects stored in database and their ids.

> Cheers,
> Esteban_______________________________________________
> Magma mailing list
> [hidden email]
> http://lists.squeakfoundation.org/mailman/listinfo/magma
>



--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma
Reply | Threaded
Open this post in threaded view
|

Re: difference between MaDictionary and MagmaDictionary

fvozzi
In reply to this post by EstebanLM
Hi,
I'm using MagmaPreallocatedDictionary and it works fine and fast. I think that you must use it.

Facu

On Mon, Dec 13, 2010 at 9:40 PM, Esteban Lorenzano <[hidden email]> wrote:
Hi,
I wonder what's the difference between MaDictionary and MagmaDictionary... and which should I use... is indistinct?

Cheers,
Esteban_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma


_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma
Reply | Threaded
Open this post in threaded view
|

Re: difference between MaDictionary and MagmaDictionary

EstebanLM
In reply to this post by Igor Stasenko
mmm...
AFAIU the objective of using magma collections is to avoid the read/write of big collections.... and I have this structure:

MaDictionary
        key -> MagmaCollection (big)
        key -> MagmaCollection (big)
        key -> MagmaCollection (big)

If I replace MaDictionary with Dictionary, I will be reading/writing the full dictionary each time I add an element to the dictionary... isn't?

btw... I'm using an unique MagmaSession to connect to repository (no real need of other... no more than 3 users connected at the same time)... and the real reason I'm asking this is because my application continues crashing each one or two days, and I can't figure out what's happening :(
So, any "best practice" I can use would be really welcome (I'm a little bit lost here... I'm new to magma in production, used just for toy project before this :( )

Cheers,
Esteban

El 14/12/2010, a las 6:12a.m., Igor Stasenko escribió:

> On 14 December 2010 01:40, Esteban Lorenzano <[hidden email]> wrote:
>> Hi,
>> I wonder what's the difference between MaDictionary and MagmaDictionary... and which should I use... is indistinct?
>>
>
> I think you better do not use these dicts. They are mainly for
> internal use only.
> They are optimized for speed not for space. And taking part in keeping
> correspondence between
> objects stored in database and their ids.
>
>> Cheers,
>> Esteban_______________________________________________
>> Magma mailing list
>> [hidden email]
>> http://lists.squeakfoundation.org/mailman/listinfo/magma
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma
Reply | Threaded
Open this post in threaded view
|

Re: difference between MaDictionary and MagmaDictionary

fvozzi


On Tue, Dec 14, 2010 at 9:29 AM, Esteban Lorenzano <[hidden email]> wrote:
mmm...
AFAIU the objective of using magma collections is to avoid the read/write of big collections.... and I have this structure:

MaDictionary
       key -> MagmaCollection (big)
       key -> MagmaCollection (big)
       key -> MagmaCollection (big)

If I replace MaDictionary with Dictionary, I will be reading/writing the full dictionary each time I add an element to the dictionary... isn't?

I think that you need replace MaDictionary with MagmaPreallocatedDictionary not with standard Dictionary.
 

btw... I'm using an unique MagmaSession to connect to repository (no real need of other... no more than 3 users connected at the same time)... and the real reason I'm asking this is because my application continues crashing each one or two days, and I can't figure out what's happening :(

Do you think that MaDicitonary is causing your crash ? 
 
So, any "best practice" I can use would be really welcome (I'm a little bit lost here... I'm new to magma in production, used just for toy project before this :( )

I'm using a session peer user, my root is my singleton (ReportSystem unique instance) and I don't have MaDictionary. I'm using MagmaPreallocatedDictionary and MagmaCollection with standard OrderedCollection for many small objects collections on my model.
We're testing with no more than 3 concurrent users (for now) but I didn't have any crash so far.


Cheers,
Esteban

See you,
Facu 

El 14/12/2010, a las 6:12a.m., Igor Stasenko escribió:

> On 14 December 2010 01:40, Esteban Lorenzano <[hidden email]> wrote:
>> Hi,
>> I wonder what's the difference between MaDictionary and MagmaDictionary... and which should I use... is indistinct?
>>
>
> I think you better do not use these dicts. They are mainly for
> internal use only.
> They are optimized for speed not for space. And taking part in keeping
> correspondence between
> objects stored in database and their ids.
>
>> Cheers,
>> Esteban_______________________________________________
>> Magma mailing list
>> [hidden email]
>> http://lists.squeakfoundation.org/mailman/listinfo/magma
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma


_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma
Reply | Threaded
Open this post in threaded view
|

Re: difference between MaDictionary and MagmaDictionary

Chris Muller-3
In reply to this post by Igor Stasenko
Igor is right; the MaDictionray hierarchy is intended for internal
use, however I see no problem with using a persistent MaDictionary.

However, there is no inherent advantage to doing so; so if someone
feels compelled to use it I'd like to know why.

 - Chris

On Tue, Dec 14, 2010 at 3:12 AM, Igor Stasenko <[hidden email]> wrote:

> On 14 December 2010 01:40, Esteban Lorenzano <[hidden email]> wrote:
>> Hi,
>> I wonder what's the difference between MaDictionary and MagmaDictionary... and which should I use... is indistinct?
>>
>
> I think you better do not use these dicts. They are mainly for
> internal use only.
> They are optimized for speed not for space. And taking part in keeping
> correspondence between
> objects stored in database and their ids.
>
>> Cheers,
>> Esteban_______________________________________________
>> Magma mailing list
>> [hidden email]
>> http://lists.squeakfoundation.org/mailman/listinfo/magma
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
> _______________________________________________
> Magma mailing list
> [hidden email]
> http://lists.squeakfoundation.org/mailman/listinfo/magma
>
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma
Reply | Threaded
Open this post in threaded view
|

Re: difference between MaDictionary and MagmaDictionary

Chris Muller-3
In reply to this post by EstebanLM
Esteban, check :

  MaObjectSerializerPreferences weakIdentityKeyDictionaryClass

and make sure it answers " MaWeakIdentityKeyDictionary" and not
anything with the word "Auto" in it.

If it does, then:

  MaObjectSerializerPreferences weakIdentityKeyDictionaryClass:
MaWeakIdentityKeyDictionary

and restart your entire application (new server and sessions).  See if
your crashing goes away.

 - Chris

On Tue, Dec 14, 2010 at 6:29 AM, Esteban Lorenzano <[hidden email]> wrote:

> mmm...
> AFAIU the objective of using magma collections is to avoid the read/write of big collections.... and I have this structure:
>
> MaDictionary
>        key -> MagmaCollection (big)
>        key -> MagmaCollection (big)
>        key -> MagmaCollection (big)
>
> If I replace MaDictionary with Dictionary, I will be reading/writing the full dictionary each time I add an element to the dictionary... isn't?
>
> btw... I'm using an unique MagmaSession to connect to repository (no real need of other... no more than 3 users connected at the same time)... and the real reason I'm asking this is because my application continues crashing each one or two days, and I can't figure out what's happening :(
> So, any "best practice" I can use would be really welcome (I'm a little bit lost here... I'm new to magma in production, used just for toy project before this :( )
>
> Cheers,
> Esteban
>
> El 14/12/2010, a las 6:12a.m., Igor Stasenko escribió:
>
>> On 14 December 2010 01:40, Esteban Lorenzano <[hidden email]> wrote:
>>> Hi,
>>> I wonder what's the difference between MaDictionary and MagmaDictionary... and which should I use... is indistinct?
>>>
>>
>> I think you better do not use these dicts. They are mainly for
>> internal use only.
>> They are optimized for speed not for space. And taking part in keeping
>> correspondence between
>> objects stored in database and their ids.
>>
>>> Cheers,
>>> Esteban_______________________________________________
>>> Magma mailing list
>>> [hidden email]
>>> http://lists.squeakfoundation.org/mailman/listinfo/magma
>>>
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>
> _______________________________________________
> Magma mailing list
> [hidden email]
> http://lists.squeakfoundation.org/mailman/listinfo/magma
>
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma
Reply | Threaded
Open this post in threaded view
|

Re: difference between MaDictionary and MagmaDictionary

Chris Muller-3
In reply to this post by EstebanLM
> AFAIU the objective of using magma collections is to avoid the read/write of big collections.... and I have this structure:
>
> MaDictionary
>        key -> MagmaCollection (big)
>        key -> MagmaCollection (big)
>        key -> MagmaCollection (big)

Esteban, Please pardon my saying, but you should "make an object" to
replace of these layers of Dictionary's...    :-)

 - Chris
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma
Reply | Threaded
Open this post in threaded view
|

Re: difference between MaDictionary and MagmaDictionary

Chris Muller-3
In reply to this post by fvozzi
>> If I replace MaDictionary with Dictionary, I will be reading/writing the
>> full dictionary each time I add an element to the dictionary... isn't?

Yes, that's true.  But a MaDictionary only saves you that if you have
a lot of collisions.

> I think that you need replace MaDictionary with MagmaPreallocatedDictionary
> not with standard Dictionary.

No, this is wrong, please see how MagmaPreallocatedDictionary works.
It is for very large dictionary's, Esteban is using a MaDictionary for
his root object, and my guess is it only has a few entries in it..

 - Chris
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma
Reply | Threaded
Open this post in threaded view
|

Re: difference between MaDictionary and MagmaDictionary

EstebanLM
In reply to this post by Chris Muller-3
no, it is MaWeakIdentityKeyDictionary :(
(and my application is still crashing... today again)

cheers,
Esteban
 
El 14/12/2010, a las 5:32p.m., Chris Muller escribió:

> Esteban, check :
>
>  MaObjectSerializerPreferences weakIdentityKeyDictionaryClass
>
> and make sure it answers " MaWeakIdentityKeyDictionary" and not
> anything with the word "Auto" in it.
>
> If it does, then:
>
>  MaObjectSerializerPreferences weakIdentityKeyDictionaryClass:
> MaWeakIdentityKeyDictionary
>
> and restart your entire application (new server and sessions).  See if
> your crashing goes away.
>
> - Chris
>
> On Tue, Dec 14, 2010 at 6:29 AM, Esteban Lorenzano <[hidden email]> wrote:
>> mmm...
>> AFAIU the objective of using magma collections is to avoid the read/write of big collections.... and I have this structure:
>>
>> MaDictionary
>>        key -> MagmaCollection (big)
>>        key -> MagmaCollection (big)
>>        key -> MagmaCollection (big)
>>
>> If I replace MaDictionary with Dictionary, I will be reading/writing the full dictionary each time I add an element to the dictionary... isn't?
>>
>> btw... I'm using an unique MagmaSession to connect to repository (no real need of other... no more than 3 users connected at the same time)... and the real reason I'm asking this is because my application continues crashing each one or two days, and I can't figure out what's happening :(
>> So, any "best practice" I can use would be really welcome (I'm a little bit lost here... I'm new to magma in production, used just for toy project before this :( )
>>
>> Cheers,
>> Esteban
>>
>> El 14/12/2010, a las 6:12a.m., Igor Stasenko escribió:
>>
>>> On 14 December 2010 01:40, Esteban Lorenzano <[hidden email]> wrote:
>>>> Hi,
>>>> I wonder what's the difference between MaDictionary and MagmaDictionary... and which should I use... is indistinct?
>>>>
>>>
>>> I think you better do not use these dicts. They are mainly for
>>> internal use only.
>>> They are optimized for speed not for space. And taking part in keeping
>>> correspondence between
>>> objects stored in database and their ids.
>>>
>>>> Cheers,
>>>> Esteban_______________________________________________
>>>> Magma mailing list
>>>> [hidden email]
>>>> http://lists.squeakfoundation.org/mailman/listinfo/magma
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko AKA sig.
>>
>> _______________________________________________
>> Magma mailing list
>> [hidden email]
>> http://lists.squeakfoundation.org/mailman/listinfo/magma
>>

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma
Reply | Threaded
Open this post in threaded view
|

Re: difference between MaDictionary and MagmaDictionary

EstebanLM
In reply to this post by Chris Muller-3
ok, and what about using MagmaDictionary? better? or the same?

cheers,
Esteban

El 14/12/2010, a las 5:40p.m., Chris Muller escribió:

>>> If I replace MaDictionary with Dictionary, I will be reading/writing the
>>> full dictionary each time I add an element to the dictionary... isn't?
>
> Yes, that's true.  But a MaDictionary only saves you that if you have
> a lot of collisions.
>
>> I think that you need replace MaDictionary with MagmaPreallocatedDictionary
>> not with standard Dictionary.
>
> No, this is wrong, please see how MagmaPreallocatedDictionary works.
> It is for very large dictionary's, Esteban is using a MaDictionary for
> his root object, and my guess is it only has a few entries in it..
>
> - Chris

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma
Reply | Threaded
Open this post in threaded view
|

Re: difference between MaDictionary and MagmaDictionary

Chris Muller-3
MagmaDictionary is just a normal MagmaCollection with an index on
#hash (actually, #hash32 because its just a 32-bit index).  Please see
it in a hierarchy browser.  As a subclass of MagmaCollection, it
inherits its characteristics...  Which is actually unfortunate because
those characteristics of regular MagmaCollections have highest
throughput when answering _batches_ of rows.  But dictionary's are
accessed by #at:, just one object at a time.

This is why I wrote MagmaPreallocatedDictionary.  It is based on a MagmaArray.

A MagmaArray is an auto-growing, but never shrinking, Array that can
be very large.  Please see the test cases whichh reference MagmaArray
for the actual semantics.

MagmaArrays are as fast as possible, always just two disk seeks,
6-bytes each, to get the location of the contents of any object.
That's why MagmaPreallocatedDictionary is much faster than a
MagmaCollection, which is based on a MaHashIndex file-structure for
supporting range-based querying.

 - Chris




On Tue, Dec 14, 2010 at 4:57 PM, Esteban Lorenzano <[hidden email]> wrote:

> ok, and what about using MagmaDictionary? better? or the same?
>
> cheers,
> Esteban
>
> El 14/12/2010, a las 5:40p.m., Chris Muller escribió:
>
>>>> If I replace MaDictionary with Dictionary, I will be reading/writing the
>>>> full dictionary each time I add an element to the dictionary... isn't?
>>
>> Yes, that's true.  But a MaDictionary only saves you that if you have
>> a lot of collisions.
>>
>>> I think that you need replace MaDictionary with MagmaPreallocatedDictionary
>>> not with standard Dictionary.
>>
>> No, this is wrong, please see how MagmaPreallocatedDictionary works.
>> It is for very large dictionary's, Esteban is using a MaDictionary for
>> his root object, and my guess is it only has a few entries in it..
>>
>> - Chris
>
> _______________________________________________
> Magma mailing list
> [hidden email]
> http://lists.squeakfoundation.org/mailman/listinfo/magma
>
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma
Reply | Threaded
Open this post in threaded view
|

Re: difference between MaDictionary and MagmaDictionary

ching
Hi Chris,

Would a MagmaPreallocatedDictionary be suitable for a collection of say, 100 million names and addresses?

ching

On Wed, Dec 15, 2010 at 10:00 AM, Chris Muller <[hidden email]> wrote:
MagmaDictionary is just a normal MagmaCollection with an index on
#hash (actually, #hash32 because its just a 32-bit index).  Please see
it in a hierarchy browser.  As a subclass of MagmaCollection, it
inherits its characteristics...  Which is actually unfortunate because
those characteristics of regular MagmaCollections have highest
throughput when answering _batches_ of rows.  But dictionary's are
accessed by #at:, just one object at a time.

This is why I wrote MagmaPreallocatedDictionary.  It is based on a MagmaArray.

A MagmaArray is an auto-growing, but never shrinking, Array that can
be very large.  Please see the test cases whichh reference MagmaArray
for the actual semantics.

MagmaArrays are as fast as possible, always just two disk seeks,
6-bytes each, to get the location of the contents of any object.
That's why MagmaPreallocatedDictionary is much faster than a
MagmaCollection, which is based on a MaHashIndex file-structure for
supporting range-based querying.

 - Chris




On Tue, Dec 14, 2010 at 4:57 PM, Esteban Lorenzano <[hidden email]> wrote:
> ok, and what about using MagmaDictionary? better? or the same?
>
> cheers,
> Esteban
>
> El 14/12/2010, a las 5:40p.m., Chris Muller escribió:
>
>>>> If I replace MaDictionary with Dictionary, I will be reading/writing the
>>>> full dictionary each time I add an element to the dictionary... isn't?
>>
>> Yes, that's true.  But a MaDictionary only saves you that if you have
>> a lot of collisions.
>>
>>> I think that you need replace MaDictionary with MagmaPreallocatedDictionary
>>> not with standard Dictionary.
>>
>> No, this is wrong, please see how MagmaPreallocatedDictionary works.
>> It is for very large dictionary's, Esteban is using a MaDictionary for
>> his root object, and my guess is it only has a few entries in it..
>>
>> - Chris
>
> _______________________________________________
> Magma mailing list
> [hidden email]
> http://lists.squeakfoundation.org/mailman/listinfo/magma
>
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma


_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma
Reply | Threaded
Open this post in threaded view
|

Re: difference between MaDictionary and MagmaDictionary

Chris Muller-3
It doesn't matter how many or what type the objects are, just the
quality of their #hash.

On Tue, Dec 14, 2010 at 8:16 PM, Ching de la Serna <[hidden email]> wrote:

> Hi Chris,
>
> Would a MagmaPreallocatedDictionary be suitable for a collection of say, 100
> million names and addresses?
>
> ching
>
> On Wed, Dec 15, 2010 at 10:00 AM, Chris Muller <[hidden email]> wrote:
>>
>> MagmaDictionary is just a normal MagmaCollection with an index on
>> #hash (actually, #hash32 because its just a 32-bit index).  Please see
>> it in a hierarchy browser.  As a subclass of MagmaCollection, it
>> inherits its characteristics...  Which is actually unfortunate because
>> those characteristics of regular MagmaCollections have highest
>> throughput when answering _batches_ of rows.  But dictionary's are
>> accessed by #at:, just one object at a time.
>>
>> This is why I wrote MagmaPreallocatedDictionary.  It is based on a
>> MagmaArray.
>>
>> A MagmaArray is an auto-growing, but never shrinking, Array that can
>> be very large.  Please see the test cases whichh reference MagmaArray
>> for the actual semantics.
>>
>> MagmaArrays are as fast as possible, always just two disk seeks,
>> 6-bytes each, to get the location of the contents of any object.
>> That's why MagmaPreallocatedDictionary is much faster than a
>> MagmaCollection, which is based on a MaHashIndex file-structure for
>> supporting range-based querying.
>>
>>  - Chris
>>
>>
>>
>>
>> On Tue, Dec 14, 2010 at 4:57 PM, Esteban Lorenzano <[hidden email]>
>> wrote:
>> > ok, and what about using MagmaDictionary? better? or the same?
>> >
>> > cheers,
>> > Esteban
>> >
>> > El 14/12/2010, a las 5:40p.m., Chris Muller escribió:
>> >
>> >>>> If I replace MaDictionary with Dictionary, I will be reading/writing
>> >>>> the
>> >>>> full dictionary each time I add an element to the dictionary...
>> >>>> isn't?
>> >>
>> >> Yes, that's true.  But a MaDictionary only saves you that if you have
>> >> a lot of collisions.
>> >>
>> >>> I think that you need replace MaDictionary with
>> >>> MagmaPreallocatedDictionary
>> >>> not with standard Dictionary.
>> >>
>> >> No, this is wrong, please see how MagmaPreallocatedDictionary works.
>> >> It is for very large dictionary's, Esteban is using a MaDictionary for
>> >> his root object, and my guess is it only has a few entries in it..
>> >>
>> >> - Chris
>> >
>> > _______________________________________________
>> > Magma mailing list
>> > [hidden email]
>> > http://lists.squeakfoundation.org/mailman/listinfo/magma
>> >
>> _______________________________________________
>> Magma mailing list
>> [hidden email]
>> http://lists.squeakfoundation.org/mailman/listinfo/magma
>
>
> _______________________________________________
> Magma mailing list
> [hidden email]
> http://lists.squeakfoundation.org/mailman/listinfo/magma
>
>
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma