Smalltalk › Frameworks & Tools › Magma

Collections issue

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

7 messages Options

fvozzi

Collections issue

Hi,

I'm using Magma r42 on Pharo 1.0 and I have this situation:

"2356 Cities"
Time millisecondsToRun: [mySession root states first cities do: [:each | Transcript show: each]] "16178 milliseconds"

mySession root answer a singleton that have 23 states and the first state has 2356 cities. Is 16178 milliseconds the expected time for accessing to it?

states and cities are OrderedCollection instances. Must I change it to MagmaCollection for better performance?

Thanks,

Facundo

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

Igor Stasenko

Re: Collections issue

On 17 May 2010 03:05, Facundo Vozzi <[hidden email]> wrote:

> Hi,
> I'm using Magma r42 on Pharo 1.0 and I have this situation:
>
> "2356 Cities"
> Time millisecondsToRun: [mySession root states first cities do: [:each
> | Transcript show: each]] "16178 milliseconds"
>
> mySession root answer a singleton that have 23 states and the first state
> has 2356 cities. Is 16178 milliseconds the expected time for accessing to
> it?
>
> states and cities are OrderedCollection instances. Must I change it to
> MagmaCollection for better performance?
>

Yes. Alternatively you could change the magma read strategy, to tell it
to fully reify all objects in your collection.
But for large collections, use of MagmaCollection is preferable.

> Thanks,
> Facundo
> _______________________________________________
> Magma mailing list
> [hidden email]
> http://lists.squeakfoundation.org/mailman/listinfo/magma
>
>

--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

Brent Pinkney-2

Re: Collections issue

In reply to this post by fvozzi

Hi,

> Hi,
> I'm using Magma r42 on Pharo 1.0 and I have this situation:
>
> "2356 Cities"
> Time millisecondsToRun:
> [mySession root states first cities do: [:each
>
> | Transcript show: each]] "16178 milliseconds"
>
> mySession root answer a singleton that have 23 states and the first state
> has 2356 cities. Is 16178 milliseconds the expected time for accessing to
> it?
> states and cities are OrderedCollection instances. Must I change it to
> MagmaCollection for better performance?

Hi Facundo,

I recommend you think in terms of "poxy agitation" and "materialisation" to
understand how a _transparent_ database works.

01. When you evaluate "mySession root" there is a rount-trip to the Magma server get the root object.
02. This then gets materialised if it it not already in the image.
03. All the root object's instance variables are proxies.
04. When you send #states, you agitate the proxy in the 'state' instance variable.
05. The proxy fetches the OrderedCollection and all 50 states (but not their instance variables).
06. These are materialised and the proxy replaced.
07. When you send first, the newly materialised OrderedCollection responds immediately.
08. When you send #cities, you agistate the proxy in the 'cities' instance variable.
09. The proxy fetches the OrderedCollection and all 2356 cities (but not their instance variables).
10. These are materialised and the proxy replaced.
11. When you send #do: it is executed by the OrderedCollection immediately.
12. The #do: sends #value: to the block passing the first City instance.
13. The blocks sends #printString, via #show: to the City.
14. Assuming the #printString prints the city's name, the name instance variable is agitated.
15. The proxy fetches the name String and materialised it.
16. The city is shown on the Transcript.
17. Repeat 13-17 another 2355 times.

So,
a) Large OrderedCollections are slow becuase the entire OrderedCollection is materialised,
even if you only want the first State.

b) Consider MagmaCollections to fetch a State by name:
ca = session root state where: [ :s | s name = 'California' ].

b) Materialisation of complex graphs of real Smalltalk objects is much slower than reading a
row of primitive types from an SQL table. Alas.

c) You can minimise materialisation round-trips by using a ReadStrategy. For example, a City
should _always_ materialise its 'name' instance variable whenever a City is materialised.

Respond to the list if you need more help.

Brent
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

fvozzi

Re: Collections issue

Hi Brent,

Hi Facundo,

I recommend you think in terms of "poxy agitation" and "materialisation" to
understand how a _transparent_ database works.

01. When you evaluate "mySession root" there is a rount-trip to the Magma server get the root object.
02. This then gets materialised if it it not already in the image.
03. All the root object's instance variables are proxies.
04. When you send #states, you agitate the proxy in the 'state' instance variable.
05. The proxy fetches the OrderedCollection and all 50 states (but not their instance variables).
06. These are materialised and the proxy replaced.
07. When you send first, the newly materialised OrderedCollection responds immediately.
08. When you send #cities, you agistate the proxy in the 'cities' instance variable.
09. The proxy fetches the OrderedCollection and all 2356 cities (but not their instance variables).
10. These are materialised and the proxy replaced.
11. When you send #do: it is executed by the OrderedCollection immediately.
12. The #do: sends #value: to the block passing the first City instance.
13. The blocks sends #printString, via #show: to the City.
14. Assuming the #printString prints the city's name, the name instance variable is agitated.
15. The proxy fetches the name String and materialised it.
16. The city is shown on the Transcript.
17. Repeat 13-17 another 2355 times.

Yes, I read magma wiki documentations but your explication is very clear, thanks.

So,
a) Large OrderedCollections are slow becuase the entire OrderedCollection is materialised,
even if you only want the first State.

b) Consider MagmaCollections to fetch a State by name:
ca = session root state where: [ :s | s name = 'California' ].

b) Materialisation of complex graphs of real Smalltalk objects is much slower than reading a
row of primitive types from an SQL table. Alas.

c) You can minimise materialisation round-trips by using a ReadStrategy. For example, a City
should _always_ materialise its 'name' instance variable whenever a City is materialised.

I think that In this case c is the better option because I need a drop down with states and when the user choice one of them I need other drop down with its cities. Do you agree ?

Respond to the list if you need more help.

Thanks you,

Facundo

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

fvozzi

Re: Collections issue

In reply to this post by fvozzi

Hi Balázs,

Was the Transcript open? Because #show: is dog slow.
For me it takes some minutes to write 2356 things to the Transcript.

Yes I know that but in this case a put show: for force instances materialization because I need the name instance variable initialized from the repository.

Only for information these are my show: times,

"Closed transcript"

Time millisecondsToRun: [

1 to: 2356 do: [:each | Transcript show: 'String', each asString]] "10818 milliseconds"

"Open transcript"

Time millisecondsToRun: [

1 to: 2356 do: [:each | Transcript show: 'String', each asString]] "117107 milliseconds"

Facundo

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

fvozzi

Re: Collections issue

In reply to this post by Igor Stasenko

Hi Igor,

Yes. Alternatively you could change the magma read strategy, to tell it
to fully reify all objects in your collection.
But for large collections, use of MagmaCollection is preferable.

In your opinion or expertise, how many objects are necesary to consider a collection how a large collection ?

Facundo

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

Igor Stasenko

Re: Collections issue

On 17 May 2010 18:31, Facundo Vozzi <[hidden email]> wrote:

> Hi Igor,
>
>>
>> Yes. Alternatively you could change the magma read strategy, to tell it
>> to fully reify all objects in your collection.
>> But for large collections, use of MagmaCollection is preferable.
>
> In your opinion or expertise, how many objects are necesary to consider a
> collection how a large collection ?
>

Well, i think you should choose based on practical criteria:
how well your system scales and how fast it runs.

Also, take into account that reification of a large amounts of data
will take a considerable amounts of time,
no matter what kind of collection you using.
So, you should focus on organizing your data & queries efficiently ,
in such way that for most use cases,
it won't require a high traffic between client and server.

> Facundo

--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma