Smalltalk › Pharo › Pharo Smalltalk Developers

Configuration loading 1.4 vs 2.0

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

12 messages Options

Camillo Bruni-3

Configuration loading 1.4 vs 2.0

After Igor's remark about the loading speed of configs in 2.0
I ran twice the same configuration in 1.4 and 2.0 and things load
incredibly slow under 2.0

Subjectively it seems that 2.0 spends ages loading Metacello / Gofer stuff.

I'll dive a bit into it to see if somethings stands out...

The two images with the Profiler opened:
http://dl.dropbox.com/u/18184943/LoadConfig1.4.zip
http://dl.dropbox.com/u/18184943/LoadConfig2.0.zip

Camillo Bruni-3

Re: Configuration loading 1.4 vs 2.0

On 2012-09-09, at 14:14, Mariano Martinez Peck <[hidden email]> wrote:

> I think it has to do with the change in #readableFileNames

Indeed it metacello does this big mistake of gathering all the local gofer-references first.

But that would mean on an empty package-cache it should load the same way, but even
then it's still considerably slower in 2.0

There's also an additional overhead by the announcement system for recategorizing methods..

Camillo Bruni-3

Re: Configuration loading 1.4 vs 2.0

In reply to this post by Camillo Bruni-3

On 2012-09-09, at 14:24, Mariano Martinez Peck <[hidden email]> wrote:

> I guess you tried both with an empty package-cache, right?

yes, not much of a diff...

Mariano Martinez Peck

Re: Configuration loading 1.4 vs 2.0

I think instVar cacheFileNames is never initialized and hence, it follows

cacheFileNames == true ifFalse: [ ^ self loadAllFileNames ].

On Sun, Sep 9, 2012 at 2:35 PM, Camillo Bruni <[hidden email]> wrote:

On 2012-09-09, at 14:24, Mariano Martinez Peck <[hidden email]> wrote:

> I guess you tried both with an empty package-cache, right?

yes, not much of a diff...

--
Mariano
http://marianopeck.wordpress.com

Camillo Bruni-3

Re: Configuration loading 1.4 vs 2.0

On 2012-09-09, at 15:12, Mariano Martinez Peck <[hidden email]> wrote:

> I think instVar cacheFileNames is never initialized and hence, it follows
>
> cacheFileNames == true ifFalse: [ ^ self loadAllFileNames ].

where is that?

> On Sun, Sep 9, 2012 at 2:35 PM, Camillo Bruni <[hidden email]>wrote:
>
>>
>> On 2012-09-09, at 14:24, Mariano Martinez Peck <[hidden email]>
>> wrote:
>>
>>> I guess you tried both with an empty package-cache, right?
>>
>> yes, not much of a diff...

Mariano Martinez Peck

Re: Configuration loading 1.4 vs 2.0

allFileNames

cacheFileNames == true ifFalse: [ ^ self loadAllFileNames ].

^ allFileNames ifNil: [

allFileNames := self loadAllFileNames]

If you put a halt in the first if, you will see it stops... because cacheFileNames is nil. In 1.4 you have:

allFileNamesOrCache

^ allFileNames ifNil: [self allFileNames]

so it kind of always cached... so maybe this is a difference that affects the results?

On Sun, Sep 9, 2012 at 3:14 PM, Camillo Bruni <[hidden email]> wrote:

On 2012-09-09, at 15:12, Mariano Martinez Peck <[hidden email]> wrote:

> I think instVar cacheFileNames is never initialized and hence, it follows
>
> cacheFileNames == true ifFalse: [ ^ self loadAllFileNames ].

where is that?

> On Sun, Sep 9, 2012 at 2:35 PM, Camillo Bruni <[hidden email]>wrote:
>
>>
>> On 2012-09-09, at 14:24, Mariano Martinez Peck <[hidden email]>
>> wrote:
>>
>>> I guess you tried both with an empty package-cache, right?
>>
>> yes, not much of a diff...

--
Mariano
http://marianopeck.wordpress.com

Camillo Bruni-3

Re: Configuration loading 1.4 vs 2.0

On 2012-09-09, at 15:16, Mariano Martinez Peck <[hidden email]> wrote:

> allFileNames
> cacheFileNames == true ifFalse: [ ^ self loadAllFileNames ].
> ^ allFileNames ifNil: [
> allFileNames := self loadAllFileNames]
>
>
> If you put a halt in the first if, you will see it stops...
> because cacheFileNames is nil. In 1.4 you have:
>
> allFileNamesOrCache
> ^ allFileNames ifNil: [self allFileNames]
>
> so it kind of always cached... so maybe this is a difference that affects
> the results?

humm you might be right |(. so metacello should explicitely tell to cache stuff in 2.0 then

Mariano Martinez Peck

Re: Configuration loading 1.4 vs 2.0

On Sun, Sep 9, 2012 at 3:20 PM, Camillo Bruni <[hidden email]> wrote:

On 2012-09-09, at 15:16, Mariano Martinez Peck <[hidden email]> wrote:

> allFileNames
> cacheFileNames == true ifFalse: [ ^ self loadAllFileNames ].
> ^ allFileNames ifNil: [
> allFileNames := self loadAllFileNames]
>
>
> If you put a halt in the first if, you will see it stops...
> because cacheFileNames is nil. In 1.4 you have:
>
> allFileNamesOrCache
> ^ allFileNames ifNil: [self allFileNames]
>
> so it kind of always cached... so maybe this is a difference that affects
> the results?

humm you might be right |(. so metacello should explicitely tell to cache stuff in 2.0 then

I would do

MCFileBasedRepository >> initialize

cacheFileNames := true

or something like test. But first, we should be sure this was really one of the causes of the slowdown

--
Mariano
http://marianopeck.wordpress.com

Mariano Martinez Peck

Re: Configuration loading 1.4 vs 2.0

Send me the doIt to install your confs and I will try to try it.

On Sun, Sep 9, 2012 at 3:25 PM, Mariano Martinez Peck <[hidden email]> wrote:

On Sun, Sep 9, 2012 at 3:20 PM, Camillo Bruni <[hidden email]> wrote:

On 2012-09-09, at 15:16, Mariano Martinez Peck <[hidden email]> wrote:

> allFileNames
> cacheFileNames == true ifFalse: [ ^ self loadAllFileNames ].
> ^ allFileNames ifNil: [
> allFileNames := self loadAllFileNames]
>
>
> If you put a halt in the first if, you will see it stops...
> because cacheFileNames is nil. In 1.4 you have:
>
> allFileNamesOrCache
> ^ allFileNames ifNil: [self allFileNames]
>
> so it kind of always cached... so maybe this is a difference that affects
> the results?

humm you might be right |(. so metacello should explicitely tell to cache stuff in 2.0 then

I would do

MCFileBasedRepository >> initialize
cacheFileNames := true

or something like test. But first, we should be sure this was really one of the causes of the slowdown

--
Mariano
http://marianopeck.wordpress.com

--
Mariano
http://marianopeck.wordpress.com

Camillo Bruni-3

Re: Configuration loading 1.4 vs 2.0

In reply to this post by Mariano Martinez Peck

On 2012-09-09, at 15:25, Mariano Martinez Peck <[hidden email]> wrote:

> On Sun, Sep 9, 2012 at 3:20 PM, Camillo Bruni <[hidden email]>wrote:
>
>>
>> On 2012-09-09, at 15:16, Mariano Martinez Peck <[hidden email]>
>> wrote:
>>
>>> allFileNames
>>> cacheFileNames == true ifFalse: [ ^ self loadAllFileNames ].
>>> ^ allFileNames ifNil: [
>>> allFileNames := self loadAllFileNames]
>>>
>>>
>>> If you put a halt in the first if, you will see it stops...
>>> because cacheFileNames is nil. In 1.4 you have:
>>>
>>> allFileNamesOrCache
>>> ^ allFileNames ifNil: [self allFileNames]
>>>
>>> so it kind of always cached... so maybe this is a difference that affects
>>> the results?
>>
>>
>> humm you might be right |(. so metacello should explicitely tell to cache
>> stuff in 2.0 then
>>
>>
>>
>
> I would do
>
> MCFileBasedRepository >> initialize
> cacheFileNames := true
>
> or something like test. But first, we should be sure this was really one of
> the causes of the slowdown

you'll break the whole download optimizations if you enable these caches there,
since it will never find the newly downloaded files again :P

I think there is a method, something like cacheAllFileNamesDuring: or so...

Camillo Bruni-3

Re: Configuration loading 1.4 vs 2.0

In reply to this post by Mariano Martinez Peck

Doesn't really matter which config you use:

Gofer it
squeaksource: 'NativeBoost';
package: 'ConfigurationOfNativeBoost';
load.

ConfigurationOfNativeBoost loadDevelopment

On 2012-09-09, at 15:27, Mariano Martinez Peck <[hidden email]> wrote:

> Send me the doIt to install your confs and I will try to try it.
>
> On Sun, Sep 9, 2012 at 3:25 PM, Mariano Martinez Peck <[hidden email]
>> wrote:
>
>>
>>
>> On Sun, Sep 9, 2012 at 3:20 PM, Camillo Bruni <[hidden email]>wrote:
>>
>>>
>>> On 2012-09-09, at 15:16, Mariano Martinez Peck <[hidden email]>
>>> wrote:
>>>
>>>> allFileNames
>>>> cacheFileNames == true ifFalse: [ ^ self loadAllFileNames ].
>>>> ^ allFileNames ifNil: [
>>>> allFileNames := self loadAllFileNames]
>>>>
>>>>
>>>> If you put a halt in the first if, you will see it stops...
>>>> because cacheFileNames is nil. In 1.4 you have:
>>>>
>>>> allFileNamesOrCache
>>>> ^ allFileNames ifNil: [self allFileNames]
>>>>
>>>> so it kind of always cached... so maybe this is a difference that
>>> affects
>>>> the results?
>>>
>>>
>>> humm you might be right |(. so metacello should explicitely tell to cache
>>> stuff in 2.0 then
>>>
>>>
>>>
>>
>> I would do
>>
>> MCFileBasedRepository >> initialize
>> cacheFileNames := true
>>
>> or something like test. But first, we should be sure this was really one
>> of the causes of the slowdown
>>
>>
>>
>> --
>> Mariano
>> http://marianopeck.wordpress.com
>>
>>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com

Stéphane Ducasse

Re: Configuration loading 1.4 vs 2.0

In reply to this post by Camillo Bruni-3

Thanks for checking that because we should address it.

Stef

On Sep 9, 2012, at 1:17 PM, Camillo Bruni wrote:

> After Igor's remark about the loading speed of configs in 2.0
> I ran twice the same configuration in 1.4 and 2.0 and things load
> incredibly slow under 2.0
>
> Subjectively it seems that 2.0 spends ages loading Metacello / Gofer stuff.
>
> I'll dive a bit into it to see if somethings stands out...
>
> The two images with the Profiler opened:
> http://dl.dropbox.com/u/18184943/LoadConfig1.4.zip
> http://dl.dropbox.com/u/18184943/LoadConfig2.0.zip
>