Speeding up Pharo 1.1

> On Mon, 18 Oct 2010, Nicolas Cellier wrote:
>
>> 2010/10/18 Igor Stasenko <[hidden email]>:
>>>
>>> On 18 October 2010 23:18, Nicolas Cellier
>>> <[hidden email]> wrote:
>>>>
>>>> 2010/10/17 Levente Uzonyi <[hidden email]>:
>>>>>
>>>>> On Sat, 16 Oct 2010, Bart Veenstra wrote:
>>>>>
>>>>>> Hi list,
>>>>>>
>>>>>> I have been working with Pharo for almost a month now, and I suspect
>>>>>> that the performance is degrading fast. UI tasks takes several seconds
>>>>>> to react to my keyboard.
>>>>>
>>>>> That kind of sluggishness is probably related to finalization/gc.
>>>>> Please
>>>>> send us the result of the following expression:
>>>>>
>>>>> (WeakArray classPool at: #FinalizationDependents)
>>>>>        select: [ :each | each notNil ]
>>>>>        thenCollect: [ :each | each class -> each size ]
>>>>>
>>>>
>>>> While updating pharo 1.2, after an EndOfCentralDirectory error, I got
>>>> a very unresponsive image...
>>>>
>>>> ((WeakArray classPool at: #FinalizationDependents) as: Array)
>>>>       select: [ :each | each notNil ]
>>>>       thenCollect: [ :each | each class -> each size]
>>>> ->
>>>> {(WeakIdentityKeyDictionary->22370).
>>>
>>> This is an MC cache. And major reason of image slowdown.
>>>
>>
>> Levente did this simplification in trunk:
>>
>> cachedDefinitions
>>        ^definitions ifNil: [ definitions := WeakIdentityKeyDictionary new
>> ]
>>
>> It would be worth a try in Pharo.
>
> IIRC Pharo's WeakKeyDictionary (and subclasses) don't work properly if
> they're not registered to the finalization process. So this won't work as
> long as that's not fixed.
>

>
> Levente
>
>>
>> Nicolas
>>
>>>> (WeakRegistry->0).
>>>> (WeakRegistry->0). (WeakRegistry->0). (WeakRegistry->2)}
>>>>
>>>> I tried to inspect it 5 minutes ago, but hey, answer in another email...
>>>> Ah, OK, cmd+shift+. just worked. I had to interrupt
>>>> #finalizationProcess itself finally...
>>>> Oh inspect is sorting the keys... unusable.
>>>> So what's in the array ? Things like
>>>>
>>>> (ScriptLoader>>#update12125 "a CompiledMethod(0)")->a
>>>> MCMethodDefinition(ScriptLoader>>update12125)
>>>> (PragmaCollector>>#release "a CompiledMethod(964689920)")->a
>>>> MCMethodDefinition(PragmaCollector>>release)
>>>> (Float>>#arcSin "a CompiledMethod(358088704)")->a
>>>> MCMethodDefinition(Float>>arcSin)
>>>>
>>>> I don't understand yet, but I begin to gather clues why updating pharo
>>>> goes so badly...
>>>>
>>>> Nicolas
>>>>
>>>> _______________________________________________
>>>> Pharo-project mailing list
>>>> [hidden email]
>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko AKA sig.
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

> 2010/10/19 Levente Uzonyi<[hidden email]>:
>> On Mon, 18 Oct 2010, Nicolas Cellier wrote:
>>
>>> 2010/10/18 Igor Stasenko<[hidden email]>:
>>>> On 18 October 2010 23:18, Nicolas Cellier
>>>> <[hidden email]>  wrote:
>>>>> 2010/10/17 Levente Uzonyi<[hidden email]>:
>>>>>> On Sat, 16 Oct 2010, Bart Veenstra wrote:
>>>>>>
>>>>>>> Hi list,
>>>>>>>
>>>>>>> I have been working with Pharo for almost a month now, and I suspect
>>>>>>> that the performance is degrading fast. UI tasks takes several seconds
>>>>>>> to react to my keyboard.
>>>>>> That kind of sluggishness is probably related to finalization/gc.
>>>>>> Please
>>>>>> send us the result of the following expression:
>>>>>>
>>>>>> (WeakArray classPool at: #FinalizationDependents)
>>>>>>         select: [ :each | each notNil ]
>>>>>>         thenCollect: [ :each | each class ->  each size ]
>>>>>>
>>>>> While updating pharo 1.2, after an EndOfCentralDirectory error, I got
>>>>> a very unresponsive image...
>>>>>
>>>>> ((WeakArray classPool at: #FinalizationDependents) as: Array)
>>>>>        select: [ :each | each notNil ]
>>>>>        thenCollect: [ :each | each class ->  each size]
>>>>> ->
>>>>> {(WeakIdentityKeyDictionary->22370).
>>>> This is an MC cache. And major reason of image slowdown.
>>>>
>>> Levente did this simplification in trunk:
>>>
>>> cachedDefinitions
>>>         ^definitions ifNil: [ definitions := WeakIdentityKeyDictionary new
>>> ]
>>>
>>> It would be worth a try in Pharo.
>> IIRC Pharo's WeakKeyDictionary (and subclasses) don't work properly if
>> they're not registered to the finalization process. So this won't work as
>> long as that's not fixed.
>>
> Huh? Can you provide a code to show it? Test case please?
>

>  On 18.10.2010 23:21, Igor Stasenko wrote:
>>
>> 2010/10/19 Levente Uzonyi<[hidden email]>:
>>>
>>> On Mon, 18 Oct 2010, Nicolas Cellier wrote:
>>>
>>>> 2010/10/18 Igor Stasenko<[hidden email]>:
>>>>>
>>>>> On 18 October 2010 23:18, Nicolas Cellier
>>>>> <[hidden email]>  wrote:
>>>>>>
>>>>>> 2010/10/17 Levente Uzonyi<[hidden email]>:
>>>>>>>
>>>>>>> On Sat, 16 Oct 2010, Bart Veenstra wrote:
>>>>>>>
>>>>>>>> Hi list,
>>>>>>>>
>>>>>>>> I have been working with Pharo for almost a month now, and I suspect
>>>>>>>> that the performance is degrading fast. UI tasks takes several
>>>>>>>> seconds
>>>>>>>> to react to my keyboard.
>>>>>>>
>>>>>>> That kind of sluggishness is probably related to finalization/gc.
>>>>>>> Please
>>>>>>> send us the result of the following expression:
>>>>>>>
>>>>>>> (WeakArray classPool at: #FinalizationDependents)
>>>>>>>        select: [ :each | each notNil ]
>>>>>>>        thenCollect: [ :each | each class ->  each size ]
>>>>>>>
>>>>>> While updating pharo 1.2, after an EndOfCentralDirectory error, I got
>>>>>> a very unresponsive image...
>>>>>>
>>>>>> ((WeakArray classPool at: #FinalizationDependents) as: Array)
>>>>>>       select: [ :each | each notNil ]
>>>>>>       thenCollect: [ :each | each class ->  each size]
>>>>>> ->
>>>>>> {(WeakIdentityKeyDictionary->22370).
>>>>>
>>>>> This is an MC cache. And major reason of image slowdown.
>>>>>
>>>> Levente did this simplification in trunk:
>>>>
>>>> cachedDefinitions
>>>>        ^definitions ifNil: [ definitions := WeakIdentityKeyDictionary
>>>> new
>>>> ]
>>>>
>>>> It would be worth a try in Pharo.
>>>
>>> IIRC Pharo's WeakKeyDictionary (and subclasses) don't work properly if
>>> they're not registered to the finalization process. So this won't work as
>>> long as that's not fixed.
>>>
>> Huh? Can you provide a code to show it? Test case please?
>>
> Just add lots of objects which will be GC'd to an unregistered
> WeakKeyDictionary.
> After X objects are gc'd, you'd still have X nil-keyed associations in the
> dictionary.
> Add Y objects, gc those, and you have X+Y nil-keys in the dict.
> (If dict grown/rehashed, all nicely placed index 1 onwards)
>

> On 19 October 2010 00:30, Henrik Sperre Johansen
> <[hidden email]>  wrote:
>>   On 18.10.2010 23:21, Igor Stasenko wrote:
>>> 2010/10/19 Levente Uzonyi<[hidden email]>:
>>>> On Mon, 18 Oct 2010, Nicolas Cellier wrote:
>>>>
>>>>> 2010/10/18 Igor Stasenko<[hidden email]>:
>>>>>> On 18 October 2010 23:18, Nicolas Cellier
>>>>>> <[hidden email]>    wrote:
>>>>>>> 2010/10/17 Levente Uzonyi<[hidden email]>:
>>>>>>>> On Sat, 16 Oct 2010, Bart Veenstra wrote:
>>>>>>>>
>>>>>>>>> Hi list,
>>>>>>>>>
>>>>>>>>> I have been working with Pharo for almost a month now, and I suspect
>>>>>>>>> that the performance is degrading fast. UI tasks takes several
>>>>>>>>> seconds
>>>>>>>>> to react to my keyboard.
>>>>>>>> That kind of sluggishness is probably related to finalization/gc.
>>>>>>>> Please
>>>>>>>> send us the result of the following expression:
>>>>>>>>
>>>>>>>> (WeakArray classPool at: #FinalizationDependents)
>>>>>>>>         select: [ :each | each notNil ]
>>>>>>>>         thenCollect: [ :each | each class ->    each size ]
>>>>>>>>
>>>>>>> While updating pharo 1.2, after an EndOfCentralDirectory error, I got
>>>>>>> a very unresponsive image...
>>>>>>>
>>>>>>> ((WeakArray classPool at: #FinalizationDependents) as: Array)
>>>>>>>        select: [ :each | each notNil ]
>>>>>>>        thenCollect: [ :each | each class ->    each size]
>>>>>>> ->
>>>>>>> {(WeakIdentityKeyDictionary->22370).
>>>>>> This is an MC cache. And major reason of image slowdown.
>>>>>>
>>>>> Levente did this simplification in trunk:
>>>>>
>>>>> cachedDefinitions
>>>>>         ^definitions ifNil: [ definitions := WeakIdentityKeyDictionary
>>>>> new
>>>>> ]
>>>>>
>>>>> It would be worth a try in Pharo.
>>>> IIRC Pharo's WeakKeyDictionary (and subclasses) don't work properly if
>>>> they're not registered to the finalization process. So this won't work as
>>>> long as that's not fixed.
>>>>
>>> Huh? Can you provide a code to show it? Test case please?
>>>
>> Just add lots of objects which will be GC'd to an unregistered
>> WeakKeyDictionary.
>> After X objects are gc'd, you'd still have X nil-keyed associations in the
>> dictionary.
>> Add Y objects, gc those, and you have X+Y nil-keys in the dict.
>> (If dict grown/rehashed, all nicely placed index 1 onwards)
>>
> it doesn't sounds like incorrect behavior (i.e. pruning a nil-ed keys
> before #finalizeValues could visit them).
>

> On Tue, 19 Oct 2010, Igor Stasenko wrote:
>
>> On 19 October 2010 00:30, Henrik Sperre Johansen
>
> <[hidden email]> wrote:
>>
>>  On 18.10.2010 23:21, Igor Stasenko wrote:
>>>
>>> 2010/10/19 Levente Uzonyi<[hidden email]>:
>>>>
>>>> On Mon, 18 Oct 2010, Nicolas Cellier wrote:
>>>>
>>>>> 2010/10/18 Igor Stasenko<[hidden email]>:
>>>>>>
>>>>>> On 18 October 2010 23:18, Nicolas Cellier
>>>>>> <[hidden email]>  wrote:
>>>>>>>
>>>>>>> 2010/10/17 Levente Uzonyi<[hidden email]>:
>>>>>>>>
>>>>>>>> On Sat, 16 Oct 2010, Bart Veenstra wrote:
>>>>>>>>
>>>>>>>>> Hi list,
>>>>>>>>>
>>>>>>>>> I have been working with Pharo for almost a month now, and I
>>>>>>>>> suspect
>>>>>>>>> that the performance is degrading fast. UI tasks takes several
>>>>>>>>> seconds
>>>>>>>>> to react to my keyboard.
>>>>>>>>
>>>>>>>> That kind of sluggishness is probably related to finalization/gc.
>>>>>>>> Please
>>>>>>>> send us the result of the following expression:
>>>>>>>>
>>>>>>>> (WeakArray classPool at: #FinalizationDependents)
>>>>>>>>        select: [ :each | each notNil ]
>>>>>>>>        thenCollect: [ :each | each class ->  each size ]
>>>>>>>>
>>>>>>> While updating pharo 1.2, after an EndOfCentralDirectory error, I got
>>>>>>> a very unresponsive image...
>>>>>>>
>>>>>>> ((WeakArray classPool at: #FinalizationDependents) as: Array)
>>>>>>>       select: [ :each | each notNil ]
>>>>>>>       thenCollect: [ :each | each class ->  each size]
>>>>>>> ->
>>>>>>> {(WeakIdentityKeyDictionary->22370).
>>>>>>
>>>>>> This is an MC cache. And major reason of image slowdown.
>>>>>>
>>>>> Levente did this simplification in trunk:
>>>>>
>>>>> cachedDefinitions
>>>>>        ^definitions ifNil: [ definitions := WeakIdentityKeyDictionary
>>>>> new
>>>>> ]
>>>>>
>>>>> It would be worth a try in Pharo.
>>>>
>>>> IIRC Pharo's WeakKeyDictionary (and subclasses) don't work properly if
>>>> they're not registered to the finalization process. So this won't work
>>>> as
>>>> long as that's not fixed.
>>>>
>>> Huh? Can you provide a code to show it? Test case please?
>>>
>> Just add lots of objects which will be GC'd to an unregistered
>> WeakKeyDictionary.
>> After X objects are gc'd, you'd still have X nil-keyed associations in the
>> dictionary.
>> Add Y objects, gc those, and you have X+Y nil-keys in the dict.
>> (If dict grown/rehashed, all nicely placed index 1 onwards)
>>
>
> it doesn't sounds like incorrect behavior (i.e. pruning a nil-ed keys
> before #finalizeValues could visit them).
>
>
> It's incorrect, because WeakKeyDictionaries are not designed to be accessed
> concurrently, but this can happen if the dictionary is registered to the
> finalization process.

>
> Levente
>
>
>> Cheers,
>> Henry
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

> 2010/10/17 Levente Uzonyi <[hidden email]>:
>> On Sat, 16 Oct 2010, Bart Veenstra wrote:
>>
>>> Hi list,
>>>
>>> I have been working with Pharo for almost a month now, and I suspect
>>> that the performance is degrading fast. UI tasks takes several seconds
>>> to react to my keyboard.
>>
>> That kind of sluggishness is probably related to finalization/gc. Please
>> send us the result of the following expression:
>>
>> (WeakArray classPool at: #FinalizationDependents)
>>        select: [ :each | each notNil ]
>>        thenCollect: [ :each | each class -> each size ]
>>
>
> While updating pharo 1.2, after an EndOfCentralDirectory error, I got
> a very unresponsive image...

> ((WeakArray classPool at: #FinalizationDependents) as: Array)
>       select: [ :each | each notNil ]
>       thenCollect: [ :each | each class -> each size]
> ->
> {(WeakIdentityKeyDictionary->22370). (WeakRegistry->0).
> (WeakRegistry->0). (WeakRegistry->0). (WeakRegistry->2)}
>
> I tried to inspect it 5 minutes ago, but hey, answer in another email...
> Ah, OK, cmd+shift+. just worked. I had to interrupt
> #finalizationProcess itself finally...
> Oh inspect is sorting the keys... unusable.
> So what's in the array ? Things like
>
> (ScriptLoader>>#update12125 "a CompiledMethod(0)")->a
> MCMethodDefinition(ScriptLoader>>update12125)
> (PragmaCollector>>#release "a CompiledMethod(964689920)")->a
> MCMethodDefinition(PragmaCollector>>release)
> (Float>>#arcSin "a CompiledMethod(358088704)")->a
> MCMethodDefinition(Float>>arcSin)
>
> I don't understand yet, but I begin to gather clues why updating pharo
> goes so badly...

> On Oct 18, 2010, at 10:19 PM, Nicolas Cellier wrote:
>
>> 2010/10/17 Levente Uzonyi<[hidden email]>:
>>> On Sat, 16 Oct 2010, Bart Veenstra wrote:
>>>
>>>> Hi list,
>>>>
>>>> I have been working with Pharo for almost a month now, and I suspect
>>>> that the performance is degrading fast. UI tasks takes several seconds
>>>> to react to my keyboard.
>>> That kind of sluggishness is probably related to finalization/gc. Please
>>> send us the result of the following expression:
>>>
>>> (WeakArray classPool at: #FinalizationDependents)
>>>         select: [ :each | each notNil ]
>>>         thenCollect: [ :each | each class ->  each size ]
>>>
>> While updating pharo 1.2, after an EndOfCentralDirectory error, I got
>> a very unresponsive image...
> can you reproduce it?
> Which image ?
> Which vm ?
> Never heard about this error before

> On 19.10.2010 09:17, Stéphane Ducasse wrote:
>> How can we make progress on this issue?
>>
>> Stef
> As far as I can tell (again, I haven't actually experienced this slowdown), any of:
> - Port Levente's WeakKeyDictionary, which is much faster.
> - Wait for new VM's supporting Igors finalization scheme, which should be much faster (haven't tested it personally)
> - Rewrite the MC package caching to not use weak collections/ be smarter about what it keeps.
>
> Cheers,
> Henry
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project