(igor) Help for 1.1: condenseChanges does not work anymore

> | empty |
> empty := Set new.
> Smalltalk allClassesAndTraitsDo: [:classOrTrait |
> empty addAll: ( classOrTrait methodDict select: [:m | m sourcePointer =0 ])
> ].
> empty isEmpty
> -->> prints true
>
> which means there is no installed methods with sourcePointer = 0 (and
> hence with no source code) after condensing the changes.
>
>
> But what makes me really suspicious is following:
>
> CompiledMethod allInstances size 55797
> (CompiledMethod allInstances reject: #isInstalled) size  12586
>
> it looks like #isInstalled is buggy, or otherwise indicating that
> there is problem with image,
> which keeps so many non-installed methods hanging around.
> I would simply put a following at the beginning of #condenseChanges:
>
> (CompiledMethod allInstances reject: #isInstalled) size > 0 ifTrue: [
> self error: 'first, get rid of any uninstalled methods' ].
>
> So, then you would not need to use #destroySourcePointer :)
>
> Here an alternative version of #destroySourcePointer. But it doesn't
> makes any big difference.
>
> destroySourcePointer
>
> "We can't change the trailer of existing method, since
> it could have completely different format. Therefore we need to
> generate a copy with new trailer, and then #become it"
> | copy |
> copy := self copyWithTrailerBytes: CompiledMethodTrailer empty.
> self becomeForward: copy.
> ^ copy
>
>
> P.S. runned it on 1.1-11343-UNSTABLE
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

> After condensing changes and #destroySourcePointer,
> the garbage 12k methods magically disappeared, so now its just:
>
> (CompiledMethod allInstances reject: #isInstalled) size 67
>
> and image after saving shrunk from 14M to 13M
> probably some cache somewhere not flushed..
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

> And the last thing...
> since new trailers using #becomeForward:
> no wonder that this loop turns into an infinite:
>
> CompiledMethod allInstancesDo: [:e | e isInstalled ifFalse: [e
> destroySourcePointer]].
>
> because #allInstancesDo: walks a heap, and assuming that there is no
> new instances of it created
> during walking. But destroySourcePointer creating a new instance, so
> it can never finish! :)
>
> i just replaced it with following:
>
> (CompiledMethod allInstances reject: #isInstalled) do: [:e | e
> isInstalled ifFalse: [e destroySourcePointer]].

>
> and made it to the end.. took about 5 minutes to wait. (12k methods +
> become on every method is a little time consuming ;)
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

> Lukas did a cool test that checks whether the we can parse the method.
>
>
>    
>> and made it to the end.. took about 5 minutes to wait. (12k methods +
>> become on every method is a little time consuming ;)
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>      
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
>    

> Hi all,
>>> i just replaced it with following:
>>>
>>> (CompiledMethod allInstances reject: #isInstalled) do: [:e | e
>>> isInstalled ifFalse: [e destroySourcePointer]].
>>>    
>> Yes clearly a conceptual bug.
>> But with the solution alain proposed ie not destroying while iterating on methods does not work
>> (yes it condenses well but accessing some methods is broken).
>>  
> yes but Igor solution is a little bit different because 'e isInstalled' is tested twice,
> one for the reject: and the second in the do: block.
> But unfortunately, this solution seems to be broken to. It works one time, but while trying to
> condenseChanges again several time, my squeak process is killed without flushing any log.
> Alain
>> Lukas did a cool test that checks whether the we can parse the method.
>>
>>
>>  
>>> and made it to the end.. took about 5 minutes to wait. (12k methods +
>>> become on every method is a little time consuming ;)
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko AKA sig.
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>    
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>>  
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

> Alain
> with the latest unstable it seems to work.
> You just have to wait a lot once the progress bar is finished to display.
>
> Then we got only 34/35 methods not installed which is ok.
> I will fix condenseSources then :)
> Stef
>
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

> Hi all,
>>>
>>> i just replaced it with following:
>>>
>>> (CompiledMethod allInstances reject: #isInstalled) do: [:e | e
>>> isInstalled ifFalse: [e destroySourcePointer]].
>>>
>>
>> Yes clearly a conceptual bug.
>> But with the solution alain proposed ie not destroying while iterating on
>> methods does not work
>> (yes it condenses well but accessing some methods is broken).
>>
>
> yes but Igor solution is a little bit different because 'e isInstalled' is
> tested twice,
> one for the reject: and the second in the do: block.

> Alain
>>
>> Lukas did a cool test that checks whether the we can parse the method.
>>
>>
>>
>>>
>>> and made it to the end.. took about 5 minutes to wait. (12k methods +
>>> become on every method is a little time consuming ;)
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko AKA sig.
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>>
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

> On 9 May 2010 13:29, Alain Plantec <[hidden email]> wrote:
>> Hi all,
>>>>
>>>> i just replaced it with following:
>>>>
>>>> (CompiledMethod allInstances reject: #isInstalled) do: [:e | e
>>>> isInstalled ifFalse: [e destroySourcePointer]].
>>>>
>>>
>>> Yes clearly a conceptual bug.
>>> But with the solution alain proposed ie not destroying while iterating on
>>> methods does not work
>>> (yes it condenses well but accessing some methods is broken).
>>>
>>
>> yes but Igor solution is a little bit different because 'e isInstalled' is
>> tested twice,
>> one for the reject: and the second in the do: block.
>
> This was a result of lack of attention, not my real intent. :)
>
> In bug entry for this issue i added a method which does just:
> (CompiledMethod allInstances reject: #isInstalled) do: #destroySourcePointer.
>
> there's no way how method can become installed when you changin its trailer.
> So, a single test #isInstalled is sufficient.
> A more faster way is to use a mass-become.
> But i think this is not a place where you need to care too much about
> performance.
>
>> But unfortunately, this solution seems to be broken to. It works one time,
>> but while trying to
>> condenseChanges again several time, my squeak process is killed without
>> flushing any log.
>
> You mean running #condenseChanges multiple times in a row, or
> interrupting it and then start over again?
> You should know, that it is important to not interrupt it in the
> middle, because when you doing that,
> part of your methods having old source pointers, while rest part
> having new ones,
> and there is no way to tell, which one is correct.
>
>> Alain
>>>
>>> Lukas did a cool test that checks whether the we can parse the method.
>>>
>>>
>>>
>>>>
>>>> and made it to the end.. took about 5 minutes to wait. (12k methods +
>>>> become on every method is a little time consuming ;)
>>>>
>>>> --
>>>> Best regards,
>>>> Igor Stasenko AKA sig.
>>>>
>>>> _______________________________________________
>>>> Pharo-project mailing list
>>>> [hidden email]
>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>
>>>
>>
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>

> Probably , a more crash-proof #condenseChanges
> would be to use a mass-become.
> So, you are not modifying existing methods, but  making their copies
> with updated source pointers
> and copying their source into new changes file.
> Then you do a single mass-become and so, all old methods get swapped
> with new ones using a single massive blow.
> In that way, its more safe, since if you interrupt the
> #condenseChanges in the middle,
> you can use debugger and browse the code , and if you decide to abort
> it, nothing bad happen,
> since methods with new source pointers is held in your temp collection
> and not yet applied.
>
> On 9 May 2010 19:48, Igor Stasenko <[hidden email]> wrote:
>> On 9 May 2010 13:29, Alain Plantec <[hidden email]> wrote:
>>> Hi all,
>>>>>
>>>>> i just replaced it with following:
>>>>>
>>>>> (CompiledMethod allInstances reject: #isInstalled) do: [:e | e
>>>>> isInstalled ifFalse: [e destroySourcePointer]].
>>>>>
>>>>
>>>> Yes clearly a conceptual bug.
>>>> But with the solution alain proposed ie not destroying while iterating on
>>>> methods does not work
>>>> (yes it condenses well but accessing some methods is broken).
>>>>
>>>
>>> yes but Igor solution is a little bit different because 'e isInstalled' is
>>> tested twice,
>>> one for the reject: and the second in the do: block.
>>
>> This was a result of lack of attention, not my real intent. :)
>>
>> In bug entry for this issue i added a method which does just:
>> (CompiledMethod allInstances reject: #isInstalled) do: #destroySourcePointer.
>>
>> there's no way how method can become installed when you changin its trailer.
>> So, a single test #isInstalled is sufficient.
>> A more faster way is to use a mass-become.
>> But i think this is not a place where you need to care too much about
>> performance.
>>
>>> But unfortunately, this solution seems to be broken to. It works one time,
>>> but while trying to
>>> condenseChanges again several time, my squeak process is killed without
>>> flushing any log.
>>
>> You mean running #condenseChanges multiple times in a row, or
>> interrupting it and then start over again?
>> You should know, that it is important to not interrupt it in the
>> middle, because when you doing that,
>> part of your methods having old source pointers, while rest part
>> having new ones,
>> and there is no way to tell, which one is correct.
>>
>>> Alain
>>>>
>>>> Lukas did a cool test that checks whether the we can parse the method.
>>>>
>>>>
>>>>
>>>>>
>>>>> and made it to the end.. took about 5 minutes to wait. (12k methods +
>>>>> become on every method is a little time consuming ;)
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Igor Stasenko AKA sig.
>>>>>
>>>>> _______________________________________________
>>>>> Pharo-project mailing list
>>>>> [hidden email]
>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>>
>>>>
>>>> _______________________________________________
>>>> Pharo-project mailing list
>>>> [hidden email]
>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>
>>>>
>>>
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

> Le 09/05/2010 18:48, Igor Stasenko a écrit :
>>
>> On 9 May 2010 13:29, Alain Plantec<[hidden email]>  wrote:
>>
>>>
>> This was a result of lack of attention, not my real intent. :)
>>
>
> ok, so we have the same version
>>
>> But unfortunately, this solution seems to be broken to. It works one time,
>>>
>>> but while trying to
>>> condenseChanges again several time, my squeak process is killed without
>>> flushing any log.
>>>
>>
>> You mean running #condenseChanges multiple times in a row, or
>> interrupting it and then start over again?
>> You should know, that it is important to not interrupt it in the
>> middle, because when you doing that,
>> part of your methods having old source pointers, while rest part
>> having new ones,
>> and there is no way to tell, which one is correct.
>>
>
> I take a fresh image.
> SmalltalkImage current condenseChanges.
> then
> I update some methods individually and save my image,
> then
> SmalltalkImage current condenseChanges.
> and it crash
>