Smalltalk › Squeak › Squeak - Dev

Finalization enhancement for 4.2

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

7 messages Options

Chris Muller-3

Finalization enhancement for 4.2

I have packaged up Igor's change-sets for the finalization fix:

http://bugs.squeak.org/view.php?id=7525
http://bugs.squeak.org/view.php?id=7473

into their respective MC packages and uploaded them to the Inbox.

Recall the original discussion:

http://lists.squeakfoundation.org/pipermail/vm-dev/2010-March/003958.html

I have been running with Igor's enhanced VM and image-side
finalization patches for a few weeks now. The results are striking.
Squeak applications using large Weak-collections deteriorate to a
point of unusability without this fix because Squeak spends 95% of its
time in the finalization process, leaving no time for application
processing. With this fix installed, CPU utilization is 0% when the
application is idle, providing excellent responsiveness.

This is a big, big, win for any application that uses Weak
collections, and a win for Squeak overall because it's own use of
weak-collections is enhanced.

While it seems to be backward compatible, e.g. I am able to run the
enhanced VM on older images (including 3.9), and I am able to run with
these image changes on an older VM, both are required to take
advantage of the enhanced finalization.

I would like to push this into the trunk, but will need help from the
VMMaker developers for that piece. David?

For Magma, this wil be the most exciting and important fix since
#becomeForward:copyHash: introduced back in 2004.

- Chris

David T. Lewis

Re: Finalization enhancement for 4.2

CC to vm-dev list, follow up on technical issues should go to vm-dev.

Chris, Igor, this sounds very encouraging. Yes, I'll help on the
VM side, with guidance from Andreas and others (I am not technically
competent on this issue). I'll probably be off list for a few days,
but will catch up on this next week.

Andreas, Eliot, others - any advice or guidance?

Dave

On Thu, May 20, 2010 at 10:14:00AM -0500, Chris Muller wrote:

> I have packaged up Igor's change-sets for the finalization fix:
>
> http://bugs.squeak.org/view.php?id=7525
> http://bugs.squeak.org/view.php?id=7473
>
> into their respective MC packages and uploaded them to the Inbox.
>
> Recall the original discussion:
>
> http://lists.squeakfoundation.org/pipermail/vm-dev/2010-March/003958.html
>
> I have been running with Igor's enhanced VM and image-side
> finalization patches for a few weeks now. The results are striking.
> Squeak applications using large Weak-collections deteriorate to a
> point of unusability without this fix because Squeak spends 95% of its
> time in the finalization process, leaving no time for application
> processing. With this fix installed, CPU utilization is 0% when the
> application is idle, providing excellent responsiveness.
>
> This is a big, big, win for any application that uses Weak
> collections, and a win for Squeak overall because it's own use of
> weak-collections is enhanced.
>
> While it seems to be backward compatible, e.g. I am able to run the
> enhanced VM on older images (including 3.9), and I am able to run with
> these image changes on an older VM, both are required to take
> advantage of the enhanced finalization.
>
> I would like to push this into the trunk, but will need help from the
> VMMaker developers for that piece. David?
>
> For Magma, this wil be the most exciting and important fix since
> #becomeForward:copyHash: introduced back in 2004.
>
> - Chris

Andreas.Raab

Re: Finalization enhancement for 4.2

On 5/20/2010 8:59 AM, David T. Lewis wrote:
> CC to vm-dev list, follow up on technical issues should go to vm-dev.
>
> Chris, Igor, this sounds very encouraging. Yes, I'll help on the
> VM side, with guidance from Andreas and others (I am not technically
> competent on this issue). I'll probably be off list for a few days,
> but will catch up on this next week.
>
> Andreas, Eliot, others - any advice or guidance?

It's the same stuff we talked about a while ago on vm-dev. I'm
definitely +1 on it. It's simple and straightforward and solves some
real issues in finalization.

Cheers,
- Andreas

> On Thu, May 20, 2010 at 10:14:00AM -0500, Chris Muller wrote:
>> I have packaged up Igor's change-sets for the finalization fix:
>>
>> http://bugs.squeak.org/view.php?id=7525
>> http://bugs.squeak.org/view.php?id=7473
>>
>> into their respective MC packages and uploaded them to the Inbox.
>>
>> Recall the original discussion:
>>
>> http://lists.squeakfoundation.org/pipermail/vm-dev/2010-March/003958.html
>>
>> I have been running with Igor's enhanced VM and image-side
>> finalization patches for a few weeks now. The results are striking.
>> Squeak applications using large Weak-collections deteriorate to a
>> point of unusability without this fix because Squeak spends 95% of its
>> time in the finalization process, leaving no time for application
>> processing. With this fix installed, CPU utilization is 0% when the
>> application is idle, providing excellent responsiveness.
>>
>> This is a big, big, win for any application that uses Weak
>> collections, and a win for Squeak overall because it's own use of
>> weak-collections is enhanced.
>>
>> While it seems to be backward compatible, e.g. I am able to run the
>> enhanced VM on older images (including 3.9), and I am able to run with
>> these image changes on an older VM, both are required to take
>> advantage of the enhanced finalization.
>>
>> I would like to push this into the trunk, but will need help from the
>> VMMaker developers for that piece. David?
>>
>> For Magma, this wil be the most exciting and important fix since
>> #becomeForward:copyHash: introduced back in 2004.
>>
>> - Chris
>
>

Levente Uzonyi-2

Re: Finalization enhancement for 4.2

In reply to this post by Chris Muller-3

On Thu, 20 May 2010, Chris Muller wrote:

A few notes:
- since 4.1 only WeakRegistries have to be registered to the finalization
process. Weak dictionaries with a custom finalizers can also benefit
from registration, but they should work without it. The custom finalizers
were also introduced in 4.1, so most weak collections are not affected in
4.1.
- WeakFinalizationRegistry lacks synchronization, so you have to be
careful when you try to replace the use of WeakRegistry with
WeakFinalizationRegistry. (this was also discussed in the above thread)

>
> This is a big, big, win for any application that uses Weak
> collections, and a win for Squeak overall because it's own use of
> weak-collections is enhanced.

See above.

>
> While it seems to be backward compatible, e.g. I am able to run the
> enhanced VM on older images (including 3.9), and I am able to run with
> these image changes on an older VM, both are required to take
> advantage of the enhanced finalization.

See above.

Levente

>
> I would like to push this into the trunk, but will need help from the
> VMMaker developers for that piece. David?
>
> For Magma, this wil be the most exciting and important fix since
> #becomeForward:copyHash: introduced back in 2004.
>
> - Chris
>
>

Chris Muller-3

Re: Finalization enhancement for 4.2

> - WeakFinalizationRegistry lacks synchronization, so you have to be careful
> when you try to replace the use of WeakRegistry with
> WeakFinalizationRegistry. (this was also discussed in the above thread)

Oh, right, I will fix that..

Eliot Miranda-2

Re: Finalization enhancement for 4.2

In reply to this post by David T. Lewis

On Thu, May 20, 2010 at 8:59 AM, David T. Lewis <[hidden email]> wrote:

CC to vm-dev list, follow up on technical issues should go to vm-dev.

Chris, Igor, this sounds very encouraging. Yes, I'll help on the
VM side, with guidance from Andreas and others (I am not technically
competent on this issue). I'll probably be off list for a few days,
but will catch up on this next week.

Andreas, Eliot, others - any advice or guidance?

As long as there are really good comments on how to use this stuff I'm happy. There should be class comments in the relevant classes and pointers to them in the relevant methods. The comments need to explain what are the registration requirements and general usage patterns from both the user perspective and the implementor/extender perspective.

Dave

On Thu, May 20, 2010 at 10:14:00AM -0500, Chris Muller wrote:
> I have packaged up Igor's change-sets for the finalization fix:
>
> http://bugs.squeak.org/view.php?id=7525
> http://bugs.squeak.org/view.php?id=7473
>
> into their respective MC packages and uploaded them to the Inbox.
>
> Recall the original discussion:
>
> http://lists.squeakfoundation.org/pipermail/vm-dev/2010-March/003958.html
>
> I have been running with Igor's enhanced VM and image-side
> finalization patches for a few weeks now. The results are striking.
> Squeak applications using large Weak-collections deteriorate to a
> point of unusability without this fix because Squeak spends 95% of its
> time in the finalization process, leaving no time for application
> processing. With this fix installed, CPU utilization is 0% when the
> application is idle, providing excellent responsiveness.
>
> This is a big, big, win for any application that uses Weak
> collections, and a win for Squeak overall because it's own use of
> weak-collections is enhanced.
>
> While it seems to be backward compatible, e.g. I am able to run the
> enhanced VM on older images (including 3.9), and I am able to run with
> these image changes on an older VM, both are required to take
> advantage of the enhanced finalization.
>
> I would like to push this into the trunk, but will need help from the
> VMMaker developers for that piece. David?
>
> For Magma, this wil be the most exciting and important fix since
> #becomeForward:copyHash: introduced back in 2004.
>
> - Chris

Igor Stasenko

Re: [Pharo-project] Fwd: [squeak-dev] Finalization enhancement for 4.2

In reply to this post by David T. Lewis

Dear all,
I will make a better implementation at image side, and cover
everything with comments, as Eliot suggested.

On 20 May 2010 22:23, Stéphane Ducasse <[hidden email]> wrote:
> igor
>
> I have a question about your new finalization. Can we add it to Pharo?

yes, you can. But without VM modifications it won't work. That's why i
postponed image-side integration
for a better time(s).
Old images, how Chris already mentioned, don't need to be aware of
changes on VM side and will continue
to function as if nothing happen.

> Should we replace something else (I mean it is something to replace an existing solution).

Well, it works best if you having a lot of weak refs, while only few
of them dying time to time.
You are free to implement a post-mortem action however you like (but
this is also true for current implementation,
except that new scheme is much more effective).

> How does it interact with existing code?

I will replace an existing WeakRegistry, which will work exactly as an old one.
Also, if you want, i can add a fallback mechanism, if you will run
your image using older VM.

Any other uses of new finalization is up to you :)

As you may guess, a Magma exploits new finalization enhancement for own needs.
For this, I implemented a specialized weak dictionaries, which clean
themselves automatically
(Magma don't needs to deal with finalization of objects, it just needs
to keep buffers free of garbage).
This was a real-world use of new features, which shown us its benefits.
Thanks to Cris for taking care integrating and testing it.

> You see I will learn a lot again :)
>
> Stef
>
>
>> From: "David T. Lewis" <[hidden email]>
>> Date: May 20, 2010 5:59:07 PM GMT+02:00
>> To: [hidden email], The general-purpose Squeak developers list <[hidden email]>
>> Cc: [hidden email]
>> Subject: Re: [squeak-dev] Finalization enhancement for 4.2
>> Reply-To: The general-purpose Squeak developers list <[hidden email]>
>>
>> CC to vm-dev list, follow up on technical issues should go to vm-dev.
>>
>> Chris, Igor, this sounds very encouraging. Yes, I'll help on the
>> VM side, with guidance from Andreas and others (I am not technically
>> competent on this issue). I'll probably be off list for a few days,
>> but will catch up on this next week.
>>
>> Andreas, Eliot, others - any advice or guidance?
>>
>> Dave
>>
>> On Thu, May 20, 2010 at 10:14:00AM -0500, Chris Muller wrote:
>>> I have packaged up Igor's change-sets for the finalization fix:
>>>
>>> http://bugs.squeak.org/view.php?id=7525
>>> http://bugs.squeak.org/view.php?id=7473
>>>
>>> into their respective MC packages and uploaded them to the Inbox.
>>>
>>> Recall the original discussion:
>>>
>>> http://lists.squeakfoundation.org/pipermail/vm-dev/2010-March/003958.html
>>>
>>> I have been running with Igor's enhanced VM and image-side
>>> finalization patches for a few weeks now. The results are striking.
>>> Squeak applications using large Weak-collections deteriorate to a
>>> point of unusability without this fix because Squeak spends 95% of its
>>> time in the finalization process, leaving no time for application
>>> processing. With this fix installed, CPU utilization is 0% when the
>>> application is idle, providing excellent responsiveness.
>>>
>>> This is a big, big, win for any application that uses Weak
>>> collections, and a win for Squeak overall because it's own use of
>>> weak-collections is enhanced.
>>>
>>> While it seems to be backward compatible, e.g. I am able to run the
>>> enhanced VM on older images (including 3.9), and I am able to run with
>>> these image changes on an older VM, both are required to take
>>> advantage of the enhanced finalization.
>>>
>>> I would like to push this into the trunk, but will need help from the
>>> VMMaker developers for that piece. David?
>>>
>>> For Magma, this wil be the most exciting and important fix since
>>> #becomeForward:copyHash: introduced back in 2004.
>>>
>>> - Chris
>>
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

--
Best regards,
Igor Stasenko AKA sig.