Issue 5167 in pharo: Fix weak finalization thrashing

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

Issue 5167 in pharo: Fix weak finalization thrashing

pharo
Status: Accepted
Owner: [hidden email]
CC: [hidden email]

New issue 5167 by [hidden email]: Fix weak finalization thrashing
http://code.google.com/p/pharo/issues/detail?id=5167

==================== Summary ====================

Name: Collections-dtl.469
Author: dtl
Time: 30 December 2011, 12:57:50.064 am
UUID: f8d7d7a5-e127-4108-95ae-6ef48ac43d86
Ancestors: Collections-cmm.468

Fix weak finalization thrashing, root cause of user interrupt issues.

Perform the check for VM support of new finalization once at image startUp  
time. This prevents the weak finalization process from creating new weak  
references that result in thrashing between VM and image as the VM signals  
the the image to clean up weak references, and the finalization process  
produces additional weak to be removed.

With this change the finalization process is much less active, and user  
interrupt handling works as intended such that any of the following can  
interrupted in the expected way:

  "[true] whileTrue"
  "[[true] whileTrue] forkAt: Processor userSchedulingPriority + 1"
  "Smalltalk createStackOverflow"
  "[Smalltalk createStackOverflow] forkAt: Processor userSchedulingPriority  
+ 1"

=============== Diff against Collections-cmm.468 ===============

Item was changed:
  ----- Method: WeakArray class>>finalizationProcess (in category 'private')  
-----
  finalizationProcess

+       | initialized |
+       initialized := false.
+       [FinalizationSemaphore wait.
+       initialized ifFalse: ["check VM capability once at image startup  
time"
+               WeakFinalizationList initTestPair.
+               Smalltalk garbageCollect.
-       [ WeakFinalizationList initTestPair.
-       FinalizationSemaphore wait.
-       FinalizationLock critical:
-               [
               WeakFinalizationList checkTestPair.
+               initialized := true].
+       FinalizationLock critical:
+               [FinalizationDependents do:
+                       [ :weakDependent |
-               FinalizationDependents do:
-                       [:weakDependent |
                       weakDependent ifNotNil:
                               [weakDependent finalizeValues]]]
               ifError:
+               [:msg :rcvr | rcvr error: msg]] repeat!
-               [:msg :rcvr | rcvr error: msg] ] repeat!






well, i have doubts that it causing a problem. But yes, a single check
at startup is enough,
except that forcing full GC makes startup slower.
So, making it lazy will be better.

On Jan 10, 2012, at 1:22 PM, Igor Stasenko wrote:

hmm.. but where the startup code?
do they use #restartFinalizationProcess in some startup method?

On 7 January 2012 22:01, Stéphane Ducasse <[hidden email]> wrote:


[hidden email] via lists.squeakfoundation.org
12/30/11 (8 days ago)


to squeak-dev, packages

David T. Lewis uploaded a new version of Collections to project The Trunk:
http://source.squeak.org/trunk/Collections-dtl.469.mcz

==================== Summary ====================

Name: Collections-dtl.469
Author: dtl
Time: 30 December 2011, 12:57:50.064 am
UUID: f8d7d7a5-e127-4108-95ae-6ef48ac43d86
Ancestors: Collections-cmm.468

Fix weak finalization thrashing, root cause of user interrupt issues.

Perform the check for VM support of new finalization once at image startUp  
time. This prevents the weak finalization process from creating new weak  
references that result in thrashing between VM and image as the VM signals  
the the image to clean up weak references, and the finalization process  
produces additional weak to be removed.

With this change the finalization process is much less active, and user  
interrupt handling works as intended such that any of the following can  
interrupted in the expected way:

  "[true] whileTrue"
  "[[true] whileTrue] forkAt: Processor userSchedulingPriority + 1"
  "Smalltalk createStackOverflow"
  "[Smalltalk createStackOverflow] forkAt: Processor userSchedulingPriority  
+ 1"

=============== Diff against Collections-cmm.468 ===============

Item was changed:
  ----- Method: WeakArray class>>finalizationProcess (in category 'private')  
-----
  finalizationProcess

+       | initialized |
+       initialized := false.
+       [FinalizationSemaphore wait.
+       initialized ifFalse: ["check VM capability once at image startup  
time"
+               WeakFinalizationList initTestPair.
+               Smalltalk garbageCollect.
-       [ WeakFinalizationList initTestPair.
-       FinalizationSemaphore wait.
-       FinalizationLock critical:
-               [
               WeakFinalizationList checkTestPair.
+               initialized := true].
+       FinalizationLock critical:
+               [FinalizationDependents do:
+                       [ :weakDependent |
-               FinalizationDependents do:
-                       [:weakDependent |
                       weakDependent ifNotNil:
                               [weakDependent finalizeValues]]]
               ifError:
+               [:msg :rcvr | rcvr error: msg]] repeat!
-               [:msg :rcvr | rcvr error: msg] ] repeat!



--
Best regards,
Igor Stasenko.





_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 5167 in pharo: Fix weak finalization thrashing

pharo

Comment #1 on issue 5167 by [hidden email]: Fix weak finalization  
thrashing
http://code.google.com/p/pharo/issues/detail?id=5167

Currently today, #hasNewFinalization can  return incorrect answers
(If an image is saved from one VM where HasNewFinalization has been set,  
and #hasNewFinalization is called before a finalization cycle triggers  
after image being loaded on a VM where support status is opposite than that  
of the first)
This includes the dangerous false positive, do you know what impact this  
can have on its users?
AFAICT, WeakRegistry finalizeValues seems fine (since it is guaranteed to  
not run before the finalization process triggers it)

But I'm somewhat unsure of add:executor: if ran in a thread with priority  
>= finalization process:
1. User thread is running
2. VM does GC, signals finalization sema
3. User thread resumes before finalization thread sets HasNewFinalization  
to false (since it has highest pri)
4. User thread uses add:executor:, and replaces un-finalized entries with  
nil keys...

Also, the use in announcements does not seem nearly as defensively coded,  
though I haven't looked very close.


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 5167 in pharo: Fix weak finalization thrashing

pharo

Comment #2 on issue 5167 by [hidden email]: Fix weak finalization  
thrashing
http://code.google.com/p/pharo/issues/detail?id=5167

If this turns out to be more than a theoretical problem, I'd suggest  
changes something like those below.
It both resolves the object creation in finalization thread mentioned in  
issue, as well as pretty much guarantee
the failure case above can't happen
(Exception being if finalization is used before restartFinalizationProcess  
during startup... )

WeakArray class:

finalizationProcess
        WeakFinalizationList initTestPair.
        [true] whileTrue: [
                FinalizationSemaphore wait.
                FinalizationLock critical: [
                        WeakFinalizationList checkTestPair.
                        ...

WeakFinalizationList class:
initTestPair
        HasNewFinalization := nil.
        TestItem := WeakFinalizerItem new list: (TestList := self new) object:  
Object new.

checkTestPair
        TestList ifNotNil:
                [:test | HasNewFinalization := TestList swapWithNil notNil.
                           TestList := nil]

hasNewFinalization
        HasNewFinalization == nil ifTrue: [Smalltalk vm garbageCollect.
                                                                                 
self  
checkTestPair].
        ^HasNewFinalization

initialize
   ***remove***




_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 5167 in pharo: Fix weak finalization thrashing

pharo

Comment #3 on issue 5167 by [hidden email]: Fix weak finalization  
thrashing
http://code.google.com/p/pharo/issues/detail?id=5167

Thanks henrik




_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 5167 in pharo: Fix weak finalization thrashing

pharo
Updates:
        Labels: Milestone-1.4

Comment #4 on issue 5167 by [hidden email]: Fix weak finalization  
thrashing
http://code.google.com/p/pharo/issues/detail?id=5167

Igor can you add that to your todo list?


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 5167 in pharo: Fix weak finalization thrashing

pharo

Comment #5 on issue 5167 by [hidden email]: Fix weak finalization  
thrashing
http://code.google.com/p/pharo/issues/detail?id=5167

to ensure safe boot,
  i think in
WeakArray>>startUp,
we should clear the HasNewFinalization to nil.

but "the root cause of interrupt issues" is the interrupt handling logic  
itself.
instead of interrupting runaway process, it shoots into the sky , and  
boom.. interrupts the finalization process.
Since a) a runaway process producing a lot of garbage, causing GC to  
trigger often
which of course leads to b) waking up the finalization process
and since c) finalization process usually having higher priority than  
runaway process
and so, there's high chances that when you hit interrupt key, you  
interrupting finalization..


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 5167 in pharo: Fix weak finalization thrashing

pharo

Comment #6 on issue 5167 by [hidden email]: Fix weak finalization  
thrashing
http://code.google.com/p/pharo/issues/detail?id=5167

There's one more thing about interrupts:
  a debugger is scheduled at ui process.

so, if interrupt handler interrupts wrong process , a runaway process keeps  
working,
keep blocking UI process.. and this is why you cannot see a debugger  
immediately and forced to press interrupt key multiple times.


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 5167 in pharo: Fix weak finalization thrashing

pharo

Comment #7 on issue 5167 by [hidden email]: Fix weak finalization  
thrashing
http://code.google.com/p/pharo/issues/detail?id=5167

There's one more thing about interrupts:
  a debugger is scheduled at ui process.

so, if interrupt handler interrupts wrong process , a runaway process keeps  
working,
keep blocking UI process.. and this is why you cannot see a debugger  
immediately and forced to press interrupt key multiple times.

Now try with this changeset

Attachments:
        interrupt.1.cs  1.8 KB


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 5167 in pharo: Fix weak finalization thrashing

pharo
Updates:
        Status: FixToInclude

Comment #8 on issue 5167 by [hidden email]: Fix weak finalization  
thrashing
http://code.google.com/p/pharo/issues/detail?id=5167

more fixes for interrupting attached

Attachments:
        interrupt2.1.cs  2.4 KB


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 5167 in pharo: Fix weak finalization thrashing

pharo
Updates:
        Status: Workneeded

Comment #9 on issue 5167 by [hidden email]: Fix weak finalization  
thrashing
http://code.google.com/p/pharo/issues/detail?id=5167

(No comment was entered for this change.)


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 5167 in pharo: Fix weak finalization thrashing

pharo

Comment #10 on issue 5167 by [hidden email]: Fix weak finalization  
thrashing
http://code.google.com/p/pharo/issues/detail?id=5167

see another fix in the PharoInbox

  Polymorph-Widgets-CamilloBruni.578


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 5167 in pharo: Fix weak finalization thrashing

pharo
Updates:
        Cc: [hidden email]

Comment #11 on issue 5167 by [hidden email]: Fix weak finalization  
thrashing
http://code.google.com/p/pharo/issues/detail?id=5167

(No comment was entered for this change.)


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 5167 in pharo: Fix weak finalization thrashing

pharo
Updates:
        Status: FixToInclude

Comment #12 on issue 5167 by [hidden email]: Fix weak finalization  
thrashing
http://code.google.com/p/pharo/issues/detail?id=5167

(No comment was entered for this change.)


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 5167 in pharo: Fix weak finalization thrashing

pharo
Updates:
        Status: Workneeded

Comment #13 on issue 5167 by [hidden email]: Fix weak finalization  
thrashing
http://code.google.com/p/pharo/issues/detail?id=5167

in 14317


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 5167 in pharo: Fix weak finalization thrashing

pharo

Comment #14 on issue 5167 by [hidden email]: Fix weak finalization  
thrashing
http://code.google.com/p/pharo/issues/detail?id=5167

Polymorph-Widgets-CamilloBruni.578 is result of my overlook.

the method should be reverted back to original version.
there is no need to do suspend before add-deferred-action.


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 5167 in pharo: Fix weak finalization thrashing

pharo
Updates:
        Labels: Type-Bug

Comment #15 on issue 5167 by [hidden email]: Fix weak finalization  
thrashing
http://code.google.com/p/pharo/issues/detail?id=5167

(No comment was entered for this change.)


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 5167 in pharo: Fix weak finalization thrashing

pharo
Updates:
        Labels: -Milestone-1.4

Comment #16 on issue 5167 by [hidden email]: Fix weak finalization  
thrashing
http://code.google.com/p/pharo/issues/detail?id=5167

I think we will not work on this in 1.4, but later...


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 5167 in pharo: Fix weak finalization thrashing

pharo
Updates:
        Status: WorkNeeded

Comment #17 on issue 5167 by [hidden email]: Fix weak finalization  
thrashing
http://code.google.com/p/pharo/issues/detail?id=5167

(No comment was entered for this change.)


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 5167 in pharo: Fix weak finalization thrashing

pharo

Comment #18 on issue 5167 by [hidden email]: Fix weak finalization  
thrashing
http://code.google.com/p/pharo/issues/detail?id=5167

Igor can you add this bug entry to your todo we should flush these and I  
cannot because I'm not good enough :(


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 5167 in pharo: Fix weak finalization thrashing

pharo
Updates:
        Status: Closed

Comment #19 on issue 5167 by [hidden email]: Fix weak finalization  
thrashing
http://code.google.com/p/pharo/issues/detail?id=5167

Lets close this issue, and reopen a new one if needed. First, the title is  
misleading, second the change(s) is integrated already.


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker