Another finalization concern: error handling

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
39 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Another finalization concern: error handling

Igor Stasenko
Hello,

here a situation, with which we can deal in more safer manner:

Suppose you have a weak registry, populated by different objects and
their executors.

Now, when some of them died, a weak registry performs finalization.

The potential danger is , that if there's an error triggered by some
executor(s),
then rest of executors will have no chance to run and will be
forgotten, causing memory leakage.

What you think, should we handle this more graciously?

(Consider a following meta-code)

WeakRegistry>>finalizeValues
| executors |
  executors := self gatherExecutorsToRun.

  executors do: [:ex |
     [ ex finalize ] fork.
  ].

in this way, if any executor's #finalize causing error, it won't
interfere with other executors, and they will accomplish their task
normally.
Of course, i'm not saying that we should use #fork for this, because
it is costly. Similar could be done w/o forking.
I just wanted to show a simplest code with which we could achieve a
more gracious error handling.

P.S. of course, in a first place it would be good to make sure that we
writing executors, which can't cause an error during finalization.
But bad things happen, and we should make sure that rest of system
won't be put on its knees because of some stupid bug in a single
#finalize.

--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: Another finalization concern: error handling

Igor Stasenko
On 11 October 2010 14:40, Igor Stasenko <[hidden email]> wrote:

> Hello,
>
> here a situation, with which we can deal in more safer manner:
>
> Suppose you have a weak registry, populated by different objects and
> their executors.
>
> Now, when some of them died, a weak registry performs finalization.
>
> The potential danger is , that if there's an error triggered by some
> executor(s),
> then rest of executors will have no chance to run and will be
> forgotten, causing memory leakage.
>
> What you think, should we handle this more graciously?
>
> (Consider a following meta-code)
>
> WeakRegistry>>finalizeValues
> | executors |
>  executors := self gatherExecutorsToRun.
>
>  executors do: [:ex |
>     [ ex finalize ] fork.
>  ].
>

oh, and this gets even more complicated, if we keep supporting
multiple finalizers per single object. :)

executors do: [:ex |
  ex hasMultipleExecutors ifTrue: [ ex do: [ :eex |   [ eex finalize ] fork ] ]
  ifFalse: [  [ ex finalize ] fork ].



> in this way, if any executor's #finalize causing error, it won't
> interfere with other executors, and they will accomplish their task
> normally.
> Of course, i'm not saying that we should use #fork for this, because
> it is costly. Similar could be done w/o forking.
> I just wanted to show a simplest code with which we could achieve a
> more gracious error handling.
>
> P.S. of course, in a first place it would be good to make sure that we
> writing executors, which can't cause an error during finalization.
> But bad things happen, and we should make sure that rest of system
> won't be put on its knees because of some stupid bug in a single
> #finalize.
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: Another finalization concern: error handling

Levente Uzonyi-2
In reply to this post by Igor Stasenko
On Mon, 11 Oct 2010, Igor Stasenko wrote:

> Hello,
>
> here a situation, with which we can deal in more safer manner:
>
> Suppose you have a weak registry, populated by different objects and
> their executors.
>
> Now, when some of them died, a weak registry performs finalization.
>
> The potential danger is , that if there's an error triggered by some
> executor(s),
> then rest of executors will have no chance to run and will be
> forgotten, causing memory leakage.
>
> What you think, should we handle this more graciously?
>
> (Consider a following meta-code)
>
> WeakRegistry>>finalizeValues
> | executors |
>  executors := self gatherExecutorsToRun.
>
>  executors do: [:ex |
>     [ ex finalize ] fork.
>  ].

The "gatherExecutorsToRun" part was implemented in the previous version
of WeakRegistry (history is not available from the image...). #finalize
wasn't sent from the protected block. The current version lacks it, so it
will deadlock if a finalizer will try to access the same WeakRegistry.
Other kind of deadlocks are also possible. For example when a finalizer
can access a semaphore which is also used by another process that
uses the same semaphore and also uses the WeakRegistry, but locks them in
a different order (JNIPort).
This only happens if the VM doesn't support the new finalization scheme
which is the case for most current VMs.

There is another issue with removal, and this affects the new finalization
scheme too, because the finalizer of valuesDictionary is #finalizeValues.
This means that finalization can happen in a process other than the
finalization process. This can lead to random errors.

So I think parts of the old WeakRegistry implementation should be
restored, like:
- WeakRegistry should collect executors
- executors should be evaluated outside the protected block

>
> in this way, if any executor's #finalize causing error, it won't
> interfere with other executors, and they will accomplish their task
> normally.
> Of course, i'm not saying that we should use #fork for this, because
> it is costly. Similar could be done w/o forking.
> I just wanted to show a simplest code with which we could achieve a
> more gracious error handling.

Wouldn't it be better to use an exception handler?


Levente

>
> P.S. of course, in a first place it would be good to make sure that we
> writing executors, which can't cause an error during finalization.
> But bad things happen, and we should make sure that rest of system
> won't be put on its knees because of some stupid bug in a single
> #finalize.
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: Another finalization concern: error handling

Schwab,Wilhelm K
Levente,

A similar discussion arose around Dolphin's event (#trigger*) mechanism.  My recollection is that it was not fully addressed due to performance concerns.  Forking and error handlers both have their costs.  I'm not saying we should necessarily follow (we probably should not), though with careful design, an interrupted chain of events might survive to be handled on a subsequent attempt.

I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point.

Bill



________________________________________
From: [hidden email] [[hidden email]] On Behalf Of Levente Uzonyi [[hidden email]]
Sent: Monday, October 11, 2010 9:51 AM
To: Pharo Development
Cc: The general-purpose Squeak developers list
Subject: Re: [Pharo-project] Another finalization concern: error handling

On Mon, 11 Oct 2010, Igor Stasenko wrote:

> Hello,
>
> here a situation, with which we can deal in more safer manner:
>
> Suppose you have a weak registry, populated by different objects and
> their executors.
>
> Now, when some of them died, a weak registry performs finalization.
>
> The potential danger is , that if there's an error triggered by some
> executor(s),
> then rest of executors will have no chance to run and will be
> forgotten, causing memory leakage.
>
> What you think, should we handle this more graciously?
>
> (Consider a following meta-code)
>
> WeakRegistry>>finalizeValues
> | executors |
>  executors := self gatherExecutorsToRun.
>
>  executors do: [:ex |
>     [ ex finalize ] fork.
>  ].

The "gatherExecutorsToRun" part was implemented in the previous version
of WeakRegistry (history is not available from the image...). #finalize
wasn't sent from the protected block. The current version lacks it, so it
will deadlock if a finalizer will try to access the same WeakRegistry.
Other kind of deadlocks are also possible. For example when a finalizer
can access a semaphore which is also used by another process that
uses the same semaphore and also uses the WeakRegistry, but locks them in
a different order (JNIPort).
This only happens if the VM doesn't support the new finalization scheme
which is the case for most current VMs.

There is another issue with removal, and this affects the new finalization
scheme too, because the finalizer of valuesDictionary is #finalizeValues.
This means that finalization can happen in a process other than the
finalization process. This can lead to random errors.

So I think parts of the old WeakRegistry implementation should be
restored, like:
- WeakRegistry should collect executors
- executors should be evaluated outside the protected block

>
> in this way, if any executor's #finalize causing error, it won't
> interfere with other executors, and they will accomplish their task
> normally.
> Of course, i'm not saying that we should use #fork for this, because
> it is costly. Similar could be done w/o forking.
> I just wanted to show a simplest code with which we could achieve a
> more gracious error handling.

Wouldn't it be better to use an exception handler?


Levente

>
> P.S. of course, in a first place it would be good to make sure that we
> writing executors, which can't cause an error during finalization.
> But bad things happen, and we should make sure that rest of system
> won't be put on its knees because of some stupid bug in a single
> #finalize.
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: Another finalization concern: error handling

Levente Uzonyi-2
On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote:

> Levente,
>
> A similar discussion arose around Dolphin's event (#trigger*) mechanism.  My recollection is that it was not fully addressed due to performance concerns.  Forking and error handlers both have their costs.  I'm not saying we should necessarily follow (we probably should not), though with careful design, an interrupted chain of events might survive to be handled on a subsequent attempt.
>
> I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point.

Smalltalk is not C. Try this:

| file |
file := StandardFileStream fileNamed: 'foo.txt'.
file close.
file primClose: (file instVarNamed: #fileID).
"Those pesky plugins save us all the time. ;)"


Levente

>
> Bill
>
>
>
> ________________________________________
> From: [hidden email] [[hidden email]] On Behalf Of Levente Uzonyi [[hidden email]]
> Sent: Monday, October 11, 2010 9:51 AM
> To: Pharo Development
> Cc: The general-purpose Squeak developers list
> Subject: Re: [Pharo-project] Another finalization concern: error handling
>
> On Mon, 11 Oct 2010, Igor Stasenko wrote:
>
>> Hello,
>>
>> here a situation, with which we can deal in more safer manner:
>>
>> Suppose you have a weak registry, populated by different objects and
>> their executors.
>>
>> Now, when some of them died, a weak registry performs finalization.
>>
>> The potential danger is , that if there's an error triggered by some
>> executor(s),
>> then rest of executors will have no chance to run and will be
>> forgotten, causing memory leakage.
>>
>> What you think, should we handle this more graciously?
>>
>> (Consider a following meta-code)
>>
>> WeakRegistry>>finalizeValues
>> | executors |
>>  executors := self gatherExecutorsToRun.
>>
>>  executors do: [:ex |
>>     [ ex finalize ] fork.
>>  ].
>
> The "gatherExecutorsToRun" part was implemented in the previous version
> of WeakRegistry (history is not available from the image...). #finalize
> wasn't sent from the protected block. The current version lacks it, so it
> will deadlock if a finalizer will try to access the same WeakRegistry.
> Other kind of deadlocks are also possible. For example when a finalizer
> can access a semaphore which is also used by another process that
> uses the same semaphore and also uses the WeakRegistry, but locks them in
> a different order (JNIPort).
> This only happens if the VM doesn't support the new finalization scheme
> which is the case for most current VMs.
>
> There is another issue with removal, and this affects the new finalization
> scheme too, because the finalizer of valuesDictionary is #finalizeValues.
> This means that finalization can happen in a process other than the
> finalization process. This can lead to random errors.
>
> So I think parts of the old WeakRegistry implementation should be
> restored, like:
> - WeakRegistry should collect executors
> - executors should be evaluated outside the protected block
>
>>
>> in this way, if any executor's #finalize causing error, it won't
>> interfere with other executors, and they will accomplish their task
>> normally.
>> Of course, i'm not saying that we should use #fork for this, because
>> it is costly. Similar could be done w/o forking.
>> I just wanted to show a simplest code with which we could achieve a
>> more gracious error handling.
>
> Wouldn't it be better to use an exception handler?
>
>
> Levente
>
>>
>> P.S. of course, in a first place it would be good to make sure that we
>> writing executors, which can't cause an error during finalization.
>> But bad things happen, and we should make sure that rest of system
>> won't be put on its knees because of some stupid bug in a single
>> #finalize.
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Another finalization concern: error handling

Igor Stasenko
In reply to this post by Levente Uzonyi-2
On 11 October 2010 16:51, Levente Uzonyi <[hidden email]> wrote:

> On Mon, 11 Oct 2010, Igor Stasenko wrote:
>
>> Hello,
>>
>> here a situation, with which we can deal in more safer manner:
>>
>> Suppose you have a weak registry, populated by different objects and
>> their executors.
>>
>> Now, when some of them died, a weak registry performs finalization.
>>
>> The potential danger is , that if there's an error triggered by some
>> executor(s),
>> then rest of executors will have no chance to run and will be
>> forgotten, causing memory leakage.
>>
>> What you think, should we handle this more graciously?
>>
>> (Consider a following meta-code)
>>
>> WeakRegistry>>finalizeValues
>> | executors |
>>  executors := self gatherExecutorsToRun.
>>
>>  executors do: [:ex |
>>    [ ex finalize ] fork.
>>  ].
>
> The "gatherExecutorsToRun" part was implemented in the previous version of
> WeakRegistry (history is not available from the image...). #finalize wasn't
> sent from the protected block. The current version lacks it, so it will
> deadlock if a finalizer will try to access the same WeakRegistry. Other kind
> of deadlocks are also possible. For example when a finalizer can access a
> semaphore which is also used by another process that uses the same semaphore
> and also uses the WeakRegistry, but locks them in a different order
> (JNIPort).
> This only happens if the VM doesn't support the new finalization scheme
> which is the case for most current VMs.
>
> There is another issue with removal, and this affects the new finalization
> scheme too, because the finalizer of valuesDictionary is #finalizeValues.
> This means that finalization can happen in a process other than the
> finalization process. This can lead to random errors.
>
err no. valuesDictionary are held privately by weak registry. and any
access to it possible only
through instance of weak registry, which always using protect: [].

> So I think parts of the old WeakRegistry implementation should be restored,
> like:
> - WeakRegistry should collect executors
> - executors should be evaluated outside the protected block
>
agreed on this one.

But weak registry lost this feature before my intervention:

This is a code from Squeak 4.1. image:
finalizeValues
        "Some of our elements may have gone away. Look for those and activate
the associated executors."
        self protected: [ valueDictionary finalizeValues ]

ul 2/22/2010 14:23 · finalization · 2 implementors · in no change set ·

So, i thought, it not a big sacrifice, besides it runs faster.
And to my thinking, the code which needs to manipulate weak registry
during finalization smells badly.

>>
>> in this way, if any executor's #finalize causing error, it won't
>> interfere with other executors, and they will accomplish their task
>> normally.
>> Of course, i'm not saying that we should use #fork for this, because
>> it is costly. Similar could be done w/o forking.
>> I just wanted to show a simplest code with which we could achieve a
>> more gracious error handling.
>
> Wouldn't it be better to use an exception handler?
>

Well, its a bit complicated, since what i want is to:

 - report an error
 - but continue run #finalize for rest executors

or:

 - report an error (stop finalization process), then
 - continue running for next executor , if user press 'abandon'
 - or continue with running fixed code

If i'm not mistaken, currently, if you put halt inside a finalization
process (like in some #finalize method) and then abandon it,
you won't have finalization process in system anymore, because it will
be terminated.

>
> Levente
>
>>
>> P.S. of course, in a first place it would be good to make sure that we
>> writing executors, which can't cause an error during finalization.
>> But bad things happen, and we should make sure that rest of system
>> won't be put on its knees because of some stupid bug in a single
>> #finalize.
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>
>



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: Another finalization concern: error handling

Schwab,Wilhelm K
In reply to this post by Levente Uzonyi-2
Levente,

Ok, but just because the system saves us at the last instant does not mean that we should be going out of our way to multiply free external resources.  Files are well known to the vm; other things (GSL vectors/matrices comes to mind) will not enjoy such protections.  This strikes me as a feature that got added when errors arose from attempts to add redundant executors, ultimately due to lack of thread safety.  Many things that are being fixed at great cost got started (it sure seems) because someone added a feature or placed something in or too near to Object only to avoid errors vs. finding and fixing the real problem.

Maybe you have made a strong argument for multiple executors; if so, I've missed it.  Right now, it looks like a design flaw instead of a feature.

Bill



________________________________________
From: [hidden email] [[hidden email]] On Behalf Of Levente Uzonyi [[hidden email]]
Sent: Monday, October 11, 2010 10:24 AM
To: [hidden email]
Subject: Re: [Pharo-project] Another finalization concern: error handling

On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote:

> Levente,
>
> A similar discussion arose around Dolphin's event (#trigger*) mechanism.  My recollection is that it was not fully addressed due to performance concerns.  Forking and error handlers both have their costs.  I'm not saying we should necessarily follow (we probably should not), though with careful design, an interrupted chain of events might survive to be handled on a subsequent attempt.
>
> I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point.

Smalltalk is not C. Try this:

| file |
file := StandardFileStream fileNamed: 'foo.txt'.
file close.
file primClose: (file instVarNamed: #fileID).
"Those pesky plugins save us all the time. ;)"


Levente

>
> Bill
>
>
>
> ________________________________________
> From: [hidden email] [[hidden email]] On Behalf Of Levente Uzonyi [[hidden email]]
> Sent: Monday, October 11, 2010 9:51 AM
> To: Pharo Development
> Cc: The general-purpose Squeak developers list
> Subject: Re: [Pharo-project] Another finalization concern: error handling
>
> On Mon, 11 Oct 2010, Igor Stasenko wrote:
>
>> Hello,
>>
>> here a situation, with which we can deal in more safer manner:
>>
>> Suppose you have a weak registry, populated by different objects and
>> their executors.
>>
>> Now, when some of them died, a weak registry performs finalization.
>>
>> The potential danger is , that if there's an error triggered by some
>> executor(s),
>> then rest of executors will have no chance to run and will be
>> forgotten, causing memory leakage.
>>
>> What you think, should we handle this more graciously?
>>
>> (Consider a following meta-code)
>>
>> WeakRegistry>>finalizeValues
>> | executors |
>>  executors := self gatherExecutorsToRun.
>>
>>  executors do: [:ex |
>>     [ ex finalize ] fork.
>>  ].
>
> The "gatherExecutorsToRun" part was implemented in the previous version
> of WeakRegistry (history is not available from the image...). #finalize
> wasn't sent from the protected block. The current version lacks it, so it
> will deadlock if a finalizer will try to access the same WeakRegistry.
> Other kind of deadlocks are also possible. For example when a finalizer
> can access a semaphore which is also used by another process that
> uses the same semaphore and also uses the WeakRegistry, but locks them in
> a different order (JNIPort).
> This only happens if the VM doesn't support the new finalization scheme
> which is the case for most current VMs.
>
> There is another issue with removal, and this affects the new finalization
> scheme too, because the finalizer of valuesDictionary is #finalizeValues.
> This means that finalization can happen in a process other than the
> finalization process. This can lead to random errors.
>
> So I think parts of the old WeakRegistry implementation should be
> restored, like:
> - WeakRegistry should collect executors
> - executors should be evaluated outside the protected block
>
>>
>> in this way, if any executor's #finalize causing error, it won't
>> interfere with other executors, and they will accomplish their task
>> normally.
>> Of course, i'm not saying that we should use #fork for this, because
>> it is costly. Similar could be done w/o forking.
>> I just wanted to show a simplest code with which we could achieve a
>> more gracious error handling.
>
> Wouldn't it be better to use an exception handler?
>
>
> Levente
>
>>
>> P.S. of course, in a first place it would be good to make sure that we
>> writing executors, which can't cause an error during finalization.
>> But bad things happen, and we should make sure that rest of system
>> won't be put on its knees because of some stupid bug in a single
>> #finalize.
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Another finalization concern: error handling

Levente Uzonyi-2
In reply to this post by Igor Stasenko
On Mon, 11 Oct 2010, Igor Stasenko wrote:

> On 11 October 2010 16:51, Levente Uzonyi <[hidden email]> wrote:
> On Mon, 11 Oct 2010, Igor Stasenko wrote:
>
>> Hello,
>>
>> here a situation, with which we can deal in more safer manner:
>>
>> Suppose you have a weak registry, populated by different objects and
>> their executors.
>>
>> Now, when some of them died, a weak registry performs finalization.
>>
>> The potential danger is , that if there's an error triggered by some
>> executor(s),
>> then rest of executors will have no chance to run and will be
>> forgotten, causing memory leakage.
>>
>> What you think, should we handle this more graciously?
>>
>> (Consider a following meta-code)
>>
>> WeakRegistry>>finalizeValues
>> | executors |
>>  executors := self gatherExecutorsToRun.
>>
>>  executors do: [:ex |
>>    [ ex finalize ] fork.
>>  ].
>
> The "gatherExecutorsToRun" part was implemented in the previous version of
> WeakRegistry (history is not available from the image...). #finalize wasn't
> sent from the protected block. The current version lacks it, so it will
> deadlock if a finalizer will try to access the same WeakRegistry. Other kind
> of deadlocks are also possible. For example when a finalizer can access a
> semaphore which is also used by another process that uses the same semaphore
> and also uses the WeakRegistry, but locks them in a different order
> (JNIPort).
> This only happens if the VM doesn't support the new finalization scheme
> which is the case for most current VMs.
>
> There is another issue with removal, and this affects the new finalization
> scheme too, because the finalizer of valuesDictionary is #finalizeValues.
> This means that finalization can happen in a process other than the
> finalization process. This can lead to random errors.
>
err no. valuesDictionary are held privately by weak registry. and any
access to it possible only
through instance of weak registry, which always using protect: [].


Hm. You missed the point. When someone sends #remove: to the WeakRegistry,
it will send #removeKey:ifAbsent to valuesDictionary. If valuesDictionary
has an association in the removed element's chain which key was garbage
collected, then it will be finalized. (See WeakKeyDictionary >>
#removeKey:ifAbsent:). Since the currently installed finalizer is
#finalizeValues, the finalization will be done by a process other than the
finalization process.

> So I think parts of the old WeakRegistry implementation should be restored,
> like:
> - WeakRegistry should collect executors
> - executors should be evaluated outside the protected block
>
agreed on this one.

But weak registry lost this feature before my intervention:

This is a code from Squeak 4.1. image:
finalizeValues
  "Some of our elements may have gone away. Look for those and activate
the associated executors."
  self protected: [ valueDictionary finalizeValues ]

ul 2/22/2010 14:23 ˙˙ finalization ˙˙ 2 implementors ˙˙ in no change set ˙˙

So, i thought, it not a big sacrifice, besides it runs faster.
And to my thinking, the code which needs to manipulate weak registry
during finalization smells badly.


I was talking about the code in the Squeak 4.2 code.

>>
>> in this way, if any executor's #finalize causing error, it won't
>> interfere with other executors, and they will accomplish their task
>> normally.
>> Of course, i'm not saying that we should use #fork for this, because
>> it is costly. Similar could be done w/o forking.
>> I just wanted to show a simplest code with which we could achieve a
>> more gracious error handling.
>
> Wouldn't it be better to use an exception handler?
>
Well, its a bit complicated, since what i want is to:

  - report an error
  - but continue run #finalize for rest executors

or:

  - report an error (stop finalization process), then
  - continue running for next executor , if user press 'abandon'
  - or continue with running fixed code

If i'm not mistaken, currently, if you put halt inside a finalization
process (like in some #finalize method) and then abandon it,
you won't have finalization process in system anymore, because it will
be terminated.


That's right. I more like the first version, because not all systems are
interactive. Stopping the finalization process for a long time cause a lot
of problems. Btw, WeakArray >> #finalizationProcess has a similar issue.


Levente

>
> Levente
>
>>
>> P.S. of course, in a first place it would be good to make sure that we
>> writing executors, which can't cause an error during finalization.
>> But bad things happen, and we should make sure that rest of system
>> won't be put on its knees because of some stupid bug in a single
>> #finalize.
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>
>


--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: Another finalization concern: error handling

Igor Stasenko
In reply to this post by Levente Uzonyi-2
On 11 October 2010 17:24, Levente Uzonyi <[hidden email]> wrote:

> On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote:
>
>> Levente,
>>
>> A similar discussion arose around Dolphin's event (#trigger*) mechanism.
>>  My recollection is that it was not fully addressed due to performance
>> concerns.  Forking and error handlers both have their costs.  I'm not saying
>> we should necessarily follow (we probably should not), though with careful
>> design, an interrupted chain of events might survive to be handled on a
>> subsequent attempt.
>>
>> I am far more worried about having multiple executors per object (when did
>> p=malloc();free(p);free(p);free(p) become good style?) than I am about
>> getting the finalizer process itself completely robust at this point.
>
> Smalltalk is not C. Try this:
>
> | file |
> file := StandardFileStream fileNamed: 'foo.txt'.
> file close.
> file primClose: (file instVarNamed: #fileID).
> "Those pesky plugins save us all the time. ;)"
>
>

Don't let me starting again on this.
You proposing to care about it in multiple various places, where we
could fix it in a single one.
It is like adding ifNotNil: test at each place you using setter,
instead of putting a single ifNotNil: test inside a setter itself.


try this:

coll := OrderedCollection new.
obj := Object new.
wrapper := WeakArray with: obj.
coll add: wrapper.

obj toFinalizeSend: #remove: to: coll with: wrapper.
obj toFinalizeSend: #remove: to: coll with: wrapper.
obj toFinalizeSend: #remove: to: coll with: wrapper.
obj toFinalizeSend: #remove: to: coll with: wrapper.

obj := nil.
Smalltalk garbageCollect.


The above works silently only because ObjectFinalizer simply swallows
any errors:

ObjectFinalizer>>finalize
        "Finalize the resource associated with the receiver. This message
should only be sent during the finalization process. There is NO
garantuee that the resource associated with the receiver hasn't been
free'd before so take care that you don't run into trouble - this all
may happen with interrupt priority."
        [self value] on: Error do:[:ex| ex return].

now, replace this implementation with just
self value
and you'll see what will happen.

Such behavior is completely unacceptable in terms of finding the bugs
& problems in your code.



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Another finalization concern: error handling

Schwab,Wilhelm K
In reply to this post by Igor Stasenko
Dolphin has all of its required processes rigged to restart themselves if terminated; we must follow that lead.  As far as getting error information from one thread to another, in one case I grab a callstack (just the This>>that, That>>this text with line feeds) and capture it to be included as text with an error signaled from another thread that was blocked while the offending thread was doing its thing.  It's not perfect, but it at least shows something about what the system was trying to do, and most importantly, that there was an error vs. just ignoring it.

Bill


________________________________________
From: [hidden email] [[hidden email]] On Behalf Of Igor Stasenko [[hidden email]]
Sent: Monday, October 11, 2010 10:30 AM
To: The general-purpose Squeak developers list; Pharo Development
Subject: Re: [Pharo-project] [squeak-dev] Re: Another finalization concern: error handling

>
> Wouldn't it be better to use an exception handler?
>

Well, its a bit complicated, since what i want is to:

 - report an error
 - but continue run #finalize for rest executors

or:

 - report an error (stop finalization process), then
 - continue running for next executor , if user press 'abandon'
 - or continue with running fixed code

If i'm not mistaken, currently, if you put halt inside a finalization
process (like in some #finalize method) and then abandon it,
you won't have finalization process in system anymore, because it will
be terminated.

>
> Levente
>
>>
>> P.S. of course, in a first place it would be good to make sure that we
>> writing executors, which can't cause an error during finalization.
>> But bad things happen, and we should make sure that rest of system
>> won't be put on its knees because of some stupid bug in a single
>> #finalize.
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>
>



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: Another finalization concern: error handling

Levente Uzonyi-2
In reply to this post by Schwab,Wilhelm K
On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote:

> Levente,
>
> Ok, but just because the system saves us at the last instant does not mean that we should be going out of our way to multiply free external resources.  Files are well known to the vm; other things (GSL vectors/matrices comes to mind) will not enjoy such protections.  This strikes me as a feature that got added when errors arose from attempts to add redundant executors, ultimately due to lack of thread safety.  Many things that are being fixed at great cost got started (it sure seems) because someone added a feature or placed something in or too near to Object only to avoid errors vs. finding and fixing the real problem.
>
> Maybe you have made a strong argument for multiple executors; if so, I've missed it.  Right now, it looks like a design flaw instead of a feature.

There's code that relies on multiple executors. Every object understands
#toFinalizeSend:to:with:. When you use it, you expect that your finalizer
will be evaluated when the object is garbage collected.

Btw I offered a simple solution for cases like yours. The behavior of
WeakRegistry >> #add:executor: can be easily made pluggable when an
object has an executor in the registry.


Levente

P.S.: how will you use GSL knowing that it's license is GPL? Will you
keep the code for yourself or release it under the GPL license?

>
> Bill
>
>
>
> ________________________________________
> From: [hidden email] [[hidden email]] On Behalf Of Levente Uzonyi [[hidden email]]
> Sent: Monday, October 11, 2010 10:24 AM
> To: [hidden email]
> Subject: Re: [Pharo-project] Another finalization concern: error handling
>
> On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote:
>
>> Levente,
>>
>> A similar discussion arose around Dolphin's event (#trigger*) mechanism.  My recollection is that it was not fully addressed due to performance concerns.  Forking and error handlers both have their costs.  I'm not saying we should necessarily follow (we probably should not), though with careful design, an interrupted chain of events might survive to be handled on a subsequent attempt.
>>
>> I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point.
>
> Smalltalk is not C. Try this:
>
> | file |
> file := StandardFileStream fileNamed: 'foo.txt'.
> file close.
> file primClose: (file instVarNamed: #fileID).
> "Those pesky plugins save us all the time. ;)"
>
>
> Levente
>
>>
>> Bill
>>
>>
>>
>> ________________________________________
>> From: [hidden email] [[hidden email]] On Behalf Of Levente Uzonyi [[hidden email]]
>> Sent: Monday, October 11, 2010 9:51 AM
>> To: Pharo Development
>> Cc: The general-purpose Squeak developers list
>> Subject: Re: [Pharo-project] Another finalization concern: error handling
>>
>> On Mon, 11 Oct 2010, Igor Stasenko wrote:
>>
>>> Hello,
>>>
>>> here a situation, with which we can deal in more safer manner:
>>>
>>> Suppose you have a weak registry, populated by different objects and
>>> their executors.
>>>
>>> Now, when some of them died, a weak registry performs finalization.
>>>
>>> The potential danger is , that if there's an error triggered by some
>>> executor(s),
>>> then rest of executors will have no chance to run and will be
>>> forgotten, causing memory leakage.
>>>
>>> What you think, should we handle this more graciously?
>>>
>>> (Consider a following meta-code)
>>>
>>> WeakRegistry>>finalizeValues
>>> | executors |
>>>  executors := self gatherExecutorsToRun.
>>>
>>>  executors do: [:ex |
>>>     [ ex finalize ] fork.
>>>  ].
>>
>> The "gatherExecutorsToRun" part was implemented in the previous version
>> of WeakRegistry (history is not available from the image...). #finalize
>> wasn't sent from the protected block. The current version lacks it, so it
>> will deadlock if a finalizer will try to access the same WeakRegistry.
>> Other kind of deadlocks are also possible. For example when a finalizer
>> can access a semaphore which is also used by another process that
>> uses the same semaphore and also uses the WeakRegistry, but locks them in
>> a different order (JNIPort).
>> This only happens if the VM doesn't support the new finalization scheme
>> which is the case for most current VMs.
>>
>> There is another issue with removal, and this affects the new finalization
>> scheme too, because the finalizer of valuesDictionary is #finalizeValues.
>> This means that finalization can happen in a process other than the
>> finalization process. This can lead to random errors.
>>
>> So I think parts of the old WeakRegistry implementation should be
>> restored, like:
>> - WeakRegistry should collect executors
>> - executors should be evaluated outside the protected block
>>
>>>
>>> in this way, if any executor's #finalize causing error, it won't
>>> interfere with other executors, and they will accomplish their task
>>> normally.
>>> Of course, i'm not saying that we should use #fork for this, because
>>> it is costly. Similar could be done w/o forking.
>>> I just wanted to show a simplest code with which we could achieve a
>>> more gracious error handling.
>>
>> Wouldn't it be better to use an exception handler?
>>
>>
>> Levente
>>
>>>
>>> P.S. of course, in a first place it would be good to make sure that we
>>> writing executors, which can't cause an error during finalization.
>>> But bad things happen, and we should make sure that rest of system
>>> won't be put on its knees because of some stupid bug in a single
>>> #finalize.
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko AKA sig.
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: Another finalization concern: error handling

Levente Uzonyi-2
In reply to this post by Igor Stasenko
On Mon, 11 Oct 2010, Igor Stasenko wrote:

> On 11 October 2010 17:24, Levente Uzonyi <[hidden email]> wrote:
> On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote:
>
>> Levente,
>>
>> A similar discussion arose around Dolphin's event (#trigger*) mechanism.
>>  My recollection is that it was not fully addressed due to performance
>> concerns.  Forking and error handlers both have their costs.  I'm not saying
>> we should necessarily follow (we probably should not), though with careful
>> design, an interrupted chain of events might survive to be handled on a
>> subsequent attempt.
>>
>> I am far more worried about having multiple executors per object (when did
>> p=malloc();free(p);free(p);free(p) become good style?) than I am about
>> getting the finalizer process itself completely robust at this point.
>
> Smalltalk is not C. Try this:
>
> | file |
> file := StandardFileStream fileNamed: 'foo.txt'.
> file close.
> file primClose: (file instVarNamed: #fileID).
> "Those pesky plugins save us all the time. ;)"
>
>
Don't let me starting again on this.
You proposing to care about it in multiple various places, where we
could fix it in a single one.
It is like adding ifNotNil: test at each place you using setter,
instead of putting a single ifNotNil: test inside a setter itself.


try this:

coll := OrderedCollection new.
obj := Object new.
wrapper := WeakArray with: obj.
coll add: wrapper.

obj toFinalizeSend: #remove: to: coll with: wrapper.
obj toFinalizeSend: #remove: to: coll with: wrapper.
obj toFinalizeSend: #remove: to: coll with: wrapper.
obj toFinalizeSend: #remove: to: coll with: wrapper.

obj := nil.
Smalltalk garbageCollect.


The above works silently only because ObjectFinalizer simply swallows
any errors:

ObjectFinalizer>>finalize
  "Finalize the resource associated with the receiver. This message
should only be sent during the finalization process. There is NO
garantuee that the resource associated with the receiver hasn't been
free'd before so take care that you don't run into trouble - this all
may happen with interrupt priority."
  [self value] on: Error do:[:ex| ex return].

now, replace this implementation with just
self value
and you'll see what will happen.

Such behavior is completely unacceptable in terms of finding the bugs
& problems in your code.



Interrupting the finalization process is also a bad idea, so we need a
better solution.


Levente

--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: Another finalization concern: error handling

Igor Stasenko
In reply to this post by Schwab,Wilhelm K
On 11 October 2010 17:34, Schwab,Wilhelm K <[hidden email]> wrote:
> Levente,
>
> Ok, but just because the system saves us at the last instant does not mean that we should be going out of our way to multiply free external resources.  Files are well known to the vm; other things (GSL vectors/matrices comes to mind) will not enjoy such protections.  This strikes me as a feature that got added when errors arose from attempts to add redundant executors, ultimately due to lack of thread safety.  Many things that are being fixed at great cost got started (it sure seems) because someone added a feature or placed something in or too near to Object only to avoid errors vs. finding and fixing the real problem.
>
> Maybe you have made a strong argument for multiple executors; if so, I've missed it.  Right now, it looks like a design flaw instead of a feature.
>

I second that. Knowing that VM/OS does the best to not pushish those
who attempting to free already freed resource, doesn't means that we
should be careless about it!

OS tries the best to not give away a same file handle for newly opened
file as recently closed file(s).
malloc() tries the best to not give away the pointer to same memory
region, which was recently freed.
But obviously, a well-designed application should never rely on such behavior.


--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: Another finalization concern: error handling

Schwab,Wilhelm K
In reply to this post by Igor Stasenko
Sig,

I'm not saying to leave the finalizer wide open to the wrath of errors from poorly coded objects.  I am saying that we should strive for one finalizer per object (preferably the object itself) and that to get there, we need to make the registry thread safe and have some vm support.  Failing the latter, the object can't be its own executor, but there still should be at most one executor per object.  Any errors should be raised on the attempt to create the second executor.  Arrange for that, and somebody somewhere will ask whether all of this should be thread safe.

BTW, Dolphin treats #beFinalizable/#beUnfinalizable by manipulating a mask that controls the vm behavior - haven't looked in a while, but it must be part of the object header.  An object can thus be marked as finalizable or not any number of times during its life; no objects are created thanks to the vm support.

Bill




________________________________________
From: [hidden email] [[hidden email]] On Behalf Of Igor Stasenko [[hidden email]]
Sent: Monday, October 11, 2010 10:47 AM
To: [hidden email]
Subject: Re: [Pharo-project] Another finalization concern: error handling

On 11 October 2010 17:24, Levente Uzonyi <[hidden email]> wrote:

> On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote:
>
>> Levente,
>>
>> A similar discussion arose around Dolphin's event (#trigger*) mechanism.
>>  My recollection is that it was not fully addressed due to performance
>> concerns.  Forking and error handlers both have their costs.  I'm not saying
>> we should necessarily follow (we probably should not), though with careful
>> design, an interrupted chain of events might survive to be handled on a
>> subsequent attempt.
>>
>> I am far more worried about having multiple executors per object (when did
>> p=malloc();free(p);free(p);free(p) become good style?) than I am about
>> getting the finalizer process itself completely robust at this point.
>
> Smalltalk is not C. Try this:
>
> | file |
> file := StandardFileStream fileNamed: 'foo.txt'.
> file close.
> file primClose: (file instVarNamed: #fileID).
> "Those pesky plugins save us all the time. ;)"
>
>

Don't let me starting again on this.
You proposing to care about it in multiple various places, where we
could fix it in a single one.
It is like adding ifNotNil: test at each place you using setter,
instead of putting a single ifNotNil: test inside a setter itself.


try this:

coll := OrderedCollection new.
obj := Object new.
wrapper := WeakArray with: obj.
coll add: wrapper.

obj toFinalizeSend: #remove: to: coll with: wrapper.
obj toFinalizeSend: #remove: to: coll with: wrapper.
obj toFinalizeSend: #remove: to: coll with: wrapper.
obj toFinalizeSend: #remove: to: coll with: wrapper.

obj := nil.
Smalltalk garbageCollect.


The above works silently only because ObjectFinalizer simply swallows
any errors:

ObjectFinalizer>>finalize
        "Finalize the resource associated with the receiver. This message
should only be sent during the finalization process. There is NO
garantuee that the resource associated with the receiver hasn't been
free'd before so take care that you don't run into trouble - this all
may happen with interrupt priority."
        [self value] on: Error do:[:ex| ex return].

now, replace this implementation with just
self value
and you'll see what will happen.

Such behavior is completely unacceptable in terms of finding the bugs
& problems in your code.



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: Another finalization concern: error handling

Schwab,Wilhelm K
In reply to this post by Levente Uzonyi-2
GPL is a problem.  It means that nothing I write on top of GSL gets out the door.  However, I see no reason not to give others the same mix of capability and concern in the form of an interface to it.



________________________________________
From: [hidden email] [[hidden email]] On Behalf Of Levente Uzonyi [[hidden email]]
Sent: Monday, October 11, 2010 10:59 AM
To: [hidden email]
Subject: Re: [Pharo-project] Another finalization concern: error handling

On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote:

> Levente,
>
> Ok, but just because the system saves us at the last instant does not mean that we should be going out of our way to multiply free external resources.  Files are well known to the vm; other things (GSL vectors/matrices comes to mind) will not enjoy such protections.  This strikes me as a feature that got added when errors arose from attempts to add redundant executors, ultimately due to lack of thread safety.  Many things that are being fixed at great cost got started (it sure seems) because someone added a feature or placed something in or too near to Object only to avoid errors vs. finding and fixing the real problem.
>
> Maybe you have made a strong argument for multiple executors; if so, I've missed it.  Right now, it looks like a design flaw instead of a feature.

There's code that relies on multiple executors. Every object understands
#toFinalizeSend:to:with:. When you use it, you expect that your finalizer
will be evaluated when the object is garbage collected.

Btw I offered a simple solution for cases like yours. The behavior of
WeakRegistry >> #add:executor: can be easily made pluggable when an
object has an executor in the registry.


Levente

P.S.: how will you use GSL knowing that it's license is GPL? Will you
keep the code for yourself or release it under the GPL license?

>
> Bill
>
>
>
> ________________________________________
> From: [hidden email] [[hidden email]] On Behalf Of Levente Uzonyi [[hidden email]]
> Sent: Monday, October 11, 2010 10:24 AM
> To: [hidden email]
> Subject: Re: [Pharo-project] Another finalization concern: error handling
>
> On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote:
>
>> Levente,
>>
>> A similar discussion arose around Dolphin's event (#trigger*) mechanism.  My recollection is that it was not fully addressed due to performance concerns.  Forking and error handlers both have their costs.  I'm not saying we should necessarily follow (we probably should not), though with careful design, an interrupted chain of events might survive to be handled on a subsequent attempt.
>>
>> I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point.
>
> Smalltalk is not C. Try this:
>
> | file |
> file := StandardFileStream fileNamed: 'foo.txt'.
> file close.
> file primClose: (file instVarNamed: #fileID).
> "Those pesky plugins save us all the time. ;)"
>
>
> Levente
>
>>
>> Bill
>>
>>
>>
>> ________________________________________
>> From: [hidden email] [[hidden email]] On Behalf Of Levente Uzonyi [[hidden email]]
>> Sent: Monday, October 11, 2010 9:51 AM
>> To: Pharo Development
>> Cc: The general-purpose Squeak developers list
>> Subject: Re: [Pharo-project] Another finalization concern: error handling
>>
>> On Mon, 11 Oct 2010, Igor Stasenko wrote:
>>
>>> Hello,
>>>
>>> here a situation, with which we can deal in more safer manner:
>>>
>>> Suppose you have a weak registry, populated by different objects and
>>> their executors.
>>>
>>> Now, when some of them died, a weak registry performs finalization.
>>>
>>> The potential danger is , that if there's an error triggered by some
>>> executor(s),
>>> then rest of executors will have no chance to run and will be
>>> forgotten, causing memory leakage.
>>>
>>> What you think, should we handle this more graciously?
>>>
>>> (Consider a following meta-code)
>>>
>>> WeakRegistry>>finalizeValues
>>> | executors |
>>>  executors := self gatherExecutorsToRun.
>>>
>>>  executors do: [:ex |
>>>     [ ex finalize ] fork.
>>>  ].
>>
>> The "gatherExecutorsToRun" part was implemented in the previous version
>> of WeakRegistry (history is not available from the image...). #finalize
>> wasn't sent from the protected block. The current version lacks it, so it
>> will deadlock if a finalizer will try to access the same WeakRegistry.
>> Other kind of deadlocks are also possible. For example when a finalizer
>> can access a semaphore which is also used by another process that
>> uses the same semaphore and also uses the WeakRegistry, but locks them in
>> a different order (JNIPort).
>> This only happens if the VM doesn't support the new finalization scheme
>> which is the case for most current VMs.
>>
>> There is another issue with removal, and this affects the new finalization
>> scheme too, because the finalizer of valuesDictionary is #finalizeValues.
>> This means that finalization can happen in a process other than the
>> finalization process. This can lead to random errors.
>>
>> So I think parts of the old WeakRegistry implementation should be
>> restored, like:
>> - WeakRegistry should collect executors
>> - executors should be evaluated outside the protected block
>>
>>>
>>> in this way, if any executor's #finalize causing error, it won't
>>> interfere with other executors, and they will accomplish their task
>>> normally.
>>> Of course, i'm not saying that we should use #fork for this, because
>>> it is costly. Similar could be done w/o forking.
>>> I just wanted to show a simplest code with which we could achieve a
>>> more gracious error handling.
>>
>> Wouldn't it be better to use an exception handler?
>>
>>
>> Levente
>>
>>>
>>> P.S. of course, in a first place it would be good to make sure that we
>>> writing executors, which can't cause an error during finalization.
>>> But bad things happen, and we should make sure that rest of system
>>> won't be put on its knees because of some stupid bug in a single
>>> #finalize.
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko AKA sig.
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: Another finalization concern: error handling

Stéphane Ducasse
In reply to this post by Levente Uzonyi-2
Arghhhhh

>>>
>
> ObjectFinalizer>>finalize
> "Finalize the resource associated with the receiver. This message
> should only be sent during the finalization process. There is NO
> garantuee that the resource associated with the receiver hasn't been
> free'd before so take care that you don't run into trouble - this all
> may happen with interrupt priority."
> [self value] on: Error do:[:ex| ex return].
>
> now, replace this implementation with just
> self value
> and you'll see what will happen.
>
> Such behavior is completely unacceptable in terms of finding the bugs
> & problems in your code.

yes please fix.... arghhhh

Stef

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: Another finalization concern: error handling

Henrik Sperre Johansen
In reply to this post by Schwab,Wilhelm K

On Oct 11, 2010, at 4:10 13PM, Schwab,Wilhelm K wrote:

>
> I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point.
>
> Bill

I fail to see how a blatantly obvious misuse of a feature becomes a valid argument against supporting that feature, or something you should worry alot about.
You can do Object become: nil, which is also not good style, but it's certainly not an valid argument for not allowing #become:.
Nor do I lose any sleep thinking of ways people could possibly abuse #become:

Cheers,
Henry
_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: Another finalization concern: error handling

Schwab,Wilhelm K
Henry,

Ok, what valid use of multiple executors have I missed?

Bill



________________________________________
From: [hidden email] [[hidden email]] On Behalf Of Henrik Johansen [[hidden email]]
Sent: Monday, October 11, 2010 11:23 AM
To: [hidden email]
Subject: Re: [Pharo-project] Another finalization concern: error handling

On Oct 11, 2010, at 4:10 13PM, Schwab,Wilhelm K wrote:

>
> I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point.
>
> Bill

I fail to see how a blatantly obvious misuse of a feature becomes a valid argument against supporting that feature, or something you should worry alot about.
You can do Object become: nil, which is also not good style, but it's certainly not an valid argument for not allowing #become:.
Nor do I lose any sleep thinking of ways people could possibly abuse #become:

Cheers,
Henry
_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: Another finalization concern: error handling

Igor Stasenko
In reply to this post by Henrik Sperre Johansen
On 11 October 2010 18:23, Henrik Johansen <[hidden email]> wrote:

>
> On Oct 11, 2010, at 4:10 13PM, Schwab,Wilhelm K wrote:
>
>>
>> I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point.
>>
>> Bill
>
> I fail to see how a blatantly obvious misuse of a feature becomes a valid argument against supporting that feature, or something you should worry alot about.
> You can do Object become: nil, which is also not good style, but it's certainly not an valid argument for not allowing #become:.
> Nor do I lose any sleep thinking of ways people could possibly abuse #become:
>

Henrik, in previous discussion i shown that multiple finalizers could
be effectively replaced with multiple
private weak registries.
There is one good reason for that: do not let anyone else to do
something 'good' with your private stuff.
Because 'good' quite often turns out to be 'bad', once you leaking a
reference to some critical object outside your model.

See
http://www.erights.org/index.html#SmartContracts

multiple finalizers per object is not a product of a smart contract.
Because it means that same object accessible by two different,
unrelated frameworks, which then will attempt to race for its
ownership as to 'who's gonna finalize it first', or 'who having right
to remove it from weak registry etc'.


More to that, in context of current discussion, i see that multiple
finalizers adding a lot of complexity for implementing
correct error handling.



> Cheers,
> Henry
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>



--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: Another finalization concern: error handling

Igor Stasenko
Meanwhile, i'll try to implement two test cases for WeakRegistryTest.

One, should cover following:

coll := OrderedCollection new.
obj := Object new.
wrapper := WeakArray with: obj.
coll add: wrapper.

obj toFinalizeSend: #remove: to: coll with: wrapper.
obj toFinalizeSend: #remove: to: coll with: wrapper.
obj toFinalizeSend: #remove: to: coll with: wrapper.
obj toFinalizeSend: #remove: to: coll with: wrapper.

obj := nil.
Smalltalk garbageCollect.

i.e somehow, user should be notified that there is an error during finalization.

And second test case is to make sure that if one finalizer unable to
complete due to error,
the other ones (and finalization process itself) should continue
running, skipping over errorneous finalizer.

--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
12