Another finalization concern: error handling
Locked
12
12
Another finalization concern: error handling
 
|
Hello,
here a situation, with which we can deal in more safer manner: Suppose you have a weak registry, populated by different objects and their executors. Now, when some of them died, a weak registry performs finalization. The potential danger is , that if there's an error triggered by some executor(s), then rest of executors will have no chance to run and will be forgotten, causing memory leakage. What you think, should we handle this more graciously? (Consider a following meta-code) WeakRegistry>>finalizeValues | executors | executors := self gatherExecutorsToRun. executors do: [:ex | [ ex finalize ] fork. ]. in this way, if any executor's #finalize causing error, it won't interfere with other executors, and they will accomplish their task normally. Of course, i'm not saying that we should use #fork for this, because it is costly. Similar could be done w/o forking. I just wanted to show a simplest code with which we could achieve a more gracious error handling. P.S. of course, in a first place it would be good to make sure that we writing executors, which can't cause an error during finalization. But bad things happen, and we should make sure that rest of system won't be put on its knees because of some stupid bug in a single #finalize. -- Best regards, Igor Stasenko AKA sig. |
Re: Another finalization concern: error handling
|
|
On 11 October 2010 14:40, Igor Stasenko <[hidden email]> wrote:
> Hello, > > here a situation, with which we can deal in more safer manner: > > Suppose you have a weak registry, populated by different objects and > their executors. > > Now, when some of them died, a weak registry performs finalization. > > The potential danger is , that if there's an error triggered by some > executor(s), > then rest of executors will have no chance to run and will be > forgotten, causing memory leakage. > > What you think, should we handle this more graciously? > > (Consider a following meta-code) > > WeakRegistry>>finalizeValues > | executors | > executors := self gatherExecutorsToRun. > > executors do: [:ex | > [ ex finalize ] fork. > ]. > oh, and this gets even more complicated, if we keep supporting multiple finalizers per single object. :) executors do: [:ex | ex hasMultipleExecutors ifTrue: [ ex do: [ :eex | [ eex finalize ] fork ] ] ifFalse: [ [ ex finalize ] fork ]. > in this way, if any executor's #finalize causing error, it won't > interfere with other executors, and they will accomplish their task > normally. > Of course, i'm not saying that we should use #fork for this, because > it is costly. Similar could be done w/o forking. > I just wanted to show a simplest code with which we could achieve a > more gracious error handling. > > P.S. of course, in a first place it would be good to make sure that we > writing executors, which can't cause an error during finalization. > But bad things happen, and we should make sure that rest of system > won't be put on its knees because of some stupid bug in a single > #finalize. > > -- > Best regards, > Igor Stasenko AKA sig. > -- Best regards, Igor Stasenko AKA sig. |
Re: [Pharo-project] Another finalization concern: error handling
|
|
In reply to this post by Igor Stasenko
On Mon, 11 Oct 2010, Igor Stasenko wrote:
> Hello, > > here a situation, with which we can deal in more safer manner: > > Suppose you have a weak registry, populated by different objects and > their executors. > > Now, when some of them died, a weak registry performs finalization. > > The potential danger is , that if there's an error triggered by some > executor(s), > then rest of executors will have no chance to run and will be > forgotten, causing memory leakage. > > What you think, should we handle this more graciously? > > (Consider a following meta-code) > > WeakRegistry>>finalizeValues > | executors | > executors := self gatherExecutorsToRun. > > executors do: [:ex | > [ ex finalize ] fork. > ]. The "gatherExecutorsToRun" part was implemented in the previous version of WeakRegistry (history is not available from the image...). #finalize wasn't sent from the protected block. The current version lacks it, so it will deadlock if a finalizer will try to access the same WeakRegistry. Other kind of deadlocks are also possible. For example when a finalizer can access a semaphore which is also used by another process that uses the same semaphore and also uses the WeakRegistry, but locks them in a different order (JNIPort). This only happens if the VM doesn't support the new finalization scheme which is the case for most current VMs. There is another issue with removal, and this affects the new finalization scheme too, because the finalizer of valuesDictionary is #finalizeValues. This means that finalization can happen in a process other than the finalization process. This can lead to random errors. So I think parts of the old WeakRegistry implementation should be restored, like: - WeakRegistry should collect executors - executors should be evaluated outside the protected block > > in this way, if any executor's #finalize causing error, it won't > interfere with other executors, and they will accomplish their task > normally. > Of course, i'm not saying that we should use #fork for this, because > it is costly. Similar could be done w/o forking. > I just wanted to show a simplest code with which we could achieve a > more gracious error handling. Wouldn't it be better to use an exception handler? Levente > > P.S. of course, in a first place it would be good to make sure that we > writing executors, which can't cause an error during finalization. > But bad things happen, and we should make sure that rest of system > won't be put on its knees because of some stupid bug in a single > #finalize. > > -- > Best regards, > Igor Stasenko AKA sig. > > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > |
Re: [Pharo-project] Another finalization concern: error handling
|
|
On 11 October 2010 16:51, Levente Uzonyi <[hidden email]> wrote:
> On Mon, 11 Oct 2010, Igor Stasenko wrote: > >> Hello, >> >> here a situation, with which we can deal in more safer manner: >> >> Suppose you have a weak registry, populated by different objects and >> their executors. >> >> Now, when some of them died, a weak registry performs finalization. >> >> The potential danger is , that if there's an error triggered by some >> executor(s), >> then rest of executors will have no chance to run and will be >> forgotten, causing memory leakage. >> >> What you think, should we handle this more graciously? >> >> (Consider a following meta-code) >> >> WeakRegistry>>finalizeValues >> | executors | >> executors := self gatherExecutorsToRun. >> >> executors do: [:ex | >> [ ex finalize ] fork. >> ]. > > The "gatherExecutorsToRun" part was implemented in the previous version of > WeakRegistry (history is not available from the image...). #finalize wasn't > sent from the protected block. The current version lacks it, so it will > deadlock if a finalizer will try to access the same WeakRegistry. Other kind > of deadlocks are also possible. For example when a finalizer can access a > semaphore which is also used by another process that uses the same semaphore > and also uses the WeakRegistry, but locks them in a different order > (JNIPort). > This only happens if the VM doesn't support the new finalization scheme > which is the case for most current VMs. > > There is another issue with removal, and this affects the new finalization > scheme too, because the finalizer of valuesDictionary is #finalizeValues. > This means that finalization can happen in a process other than the > finalization process. This can lead to random errors. > access to it possible only through instance of weak registry, which always using protect: []. > So I think parts of the old WeakRegistry implementation should be restored, > like: > - WeakRegistry should collect executors > - executors should be evaluated outside the protected block > agreed on this one. But weak registry lost this feature before my intervention: This is a code from Squeak 4.1. image: finalizeValues "Some of our elements may have gone away. Look for those and activate the associated executors." self protected: [ valueDictionary finalizeValues ] ul 2/22/2010 14:23 · finalization · 2 implementors · in no change set · So, i thought, it not a big sacrifice, besides it runs faster. And to my thinking, the code which needs to manipulate weak registry during finalization smells badly. >> >> in this way, if any executor's #finalize causing error, it won't >> interfere with other executors, and they will accomplish their task >> normally. >> Of course, i'm not saying that we should use #fork for this, because >> it is costly. Similar could be done w/o forking. >> I just wanted to show a simplest code with which we could achieve a >> more gracious error handling. > > Wouldn't it be better to use an exception handler? > Well, its a bit complicated, since what i want is to: - report an error - but continue run #finalize for rest executors or: - report an error (stop finalization process), then - continue running for next executor , if user press 'abandon' - or continue with running fixed code If i'm not mistaken, currently, if you put halt inside a finalization process (like in some #finalize method) and then abandon it, you won't have finalization process in system anymore, because it will be terminated. > > Levente > >> >> P.S. of course, in a first place it would be good to make sure that we >> writing executors, which can't cause an error during finalization. >> But bad things happen, and we should make sure that rest of system >> won't be put on its knees because of some stupid bug in a single >> #finalize. >> >> -- >> Best regards, >> Igor Stasenko AKA sig. >> >> _______________________________________________ >> Pharo-project mailing list >> [hidden email] >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > > -- Best regards, Igor Stasenko AKA sig. |
Re: [Pharo-project] [squeak-dev] Re: Another finalization concern: error handling
|
|
On Mon, 11 Oct 2010, Igor Stasenko wrote:
> On 11 October 2010 16:51, Levente Uzonyi <[hidden email]> wrote: > On Mon, 11 Oct 2010, Igor Stasenko wrote: > >> Hello, >> >> here a situation, with which we can deal in more safer manner: >> >> Suppose you have a weak registry, populated by different objects and >> their executors. >> >> Now, when some of them died, a weak registry performs finalization. >> >> The potential danger is , that if there's an error triggered by some >> executor(s), >> then rest of executors will have no chance to run and will be >> forgotten, causing memory leakage. >> >> What you think, should we handle this more graciously? >> >> (Consider a following meta-code) >> >> WeakRegistry>>finalizeValues >> | executors | >> executors := self gatherExecutorsToRun. >> >> executors do: [:ex | >> [ ex finalize ] fork. >> ]. > > The "gatherExecutorsToRun" part was implemented in the previous version of > WeakRegistry (history is not available from the image...). #finalize wasn't > sent from the protected block. The current version lacks it, so it will > deadlock if a finalizer will try to access the same WeakRegistry. Other kind > of deadlocks are also possible. For example when a finalizer can access a > semaphore which is also used by another process that uses the same semaphore > and also uses the WeakRegistry, but locks them in a different order > (JNIPort). > This only happens if the VM doesn't support the new finalization scheme > which is the case for most current VMs. > > There is another issue with removal, and this affects the new finalization > scheme too, because the finalizer of valuesDictionary is #finalizeValues. > This means that finalization can happen in a process other than the > finalization process. This can lead to random errors. > through instance of weak registry, which always using protect: []. Hm. You missed the point. When someone sends #remove: to the WeakRegistry, it will send #removeKey:ifAbsent to valuesDictionary. If valuesDictionary has an association in the removed element's chain which key was garbage collected, then it will be finalized. (See WeakKeyDictionary >> #removeKey:ifAbsent:). Since the currently installed finalizer is #finalizeValues, the finalization will be done by a process other than the finalization process. > So I think parts of the old WeakRegistry implementation should be restored, > like: > - WeakRegistry should collect executors > - executors should be evaluated outside the protected block > agreed on this one. But weak registry lost this feature before my intervention: This is a code from Squeak 4.1. image: finalizeValues "Some of our elements may have gone away. Look for those and activate the associated executors." self protected: [ valueDictionary finalizeValues ] ul 2/22/2010 14:23 ˙˙ finalization ˙˙ 2 implementors ˙˙ in no change set ˙˙ So, i thought, it not a big sacrifice, besides it runs faster. And to my thinking, the code which needs to manipulate weak registry during finalization smells badly. I was talking about the code in the Squeak 4.2 code. >> >> in this way, if any executor's #finalize causing error, it won't >> interfere with other executors, and they will accomplish their task >> normally. >> Of course, i'm not saying that we should use #fork for this, because >> it is costly. Similar could be done w/o forking. >> I just wanted to show a simplest code with which we could achieve a >> more gracious error handling. > > Wouldn't it be better to use an exception handler? > - report an error - but continue run #finalize for rest executors or: - report an error (stop finalization process), then - continue running for next executor , if user press 'abandon' - or continue with running fixed code If i'm not mistaken, currently, if you put halt inside a finalization process (like in some #finalize method) and then abandon it, you won't have finalization process in system anymore, because it will be terminated. That's right. I more like the first version, because not all systems are interactive. Stopping the finalization process for a long time cause a lot of problems. Btw, WeakArray >> #finalizationProcess has a similar issue. Levente > > Levente > >> >> P.S. of course, in a first place it would be good to make sure that we >> writing executors, which can't cause an error during finalization. >> But bad things happen, and we should make sure that rest of system >> won't be put on its knees because of some stupid bug in a single >> #finalize. >> >> -- >> Best regards, >> Igor Stasenko AKA sig. >> >> _______________________________________________ >> Pharo-project mailing list >> [hidden email] >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > > -- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Another finalization concern: error handling
|
|
In reply to this post by Levente Uzonyi-2
On 11 October 2010 17:24, Levente Uzonyi <[hidden email]> wrote:
> On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote: > >> Levente, >> >> A similar discussion arose around Dolphin's event (#trigger*) mechanism. >> My recollection is that it was not fully addressed due to performance >> concerns. Forking and error handlers both have their costs. I'm not saying >> we should necessarily follow (we probably should not), though with careful >> design, an interrupted chain of events might survive to be handled on a >> subsequent attempt. >> >> I am far more worried about having multiple executors per object (when did >> p=malloc();free(p);free(p);free(p) become good style?) than I am about >> getting the finalizer process itself completely robust at this point. > > Smalltalk is not C. Try this: > > | file | > file := StandardFileStream fileNamed: 'foo.txt'. > file close. > file primClose: (file instVarNamed: #fileID). > "Those pesky plugins save us all the time. ;)" > > Don't let me starting again on this. You proposing to care about it in multiple various places, where we could fix it in a single one. It is like adding ifNotNil: test at each place you using setter, instead of putting a single ifNotNil: test inside a setter itself. try this: coll := OrderedCollection new. obj := Object new. wrapper := WeakArray with: obj. coll add: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj := nil. Smalltalk garbageCollect. The above works silently only because ObjectFinalizer simply swallows any errors: ObjectFinalizer>>finalize "Finalize the resource associated with the receiver. This message should only be sent during the finalization process. There is NO garantuee that the resource associated with the receiver hasn't been free'd before so take care that you don't run into trouble - this all may happen with interrupt priority." [self value] on: Error do:[:ex| ex return]. now, replace this implementation with just self value and you'll see what will happen. Such behavior is completely unacceptable in terms of finding the bugs & problems in your code. -- Best regards, Igor Stasenko AKA sig. |
Re: Another finalization concern: error handling
|
|
On Mon, 11 Oct 2010, Igor Stasenko wrote:
> On 11 October 2010 17:24, Levente Uzonyi <[hidden email]> wrote: >> On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote: >> >>> Levente, >>> >>> A similar discussion arose around Dolphin's event (#trigger*) mechanism. >>> My recollection is that it was not fully addressed due to performance >>> concerns. Forking and error handlers both have their costs. I'm not saying >>> we should necessarily follow (we probably should not), though with careful >>> design, an interrupted chain of events might survive to be handled on a >>> subsequent attempt. >>> >>> I am far more worried about having multiple executors per object (when did >>> p=malloc();free(p);free(p);free(p) become good style?) than I am about >>> getting the finalizer process itself completely robust at this point. >> >> Smalltalk is not C. Try this: >> >> | file | >> file := StandardFileStream fileNamed: 'foo.txt'. >> file close. >> file primClose: (file instVarNamed: #fileID). >> "Those pesky plugins save us all the time. ;)" >> >> > > Don't let me starting again on this. > You proposing to care about it in multiple various places, where we > could fix it in a single one. > It is like adding ifNotNil: test at each place you using setter, > instead of putting a single ifNotNil: test inside a setter itself. > > > try this: > > coll := OrderedCollection new. > obj := Object new. > wrapper := WeakArray with: obj. > coll add: wrapper. > > obj toFinalizeSend: #remove: to: coll with: wrapper. > obj toFinalizeSend: #remove: to: coll with: wrapper. > obj toFinalizeSend: #remove: to: coll with: wrapper. > obj toFinalizeSend: #remove: to: coll with: wrapper. > > obj := nil. > Smalltalk garbageCollect. > > > The above works silently only because ObjectFinalizer simply swallows > any errors: > > ObjectFinalizer>>finalize > "Finalize the resource associated with the receiver. This message > should only be sent during the finalization process. There is NO > garantuee that the resource associated with the receiver hasn't been > free'd before so take care that you don't run into trouble - this all > may happen with interrupt priority." > [self value] on: Error do:[:ex| ex return]. > > now, replace this implementation with just > self value > and you'll see what will happen. > > Such behavior is completely unacceptable in terms of finding the bugs > & problems in your code. > Interrupting the finalization process is also a bad idea, so we need a better solution. Levente > > > -- > Best regards, > Igor Stasenko AKA sig. > > |
Re: Another finalization concern: error handling
|
|
2010/10/11 Levente Uzonyi <[hidden email]>:
> On Mon, 11 Oct 2010, Igor Stasenko wrote: > >> On 11 October 2010 17:24, Levente Uzonyi <[hidden email]> wrote: >>> >>> On Mon, 11 Oct 2010, Schwab,Wilhelm K wrote: >>> >>>> Levente, >>>> >>>> A similar discussion arose around Dolphin's event (#trigger*) mechanism. >>>> My recollection is that it was not fully addressed due to performance >>>> concerns. Forking and error handlers both have their costs. I'm not >>>> saying >>>> we should necessarily follow (we probably should not), though with >>>> careful >>>> design, an interrupted chain of events might survive to be handled on a >>>> subsequent attempt. >>>> >>>> I am far more worried about having multiple executors per object (when >>>> did >>>> p=malloc();free(p);free(p);free(p) become good style?) than I am about >>>> getting the finalizer process itself completely robust at this point. >>> >>> Smalltalk is not C. Try this: >>> >>> | file | >>> file := StandardFileStream fileNamed: 'foo.txt'. >>> file close. >>> file primClose: (file instVarNamed: #fileID). >>> "Those pesky plugins save us all the time. ;)" >>> >>> >> >> Don't let me starting again on this. >> You proposing to care about it in multiple various places, where we >> could fix it in a single one. >> It is like adding ifNotNil: test at each place you using setter, >> instead of putting a single ifNotNil: test inside a setter itself. >> >> >> try this: >> >> coll := OrderedCollection new. >> obj := Object new. >> wrapper := WeakArray with: obj. >> coll add: wrapper. >> >> obj toFinalizeSend: #remove: to: coll with: wrapper. >> obj toFinalizeSend: #remove: to: coll with: wrapper. >> obj toFinalizeSend: #remove: to: coll with: wrapper. >> obj toFinalizeSend: #remove: to: coll with: wrapper. >> >> obj := nil. >> Smalltalk garbageCollect. >> >> >> The above works silently only because ObjectFinalizer simply swallows >> any errors: >> >> ObjectFinalizer>>finalize >> "Finalize the resource associated with the receiver. This message >> should only be sent during the finalization process. There is NO >> garantuee that the resource associated with the receiver hasn't been >> free'd before so take care that you don't run into trouble - this all >> may happen with interrupt priority." >> [self value] on: Error do:[:ex| ex return]. >> >> now, replace this implementation with just >> self value >> and you'll see what will happen. >> >> Such behavior is completely unacceptable in terms of finding the bugs >> & problems in your code. >> > > Argh. Your mail client splitted this thread too. To all: my apologies. Lets continue this thread in squeak-dev only. (choice is not based on any personal/political preference ;) ) > Interrupting the finalization process is also a bad idea, so we need a > better solution. > Yes. That's why i started this thread. I'm not looking for immediate & perfect solution, but my current intent is to let everyone know that there are issues which we need to take care of. I was also trying to poll people's reaction on it. Because from first glance it may look as a minor issue. However, for production-ready & stable environments, this issue could become a critical one. > > Levente > -- Best regards, Igor Stasenko AKA sig. |
Fwd: [Pharo-project] Another finalization concern: error handling
|
|
In reply to this post by Levente Uzonyi-2
On 11 October 2010 17:34, Schwab,Wilhelm K <[hidden email]> wrote:
> Levente, > > Ok, but just because the system saves us at the last instant does not mean that we should be going out of our way to multiply free external resources. Files are well known to the vm; other things (GSL vectors/matrices comes to mind) will not enjoy such protections. This strikes me as a feature that got added when errors arose from attempts to add redundant executors, ultimately due to lack of thread safety. Many things that are being fixed at great cost got started (it sure seems) because someone added a feature or placed something in or too near to Object only to avoid errors vs. finding and fixing the real problem. > > Maybe you have made a strong argument for multiple executors; if so, I've missed it. Right now, it looks like a design flaw instead of a feature. > I second that. Knowing that VM/OS does the best to not pushish those who attempting to free already freed resource, doesn't means that we should be careless about it! OS tries the best to not give away a same file handle for newly opened file as recently closed file(s). malloc() tries the best to not give away the pointer to same memory region, which was recently freed. But obviously, a well-designed application should never rely on such behavior. -- Best regards, Igor Stasenko AKA sig. |
Re: [Pharo-project] Another finalization concern: error handling
|
|
In reply to this post by Levente Uzonyi-2
On 11 October 2010 18:23, Henrik Johansen <[hidden email]> wrote:
> > On Oct 11, 2010, at 4:10 13PM, Schwab,Wilhelm K wrote: > >> >> I am far more worried about having multiple executors per object (when did p=malloc();free(p);free(p);free(p) become good style?) than I am about getting the finalizer process itself completely robust at this point. >> >> Bill > > I fail to see how a blatantly obvious misuse of a feature becomes a valid argument against supporting that feature, or something you should worry alot about. > You can do Object become: nil, which is also not good style, but it's certainly not an valid argument for not allowing #become:. > Nor do I lose any sleep thinking of ways people could possibly abuse #become: > Henrik, in previous discussion i shown that multiple finalizers could be effectively replaced with multiple private weak registries. There is one good reason for that: do not let anyone else to do something 'good' with your private stuff. Because 'good' quite often turns out to be 'bad', once you leaking a reference to some critical object outside your model. See http://www.erights.org/index.html#SmartContracts multiple finalizers per object is not a product of a smart contract. Because it means that same object accessible by two different, unrelated frameworks, which then will attempt to race for its ownership as to 'who's gonna finalize it first', or 'who having right to remove it from weak registry etc'. More to that, in context of current discussion, i see that multiple finalizers adding a lot of complexity for implementing correct error handling. > Cheers, > Henry > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > -- Best regards, Igor Stasenko AKA sig. |
Re: [Pharo-project] Another finalization concern: error handling
|
|
Meanwhile, i'll try to implement two test cases for WeakRegistryTest.
One, should cover following: coll := OrderedCollection new. obj := Object new. wrapper := WeakArray with: obj. coll add: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj toFinalizeSend: #remove: to: coll with: wrapper. obj := nil. Smalltalk garbageCollect. i.e somehow, user should be notified that there is an error during finalization. And second test case is to make sure that if one finalizer unable to complete due to error, the other ones (and finalization process itself) should continue running, skipping over errorneous finalizer. -- Best regards, Igor Stasenko AKA sig. |
Re: [Pharo-project] Another finalization concern: error handling
|
|
On 11 October 2010 22:49, Schwab,Wilhelm K <[hidden email]> wrote:
> Sig, > > The most important words in there are "critical section." Carry on :) > Oh, please. This is not too hard to code. My mind rolling around following choice(s) (there may be others i don't see). What would be a proper way to handle error during #finalize. [ executor finalize ] on: Error do: [:ex | self handleFinalizationError: ex "where self is registry" ]. or: [ executor finalize ] on: Error do: [:ex | executor handleFinalizationError: ex ]. of course, i should catch this error in test, so i can verify that: a) test is get notified upon synthetically made error b) no matter what i do inside error handler (up to 'Processor activeProcess terminate'), a finalization process continues working (or restarts without losing remainder of executors). Also, i used #ensure: and #ifCurtailed: but i tend to forget where they are applicable and how. So, little help in this regard will be wellcome. > Bill > -- Best regards, Igor Stasenko AKA sig. |
Re: [Pharo-project] Another finalization concern: error handling
|
|
On 10/11/2010 1:17 PM, Igor Stasenko wrote:
> On 11 October 2010 22:49, Schwab,Wilhelm K<[hidden email]> wrote: >> Sig, >> >> The most important words in there are "critical section." Carry on :) >> > > Oh, please. This is not too hard to code. > > My mind rolling around following choice(s) (there may be others i don't see). > What would be a proper way to handle error during #finalize. > > [ executor finalize ] on: Error do: [:ex | > self handleFinalizationError: ex "where self is registry" > ]. > > or: > > [ executor finalize ] on: Error do: [:ex | > executor handleFinalizationError: ex > ]. Make it a parameter (block) so that clients can tell the registry how they'd like to handle errors. I.e., something like: [executor finalize] on: Error do:[:ex| errorHandler ifNotNil:[errorHandler value: ex value: executor]. ]. Cheers, - Andreas > of course, i should catch this error in test, so i can verify that: > > a) test is get notified upon synthetically made error > b) no matter what i do inside error handler (up to 'Processor > activeProcess terminate'), a finalization process continues working > (or restarts without losing remainder of executors). > > > Also, i used #ensure: and #ifCurtailed: but i tend to forget where > they are applicable and how. > So, little help in this regard will be wellcome. > >> Bill >> |
Re: [Pharo-project] Another finalization concern: error handling
|
|
On 11 October 2010 23:33, Andreas Raab <[hidden email]> wrote:
> On 10/11/2010 1:17 PM, Igor Stasenko wrote: >> >> On 11 October 2010 22:49, Schwab,Wilhelm K<[hidden email]> wrote: >>> >>> Sig, >>> >>> The most important words in there are "critical section." Carry on :) >>> >> >> Oh, please. This is not too hard to code. >> >> My mind rolling around following choice(s) (there may be others i don't >> see). >> What would be a proper way to handle error during #finalize. >> >> [ executor finalize ] on: Error do: [:ex | >> self handleFinalizationError: ex "where self is registry" >> ]. >> >> or: >> >> [ executor finalize ] on: Error do: [:ex | >> executor handleFinalizationError: ex >> ]. > > Make it a parameter (block) so that clients can tell the registry how they'd > like to handle errors. I.e., something like: > > [executor finalize] on: Error do:[:ex| > errorHandler ifNotNil:[errorHandler value: ex value: executor]. right? > ]. > > Cheers, > - Andreas > >> of course, i should catch this error in test, so i can verify that: >> >> a) test is get notified upon synthetically made error >> b) no matter what i do inside error handler (up to 'Processor >> activeProcess terminate'), a finalization process continues working >> (or restarts without losing remainder of executors). >> >> >> Also, i used #ensure: and #ifCurtailed: but i tend to forget where >> they are applicable and how. >> So, little help in this regard will be wellcome. >> >>> Bill >>> > > > -- Best regards, Igor Stasenko AKA sig. |
Re: [Pharo-project] Another finalization concern: error handling
|
|
In reply to this post by Igor Stasenko
Igor,
take a look at the VisualWorks finalization code. There a process is spawned to run each finalizer, and there is a throttling mechanism to ensure that the system does not create a flood of processes when lots of objects are to be finalized. Instead, a new finalization process is created only when the previous one starts to run. A new finalizer process is created when the previous one starts because, due to errors, the previous one may never finish, but it will always start.
outerFinalizationLoop "Forks a process that will send every finalizable object (weak indexable class or Ephemeron) on the finalization queue the
mourn message. Separate processes are used to fetch finalizable objects and to finalize them so that if errors occur
in finalization they don't stop the finalization mechanism altogether. This method uses the ratchet semaphore to arrange
that each new processes is only forked after the previous process has made progess. Hence this avoids creating many
new processes that can't run because they have lower priority than the finalization process, and in the process possibly
running out of memory." | theBereaved ratchet | ratchet := Semaphore new.
[FinalizationSemaphore wait. theBereaved := [self fetchFromFinalizationQueue]
on: self queueOverflowSignal do: [:ex | ex return: #overflow].
theBereaved == nil ifFalse: [[ratchet signal. self innerFinalizationLoopWith: theBereaved]
forkAt: Processor lowIOPriority. ratchet wait]] repeat innerFinalizationLoopWith: initialValue "Sends every finalizable object (weak indexable class or Ephemeron) on the finalization queue the mourn message."
initialValue == #overflow ifTrue: [^self handleFinalizationQueueOverflow].
[| theBereaved | theBereaved := initialValue. [theBereaved mourn.
(theBereaved := self fetchFromFinalizationQueue) ~~ nil] whileTrue] on: self queueOverflowSignal
do: [:ex | self handleFinalizationQueueOverflow. ex return]
HTH Eliot On Mon, Oct 11, 2010 at 1:17 PM, Igor Stasenko <[hidden email]> wrote:
|
Re: [Pharo-project] Another finalization concern: error handling
|
|
On 12 October 2010 00:02, Eliot Miranda <[hidden email]> wrote:
> Igor, > take a look at the VisualWorks finalization code. There a process is > spawned to run each finalizer, and there is a throttling mechanism to ensure > that the system does not create a flood of processes when lots of objects > are to be finalized. Instead, a new finalization process is created only > when the previous one starts to run. A new finalizer process is created > when the previous one starts because, due to errors, the previous one may > never finish, but it will always start. This is a little bit confusing. Did you mean: new finalization process is created only when the previous one _stops_ to run and A new finalizer process is created when the previous one _stops_ because, due to errors ? > outerFinalizationLoop > "Forks a process that will send every finalizable object (weak > indexable class or Ephemeron) on the finalization queue the > mourn message. Separate processes are used to fetch > finalizable objects and to finalize them so that if errors occur > in finalization they don't stop the finalization mechanism > altogether. This method uses the ratchet semaphore to arrange > that each new processes is only forked after the previous > process has made progess. Hence this avoids creating many > new processes that can't run because they have lower priority > than the finalization process, and in the process possibly > running out of memory." > | theBereaved ratchet | > ratchet := Semaphore new. > [FinalizationSemaphore wait. > theBereaved := [self fetchFromFinalizationQueue] > on: self queueOverflowSignal > do: [:ex | ex return: #overflow]. > theBereaved == nil ifFalse: > [[ratchet signal. > self innerFinalizationLoopWith: theBereaved] > forkAt: Processor lowIOPriority. > ratchet wait]] repeat > innerFinalizationLoopWith: initialValue > "Sends every finalizable object (weak indexable class or Ephemeron) on the > finalization queue the mourn message." > initialValue == #overflow ifTrue: > [^self handleFinalizationQueueOverflow]. > [| theBereaved | > theBereaved := initialValue. > [theBereaved mourn. > (theBereaved := self fetchFromFinalizationQueue) ~~ nil] whileTrue] > on: self queueOverflowSignal > do: [:ex | > self handleFinalizationQueueOverflow. > ex return] > > (there are details like the use of handleFinalizationQueueOverflow that you > can ignore) > HTH > Eliot > On Mon, Oct 11, 2010 at 1:17 PM, Igor Stasenko <[hidden email]> wrote: >> >> On 11 October 2010 22:49, Schwab,Wilhelm K <[hidden email]> wrote: >> > Sig, >> > >> > The most important words in there are "critical section." Carry on :) >> > >> >> Oh, please. This is not too hard to code. >> >> My mind rolling around following choice(s) (there may be others i don't >> see). >> What would be a proper way to handle error during #finalize. >> >> [ executor finalize ] on: Error do: [:ex | >> self handleFinalizationError: ex "where self is registry" >> ]. >> >> or: >> >> [ executor finalize ] on: Error do: [:ex | >> executor handleFinalizationError: ex >> ]. >> >> >> of course, i should catch this error in test, so i can verify that: >> >> a) test is get notified upon synthetically made error >> b) no matter what i do inside error handler (up to 'Processor >> activeProcess terminate'), a finalization process continues working >> (or restarts without losing remainder of executors). >> >> >> Also, i used #ensure: and #ifCurtailed: but i tend to forget where >> they are applicable and how. >> So, little help in this regard will be wellcome. >> >> > Bill >> > >> -- >> Best regards, >> Igor Stasenko AKA sig. >> > > > > > -- Best regards, Igor Stasenko AKA sig. |
Re: [Pharo-project] Another finalization concern: error handling
|
|
In reply to this post by Eliot Miranda-2
On 12 October 2010 00:02, Eliot Miranda <[hidden email]> wrote:
> Igor, [ snip ] > outerFinalizationLoop [ snip ] > (there are details like the use of handleFinalizationQueueOverflow that you > can ignore) > HTH > Eliot Okay, i got the idea. Each finalizer (or executor) is first pushed to shared queue. Then there is a separate process which fetching them one by one from queue and sends #finalize. If that process dies (due to some error), then queue starts growing up, up to the point that outer process (responsible for pushing finalizers to queue), starts another process to feed from it. Right? -- Best regards, Igor Stasenko AKA sig. |
Re: [Pharo-project] Another finalization concern: error handling
|
|
Sorry for blowing up this thread. :)
The VW implementation actually is very close to my draft idea, to fork new process for each #finalize, so it won't interfere with others, and on error, it won't stop stones rolling: executors do: [:ex | [ ex finalize ] fork ]. If there's no objections, we could arm this idea (use forked process, close to what VW does). Or should we do the hard way and avoid forking? P.S. btw, are we guilty already for violating copyright law here by looking at VW code? :) -- Best regards, Igor Stasenko AKA sig. |
Re: [Pharo-project] Another finalization concern: error handling
|
|
In reply to this post by Igor Stasenko
On Mon, Oct 11, 2010 at 3:07 PM, Igor Stasenko <[hidden email]> wrote:
No. The top-level loop spawns processes at a lower priority to run finalization; I'll call these sub-processes. If one creates a sub-process as soon as an object is available to finalize then the system can end up creating many many sub-processes, none of which can run since the top-level process is busy creating sub-processes. If one creates a sub-process only when a previous sub-process terminates then the system is no different to now; if an error occurs in a sub-process taht prevents it teminating then no more sub-processes are created and finalization stops.
If instead one creates a new sub-process when a previous sub-process starts executing (starts executing is not the same point in time as create the process) then there are only ever two subprocesses, the one already running, and the one that was created when the first started running, but has not started running since cooperative scheduling means it is queued behind the previous sub-proecss. However, as soon as an error occurs in the previous sub-process, the one that was created when the sub-process with the error started will start running and create another one.
Read the VisualWorks source code and it will, I hope, become clear.
No. See above. When the sub-process *starts*. best Eliot
|
Re: [Pharo-project] Another finalization concern: error handling
|
|
In reply to this post by Igor Stasenko
On Mon, Oct 11, 2010 at 3:30 PM, Igor Stasenko <[hidden email]> wrote: Sorry for blowing up this thread. :) No. They haven't patented the algorithm. The code is freely accessible (its from vw non-commercial). Just don't copy the code.
best, Eliot
|
«
Return to Squeak - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |