Handling exceptions during callback

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Handling exceptions during callback

Igor Stasenko
 
Hello,
Andreas and Eliot,

i direct my question mainly to you , because you are most experienced
in this area, but if there's others who want to share their thoughts,
please do so.

So, the question is, what to do, if an error triggered during a callback.
My concern is, that sometimes its unsafe to do return from callback
with invalid value.

For example, if callback's caller expects a pointer to something,
returning a default value (NULL)
will have the same effect as simply terminating VM :)

Here's my snippet of code, which i coded so far:

block: aBlock

        self numberOfArguments = aBlock numArgs ifFalse: [
                self error: 'Invalid number of arguments' ].
       
        context := [ "here we entering a callback"
     [
                [ | args |  args := self readArguments.
                  self leave: (aBlock valueWithArguments: args) ]
                        ifCurtailed: [ self error: 'attempt to non-local return across a callback' ]
               
            ] repeat.
        ] asContext


when callback is activated, an interpreter should enter the context ,
stored in callback's context ivar.

The best what i invented so far, is to wrap the whole thing in repeat block.
So, if there's error happens, there could be at least chance that user
(like me ;) might supply the right argument to the #leave:  method,
using debugger.
A leave: method serves to leave callback , but also might fail, if it
can't coerce a return value to corresponding C equivalent.

Of course, there's many other variants, like
self leave: ( [ aBlock valueWithArguments: args ] ifCurtailed: [ self
askUserToEnterTheValue ] )

i'd like to know, what you think would be the best variant.


--
Best regards,
Igor Stasenko AKA sig.
Reply | Threaded
Open this post in threaded view
|

Re: Handling exceptions during callback

Andreas.Raab
 
Hi Igor -

I don't think there's a general approach for the problem. After quickly
checking our internal uses of callbacks I found that usage basically
looks like this:

runCallbackProcess
        "Run the callback process"
        [true] whileTrue:[
                CallbackSemaphore wait.
                [self handleCallback] ensure:[self callbackReturn].
        ].

handleCallback
        "Handle a callback"
        | nArgs rcvr selector args result |
        nArgs := self callbackGetArgCount.
        rcvr := self callbackGetArg: 1.
        selector := self callbackGetArg: 2.
        args := (3 to: nArgs) collect:[:i| self callbackGetArg: i].
        result := [rcvr callback: selector asSymbol args: args] on: Error do:[:ex|
                ex return: nil.
        ].
        self callbackResult: result.

(this is slightly simplified from original code) The point there is that
if there's an error during the callback we simply return some predefined
value (nil in the above) and assume that the caller can deal with that
default return value.

The alternative (that I had considered but dropped as adding too much
complexity for to little value) was having an explicit error indication
along the lines of:

        result := [rcvr callback: selector asSymbol args: args] on: Error do:[:ex|
                "Signal underlying code that we failed"
                self callbackError: ex description.
                ex return: nil. "still sets a default return value"
        ].
        self callbackResult: result.

But this then assumes that the callback machinery itself has some notion
of failure which is generally not the case.

The interesting point here is that since callbacks *can* fail there must
be some way by which one can indicate to the caller that a failure has
occurred. However, that is exactly why I decided against having the
callbackError: call - when you handle the callbacks you absolutely need
to wrap them properly in an error handler and do whatever is appropriate
to return from the callback in the case of an error. And that, of
course, is specific to the callback in question and cannot be
implemented by the callback machinery in general.

And if your callback has no way to indicate failure, then you've got two
choices: 1) Return some madeup value or 2) Fail and die. There are
really no other options.

So the short answer to your question is that to implement a callback
properly you *must* specify how to respond in the case of an error. It's
not optional and in fact, you might make that part of the interface of a
callback, i.e., *always* require an error handler for the client to
indicate how to respond to an error. This could be as simple as
something like:

        Callback action:[self doSomething] ifError:[0]. "returns zero on errors"

or it could be more complex, but you might consider making this
explicitly part of the callback interface.

The other issue in this regard is how to debug failures. We punt on
this. We print a callstack to indicate error and then we return the
default value. So there's no other information than the callstack. You
probably want to look at Newspeak to find out how to debug callbacks;
there must be a few 'tricks' for how to ensure proper debug and return
semantics. We don't have any such uses in Teleplace at this point.

Cheers,
   - Andreas

On 5/3/2010 4:31 PM, Igor Stasenko wrote:

> Hello,
> Andreas and Eliot,
>
> i direct my question mainly to you , because you are most experienced
> in this area, but if there's others who want to share their thoughts,
> please do so.
>
> So, the question is, what to do, if an error triggered during a callback.
> My concern is, that sometimes its unsafe to do return from callback
> with invalid value.
>
> For example, if callback's caller expects a pointer to something,
> returning a default value (NULL)
> will have the same effect as simply terminating VM :)
>
> Here's my snippet of code, which i coded so far:
>
> block: aBlock
>
> self numberOfArguments = aBlock numArgs ifFalse: [
> self error: 'Invalid number of arguments' ].
>
> context := [ "here we entering a callback"
>      [
> [ | args |  args := self readArguments.
>  self leave: (aBlock valueWithArguments: args) ]
> ifCurtailed: [ self error: 'attempt to non-local return across a callback' ]
>
>    ] repeat.
> ] asContext
>
>
> when callback is activated, an interpreter should enter the context ,
> stored in callback's context ivar.
>
> The best what i invented so far, is to wrap the whole thing in repeat block.
> So, if there's error happens, there could be at least chance that user
> (like me ;) might supply the right argument to the #leave:  method,
> using debugger.
> A leave: method serves to leave callback , but also might fail, if it
> can't coerce a return value to corresponding C equivalent.
>
> Of course, there's many other variants, like
> self leave: ( [ aBlock valueWithArguments: args ] ifCurtailed: [ self
> askUserToEnterTheValue ] )
>
> i'd like to know, what you think would be the best variant.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Handling exceptions during callback

Igor Stasenko

On 4 May 2010 07:14, Andreas Raab <[hidden email]> wrote:

>
> Hi Igor -
>
> I don't think there's a general approach for the problem. After quickly
> checking our internal uses of callbacks I found that usage basically looks
> like this:
>
> runCallbackProcess
>        "Run the callback process"
>        [true] whileTrue:[
>                CallbackSemaphore wait.
>                [self handleCallback] ensure:[self callbackReturn].
>        ].
>
> handleCallback
>        "Handle a callback"
>        | nArgs rcvr selector args result |
>        nArgs := self callbackGetArgCount.
>        rcvr := self callbackGetArg: 1.
>        selector := self callbackGetArg: 2.
>        args := (3 to: nArgs) collect:[:i| self callbackGetArg: i].
>        result := [rcvr callback: selector asSymbol args: args] on: Error
> do:[:ex|
>                ex return: nil.
>        ].
>        self callbackResult: result.
>
> (this is slightly simplified from original code) The point there is that if
> there's an error during the callback we simply return some predefined value
> (nil in the above) and assume that the caller can deal with that default
> return value.
>
> The alternative (that I had considered but dropped as adding too much
> complexity for to little value) was having an explicit error indication
> along the lines of:
>
>        result := [rcvr callback: selector asSymbol args: args] on: Error
> do:[:ex|
>                "Signal underlying code that we failed"
>                self callbackError: ex description.
>                ex return: nil. "still sets a default return value"
>        ].
>        self callbackResult: result.
>
> But this then assumes that the callback machinery itself has some notion of
> failure which is generally not the case.
>
> The interesting point here is that since callbacks *can* fail there must be
> some way by which one can indicate to the caller that a failure has
> occurred. However, that is exactly why I decided against having the
> callbackError: call - when you handle the callbacks you absolutely need to
> wrap them properly in an error handler and do whatever is appropriate to
> return from the callback in the case of an error. And that, of course, is
> specific to the callback in question and cannot be implemented by the
> callback machinery in general.
>
> And if your callback has no way to indicate failure, then you've got two
> choices: 1) Return some madeup value or 2) Fail and die. There are really no
> other options.
>
> So the short answer to your question is that to implement a callback
> properly you *must* specify how to respond in the case of an error. It's not
> optional and in fact, you might make that part of the interface of a
> callback, i.e., *always* require an error handler for the client to indicate
> how to respond to an error. This could be as simple as something like:
>
>        Callback action:[self doSomething] ifError:[0]. "returns zero on
> errors"
>
> or it could be more complex, but you might consider making this explicitly
> part of the callback interface.
>
> The other issue in this regard is how to debug failures. We punt on this. We
> print a callstack to indicate error and then we return the default value. So
> there's no other information than the callstack. You probably want to look
> at Newspeak to find out how to debug callbacks; there must be a few 'tricks'
> for how to ensure proper debug and return semantics. We don't have any such
> uses in Teleplace at this point.
>

Thanks for detailed explanation. Indeed, there's not much what can be
done facing the requirement, that
be it error or not, you are still have to return from callback.
Since i will demand from callback users to have an unique class for
each callback type anyways,
then you can override #defaultReturnValue in a subclass and supply
value, which is safe to return.
And for extreme cases, when you want to be 100% sure, you can always
implement callback natively
and don't even bother entering an interpreter loop and passing its
arguments to a language side.
But this is a simple case, which i already having, so its not so interesting ;)

> Cheers,
>  - Andreas
>
> On 5/3/2010 4:31 PM, Igor Stasenko wrote:
>>



--
Best regards,
Igor Stasenko AKA sig.
Reply | Threaded
Open this post in threaded view
|

Re: Handling exceptions during callback

Eliot Miranda-2
In reply to this post by Igor Stasenko
 
Hi Igor,

On Mon, May 3, 2010 at 4:31 PM, Igor Stasenko <[hidden email]> wrote:
Hello,
Andreas and Eliot,

i direct my question mainly to you , because you are most experienced
in this area, but if there's others who want to share their thoughts,
please do so.

So, the question is, what to do, if an error triggered during a callback.
My concern is, that sometimes its unsafe to do return from callback
with invalid value.

At one (low) level it depends on what kind of code you're calling.  Essentially the problem is how to cut-back the C call stack.  Since the FFI machinery knows the extent of the C stack between the FFI callout and the callback it is straight-forward to provide an escape primitive that cuts back the C stack to the point of the FFI callout within which the callback occurred.  But one can't simply cut-back the stack if one is calling code that depends on unwind-protect (e.g. C++).  If you want to support C++ the FFI callout could use a try-catch around the actual callout (something you'd want to do anyway if you wanted the FFI to be able to catch exceptions and answer them as primitive failures).  To cut-back the stack the primitive  set a flag and throw a special exception that would be caught by the try-catch in the FFI callout.  The catch block would test for the flag and do a special return, e.g. return a failure value or fail the FFI callout primitive.  I like the primitive failure response because the response to FFI failures could convert the failure into a Smalltalk exception that could be engineered to continue propagating the exception that caused you to want to unwind the callback.

Another approach is to say that this is a fatal error in a deployed application (because, as you can see from the above, you need some complex machinery to deal with the situation generally) and that during development all one is able to do is inform the programmer of what has happened so they can try and avoid it.

Another approach (which IIRC is the one VisualWorks takes) is just to return some default error value to the callback.  The VW machinery puts a generic exception handler round the invocation of Smalltalk code from the callback trampoline so that it catches any uncaught exception raised within the Smalltalk side of the callback.  The exception handler arranges to answer a default value to the callback and consume the exception.

I think you'll be able to find some papers on propagating exceptions across language boundaries (which is what this is really about) and see what high-quality FFIs do here.  Finding the cost/benefit justification for implementing it is another thing altogether :)  It seems to me that a reasonable approach is as follows:

1.  no attempt is made to propagate Smalltalk exceptions to foreign code or vice verse.  e.g. if an exception occurs in Smalltalk code invoked in the context of a callback then it can propagate freely beyond the FFI callout within which the callback occurred.

2. given that exceptions are not propagated what we require is correct propagation of unwinds (invocation of finally blocks in C++) to foreign code if and when the Smalltalk stack is unwound around an FFI callout/callback invocation.

To implement this one needs two things
- to put an unwind-protect around the invocation of the Smalltalk code from a callback
- to provide a primitive that will unwind the C stack back from a callback to its corresponding FFI callout /and not/ return from the FFI callout, instead somehow returning from the primitive invocation.

The last thing isn't that hard to do because we have contexts.  The unwind-a-callback primitive can save the calling context and the try-catch in the FFI callout can arrange to return to it, not to the sender of the FFI invocation.

So when the Smalltalk stack is unwound as the last part of handling the exception within the callback we cause any finally blocks in C++ to be run as the C stack is unwound from our special callback-unwind primitive.  A stack containing multiple callout/callback invocations can be correctly unwound over multiple callout/callback invocations.  If within a callout/callback invocation there are callouts and callbacks to other language systems invisible to us then providing these also support correct unwinding we won't break anything because we're unwinding correctly.

Does this make any sense?

best
Eliot


HTH
Eliot


For example, if callback's caller expects a pointer to something,
returning a default value (NULL)
will have the same effect as simply terminating VM :)

Here's my snippet of code, which i coded so far:

block: aBlock

       self numberOfArguments = aBlock numArgs ifFalse: [
               self error: 'Invalid number of arguments' ].

       context := [    "here we entering a callback"
           [
               [ | args |  args := self readArguments.
                 self leave: (aBlock valueWithArguments: args) ]
                       ifCurtailed: [ self error: 'attempt to non-local return across a callback' ]

           ] repeat.
       ] asContext


when callback is activated, an interpreter should enter the context ,
stored in callback's context ivar.

The best what i invented so far, is to wrap the whole thing in repeat block.
So, if there's error happens, there could be at least chance that user
(like me ;) might supply the right argument to the #leave:  method,
using debugger.
A leave: method serves to leave callback , but also might fail, if it
can't coerce a return value to corresponding C equivalent.

Of course, there's many other variants, like
self leave: ( [ aBlock valueWithArguments: args ] ifCurtailed: [ self
askUserToEnterTheValue ] )

i'd like to know, what you think would be the best variant.


--
Best regards,
Igor Stasenko AKA sig.