Exceptions with Continuations
Locked
 
|
It always bugged me that SUnit has to rerun the tests again when you want to debug them. So I made a little use-case where exceptions have an optional continuation, so you can resume/debug them later on.
Here's a small example: |e| [ 0 / 0 ] on: Error do: [ :error| e := error freeze ]. e debug freeze simply creates a continuation (at the moment a full one, but that could be of course reduced to a partial one by the TestRunner). Now when you would click on the failing test result you would simply have to activate the corresponding error by sending #debug to it. This would have a major advantage for non-deterministic tests (which of course they should not be). Because in this case you're lost with the current SUnit since the debug stack will differ from the actual test-run which produced the error. Needed Changes: - Continuations (the one from Seaside will just do fine) - Exception >> #freeze saving the continuation - TestCase has to store the actual exception |
Re: Exceptions with Continuations
|
|
On 26 August 2011 14:54, Camillo Bruni <[hidden email]> wrote:
> It always bugged me that SUnit has to rerun the tests again when you want to debug them. So I made a little use-case where exceptions have an optional continuation, so you can resume/debug them later on. > > Here's a small example: > > |e| > [ 0 / 0 ] > on: Error > do: [ :error| e := error freeze ]. > e debug > > > freeze simply creates a continuation (at the moment a full one, but that could be of course reduced to a partial one by the TestRunner). Now when you would click on the failing test result you would simply have to activate the corresponding error by sending #debug to it. > > This would have a major advantage for non-deterministic tests (which of course they should not be). Because in this case you're lost with the current SUnit since the debug stack will differ from the actual test-run which produced the error. > > > Needed Changes: > - Continuations (the one from Seaside will just do fine) > - Exception >> #freeze saving the continuation > - TestCase has to store the actual exception Hi Camillo, Indeed, partial continuations would make an excellent addition to SUnit. Levente & I discussed doing do a short while back in squeak-dev. Have you had a look at http://www.squeaksource.com/Control.html by any chance? It implements the shift/reset control operator, which is almost what Seaside uses. frank |
Re: Exceptions with Continuations
|
|
On 2011-08-26, at 16:02, Frank Shearar wrote: > On 26 August 2011 14:54, Camillo Bruni <[hidden email]> wrote: >> It always bugged me that SUnit has to rerun the tests again when you want to debug them. So I made a little use-case where exceptions have an optional continuation, so you can resume/debug them later on. >> >> Here's a small example: >> >> |e| >> [ 0 / 0 ] >> on: Error >> do: [ :error| e := error freeze ]. >> e debug >> >> >> freeze simply creates a continuation (at the moment a full one, but that could be of course reduced to a partial one by the TestRunner). Now when you would click on the failing test result you would simply have to activate the corresponding error by sending #debug to it. >> >> This would have a major advantage for non-deterministic tests (which of course they should not be). Because in this case you're lost with the current SUnit since the debug stack will differ from the actual test-run which produced the error. >> >> >> Needed Changes: >> - Continuations (the one from Seaside will just do fine) >> - Exception >> #freeze saving the continuation >> - TestCase has to store the actual exception > > Hi Camillo, > > Indeed, partial continuations would make an excellent addition to > SUnit. Levente & I discussed doing do a short while back in > squeak-dev. Have you had a look at > http://www.squeaksource.com/Control.html by any chance? It implements > the shift/reset control operator, which is almost what Seaside uses. > > frank No I was unaware of that, having a look at it right now. thanks |
Re: Exceptions with Continuations
|
|
Storing a continuation per failure/error potentially consumes a lot of memory. Memory leaks are also the reason why SUnit doesn't even remeber the exception (which only refer to one or two unwound stack frames).
Furthermore continuations interfeer with the guarantees of tearDown methods in test-cases and test-resources. This opens a box full of worms: when do you run them, how often do you run them, how do you ensure a consistent state, ...? Lukas On Friday, 26 August 2011, Camillo Bruni <[hidden email]> wrote: > > On 2011-08-26, at 16:02, Frank Shearar wrote: > >> On 26 August 2011 14:54, Camillo Bruni <[hidden email]> wrote: >>> It always bugged me that SUnit has to rerun the tests again when you want to debug them. So I made a little use-case where exceptions have an optional continuation, so you can resume/debug them later on. >>> >>> Here's a small example: >>> >>> |e| >>> [ 0 / 0 ] >>> on: Error >>> do: [ :error| e := error freeze ]. >>> e debug >>> >>> >>> freeze simply creates a continuation (at the moment a full one, but that could be of course reduced to a partial one by the TestRunner). Now when you would click on the failing test result you would simply have to activate the corresponding error by sending #debug to it. >>> >>> This would have a major advantage for non-deterministic tests (which of course they should not be). Because in this case you're lost with the current SUnit since the debug stack will differ from the actual test-run which produced the error. >>> >>> >>> Needed Changes: >>> - Continuations (the one from Seaside will just do fine) >>> - Exception >> #freeze saving the continuation >>> - TestCase has to store the actual exception >> >> Hi Camillo, >> >> Indeed, partial continuations would make an excellent addition to >> SUnit. Levente & I discussed doing do a short while back in >> squeak-dev. Have you had a look at >> http://www.squeaksource.com/Control.html by any chance? It implements >> the shift/reset control operator, which is almost what Seaside uses. >> >> frank > > No I was unaware of that, having a look at it right now. > > thanks > -- Lukas Renggli www.lukas-renggli.ch |
Re: Exceptions with Continuations
|
|
On 2011-08-26, at 16:28, Lukas Renggli wrote: Storing a continuation per failure/error potentially consumes a lot of memory. Memory leaks are also the reason why SUnit doesn't even remeber the exception (which only refer to one or two unwound stack frames). point taken. Furthermore continuations interfeer with the guarantees of tearDown methods in test-cases and test-resources. This opens a box full of worms: when do you run them, how often do you run them, how do you ensure a consistent state, …? right, but they might also help debugging annoying nondeterministic edge cases. so I will implement it with an optional setting in the TestRunner. When you find an annoying test that won't run consistently you can rerun it with the continuation being recorded. Lukas |
Re: Exceptions with Continuations
|
|
Why don't you just debug the test then? If you have to enable and rerun the test anyway, you can as well tell the test to open a debugger in case of a problem. Isn't this what "MyTest debug: #testMe" does?
Lukas On Friday, 26 August 2011, Camillo Bruni <[hidden email]> wrote: > > On 2011-08-26, at 16:28, Lukas Renggli wrote: > > Storing a continuation per failure/error potentially consumes a lot of memory. Memory leaks are also the reason why SUnit doesn't even remeber the exception (which only refer to one or two unwound stack frames). > > point taken. > > Furthermore continuations interfeer with the guarantees of tearDown methods in test-cases and test-resources. This opens a box full of worms: when do you run them, how often do you run them, how do you ensure a consistent state, …? > > right, but they might also help debugging annoying nondeterministic edge cases. so I will implement it with an optional setting in the TestRunner. When you find an annoying test that won't run consistently you can rerun it with the continuation being recorded. > > Lukas > > On Friday, 26 August 2011, Camillo Bruni <[hidden email]> wrote: >> >> On 2011-08-26, at 16:02, Frank Shearar wrote: >> >>> On 26 August 2011 14:54, Camillo Bruni <[hidden email]> wrote: >>>> It always bugged me that SUnit has to rerun the tests again when you want to debug them. So I made a little use-case where exceptions have an optional continuation, so you can resume/debug them later on. >>>> >>>> Here's a small example: >>>> >>>> |e| >>>> [ 0 / 0 ] >>>> on: Error >>>> do: [ :error| e := error freeze ]. >>>> e debug >>>> >>>> >>>> freeze simply creates a continuation (at the moment a full one, but that could be of course reduced to a partial one by the TestRunner). Now when you would click on the failing test result you would simply have to activate the corresponding error by sending #debug to it. >>>> >>>> This would have a major advantage for non-deterministic tests (which of course they should not be). Because in this case you're lost with the current SUnit since the debug stack will differ from the actual test-run which produced the error. >>>> >>>> >>>> Needed Changes: >>>> - Continuations (the one from Seaside will just do fine) >>>> - Exception >> #freeze saving the continuation >>>> - TestCase has to store the actual exception >>> >>> Hi Camillo, >>> >>> Indeed, partial continuations would make an excellent addition to >>> SUnit. Levente & I discussed doing do a short while back in >>> squeak-dev. Have you had a look at >>> http://www.squeaksource.com/Control.html by any chance? It implements >>> the shift/reset control operator, which is almost what Seaside uses. >>> >>> frank >> >> No I was unaware of that, having a look at it right now. >> >> thanks >> > > -- > Lukas Renggli > www.lukas-renggli.ch <http://www.lukas-renggli.ch/> > -- Lukas Renggli www.lukas-renggli.ch |
Re: Exceptions with Continuations
|
|
well the problem with a non deterministic test is that the UI might not correspond to the opened debugger :)
another application I see here is for instance when running tests in CI and you see only a couple of tests failing, rerun them again, save the image. when you open the tests locally you can directly see where it failed on the server (e.g. very annoying with platform dependent stuff) And I personally would prefer this as default behavior, even though it might not fully work on certain tests (so I will add it as an option). Ah, and long running tests which fail, are debuggable instantaneously ;) cami On 2011-08-26, at 16:54, Lukas Renggli wrote: Why don't you just debug the test then? If you have to enable and rerun the test anyway, you can as well tell the test to open a debugger in case of a problem. Isn't this what "MyTest debug: #testMe" does? |
Re: Exceptions with Continuations
|
|
On 26 August 2011 17:19, Camillo Bruni <[hidden email]> wrote:
> well the problem with a non deterministic test is that the UI might not > correspond to the opened debugger :) I don't understand. #debug: simply runs the test without wrapping them into exception handlers, so it will just open a debugger on the spot where the problem happens. So if you test fails once in a while you should be able to reproduce it with 1000 timesRepeat: [ MyTest debug: #testMe ] and be exactly in the execution where the problem happend. > another application I see here is for instance when running tests in CI and > you see only a couple of tests failing, rerun them again, save the image. > when you open the tests locally you can directly see where it failed on the > server (e.g. very annoying with platform dependent stuff) > And I personally would prefer this as default behavior, even though it might > not fully work on certain tests (so I will add it as an option). Ah, and > long running tests which fail, are debuggable instantaneously ;) Yeah maybe, but then again you can just run the tests with #debug: and leave the debuggers open in the image, no? Lukas -- Lukas Renggli www.lukas-renggli.ch |
«
Return to Pharo Smalltalk Developers
|
1 view|%1 views
| Free forum by Nabble | Edit this page |