Expected Failures Survey

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Expected Failures Survey

keith1y
I would like to know if anyone actually uses the #expectedFailure
mechanism in SUnit as is?

TestResult appears to me to be needlessly complicated by the whole
concept of expected this and unexpected thats.

If there is an alternative way of categorising and running expected
failures, would anyone object if TestResult were simplified somewhat?

many thanks in  advance

Keith

       
       
               
___________________________________________________________
All new Yahoo! Mail "The new Interface is stunning in its simplicity and ease of use." - PC Magazine
http://uk.docs.yahoo.com/nowyoucan.html

Reply | Threaded
Open this post in threaded view
|

Re: Expected Failures Survey

Damien Cassou
2007/2/22, Keith Hodges <[hidden email]>:
> I would like to know if anyone actually uses the #expectedFailure
> mechanism in SUnit as is?

I use it a bit. It's usefull for writing tests for the squeak classes
while not able to correct bugs.

--
Damien Cassou

Reply | Threaded
Open this post in threaded view
|

Re: Expected Failures Survey

keith1y
Damien Cassou wrote:
> 2007/2/22, Keith Hodges <[hidden email]>:
>> I would like to know if anyone actually uses the #expectedFailure
>> mechanism in SUnit as is?
>
> I use it a bit. It's usefull for writing tests for the squeak classes
> while not able to correct bugs.
>
So if there was an alternative (easier) means of marking those tests
than implementing #expectedFailures,  you wouldnt mind if
#expectedFailures was deprecated?

Keith

               
___________________________________________________________
Now you can scan emails quickly with a reading pane. Get the new Yahoo! Mail. http://uk.docs.yahoo.com/nowyoucan.html

Reply | Threaded
Open this post in threaded view
|

Re: Expected Failures Survey

Andreas.Raab
In reply to this post by keith1y
Keith Hodges wrote:
> I would like to know if anyone actually uses the #expectedFailure
> mechanism in SUnit as is?

We have used it in Croquet to point out known differences between platforms.

> TestResult appears to me to be needlessly complicated by the whole
> concept of expected this and unexpected thats.

As a user (and not an implementor) the complexity doesn't disturb me one
bit. If it can be tidied up it's certainly great but if that means
breaking the test framework I'm not to fond of the idea.

> If there is an alternative way of categorising and running expected
> failures, would anyone object if TestResult were simplified somewhat?

If the only benefit is a simpler implementation I fail to see the value
of the change for the people whose code this breaks. How about making it
simpler and keeping it compatible instead?

Cheers,
   - Andreas

Reply | Threaded
Open this post in threaded view
|

Re: Expected Failures Survey

keith1y
Andreas Raab wrote:
> Keith Hodges wrote:
>> I would like to know if anyone actually uses the #expectedFailure
>> mechanism in SUnit as is?
>
> We have used it in Croquet to point out known differences between
> platforms.
>
That makes perfect sense, I have attempted to craft a fairly
comprehensive alternative solution for marking tests for various cases
like this, in a class called TestCaseVersioned.

This enables you to categorise a test as of a specific platform, a
specific release, a specific vm version, or even a specific release or
later, or earlier.

TestRunner presents the option to filter out or select tests that are
not expected to work for the current release, current vm version and
current platform.

>> TestResult appears to me to be needlessly complicated by the whole
>> concept of expected this and unexpected thats.
>
> As a user (and not an implementor) the complexity doesn't disturb me
> one bit. If it can be tidied up it's certainly great but if that means
> breaking the test framework I'm not to fond of the idea.
>
>> If there is an alternative way of categorising and running expected
>> failures, would anyone object if TestResult were simplified somewhat?
>
> If the only benefit is a simpler implementation I fail to see the
> value of the change for the people whose code this breaks. How about
> making it simpler and keeping it compatible instead?
>
> Cheers,
>   - Andreas
I am proposing to tidy TestResult to contain normal, failure, errors and
passes, but would then have to come up with a scheme for mapping
TestCase-#expectedFailures into the other scheme. Its doable I guess.

Keith

               
___________________________________________________________
Now you can scan emails quickly with a reading pane. Get the new Yahoo! Mail. http://uk.docs.yahoo.com/nowyoucan.html

Reply | Threaded
Open this post in threaded view
|

Re: Expected Failures Survey

Damien Cassou
In reply to this post by keith1y
2007/2/22, Keith Hodges <[hidden email]>:

> Damien Cassou wrote:
> > 2007/2/22, Keith Hodges <[hidden email]>:
> >> I would like to know if anyone actually uses the #expectedFailure
> >> mechanism in SUnit as is?
> >
> > I use it a bit. It's usefull for writing tests for the squeak classes
> > while not able to correct bugs.
> >
> So if there was an alternative (easier) means of marking those tests
> than implementing #expectedFailures,  you wouldnt mind if
> #expectedFailures was deprecated?

I don't mind :-)


--
Damien Cassou

Reply | Threaded
Open this post in threaded view
|

Re: Expected Failures Survey

Giovanni Corriga
In reply to this post by Andreas.Raab
Il giorno gio, 22/02/2007 alle 21.47 -0800, Andreas Raab ha scritto:
> Keith Hodges wrote:
> > I would like to know if anyone actually uses the #expectedFailure
> > mechanism in SUnit as is?
>
> We have used it in Croquet to point out known differences between platforms.

Andreas,

can you provide an example? There's something about the whole concept of
expected failures and errors that leaves me perplexed.

        Giovanni


Reply | Threaded
Open this post in threaded view
|

Re: Expected Failures Survey

Andreas.Raab
Giovanni Corriga wrote:

> Il giorno gio, 22/02/2007 alle 21.47 -0800, Andreas Raab ha scritto:
>> Keith Hodges wrote:
>>> I would like to know if anyone actually uses the #expectedFailure
>>> mechanism in SUnit as is?
>> We have used it in Croquet to point out known differences between platforms.
>
> Andreas,
>
> can you provide an example? There's something about the whole concept of
> expected failures and errors that leaves me perplexed.

Sure. In CroquetVMTests we have a number of tests to make sure we have
all the required plugins. I would like to count joystick and midi plugin
to those but the Unix VM doesn't have them. Therefore:

CroquetVMTests>>expectedFailures
   Smalltalk platformName = 'unix'
     ifTrue:[^#(testJoystickTabletPlugin testMIDIPlugin)].
   ^#()

Cheers,
   - Andreas

Reply | Threaded
Open this post in threaded view
|

Re: Expected Failures Survey

Giovanni Corriga
Il giorno ven, 23/02/2007 alle 10.15 -0800, Andreas Raab ha scritto:

> Giovanni Corriga wrote:
> > Il giorno gio, 22/02/2007 alle 21.47 -0800, Andreas Raab ha scritto:
> >> Keith Hodges wrote:
> >>> I would like to know if anyone actually uses the #expectedFailure
> >>> mechanism in SUnit as is?
> >> We have used it in Croquet to point out known differences between platforms.
> >
> > Andreas,
> >
> > can you provide an example? There's something about the whole concept of
> > expected failures and errors that leaves me perplexed.
>
> Sure. In CroquetVMTests we have a number of tests to make sure we have
> all the required plugins. I would like to count joystick and midi plugin
> to those but the Unix VM doesn't have them. Therefore:
>
> CroquetVMTests>>expectedFailures
>    Smalltalk platformName = 'unix'
>      ifTrue:[^#(testJoystickTabletPlugin testMIDIPlugin)].
>    ^#()

Aaah, I see. Personally, I'd have overloaded #suite instead of
implementing an expected failures system. But since it's already therem
using it does no harm.

        Giovanni