Expected Failures Survey

I would like to know if anyone actually uses the #expectedFailure
mechanism in SUnit as is?

TestResult appears to me to be needlessly complicated by the whole
concept of expected this and unexpected thats.

If there is an alternative way of categorising and running expected
failures, would anyone object if TestResult were simplified somewhat?

many thanks in  advance

Keith

       
       
               
___________________________________________________________
All new Yahoo! Mail "The new Interface is stunning in its simplicity and ease of use." - PC Magazine
http://uk.docs.yahoo.com/nowyoucan.html

2007/2/22, Keith Hodges <[hidden email]>:
> I would like to know if anyone actually uses the #expectedFailure
> mechanism in SUnit as is?

I use it a bit. It's usefull for writing tests for the squeak classes
while not able to correct bugs.

--
Damien Cassou

Damien Cassou wrote:
> 2007/2/22, Keith Hodges <[hidden email]>:
>> I would like to know if anyone actually uses the #expectedFailure
>> mechanism in SUnit as is?
>
> I use it a bit. It's usefull for writing tests for the squeak classes
> while not able to correct bugs.
>
So if there was an alternative (easier) means of marking those tests
than implementing #expectedFailures,  you wouldnt mind if
#expectedFailures was deprecated?

Keith

               
___________________________________________________________
Now you can scan emails quickly with a reading pane. Get the new Yahoo! Mail. http://uk.docs.yahoo.com/nowyoucan.html

Keith Hodges wrote:
> I would like to know if anyone actually uses the #expectedFailure
> mechanism in SUnit as is?

We have used it in Croquet to point out known differences between platforms.

> TestResult appears to me to be needlessly complicated by the whole
> concept of expected this and unexpected thats.

As a user (and not an implementor) the complexity doesn't disturb me one
bit. If it can be tidied up it's certainly great but if that means
breaking the test framework I'm not to fond of the idea.

> If there is an alternative way of categorising and running expected
> failures, would anyone object if TestResult were simplified somewhat?

If the only benefit is a simpler implementation I fail to see the value
of the change for the people whose code this breaks. How about making it
simpler and keeping it compatible instead?

Cheers,
   - Andreas

Andreas Raab wrote:
> Keith Hodges wrote:
>> I would like to know if anyone actually uses the #expectedFailure
>> mechanism in SUnit as is?
>
> We have used it in Croquet to point out known differences between
> platforms.
>
That makes perfect sense, I have attempted to craft a fairly
comprehensive alternative solution for marking tests for various cases
like this, in a class called TestCaseVersioned.

This enables you to categorise a test as of a specific platform, a
specific release, a specific vm version, or even a specific release or
later, or earlier.

TestRunner presents the option to filter out or select tests that are
not expected to work for the current release, current vm version and
current platform.
I am proposing to tidy TestResult to contain normal, failure, errors and
passes, but would then have to come up with a scheme for mapping
TestCase-#expectedFailures into the other scheme. Its doable I guess.

Keith

               
___________________________________________________________
Now you can scan emails quickly with a reading pane. Get the new Yahoo! Mail. http://uk.docs.yahoo.com/nowyoucan.html

2007/2/22, Keith Hodges <[hidden email]>:
I don't mind :-)


--
Damien Cassou

Il giorno gio, 22/02/2007 alle 21.47 -0800, Andreas Raab ha scritto:
> Keith Hodges wrote:
> > I would like to know if anyone actually uses the #expectedFailure
> > mechanism in SUnit as is?
>
> We have used it in Croquet to point out known differences between platforms.

Andreas,

can you provide an example? There's something about the whole concept of
expected failures and errors that leaves me perplexed.

        Giovanni

Giovanni Corriga wrote:
Sure. In CroquetVMTests we have a number of tests to make sure we have
all the required plugins. I would like to count joystick and midi plugin
to those but the Unix VM doesn't have them. Therefore:

CroquetVMTests>>expectedFailures
   Smalltalk platformName = 'unix'
     ifTrue:[^#(testJoystickTabletPlugin testMIDIPlugin)].
   ^#()

Cheers,
   - Andreas

Il giorno ven, 23/02/2007 alle 10.15 -0800, Andreas Raab ha scritto:
Aaah, I see. Personally, I'd have overloaded #suite instead of
implementing an expected failures system. But since it's already therem
using it does no harm.

        Giovanni