ExpectedFailures in SUnit

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

ExpectedFailures in SUnit

niko.schwarz
Hi,

Currently, to mark a test as an expected failure, `expectedFailures`
should be overwritten
(http://stackoverflow.com/questions/3976596/how-to-mark-expected-failures-in-sunit).
That's a bit tricky to figure out, so I added a comment to the method:
SUnit-NikoSchwarz.114. However, I think it's still a usability bug:
marking something as an expectedFailure isn't visible inside the test.

I think expectedFailures should look through the pragmas of its tests
and return an array of all method names that contain the pragma
<expectedFailure>.

Cheers,

Niko

--
http://scg.unibe.ch/staff/Schwarz
twitter.com/nes1983
Tel: +41 076 235 8683

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: ExpectedFailures in SUnit

Mariano Martinez Peck


On Wed, Oct 20, 2010 at 12:15 PM, Niko Schwarz <[hidden email]> wrote:
Hi,

Currently, to mark a test as an expected failure, `expectedFailures`
should be overwritten
(http://stackoverflow.com/questions/3976596/how-to-mark-expected-failures-in-sunit).
That's a bit tricky to figure out, so I added a comment to the method:
SUnit-NikoSchwarz.114. However, I think it's still a usability bug:
marking something as an expectedFailure isn't visible inside the test.

I think expectedFailures should look through the pragmas of its tests
and return an array of all method names that contain the pragma
<expectedFailure>.

Yes, that's better. But in addition, sometimes we need conditions. For example:

Smalltalk listPlugins contains: 'blahhh' ifFalse[ expectedFailures add: 'testBlah' ]

or whatever condition. Can you do that with pragmas?

the other problem is that we don't have expectedErrors as well. This is bad and confusing.

cheers

mariano

 

 

Cheers,

Niko

--
http://scg.unibe.ch/staff/Schwarz
twitter.com/nes1983
Tel: +41 076 235 8683

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: ExpectedFailures in SUnit

Mariano Martinez Peck
In reply to this post by niko.schwarz


On Wed, Oct 20, 2010 at 12:15 PM, Niko Schwarz <[hidden email]> wrote:
Hi,

Currently, to mark a test as an expected failure, `expectedFailures`
should be overwritten
(http://stackoverflow.com/questions/3976596/how-to-mark-expected-failures-in-sunit).
That's a bit tricky to figure out, so I added a comment to the method:
SUnit-NikoSchwarz.114.

Did you open a ticket?  otherwise will be forgoten and nobody will integrate it.

Cheers

Mariano
 
However, I think it's still a usability bug:
marking something as an expectedFailure isn't visible inside the test.

I think expectedFailures should look through the pragmas of its tests
and return an array of all method names that contain the pragma
<expectedFailure>.

Cheers,

Niko

--
http://scg.unibe.ch/staff/Schwarz
twitter.com/nes1983
Tel: +41 076 235 8683

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: ExpectedFailures in SUnit

Stéphane Ducasse
In reply to this post by niko.schwarz
+1
the problem is that we should sync with nial for SUNit but since brest I do not know what really make progress.
I still want to know the status of a test. what is red or not when it was packaged and shipped or is is my system the problem

Stef
On Oct 20, 2010, at 1:41 PM, Mariano Martinez Peck wrote:

>
>
> On Wed, Oct 20, 2010 at 12:15 PM, Niko Schwarz <[hidden email]> wrote:
> Hi,
>
> Currently, to mark a test as an expected failure, `expectedFailures`
> should be overwritten
> (http://stackoverflow.com/questions/3976596/how-to-mark-expected-failures-in-sunit).
> That's a bit tricky to figure out, so I added a comment to the method:
> SUnit-NikoSchwarz.114. However, I think it's still a usability bug:
> marking something as an expectedFailure isn't visible inside the test.
>
> I think expectedFailures should look through the pragmas of its tests
> and return an array of all method names that contain the pragma
> <expectedFailure>.
>
> Yes, that's better. But in addition, sometimes we need conditions. For example:
>
> Smalltalk listPlugins contains: 'blahhh' ifFalse[ expectedFailures add: 'testBlah' ]
>
> or whatever condition. Can you do that with pragmas?
>
> the other problem is that we don't have expectedErrors as well. This is bad and confusing.
>
> cheers
>
> mariano
>
>  
>
>  
>
> Cheers,
>
> Niko
>
> --
> http://scg.unibe.ch/staff/Schwarz
> twitter.com/nes1983
> Tel: +41 076 235 8683
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: ExpectedFailures in SUnit

niko.schwarz
I created an issue and submitted a fix:
http://code.google.com/p/pharo/issues/detail?id=3143

Niko

On Wed, Oct 20, 2010 at 3:48 PM, Stéphane Ducasse
<[hidden email]> wrote:

> +1
> the problem is that we should sync with nial for SUNit but since brest I do not know what really make progress.
> I still want to know the status of a test. what is red or not when it was packaged and shipped or is is my system the problem
>
> Stef
> On Oct 20, 2010, at 1:41 PM, Mariano Martinez Peck wrote:
>
>>
>>
>> On Wed, Oct 20, 2010 at 12:15 PM, Niko Schwarz <[hidden email]> wrote:
>> Hi,
>>
>> Currently, to mark a test as an expected failure, `expectedFailures`
>> should be overwritten
>> (http://stackoverflow.com/questions/3976596/how-to-mark-expected-failures-in-sunit).
>> That's a bit tricky to figure out, so I added a comment to the method:
>> SUnit-NikoSchwarz.114. However, I think it's still a usability bug:
>> marking something as an expectedFailure isn't visible inside the test.
>>
>> I think expectedFailures should look through the pragmas of its tests
>> and return an array of all method names that contain the pragma
>> <expectedFailure>.
>>
>> Yes, that's better. But in addition, sometimes we need conditions. For example:
>>
>> Smalltalk listPlugins contains: 'blahhh' ifFalse[ expectedFailures add: 'testBlah' ]
>>
>> or whatever condition. Can you do that with pragmas?
>>
>> the other problem is that we don't have expectedErrors as well. This is bad and confusing.
>>
>> cheers
>>
>> mariano
>>
>>
>>
>>
>>
>> Cheers,
>>
>> Niko
>>
>> --
>> http://scg.unibe.ch/staff/Schwarz
>> twitter.com/nes1983
>> Tel: +41 076 235 8683
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>



--
http://scg.unibe.ch/staff/Schwarz
twitter.com/nes1983
Tel: +41 076 235 8683

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Reply | Threaded
Open this post in threaded view
|

Re: ExpectedFailures in SUnit

Stéphane Ducasse
In reply to this post by Stéphane Ducasse
Tx niko


Stef

On Oct 23, 2010, at 3:52 PM, Niko Schwarz wrote:

> I created an issue and submitted a fix:
> http://code.google.com/p/pharo/issues/detail?id=3143
>
> Niko
>
> On Wed, Oct 20, 2010 at 3:48 PM, Stéphane Ducasse
> <[hidden email]> wrote:
>> +1
>> the problem is that we should sync with nial for SUNit but since brest I do not know what really make progress.
>> I still want to know the status of a test. what is red or not when it was packaged and shipped or is is my system the problem
>>
>> Stef
>> On Oct 20, 2010, at 1:41 PM, Mariano Martinez Peck wrote:
>>
>>>
>>>
>>> On Wed, Oct 20, 2010 at 12:15 PM, Niko Schwarz <[hidden email]> wrote:
>>> Hi,
>>>
>>> Currently, to mark a test as an expected failure, `expectedFailures`
>>> should be overwritten
>>> (http://stackoverflow.com/questions/3976596/how-to-mark-expected-failures-in-sunit).
>>> That's a bit tricky to figure out, so I added a comment to the method:
>>> SUnit-NikoSchwarz.114. However, I think it's still a usability bug:
>>> marking something as an expectedFailure isn't visible inside the test.
>>>
>>> I think expectedFailures should look through the pragmas of its tests
>>> and return an array of all method names that contain the pragma
>>> <expectedFailure>.
>>>
>>> Yes, that's better. But in addition, sometimes we need conditions. For example:
>>>
>>> Smalltalk listPlugins contains: 'blahhh' ifFalse[ expectedFailures add: 'testBlah' ]
>>>
>>> or whatever condition. Can you do that with pragmas?
>>>
>>> the other problem is that we don't have expectedErrors as well. This is bad and confusing.
>>>
>>> cheers
>>>
>>> mariano
>>>
>>>
>>>
>>>
>>>
>>> Cheers,
>>>
>>> Niko
>>>
>>> --
>>> http://scg.unibe.ch/staff/Schwarz
>>> twitter.com/nes1983
>>> Tel: +41 076 235 8683
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>
>
>
> --
> http://scg.unibe.ch/staff/Schwarz
> twitter.com/nes1983
> Tel: +41 076 235 8683
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project


_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project