Smalltalk › Squeak › Squeak - Dev

Tests are yellow!

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

3 messages Options

Ralph Johnson

Tests are yellow!

"Tests are yellow" doesn't sound nearly as nice as "Tests are green",
does it? That is exactly what I felt when I got rid of the last
failing test for Squeak 3.9 and the bar on the Test Runner was *still*
yellow! It says that there are 2 expected failures. However, it
doesn't list any methods. I don't know whether this means that it
was a failure, but it was supposed to be a failure, or whether it was
supposed to be a failure, but wasn't. If everything is OK, why is the
Test Runner yellow? If everything is not OK, why doesn't it tell me
which tests are indicating a problem? Can you help me either make the
Test Runner green or figure out which test is the problem so I can fix
it?

I have attached two change files for your amusement. They are both
for Squeak3.9-finel-7067. The first will, despite its name, make the
tests yellow. There will be no tests left to fix, though, so it
illustrates what I am complaining about. Hans-Martin Mosner did most
of the work to make this happen. Andreas Raab said we could delete
the FloatMathPluginTests and Colin Putney said we could delte the
ToolBuilderTest. All I did was ask the original question, delete
tests people said could be deleted, and move the other failing tests
to subclasses in another category, which I deleted. I have also
attached this other cateogry, which is called FailingTests. Don't
file it in unless you want the Test Runner to be red!

-Ralph Johnson

Make39Green.cs (27K) Download Attachment

FailingTests.st (15K) Download Attachment

Giovanni Corriga

Re: Tests are yellow!

Il giorno sab, 25/11/2006 alle 07.26 -0600, Ralph Johnson ha scritto:

What's the rationale for having expected failures in the first place?

Giovanni

Adrian Lienhard

Re: Tests are yellow!

In reply to this post by Ralph Johnson

Hi Ralph,

see implementors of #expectedFailures

IIRC I tagged two tests of Andrew Black as expected failures because
they document a problem in the compiler (and I didn't expect that the
compiler will be fixed soon).

Concerning the behavior of the test runner, I think it should turn
green in this situation. In case the expected failures actually pass,
it should be yellow (and list the methods that passed in the failures
list).

Cheers,
Adrian
___________________
Adrian Lienhard
www.adrian-lienhard.ch

On Nov 25, 2006, at 14:26 , Ralph Johnson wrote:

> "Tests are yellow" doesn't sound nearly as nice as "Tests are green",
> does it? That is exactly what I felt when I got rid of the last
> failing test for Squeak 3.9 and the bar on the Test Runner was *still*
> yellow! It says that there are 2 expected failures. However, it
> doesn't list any methods. I don't know whether this means that it
> was a failure, but it was supposed to be a failure, or whether it was
> supposed to be a failure, but wasn't. If everything is OK, why is the
> Test Runner yellow? If everything is not OK, why doesn't it tell me
> which tests are indicating a problem? Can you help me either make the
> Test Runner green or figure out which test is the problem so I can fix
> it?
>
> I have attached two change files for your amusement. They are both
> for Squeak3.9-finel-7067. The first will, despite its name, make the
> tests yellow. There will be no tests left to fix, though, so it
> illustrates what I am complaining about. Hans-Martin Mosner did most
> of the work to make this happen. Andreas Raab said we could delete
> the FloatMathPluginTests and Colin Putney said we could delte the
> ToolBuilderTest. All I did was ask the original question, delete
> tests people said could be deleted, and move the other failing tests
> to subclasses in another category, which I deleted. I have also
> attached this other cateogry, which is called FailingTests. Don't
> file it in unless you want the Test Runner to be red!
>
> -Ralph Johnson
> <Make39Green.cs>
> <FailingTests.st>
>