Smalltalk › Usenets › Dolphin Smalltalk

Latest Dolphin patches have broken Unit Test running of individual suites

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

9 messages Options

Tim M

Latest Dolphin patches have broken Unit Test running of individual suites

I was just packaging up a new release of Intelli-Dolphin and have
noticed that when I run unit tests now (in a clean image without
intelli) - the test runner now creates a suite for every test method.
This seems to have happened in the latest patches (although I had to
download the latest Dolphin setup as I couldn't apply patch 2 as it
said my installer was out of date).

Thus if I right click on a TestCase and pick Run Tests - I USED TO get
a single suite in the left pane of the test runner, and picking that
suite would then show me the test cases (e.g. one for each test
method).

It may be releated to something that has changed in the latest patches
- in Intelli I notice that where I used to do:

filteredClasses
do: [:each | testSuite addTest: (each buildSuiteFromEnvironment:
BrowserEnvironment new)].

The #buildSuiteFromEnvironment: method is now gone. It seems like I now
should write:

filteredClasses
do: [:each | testSuite addTest: (each buildSuiteFromSelectors)].

Blair McGlashan-3

Re: Latest Dolphin patches have broken Unit Test running of individual suites

"TimM" <[hidden email]> wrote in message
news:[hidden email]...

>I was just packaging up a new release of Intelli-Dolphin and have
> noticed that when I run unit tests now (in a clean image without
> intelli) - the test runner now creates a suite for every test method.
> This seems to have happened in the latest patches (although I had to
> download the latest Dolphin setup as I couldn't apply patch 2 as it
> said my installer was out of date).
>
> Thus if I right click on a TestCase and pick Run Tests - I USED TO get
> a single suite in the left pane of the test runner, and picking that
> suite would then show me the test cases (e.g. one for each test
> method).
>

The behaviour in 6.02 has reverted to the behaviour in Dolphin 5.x, which I
assume is the way that Jeff had originally intended it should work. There
was a bug in the original D6 release in that the SUnit Browser shown by
browsing the test cases on a class did not include inherited test cases.
This wasn't intentional, but I don't think it is "broken". If the browser is
invoked to display either all tests in your image, or all tests in a
package, then you will get the TestCases appearing as suites, when browsing
a single test case it is expanded out. Logically the original D6 behaviour
was perhaps more correct, but the D5 behaviour seems more usable in
practice.

What does everyone think, should it display a single suite, or expand it out
in the main list?

> It may be releated to something that has changed in the latest patches
> - in Intelli I notice that where I used to do:
>
> filteredClasses
> do: [:each | testSuite addTest: (each buildSuiteFromEnvironment:
> BrowserEnvironment new)].
>
> The #buildSuiteFromEnvironment: method is now gone. It seems like I now
> should write:
>
> filteredClasses
> do: [:each | testSuite addTest: (each buildSuiteFromSelectors)].
>

I'm not sure, since I don't know what you are trying to achieve.
#buildSuiteFromEnvironment: was an extension method that we'd added, its
purpose being as named (i.e. to construct a TestSuite from only those test
cases defined in a BrowserEnvironment). It was implemented in order to
support browsing tests by package (i.e. to allow a TestCase to have loose
test cases that would be selectively shown when one initiated from the
System Browser based on package selection). However it was complex and
didn't work correctly. Rather than attempt to correct it, we removed it and
reverted to the basic TestSuite construction methods by SUnit.

Regards

Blair

Tim M

Re: Latest Dolphin patches have broken Unit Test running of individual suites

>If the browser is invoked to display either all tests in
> your image, or all tests in a package, then you will get the TestCases
> appearing as suites, when browsing a single test case it is expanded
> out. Logically the original D6 behaviour was perhaps more correct, but
> the D5 behaviour seems more usable in practice.
>
> What does everyone think, should it display a single suite, or expand
> it out in the main list?

I think the previous D6 behavior was the correct and most useful one - I
have a lot of tests I now find that I have to scroll up and down the list
to see them - its quite distracting. It also makes the right pane of the
Test browser meaningless as it just shows you exactly what the left side
is already showing you...

Before when it grouped your tests by TestSuite (remember that TestSuite is
a composite) and then showed the individual TestCases on the right it made
far more sense - it also means that the columns in the UI were useful - ie.
Correct, Failures, Errors etc - now they are just binary 1/0 columns, before
you could see that TestCase XYZ had 10 correct, 5 failures and 0 errors.
You clicked on that row to see the details.

It seems to me that the "bug" is here:

TestSuite>>buildSuite
| copy |
copy := self class named: self name.
copy addTests: self allTests.
^copy

if self tests is:

an OrderedCollection(a TestSuite a TestSuite a TestSuite)

then the copy should contain the same collection of tests, instead it has
them all flattened out e.g.

an OrderedCollection(MockeryTests>>testAnyOfSignalWithSeveralMatchingConstraints
MockeryTests>>testAnyOfSignalWithSingleMatchingConstraint MockeryTests>>testAnyOfWithDifferentParameters
...)

So I would argue that this method should be something like:

buildSuite
| copy |
copy := self class named: self name.
self tests do: [:each |
copy addTest: each copy].
^copy

Tim

Chris Uppal-3

Re: Latest Dolphin patches have broken Unit Test running of individual suites

In reply to this post by Blair McGlashan-3

Blair,

> There was a bug in the original D6 release in that the SUnit Browser
> shown by browsing the test cases on a class did not include inherited
> test cases. This wasn't intentional, but I don't think it is "broken". [...]
>
> What does everyone think, should it display a single suite, or expand it
> out in the main list?

I don't care too much about how tests are grouped in the browser, but it's
important to me that the bug doesn't come back. Nearly all of my tests are
inherited in one way or another.

-- chris

James Foster-3

Re: Latest Dolphin patches have broken Unit Test running of individual suites

In reply to this post by Blair McGlashan-3

It appears to me that there are two issues: (1) inherited tests went missing
in 6.02 (making the SUnitBrowser largely worthless in many cases); and (2)
the tests are split out of their class-based suites and each test is put in
a separate suite (making the two-pane UI and the count columns pointless).

The first issue appears to be related to PackageSelector>>buildTestSuite
which filters out all inherited tests. A simple work-around is to modify the
#ifTrue: block to be [suite addTest: eachClass suite].

The second issue appears to be related to both
PackageSelector>>buildTestSuite (which needs the fix in the prior paragraph)
and TestSuite>>buildSuite, where the modification provided by "macta" on
2/14/2006 seems to work (if combined with the other fix).

I'd appreciate it if these two issues were addressed in the next update by
including the above fixes or something else that solves the problems.

James Foster

"Blair McGlashan" <[hidden email]> wrote in message
news:[hidden email]...

> "TimM" <[hidden email]> wrote in message
> news:[hidden email]...
>>I was just packaging up a new release of Intelli-Dolphin and have
>> noticed that when I run unit tests now (in a clean image without
>> intelli) - the test runner now creates a suite for every test method.
>> This seems to have happened in the latest patches (although I had to
>> download the latest Dolphin setup as I couldn't apply patch 2 as it
>> said my installer was out of date).
>>
>> Thus if I right click on a TestCase and pick Run Tests - I USED TO get
>> a single suite in the left pane of the test runner, and picking that
>> suite would then show me the test cases (e.g. one for each test
>> method).
>>
>
> The behaviour in 6.02 has reverted to the behaviour in Dolphin 5.x, which
> I assume is the way that Jeff had originally intended it should work.
> There was a bug in the original D6 release in that the SUnit Browser shown
> by browsing the test cases on a class did not include inherited test
> cases. This wasn't intentional, but I don't think it is "broken". If the
> browser is invoked to display either all tests in your image, or all tests
> in a package, then you will get the TestCases appearing as suites, when
> browsing a single test case it is expanded out. Logically the original D6
> behaviour was perhaps more correct, but the D5 behaviour seems more usable
> in practice.
>
> What does everyone think, should it display a single suite, or expand it
> out in the main list?
>
>> It may be releated to something that has changed in the latest patches
>> - in Intelli I notice that where I used to do:
>>
>> filteredClasses
>> do: [:each | testSuite addTest: (each buildSuiteFromEnvironment:
>> BrowserEnvironment new)].
>>
>> The #buildSuiteFromEnvironment: method is now gone. It seems like I now
>> should write:
>>
>> filteredClasses
>> do: [:each | testSuite addTest: (each buildSuiteFromSelectors)].
>>
>
> I'm not sure, since I don't know what you are trying to achieve.
> #buildSuiteFromEnvironment: was an extension method that we'd added, its
> purpose being as named (i.e. to construct a TestSuite from only those test
> cases defined in a BrowserEnvironment). It was implemented in order to
> support browsing tests by package (i.e. to allow a TestCase to have loose
> test cases that would be selectively shown when one initiated from the
> System Browser based on package selection). However it was complex and
> didn't work correctly. Rather than attempt to correct it, we removed it
> and reverted to the basic TestSuite construction methods by SUnit.
>
> Regards
>
> Blair
>

Schwab,Wilhelm K

Re: Latest Dolphin patches have broken Unit Test running of individual suites

Blair,

>> I'm not sure, since I don't know what you are trying to achieve.
>> #buildSuiteFromEnvironment: was an extension method that we'd added, its
>> purpose being as named (i.e. to construct a TestSuite from only those test
>> cases defined in a BrowserEnvironment). It was implemented in order to
>> support browsing tests by package (i.e. to allow a TestCase to have loose
>> test cases that would be selectively shown when one initiated from the
>> System Browser based on package selection). However it was complex and
>> didn't work correctly. Rather than attempt to correct it, we removed it
>> and reverted to the basic TestSuite construction methods by SUnit.

Have you looked at my filtered browser goodie? It launches the test
browser on a suite that has always seemed reasonable to me, though I do
not claim to have created exhaustive tests. I would welcome bug reports
if appropriate; feel free to use/modify the code if it helps.

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]

Steve Alan Waring

Re: Latest Dolphin patches have broken Unit Test running of individual suites

In reply to this post by James Foster-3

One more issue that I have come across:

In D6, the SUnitBrowser can take 4-5 seconds to open on a particular
TestCase, if the image contains a lot of TestCases.

I think the problem is in SUnitBrowser class>>openOnTestCase:

This method firstly displays the presenter on the #defaultModel ...
which builds a full suite, and then sets a new model with only the
required tests. Building a full suite can take a while.

In D5, SUnitBrowser class>>openOnTestCase: did not open on the
#defaultModel, but I can see this method has had major changes since
D5.

Changing it to the method below fixed the speed issue ... not sure if
there are any side effects.

!SUnitBrowser class methodsFor!

openOnTestCase: aTestCase
^self showOn: (SUnitBrowserModel newForUI: nil builder: aTestCase)! !

Steve
--
Steve Waring

Steve Alan Waring

Re: Latest Dolphin patches have broken Unit Test running of individual suites

> Changing it to the method below fixed the speed issue ... not sure if
> there are any side effects.

Side effects are that it breaks reuseIfOpen and opening in an ideaspace
:(

Steve
--
[hidden email]

James Foster-3

Re: Latest Dolphin patches have broken Unit Test running of individual suites

In reply to this post by Blair McGlashan-3

Blair,

It appears to me that there are two issues: (1) inherited tests went missing
in 6.02 (making the SUnitBrowser largely worthless in many cases); and (2)
the tests are split out of their class-based suites and each test is put in
a separate suite (making the two-pane UI and the count columns pointless).

The first issue appears to be related to PackageSelector>>buildTestSuite
which filters out all inherited tests. A simple work-around is to modify the
#ifTrue: block to be [suite addTest: eachClass suite].

The second issue appears to be related to both
PackageSelector>>buildTestSuite (which needs the fix in the prior paragraph)
and TestSuite>>buildSuite, where the modification provided by "macta" on
2/14/2006 seems to work (if combined with the other fix).

Do you think these two issues can be noted as bugs/feature requests?

James Foster

"Blair McGlashan" <[hidden email]> wrote in message
news:[hidden email]...