The Inbox: SUnitTools-ct.5.mcz

Am 31.05.2019 14:15:43 schrieb [hidden email] <[hidden email]>:

A new version of SUnitTools was added to project The Inbox:
http://source.squeak.org/inbox/SUnitTools-ct.5.mcz

==================== Summary ====================

Name: SUnitTools-ct.5
Author: ct
Time: 31 May 2019, 2:15:36.714053 pm
UUID: 397d3b85-7ac5-3645-baa9-b9f848d90880
Ancestors: SUnitTools-tcj.4

Change CodeHolder>>testRunSuite: -- if multiple tests fail, instead of a UIManager dialog, open a TestRunner window that provides specific information.

Depends on SUnitGUI-ct.70.

=============== Diff against SUnitTools-tcj.4 ===============

Item was changed:
----- Method: CodeHolder>>testRunSuite: (in category '*SUnitTools-running') -----
testRunSuite: suite

+ | runner result |
+ (suite isKindOf: TestCase)
+ ifFalse: [
+ runner := TestRunner newForSuite: suite.
+ result := runner runAll; result]
+ ifTrue: [result := suite run].
- | result |
- result := suite run.

(result respondsTo: #dispatchResultsIntoHistory)
ifTrue: [result dispatchResultsIntoHistory].

result hasPassed ifTrue: [^ self].

+ result defects size = 1
+ ifTrue: [result defects anyOne debug]
+ ifFalse: [runner open]!
- (result defects size = 1
- ifTrue: [result defects anyOne]
- ifFalse: [UIManager default
- chooseFrom: (result defects collect: [:each | each class name , '>>' , each selector printString])
- values: result defects
- title: ('{1} passes, {2} failures, {3} errors\\Debug a failure or error?' format: {
- result runCount . result failureCount . result errorCount}) withCRs]
- ) ifNotNil: [:defect | defect debug].
- !

> What are the specific information in the test runner you are referring to?

It is "reentrant", you do not need to re-run all tests after debugging one. A DialogBox would need to be manually set as non-exclusive.

> If the test runner would be a better option than the dialog, wouldn't we want to show it even if only a single test fails (result defects size = 1)?

You're right, we could :)

It is modally exclusive but not reentrant.

(I believe long test selector were displayed abbreviated, but I can't reproduce that.)

Also, DialogBoxes are not resizable.

On the other hand, I actually avoided using TestRunner in the last time, as it has a few convenience drawbacks:

1. I repeatedly faced an issue where a test, after debugging it, was not tear-downed correctly. Will try to reproduce that.

2. You cannot select a test without running it (impractical if you would like to open the menu, for example to browse it). Vice versa, you cannot run a test without selecting it, so the keyboard handling (such as navigation using arrow keys) is quite restricted ...

3. TestRunner is not aware of any additions to the test list (maybe we need something like #didCodeChangeElsewhere)

But your words "specific information" sound interesting to me. Would it be an option to turn the TestRunner upside down, aiming to display the last error information for each test?

Some wild ideas :-)

• allow selection of single test cases; instead of separately displaying all errors/failures, display their success with an icon
• add another panel to display information for the selected test
  • this might include the failing line of the test method, the full error message (displayed in a big TextMorph, not as a one-liner with insufficient space), maybe the context stack - basically something like a collapsed debugger view
  • I had an idea in the back of my mind for a few months: One could set up a small extension-based framework that provides additional information about the failing assertion (if you're interested, I can tell you more) - we could display these information here
    
    • for example, in [self assert: [string allSatisfy: #isDigit]], instead of "AssertionFailed", the failure description could be something like "expected all elements of <'123a5'> to satisfy <#isDigit>, but <'a'> at index <4> didn't"
    • ideally, the information would not be restricted to textual representation (why not import a diff view or an Inspector into the TestRunner? :) )
  • It would be also possible to display *all* failures, not only the first one (I once read about resumable TestFailure implementations, why is TestFailure not marked as resumable by default?)
• If there are only few tests, maybe the test runner could suspend the failing/erroneous ones (actually their threads) instead of shutting them down? The user could then resume a test and directly jump to the raised Exception in debugger, without needing to wait for the test to re-run. Especially in case of heavy tests this might be a great help to some users.
• By always displaying the source of the selected test method, we could improve the overall liveness of SUnit