ResumableTestFailure

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

ResumableTestFailure

Christoph Thiede

Hi all! :-)


I was just working a bit with Squeak by Example (the book) and found out that resumable assertions as described in the book (see chapter SUnit, Advanced, Continuing after a failure) do not any longer work in a trunk image. If you write a test case with some resumable assertions, turn on logging, and run the test, only the first failure will be logged, not the rest, too.


(Methods to reproduce:

isLogging

    ^  true "You will need to open a Transcript"

testResumable

    #(1 2 3 4 5) do: [:i |

        self assert: i even description: ['{1} is not even' format: {i}] resumable: true]

)


A second issue I experienced frequently when I felt the desire to work with resumable assertions was that they are basically only available for two selectors: #assert:description:resumable: and #deny:description:identical:. I often would like to insert a resumable into any test without being that verbose, and nearly redoubling the number of assertion selectors by adding a resumable argument to each of them did not feel right, either.


Please find the attached changeset, which fixes both issues.

  • In TestResult >> #runCase:, resumable assertions are identified and - finally - resumed, after marking the test as failed. Unfortunately, other test runners (yes, I'm thinking of smalltalkCI) will need to reimplement this feature unless they reuse the TestResult implementation.
  • In TestCase, I refactored some methods that raise assertion failures and added the most basic #signalFailure:resumable: as the single source of signalization. Instead of setting the resumable bit manually for each assertion, you now can also make the whole test execution resumable by:
    • overriding #isResumable,
    • setting a resumable flag using #resumable:,
    • setting a resumable flag temporarily using #beResumable:during:, or
    • adding a <resumable> or <resumable: true> pragma to the test method.
  • In addition, I tested the added and fixed behavior in ResumableTestFailureTestCase.


Looking forward to your review! :-)


Best,

Christoph




resumable.1.cs (7K) Download Attachment
Carpe Squeak!
Reply | Threaded
Open this post in threaded view
|

Re: ResumableTestFailure

Christoph Thiede

PS: I ran all tests in a fresh trunk image before and after loading the changeset, there were no regressions.


Von: Squeak-dev <[hidden email]> im Auftrag von Thiede, Christoph
Gesendet: Dienstag, 29. September 2020 01:18:07
An: Squeak Dev
Betreff: [squeak-dev] ResumableTestFailure
 

Hi all! :-)


I was just working a bit with Squeak by Example (the book) and found out that resumable assertions as described in the book (see chapter SUnit, Advanced, Continuing after a failure) do not any longer work in a trunk image. If you write a test case with some resumable assertions, turn on logging, and run the test, only the first failure will be logged, not the rest, too.


(Methods to reproduce:

isLogging

    ^  true "You will need to open a Transcript"

testResumable

    #(1 2 3 4 5) do: [:i |

        self assert: i even description: ['{1} is not even' format: {i}] resumable: true]

)


A second issue I experienced frequently when I felt the desire to work with resumable assertions was that they are basically only available for two selectors: #assert:description:resumable: and #deny:description:identical:. I often would like to insert a resumable into any test without being that verbose, and nearly redoubling the number of assertion selectors by adding a resumable argument to each of them did not feel right, either.


Please find the attached changeset, which fixes both issues.

  • In TestResult >> #runCase:, resumable assertions are identified and - finally - resumed, after marking the test as failed. Unfortunately, other test runners (yes, I'm thinking of smalltalkCI) will need to reimplement this feature unless they reuse the TestResult implementation.
  • In TestCase, I refactored some methods that raise assertion failures and added the most basic #signalFailure:resumable: as the single source of signalization. Instead of setting the resumable bit manually for each assertion, you now can also make the whole test execution resumable by:
    • overriding #isResumable,
    • setting a resumable flag using #resumable:,
    • setting a resumable flag temporarily using #beResumable:during:, or
    • adding a <resumable> or <resumable: true> pragma to the test method.
  • In addition, I tested the added and fixed behavior in ResumableTestFailureTestCase.


Looking forward to your review! :-)


Best,

Christoph



Carpe Squeak!