Squeak's AssertionFailure vs. SUnit's TestFailure

Hi, there.

1 to: aCollection size do: [:index |

    aCollection remove: (aCollection at: index)].

self assert: [aCollection isEmpty].

1 to: aCollection size do: [:index |

    aCollection remove: (aCollection at: index)].

aCollection isEmpty ifFalse: [

    self error: 'Internal error occurred! Collection is not empty'

    "Unless there is a programmer debugging, execution won't be continued"].

Hi Christoph.

Am 11.11.2019 22:29:22 schrieb Thiede, Christoph <[hidden email]>:

> I understand that Halt is just a way to invite users to debug programming Error, but an Error works equally well.

Related question: Why was Halt designed to behave exactly like an Error? I understand Halt as a note for a programmer to investigate something, but imho it does not indicate that something went wrong, so I would not expect it to abort the process (neglecting the fact that there is usually a debugger).

Imagine something like a ProductionToolSet (similar to the approach in https://wiki.squeak.org/squeak/6633), I would assume it should ignore Halts and should just resume. So wouldn't it be better if Halt raised an UnhandledHalt, which could be ignored by certain ToolSets? (I would even propose this behavior for the CommandLineToolSet.)

Please see the attached changeset.

---

Concerning AssertionFailures, I think outside of tests, this code

1 to: aCollection size do: [:index |

    aCollection remove: (aCollection at: index)].

self assert: [aCollection isEmpty].

should be broadly equivalent to

1 to: aCollection size do: [:index |

    aCollection remove: (aCollection at: index)].

aCollection isEmpty ifFalse: [

    self error: 'Internal error occurred! Collection is not empty'

    "Unless there is a programmer debugging, execution won't be continued"].

So I would agree with Marcel's proposal to derive AssertionFailure from Error.

Just a few days ago, I realized that an AssertionFaillure raised in a drawing method would damage the image, even if we catch "Error, Warning, Halt".

So it seems the right approach to me to refine the definition of exception classes instead of enumerating them at multiple different places.

Best,

Christoph

---

Von: Squeak-dev <[hidden email]> im Auftrag von Nicolas Cellier <[hidden email]>
Gesendet: Montag, 11. November 2019 21:24 Uhr
An: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] Squeak's AssertionFailure vs. SUnit's TestFailure

 

Hi Marcel,

Halt is a breakpoint explicitly set in the source code.

In some way, Assertion could be viewed as conditional halt explicitly set in the source code...

However, I don't like this POV. IMO AssertionFailure should not be subclass of Halt.

Assertion failures are failure to fullfil some pre/post condition, and as such programming Error.

I understand that Halt is just a way to invite users to debug programming Error, but an Error works equally well.

Why using Halt?

Le lun. 11 nov. 2019 à 16:03, Marcel Taeumel <[hidden email]> a écrit :

Hi, there.

The Kernel package provides "AssertionFailure" through "self assert: ...".

The SUnit package provides "TestFailure" through "self assert: ..." in test cases.

AssertionFailure is a subclass of Halt.

TestFailure is a subclass of Exception.

Now, the test runner catches all errors (red) and all test failures (yellow). However, the rest runner stops working if an AssertionFailure is raised. The Decompiler can do that, for example. In that case, the user cannot easily get an overview of all non-passing tests.

Here are my questions:

- Should AssertionFailure be an Error instead of a Halt?

- Should SUnit treat "real" AssertionFailures also as errors (red)?

Find attached two changesets with two different strategies.

See TestResult >> #runCase: to see which exceptions are caught during test execution.

Best,

Marcel

Hi Christoph.

Am 12.11.2019 10:15:05 schrieb Marcel Taeumel <[hidden email]>:

Hi Christoph.

> Why was Halt designed to behave exactly like an Error?

Well, it is not. Halt can be proceeded (in the debugger) (or resumed) without caution. :-) In Squeak's exception-handling terms, Halt seems to be a mix of Error and Notification.

> ... so I would not expect it to abort the process ...

It does not. A Halt just suspends the process.

> So wouldn't it be better if Halt raised an UnhandledHalt, which could be ignored by certain ToolSets?

Hmm... it is not that difficult to check for "Halt" in a toolset's #handleError:. I think that such halt's should not stay in code anyway. So, it is just a quick way to check something in the control flow. 

Best,

Marcel

Am 11.11.2019 22:29:22 schrieb Thiede, Christoph <[hidden email]>:

> I understand that Halt is just a way to invite users to debug programming Error, but an Error works equally well.

Related question: Why was Halt designed to behave exactly like an Error? I understand Halt as a note for a programmer to investigate something, but imho it does not indicate that something went wrong, so I would not expect it to abort the process (neglecting the fact that there is usually a debugger).

Imagine something like a ProductionToolSet (similar to the approach in https://wiki.squeak.org/squeak/6633), I would assume it should ignore Halts and should just resume. So wouldn't it be better if Halt raised an UnhandledHalt, which could be ignored by certain ToolSets? (I would even propose this behavior for the CommandLineToolSet.)

Please see the attached changeset.

---

Concerning AssertionFailures, I think outside of tests, this code

1 to: aCollection size do: [:index |

    aCollection remove: (aCollection at: index)].

self assert: [aCollection isEmpty].

should be broadly equivalent to

1 to: aCollection size do: [:index |

    aCollection remove: (aCollection at: index)].

aCollection isEmpty ifFalse: [

    self error: 'Internal error occurred! Collection is not empty'

    "Unless there is a programmer debugging, execution won't be continued"].

So I would agree with Marcel's proposal to derive AssertionFailure from Error.

Just a few days ago, I realized that an AssertionFaillure raised in a drawing method would damage the image, even if we catch "Error, Warning, Halt".

So it seems the right approach to me to refine the definition of exception classes instead of enumerating them at multiple different places.

Best,

Christoph

---

Von: Squeak-dev <[hidden email]> im Auftrag von Nicolas Cellier <[hidden email]>
Gesendet: Montag, 11. November 2019 21:24 Uhr
An: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] Squeak's AssertionFailure vs. SUnit's TestFailure

 

Hi Marcel,

Halt is a breakpoint explicitly set in the source code.

In some way, Assertion could be viewed as conditional halt explicitly set in the source code...

However, I don't like this POV. IMO AssertionFailure should not be subclass of Halt.

Assertion failures are failure to fullfil some pre/post condition, and as such programming Error.

I understand that Halt is just a way to invite users to debug programming Error, but an Error works equally well.

Why using Halt?

Le lun. 11 nov. 2019 à 16:03, Marcel Taeumel <[hidden email]> a écrit :

Hi, there.

The Kernel package provides "AssertionFailure" through "self assert: ...".

The SUnit package provides "TestFailure" through "self assert: ..." in test cases.

AssertionFailure is a subclass of Halt.

TestFailure is a subclass of Exception.

Now, the test runner catches all errors (red) and all test failures (yellow). However, the rest runner stops working if an AssertionFailure is raised. The Decompiler can do that, for example. In that case, the user cannot easily get an overview of all non-passing tests.

Here are my questions:

- Should AssertionFailure be an Error instead of a Halt?

- Should SUnit treat "real" AssertionFailures also as errors (red)?

Find attached two changesets with two different strategies.

See TestResult >> #runCase: to see which exceptions are caught during test execution.

Best,

Marcel

> Then again, what are notifications that need user input to resume automatically?

Well, a halt, nothing else so far, I guess :-)

> Hmm... it is not that difficult to check for "Halt" in a toolset's #handleError:

I think I see your point of holding the ToolSet interface clean, but making the halt check in a specific toolset implementation would stop us from providing different variation points to each toolset implementation, wouldn't it?

Best,

Christoph

Von: Squeak-dev <[hidden email]> im Auftrag von Taeumel, Marcel
Gesendet: Dienstag, 12. November 2019 10:17:57
An: John Pfersich via Squeak-dev
Betreff: Re: [squeak-dev] Squeak's AssertionFailure vs. SUnit's TestFailure

> Well, a halt, nothing else so far, I guess :-)

Am 14.11.2019 15:42:00 schrieb Thiede, Christoph <[hidden email]>:

> Then again, what are notifications that need user input to resume automatically?

Well, a halt, nothing else so far, I guess :-)

> Hmm... it is not that difficult to check for "Halt" in a toolset's #handleError:

I think I see your point of holding the ToolSet interface clean, but making the halt check in a specific toolset implementation would stop us from providing different variation points to each toolset implementation, wouldn't it?

Best,

Christoph

---

Von: Squeak-dev <[hidden email]> im Auftrag von Taeumel, Marcel
Gesendet: Dienstag, 12. November 2019 10:17:57
An: John Pfersich via Squeak-dev
Betreff: Re: [squeak-dev] Squeak's AssertionFailure vs. SUnit's TestFailure

 

Hi Christoph.

> Please see the attached changeset.

Ah, I see. In #defaultAction, Squeak treats a Halt similar to an Error. Then again, what are notifications that need user input to resume automatically? Some design challenges to tackle here ... :-)

Best,

Marcel

Am 12.11.2019 10:15:05 schrieb Marcel Taeumel <[hidden email]>:

Hi Christoph.

> Why was Halt designed to behave exactly like an Error?

Well, it is not. Halt can be proceeded (in the debugger) (or resumed) without caution. :-) In Squeak's exception-handling terms, Halt seems to be a mix of Error and Notification.

> ... so I would not expect it to abort the process ...

It does not. A Halt just suspends the process.

> So wouldn't it be better if Halt raised an UnhandledHalt, which could be ignored by certain ToolSets?

Hmm... it is not that difficult to check for "Halt" in a toolset's #handleError:. I think that such halt's should not stay in code anyway. So, it is just a quick way to check something in the control flow. 

Best,

Marcel

Am 11.11.2019 22:29:22 schrieb Thiede, Christoph <[hidden email]>:

> I understand that Halt is just a way to invite users to debug programming Error, but an Error works equally well.

Related question: Why was Halt designed to behave exactly like an Error? I understand Halt as a note for a programmer to investigate something, but imho it does not indicate that something went wrong, so I would not expect it to abort the process (neglecting the fact that there is usually a debugger).

Imagine something like a ProductionToolSet (similar to the approach in https://wiki.squeak.org/squeak/6633), I would assume it should ignore Halts and should just resume. So wouldn't it be better if Halt raised an UnhandledHalt, which could be ignored by certain ToolSets? (I would even propose this behavior for the CommandLineToolSet.)

Please see the attached changeset.

---

Concerning AssertionFailures, I think outside of tests, this code

1 to: aCollection size do: [:index |

    aCollection remove: (aCollection at: index)].

self assert: [aCollection isEmpty].

should be broadly equivalent to

1 to: aCollection size do: [:index |

    aCollection remove: (aCollection at: index)].

aCollection isEmpty ifFalse: [

    self error: 'Internal error occurred! Collection is not empty'

    "Unless there is a programmer debugging, execution won't be continued"].

So I would agree with Marcel's proposal to derive AssertionFailure from Error.

Just a few days ago, I realized that an AssertionFaillure raised in a drawing method would damage the image, even if we catch "Error, Warning, Halt".

So it seems the right approach to me to refine the definition of exception classes instead of enumerating them at multiple different places.

Best,

Christoph

---

Von: Squeak-dev <[hidden email]> im Auftrag von Nicolas Cellier <[hidden email]>
Gesendet: Montag, 11. November 2019 21:24 Uhr
An: The general-purpose Squeak developers list
Betreff: Re: [squeak-dev] Squeak's AssertionFailure vs. SUnit's TestFailure

 

Hi Marcel,

Halt is a breakpoint explicitly set in the source code.

In some way, Assertion could be viewed as conditional halt explicitly set in the source code...

However, I don't like this POV. IMO AssertionFailure should not be subclass of Halt.

Assertion failures are failure to fullfil some pre/post condition, and as such programming Error.

I understand that Halt is just a way to invite users to debug programming Error, but an Error works equally well.

Why using Halt?

Le lun. 11 nov. 2019 à 16:03, Marcel Taeumel <[hidden email]> a écrit :

Hi, there.

The Kernel package provides "AssertionFailure" through "self assert: ...".

The SUnit package provides "TestFailure" through "self assert: ..." in test cases.

AssertionFailure is a subclass of Halt.

TestFailure is a subclass of Exception.

Now, the test runner catches all errors (red) and all test failures (yellow). However, the rest runner stops working if an AssertionFailure is raised. The Decompiler can do that, for example. In that case, the user cannot easily get an overview of all non-passing tests.

Here are my questions:

- Should AssertionFailure be an Error instead of a Halt?

- Should SUnit treat "real" AssertionFailures also as errors (red)?

Find attached two changesets with two different strategies.

See TestResult >> #runCase: to see which exceptions are caught during test execution.

Best,

Marcel