Smalltalk › Pharo › Pharo Smalltalk Users

Why does the test runner show red when I correct a test?

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

47 messages Options

123

Richard Sargent

Re: Why does the test runner show red when I correct a test?

Administrator

So after a restart we would expect green

Yes.

Add having the test tool able to automate the tear-down and restart of the test would be a good automation.

On Fri, Nov 10, 2017 at 11:41 AM, stephan <[hidden email]> wrote:

On 10-11-17 18:56, Richard Sargent wrote:

That would be fine.
The point is that, without running the test in its entirety, from start to finish, without interruption, error, or failure, one cannot claim success.

So after a restart we whould expect green

Stephan

Ben Coman

Re: Why does the test runner show red when I correct a test?

In reply to this post by Richard Sargent

The CI sometimes fails due to heisenbugs, which of course we should fix but mostly someone just kicks the CI off again and the test passes and the code is integrated. So these heisenbugs slow down development when the CI fails, and also we are not gaining any systematic statistics on where the heisenbugs come from.

So for a while I've thought it would be useful for the CI to automatically rerun a failed test to bypass the heisenbugs and at the same time log them to a central location for anaylsis. So I think such a feature could also suit this situation. After you correct the code and resume execution to the end, the test is automatically rerun and turns green, or fails again bringing you straight back to the debugger. Actually that last itself would be a useful feature for a tight cycle dev/testing a Test.

cheers -ben

On Sat, Nov 11, 2017 at 3:45 AM, Richard Sargent <[hidden email]> wrote:

I hear you and I understand your pain.

However, if you corrected the problem, not by a code change, but by playing in the object's inspector or otherwise causing its internal state to change, and then resumed from the debugger, would you still claim the method was successful and should be coloured green?

The only thing we can claim is that there were or were not further errors in the test. Colour it red if there were, but you cannot honestly colour it green. The code doesn't actually work.

On Fri, Nov 10, 2017 at 11:29 AM, Tim Mackinnon <[hidden email]> wrote:
My specific usecase is from a pragmatic TDD perspective - failing test, in the debugger you fix the test and press proceed - expecting green. Getting red, and then immediately running again to get red takes away from our story of love coding and loving your debugger - and even Cassie me to mistrust the tools.

I get the idea that you can jiffy in the debugger and cause a false pass - but I feel you are penalised for doing it right at the moment.

Of course these tests will get run again, but I like the idea that if I did it right, it should recognise this, not incur an extra click and moment of doubt.

A button to rerun the whole lot, or automatically rerun, or just run it would work for me.

Tim

Sent from my iPhone

On 10 Nov 2017, at 17:56, Richard Sargent <[hidden email]> wrote:

That would be fine.
The point is that, without running the test in its entirety, from start to finish, without interruption, error, or failure, one cannot claim success.

On Fri, Nov 10, 2017 at 9:34 AM, Sean P. DeNigris <[hidden email]> wrote:
Richard Sargent wrote
> The only reliable conclusion one can make from such an interrupted run is
> whether it failed again. So, it would be possible to determine that the
> test should be coloured red, but it is impossible to *reliably* claim that
> the test should be coloured green.

What if we ran the test again as if from the browser/runner before setting
the icon?

-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Tim Mackinnon

Re: Why does the test runner show red when I correct a test?

In reply to this post by Richard Sargent

Richard - I better understand what you are saying now. If you change the method and save it (hence restarting on the stack) I would expect it to turn green if I press resume. That is the TDD flow I expect, and one which sunit and coding in the debugger was predicated on.

However, there is the potential that some memory object that cached a result and is now running a second time could be the cause of a pass and so it is remotely possible that this is a false pass…. (And this I think is the crux of your argument - if any memory object is affected, theoretically you should rerun the whole transaction from a virgin state - which is effectively rerun the test from the beginning). So I guess we are discussing that we don’t have fully transactional memory executions?

However I would argue that its way more common that you edit a method and fix a text and have 0 side effects than mucking around in memory or having something that rerunning locally messes up memory as well. So its much more useful to get the confirmation of green and move on. YES you could have a subtle error, and when you re-run it may later go red - but I would favour the 99% case as its a annoying if you are doing TDD.

Tim

On 10 Nov 2017, at 19:45, Richard Sargent <[hidden email]> wrote:

I hear you and I understand your pain.

However, if you corrected the problem, not by a code change, but by playing in the object's inspector or otherwise causing its internal state to change, and then resumed from the debugger, would you still claim the method was successful and should be coloured green?

The only thing we can claim is that there were or were not further errors in the test. Colour it red if there were, but you cannot honestly colour it green. The code doesn't actually work.

On Fri, Nov 10, 2017 at 11:29 AM, Tim Mackinnon <[hidden email]> wrote:
My specific usecase is from a pragmatic TDD perspective - failing test, in the debugger you fix the test and press proceed - expecting green. Getting red, and then immediately running again to get red takes away from our story of love coding and loving your debugger - and even Cassie me to mistrust the tools.

I get the idea that you can jiffy in the debugger and cause a false pass - but I feel you are penalised for doing it right at the moment.

Of course these tests will get run again, but I like the idea that if I did it right, it should recognise this, not incur an extra click and moment of doubt.

A button to rerun the whole lot, or automatically rerun, or just run it would work for me.

Tim

Sent from my iPhone

On 10 Nov 2017, at 17:56, Richard Sargent <[hidden email]> wrote:

That would be fine.
The point is that, without running the test in its entirety, from start to finish, without interruption, error, or failure, one cannot claim success.

On Fri, Nov 10, 2017 at 9:34 AM, Sean P. DeNigris <[hidden email]> wrote:
Richard Sargent wrote
> The only reliable conclusion one can make from such an interrupted run is
> whether it failed again. So, it would be possible to determine that the
> test should be coloured red, but it is impossible to *reliably* claim that
> the test should be coloured green.

What if we ran the test again as if from the browser/runner before setting
the icon?

-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Richard Sargent (again)

Re: Why does the test runner show red when I correct a test?

>> I would expect it to turn green if I press resume.

I disagree with your expectations. You are arguing that we should operate is if "probably correct" is the same as "correct". That's why we have ****ty software.

I have no objection to leaving the method uncoloured when you resume after correcting the error.

I have no objection to eliminating the red colouring in these scenarios.

I strongly object to pretending that it is known to be correct.

In my experience, we end up with better results when we act on what we absolutely know to be factual and stop relying on guessing.

From: Pharo-users [mailto:[hidden email]] On Behalf Of Tim Mackinnon
Sent: November 14, 2017 06:19
To: Any question about pharo is welcome
Subject: Re: [Pharo-users] Why does the test runner show red when I correct a test?

Tim

On 10 Nov 2017, at 19:45, Richard Sargent <[hidden email]> wrote:

I hear you and I understand your pain.

However, if you corrected the problem, not by a code change, but by playing in the object's inspector or otherwise causing its internal state to change, and then resumed from the debugger, would you still claim the method was successful and should be coloured green?

The only thing we can claim is that there were or were not further errors in the test. Colour it red if there were, but you cannot honestly colour it green. The code doesn't actually work.

On Fri, Nov 10, 2017 at 11:29 AM, Tim Mackinnon <[hidden email]> wrote:

My specific usecase is from a pragmatic TDD perspective - failing test, in the debugger you fix the test and press proceed - expecting green. Getting red, and then immediately running again to get red takes away from our story of love coding and loving your debugger - and even Cassie me to mistrust the tools.

I get the idea that you can jiffy in the debugger and cause a false pass - but I feel you are penalised for doing it right at the moment.

Of course these tests will get run again, but I like the idea that if I did it right, it should recognise this, not incur an extra click and moment of doubt.

A button to rerun the whole lot, or automatically rerun, or just run it would work for me.

Tim

Sent from my iPhone

On 10 Nov 2017, at 17:56, Richard Sargent <[hidden email]> wrote:

That would be fine.
The point is that, without running the test in its entirety, from start to finish, without interruption, error, or failure, one cannot claim success.

On Fri, Nov 10, 2017 at 9:34 AM, Sean P. DeNigris <[hidden email]> wrote:

Richard Sargent wrote
> The only reliable conclusion one can make from such an interrupted run is
> whether it failed again. So, it would be possible to determine that the
> test should be coloured red, but it is impossible to *reliably* claim that
> the test should be coloured green.

What if we ran the test again as if from the browser/runner before setting
the icon?

-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Ben Coman

Re: Why does the test runner show red when I correct a test?

On 15 November 2017 at 00:14, Richard Sargent <[hidden email]> wrote:

>> I would expect it to turn green if I press resume.

I disagree with your expectations. You are arguing that we should operate is if "probably correct" is the same as "correct". That's why we have ****ty software.

I have no objection to leaving the method uncoloured when you resume after correcting the error.

I have no objection to eliminating the red colouring in these scenarios.

I strongly object to pretending that it is known to be correct.

Or it could go to Amber, half-way between green & red to mean probably correct.

cheers -ben

In my experience, we end up with better results when we act on what we absolutely know to be factual and stop relying on guessing.

From: Pharo-users [mailto:[hidden email]] On Behalf Of Tim Mackinnon
Sent: November 14, 2017 06:19
To: Any question about pharo is welcome
Subject: Re: [Pharo-users] Why does the test runner show red when I correct a test?

Richard - I better understand what you are saying now. If you change the method and save it (hence restarting on the stack) I would expect it to turn green if I press resume. That is the TDD flow I expect, and one which sunit and coding in the debugger was predicated on.

However, there is the potential that some memory object that cached a result and is now running a second time could be the cause of a pass and so it is remotely possible that this is a false pass…. (And this I think is the crux of your argument - if any memory object is affected, theoretically you should rerun the whole transaction from a virgin state - which is effectively rerun the test from the beginning). So I guess we are discussing that we don’t have fully transactional memory executions?

However I would argue that its way more common that you edit a method and fix a text and have 0 side effects than mucking around in memory or having something that rerunning locally messes up memory as well. So its much more useful to get the confirmation of green and move on. YES you could have a subtle error, and when you re-run it may later go red - but I would favour the 99% case as its a annoying if you are doing TDD.

Tim

On 10 Nov 2017, at 19:45, Richard Sargent <[hidden email]> wrote:

I hear you and I understand your pain.

However, if you corrected the problem, not by a code change, but by playing in the object's inspector or otherwise causing its internal state to change, and then resumed from the debugger, would you still claim the method was successful and should be coloured green?

The only thing we can claim is that there were or were not further errors in the test. Colour it red if there were, but you cannot honestly colour it green. The code doesn't actually work.

On Fri, Nov 10, 2017 at 11:29 AM, Tim Mackinnon <[hidden email]> wrote:

My specific usecase is from a pragmatic TDD perspective - failing test, in the debugger you fix the test and press proceed - expecting green. Getting red, and then immediately running again to get red takes away from our story of love coding and loving your debugger - and even Cassie me to mistrust the tools.

I get the idea that you can jiffy in the debugger and cause a false pass - but I feel you are penalised for doing it right at the moment.

Of course these tests will get run again, but I like the idea that if I did it right, it should recognise this, not incur an extra click and moment of doubt.

A button to rerun the whole lot, or automatically rerun, or just run it would work for me.

Tim

Sent from my iPhone

On 10 Nov 2017, at 17:56, Richard Sargent <[hidden email]> wrote:

That would be fine.
The point is that, without running the test in its entirety, from start to finish, without interruption, error, or failure, one cannot claim success.

On Fri, Nov 10, 2017 at 9:34 AM, Sean P. DeNigris <[hidden email]> wrote:

Richard Sargent wrote
> The only reliable conclusion one can make from such an interrupted run is
> whether it failed again. So, it would be possible to determine that the
> test should be coloured red, but it is impossible to *reliably* claim that
> the test should be coloured green.

What if we ran the test again as if from the browser/runner before setting
the icon?

-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Sean P. DeNigris

Re: Why does the test runner show red when I correct a test?

Administrator

Ben Coman wrote
> Or it could go to Amber, half-way between green & red to mean probably
> correct.

Ha ha.

Again, it seems that just automatically rerunning the test immediately after
a human-manipulated run and setting the color based on that second run
addresses all points on both sides, no?

-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Cheers,
Sean

Richard Sargent

Re: Why does the test runner show red when I correct a test?

Administrator

Again, it seems that just automatically rerunning the test immediately after a human-manipulated run and setting the color based on that second run

Absolutely!

(Although, I was kind of looking forward to the third colour discussions. My vote would have been for paisley!)

On Tue, Nov 14, 2017 at 4:49 PM, Sean P. DeNigris <[hidden email]> wrote:

Ben Coman wrote
> Or it could go to Amber, half-way between green & red to mean probably
> correct.

Ha ha.

Again, it seems that just automatically rerunning the test immediately after
a human-manipulated run and setting the color based on that second run
addresses all points on both sides, no?

-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Stephan Eggermont-3

Re: Why does the test runner show red when I correct a test?

In reply to this post by Sean P. DeNigris

Op 15-11-2017 om 01:49 schreef Sean P. DeNigris:
> Again, it seems that just automatically rerunning the test immediately after
> a human-manipulated run and setting the color based on that second run
> addresses all points on both sides, no?

How many times do you want to restart execution? I have writen enough
code where running it twice was not enough to get to green :)

Stephan

Richard Sargent

Re: Why does the test runner show red when I correct a test?

Administrator

I would say that is an entirely different question. If a test isn't stable, you have a completely different scenario from a test that interrupts execution and is resumed.

On Nov 14, 2017 20:45, "Stephan Eggermont" <[hidden email]> wrote:

Op 15-11-2017 om 01:49 schreef Sean P. DeNigris:

Again, it seems that just automatically rerunning the test immediately after
a human-manipulated run and setting the color based on that second run
addresses all points on both sides, no?

How many times do you want to restart execution? I have writen enough code where running it twice was not enough to get to green :)

Stephan

Prof. Andrew P. Black

Re: Why does the test runner show red when I correct a test?

In reply to this post by Tim Mackinnon

> On Nov 15, 2017, at 08:30, Richard Sargent <[hidden email]> wrote:
>
> I would say that is an entirely different question

You are right: this is a different scenario. But the suggested fix sounds like a good one to me. After editing a test, even in the debugger, it makes sense to run the test (again?), and display the appropriate colored balloon.

Andrew

Denis Kudriashov

Re: Why does the test runner show red when I correct a test?

In reply to this post by Sean P. DeNigris

2017-11-15 1:49 GMT+01:00 Sean P. DeNigris <[hidden email]>:

Ben Coman wrote
> Or it could go to Amber, half-way between green & red to mean probably
> correct.

Ha ha.

Again, it seems that just automatically rerunning the test immediately after
a human-manipulated run and setting the color based on that second run
addresses all points on both sides, no?

Except that sometimes we are debugging slow test and running it second time automatically after "proceed" can be not appropriate.

We are talking about single test run. If user have any doubts about result It is his responsibility to rerun the test. User knows what he is doing when he debug and fix the test. No intelligence is required here.

And anyway current fix just provides consistent behaviour to debugging from explicit breakpoint/halt. In that case the result was always in sync with debug session.

-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Guillermo Polito

Re: Why does the test runner show red when I correct a test?

And just putting it back to gray? As "not run"?

On Wed, Nov 15, 2017 at 10:44 AM, Denis Kudriashov <[hidden email]> wrote:

2017-11-15 1:49 GMT+01:00 Sean P. DeNigris <[hidden email]>:
Ben Coman wrote
> Or it could go to Amber, half-way between green & red to mean probably
> correct.

Ha ha.

Again, it seems that just automatically rerunning the test immediately after
a human-manipulated run and setting the color based on that second run
addresses all points on both sides, no?

Except that sometimes we are debugging slow test and running it second time automatically after "proceed" can be not appropriate.
We are talking about single test run. If user have any doubts about result It is his responsibility to rerun the test. User knows what he is doing when he debug and fix the test. No intelligence is required here.

And anyway current fix just provides consistent behaviour to debugging from explicit breakpoint/halt. In that case the result was always in sync with debug session.

-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: +33 06 52 70 66 13

Denis Kudriashov

Re: Why does the test runner show red when I correct a test?

2017-11-15 11:00 GMT+01:00 Guillermo Polito <[hidden email]>:

And just putting it back to gray? As "not run"?

We can implement any logic.

Personally I need current behaviour.

On Wed, Nov 15, 2017 at 10:44 AM, Denis Kudriashov <[hidden email]> wrote:
2017-11-15 1:49 GMT+01:00 Sean P. DeNigris <[hidden email]>:
Ben Coman wrote
> Or it could go to Amber, half-way between green & red to mean probably
> correct.

Ha ha.

Again, it seems that just automatically rerunning the test immediately after
a human-manipulated run and setting the color based on that second run
addresses all points on both sides, no?

Except that sometimes we are debugging slow test and running it second time automatically after "proceed" can be not appropriate.
We are talking about single test run. If user have any doubts about result It is his responsibility to rerun the test. User knows what he is doing when he debug and fix the test. No intelligence is required here.

And anyway current fix just provides consistent behaviour to debugging from explicit breakpoint/halt. In that case the result was always in sync with debug session.

-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

--

Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13

Guillermo Polito

Re: Why does the test runner show red when I correct a test?

On Wed, Nov 15, 2017 at 11:06 AM, Denis Kudriashov <[hidden email]> wrote:

2017-11-15 11:00 GMT+01:00 Guillermo Polito <[hidden email]>:
And just putting it back to gray? As "not run"?

We can implement any logic.
Personally I need current behaviour.

But it is not about you personally. It is about implementing the most common and the less strange for newcomers.

On Wed, Nov 15, 2017 at 10:44 AM, Denis Kudriashov <[hidden email]> wrote:
2017-11-15 1:49 GMT+01:00 Sean P. DeNigris <[hidden email]>:
Ben Coman wrote
> Or it could go to Amber, half-way between green & red to mean probably
> correct.

Ha ha.

Again, it seems that just automatically rerunning the test immediately after
a human-manipulated run and setting the color based on that second run
addresses all points on both sides, no?

Except that sometimes we are debugging slow test and running it second time automatically after "proceed" can be not appropriate.
We are talking about single test run. If user have any doubts about result It is his responsibility to rerun the test. User knows what he is doing when he debug and fix the test. No intelligence is required here.

And anyway current fix just provides consistent behaviour to debugging from explicit breakpoint/halt. In that case the result was always in sync with debug session.

-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

--

Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: +33 06 52 70 66 13

Denis Kudriashov

Re: Why does the test runner show red when I correct a test?

2017-11-15 11:08 GMT+01:00 Guillermo Polito <[hidden email]>:

On Wed, Nov 15, 2017 at 11:06 AM, Denis Kudriashov <[hidden email]> wrote:

2017-11-15 11:00 GMT+01:00 Guillermo Polito <[hidden email]>:
And just putting it back to gray? As "not run"?

We can implement any logic.
Personally I need current behaviour.

But it is not about you personally. It is about implementing the most common and the less strange for newcomers.

To know what is the most common case people should tell personal opinion.

And in this thread only Richard was against current logic.

On Wed, Nov 15, 2017 at 10:44 AM, Denis Kudriashov <[hidden email]> wrote:
2017-11-15 1:49 GMT+01:00 Sean P. DeNigris <[hidden email]>:
Ben Coman wrote
> Or it could go to Amber, half-way between green & red to mean probably
> correct.

Ha ha.

Again, it seems that just automatically rerunning the test immediately after
a human-manipulated run and setting the color based on that second run
addresses all points on both sides, no?

Except that sometimes we are debugging slow test and running it second time automatically after "proceed" can be not appropriate.
We are talking about single test run. If user have any doubts about result It is his responsibility to rerun the test. User knows what he is doing when he debug and fix the test. No intelligence is required here.

And anyway current fix just provides consistent behaviour to debugging from explicit breakpoint/halt. In that case the result was always in sync with debug session.

-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

--

Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13

--

Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13

Guillermo Polito

Re: Why does the test runner show red when I correct a test?

On Wed, Nov 15, 2017 at 11:41 AM, Denis Kudriashov <[hidden email]> wrote:

2017-11-15 11:08 GMT+01:00 Guillermo Polito <[hidden email]>:
On Wed, Nov 15, 2017 at 11:06 AM, Denis Kudriashov <[hidden email]> wrote:

2017-11-15 11:00 GMT+01:00 Guillermo Polito <[hidden email]>:
And just putting it back to gray? As "not run"?

We can implement any logic.
Personally I need current behaviour.

But it is not about you personally. It is about implementing the most common and the less strange for newcomers.

To know what is the most common case people should tell personal opinion.
And in this thread only Richard was against current logic.

But you're assuming here that:

- people that is not reading this email do not care and don't have a say

- so pleople that is not subscribed to the mailing list don't care

- and that includes newbies

Our role of experienced guys it not only look after "our" best defaults. But also after the defaults of people without experience.

I think that setting the button back to gray is a good behaviour.

- it is the same thing that happens once you modify a method (which is what is happening during debugging)

- it explicitly says "please rerun the test because you may have introduced side effects"

Unless you make the debugger more intelligent, you cannot be sure that the result you obtained at the end of the test is really reproducible. And moreover, to be able to make such assumption you should be an expert that understands how the underlying framework behaves.

On Wed, Nov 15, 2017 at 10:44 AM, Denis Kudriashov <[hidden email]> wrote:
2017-11-15 1:49 GMT+01:00 Sean P. DeNigris <[hidden email]>:
Ben Coman wrote
> Or it could go to Amber, half-way between green & red to mean probably
> correct.

Ha ha.

Again, it seems that just automatically rerunning the test immediately after
a human-manipulated run and setting the color based on that second run
addresses all points on both sides, no?

Except that sometimes we are debugging slow test and running it second time automatically after "proceed" can be not appropriate.
We are talking about single test run. If user have any doubts about result It is his responsibility to rerun the test. User knows what he is doing when he debug and fix the test. No intelligence is required here.

And anyway current fix just provides consistent behaviour to debugging from explicit breakpoint/halt. In that case the result was always in sync with debug session.

-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

--

Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13

--

Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: +33 06 52 70 66 13

Richard Sargent

Re: Why does the test runner show red when I correct a test?

Administrator

In reply to this post by Denis Kudriashov

And in this thread only Richard was against current logic.

I am against colouring it green without having run the test without interruption or error. I don't think that is the current logic.

On Wed, Nov 15, 2017 at 2:41 AM, Denis Kudriashov <[hidden email]> wrote:

2017-11-15 11:08 GMT+01:00 Guillermo Polito <[hidden email]>:
On Wed, Nov 15, 2017 at 11:06 AM, Denis Kudriashov <[hidden email]> wrote:

2017-11-15 11:00 GMT+01:00 Guillermo Polito <[hidden email]>:
And just putting it back to gray? As "not run"?

We can implement any logic.
Personally I need current behaviour.

But it is not about you personally. It is about implementing the most common and the less strange for newcomers.

To know what is the most common case people should tell personal opinion.
And in this thread only Richard was against current logic.

On Wed, Nov 15, 2017 at 10:44 AM, Denis Kudriashov <[hidden email]> wrote:
2017-11-15 1:49 GMT+01:00 Sean P. DeNigris <[hidden email]>:
Ben Coman wrote
> Or it could go to Amber, half-way between green & red to mean probably
> correct.

Ha ha.

Again, it seems that just automatically rerunning the test immediately after
a human-manipulated run and setting the color based on that second run
addresses all points on both sides, no?

Except that sometimes we are debugging slow test and running it second time automatically after "proceed" can be not appropriate.
We are talking about single test run. If user have any doubts about result It is his responsibility to rerun the test. User knows what he is doing when he debug and fix the test. No intelligence is required here.

And anyway current fix just provides consistent behaviour to debugging from explicit breakpoint/halt. In that case the result was always in sync with debug session.

-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

--

Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13

--

Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13

Richard Sargent

Re: Why does the test runner show red when I correct a test?

Administrator

In reply to this post by Guillermo Polito

I think that setting the button back to gray is a good behaviour.
- it is the same thing that happens once you modify a method (which is what is happening during debugging)
- it explicitly says "please rerun the test because you may have introduced side effects"

I would go a little further. Any method modified by the developer during the course of running a test voids the ability to claim the test succeeded. Likewise, for any object editted in an inspector.

In other words, if the test run was interrupted, it cannot be considered successful. Keep things simple. (as simple as possible)

On Wed, Nov 15, 2017 at 2:49 AM, Guillermo Polito <[hidden email]> wrote:

On Wed, Nov 15, 2017 at 11:41 AM, Denis Kudriashov <[hidden email]> wrote:

2017-11-15 11:08 GMT+01:00 Guillermo Polito <[hidden email]>:
On Wed, Nov 15, 2017 at 11:06 AM, Denis Kudriashov <[hidden email]> wrote:

2017-11-15 11:00 GMT+01:00 Guillermo Polito <[hidden email]>:
And just putting it back to gray? As "not run"?

We can implement any logic.
Personally I need current behaviour.

But it is not about you personally. It is about implementing the most common and the less strange for newcomers.

To know what is the most common case people should tell personal opinion.
And in this thread only Richard was against current logic.

But you're assuming here that:
- people that is not reading this email do not care and don't have a say
- so pleople that is not subscribed to the mailing list don't care
- and that includes newbies

Our role of experienced guys it not only look after "our" best defaults. But also after the defaults of people without experience.

I think that setting the button back to gray is a good behaviour.
- it is the same thing that happens once you modify a method (which is what is happening during debugging)
- it explicitly says "please rerun the test because you may have introduced side effects"

Unless you make the debugger more intelligent, you cannot be sure that the result you obtained at the end of the test is really reproducible. And moreover, to be able to make such assumption you should be an expert that understands how the underlying framework behaves.

On Wed, Nov 15, 2017 at 10:44 AM, Denis Kudriashov <[hidden email]> wrote:
2017-11-15 1:49 GMT+01:00 Sean P. DeNigris <[hidden email]>:
Ben Coman wrote
> Or it could go to Amber, half-way between green & red to mean probably
> correct.

Ha ha.

Again, it seems that just automatically rerunning the test immediately after
a human-manipulated run and setting the color based on that second run
addresses all points on both sides, no?

Except that sometimes we are debugging slow test and running it second time automatically after "proceed" can be not appropriate.
We are talking about single test run. If user have any doubts about result It is his responsibility to rerun the test. User knows what he is doing when he debug and fix the test. No intelligence is required here.

And anyway current fix just provides consistent behaviour to debugging from explicit breakpoint/halt. In that case the result was always in sync with debug session.

-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

--



Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13

--



Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13

--



Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13

Denis Kudriashov

Re: Why does the test runner show red when I correct a test?

In reply to this post by Richard Sargent

2017-11-15 17:16 GMT+01:00 Richard Sargent <[hidden email]>:

And in this thread only Richard was against current logic.

I am against colouring it green without having run the test without interruption or error. I don't think that is the current logic.

Fix is already in Pharo 7. So it is current logic now.

And it was same for manual halt (instead of failure).

On Wed, Nov 15, 2017 at 2:41 AM, Denis Kudriashov <[hidden email]> wrote:

2017-11-15 11:08 GMT+01:00 Guillermo Polito <[hidden email]>:
On Wed, Nov 15, 2017 at 11:06 AM, Denis Kudriashov <[hidden email]> wrote:

2017-11-15 11:00 GMT+01:00 Guillermo Polito <[hidden email]>:
And just putting it back to gray? As "not run"?

We can implement any logic.
Personally I need current behaviour.

But it is not about you personally. It is about implementing the most common and the less strange for newcomers.

To know what is the most common case people should tell personal opinion.
And in this thread only Richard was against current logic.

On Wed, Nov 15, 2017 at 10:44 AM, Denis Kudriashov <[hidden email]> wrote:
2017-11-15 1:49 GMT+01:00 Sean P. DeNigris <[hidden email]>:
Ben Coman wrote
> Or it could go to Amber, half-way between green & red to mean probably
> correct.

Ha ha.

Again, it seems that just automatically rerunning the test immediately after
a human-manipulated run and setting the color based on that second run
addresses all points on both sides, no?

Except that sometimes we are debugging slow test and running it second time automatically after "proceed" can be not appropriate.
We are talking about single test run. If user have any doubts about result It is his responsibility to rerun the test. User knows what he is doing when he debug and fix the test. No intelligence is required here.

And anyway current fix just provides consistent behaviour to debugging from explicit breakpoint/halt. In that case the result was always in sync with debug session.

-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

--

Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13

--

Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13

Sean P. DeNigris

Re: Why does the test runner show red when I correct a test?

Administrator

In reply to this post by Richard Sargent

Richard Sargent wrote
> I would go a little further. Any method modified by the developer during
> the course of running a test voids the ability to claim the test
> succeeded.
> Likewise, for any object editted in an inspector.

That makes sense to me.

-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Cheers,
Sean

123