Why does the test runner show red when I correct a test?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
47 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Why does the test runner show red when I correct a test?

Tim Mackinnon
Hi - I really like the build in test runner in the Pharo browsers, and I was preparing a talk to show how great TDD is in Pharo and how we aren’t ashamed of our debugger when testing (it augments the experience in fact - letting you poke around and get your thoughts straight).

However - if I pick rerun in the test runner debugger - and step through a test and then correct the failing code, and hit resume - the browser always shows a red failure, even though the execution is now correct. I have to run the test again.

This doesn’t seem right to me - are we missing a success event or something?


Tim
Reply | Threaded
Open this post in threaded view
|

Re: Why does the test runner show red when I correct a test?

Denis Kudriashov
Hi Tim.

Thank's for reporting. It forces me to fix. I always noticed it but never take it seriously :)

2017-11-09 11:32 GMT+01:00 Tim Mackinnon <[hidden email]>:
Hi - I really like the build in test runner in the Pharo browsers, and I was preparing a talk to show how great TDD is in Pharo and how we aren’t ashamed of our debugger when testing (it augments the experience in fact - letting you poke around and get your thoughts straight).

However - if I pick rerun in the test runner debugger - and step through a test and then correct the failing code, and hit resume - the browser always shows a red failure, even though the execution is now correct. I have to run the test again.

This doesn’t seem right to me - are we missing a success event or something?


Tim

Reply | Threaded
Open this post in threaded view
|

Re: Why does the test runner show red when I correct a test?

Denis Kudriashov
And now it is in latest Pharo

2017-11-09 12:16 GMT+01:00 Denis Kudriashov <[hidden email]>:
Hi Tim.

Thank's for reporting. It forces me to fix. I always noticed it but never take it seriously :)

2017-11-09 11:32 GMT+01:00 Tim Mackinnon <[hidden email]>:
Hi - I really like the build in test runner in the Pharo browsers, and I was preparing a talk to show how great TDD is in Pharo and how we aren’t ashamed of our debugger when testing (it augments the experience in fact - letting you poke around and get your thoughts straight).

However - if I pick rerun in the test runner debugger - and step through a test and then correct the failing code, and hit resume - the browser always shows a red failure, even though the execution is now correct. I have to run the test again.

This doesn’t seem right to me - are we missing a success event or something?


Tim


Reply | Threaded
Open this post in threaded view
|

Re: Why does the test runner show red when I correct a test?

Tim Mackinnon
Thanks for looking at this - there is an issue however - when you apply that change (at least in a Pharo 6.1 image) - it shows green even when a test fails? So I think its turned one problem into the opposite one.

Unfortunately I haven’t got a chance to look a bit deeper to help - but it might be worth rolling back this change for now. We should fix it though - and the answer must be in the area you have identified.

tim

On 9 Nov 2017, at 12:43, Denis Kudriashov <[hidden email]> wrote:

And now it is in latest Pharo

2017-11-09 12:16 GMT+01:00 Denis Kudriashov <[hidden email]>:
Hi Tim.

Thank's for reporting. It forces me to fix. I always noticed it but never take it seriously :)

2017-11-09 11:32 GMT+01:00 Tim Mackinnon <[hidden email]>:
Hi - I really like the build in test runner in the Pharo browsers, and I was preparing a talk to show how great TDD is in Pharo and how we aren’t ashamed of our debugger when testing (it augments the experience in fact - letting you poke around and get your thoughts straight).

However - if I pick rerun in the test runner debugger - and step through a test and then correct the failing code, and hit resume - the browser always shows a red failure, even though the execution is now correct. I have to run the test again.

This doesn’t seem right to me - are we missing a success event or something?


Tim



Reply | Threaded
Open this post in threaded view
|

Re: Why does the test runner show red when I correct a test?

Richard Sargent
Administrator
I saw this earlier and thought it was wrong to make that change. You cannot assign a colour to the test if it does not run all the way through without interruption. You cannot know whether it truly succeeded or not until you finally run it all the way through without error or interruption.

Leave it uncoloured until you (can) know for certain.


On Thu, Nov 9, 2017 at 9:29 AM, Tim Mackinnon <[hidden email]> wrote:
Thanks for looking at this - there is an issue however - when you apply that change (at least in a Pharo 6.1 image) - it shows green even when a test fails? So I think its turned one problem into the opposite one.

Unfortunately I haven’t got a chance to look a bit deeper to help - but it might be worth rolling back this change for now. We should fix it though - and the answer must be in the area you have identified.

tim

On 9 Nov 2017, at 12:43, Denis Kudriashov <[hidden email]> wrote:

And now it is in latest Pharo

2017-11-09 12:16 GMT+01:00 Denis Kudriashov <[hidden email]>:
Hi Tim.

Thank's for reporting. It forces me to fix. I always noticed it but never take it seriously :)

2017-11-09 11:32 GMT+01:00 Tim Mackinnon <[hidden email]>:
Hi - I really like the build in test runner in the Pharo browsers, and I was preparing a talk to show how great TDD is in Pharo and how we aren’t ashamed of our debugger when testing (it augments the experience in fact - letting you poke around and get your thoughts straight).

However - if I pick rerun in the test runner debugger - and step through a test and then correct the failing code, and hit resume - the browser always shows a red failure, even though the execution is now correct. I have to run the test again.

This doesn’t seem right to me - are we missing a success event or something?


Tim




Reply | Threaded
Open this post in threaded view
|

Re: Why does the test runner show red when I correct a test?

Denis Kudriashov
In reply to this post by Tim Mackinnon
In Pharo 7 it works.

2017-11-09 18:29 GMT+01:00 Tim Mackinnon <[hidden email]>:
Thanks for looking at this - there is an issue however - when you apply that change (at least in a Pharo 6.1 image) - it shows green even when a test fails? So I think its turned one problem into the opposite one.

Unfortunately I haven’t got a chance to look a bit deeper to help - but it might be worth rolling back this change for now. We should fix it though - and the answer must be in the area you have identified.

tim

On 9 Nov 2017, at 12:43, Denis Kudriashov <[hidden email]> wrote:

And now it is in latest Pharo

2017-11-09 12:16 GMT+01:00 Denis Kudriashov <[hidden email]>:
Hi Tim.

Thank's for reporting. It forces me to fix. I always noticed it but never take it seriously :)

2017-11-09 11:32 GMT+01:00 Tim Mackinnon <[hidden email]>:
Hi - I really like the build in test runner in the Pharo browsers, and I was preparing a talk to show how great TDD is in Pharo and how we aren’t ashamed of our debugger when testing (it augments the experience in fact - letting you poke around and get your thoughts straight).

However - if I pick rerun in the test runner debugger - and step through a test and then correct the failing code, and hit resume - the browser always shows a red failure, even though the execution is now correct. I have to run the test again.

This doesn’t seem right to me - are we missing a success event or something?


Tim




Reply | Threaded
Open this post in threaded view
|

Re: Why does the test runner show red when I correct a test?

Tim Mackinnon
Thanks Denis - I'll try my simple TDD test case in Pharo 7 (I haven't tried it yet - probably time to).

I think it is correct to show green if you fix your problem in the debugger and then proceed and  it passes all assertions. But it should equally fail if you proceed and it asserts false or craps out.

Strange it would vary in 6.1 - but I'll check.

Tim

Sent from my iPhone



Sent from my iPhone
On 9 Nov 2017, at 19:54, Denis Kudriashov <[hidden email]> wrote:

In Pharo 7 it works.

2017-11-09 18:29 GMT+01:00 Tim Mackinnon <[hidden email]>:
Thanks for looking at this - there is an issue however - when you apply that change (at least in a Pharo 6.1 image) - it shows green even when a test fails? So I think its turned one problem into the opposite one.

Unfortunately I haven’t got a chance to look a bit deeper to help - but it might be worth rolling back this change for now. We should fix it though - and the answer must be in the area you have identified.

tim

On 9 Nov 2017, at 12:43, Denis Kudriashov <[hidden email]> wrote:

And now it is in latest Pharo

2017-11-09 12:16 GMT+01:00 Denis Kudriashov <[hidden email]>:
Hi Tim.

Thank's for reporting. It forces me to fix. I always noticed it but never take it seriously :)

2017-11-09 11:32 GMT+01:00 Tim Mackinnon <[hidden email]>:
Hi - I really like the build in test runner in the Pharo browsers, and I was preparing a talk to show how great TDD is in Pharo and how we aren’t ashamed of our debugger when testing (it augments the experience in fact - letting you poke around and get your thoughts straight).

However - if I pick rerun in the test runner debugger - and step through a test and then correct the failing code, and hit resume - the browser always shows a red failure, even though the execution is now correct. I have to run the test again.

This doesn’t seem right to me - are we missing a success event or something?


Tim




Reply | Threaded
Open this post in threaded view
|

Re: Why does the test runner show red when I correct a test?

Richard Sargent
Administrator
I think it is correct to show green if you fix your problem in the debugger and then proceed and  it passes all assertions. But it should equally fail if you proceed and it asserts false or craps out.

It would be a lie. The truth is only that from that point on, it is correct. You have no guarantee that your correction hasn't broken something that would occur in the next run prior to the point at which you applied the correction.

Hence my assertion that you cannot determine the colour until the next run.


On Thu, Nov 9, 2017 at 2:47 PM, Tim Mackinnon <[hidden email]> wrote:
Thanks Denis - I'll try my simple TDD test case in Pharo 7 (I haven't tried it yet - probably time to).

I think it is correct to show green if you fix your problem in the debugger and then proceed and  it passes all assertions. But it should equally fail if you proceed and it asserts false or craps out.

Strange it would vary in 6.1 - but I'll check.

Tim

Sent from my iPhone



Sent from my iPhone
On 9 Nov 2017, at 19:54, Denis Kudriashov <[hidden email]> wrote:

In Pharo 7 it works.

2017-11-09 18:29 GMT+01:00 Tim Mackinnon <[hidden email]>:
Thanks for looking at this - there is an issue however - when you apply that change (at least in a Pharo 6.1 image) - it shows green even when a test fails? So I think its turned one problem into the opposite one.

Unfortunately I haven’t got a chance to look a bit deeper to help - but it might be worth rolling back this change for now. We should fix it though - and the answer must be in the area you have identified.

tim

On 9 Nov 2017, at 12:43, Denis Kudriashov <[hidden email]> wrote:

And now it is in latest Pharo

2017-11-09 12:16 GMT+01:00 Denis Kudriashov <[hidden email]>:
Hi Tim.

Thank's for reporting. It forces me to fix. I always noticed it but never take it seriously :)

2017-11-09 11:32 GMT+01:00 Tim Mackinnon <[hidden email]>:
Hi - I really like the build in test runner in the Pharo browsers, and I was preparing a talk to show how great TDD is in Pharo and how we aren’t ashamed of our debugger when testing (it augments the experience in fact - letting you poke around and get your thoughts straight).

However - if I pick rerun in the test runner debugger - and step through a test and then correct the failing code, and hit resume - the browser always shows a red failure, even though the execution is now correct. I have to run the test again.

This doesn’t seem right to me - are we missing a success event or something?


Tim





Reply | Threaded
Open this post in threaded view
|

Re: Why does the test runner show red when I correct a test?

Denis Kudriashov
In reply to this post by Tim Mackinnon
And give me concrete steps to see that it is not working (if you will find it) 

2017-11-09 23:47 GMT+01:00 Tim Mackinnon <[hidden email]>:
Thanks Denis - I'll try my simple TDD test case in Pharo 7 (I haven't tried it yet - probably time to).

I think it is correct to show green if you fix your problem in the debugger and then proceed and  it passes all assertions. But it should equally fail if you proceed and it asserts false or craps out.

Strange it would vary in 6.1 - but I'll check.

Tim

Sent from my iPhone



Sent from my iPhone
On 9 Nov 2017, at 19:54, Denis Kudriashov <[hidden email]> wrote:

In Pharo 7 it works.

2017-11-09 18:29 GMT+01:00 Tim Mackinnon <[hidden email]>:
Thanks for looking at this - there is an issue however - when you apply that change (at least in a Pharo 6.1 image) - it shows green even when a test fails? So I think its turned one problem into the opposite one.

Unfortunately I haven’t got a chance to look a bit deeper to help - but it might be worth rolling back this change for now. We should fix it though - and the answer must be in the area you have identified.

tim

On 9 Nov 2017, at 12:43, Denis Kudriashov <[hidden email]> wrote:

And now it is in latest Pharo

2017-11-09 12:16 GMT+01:00 Denis Kudriashov <[hidden email]>:
Hi Tim.

Thank's for reporting. It forces me to fix. I always noticed it but never take it seriously :)

2017-11-09 11:32 GMT+01:00 Tim Mackinnon <[hidden email]>:
Hi - I really like the build in test runner in the Pharo browsers, and I was preparing a talk to show how great TDD is in Pharo and how we aren’t ashamed of our debugger when testing (it augments the experience in fact - letting you poke around and get your thoughts straight).

However - if I pick rerun in the test runner debugger - and step through a test and then correct the failing code, and hit resume - the browser always shows a red failure, even though the execution is now correct. I have to run the test again.

This doesn’t seem right to me - are we missing a success event or something?


Tim





Reply | Threaded
Open this post in threaded view
|

Re: Why does the test runner show red when I correct a test?

Denis Kudriashov
In reply to this post by Richard Sargent
Hi Richard

2017-11-09 23:51 GMT+01:00 Richard Sargent <[hidden email]>:
I think it is correct to show green if you fix your problem in the debugger and then proceed and  it passes all assertions. But it should equally fail if you proceed and it asserts false or craps out.

It would be a lie. The truth is only that from that point on, it is correct. You have no guarantee that your correction hasn't broken something that would occur in the next run prior to the point at which you applied the correction.

I think it is correct because icon shows the last run state. It is not depends on the way how user manages to pass or fail the test. 
So user can hack the test from the debugger and make it pass or fail using inspector, scripting or whatever interaction with available objects. When he closes the debugger (by proceed or window close) it is expected to see the result of this hacking in the UI. So test icon should be updated accordingly. 
Then it is up to user to check how test is working in normal way.
 

Hence my assertion that you cannot determine the colour until the next run.

 


On Thu, Nov 9, 2017 at 2:47 PM, Tim Mackinnon <[hidden email]> wrote:
Thanks Denis - I'll try my simple TDD test case in Pharo 7 (I haven't tried it yet - probably time to).

I think it is correct to show green if you fix your problem in the debugger and then proceed and  it passes all assertions. But it should equally fail if you proceed and it asserts false or craps out.

Strange it would vary in 6.1 - but I'll check.

Tim

Sent from my iPhone



Sent from my iPhone
On 9 Nov 2017, at 19:54, Denis Kudriashov <[hidden email]> wrote:

In Pharo 7 it works.

2017-11-09 18:29 GMT+01:00 Tim Mackinnon <[hidden email]>:
Thanks for looking at this - there is an issue however - when you apply that change (at least in a Pharo 6.1 image) - it shows green even when a test fails? So I think its turned one problem into the opposite one.

Unfortunately I haven’t got a chance to look a bit deeper to help - but it might be worth rolling back this change for now. We should fix it though - and the answer must be in the area you have identified.

tim

On 9 Nov 2017, at 12:43, Denis Kudriashov <[hidden email]> wrote:

And now it is in latest Pharo

2017-11-09 12:16 GMT+01:00 Denis Kudriashov <[hidden email]>:
Hi Tim.

Thank's for reporting. It forces me to fix. I always noticed it but never take it seriously :)

2017-11-09 11:32 GMT+01:00 Tim Mackinnon <[hidden email]>:
Hi - I really like the build in test runner in the Pharo browsers, and I was preparing a talk to show how great TDD is in Pharo and how we aren’t ashamed of our debugger when testing (it augments the experience in fact - letting you poke around and get your thoughts straight).

However - if I pick rerun in the test runner debugger - and step through a test and then correct the failing code, and hit resume - the browser always shows a red failure, even though the execution is now correct. I have to run the test again.

This doesn’t seem right to me - are we missing a success event or something?


Tim






Reply | Threaded
Open this post in threaded view
|

Re: Why does the test runner show red when I correct a test?

Richard Sargent
Administrator
I think it is correct because icon shows the last run state.
My claim is that is not a true statement.

Then it is up to user to check how test is working in normal way.
This is my point. You must do this or you cannot know.

The first statement should say "the last run state from the point of interruption". Any changes made in the code after the interruption invalidate your knowledge of the correctness of the code leading up to that point.

The only reliable conclusion one can make from such an interrupted run is whether it failed again. So, it would be possible to determine that the test should be coloured red, but it is impossible to *reliably* claim that the test should be coloured green.


Many people in our industry, our profession have a disturbing tendency to be vague and practice self-deception. Take the concept of a "bug"as an example. They don't "fly into" our code; we *put* them in it. *We* put them in. That's why they should be called errors or defects.

Likewise, claiming that a test that failed run successfully and uninterruptedly to completion "succeeded" is a fallacy. In general, the test runner cannot make that claim. It lacks the knowledge to do so.

On Fri, Nov 10, 2017 at 1:21 AM, Denis Kudriashov <[hidden email]> wrote:
Hi Richard

2017-11-09 23:51 GMT+01:00 Richard Sargent <[hidden email]>:
I think it is correct to show green if you fix your problem in the debugger and then proceed and  it passes all assertions. But it should equally fail if you proceed and it asserts false or craps out.

It would be a lie. The truth is only that from that point on, it is correct. You have no guarantee that your correction hasn't broken something that would occur in the next run prior to the point at which you applied the correction.

I think it is correct because icon shows the last run state. It is not depends on the way how user manages to pass or fail the test. 
So user can hack the test from the debugger and make it pass or fail using inspector, scripting or whatever interaction with available objects. When he closes the debugger (by proceed or window close) it is expected to see the result of this hacking in the UI. So test icon should be updated accordingly. 
Then it is up to user to check how test is working in normal way.
 

Hence my assertion that you cannot determine the colour until the next run.

 


On Thu, Nov 9, 2017 at 2:47 PM, Tim Mackinnon <[hidden email]> wrote:
Thanks Denis - I'll try my simple TDD test case in Pharo 7 (I haven't tried it yet - probably time to).

I think it is correct to show green if you fix your problem in the debugger and then proceed and  it passes all assertions. But it should equally fail if you proceed and it asserts false or craps out.

Strange it would vary in 6.1 - but I'll check.

Tim

Sent from my iPhone



Sent from my iPhone
On 9 Nov 2017, at 19:54, Denis Kudriashov <[hidden email]> wrote:

In Pharo 7 it works.

2017-11-09 18:29 GMT+01:00 Tim Mackinnon <[hidden email]>:
Thanks for looking at this - there is an issue however - when you apply that change (at least in a Pharo 6.1 image) - it shows green even when a test fails? So I think its turned one problem into the opposite one.

Unfortunately I haven’t got a chance to look a bit deeper to help - but it might be worth rolling back this change for now. We should fix it though - and the answer must be in the area you have identified.

tim

On 9 Nov 2017, at 12:43, Denis Kudriashov <[hidden email]> wrote:

And now it is in latest Pharo

2017-11-09 12:16 GMT+01:00 Denis Kudriashov <[hidden email]>:
Hi Tim.

Thank's for reporting. It forces me to fix. I always noticed it but never take it seriously :)

2017-11-09 11:32 GMT+01:00 Tim Mackinnon <[hidden email]>:
Hi - I really like the build in test runner in the Pharo browsers, and I was preparing a talk to show how great TDD is in Pharo and how we aren’t ashamed of our debugger when testing (it augments the experience in fact - letting you poke around and get your thoughts straight).

However - if I pick rerun in the test runner debugger - and step through a test and then correct the failing code, and hit resume - the browser always shows a red failure, even though the execution is now correct. I have to run the test again.

This doesn’t seem right to me - are we missing a success event or something?


Tim







Reply | Threaded
Open this post in threaded view
|

Re: Why does the test runner show red when I correct a test?

Sean P. DeNigris
Administrator
Richard Sargent wrote
> The only reliable conclusion one can make from such an interrupted run is
> whether it failed again. So, it would be possible to determine that the
> test should be coloured red, but it is impossible to *reliably* claim that
> the test should be coloured green.

What if we ran the test again as if from the browser/runner before setting
the icon?



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: Why does the test runner show red when I correct a test?

Richard Sargent
Administrator
That would be fine.
The point is that, without running the test in its entirety, from start to finish, without interruption, error, or failure, one cannot claim success.

On Fri, Nov 10, 2017 at 9:34 AM, Sean P. DeNigris <[hidden email]> wrote:
Richard Sargent wrote
> The only reliable conclusion one can make from such an interrupted run is
> whether it failed again. So, it would be possible to determine that the
> test should be coloured red, but it is impossible to *reliably* claim that
> the test should be coloured green.

What if we ran the test again as if from the browser/runner before setting
the icon?



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html


Reply | Threaded
Open this post in threaded view
|

Re: Why does the test runner show red when I correct a test?

Kjell Godo
In reply to this post by Richard Sargent

On Fri, Nov 10, 2017 at 08:12 Richard Sargent <[hidden email]> wrote:
I think it is correct because icon shows the last run state.
My claim is that is not a true statement.


Then it is up to user to check how test is working in normal way.
This is my point. You must do this or you cannot know.

The first statement should say "the last run state from the point of interruption". Any changes made in the code after the interruption invalidate your knowledge of the correctness of the code leading up to that point.

The only reliable conclusion one can make from such an interrupted run is whether it failed again. So, it would be possible to determine that the test should be coloured red, but it is impossible to *reliably* claim that the test should be coloured green.

+1



Many people in our industry, our profession have a disturbing tendency to be vague and practice self-deception. Take the concept of a "bug"as an example. They don't "fly into" our code; we *put* them in it. *We* put them in. That's why they should be called errors or defects.

Likewise, claiming that a test that failed run successfully and uninterruptedly to completion "succeeded" is a fallacy. In general, the test runner cannot make that claim. It lacks the knowledge to do so.

On Fri, Nov 10, 2017 at 1:21 AM, Denis Kudriashov <[hidden email]> wrote:
Hi Richard

2017-11-09 23:51 GMT+01:00 Richard Sargent <[hidden email]>:
I think it is correct to show green if you fix your problem in the debugger and then proceed and  it passes all assertions. But it should equally fail if you proceed and it asserts false or craps out.

It would be a lie. The truth is only that from that point on, it is correct. You have no guarantee that your correction hasn't broken something that would occur in the next run prior to the point at which you applied the correction.

I think it is correct because icon shows the last run state. It is not depends on the way how user manages to pass or fail the test. 
So user can hack the test from the debugger and make it pass or fail using inspector, scripting or whatever interaction with available objects. When he closes the debugger (by proceed or window close) it is expected to see the result of this hacking in the UI. So test icon should be updated accordingly. 
Then it is up to user to check how test is working in normal way.
 

Hence my assertion that you cannot determine the colour until the next run.

 


On Thu, Nov 9, 2017 at 2:47 PM, Tim Mackinnon <[hidden email]> wrote:
Thanks Denis - I'll try my simple TDD test case in Pharo 7 (I haven't tried it yet - probably time to).

I think it is correct to show green if you fix your problem in the debugger and then proceed and  it passes all assertions. But it should equally fail if you proceed and it asserts false or craps out.

Strange it would vary in 6.1 - but I'll check.

Tim

Sent from my iPhone



Sent from my iPhone
On 9 Nov 2017, at 19:54, Denis Kudriashov <[hidden email]> wrote:

In Pharo 7 it works.

2017-11-09 18:29 GMT+01:00 Tim Mackinnon <[hidden email]>:
Thanks for looking at this - there is an issue however - when you apply that change (at least in a Pharo 6.1 image) - it shows green even when a test fails? So I think its turned one problem into the opposite one.

Unfortunately I haven’t got a chance to look a bit deeper to help - but it might be worth rolling back this change for now. We should fix it though - and the answer must be in the area you have identified.

tim

On 9 Nov 2017, at 12:43, Denis Kudriashov <[hidden email]> wrote:

And now it is in latest Pharo

2017-11-09 12:16 GMT+01:00 Denis Kudriashov <[hidden email]>:
Hi Tim.

Thank's for reporting. It forces me to fix. I always noticed it but never take it seriously :)

2017-11-09 11:32 GMT+01:00 Tim Mackinnon <[hidden email]>:
Hi - I really like the build in test runner in the Pharo browsers, and I was preparing a talk to show how great TDD is in Pharo and how we aren’t ashamed of our debugger when testing (it augments the experience in fact - letting you poke around and get your thoughts straight).

However - if I pick rerun in the test runner debugger - and step through a test and then correct the failing code, and hit resume - the browser always shows a red failure, even though the execution is now correct. I have to run the test again.

This doesn’t seem right to me - are we missing a success event or something?


Tim







Reply | Threaded
Open this post in threaded view
|

Re: Why does the test runner show red when I correct atest?

aglynn42

Love this:

 

“Many people in our industry, our profession have a disturbing tendency to be vague and practice self-deception. Take the concept of a "bug" as an example. They don't "fly into" our code; we *put* them in it. *We* put them in. That's why they should be called errors or defects.”

 

Or, sometimes, “lazy stupidity” (particularly in my case).  And in many others “don’t really know what I’m doing, really, so I copied/pasted a bunch of code off random websites until it “sorta/kinda” worked.”

 

The worst is that I’ve had a number of team leads and architects on projects for whom the last is the most accurate.

 

Andrew

Reply | Threaded
Open this post in threaded view
|

Re: Why does the test runner show red when I correct atest?

Richard Sargent
Administrator
Exactly!

And don't let the words I write convince you I am immune to the same failings. Far from it! But, I try. :-)


On Fri, Nov 10, 2017 at 10:49 AM, Andrew Glynn <[hidden email]> wrote:

Love this:

 

“Many people in our industry, our profession have a disturbing tendency to be vague and practice self-deception. Take the concept of a "bug" as an example. They don't "fly into" our code; we *put* them in it. *We* put them in. That's why they should be called errors or defects.”

 

Or, sometimes, “lazy stupidity” (particularly in my case).  And in many others “don’t really know what I’m doing, really, so I copied/pasted a bunch of code off random websites until it “sorta/kinda” worked.”

 

The worst is that I’ve had a number of team leads and architects on projects for whom the last is the most accurate.

 

Andrew


Reply | Threaded
Open this post in threaded view
|

Re: Why does the test runner show red when I correct a test?

Tim Mackinnon
In reply to this post by Richard Sargent
My specific usecase is from a pragmatic TDD perspective - failing test, in the debugger you fix the test and press proceed - expecting green. Getting red, and then immediately running again to get red takes away from our story of love coding and loving your debugger - and even Cassie me to mistrust the tools.

I get the idea that you can jiffy in the debugger and cause a false pass - but I feel you are penalised for doing it right at the moment.

Of course these tests will get run again, but I like the idea that if I did it right, it should recognise this, not incur an extra click and moment of doubt.

A button to rerun the whole lot, or automatically rerun, or just run it would work for me.

Tim

Sent from my iPhone

On 10 Nov 2017, at 17:56, Richard Sargent <[hidden email]> wrote:

That would be fine.
The point is that, without running the test in its entirety, from start to finish, without interruption, error, or failure, one cannot claim success.

On Fri, Nov 10, 2017 at 9:34 AM, Sean P. DeNigris <[hidden email]> wrote:
Richard Sargent wrote
> The only reliable conclusion one can make from such an interrupted run is
> whether it failed again. So, it would be possible to determine that the
> test should be coloured red, but it is impossible to *reliably* claim that
> the test should be coloured green.

What if we ran the test again as if from the browser/runner before setting
the icon?



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html


Reply | Threaded
Open this post in threaded view
|

Re: Why does the test runner show red when I correct a test?

Tim Mackinnon
Sorry "immediately running again to get red " should have been "to get green"

Tim

Sent from my iPhone

> On 10 Nov 2017, at 19:29, Tim Mackinnon <[hidden email]> wrote:
>
> immediately running again to get red

Reply | Threaded
Open this post in threaded view
|

Re: Why does the test runner show red when I correct a test?

Stephan Eggermont-3
In reply to this post by Richard Sargent
On 10-11-17 18:56, Richard Sargent wrote:
> That would be fine.
> The point is that, without running the test in its entirety, from start
> to finish, without interruption, error, or failure, one cannot claim
> success.

So after a restart we whould expect green

Stephan


Reply | Threaded
Open this post in threaded view
|

Re: Why does the test runner show red when I correct a test?

Richard Sargent
Administrator
In reply to this post by Tim Mackinnon
I hear you and I understand your pain.

However, if you corrected the problem, not by a code change, but by playing in the object's inspector or otherwise causing its internal state to change, and then resumed from the debugger, would you still claim the method was successful and should be coloured green?

The only thing we can claim is that there were or were not further errors in the test. Colour it red if there were, but you cannot honestly colour it green. The code doesn't actually work.


On Fri, Nov 10, 2017 at 11:29 AM, Tim Mackinnon <[hidden email]> wrote:
My specific usecase is from a pragmatic TDD perspective - failing test, in the debugger you fix the test and press proceed - expecting green. Getting red, and then immediately running again to get red takes away from our story of love coding and loving your debugger - and even Cassie me to mistrust the tools.

I get the idea that you can jiffy in the debugger and cause a false pass - but I feel you are penalised for doing it right at the moment.

Of course these tests will get run again, but I like the idea that if I did it right, it should recognise this, not incur an extra click and moment of doubt.

A button to rerun the whole lot, or automatically rerun, or just run it would work for me.

Tim

Sent from my iPhone

On 10 Nov 2017, at 17:56, Richard Sargent <[hidden email]> wrote:

That would be fine.
The point is that, without running the test in its entirety, from start to finish, without interruption, error, or failure, one cannot claim success.

On Fri, Nov 10, 2017 at 9:34 AM, Sean P. DeNigris <[hidden email]> wrote:
Richard Sargent wrote
> The only reliable conclusion one can make from such an interrupted run is
> whether it failed again. So, it would be possible to determine that the
> test should be coloured red, but it is impossible to *reliably* claim that
> the test should be coloured green.

What if we ran the test again as if from the browser/runner before setting
the icon?



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html



123