Why does the test runner show red when I correct a test?
Locked
123
123
Why does the test runner show red when I correct a test?
 
|
Hi - I really like the build in test runner in the Pharo browsers, and I was preparing a talk to show how great TDD is in Pharo and how we aren’t ashamed of our debugger when testing (it augments the experience in fact - letting you poke around and get your thoughts straight).
However - if I pick rerun in the test runner debugger - and step through a test and then correct the failing code, and hit resume - the browser always shows a red failure, even though the execution is now correct. I have to run the test again. This doesn’t seem right to me - are we missing a success event or something? Tim |
Re: Why does the test runner show red when I correct a test?
|
|
Hi Tim. Thank's for reporting. It forces me to fix. I always noticed it but never take it seriously :) 2017-11-09 11:32 GMT+01:00 Tim Mackinnon <[hidden email]>: Hi - I really like the build in test runner in the Pharo browsers, and I was preparing a talk to show how great TDD is in Pharo and how we aren’t ashamed of our debugger when testing (it augments the experience in fact - letting you poke around and get your thoughts straight). |
Re: Why does the test runner show red when I correct a test?
|
|
And now it is in latest Pharo 2017-11-09 12:16 GMT+01:00 Denis Kudriashov <[hidden email]>:
|
Re: Why does the test runner show red when I correct a test?
|
|
Thanks for looking at this - there is an issue however - when you apply that change (at least in a Pharo 6.1 image) - it shows green even when a test fails? So I think its turned one problem into the opposite one.
Unfortunately I haven’t got a chance to look a bit deeper to help - but it might be worth rolling back this change for now. We should fix it though - and the answer must be in the area you have identified. tim
|
Re: Why does the test runner show red when I correct a test?
Administrator
|
I saw this earlier and thought it was wrong to make that change. You cannot assign a colour to the test if it does not run all the way through without interruption. You cannot know whether it truly succeeded or not until you finally run it all the way through without error or interruption. Leave it uncoloured until you (can) know for certain.On Thu, Nov 9, 2017 at 9:29 AM, Tim Mackinnon <[hidden email]> wrote:
|
Re: Why does the test runner show red when I correct a test?
|
|
In reply to this post by Tim Mackinnon
In Pharo 7 it works. 2017-11-09 18:29 GMT+01:00 Tim Mackinnon <[hidden email]>:
|
Re: Why does the test runner show red when I correct a test?
|
|
Thanks Denis - I'll try my simple TDD test case in Pharo 7 (I haven't tried it yet - probably time to). I think it is correct to show green if you fix your problem in the debugger and then proceed and it passes all assertions. But it should equally fail if you proceed and it asserts false or craps out. Strange it would vary in 6.1 - but I'll check. Tim Sent from my iPhone
|
Re: Why does the test runner show red when I correct a test?
|
Administrator
|
I think it is correct to show green if you fix your problem in the debugger and then proceed and it passes all assertions. But it should equally fail if you proceed and it asserts false or craps out. On Thu, Nov 9, 2017 at 2:47 PM, Tim Mackinnon <[hidden email]> wrote:
|
Re: Why does the test runner show red when I correct a test?
|
|
In reply to this post by Tim Mackinnon
And give me concrete steps to see that it is not working (if you will find it) 2017-11-09 23:47 GMT+01:00 Tim Mackinnon <[hidden email]>:
|
Re: Why does the test runner show red when I correct a test?
|
|
In reply to this post by Richard Sargent
Hi Richard
2017-11-09 23:51 GMT+01:00 Richard Sargent <[hidden email]>:
I think it is correct because icon shows the last run state. It is not depends on the way how user manages to pass or fail the test. So user can hack the test from the debugger and make it pass or fail using inspector, scripting or whatever interaction with available objects. When he closes the debugger (by proceed or window close) it is expected to see the result of this hacking in the UI. So test icon should be updated accordingly. Then it is up to user to check how test is working in normal way.
|
Re: Why does the test runner show red when I correct a test?
|
Administrator
|
I think it is correct because icon shows the last run state.My claim is that is not a true statement.
This is my point. You must do this or you cannot know. The first statement should say "the last run state from the point of interruption". Any changes made in the code after the interruption invalidate your knowledge of the correctness of the code leading up to that point. The only reliable conclusion one can make from such an interrupted run is whether it failed again. So, it would be possible to determine that the test should be coloured red, but it is impossible to *reliably* claim that the test should be coloured green. Many people in our industry, our profession have a disturbing tendency to be vague and practice self-deception. Take the concept of a "bug"as an example. They don't "fly into" our code; we *put* them in it. *We* put them in. That's why they should be called errors or defects. Likewise, claiming that a test that failed run successfully and uninterruptedly to completion "succeeded" is a fallacy. In general, the test runner cannot make that claim. It lacks the knowledge to do so. On Fri, Nov 10, 2017 at 1:21 AM, Denis Kudriashov <[hidden email]> wrote:
|
Re: Why does the test runner show red when I correct a test?
|
Administrator
|
Richard Sargent wrote
> The only reliable conclusion one can make from such an interrupted run is > whether it failed again. So, it would be possible to determine that the > test should be coloured red, but it is impossible to *reliably* claim that > the test should be coloured green. What if we ran the test again as if from the browser/runner before setting the icon? ----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
Cheers,
Sean |
Re: Why does the test runner show red when I correct a test?
|
Administrator
|
That would be fine. The point is that, without running the test in its entirety, from start to finish, without interruption, error, or failure, one cannot claim success. On Fri, Nov 10, 2017 at 9:34 AM, Sean P. DeNigris <[hidden email]> wrote: Richard Sargent wrote |
Re: Why does the test runner show red when I correct a test?
|
|
In reply to this post by Richard Sargent
On Fri, Nov 10, 2017 at 08:12 Richard Sargent <[hidden email]> wrote:
+1
|
|
|
Love this: “Many people in our industry, our profession have a disturbing tendency to be vague and practice self-deception. Take the concept of a "bug" as an example. They don't "fly into" our code; we *put* them in it. *We* put them in. That's why they should be called errors or defects.” Or, sometimes, “lazy stupidity” (particularly in my case). And in many others “don’t really know what I’m doing, really, so I copied/pasted a bunch of code off random websites until it “sorta/kinda” worked.” The worst is that I’ve had a number of team leads and architects on projects for whom the last is the most accurate. Andrew |
Re: Why does the test runner show red when I correct atest?
|
Administrator
|
Exactly! And don't let the words I write convince you I am immune to the same failings. Far from it! But, I try. :-)On Fri, Nov 10, 2017 at 10:49 AM, Andrew Glynn <[hidden email]> wrote:
|
Re: Why does the test runner show red when I correct a test?
|
|
In reply to this post by Richard Sargent
My specific usecase is from a pragmatic TDD perspective - failing test, in the debugger you fix the test and press proceed - expecting green. Getting red, and then immediately running again to get red takes away from our story of love coding and loving your debugger - and even Cassie me to mistrust the tools. I get the idea that you can jiffy in the debugger and cause a false pass - but I feel you are penalised for doing it right at the moment. Of course these tests will get run again, but I like the idea that if I did it right, it should recognise this, not incur an extra click and moment of doubt. A button to rerun the whole lot, or automatically rerun, or just run it would work for me. Tim Sent from my iPhone
|
Re: Why does the test runner show red when I correct a test?
|
|
Sorry "immediately running again to get red " should have been "to get green"
Tim Sent from my iPhone > On 10 Nov 2017, at 19:29, Tim Mackinnon <[hidden email]> wrote: > > immediately running again to get red |
Re: Why does the test runner show red when I correct a test?
|
|
In reply to this post by Richard Sargent
On 10-11-17 18:56, Richard Sargent wrote:
> That would be fine. > The point is that, without running the test in its entirety, from start > to finish, without interruption, error, or failure, one cannot claim > success. So after a restart we whould expect green Stephan |
Re: Why does the test runner show red when I correct a test?
|
Administrator
|
In reply to this post by Tim Mackinnon
I hear you and I understand your pain. However, if you corrected the problem, not by a code change, but by playing in the object's inspector or otherwise causing its internal state to change, and then resumed from the debugger, would you still claim the method was successful and should be coloured green?On Fri, Nov 10, 2017 at 11:29 AM, Tim Mackinnon <[hidden email]> wrote:
|
«
Return to Pharo Smalltalk Users
|
1 view|%1 views
| Free forum by Nabble | Edit this page |