Good reference on time on unit testing?
Locked
12
12
Good reference on time on unit testing?
 
|
Hello all,
Can anyone recommend a good reference on the amount of time one should expect to spend writing tests? I will have to be the messenger (will be wearing running shoes just in case...), but I want the message to come from a solid source. Bill |
Re: Good reference on time on unit testing?
|
|
On Sun, Feb 27, 2011 at 4:52 AM, Schwab,Wilhelm K <[hidden email]> wrote: Hello all, If you develop the TDD way (which I always do at work) then I would say this question is not valid. Your production code will be there because of tests. In legacy system, new and refactored code will be there because of tests.
IMHO the questions "how much time should I spend writing tests?" or "how many tests should I write ?" are raised when tests are written after. And sadly tests often become a burden in this case and doesn't support the pressure of releases / last minute changes / ... (I've done it this way almost 10 years ago, not good :)
In TDD the question is "which stories / scenarios do we have to implement ?". It's more a Product owner / customer point of view. The best book on TDD I've read is Dave Astels TDD: a practical guide (Indeed, this one has really changed my mind on how testing should be done, and other books I've read since don't).
Laurent
|
Re: Good reference on time on unit testing?
|
|
+100 Couldn't have said it better myself! -- Cheers, Peter.
|
Re: Good reference on time on unit testing?
|
|
In reply to this post by Schwab,Wilhelm K
Hi,
On 27.02.2011, at 04:52, Schwab,Wilhelm K wrote: > Hello all, > > Can anyone recommend a good reference on the amount of time one should expect to spend writing tests? I will have to be the messenger (will be wearing running shoes just in case...), but I want the message to come from a solid source. I find that really hard to answer. To me the problem is the question itself. I heard the "..amount of time one should expect to spend writing tests?" so often in companies and I think it was always exactly this phrase. While the question is valid it gives the impression there is something that decucts time from your "normal" development work. So the people that are asking this question are often managing people that have read something about code quality and they want to apply _this_ to their teams. The definition over time is troublesome, too. Testing is not easy. Everything you read about testing gives you the impression that everyone knows how to test and that those developers are just too lazy. And that is not true. Most developers I met had problems to see what testing is all about. The don't see interfaces as provable promises etc. So if you tell any of these developers they should spend 1 hour a day in testing than you will get tests that are counter-productive. My favorite example is the one where you have any composite object with an add method. Than the test goes like adding something via the interface and then try to access the internal array and check if it is of size 1. To me it is the same as with documentation. I prefer to have less documentation than useless documentation. So every developer is testing in some way. You either test and debug on the way in an unstructured form or you write tests. To me writing tests is not an add-on it is a change in working style. From this point of view I would state that the time I need to spend _additionally_ for testing is negative. I grew up with a print statement being the ultimate debugging tool. A print/debugging statement is added at the time of debugging and probably removed if the error seems to be fixed. That can lead to a situation where you do this multiple times. Writing the same as a test (and that forces you to produce more fine grained code) you will have a saving in time and a little assurance about regression. Regression debugging (debugging it again later) is much more time spent because you have to fix the error _and_ you need to focus again on that problem (which takes most of the time). So the point in writing tests is not to spend time but to save time. The amount of time one should expect to spend writing tests is less than the time you need to spend for testing otherwise. :) Norbert |
Re: Good reference on time on unit testing?
|
|
Norbert,
Excellent points - I take exception with only one: you assume that all developers test - that is sadly not true. I am involved with a group who seem to think that a handful of tests added at the last minute will somehow magically fix their problems. I have no problem arguing that testing (if done well) can/will reduce overall development time; the question is how much of that time one should expect to devote to writing and maintaining unit and acceptance tests? Bill ________________________________________ From: [hidden email] [[hidden email]] On Behalf Of Norbert Hartl [[hidden email]] Sent: Sunday, February 27, 2011 5:41 AM To: [hidden email] Subject: Re: [Pharo-project] Good reference on time on unit testing? Hi, On 27.02.2011, at 04:52, Schwab,Wilhelm K wrote: > Hello all, > > Can anyone recommend a good reference on the amount of time one should expect to spend writing tests? I will have to be the messenger (will be wearing running shoes just in case...), but I want the message to come from a solid source. I find that really hard to answer. To me the problem is the question itself. I heard the "..amount of time one should expect to spend writing tests?" so often in companies and I think it was always exactly this phrase. While the question is valid it gives the impression there is something that decucts time from your "normal" development work. So the people that are asking this question are often managing people that have read something about code quality and they want to apply _this_ to their teams. The definition over time is troublesome, too. Testing is not easy. Everything you read about testing gives you the impression that everyone knows how to test and that those developers are just too lazy. And that is not true. Most developers I met had problems to see what testing is all about. The don't see interfaces as provable promises etc. So if you tell any of these developers they should spend 1 hour a day in testing than you will get tests that are counter-productive. My favorite example is the one where you have any composite object with an add method. Than the test goes like adding something via the interface and then try to access the internal array and check if it is of size 1. To me it is the same as with documentation. I prefer to have less documentation than useless documentation. So every developer is testing in some way. You either test and debug on the way in an unstructured form or you write tests. To me writing tests is not an add-on it is a change in working style. From this point of view I would state that the time I need to spend _additionally_ for testing is negative. I grew up with a print statement being the ultimate debugging tool. A print/debugging statement is added at the time of debugging and probably removed if the error seems to be fixed. That can lead to a situation where you do this multiple times. Writing the same as a test (and that forces you to produce more fine grained code) you will have a saving in time and a little assurance about regression. Regression debugging (debugging it again later) is much more time spent because you have to fix the error _and_ you need to focus again on that problem (which takes most of the time). So the point in writing tests is not to spend time but to save time. The amount of time one should expect to spend writing tests is less than the time you need to spend for testing otherwise. :) Norbert |
Re: Good reference on time on unit testing?
|
|
In reply to this post by Peter Hugosson-Miller
Understood. But, doing it correctly, how much of your time do you spend on the test writing/maintenance side? I found some blog posts, but I was hoping for some peer-reviewed papers that go on the record.
I am trying to make the point you are making, and it will go over better if I can point to literature and some numbers that I suspect will be seen as sobering. Tests ARE being "written after" in this case, but assuming they change to a proper process, they still need to understand that tests take thought and work. ________________________________________ From: [hidden email] [[hidden email]] On Behalf Of Peter Hugosson-Miller [[hidden email]] Sent: Sunday, February 27, 2011 2:58 AM To: [hidden email] Subject: Re: [Pharo-project] Good reference on time on unit testing? +100 Couldn't have said it better myself! -- Cheers, Peter. On 27 feb 2011, at 08:27, laurent laffont <[hidden email]<mailto:[hidden email]>> wrote: On Sun, Feb 27, 2011 at 4:52 AM, Schwab,Wilhelm K <<mailto:[hidden email]>[hidden email]<mailto:[hidden email]>> wrote: Hello all, Can anyone recommend a good reference on the amount of time one should expect to spend writing tests? I will have to be the messenger (will be wearing running shoes just in case...), but I want the message to come from a solid source. If you develop the TDD way (which I always do at work) then I would say this question is not valid. Your production code will be there because of tests. In legacy system, new and refactored code will be there because of tests. IMHO the questions "how much time should I spend writing tests?" or "how many tests should I write ?" are raised when tests are written after. And sadly tests often become a burden in this case and doesn't support the pressure of releases / last minute changes / ... (I've done it this way almost 10 years ago, not good :) In TDD the question is "which stories / scenarios do we have to implement ?". It's more a Product owner / customer point of view. The best book on TDD I've read is Dave Astels TDD: a practical guide (Indeed, this one has really changed my mind on how testing should be done, and other books I've read since don't). Laurent Bill |
|
|
In reply to this post by Schwab,Wilhelm K
Robert Martin said in his book that one should spend ~50% of its time writing tests.
Cheers, Alexandre On 27 Feb 2011, at 00:52, Schwab,Wilhelm K wrote: > Hello all, > > Can anyone recommend a good reference on the amount of time one should expect to spend writing tests? I will have to be the messenger (will be wearing running shoes just in case...), but I want the message to come from a solid source. > > Bill > > > > -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. |
Re: Good reference on time on unit testing?
|
|
In reply to this post by Schwab,Wilhelm K
On Sun, Feb 27, 2011 at 2:12 PM, Schwab,Wilhelm K <[hidden email]> wrote: Understood. But, doing it correctly, how much of your time do you spend on the test writing/maintenance side? None. Seriously none. I never pause and say "OK now it's time to write tests and maintain existing ones". It's part of the TDD cycle: write a test, make it green, refactor. write a test, make it green, refactor. The cycle can go from 1mn to 10mn, rarely more. And if you use the Pomodoro technique, you feel it even better. Test writing/maintenance is part of programming.
Sometimes you can say "OK before implementing this new story, I would like to refactor here and here (tests + code) because I don't like it". We also do this stuff on legacy code when tests are not here. But it's far easier when we have good test coverage.
I found some blog posts, but I was hoping for some peer-reviewed papers that go on the record. When tests are written after, it's another kind of story. Then "how much time we should spend writing test" is valid. I don't have any answer.
TDD is not easy. Learning to write good tests takes time (years), investment. If someone in your team have good experience with it, he should pair program continuously with other programmers. Participating in coding-dojo , code retreats with experienced developers is great too.
Laurent.
|
Re: Good reference on time on unit testing?
|
|
In reply to this post by Schwab,Wilhelm K
On 27/02/11 7:58 AM, Schwab,Wilhelm K wrote:
> > I have no problem arguing that testing (if done well) can/will reduce > overall development time; the question is how much of that time one > should expect to devote to writing and maintaining unit and > acceptance tests? You've asked the question for the third time, so all the TDD explanations seem to have missed the mark. How about asking instead: how much test code do people write? Maybe this way, both the TDD and the "how much time" viewpoints can be accommodated (because volume of test code is roughly proportional to time to write test code). -- Yanni Chiu |
Re: Good reference on time on unit testing?
|
|
In reply to this post by Schwab,Wilhelm K
On 02/27/2011 06:12 AM, Schwab,Wilhelm K wrote:
> Understood. But, doing it correctly, how much of your time do you spend on the test writing/maintenance side? Far less time than you'll spend manually testing over the course of writing and maintaining the program. You can't avoid testing, nor the time spent doing it, the only question is do want to do it manually and haphazardly with fingers crossed hoping you didn't break anything every publish, or do it with automated tests so you know you didn't break anything every time. -- Ramon Leon http://onsmalltalk.com |
Re: Good reference on time on unit testing?
|
|
Sure. But the audience is a group that bangs out code and then fixes it later; I am trying to help them past that, and something quantitative in an ACM or IEEE journal would be harder to shoot down that our advice, not matter how correct it might be.
________________________________________ From: [hidden email] [[hidden email]] On Behalf Of Ramon Leon [[hidden email]] Sent: Sunday, February 27, 2011 11:26 AM To: [hidden email] Subject: Re: [Pharo-project] Good reference on time on unit testing? On 02/27/2011 06:12 AM, Schwab,Wilhelm K wrote: > Understood. But, doing it correctly, how much of your time do you spend on the test writing/maintenance side? Far less time than you'll spend manually testing over the course of writing and maintaining the program. You can't avoid testing, nor the time spent doing it, the only question is do want to do it manually and haphazardly with fingers crossed hoping you didn't break anything every publish, or do it with automated tests so you know you didn't break anything every time. -- Ramon Leon http://onsmalltalk.com |
Re: Good reference on time on unit testing?
|
|
In reply to this post by Yanni Chiu
I've asked the question three times because people seem to be trying to convince me (I'm on your side of this) rather than my audience.
Your idea of how much test code gets written would be a fine answer to have. One of the points I am going to make in the presentation is that tests are code too (yes, they apparently need to hear that). If expectations were realistic, I would not need numbers. ________________________________________ From: [hidden email] [[hidden email]] On Behalf Of Yanni Chiu [[hidden email]] Sent: Sunday, February 27, 2011 11:19 AM To: [hidden email] Subject: Re: [Pharo-project] Good reference on time on unit testing? On 27/02/11 7:58 AM, Schwab,Wilhelm K wrote: > > I have no problem arguing that testing (if done well) can/will reduce > overall development time; the question is how much of that time one > should expect to devote to writing and maintaining unit and > acceptance tests? You've asked the question for the third time, so all the TDD explanations seem to have missed the mark. How about asking instead: how much test code do people write? Maybe this way, both the TDD and the "how much time" viewpoints can be accommodated (because volume of test code is roughly proportional to time to write test code). -- Yanni Chiu |
Re: Good reference on time on unit testing?
|
|
In reply to this post by laurent laffont
On Sun, Feb 27, 2011 at 5:35 PM, Schwab,Wilhelm K <[hidden email]> wrote: Laurent, :) What I've learned: - people think it will take time to write tests because they don't know what is testing, they don't know what is TDD, because it's hard (yes, it's hard).
- showing them your fully tested project, 100% test coverage, happy customer is not enough (but can help to bring confidence) - showing statistics rarely works (may be for your boss, but not developers)
- buying them good books can help, but for example here in France it's hard: developers rarely read, even less when the book is technical, even less if it's for their work, even less if written in english (and only one book about TDD in French :)
What have worked for me: - the best (by far) is pair programming: but hard to convince people to do pair-programming - except when you are the manager of your team. So you can pair-program with every member and do TDD with them.
- regular internal coding-dojo (randori mode is better): easy acceptance. But that doesn't mean people will test when alone. So it have to be often (every week). - reading group (as described in Pragmatic Thinking and Learning - Andrew Hunt): take a book, a copy for each developer. Each week (or more often if possible) meet for 1 hour and do the summary of last read chapter, discuss, see how it can be applied.
It seems that to spread an idea / practice, we have to help and do things together (see Your brain at work - David Rock) Hope it can help. Laurent.
|
|
|
In reply to this post by abergel
Yes, thus for Bill this mean that for amount of time already expended you need about the same amount for writing tests. YMMV if the cyclomatic complexity (or similar metric) is higher of lower than 'average' programs.
In a more commercial scenario, you can cite Gartner's Research Note SPA-19-3568 where they put 10 to 15% of total development time for unit testing and 10 to 15% time for system [integration] testing. These numbers vary a lot, but you can find good references in Software Engineering books or periodicals. HTH -- Cesar Rabak Em 27/02/2011 11:28, Alexandre Bergel < [hidden email] > escreveu: Robert Martin said in his book that one should spend ~50% of its time writing tests. Cheers, Alexandre On 27 Feb 2011, at 00:52, Schwab,Wilhelm K wrote: > Hello all, > > Can anyone recommend a good reference on the amount of time one should expect to spend writing tests? I will have to be the messenger (will be wearing running shoes just in case...), but I want the message to come from a solid source. > > Bill > > > > -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. |
Re: Good reference on time on unit testing?
|
|
Cesar, Alexandre,
Thanks. Bill ________________________________________ From: [hidden email] [[hidden email]] On Behalf Of [hidden email] [[hidden email]] Sent: Sunday, February 27, 2011 2:24 PM To: [hidden email] Subject: Re: [Pharo-project] Good reference on time on unit testing? Yes, thus for Bill this mean that for amount of time already expended you need about the same amount for writing tests. YMMV if the cyclomatic complexity (or similar metric) is higher of lower than 'average' programs. In a more commercial scenario, you can cite Gartner's Research Note SPA-19-3568 where they put 10 to 15% of total development time for unit testing and 10 to 15% time for system [integration] testing. These numbers vary a lot, but you can find good references in Software Engineering books or periodicals. HTH -- Cesar Rabak Em 27/02/2011 11:28, Alexandre Bergel < [hidden email] > escreveu: Robert Martin said in his book that one should spend ~50% of its time writing tests. Cheers, Alexandre On 27 Feb 2011, at 00:52, Schwab,Wilhelm K wrote: > Hello all, > > Can anyone recommend a good reference on the amount of time one should expect to spend writing tests? I will have to be the messenger (will be wearing running shoes just in case...), but I want the message to come from a solid source. > > Bill > > > > -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. |
Re: Good reference on time on unit testing?
|
|
In reply to this post by Schwab,Wilhelm K
On 27.02.2011, at 13:58, Schwab,Wilhelm K wrote: > Norbert, > > Excellent points - I take exception with only one: you assume that all developers test - that is sadly not true. I am involved with a group who seem to think that a handful of tests added at the last minute will somehow magically fix their problems. > It seems I wasn't clear on this one. I'm trying to making a point that there is no distinction between "testing" - "no testing" but between "testing" - "writing tests". If you take compilation you test the code for syntax and language quirks. If a developer runs the program he develops than he is testing already. Testing and running a program is the same form that point of view. He takes input parameter and expects output parameter. That is testing. The point is just that developers do it manually and that is not reproducable. So there is no real difference between manual testing all the time and written test case. Only that repeating the testing procedure is boring and therefor supposed to be done by a machine. So if developers could see that the just need to put the work they already doing into a test will ease the work without changing much. They are just changing style and save time. That's it. Talking about 100% test coverage and holistic views about what the perfect testing could be doesn't help. > I have no problem arguing that testing (if done well) can/will reduce overall development time; the question is how much of that time one should expect to devote to writing and maintaining unit and acceptance tests? > I understand your intention but the view is inapropriate. Coding and testing aren't two things. It is something that belongs together (if you changed coding style). So that's why it is hard to estimate a time that should be spent. To me it is more comparabale to this: We all know collections are great. Imagine someone asking you "Can anyone recommend a good reference on the amount of time one should expect to spend writing code that uses collections?". What would you say? There is no answer. That doesn't mean you can't solve the problem. But you won't solving it by saying "It is 38.345 % of the time". If people don't get it you have to convince and/or teach them. I had several times where I could show the developer that he is gaining time from doing it. That works. Everything else is targeted towards excel manipulators. Norbert > ________________________________________ > From: [hidden email] [[hidden email]] On Behalf Of Norbert Hartl [[hidden email]] > Sent: Sunday, February 27, 2011 5:41 AM > To: [hidden email] > Subject: Re: [Pharo-project] Good reference on time on unit testing? > > Hi, > > On 27.02.2011, at 04:52, Schwab,Wilhelm K wrote: > >> Hello all, >> >> Can anyone recommend a good reference on the amount of time one should expect to spend writing tests? I will have to be the messenger (will be wearing running shoes just in case...), but I want the message to come from a solid source. > > I find that really hard to answer. To me the problem is the question itself. I heard the "..amount of time one should expect to spend writing tests?" so often in companies and I think it was always exactly this phrase. While the question is valid it gives the impression there is something that decucts time from your "normal" development work. So the people that are asking this question are often managing people that have read something about code quality and they want to apply _this_ to their teams. > The definition over time is troublesome, too. Testing is not easy. Everything you read about testing gives you the impression that everyone knows how to test and that those developers are just too lazy. And that is not true. Most developers I met had problems to see what testing is all about. The don't see interfaces as provable promises etc. So if you tell any of these developers they should spend 1 hour a day in testing than you will get tests that are counter-productive. My favorite example is the one where you have any composite object with an add method. Than the test goes like adding something via the interface and then try to access the internal array and check if it is of size 1. To me it is the same as with documentation. I prefer to have less documentation than useless documentation. > > So every developer is testing in some way. You either test and debug on the way in an unstructured form or you write tests. To me writing tests is not an add-on it is a change in working style. From this point of view I would state that the time I need to spend _additionally_ for testing is negative. I grew up with a print statement being the ultimate debugging tool. A print/debugging statement is added at the time of debugging and probably removed if the error seems to be fixed. That can lead to a situation where you do this multiple times. Writing the same as a test (and that forces you to produce more fine grained code) you will have a saving in time and a little assurance about regression. Regression debugging (debugging it again later) is much more time spent because you have to fix the error _and_ you need to focus again on that problem (which takes most of the time). So the point in writing tests is not to spend time but to save time. > > The amount of time one should expect to spend writing tests is less than the time you need to spend for testing otherwise. :) > > Norbert > |
Re: Good reference on time on unit testing?
|
|
I've always felt that "test driving" code actually results in negative
time spent on "testing". I spend (and everyone I know that tests well does as well) a lot less time writing code test-first than I ever would writing the same code without the tests first. Also, I spend far less time (almost none) manually testing functionality. So TDD results in net negative time difference. (Apologize if I'm incoherent, the cold and flu drugs are strong in this one.) -Steven On Sun, Feb 27, 2011 at 11:51 AM, Norbert Hartl <[hidden email]> wrote: > > On 27.02.2011, at 13:58, Schwab,Wilhelm K wrote: > >> Norbert, >> >> Excellent points - I take exception with only one: you assume that all developers test - that is sadly not true. I am involved with a group who seem to think that a handful of tests added at the last minute will somehow magically fix their problems. >> > It seems I wasn't clear on this one. I'm trying to making a point that there is no distinction between "testing" - "no testing" but between "testing" - "writing tests". If you take compilation you test the code for syntax and language quirks. If a developer runs the program he develops than he is testing already. Testing and running a program is the same form that point of view. He takes input parameter and expects output parameter. That is testing. The point is just that developers do it manually and that is not reproducable. So there is no real difference between manual testing all the time and written test case. Only that repeating the testing procedure is boring and therefor supposed to be done by a machine. So if developers could see that the just need to put the work they already doing into a test will ease the work without changing much. They are just changing style and save time. That's it. Talking about 100% test coverage and holistic views about what the perfect testing could be doesn't help. > >> I have no problem arguing that testing (if done well) can/will reduce overall development time; the question is how much of that time one should expect to devote to writing and maintaining unit and acceptance tests? >> > I understand your intention but the view is inapropriate. Coding and testing aren't two things. It is something that belongs together (if you changed coding style). So that's why it is hard to estimate a time that should be spent. To me it is more comparabale to this: We all know collections are great. Imagine someone asking you "Can anyone recommend a good reference on the amount of time one should expect to spend writing code that uses collections?". What would you say? There is no answer. That doesn't mean you can't solve the problem. But you won't solving it by saying "It is 38.345 % of the time". If people don't get it you have to convince and/or teach them. I had several times where I could show the developer that he is gaining time from doing it. That works. Everything else is targeted towards excel manipulators. > > Norbert > > >> ________________________________________ >> From: [hidden email] [[hidden email]] On Behalf Of Norbert Hartl [[hidden email]] >> Sent: Sunday, February 27, 2011 5:41 AM >> To: [hidden email] >> Subject: Re: [Pharo-project] Good reference on time on unit testing? >> >> Hi, >> >> On 27.02.2011, at 04:52, Schwab,Wilhelm K wrote: >> >>> Hello all, >>> >>> Can anyone recommend a good reference on the amount of time one should expect to spend writing tests? I will have to be the messenger (will be wearing running shoes just in case...), but I want the message to come from a solid source. >> >> I find that really hard to answer. To me the problem is the question itself. I heard the "..amount of time one should expect to spend writing tests?" so often in companies and I think it was always exactly this phrase. While the question is valid it gives the impression there is something that decucts time from your "normal" development work. So the people that are asking this question are often managing people that have read something about code quality and they want to apply _this_ to their teams. >> The definition over time is troublesome, too. Testing is not easy. Everything you read about testing gives you the impression that everyone knows how to test and that those developers are just too lazy. And that is not true. Most developers I met had problems to see what testing is all about. The don't see interfaces as provable promises etc. So if you tell any of these developers they should spend 1 hour a day in testing than you will get tests that are counter-productive. My favorite example is the one where you have any composite object with an add method. Than the test goes like adding something via the interface and then try to access the internal array and check if it is of size 1. To me it is the same as with documentation. I prefer to have less documentation than useless documentation. >> >> So every developer is testing in some way. You either test and debug on the way in an unstructured form or you write tests. To me writing tests is not an add-on it is a change in working style. From this point of view I would state that the time I need to spend _additionally_ for testing is negative. I grew up with a print statement being the ultimate debugging tool. A print/debugging statement is added at the time of debugging and probably removed if the error seems to be fixed. That can lead to a situation where you do this multiple times. Writing the same as a test (and that forces you to produce more fine grained code) you will have a saving in time and a little assurance about regression. Regression debugging (debugging it again later) is much more time spent because you have to fix the error _and_ you need to focus again on that problem (which takes most of the time). So the point in writing tests is not to spend time but to save time. >> >> The amount of time one should expect to spend writing tests is less than the time you need to spend for testing otherwise. :) >> >> Norbert >> > > > |
Re: Good reference on time on unit testing?
|
|
Steven, you're making perfect sense to me, and I think almost everyone here agrees with you.
Bill's problem (if I've understood him correctly) is that he needs to convince some pointy-haired-boss-type person, by directing him or her to a well-respected "official" statistic that "proves" what we all know to be true from experience. Sadly, I think it will be hard to find it :-( -- Cheers, Peter On Sun, Feb 27, 2011 at 9:04 PM, Steven Baker <[hidden email]> wrote:
I've always felt that "test driving" code actually results in negative |
Re: Good reference on time on unit testing?
|
|
Oh!
I never worry about "proving" it. When I've worked for clients who have insisted that I leave the testing out because they don't want to "pay extra" for the time spent, I simply don't push the tests to their repo. I test for myself, because I have a personal commitment to do the best job possible. -Steven On Sun, Feb 27, 2011 at 12:11 PM, Peter Hugosson-Miller <[hidden email]> wrote: > Steven, you're making perfect sense to me, and I think almost everyone here > agrees with you. > > Bill's problem (if I've understood him correctly) is that he needs to > convince some pointy-haired-boss-type person, by directing him or her to a > well-respected "official" statistic that "proves" what we all know to be > true from experience. Sadly, I think it will be hard to find it :-( > > -- > Cheers, > Peter > > On Sun, Feb 27, 2011 at 9:04 PM, Steven Baker <[hidden email]> > wrote: >> >> I've always felt that "test driving" code actually results in negative >> time spent on "testing". I spend (and everyone I know that tests well >> does as well) a lot less time writing code test-first than I ever >> would writing the same code without the tests first. Also, I spend far >> less time (almost none) manually testing functionality. So TDD results >> in net negative time difference. >> >> (Apologize if I'm incoherent, the cold and flu drugs are strong in this >> one.) >> >> -Steven >> >> On Sun, Feb 27, 2011 at 11:51 AM, Norbert Hartl <[hidden email]> >> wrote: >> > >> > On 27.02.2011, at 13:58, Schwab,Wilhelm K wrote: >> > >> >> Norbert, >> >> >> >> Excellent points - I take exception with only one: you assume that all >> >> developers test - that is sadly not true. I am involved with a group who >> >> seem to think that a handful of tests added at the last minute will somehow >> >> magically fix their problems. >> >> >> > It seems I wasn't clear on this one. I'm trying to making a point that >> > there is no distinction between "testing" - "no testing" but between >> > "testing" - "writing tests". If you take compilation you test the code for >> > syntax and language quirks. If a developer runs the program he develops than >> > he is testing already. Testing and running a program is the same form that >> > point of view. He takes input parameter and expects output parameter. That >> > is testing. The point is just that developers do it manually and that is not >> > reproducable. So there is no real difference between manual testing all the >> > time and written test case. Only that repeating the testing procedure is >> > boring and therefor supposed to be done by a machine. So if developers could >> > see that the just need to put the work they already doing into a test will >> > ease the work without changing much. They are just changing style and save >> > time. That's it. Talking about 100% test coverage and holistic views about >> > what the perfect testing could be doesn't help. >> > >> >> I have no problem arguing that testing (if done well) can/will reduce >> >> overall development time; the question is how much of that time one should >> >> expect to devote to writing and maintaining unit and acceptance tests? >> >> >> > I understand your intention but the view is inapropriate. Coding and >> > testing aren't two things. It is something that belongs together (if you >> > changed coding style). So that's why it is hard to estimate a time that >> > should be spent. To me it is more comparabale to this: We all know >> > collections are great. Imagine someone asking you "Can anyone recommend a >> > good reference on the amount of time one should expect to spend writing code >> > that uses collections?". What would you say? There is no answer. That >> > doesn't mean you can't solve the problem. But you won't solving it by saying >> > "It is 38.345 % of the time". If people don't get it you have to convince >> > and/or teach them. I had several times where I could show the developer that >> > he is gaining time from doing it. That works. Everything else is targeted >> > towards excel manipulators. >> > >> > Norbert >> > >> > >> >> ________________________________________ >> >> From: [hidden email] >> >> [[hidden email]] On Behalf Of Norbert Hartl >> >> [[hidden email]] >> >> Sent: Sunday, February 27, 2011 5:41 AM >> >> To: [hidden email] >> >> Subject: Re: [Pharo-project] Good reference on time on unit testing? >> >> >> >> Hi, >> >> >> >> On 27.02.2011, at 04:52, Schwab,Wilhelm K wrote: >> >> >> >>> Hello all, >> >>> >> >>> Can anyone recommend a good reference on the amount of time one should >> >>> expect to spend writing tests? I will have to be the messenger (will be >> >>> wearing running shoes just in case...), but I want the message to come from >> >>> a solid source. >> >> >> >> I find that really hard to answer. To me the problem is the question >> >> itself. I heard the "..amount of time one should expect to spend writing >> >> tests?" so often in companies and I think it was always exactly this phrase. >> >> While the question is valid it gives the impression there is something that >> >> decucts time from your "normal" development work. So the people that are >> >> asking this question are often managing people that have read something >> >> about code quality and they want to apply _this_ to their teams. >> >> The definition over time is troublesome, too. Testing is not easy. >> >> Everything you read about testing gives you the impression that everyone >> >> knows how to test and that those developers are just too lazy. And that is >> >> not true. Most developers I met had problems to see what testing is all >> >> about. The don't see interfaces as provable promises etc. So if you tell any >> >> of these developers they should spend 1 hour a day in testing than you will >> >> get tests that are counter-productive. My favorite example is the one where >> >> you have any composite object with an add method. Than the test goes like >> >> adding something via the interface and then try to access the internal array >> >> and check if it is of size 1. To me it is the same as with documentation. I >> >> prefer to have less documentation than useless documentation. >> >> >> >> So every developer is testing in some way. You either test and debug on >> >> the way in an unstructured form or you write tests. To me writing tests is >> >> not an add-on it is a change in working style. From this point of view I >> >> would state that the time I need to spend _additionally_ for testing is >> >> negative. I grew up with a print statement being the ultimate debugging >> >> tool. A print/debugging statement is added at the time of debugging and >> >> probably removed if the error seems to be fixed. That can lead to a >> >> situation where you do this multiple times. Writing the same as a test (and >> >> that forces you to produce more fine grained code) you will have a saving in >> >> time and a little assurance about regression. Regression debugging >> >> (debugging it again later) is much more time spent because you have to fix >> >> the error _and_ you need to focus again on that problem (which takes most of >> >> the time). So the point in writing tests is not to spend time but to save >> >> time. >> >> >> >> The amount of time one should expect to spend writing tests is less >> >> than the time you need to spend for testing otherwise. :) >> >> >> >> Norbert > |
Re: Good reference on time on unit testing?
|
|
In reply to this post by NorbertHartl
Norbert,
Have you ever read of shops where people throw code at features and never much bother to test what they are doing, because it is "testing's" job to catch the bugs ? I had read about it too... I understand what you are saying. Bill ________________________________________ From: [hidden email] [[hidden email]] On Behalf Of Norbert Hartl [[hidden email]] Sent: Sunday, February 27, 2011 2:51 PM To: [hidden email] Subject: Re: [Pharo-project] Good reference on time on unit testing? On 27.02.2011, at 13:58, Schwab,Wilhelm K wrote: > Norbert, > > Excellent points - I take exception with only one: you assume that all developers test - that is sadly not true. I am involved with a group who seem to think that a handful of tests added at the last minute will somehow magically fix their problems. > It seems I wasn't clear on this one. I'm trying to making a point that there is no distinction between "testing" - "no testing" but between "testing" - "writing tests". If you take compilation you test the code for syntax and language quirks. If a developer runs the program he develops than he is testing already. Testing and running a program is the same form that point of view. He takes input parameter and expects output parameter. That is testing. The point is just that developers do it manually and that is not reproducable. So there is no real difference between manual testing all the time and written test case. Only that repeating the testing procedure is boring and therefor supposed to be done by a machine. So if developers could see that the just need to put the work they already doing into a test will ease the work without changing much. They are just changing style and save time. That's it. Talking about 100% test coverage and holistic views about what the perfect testing could be doesn't help. > I have no problem arguing that testing (if done well) can/will reduce overall development time; the question is how much of that time one should expect to devote to writing and maintaining unit and acceptance tests? > I understand your intention but the view is inapropriate. Coding and testing aren't two things. It is something that belongs together (if you changed coding style). So that's why it is hard to estimate a time that should be spent. To me it is more comparabale to this: We all know collections are great. Imagine someone asking you "Can anyone recommend a good reference on the amount of time one should expect to spend writing code that uses collections?". What would you say? There is no answer. That doesn't mean you can't solve the problem. But you won't solving it by saying "It is 38.345 % of the time". If people don't get it you have to convince and/or teach them. I had several times where I could show the developer that he is gaining time from doing it. That works. Everything else is targeted towards excel manipulators. Norbert > ________________________________________ > From: [hidden email] [[hidden email]] On Behalf Of Norbert Hartl [[hidden email]] > Sent: Sunday, February 27, 2011 5:41 AM > To: [hidden email] > Subject: Re: [Pharo-project] Good reference on time on unit testing? > > Hi, > > On 27.02.2011, at 04:52, Schwab,Wilhelm K wrote: > >> Hello all, >> >> Can anyone recommend a good reference on the amount of time one should expect to spend writing tests? I will have to be the messenger (will be wearing running shoes just in case...), but I want the message to come from a solid source. > > I find that really hard to answer. To me the problem is the question itself. I heard the "..amount of time one should expect to spend writing tests?" so often in companies and I think it was always exactly this phrase. While the question is valid it gives the impression there is something that decucts time from your "normal" development work. So the people that are asking this question are often managing people that have read something about code quality and they want to apply _this_ to their teams. > The definition over time is troublesome, too. Testing is not easy. Everything you read about testing gives you the impression that everyone knows how to test and that those developers are just too lazy. And that is not true. Most developers I met had problems to see what testing is all about. The don't see interfaces as provable promises etc. So if you tell any of these developers they should spend 1 hour a day in testing than you will get tests that are counter-productive. My favorite example is the one where you have any composite object with an add method. Than the test goes like adding something via the interface and then try to access the internal array and check if it is of size 1. To me it is the same as with documentation. I prefer to have less documentation than useless documentation. > > So every developer is testing in some way. You either test and debug on the way in an unstructured form or you write tests. To me writing tests is not an add-on it is a change in working style. From this point of view I would state that the time I need to spend _additionally_ for testing is negative. I grew up with a print statement being the ultimate debugging tool. A print/debugging statement is added at the time of debugging and probably removed if the error seems to be fixed. That can lead to a situation where you do this multiple times. Writing the same as a test (and that forces you to produce more fine grained code) you will have a saving in time and a little assurance about regression. Regression debugging (debugging it again later) is much more time spent because you have to fix the error _and_ you need to focus again on that problem (which takes most of the time). So the point in writing tests is not to spend time but to save time. > > The amount of time one should expect to spend writing tests is less than the time you need to spend for testing otherwise. :) > > Norbert > |
«
Return to Pharo Smalltalk Developers
|
1 view|%1 views
| Free forum by Nabble | Edit this page |