improving the quality of the image

> Jerome Peace has been working on getting rid of unimplemented methods
> and has a lot of fixes.  You can find them at
> http://bugs.impara.de/view.php?id=4544   This is the original Mantis
> issue that he has been working on.  Most of the fixes are in "child"
> issues, but you can find them from that page.
>
> You can help by checking Jerome's fixes.  If you are familiar with the
> code he is changing, read it and see whether you can spot anything
> wrong.  If you can, post a note.  If you can't, please post a note
> that, as far as you can tell, it looks good.  If you aren't that
> familiar with the code, but are working on an application that uses
> it, please file in the changes and try out your application.  Again,
> report on the results!

> But there are many
> other things that could be checked automatically.  For example, there
> should be no unimplemented methods in the released image.
> Unfortunately, there are a lot right now, so we can't make that rule.
> But I would like to have all these fixed by the end of the 3.10 cycle
> and to be able to enforce the rule that no release has any
> unimplemented methods.
>
> Jerome Peace has been working on getting rid of unimplemented methods
> and has a lot of fixes.  You can find them at
> http://bugs.impara.de/view.php?id=4544   This is the original Mantis
> issue that he has been working on.  Most of the fixes are in "child"
> issues, but you can find them from that page.
>
> You can help by checking Jerome's fixes.  If you are familiar with the
> code he is changing, read it and see whether you can spot anything
> wrong.  If you can, post a note.  If you can't, please post a note
> that, as far as you can tell, it looks good.  If you aren't that
> familiar with the code, but are working on an application that uses
> it, please file in the changes and try out your application.  Again,
> report on the results!
>
> If two or three people try out some changes and everybody thinks they
> are OK, the release team will mark the issue as "resolved" and put it
> in the next release.
>
> We will make sure the code doesn't break any tests.  But if you don't
> try out the code then we'll have to try it, as well, and that will
> take a lot more time.  So, you can help us get more work done by
> checking out these fixes.
>
> Thanks!
>
> -Ralph Johnson
>
>

>> But there are many
>> other things that could be checked automatically.  For example, there
>> should be no unimplemented methods in the released image.
>> Unfortunately, there are a lot right now, so we can't make that rule.
>> But I would like to have all these fixed by the end of the 3.10 cycle
>> and to be able to enforce the rule that no release has any
>> unimplemented methods.
>>
>> Jerome Peace has been working on getting rid of unimplemented methods
>> and has a lot of fixes.  You can find them at
>> http://bugs.impara.de/view.php?id=4544   This is the original Mantis
>> issue that he has been working on.  Most of the fixes are in "child"
>> issues, but you can find them from that page.
>>
>> You can help by checking Jerome's fixes.  If you are familiar with  
>> the
>> code he is changing, read it and see whether you can spot anything
>> wrong.  If you can, post a note.  If you can't, please post a note
>> that, as far as you can tell, it looks good.  If you aren't that
>> familiar with the code, but are working on an application that uses
>> it, please file in the changes and try out your application.  Again,
>> report on the results!
>>
>> If two or three people try out some changes and everybody thinks they
>> are OK, the release team will mark the issue as "resolved" and put it
>> in the next release.
>>
>> We will make sure the code doesn't break any tests.  But if you don't
>> try out the code then we'll have to try it, as well, and that will
>> take a lot more time.  So, you can help us get more work done by
>> checking out these fixes.
>>
>> Thanks!
>>
>> -Ralph Johnson
>>
>>
>

>
> On Jan 26, 2007, at 16:03 , Philippe Marschall wrote:
>
> > 2007/1/26, Ralph Johnson <[hidden email]>:
> >> One of my goals for 3.10 is to improve the quality of the image.  Our
> >> first release (coming soon!) will have only green tests, and each
> >> following release will have only green tests.
> >
> > How does removing failing tests improve the quality?
>
> Woa, where does that hostility come from? There is another way to
> ensure all tests are green, besides removing the failing ones.

> >> But there are many
> >> other things that could be checked automatically.  For example, there
> >> should be no unimplemented methods in the released image.
> >> Unfortunately, there are a lot right now, so we can't make that rule.
> >> But I would like to have all these fixed by the end of the 3.10 cycle
> >> and to be able to enforce the rule that no release has any
> >> unimplemented methods.
> >>
> >> Jerome Peace has been working on getting rid of unimplemented methods
> >> and has a lot of fixes.  You can find them at
> >> http://bugs.impara.de/view.php?id=4544   This is the original Mantis
> >> issue that he has been working on.  Most of the fixes are in "child"
> >> issues, but you can find them from that page.
> >>
> >> You can help by checking Jerome's fixes.  If you are familiar with
> >> the
> >> code he is changing, read it and see whether you can spot anything
> >> wrong.  If you can, post a note.  If you can't, please post a note
> >> that, as far as you can tell, it looks good.  If you aren't that
> >> familiar with the code, but are working on an application that uses
> >> it, please file in the changes and try out your application.  Again,
> >> report on the results!
> >>
> >> If two or three people try out some changes and everybody thinks they
> >> are OK, the release team will mark the issue as "resolved" and put it
> >> in the next release.
> >>
> >> We will make sure the code doesn't break any tests.  But if you don't
> >> try out the code then we'll have to try it, as well, and that will
> >> take a lot more time.  So, you can help us get more work done by
> >> checking out these fixes.
> >>
> >> Thanks!
> >>
> >> -Ralph Johnson
> >>
> >>
> >
>
>
>
>
>
>

> 2007/1/26, Bert Freudenberg <[hidden email]>:
>>
>> On Jan 26, 2007, at 16:03 , Philippe Marschall wrote:
>>
>> > 2007/1/26, Ralph Johnson <[hidden email]>:
>> >> One of my goals for 3.10 is to improve the quality of the image.  Our
>> >> first release (coming soon!) will have only green tests, and each
>> >> following release will have only green tests.
>> >
>> > How does removing failing tests improve the quality?
>>
>> Woa, where does that hostility come from? There is another way to
>> ensure all tests are green, besides removing the failing ones.
>
> What hostility? I could not see why this improves the quality because
> to me the first step to fix a problem is to admit that you have a
> problem. Failing tests are pointer to problems for me. Removing
> failing tests because they can not be fixed today or tomorrow looked
> to me like an attempt to hide hide a problem. So I asked and now I
> know the reason why it was done.
>
> Philippe
>

>> >> But there are many
>> >> other things that could be checked automatically.  For example, there
>> >> should be no unimplemented methods in the released image.
>> >> Unfortunately, there are a lot right now, so we can't make that rule.
>> >> But I would like to have all these fixed by the end of the 3.10 cycle
>> >> and to be able to enforce the rule that no release has any
>> >> unimplemented methods.
>> >>
>> >> Jerome Peace has been working on getting rid of unimplemented methods
>> >> and has a lot of fixes.  You can find them at
>> >> http://bugs.impara.de/view.php?id=4544   This is the original Mantis
>> >> issue that he has been working on.  Most of the fixes are in "child"
>> >> issues, but you can find them from that page.
>> >>
>> >> You can help by checking Jerome's fixes.  If you are familiar with
>> >> the
>> >> code he is changing, read it and see whether you can spot anything
>> >> wrong.  If you can, post a note.  If you can't, please post a note
>> >> that, as far as you can tell, it looks good.  If you aren't that
>> >> familiar with the code, but are working on an application that uses
>> >> it, please file in the changes and try out your application.  Again,
>> >> report on the results!
>> >>
>> >> If two or three people try out some changes and everybody thinks they
>> >> are OK, the release team will mark the issue as "resolved" and put it
>> >> in the next release.
>> >>
>> >> We will make sure the code doesn't break any tests.  But if you don't
>> >> try out the code then we'll have to try it, as well, and that will
>> >> take a lot more time.  So, you can help us get more work done by
>> >> checking out these fixes.
>> >>
>> >> Thanks!
>> >>
>> >> -Ralph Johnson
>> >>
>> >>
>> >
>>
>>
>>
>>
>>
>>
>
>

> Philippe Marschall wrote:
>> 2007/1/26, Bert Freudenberg <[hidden email]>:
>>>
>>> On Jan 26, 2007, at 16:03 , Philippe Marschall wrote:
>>>
>>> > 2007/1/26, Ralph Johnson <[hidden email]>:
>>> >> One of my goals for 3.10 is to improve the quality of the image.
>>> Our
>>> >> first release (coming soon!) will have only green tests, and each
>>> >> following release will have only green tests.
>>> >
>>> > How does removing failing tests improve the quality?
>>>
>>> Woa, where does that hostility come from? There is another way to
>>> ensure all tests are green, besides removing the failing ones.

> Philippe Marschall wrote:
> > 2007/1/26, Bert Freudenberg <[hidden email]>:
> >>
> >> On Jan 26, 2007, at 16:03 , Philippe Marschall wrote:
> >>
> >> > 2007/1/26, Ralph Johnson <[hidden email]>:
> >> >> One of my goals for 3.10 is to improve the quality of the image.  Our
> >> >> first release (coming soon!) will have only green tests, and each
> >> >> following release will have only green tests.
> >> >
> >> > How does removing failing tests improve the quality?
> >>
> >> Woa, where does that hostility come from? There is another way to
> >> ensure all tests are green, besides removing the failing ones.
> >
> > What hostility? I could not see why this improves the quality because
> > to me the first step to fix a problem is to admit that you have a
> > problem. Failing tests are pointer to problems for me. Removing
> > failing tests because they can not be fixed today or tomorrow looked
> > to me like an attempt to hide hide a problem. So I asked and now I
> > know the reason why it was done.
> >
> > Philippe
> >
> Philippe, where did you read that failing tests will be removed? "First
> release will have
> only green tests" means, that all tests remain and will pass, not fail.
> There will be no
> test removal at all! I'm, pretty sure you misunderstood something.

> Elod
> >> >> But there are many
> >> >> other things that could be checked automatically.  For example, there
> >> >> should be no unimplemented methods in the released image.
> >> >> Unfortunately, there are a lot right now, so we can't make that rule.
> >> >> But I would like to have all these fixed by the end of the 3.10 cycle
> >> >> and to be able to enforce the rule that no release has any
> >> >> unimplemented methods.
> >> >>
> >> >> Jerome Peace has been working on getting rid of unimplemented methods
> >> >> and has a lot of fixes.  You can find them at
> >> >> http://bugs.impara.de/view.php?id=4544   This is the original Mantis
> >> >> issue that he has been working on.  Most of the fixes are in "child"
> >> >> issues, but you can find them from that page.
> >> >>
> >> >> You can help by checking Jerome's fixes.  If you are familiar with
> >> >> the
> >> >> code he is changing, read it and see whether you can spot anything
> >> >> wrong.  If you can, post a note.  If you can't, please post a note
> >> >> that, as far as you can tell, it looks good.  If you aren't that
> >> >> familiar with the code, but are working on an application that uses
> >> >> it, please file in the changes and try out your application.  Again,
> >> >> report on the results!
> >> >>
> >> >> If two or three people try out some changes and everybody thinks they
> >> >> are OK, the release team will mark the issue as "resolved" and put it
> >> >> in the next release.
> >> >>
> >> >> We will make sure the code doesn't break any tests.  But if you don't
> >> >> try out the code then we'll have to try it, as well, and that will
> >> >> take a lot more time.  So, you can help us get more work done by
> >> >> checking out these fixes.
> >> >>
> >> >> Thanks!
> >> >>
> >> >> -Ralph Johnson
> >> >>
> >> >>
> >> >
> >>
> >>
> >>
> >>
> >>
> >>
> >
> >
>
>
>

> 2007/1/29, Elod Kironsky <[hidden email]>:
>> Philippe Marschall wrote:
>> > 2007/1/26, Bert Freudenberg <[hidden email]>:
>> >>
>> >> On Jan 26, 2007, at 16:03 , Philippe Marschall wrote:
>> >>
>> >> > 2007/1/26, Ralph Johnson <[hidden email]>:
>> >> >> One of my goals for 3.10 is to improve the quality of the
>> image.  Our
>> >> >> first release (coming soon!) will have only green tests, and each
>> >> >> following release will have only green tests.
>> >> >
>> >> > How does removing failing tests improve the quality?
>> >>
>> >> Woa, where does that hostility come from? There is another way to
>> >> ensure all tests are green, besides removing the failing ones.
>> >
>> > What hostility? I could not see why this improves the quality because
>> > to me the first step to fix a problem is to admit that you have a
>> > problem. Failing tests are pointer to problems for me. Removing
>> > failing tests because they can not be fixed today or tomorrow looked
>> > to me like an attempt to hide hide a problem. So I asked and now I
>> > know the reason why it was done.
>> >
>> > Philippe
>> >
>> Philippe, where did you read that failing tests will be removed? "First
>> release will have
>> only green tests" means, that all tests remain and will pass, not fail.
>> There will be no
>> test removal at all! I'm, pretty sure you misunderstood something.
>
> http://bugs.impara.de/view.php?id=5527
>
> Philippe
>

> Failing tests do not belong in the image.  They belong in Mantis.  If
> you have a test that ought to run but does not then post it to Mantis.
>
> We should not release an image with failing tests.  In general, the
> 3.10 release team will not accept changes that break tests.  If the
> image contains failing tests then it is hard for people to tell that
> their proposed changed breaks them.  It is crucial that there be NO
> failing tests in the image so that anyone can easily check whether
> their change caused any of the tests to fail.  Go to the test runner,
> select all tests, run them, and make sure everything is green.
>
> There is no need to make some new way of classifying tests.  The way
> we classify tests is that tests that do not work are in Mantis as bug
> reports.  If you want to work on them then you can easily file them
> in.

>> Did you delete the tests AND the code that it tests? Just removing the
>> tests doesn't really help much if the code that it tests is still there.
>> I'm sure you agree with this, so that is why I'm a bit stumped.
>
> You completely miss the point.  Leaving broken tests in the image does
> not cause the tests to be fixed.  Broken tests have been in the image
> for years and they did not help.
>
> There are a lot of bugs in Squeak.  Squeak is, in general, of
> "experimental" quality.  It is amazing to me that people actually
> develop commercial products with it, but they do.  I would like to
> raise the quality of Squeak.  Part of this is making sure that there
> is a regression test suite that actually works, and preventing people
> from breaking it.  There are lots of other parts, of course.  The
> image should be smaller, it should have more tests, more bugs should
> be fixed, there should not be any unimplemented messages.  No one
> thing will improve quality, but a lot of little things will.
>
> In general, it is not easy to find the code that a test exercises.
> What you are asking does not  make sense.
>
> Please run the tests in 3.9, look at the ones that are broken, and try
> to figure out what code is bad and should be deleted.  It will be an
> enlightening experience.

> Ok, IMHO this is all about classification of tests. Perhaps someone has
> already proposed the following but how about:
>
> - If there is a bug and someone authors a test to show it and it enters
> the image (lets ignore *that* particular question for 3 seconds - many of
> course argue that it should not enter the image unless accompanied with a
> fix), why not mark it as "has never worked" - or something along those
> lines?
>
> That way we can keep all *working* (not marked) tests green and at the
> same time have a list of non working tests (those marked) that typically
> are all red. When something is fixed and turned green we remove the
> marker.
>
> Sure, this may be a daft idea :), just wanted to mention it.
>
> regards, Göran
>
>
>
>  

> Dear Goran,
>
> I am with you on this one. Having broken tests visible, but  
> correctly categorised is key to getting things sorted IMHO. I dont  
> think that hiding things away on mantis is not visible enough.
>
> You might like to try my improvements to TestRunner and SUnit which  
> implement these changes. You can try it by executing.
>
> Installer fixBug: 5639.
>
> best regards
>
> Keith
>> Ok, IMHO this is all about classification of tests. Perhaps  
>> someone has
>> already proposed the following but how about:
>>
>> - If there is a bug and someone authors a test to show it and it  
>> enters
>> the image (lets ignore *that* particular question for 3 seconds -  
>> many of
>> course argue that it should not enter the image unless accompanied  
>> with a
>> fix), why not mark it as "has never worked" - or something along  
>> those
>> lines?
>>
>> That way we can keep all *working* (not marked) tests green and at  
>> the
>> same time have a list of non working tests (those marked) that  
>> typically
>> are all red. When something is fixed and turned green we remove the
>> marker.
>>
>> Sure, this may be a daft idea :), just wanted to mention it.
>>
>> regards, Göran
>