Core evolution and non regression test Framework (was Ideas about sets and dictionaries)
Locked
Core evolution and non regression test Framework (was Ideas about sets and dictionaries)
 
|
2009/11/12 Igor Stasenko <[hidden email]>:
> 2009/11/12 Ralph Johnson <[hidden email]>: >>> All tests in CollectionTests are green except one unrelated >>> (ArrayTest>>testPrinting) >> >> But that is boring and obvious. >> > not for those who cares, and changed the printing behavior. But it was not me. > >> What I want to know is that you don't break any tests on any published >> Squeak -based system. I don't just mean all the tests in the standard >> Squeak images, I mean all the applications, too. >> >> I imagine that a change of this magnitude WILL break things. Then we >> can discuss how badly it breaks things and how hard it is to fix. If >> it only breaks a few applications, and they are easy to fix, then it >> is no big deal. But the main problem with your proposal is that >> changing fundamental behavior of base classes is likely to break >> applications, and you won't address that questions with tests of >> collection classes. >> > > Ralph, by putting such high demands you can kill any potential > initiative to see any changes in > existing core classes, not only mine. > Tests and only tests should cover all the cases, needed to be tested > against correct behavior. > If some application breaking up, it would mean only that either > current set of tests not sufficient, and therefore we > need better test coverage, or application plays against the rules and > breaking an encapsulation. > Testing against virtually every existing application is unreasonable > demand and nobody having time to do that. > >> -Ralph >> >> > Agree 100% with Igor if the change is about an implementation detail. Ideal in-image tests should be a specification of the API. If an external package fails, then either in-image tests are incomplete or external package violated some encapsulation. Now, if we have to modify a test, that is a specification, then we should open a discussion. That's why I support [trunk] spam in squeak-dev. Set includes: nil is an example of specification modification. A package maintainer can legitimately expect a Set doesn't contain any UndefinedObject based on previous tests. Since we should be able to move and not freeze core squeak development, while not making package maintenance a hell, we should agree on a process. Who is responsible for the release should: - clearly state compatibility issues in release notes (where are trunk release notes?) - propose known corrections or workaround for smooth transition of external packages - provide technical support if requested by package maintainers (can be in above format) I agree with Ralph that it would be great to add this one: - provide automated tests infra-structure for main public packages, (extensible by squeak users for their own private ones). The last point means we have automatic installation and test procedures like Installer/Sake for example. Then it would be package maintainer responsibility to provide istallation and tests procedures. This goal has been temporarily frozen in trunk, but will have to come back. I think it is orthogonal to trunk policy, and does not have to compete. The major difference between 3.11 and trunk process is the will to be able to backport core changes to different flavours of image. This is another discussion of course. Nicolas > > > -- > Best regards, > Igor Stasenko AKA sig. > > |
Re: Core evolution and non regression test Framework (was Ideas about sets and dictionaries)
|
|
2009/11/12 Nicolas Cellier <[hidden email]>:
> 2009/11/12 Igor Stasenko <[hidden email]>: >> 2009/11/12 Ralph Johnson <[hidden email]>: >>>> All tests in CollectionTests are green except one unrelated >>>> (ArrayTest>>testPrinting) >>> >>> But that is boring and obvious. >>> >> not for those who cares, and changed the printing behavior. But it was not me. >> >>> What I want to know is that you don't break any tests on any published >>> Squeak -based system. I don't just mean all the tests in the standard >>> Squeak images, I mean all the applications, too. >>> >>> I imagine that a change of this magnitude WILL break things. Then we >>> can discuss how badly it breaks things and how hard it is to fix. If >>> it only breaks a few applications, and they are easy to fix, then it >>> is no big deal. But the main problem with your proposal is that >>> changing fundamental behavior of base classes is likely to break >>> applications, and you won't address that questions with tests of >>> collection classes. >>> >> >> Ralph, by putting such high demands you can kill any potential >> initiative to see any changes in >> existing core classes, not only mine. >> Tests and only tests should cover all the cases, needed to be tested >> against correct behavior. >> If some application breaking up, it would mean only that either >> current set of tests not sufficient, and therefore we >> need better test coverage, or application plays against the rules and >> breaking an encapsulation. >> Testing against virtually every existing application is unreasonable >> demand and nobody having time to do that. >> >>> -Ralph >>> >>> >> > > Agree 100% with Igor if the change is about an implementation detail. > Ideal in-image tests should be a specification of the API. > If an external package fails, then either in-image tests are > incomplete or external package violated some encapsulation. > > Now, if we have to modify a test, that is a specification, then we > should open a discussion. > That's why I support [trunk] spam in squeak-dev. > > Set includes: nil is an example of specification modification. > A package maintainer can legitimately expect a Set doesn't contain any > UndefinedObject based on previous tests. > create them by intent. This is important starting point, which allows nearly painless introduction of new specification, thanks to previous implementation. And now let us think, how they appear: - by a developer who are aware that nils now can be uncluded into sets. Now he passing such set to external package routine, it automatically rings the bell in his head, because he should either use a newer version of package which already allows and using sets with nils, or if not, then developer takes a counter measures to ensure old specification conformance. Or lastly, he starts bashing the package maintainer until he adds support of new sets conformance :) The things as to me is quite simple: unless you intentionally start using sets with nils, you have to chances to break any existing code, because there will be no difference. And i can guess that most of the code could easily swallow nil as element without any breakage or need of modification. Simply because everything is an object. > Since we should be able to move and not freeze core squeak > development, while not making package maintenance a hell, we should > agree on a process. > Who is responsible for the release should: > - clearly state compatibility issues in release notes (where are trunk > release notes?) > - propose known corrections or workaround for smooth transition of > external packages > - provide technical support if requested by package maintainers (can > be in above format) This is important, and while i'm still remember that, may i ask you to do a little bureaucracy (of course if you have a time on that), to create a draft 'conduct of rules' , a set of good development practices for Squeak developers? I think this will be indeed very helpful and useful, and since you started this topic - you drive :) Then we can discuss it on squeak-dev list, board meeting, and finally ratify & place it on some public location. Then board, btw could officially have rights to ban or punish those who not following these rules , including themselves, at first place, of course :) > I agree with Ralph that it would be great to add this one: > - provide automated tests infra-structure for main public packages, > (extensible by squeak users for their own private ones). > The last point means we have automatic installation and test > procedures like Installer/Sake for example. > Then it would be package maintainer responsibility to provide > istallation and tests procedures. > This goal has been temporarily frozen in trunk, but will have to come back. > I think it is orthogonal to trunk policy, and does not have to compete. > Yes, absolutely, we eventually will get to it sooner or later. And i really hope in receiving help from Keith and Matthew in that. > The major difference between 3.11 and trunk process is the will to be > able to backport core changes to different flavours of image. This is > another discussion of course. > > Nicolas > >> >> >> -- >> Best regards, >> Igor Stasenko AKA sig. >> -- Best regards, Igor Stasenko AKA sig. |
«
Return to Squeak - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |