akuhn/SUnit, chrisma edition :)
Locked
123
123
Re: akuhn/SUnit, christmas edition :)
 
|
Alexandre Bergel <alexandre@...> writes:
> > The preferred way is to store your results into the stored history. You can > > store the history with the context menu in the test runner. To do so, right > > click in the colored status field and choose `store history`. The stored > > results are treated as the expected results then. > > Yes, but... how is this related to the expectedError? The stored history is compiled into a method on the class side. So you get the exact benefit of #expectedFailures/#expectedError. It is persistent across source control and will mark your failures/errors as expected. So you dont need #expectedError at all. The history system has two levels - "history" is put in a class variable on the class side. This information is used for the browser icons by OB and O2, such that you can see the result of the latest test run. This information is local to your image. - "stored history" is put into a compiled method on the class side. This information is used by the test runner to mark unexpected passes and expected defects (failures, errors *and* any custom defects). This information is global and persists together with your source code. Programatically they are accessible with `testClass history` and `testClass storedHistory`, and `testClass storeHistoryIntoCompiledMethod` will put the current history into stored history. Marking of unexpected results can be done between *any* two test results with `testResult markUnexpectedResults: anotherTestResult`, so other tools builders are free to compare any two results. Also you can define your very own test results (eg `aTestFinished outcome: #ignored`) and they will be processed by result marking. --AA _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: akuhn/SUnit, christmas edition :)
|
|
In reply to this post by Mariano Martinez Peck
Mariano Martinez Peck <marianopeck@...> writes:
> Hi Adrian! Thanks for trying to fix this. However, I can still reproduce the > problem. I put a halt in my expectedFailures method and it is being called > [..] The iteration throught the do: seems to be ok, but I don't know why, > the tests are still considered as error (red) instead of expectedFailures in > green. Is correct to put "outcome: TestFinished failures;" ??? I guess > it would be something like "outcome: TestFinished expectedFailures" but I > look in the class side methods of TestFinished and I only see passed, failure > and error. Let me know if I can test something more. I see, you got the same case as Alex. You declared an expected *error* in #expectedFailures. That used to work with the old SUnit but the new SUnit takes you literally and expects it to be a failure and a failure only. I will work on a solution for that. --AA _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: akuhn/SUnit, christmas edition :)
|
|
On Mon, Dec 28, 2009 at 6:42 PM, Adrian Kuhn <[hidden email]> wrote:
Ahhh yes, sorry. I just discovered that. Wait...I will suggest this: It is true what you said about expectedFailure is expected failure and not expected error. I just did it like that because I thought it was like that. But now that I see it, that is probably wrong. So, what I would do is to "fix" it but (I don't know) put is as deprecated so that all the tests of the people are still green and give them time to change it to expectedErrors or similar. After a second release you can remove this fix :) What do you think ? Cheers Mariano
_______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: akuhn/SUnit, christmas edition :)
|
|
In reply to this post by Adrian Kuhn
>>> The preferred way is to store your results into the stored
>>> history. You can >>> store the history with the context menu in the test runner. To do >>> so, right >>> click in the colored status field and choose `store history`. The >>> stored >>> results are treated as the expected results then. >> >> Yes, but... how is this related to the expectedError? > > The stored history is compiled into a method on the class side. So > you get the > exact benefit of #expectedFailures/#expectedError. It is persistent > across > source control and will mark your failures/errors as expected. So > you dont > need #expectedError at all. Yep, I know this, I pair programmed the history mechanism with Simon :-) But I still do not understand how do you distinguish a method that always fails from an expectedFailure (i.e., a method that I know will fail, but I do not want it to make my unit test red)? If we need expectedFailure, we will probably need expectedError no? Cheers, Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: akuhn/SUnit, chrisma edition :)
|
|
In reply to this post by Stéphane Ducasse
Stéphane Ducasse <stephane.ducasse@...> writes:
> Adrian I would love to see a videos of the new features you did :) A blog post is in work... --AA -- 2nd Workshop on Source Code Search etc. Submit papers by January 19, 2010. http://bit.ly/suite2010 _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: akuhn/SUnit, christmas edition :)
|
|
In reply to this post by Alexandre Bergel
This is really cool that you work on SUnit. This is really needed.
Cheers, Alexandre On 28 Dec 2009, at 18:47, Alexandre Bergel wrote: >>>> The preferred way is to store your results into the stored >>>> history. You can >>>> store the history with the context menu in the test runner. To do >>>> so, right >>>> click in the colored status field and choose `store history`. The >>>> stored >>>> results are treated as the expected results then. >>> >>> Yes, but... how is this related to the expectedError? >> >> The stored history is compiled into a method on the class side. So >> you get the >> exact benefit of #expectedFailures/#expectedError. It is persistent >> across >> source control and will mark your failures/errors as expected. So >> you dont >> need #expectedError at all. > > Yep, I know this, I pair programmed the history mechanism with > Simon :-) > But I still do not understand how do you distinguish a method that > always fails from an expectedFailure (i.e., a method that I know will > fail, but I do not want it to make my unit test red)? > If we need expectedFailure, we will probably need expectedError no? > > Cheers, > Alexandre > -- > _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: > Alexandre Bergel http://www.bergel.eu > ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > > > > > > > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: akuhn/SUnit, christmas edition :)
|
|
In reply to this post by Alexandre Bergel
Alexandre Bergel <alexandre@...> writes:
> > The stored history is compiled into a method on the class side. So you get > > the exact benefit of #expectedFailures/#expectedError. It is persistent > > across source control and will mark your failures/errors as expected. So > > you dont need #expectedError at all. > > Yep, I know this, I pair programmed the history mechanism with Simon > But I still do not understand how do you distinguish a method that > always fails from an expectedFailure (i.e., a method that I know will > fail, but I do not want it to make my unit test red)? > If we need expectedFailure, we will probably need expectedError no? This is an aspect that I changed. An outcome is unexpected if and only if it does not match what is in the stored history. (For compatibility only #expectedFailures are added to the stored history.) So the preferred way to mark an outcome as unexpected is to store the history. Try it - remove #expectedFailures - run all your tests (you'll get failures and errors, -> not green) - store the history - run all your tests again (all your defects are expected, -> green) And just for demonstration - introduce a new defect - run all your tests again (one unexpected defect, -> not green) Does this match your workflow? --AA _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: akuhn/SUnit, christmas edition :)
|
|
In reply to this post by Mariano Martinez Peck
Mariano Martinez Peck <marianopeck@...> writes:
> So, what I would do is to "fix" it but (I don't know) put is as deprecated so > that all the tests of the people are still green and give them time to change > it to expectedErrors or similar. After a second release you can remove this > fix :) What do you think? Yes, that is how I planned it ... and then you came with your dynamically generated expectations. Which the new system does not support. It's an awesome idea, so I am thinking about a way to support dynamic expectations in the new system. I have one solution in mind: given your #expectedFailures, would the following work for you? testBlob self assume: SqueakDatabaseAccessor DefaultDriver ~~ SqueakDBXDriver. ... which runs the test if and only if the default driver is not SqueakDBXDriver. This would have the same effect as your #expectedFailures implementation, where you expected an error for that case. Like this your expectation is put closer to the code. --AA _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: akuhn/SUnit, christmas edition :)
|
|
In reply to this post by Adrian Kuhn
On Mon, Dec 28, 2009 at 7:01 PM, Adrian Kuhn <[hidden email]> wrote:
The problem is if you have several others failing/error tests but that they are not expected :( Ok...I can run only the expected ones and store them, or don't run the error/failures ones and run them later
_______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: akuhn/SUnit, christmas edition :)
|
|
In reply to this post by Adrian Kuhn
On Mon, Dec 28, 2009 at 7:29 PM, Adrian Kuhn <[hidden email]> wrote:
Yes, this will work and will give me a benefit and a drawback :) Benefit: I can declare a expected failure only for a single test and not for all the tests of the class. Right now, I have situations like this and I a doing exactly that (more ugly actually), an if to see something and then a ^ self is it shouldn't be run. Drawback: Most of the times (in my case) the condition (SqueakDatabaseAccessor DefaultDriver ~~ SqueakDBXDrive) is always the same, so that's why it was easier as I did it. Here I should declare it in every test. However, this is VERYYYYYY little detail that I really don't care (I am not eve sure if this solution is better). I am bothering too much hahaha. Of course, I don't know what other people think about this but in my point of view, this is ok :) Cheers Mariano
_______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Assumptions (was Re: akuhn/SUnit, christmas edition :)
|
|
Mariano Martinez Peck <marianopeck@...> writes:
Assumptions are in the box! If you expect your test case to fail unless a given condition is met, you can now use assumptions to declare that. Gofer the latest akuhn/SUnit to get assumptions. For example, when your test case depends on the availability of a certain database you can write testBlob self assumeThat: Database default = DatabaseWithBlobs. ... if the assumption is true, the test case is run as a normal test case. If however the assumption is false, the test case is *not* run but still marked as passed. This is supposed to replace conditional #expectedFailures in the new SUnit. Typically, assumptions are put at the begin of a test case. However, you can put assumptions anywhere. The running tests case is aborted as soon as the first failing assumption is reached. PS regarding the naming: the message #assumeThat: has been preferred over a simple #assume: to make confusions with a simple #assert: less likely. --AA _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: Assumptions (was Re: akuhn/SUnit, christmas edition :)
|
|
On Mon, Dec 28, 2009 at 11:19 PM, Adrian Kuhn <[hidden email]> wrote: Mariano Martinez Peck <marianopeck@...> writes: Thanks Adrian. All your improvments are really cool. The only thing that I am concerned is that if I change all my tests to this style, Squeak people won't be able to run the tests :( because that method won't be understood. Anyway, I will si what can I do. Thank you very much for doing every day something better from SUnit!!! --AA _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: Assumptions (was Re: akuhn/SUnit, christmas edition :)
|
|
On Tue, Dec 29, 2009 at 12:17 AM, Mariano Martinez Peck <[hidden email]> wrote:
Sorry I forgot to say that I just realized about that (because you asked me if it was ok and I said that yes).
_______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: Assumptions (was Re: akuhn/SUnit, christmas edition :)
|
|
In reply to this post by Mariano Martinez Peck
Mariano Martinez Peck <marianopeck@...> writes:
> Thanks Adrian. All your improvments are really cool. The only thing that I am > concerned is that if I change all my tests to this style, Squeak people won't > be able to run the tests :( because that method won't be understood. True enough, that applies to any new feature. We cannot have evolution and full compatibility at the same time. I will try to ensure that old tests keep running. But regarding new features there is no guarantees (unless someone backports my changes). --AA _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
|
|
That's irght. You can't break eggs without baking a cake!
Dale ----- "Adrian Kuhn" <[hidden email]> wrote: | Mariano Martinez Peck <marianopeck@...> writes: | | > Thanks Adrian. All your improvments are really cool. The only thing | that I am | > concerned is that if I change all my tests to this style, Squeak | people won't | > be able to run the tests :( because that method won't be | understood. | | True enough, that applies to any new feature. We cannot have evolution | and | full compatibility at the same time. I will try to ensure that old | tests keep | running. But regarding new features there is no guarantees (unless | someone | backports my changes). | | --AA | | | | _______________________________________________ | Pharo-project mailing list | [hidden email] | http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: Assumptions (was Re: akuhn/SUnit, christmas edition :)
|
|
In reply to this post by Adrian Kuhn
On Mon, Dec 28, 2009 at 2:19 PM, Adrian Kuhn <[hidden email]> wrote:
> For example, when your test case depends on the availability of a certain > database you can write > > testBlob > self assumeThat: Database default = DatabaseWithBlobs. > ... > Can I suggest simply #assume: ? The colon kind of has the same meaning as "that" so it ends up redundant (would be the same with existing methods: #assert(That): #deny(That):). Then, having said that, #assume: sounds a bit like the test should assume the statement is true and then carry on running with that assumption. Maybe #expect: would be more descriptive? Or #precondition: ? Not sure... Julian _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: Assumptions (was Re: akuhn/SUnit, christmas edition :)
|
|
yes expect: seems better.
Stef On Dec 29, 2009, at 7:21 AM, Julian Fitzell wrote: > On Mon, Dec 28, 2009 at 2:19 PM, Adrian Kuhn <[hidden email]> wrote: >> For example, when your test case depends on the availability of a certain >> database you can write >> >> testBlob >> self assumeThat: Database default = DatabaseWithBlobs. >> ... >> > > Can I suggest simply #assume: ? The colon kind of has the same meaning > as "that" so it ends up redundant (would be the same with existing > methods: #assert(That): #deny(That):). > > Then, having said that, #assume: sounds a bit like the test should > assume the statement is true and then carry on running with that > assumption. Maybe #expect: would be more descriptive? Or > #precondition: ? Not sure... > > Julian > > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: Assumptions (was Re: akuhn/SUnit, christmas edition :)
|
|
> On Dec 29, 2009, at 7:21 AM, Julian Fitzell wrote:
> > > Then, having said that, #assume: sounds a bit like the test should > > assume the statement is true and then carry on running with that > > assumption. Maybe #expect: would be more descriptive? Or > > #precondition: ? Not sure... > > Stéphane Ducasse <stephane.ducasse@...> writes: > > yes expect: seems better. Hmm, that sound to me as if the test should fail when the expectations is not met, whereas in fact it is skipped unless the condition is true. What about something like #skipUnless: ? --AA _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: Assumptions (was Re: akuhn/SUnit, christmas edition :)
|
|
yes much much better to me.
On Dec 29, 2009, at 11:13 AM, Adrian Kuhn wrote: >> On Dec 29, 2009, at 7:21 AM, Julian Fitzell wrote: >> >>> Then, having said that, #assume: sounds a bit like the test should >>> assume the statement is true and then carry on running with that >>> assumption. Maybe #expect: would be more descriptive? Or >>> #precondition: ? Not sure... >> >> Stéphane Ducasse <stephane.ducasse@...> writes: >> >> yes expect: seems better. > > Hmm, that sound to me as if the test should fail when the expectations is > not met, whereas in fact it is skipped unless the condition is true. > > What about something like #skipUnless: ? > > --AA > > > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: Assumptions (was Re: akuhn/SUnit, christmas edition :)
|
|
In reply to this post by Adrian Kuhn
On Tue, Dec 29, 2009 at 2:13 AM, Adrian Kuhn <[hidden email]> wrote:
>> On Dec 29, 2009, at 7:21 AM, Julian Fitzell wrote: >> >> > Then, having said that, #assume: sounds a bit like the test should >> > assume the statement is true and then carry on running with that >> > assumption. Maybe #expect: would be more descriptive? Or >> > #precondition: ? Not sure... >> >> Stéphane Ducasse <stephane.ducasse@...> writes: >> >> yes expect: seems better. > > Hmm, that sound to me as if the test should fail when the expectations is > not met, whereas in fact it is skipped unless the condition is true. > > What about something like #skipUnless: ? Sure... I think #expect: could be interpreted either way; #skipUnless: pretty unambiguous. Julian _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
«
Return to Pharo Smalltalk Developers
|
1 view|%1 views
| Free forum by Nabble | Edit this page |