Smalltalk › Squeak › Squeak - Dev

[Proposal SUnit] Show number of tests and last duration

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

6 messages Options

marcel.taeumel

[Proposal SUnit] Show number of tests and last duration

Can I merge the required changes into Trunk's SUnit package? :-) It's basically a few additions to TestResult and RestTunner.

Best,

Marcel

Tobias Pape

Re: [Proposal SUnit] Show number of tests and last duration

> On 03.09.2019, at 10:35, Marcel Taeumel <[hidden email]> wrote:
>
> Can I merge the required changes into Trunk's SUnit package? :-) It's basically a few additions to TestResult and RestTunner.
>

:D
Yes please, at least something in that direction.

-t

David T. Lewis

Re: [Proposal SUnit] Show number of tests and last duration

In reply to this post by marcel.taeumel

On Tue, Sep 03, 2019 at 10:35:01AM +0200, Marcel Taeumel wrote:
> Can I merge the required changes into Trunk's SUnit package? :-) It's basically a few additions to TestResult and RestTunner.
>
>

Yes please, this is a useful enhancement.

Dave

timrowledge

Re: [Proposal SUnit] Show number of tests and last duration

In reply to this post by Tobias Pape

> On 2019-09-03, at 1:38 AM, Tobias Pape <[hidden email]> wrote:
>
>
>> On 03.09.2019, at 10:35, Marcel Taeumel <[hidden email]> wrote:
>>
>> Can I merge the required changes into Trunk's SUnit package? :-) It's basically a few additions to TestResult and RestTunner.
>>
>
> :D
> Yes please, at least something in that direction.

Likewise

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
"Bother," said Pooh. "Eeyore - ready two photon torpedoes and lock phasers on the Heffalump. Piglet, meet me in Transporter Room Three."

Chris Muller-3

Re: [Proposal SUnit] Show number of tests and last duration

In reply to this post by marcel.taeumel

A great start to detecting "performance failures". +1

Now imagine if we added a variable to TestCase, say, 'timings', a Dictionary of #testSelector -> lastRunDuration 's. Then, any test more than ~10% off its baseline would raise an Error (or.. PerformanceFailure?, whatever).

There'd be a method, #resetBaselines to reset this Dictionary under the same condition that we re-ask for developer initials -- when image detects it has moved to a new location. Since this this could mean possibly different hardware, new baselines would need established to avoid spurious failures.

From a UI perspective, access to #resetBaselines should be sufficient -- once all are addressed or "okay'ed", the #resetBaselines would allow new baselines to be captured on the next run...

- Chris

On Tue, Sep 3, 2019 at 3:35 AM Marcel Taeumel <[hidden email]> wrote:

Can I merge the required changes into Trunk's SUnit package? :-) It's basically a few additions to TestResult and RestTunner.

Best,
Marcel

marcel.taeumel

Re: [Proposal SUnit] Show number of tests and last duration

Hmm... at the moment, it would be a new entry in the TestCase's history dict besides #passes, #failures, etc.:

How to compute a reliable baseline? You know about SMark and BechmarkRunner?

http://www.squeaksource.com/SMark.html

http://www.hpi.uni-potsdam.de/hirschfeld/squeaksource/BenchmarkRunner.html

A TestCase's history dict is the mechanism to store such kind of data, I suppose. Even though I am not sure why there are no TestResults kept in such a history record. Objects! :-D

Best,

Marcel

Am 04.09.2019 00:45:22 schrieb Chris Muller <[hidden email]>:

A great start to detecting "performance failures". +1

Now imagine if we added a variable to TestCase, say, 'timings', a Dictionary of #testSelector -> lastRunDuration 's. Then, any test more than ~10% off its baseline would raise an Error (or.. PerformanceFailure?, whatever).

There'd be a method, #resetBaselines to reset this Dictionary under the same condition that we re-ask for developer initials -- when image detects it has moved to a new location. Since this this could mean possibly different hardware, new baselines would need established to avoid spurious failures.

From a UI perspective, access to #resetBaselines should be sufficient -- once all are addressed or "okay'ed", the #resetBaselines would allow new baselines to be captured on the next run...

- Chris

On Tue, Sep 3, 2019 at 3:35 AM Marcel Taeumel <[hidden email]> wrote:
Can I merge the required changes into Trunk's SUnit package? :-) It's basically a few additions to TestResult and RestTunner.

Best,
Marcel