UserInterfaceTheme testing & TestRunner

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

UserInterfaceTheme testing & TestRunner

timrowledge
I tried running the tests on UserInterfaceThem to check I hadn’t broken too much and as is so often the case on slower machines some of the tests timed out. It’s not a simply replicatable thing either, with each run producing different tests that ‘fail’ this way.

So the obvious thing to day was
a) add a timeout: pragma to the tests that seem to be slow
b) add an option to profile an individual test

While running these to play with them I managed to occasionally lock up the input process, which I’d make a wild guess might be something to do with the debugging etc in the testrunner code. It’s not reliable enough to get a good handle on it. It did, however manage to completely lock up the Pi on one occasion - not merely Squeak but the entire Pi. Attempts to login remotely had no effect. Naughty, naughty somewhere in the stack.

Whilst writing this little extension I noticed some code of a form I’m not keen on. Perhaps there is a good solid reason for it to be like this that I’m unaware of -
runFailures
        self result instVarNamed: 'failures' put: Set new.
        self runSuite: self suiteFailures.
(similar in runErrors)
There’s not often a good reason to use #instVarNamed:put: and it’s sufficiently odd that I have to wonder if there is a hidden subtlety making it appropriate.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
"Bother!" said Pooh, as Piglet pressed <START> on the Microwave...



Reply | Threaded
Open this post in threaded view
|

Re: UserInterfaceTheme testing & TestRunner

David T. Lewis
On Mon, Sep 25, 2017 at 02:43:28PM -0700, tim Rowledge wrote:

> I tried running the tests on UserInterfaceThem to check I hadn???t broken too much and as is so often the case on slower machines some of the tests timed out. It???s not a simply replicatable thing either, with each run producing different tests that ???fail??? this way.
>
> So the obvious thing to day was
> a) add a timeout: pragma to the tests that seem to be slow
> b) add an option to profile an individual test
>
> While running these to play with them I managed to occasionally lock up the input process, which I???d make a wild guess might be something to do with the debugging etc in the testrunner code. It???s not reliable enough to get a good handle on it. It did, however manage to completely lock up the Pi on one occasion - not merely Squeak but the entire Pi. Attempts to login remotely had no effect. Naughty, naughty somewhere in the stack.
>
> Whilst writing this little extension I noticed some code of a form I???m not keen on. Perhaps there is a good solid reason for it to be like this that I???m unaware of -
> runFailures
> self result instVarNamed: 'failures' put: Set new.
> self runSuite: self suiteFailures.
> (similar in runErrors)
> There???s not often a good reason to use #instVarNamed:put: and it???s sufficiently odd that I have to wonder if there is a hidden subtlety making it appropriate.
>

I am going to take a wild guess at this one, just to see if I can get
it right. My theory is that classes such as TestResult in the underlying
test framework are shared across Smalltalk dialects, so the author of
the test runner UI wanted to avoid touching those classes. There is no
TestResult>>failures: accessor, and rather than add the accessor, the UI
author (lr) elected to just poke the value in using #instVarNamed:put:
instead.

Dave


Reply | Threaded
Open this post in threaded view
|

Re: UserInterfaceTheme testing & TestRunner

marcel.taeumel
Hi Tim,

you can also overwrite #defaultTimeOut in UserInterfaceThemeTest and make use of PreferenceWizardMorph >> #hasLowPerformance, whose implementation we might improve in the future.

Best,
Marcel

Am 26.09.2017 01:44:23 schrieb David T. Lewis <[hidden email]>:

On Mon, Sep 25, 2017 at 02:43:28PM -0700, tim Rowledge wrote:
> I tried running the tests on UserInterfaceThem to check I hadn???t broken too much and as is so often the case on slower machines some of the tests timed out. It???s not a simply replicatable thing either, with each run producing different tests that ???fail??? this way.
>
> So the obvious thing to day was
> a) add a timeout: pragma to the tests that seem to be slow
> b) add an option to profile an individual test
>
> While running these to play with them I managed to occasionally lock up the input process, which I???d make a wild guess might be something to do with the debugging etc in the testrunner code. It???s not reliable enough to get a good handle on it. It did, however manage to completely lock up the Pi on one occasion - not merely Squeak but the entire Pi. Attempts to login remotely had no effect. Naughty, naughty somewhere in the stack.
>
> Whilst writing this little extension I noticed some code of a form I???m not keen on. Perhaps there is a good solid reason for it to be like this that I???m unaware of -
> runFailures
> self result instVarNamed: 'failures' put: Set new.
> self runSuite: self suiteFailures.
> (similar in runErrors)
> There???s not often a good reason to use #instVarNamed:put: and it???s sufficiently odd that I have to wonder if there is a hidden subtlety making it appropriate.
>

I am going to take a wild guess at this one, just to see if I can get
it right. My theory is that classes such as TestResult in the underlying
test framework are shared across Smalltalk dialects, so the author of
the test runner UI wanted to avoid touching those classes. There is no
TestResult>>failures: accessor, and rather than add the accessor, the UI
author (lr) elected to just poke the value in using #instVarNamed:put:
instead.

Dave




Reply | Threaded
Open this post in threaded view
|

Re: UserInterfaceTheme testing & TestRunner

timrowledge

> On 26-09-2017, at 12:42 AM, Marcel Taeumel <[hidden email]> wrote:
>
> Hi Tim,
>
> you can also overwrite #defaultTimeOut in UserInterfaceThemeTest

Hmm, hadn’t come across that one, neat.

> and make use of PreferenceWizardMorph >> #hasLowPerformance, whose implementation we might improve in the future.

Yes, a simple multiplier in the base defaultTimeout might work. I see there is already a LongTestCase class too. Another possibly useful menu option for a failed test might be just to 'try again with longer time limit'


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Manual Writer's Creed:  Garbage in, gospel out.