[Question/Bug] SUnit CodeCoverage uses #valueUnpreemptively

Hi all,

I have a question regarding to the implementation of TestRunner >> #collectCoverageFor:. Why does it use #valueUnpreemptively during running the coverage tests?

I found two major disadvantages of using #valueUnpreemptively in the TestRunner:

1. You cannot interrupt the system via cmd+dot if you lose your patience to wait for all tests to complete.
   

2. In addition, I found a quite interesting bug that makes your image unusable if you do the following:
   
   

TestCase subclass: #MyTestCase

instanceVariableNames: ''

classVariableNames: ''

poolDictionaries: ''

category: 'UserObjects'.

MyTestCase compile: 'testTimeout

[] repeat'.

TestRunner openForSuite: MyTestCase suite.

"Now press 'Run coverage' abd choose any non-empty package."

The coverage test will never end!

Why should it be necessary to use it in the TestRunner? Because we replace CompiledMethods? That's nothing WrappedBreakpoint wouldn't do, too. I replaced #valueUnpreemptively in this method by a normal #value in my image and the world did not end - thanks to dynamic forwarding, I could even keep browsing the patched methods. However, I admit it would not be a good idea to recompile any method during the coverage tests are running. (We could also use some property like Class >> #beLocked to avoid this, hypothetically).

I am looking forward to your advice.

Best,

Christoph

Am 09.04.2020 15:33:23 schrieb Thiede, Christoph <[hidden email]>:

Hi all,

I have a question regarding to the implementation of TestRunner >> #collectCoverageFor:. Why does it use #valueUnpreemptively during running the coverage tests?

I found two major disadvantages of using #valueUnpreemptively in the TestRunner:

1. You cannot interrupt the system via cmd+dot if you lose your patience to wait for all tests to complete.
   

2. In addition, I found a quite interesting bug that makes your image unusable if you do the following:
   
   

TestCase subclass: #MyTestCase

instanceVariableNames: ''

classVariableNames: ''

poolDictionaries: ''

category: 'UserObjects'.

MyTestCase compile: 'testTimeout

[] repeat'.

TestRunner openForSuite: MyTestCase suite.

"Now press 'Run coverage' abd choose any non-empty package."

The coverage test will never end!

#valueUnpreemptively's comment says that you should use it very carefully, what makes absolutely sense to me, and it has only two senders in the Trunk at all (the second one is ClassBuilder >> #update:to:).

Why should it be necessary to use it in the TestRunner? Because we replace CompiledMethods? That's nothing WrappedBreakpoint wouldn't do, too. I replaced #valueUnpreemptively in this method by a normal #value in my image and the world did not end - thanks to dynamic forwarding, I could even keep browsing the patched methods. However, I admit it would not be a good idea to recompile any method during the coverage tests are running. (We could also use some property like Class >> #beLocked to avoid this, hypothetically).

I am looking forward to your advice.

Best,

Christoph

Hi Marcel, thanks for your points! :-) I think the coverage detection should work with hard-coded priorities, too. There are just too many users that don't respect Processor activePriority etc.

Hm ... Speaking hypothetically, couldn't we check Processor activeProcess in every single TestCoverage, i.e.:

- Register the testing process when creating a coverage wrapper

- Only #mark the method if Processor activeProcess equals the testing process or a child of it?

Then we could get completely rid of global state and you should be able to run coverage testing completely asynchronously (for example in AutoTDD)! :-)

However, this could slow down coverage testing significantly, I did not yet try it out ...

Best,

Christoph