Smalltalk › Frameworks & Tools › Moose

Spy alters some test results?

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

4 messages Options

MartinW

Spy alters some test results?

Hello,
i am experimenting with the Spy framework (i hope this is the appropriate newsgroup).

I encountered the following problem:
Running unit tests through Spy gives sometimes other test results than running them with the Test Runner.

I first encountered this problem when i tried in Pharo 2:

Profiler runTestsForPackages: #('XML-Parser-Nodes' 'XML-Tests-Parser-Nodes')

When the same tests are run from the Test Runner all 126 tests pass,
test results in the Spy Profiler show 12 failures.

Is anybody working with tests in Spy? Does anybody have an idea what may cause this phenomenon and how to fix it?

Best regards,
Martin.

Ben Coman

Re: Spy alters some test results?

I have seen use of Sets in tests introduce randomness in the ordering of
results.

MartinW wrote:

> Hello,
> i am experimenting with the Spy framework (i hope this is the appropriate
> newsgroup).
>
> I encountered the following problem:
> Running unit tests through Spy gives sometimes other test results than
> running them with the Test Runner.
>
> I first encountered this problem when i tried in Pharo 2:
>
> Profiler runTestsForPackages: #('XML-Parser-Nodes' 'XML-Tests-Parser-Nodes')
>
> When the same tests are run from the Test Runner all 126 tests pass,
> test results in the Spy Profiler show 12 failures.
>
> Is anybody working with tests in Spy? Does anybody have an idea what may
> cause this phenomenon and how to fix it?
>
> Best regards,
> Martin.
>
> <http://forum.world.st/file/n4722052/Tests_with_Spy.png>
>
>
>
> --
> View this message in context: http://forum.world.st/Spy-alters-some-test-results-tp4722052.html
> Sent from the Moose mailing list archive at Nabble.com.
> _______________________________________________
> Moose-dev mailing list
> [hidden email]
> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>
>

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

jfabry

Re: Spy alters some test results?

I have had similar issues with testing PHANtom. Apparently the test runner does meta-level stuff that interferes with other packages that modify method lookup (through method wrappers or other means). I have no fundamental fix, sadly. A workaround is to run the tests outside the test runner.

On Nov 14, 2013, at 11:30 AM, Ben Coman <[hidden email]> wrote:

> I have seen use of Sets in tests introduce randomness in the ordering of results.
>
> MartinW wrote:
>> Hello,
>> i am experimenting with the Spy framework (i hope this is the appropriate
>> newsgroup).
>>
>> I encountered the following problem:
>> Running unit tests through Spy gives sometimes other test results than
>> running them with the Test Runner.
>>
>> I first encountered this problem when i tried in Pharo 2:
>>
>> Profiler runTestsForPackages: #('XML-Parser-Nodes' 'XML-Tests-Parser-Nodes')
>>
>> When the same tests are run from the Test Runner all 126 tests pass,
>> test results in the Spy Profiler show 12 failures.
>>
>> Is anybody working with tests in Spy? Does anybody have an idea what may
>> cause this phenomenon and how to fix it?
>>
>> Best regards,
>> Martin.
>>
>> <http://forum.world.st/file/n4722052/Tests_with_Spy.png>
>>
>>
>> --
>> View this message in context: http://forum.world.st/Spy-alters-some-test-results-tp4722052.html
>> Sent from the Moose mailing list archive at Nabble.com.
>> _______________________________________________
>> Moose-dev mailing list
>> [hidden email]
>> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>>
>>
>
> _______________________________________________
> Moose-dev mailing list
> [hidden email]
> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>

---> Save our in-boxes! http://emailcharter.org <---

Johan Fabry - http://pleiad.cl/~jfabry
PLEIAD lab - Computer Science Department (DCC) - University of Chile

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

abergel

Re: Spy alters some test results?

In reply to this post by Ben Coman

Spy has not been used that much recently.
Alejandro is working on Gadget profiler, and he is putting a lot of effort on that.

Martin, can you describe the scenario you are interesting in using Spy? Maybe we can achieve something simpler and more stable by using Gadget. We can have a skype meeting or something (in spanish ? :-)

Alexandre

On Nov 14, 2013, at 11:30 AM, Ben Coman <[hidden email]> wrote:

--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev