Smalltalk › Pharo › Pharo Smalltalk Developers

Need help to get a list of Pharo projects with tests

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

5 messages Options

Julien Delplanque-2

Need help to get a list of Pharo projects with tests

Hello Pharo community,

I am currently working on detecting rotten tests in Pharo projects.

Rotten tests are defined as test methods containing one or many assertions in their source code but one or many of these assertions are not executed when the test is run.

To have more details on the subject, you can check the research report related to our first definition of these tests freely available on HAL [1].

In this context, we built a test analyser which, given a Pharo package containing tests, finds rotten tests [2]. This analyser is still under development.

We would like to extend the experiment of our research report [1] and to analyse more projects in order to get a better understanding of rotten tests.

To do that, we need your help. You can help us in two ways:

1. Answer this email with links to one or many open-source Pharo projects containing tests.

2. If you want, provide us private projects or, at least, run our analyser on these private projects and give us back the results.

If you want to help us with the second option, send me a mail directly at [hidden email].

Thanks in advance.

Regards,

Julien Delplanque

Links:

[1]: https://hal.archives-ouvertes.fr/hal-01819302

[2]: https://github.com/juliendelplanque/RottenTestsFinder/

---

Julien Delplanque
Doctorant à l’Université de Lille

http://juliendelplanque.be/phd.html
Equipe Rmod, Inria

Bâtiment B 40, Avenue Halley 59650 Villeneuve d'Ascq

Numéro de téléphone: +333 59 35 86 40

SergeStinckwich

Re: Need help to get a list of Pharo projects with tests

On Thu, Jul 5, 2018 at 3:24 PM Julien <[hidden email]> wrote:

Hello Pharo community,

I am currently working on detecting rotten tests in Pharo projects.

Rotten tests are defined as test methods containing one or many assertions in their source code but one or many of these assertions are not executed when the test is run.
To have more details on the subject, you can check the research report related to our first definition of these tests freely available on HAL [1].

In this context, we built a test analyser which, given a Pharo package containing tests, finds rotten tests [2]. This analyser is still under development.

We would like to extend the experiment of our research report [1] and to analyse more projects in order to get a better understanding of rotten tests.

To do that, we need your help. You can help us in two ways:
1. Answer this email with links to one or many open-source Pharo projects containing tests.

​We have 774 tests in PolyMath: 

https://github.com/PolyMathOrg/PolyMath

Serge Stinckwich
UMI UMMISCO 209 (SU/IRD/UY1)

"Programs must be written for people to read, and only incidentally for machines to execute."

http://www.doesnotunderstand.org/

Max Leske

Re: Need help to get a list of Pharo projects with tests

Fuel: https://github.com/theseion/Fuel
Seaside: https://github.com/SeasideSt/Seaside

On 5 Jul 2018, at 16:31, Serge Stinckwich wrote:

On Thu, Jul 5, 2018 at 3:24 PM Julien <[hidden email]> wrote:
Hello Pharo community,

I am currently working on detecting rotten tests in Pharo projects.

Rotten tests are defined as test methods containing one or many assertions in their source code but one or many of these assertions are not executed when the test is run.
To have more details on the subject, you can check the research report related to our first definition of these tests freely available on HAL [1].

In this context, we built a test analyser which, given a Pharo package containing tests, finds rotten tests [2]. This analyser is still under development.

We would like to extend the experiment of our research report [1] and to analyse more projects in order to get a better understanding of rotten tests.

To do that, we need your help. You can help us in two ways:
1. Answer this email with links to one or many open-source Pharo projects containing tests.

We have 774 tests in PolyMath:
https://github.com/PolyMathOrg/PolyMath

--
Serge Stinckwich
UMI UMMISCO 209 (SU/IRD/UY1)
"Programs must be written for people to read, and only incidentally for machines to execute."
http://www.doesnotunderstand.org/

Peter Uhnak

Re: Need help to get a list of Pharo projects with tests

Hi,

can you please include in the README at the very least how it should be launched? Looking through the code I eventually found that there's a world menu entry... but that's not exactly the best way to discovery.

I've ran it on couple of projects, but it didn't show anything apart from self skip, but if you find something else, I'd be interested to see what it can detect.

https://github.com/peteruhnak?utf8=%E2%9C%93&tab=repositories&q=&type=source&language=smalltalk

Also when I filter packages and select one, the test cases list contains some random unrelated tests. When the filtering is not on, the test cases are correct.

Cheers,

Peter

On Thu, Jul 5, 2018 at 7:16 PM, Max Leske <[hidden email]> wrote:

Fuel: https://github.com/theseion/Fuel
Seaside: https://github.com/SeasideSt/Seaside

On 5 Jul 2018, at 16:31, Serge Stinckwich wrote:

On Thu, Jul 5, 2018 at 3:24 PM Julien <[hidden email]> wrote:
Hello Pharo community,

I am currently working on detecting rotten tests in Pharo projects.

Rotten tests are defined as test methods containing one or many assertions in their source code but one or many of these assertions are not executed when the test is run.
To have more details on the subject, you can check the research report related to our first definition of these tests freely available on HAL [1].

In this context, we built a test analyser which, given a Pharo package containing tests, finds rotten tests [2]. This analyser is still under development.

We would like to extend the experiment of our research report [1] and to analyse more projects in order to get a better understanding of rotten tests.

To do that, we need your help. You can help us in two ways:
1. Answer this email with links to one or many open-source Pharo projects containing tests.

We have 774 tests in PolyMath:
https://github.com/PolyMathOrg/PolyMath

--
Serge Stinckwich
UMI UMMISCO 209 (SU/IRD/UY1)
"Programs must be written for people to read, and only incidentally for machines to execute."
http://www.doesnotunderstand.org/

Julien Delplanque-2

Re: Need help to get a list of Pharo projects with tests

Thanks for all your answers, I’m collecting these projects for the experiment. :-)

Peter,

Le 5 juil. 2018 à 21:12, Peter Uhnák <[hidden email]> a écrit :

Hi,

can you please include in the README at the very least how it should be launched? Looking through the code I eventually found that there's a world menu entry... but that's not exactly the best way to discovery.

Yeah, the GitHub repository is not yet in a good state, the previous mail was not an announcement for the project in fact, I just put the link to show it exists, it’s real… :-)

Anyway, you can run it (without the GUI, because there are some bugs in it and I do not focus too much on it for now) using this:

[RottenTestsFinder analyze: (RPackage organizer packageNamed: 'Collections-Tests')] on: Warning do: #resume.

I've ran it on couple of projects, but it didn't show anything apart from self skip, but if you find something else, I'd be interested to see what it can detect.

In Collection tests you have some rotten tests. :-)

BUT, the result are not totally correct right now, it is still under development so do not expect it to work for now.

https://github.com/peteruhnak?utf8=%E2%9C%93&tab=repositories&q=&type=source&language=smalltalk

Also when I filter packages and select one, the test cases list contains some random unrelated tests. When the filtering is not on, the test cases are correct.

Yeah, I know this bug, the filter of the Spec widget does not work as I expected.

If you do not use the filters, it should work well.

You can select all tests in the second list using Cmd+a.

Cheers,

Julien