State of Testing in Squeak

> Ladies and Gentlemen,
>
> The value of a test suite that can give the developer a good degree of
> confidence in a short period of time cannot be understated. IMHO, the
> tests in Trunk take *way* too long to run. It's been bugging me for
> awhile. I'd like to be kicking off a test pass every time I finish
> loading updates, and as it stands this is not feasible. As a result, I
> don't run the tests. I wonder how many other people are having this
> experience?
>
> Would the people of the community be receptive to moving long running
> tests out to a separately loadable package?
>

> On Thu, 2009-12-03 at 15:54 -0800, Andreas Raab wrote:
>> Ken Causey wrote:
>> > On Thu, 2009-12-03 at 13:36 -0800, Ronald Spengler wrote:
>> >> Would the people of the community be receptive to moving long running
>> >> tests out to a separately loadable package?
>> >
>> > I fear that separating a subset of tests from the rest will result in
>> > them being largely forgotten and unused.
>>
>> True, but which choice is preferable: Having a few tests never run, or
>> having no tests ever run? (hey, that rhymes! ;-)
>>
>> Cheers,
>>    - Andreas
>
> I think 'never' is the wrong word.  What I'm thinking of is an option to
> not run the long running tests that defaults to running all tests and
> actually requires some small effort from the user with hopefully at
> least a little thought and recognition that the other tests do exist.
> Separating out the long running tests into a separate package likely
> means newcomers are never aware of them and the rest of us forget about
> them, they are no longer maintained, and that's the end of them.  If a
> test is truly not useful, or at least never worth the running time
> required, fine just delete it;  but I have to assume that is not
> generally the case otherwise it would have already happened.
>
> I agree that currently running all the tests is not something you do
> without thought, but it's not that bad.  I'm using a 7 or 8 year old
> system and continuing to do other work while the tests run and they
> finish in 30-45 minutes for me.
>
> Ken
>
>
>
>

> How about that then:
> - mark lengthy test with pragmas
> - add a checkbox in test runner or run all but lengthy by default, and
> then put lengthy tests in separate list, where user could run all of
> them or select which one to run?
>
> Or, by foreseeing more generic use cases, we could scan all test
> methods for pragmas like:
> <testCategory: #foo>
>
> and then give user a selection where he could pick which ones to run or not.
>
>
> 2009/12/4 Ken Causey <[hidden email]>:
>> On Thu, 2009-12-03 at 15:54 -0800, Andreas Raab wrote:
>>> Ken Causey wrote:
>>> > On Thu, 2009-12-03 at 13:36 -0800, Ronald Spengler wrote:
>>> >> Would the people of the community be receptive to moving long running
>>> >> tests out to a separately loadable package?
>>> >
>>> > I fear that separating a subset of tests from the rest will result in
>>> > them being largely forgotten and unused.
>>>
>>> True, but which choice is preferable: Having a few tests never run, or
>>> having no tests ever run? (hey, that rhymes! ;-)
>>>
>>> Cheers,
>>>    - Andreas
>>
>> I think 'never' is the wrong word.  What I'm thinking of is an option to
>> not run the long running tests that defaults to running all tests and
>> actually requires some small effort from the user with hopefully at
>> least a little thought and recognition that the other tests do exist.
>> Separating out the long running tests into a separate package likely
>> means newcomers are never aware of them and the rest of us forget about
>> them, they are no longer maintained, and that's the end of them.  If a
>> test is truly not useful, or at least never worth the running time
>> required, fine just delete it;  but I have to assume that is not
>> generally the case otherwise it would have already happened.
>>
>> I agree that currently running all the tests is not something you do
>> without thought, but it's not that bad.  I'm using a 7 or 8 year old
>> system and continuing to do other work while the tests run and they
>> finish in 30-45 minutes for me.
>>
>> Ken
>>
>>
>>
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
>

> On Thu, 3 Dec 2009, Ronald Spengler wrote:
>
>> Ken: 30 to 45 minutes? Really? I'm experiencing on the order of hours
>> on a dual core 2GHz machine. What gives?
>>
>
> 12.5 minutes on my 2.4GHz core2 duo, and it's getting faster. :)
>
> Levente
>
>

> Maybe something's weird on my end. Checking...
>
> On Thu, Dec 3, 2009 at 5:20 PM, Levente Uzonyi <[hidden email]> wrote:
>> On Thu, 3 Dec 2009, Ronald Spengler wrote:
>>
>>> Ken: 30 to 45 minutes? Really? I'm experiencing on the order of hours
>>> on a dual core 2GHz machine. What gives?
>>>
>>
>> 12.5 minutes on my 2.4GHz core2 duo, and it's getting faster. :)
>>
>> Levente
>>
>>
>
>
>
> --
> Ron
>

> On Thu, Dec 3, 2009 at 5:24 PM, Ronald Spengler <[hidden email]> wrote:
>> Maybe something's weird on my end. Checking...
>>
>> On Thu, Dec 3, 2009 at 5:20 PM, Levente Uzonyi <[hidden email]> wrote:
>>> On Thu, 3 Dec 2009, Ronald Spengler wrote:
>>>
>>>> Ken: 30 to 45 minutes? Really? I'm experiencing on the order of hours
>>>> on a dual core 2GHz machine. What gives?
>>>>
>>>
>>> 12.5 minutes on my 2.4GHz core2 duo, and it's getting faster. :)
>>>
>>> Levente
>>>
>>>
>>
>>
>>
>> --
>> Ron
>>
>
>
>
> --
> Ron
>
>