Re: Recommended SUnit testing tools?

> Date: Thu, 03 May 2012 19:35:47 -0700
> From: Andres Valloud <[hidden email]>
> Subject: Re: [vwnc] Recommended SUnit testing tools?
> To: "[hidden email]" <[hidden email]>
> Message-ID: <[hidden email]>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> You might also want to look at Assessments (bundle of the same name in
> the public Store repository, MIT license).  It offers a much more
> flexible implementation of a basic test framework, which is then used to
> execute tests from three different variants of SUnit (SUnit, SUnitToo,
> and SUnitVM) without needing to modify or override or reparent existing
> test classes.  In addition, it implements test based validation, as well
> as test based benchmarks and performance measurements.  For references,
> see Chapter 4 of "A Mentoring Course on Smalltalk" here:
>
> http://www.lulu.com/shop/search.ep?contributorId=441247
>
> as well as several conference talk slides about these, for example:
>
> http://www.youtube.com/watch?v=jeLGRjQqRf0
>
> and also see for example the paper Extreme Validation here:
>
> http://www.caesarsystems.com/resources/caesarsystems/files/Extreme_Validation.pdf.
>
>
> Assessments' flexible architecture also allows extending Assessments
> without having to override the framework itself.  This is a problem with
> extending SUnit, and it will lead to various extensions being
> incompatible with each other.

> Thought I would try out your Assessments Framework.
>
> I selected the Assessments Self Checks package, clicked Run, after a few minutes got 105 green.
> Tried same with Assessments Tools Self Checks, got 11 results, 2 skipped, 6 errors, 1 failure.
> This is on Mac 10.7.5, VW7.8.
> Perhaps I need a different version? What should I be loading?
>
> Thx,
> Ken
>
> On 2012-05-04, at 1:02 AM, [hidden email] wrote:
>
>> Date: Thu, 03 May 2012 19:35:47 -0700
>> From: Andres Valloud <[hidden email]>
>> Subject: Re: [vwnc] Recommended SUnit testing tools?
>> To: "[hidden email]" <[hidden email]>
>> Message-ID: <[hidden email]>
>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>
>> You might also want to look at Assessments (bundle of the same name in
>> the public Store repository, MIT license).  It offers a much more
>> flexible implementation of a basic test framework, which is then used to
>> execute tests from three different variants of SUnit (SUnit, SUnitToo,
>> and SUnitVM) without needing to modify or override or reparent existing
>> test classes.  In addition, it implements test based validation, as well
>> as test based benchmarks and performance measurements.  For references,
>> see Chapter 4 of "A Mentoring Course on Smalltalk" here:
>>
>> http://www.lulu.com/shop/search.ep?contributorId=441247
>>
>> as well as several conference talk slides about these, for example:
>>
>> http://www.youtube.com/watch?v=jeLGRjQqRf0
>>
>> and also see for example the paper Extreme Validation here:
>>
>> http://www.caesarsystems.com/resources/caesarsystems/files/Extreme_Validation.pdf.
>>
>>
>> Assessments' flexible architecture also allows extending Assessments
>> without having to override the framework itself.  This is a problem with
>> extending SUnit, and it will lead to various extensions being
>> incompatible with each other.
>
> _______________________________________________
> vwnc mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
>

> Those tool self tests fail on purpose to verify various failure modes are detected and handled correctly.  This is similar to the original SUnit self tests that have test methods reading "3 zork", "self assert: 1 + 1 = 3" and similar.
>
> For example, the CheckError section of the results lists various errors because of MNU #whatever.  Similarly, there are attempts to create fractions with zero denominators.  These ensure that various errors do show up in the results viewer as intended.
>
> The next section, for CheckFailure, has a single failure because a check with a selector "checkFailureShowsUpInResultViewer" failed.  This is also intentional.
>
> The next section, for CheckSkip, ensures that checks that do not run because of missing prerequisites (i.e. resources) are reported correctly.  The first noted CheckSkip is because of a prerequisite that fails with an error.  The second CheckSkip is because of a prerequisite that doesn't fail, but doesn't start either.  Note that CheckSkip is a result for a whole checklist (i.e. a test case).
>
> The next section, for PrerequisiteStartFailure, lists a couple prerequisites that fail to start because of various reasons.  These failures to start result in the CheckSkips above.
>
> The last section, for PrerequisiteStopFailure, lists a couple prerequisites that fail to stop presumably after checklists are evaluated.
>
> All these failures simply illustrate that the result viewer is working correctly.
>
> On 9/27/2012 1:41 PM, Ken G. Brown wrote:
>> Thought I would try out your Assessments Framework.
>>
>> I selected the Assessments Self Checks package, clicked Run, after a few minutes got 105 green.
>> Tried same with Assessments Tools Self Checks, got 11 results, 2 skipped, 6 errors, 1 failure.
>> This is on Mac 10.7.5, VW7.8.
>> Perhaps I need a different version? What should I be loading?
>>
>> Thx,
>> Ken
>>
>> On 2012-05-04, at 1:02 AM, [hidden email] wrote:
>>
>>> Date: Thu, 03 May 2012 19:35:47 -0700
>>> From: Andres Valloud <[hidden email]>
>>> Subject: Re: [vwnc] Recommended SUnit testing tools?
>>> To: "[hidden email]" <[hidden email]>
>>> Message-ID: <[hidden email]>
>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>
>>> You might also want to look at Assessments (bundle of the same name in
>>> the public Store repository, MIT license).  It offers a much more
>>> flexible implementation of a basic test framework, which is then used to
>>> execute tests from three different variants of SUnit (SUnit, SUnitToo,
>>> and SUnitVM) without needing to modify or override or reparent existing
>>> test classes.  In addition, it implements test based validation, as well
>>> as test based benchmarks and performance measurements.  For references,
>>> see Chapter 4 of "A Mentoring Course on Smalltalk" here:
>>>
>>> http://www.lulu.com/shop/search.ep?contributorId=441247
>>>
>>> as well as several conference talk slides about these, for example:
>>>
>>> http://www.youtube.com/watch?v=jeLGRjQqRf0
>>>
>>> and also see for example the paper Extreme Validation here:
>>>
>>> http://www.caesarsystems.com/resources/caesarsystems/files/Extreme_Validation.pdf.
>>>
>>>
>>> Assessments' flexible architecture also allows extending Assessments
>>> without having to override the framework itself.  This is a problem with
>>> extending SUnit, and it will lead to various extensions being
>>> incompatible with each other.
>>
>> _______________________________________________
>> vwnc mailing list
>> [hidden email]
>> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
>>

> Thx.
> Wouldn't there be some way to write the checks so that they would end up passing and everything would be green and happiness? As it is, the results look like something is terribly wrong.
>
> Ken
>
>
> On 2012-09-27, at 3:01 PM, Andres Valloud wrote:
>
>> Those tool self tests fail on purpose to verify various failure modes are detected and handled correctly.  This is similar to the original SUnit self tests that have test methods reading "3 zork", "self assert: 1 + 1 = 3" and similar.
>>
>> For example, the CheckError section of the results lists various errors because of MNU #whatever.  Similarly, there are attempts to create fractions with zero denominators.  These ensure that various errors do show up in the results viewer as intended.
>>
>> The next section, for CheckFailure, has a single failure because a check with a selector "checkFailureShowsUpInResultViewer" failed.  This is also intentional.
>>
>> The next section, for CheckSkip, ensures that checks that do not run because of missing prerequisites (i.e. resources) are reported correctly.  The first noted CheckSkip is because of a prerequisite that fails with an error.  The second CheckSkip is because of a prerequisite that doesn't fail, but doesn't start either.  Note that CheckSkip is a result for a whole checklist (i.e. a test case).
>>
>> The next section, for PrerequisiteStartFailure, lists a couple prerequisites that fail to start because of various reasons.  These failures to start result in the CheckSkips above.
>>
>> The last section, for PrerequisiteStopFailure, lists a couple prerequisites that fail to stop presumably after checklists are evaluated.
>>
>> All these failures simply illustrate that the result viewer is working correctly.
>>
>> On 9/27/2012 1:41 PM, Ken G. Brown wrote:
>>> Thought I would try out your Assessments Framework.
>>>
>>> I selected the Assessments Self Checks package, clicked Run, after a few minutes got 105 green.
>>> Tried same with Assessments Tools Self Checks, got 11 results, 2 skipped, 6 errors, 1 failure.
>>> This is on Mac 10.7.5, VW7.8.
>>> Perhaps I need a different version? What should I be loading?
>>>
>>> Thx,
>>> Ken
>>>
>>> On 2012-05-04, at 1:02 AM, [hidden email] wrote:
>>>
>>>> Date: Thu, 03 May 2012 19:35:47 -0700
>>>> From: Andres Valloud <[hidden email]>
>>>> Subject: Re: [vwnc] Recommended SUnit testing tools?
>>>> To: "[hidden email]" <[hidden email]>
>>>> Message-ID: <[hidden email]>
>>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>>
>>>> You might also want to look at Assessments (bundle of the same name in
>>>> the public Store repository, MIT license).  It offers a much more
>>>> flexible implementation of a basic test framework, which is then used to
>>>> execute tests from three different variants of SUnit (SUnit, SUnitToo,
>>>> and SUnitVM) without needing to modify or override or reparent existing
>>>> test classes.  In addition, it implements test based validation, as well
>>>> as test based benchmarks and performance measurements.  For references,
>>>> see Chapter 4 of "A Mentoring Course on Smalltalk" here:
>>>>
>>>> http://www.lulu.com/shop/search.ep?contributorId=441247
>>>>
>>>> as well as several conference talk slides about these, for example:
>>>>
>>>> http://www.youtube.com/watch?v=jeLGRjQqRf0
>>>>
>>>> and also see for example the paper Extreme Validation here:
>>>>
>>>> http://www.caesarsystems.com/resources/caesarsystems/files/Extreme_Validation.pdf.
>>>>
>>>>
>>>> Assessments' flexible architecture also allows extending Assessments
>>>> without having to override the framework itself.  This is a problem with
>>>> extending SUnit, and it will lead to various extensions being
>>>> incompatible with each other.
>>>
>>> _______________________________________________
>>> vwnc mailing list
>>> [hidden email]
>>> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
>>>
>
>