Canonical test cases?

> On 2/23/15, Phil (list) <[hidden email]> wrote:
> > On Mon, 2015-02-23 at 07:53 -0800, Ken.Dickey wrote:
> >> On Mon, 23 Feb 2015 08:38:44 +0000
> >> "H. Hirzel" <[hidden email]> wrote:
> >>
> >> > As Casey writes "The hard part is time and energy to do so."
> >> >
> >> > So what can be done with minimal effort and big impact?
> >>
> >> Looking around, I see that I could make some simple naming changes for
> >> test cases.
> >>
> >> E.g in package Crypto-Nacl there are test cases (class NaclTests)
> >>   testExampleHighlevel -> testNaclHiglLevelAPI
> >>   textExampleLowLevel -> testNaclLowLevelAPI
> >>
> >> We could use other words, but API is shorter than Usage, Example,
> >> Interface.
> >>
> >> Having specific test names end in 'API' would clue me in.
> >>
> >> Adding 'Nacl' means I can use the existing message finder to search for
> >> 'api' and note testNacl*API methods.  This lets me find API tests specific
> >> to Nacl.
> >>
> >> And we can do this as I get the chance.
> >>
> >> Other suggestions?
> >
> > I started playing around with a couple of example test cases to see what
> > I ran into and came up with a distinct class to store all of these per
> > test category (i.e. under Test-Files a class ApiFile which could have a
> > method testWriteTextFile)  The rationale was that it might make sense to
> > keep these test cases separate from traditional test cases which are
> > free to make calls that users of the class (i.e. who are calling the
> > supported API) should not.
>
> Phil,
>
> where do you keep this class ApiFile?
>

> The file testResults.xml which Juan added has
>
> <testcase classname="ApiFile" name="testImagePNG01WriteToFile"
> status="passed"/>
> <testcase classname="ApiFile" name="testImagePNG02ReadFromFile"
> status="passed"/>
> <testcase classname="ApiFile" name="testText01WriteToFile" status="passed"/>
> <testcase classname="ApiFile" name="testText02ReadFromFile" status="passed"/>
>
> So there is an issue of test setup which was not fullfilled in my case....
> Might be a Linux Mint issue (access rights)
>

> On 7/20/15, H. Hirzel <[hidden email]> wrote:
> > On 7/20/15, Phil (list) <[hidden email]> wrote:
> >> Hi Hannes,
> >>
> >> On Sun, 2015-07-19 at 23:52 +0000, H. Hirzel wrote:
> >>> On 2/23/15, Phil (list) <[hidden email]> wrote:
> >>> > On Mon, 2015-02-23 at 07:53 -0800, Ken.Dickey wrote:
> >>> >> On Mon, 23 Feb 2015 08:38:44 +0000
> >>> >> "H. Hirzel" <[hidden email]> wrote:
> >>> >>
> >>> >> > As Casey writes "The hard part is time and energy to do so."
> >>> >> >
> >>> >> > So what can be done with minimal effort and big impact?
> >>> >>
> >>> >> Looking around, I see that I could make some simple naming changes
> >>> >> for
> >>> >> test cases.
> >>> >>
> >>> >> E.g in package Crypto-Nacl there are test cases (class NaclTests)
> >>> >>   testExampleHighlevel -> testNaclHiglLevelAPI
> >>> >>   textExampleLowLevel -> testNaclLowLevelAPI
> >>> >>
> >>> >> We could use other words, but API is shorter than Usage, Example,
> >>> >> Interface.
> >>> >>
> >>> >> Having specific test names end in 'API' would clue me in.
> >>> >>
> >>> >> Adding 'Nacl' means I can use the existing message finder to search
> >>> >> for
> >>> >> 'api' and note testNacl*API methods.  This lets me find API tests
> >>> >> specific
> >>> >> to Nacl.
> >>> >>
> >>> >> And we can do this as I get the chance.
> >>> >>
> >>> >> Other suggestions?
> >>> >
> >>> > I started playing around with a couple of example test cases to see
> >>> > what
> >>> > I ran into and came up with a distinct class to store all of these per
> >>> > test category (i.e. under Test-Files a class ApiFile which could have
> >>> > a
> >>> > method testWriteTextFile)  The rationale was that it might make sense
> >>> > to
> >>> > keep these test cases separate from traditional test cases which are
> >>> > free to make calls that users of the class (i.e. who are calling the
> >>> > supported API) should not.
> >>>
> >>> Phil,
> >>>
> >>> where do you keep this class ApiFile?
> >>>
> >>
> >> It's in Packages/Tests.pck.st package. After loading it will be listed
> >> in the Tests-Files category.
> >
> >
> > I loaded the class ApiFile only and I get 3 tests passed and 1 failure
> > in Cuis 2.4-2424.
> >
> > Can you confirm this, Phil?
> >
> >>> regards
> >>>
> >>> Hannes
> >>>
> >>
> >> Thanks,
> >> Phil
> >>
> >>
> >> _______________________________________________
> >> Cuis mailing list
> >> [hidden email]
> >> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
> >>
> >
> _______________________________________________
> Cuis mailing list
> [hidden email]
> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org

> On Mon, 2015-07-20 at 08:16 +0000, H. Hirzel wrote:
>> The file testResults.xml which Juan added has
>>
>> <testcase classname="ApiFile" name="testImagePNG01WriteToFile"
>> status="passed"/>
>> <testcase classname="ApiFile" name="testImagePNG02ReadFromFile"
>> status="passed"/>
>> <testcase classname="ApiFile" name="testText01WriteToFile"
>> status="passed"/>
>> <testcase classname="ApiFile" name="testText02ReadFromFile"
>> status="passed"/>
>>
>> So there is an issue of test setup which was not fullfilled in my
>> case....
>> Might be a Linux Mint issue (access rights)
>>
>
> I don't have the test in front of me right now but if I recall, the
> setup code exists to create a temp directory in the same directory the
> image was launched from to write the test files. (that was the best way
> I could come up with to not introduce any platform-specific path
> handling issues to the test)
>
> If the setup code failed, then the test would have probably failed.  Did
> you manually do anything or did it just work the 2nd time you ran it?
> (if it re-ran successfully without you needing to do anything else, it
> definitely sounds like a problem in the test setup.  If it were a rights
> issue, it should have continued to fail.)
>
> The good news is we're about to change all the file/directory handling
> code so it needs to be updated anyway...
>
>> On 7/20/15, H. Hirzel <[hidden email]> wrote:
>> > On 7/20/15, Phil (list) <[hidden email]> wrote:
>> >> Hi Hannes,
>> >>
>> >> On Sun, 2015-07-19 at 23:52 +0000, H. Hirzel wrote:
>> >>> On 2/23/15, Phil (list) <[hidden email]> wrote:
>> >>> > On Mon, 2015-02-23 at 07:53 -0800, Ken.Dickey wrote:
>> >>> >> On Mon, 23 Feb 2015 08:38:44 +0000
>> >>> >> "H. Hirzel" <[hidden email]> wrote:
>> >>> >>
>> >>> >> > As Casey writes "The hard part is time and energy to do so."
>> >>> >> >
>> >>> >> > So what can be done with minimal effort and big impact?
>> >>> >>
>> >>> >> Looking around, I see that I could make some simple naming changes
>> >>> >> for
>> >>> >> test cases.
>> >>> >>
>> >>> >> E.g in package Crypto-Nacl there are test cases (class NaclTests)
>> >>> >>   testExampleHighlevel -> testNaclHiglLevelAPI
>> >>> >>   textExampleLowLevel -> testNaclLowLevelAPI
>> >>> >>
>> >>> >> We could use other words, but API is shorter than Usage, Example,
>> >>> >> Interface.
>> >>> >>
>> >>> >> Having specific test names end in 'API' would clue me in.
>> >>> >>
>> >>> >> Adding 'Nacl' means I can use the existing message finder to
>> >>> >> search
>> >>> >> for
>> >>> >> 'api' and note testNacl*API methods.  This lets me find API tests
>> >>> >> specific
>> >>> >> to Nacl.
>> >>> >>
>> >>> >> And we can do this as I get the chance.
>> >>> >>
>> >>> >> Other suggestions?
>> >>> >
>> >>> > I started playing around with a couple of example test cases to see
>> >>> > what
>> >>> > I ran into and came up with a distinct class to store all of these
>> >>> > per
>> >>> > test category (i.e. under Test-Files a class ApiFile which could
>> >>> > have
>> >>> > a
>> >>> > method testWriteTextFile)  The rationale was that it might make
>> >>> > sense
>> >>> > to
>> >>> > keep these test cases separate from traditional test cases which
>> >>> > are
>> >>> > free to make calls that users of the class (i.e. who are calling
>> >>> > the
>> >>> > supported API) should not.
>> >>>
>> >>> Phil,
>> >>>
>> >>> where do you keep this class ApiFile?
>> >>>
>> >>
>> >> It's in Packages/Tests.pck.st package. After loading it will be listed
>> >> in the Tests-Files category.
>> >
>> >
>> > I loaded the class ApiFile only and I get 3 tests passed and 1 failure
>> > in Cuis 2.4-2424.
>> >
>> > Can you confirm this, Phil?
>> >
>> >>> regards
>> >>>
>> >>> Hannes
>> >>>
>> >>
>> >> Thanks,
>> >> Phil
>> >>
>> >>
>> >> _______________________________________________
>> >> Cuis mailing list
>> >> [hidden email]
>> >> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
>> >>
>> >
>> _______________________________________________
>> Cuis mailing list
>> [hidden email]
>> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
>
>
>
> _______________________________________________
> Cuis mailing list
> [hidden email]
> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
>

> --Hannes
>
> On 7/20/15, Phil (list) <[hidden email]> wrote:
> > On Mon, 2015-07-20 at 08:16 +0000, H. Hirzel wrote:
> >> The file testResults.xml which Juan added has
> >>
> >> <testcase classname="ApiFile" name="testImagePNG01WriteToFile"
> >> status="passed"/>
> >> <testcase classname="ApiFile" name="testImagePNG02ReadFromFile"
> >> status="passed"/>
> >> <testcase classname="ApiFile" name="testText01WriteToFile"
> >> status="passed"/>
> >> <testcase classname="ApiFile" name="testText02ReadFromFile"
> >> status="passed"/>
> >>
> >> So there is an issue of test setup which was not fullfilled in my
> >> case....
> >> Might be a Linux Mint issue (access rights)
> >>
> >
> > I don't have the test in front of me right now but if I recall, the
> > setup code exists to create a temp directory in the same directory the
> > image was launched from to write the test files. (that was the best way
> > I could come up with to not introduce any platform-specific path
> > handling issues to the test)
> >
> > If the setup code failed, then the test would have probably failed.  Did
> > you manually do anything or did it just work the 2nd time you ran it?
> > (if it re-ran successfully without you needing to do anything else, it
> > definitely sounds like a problem in the test setup.  If it were a rights
> > issue, it should have continued to fail.)
> >
> > The good news is we're about to change all the file/directory handling
> > code so it needs to be updated anyway...
> >
> >> On 7/20/15, H. Hirzel <[hidden email]> wrote:
> >> > On 7/20/15, Phil (list) <[hidden email]> wrote:
> >> >> Hi Hannes,
> >> >>
> >> >> On Sun, 2015-07-19 at 23:52 +0000, H. Hirzel wrote:
> >> >>> On 2/23/15, Phil (list) <[hidden email]> wrote:
> >> >>> > On Mon, 2015-02-23 at 07:53 -0800, Ken.Dickey wrote:
> >> >>> >> On Mon, 23 Feb 2015 08:38:44 +0000
> >> >>> >> "H. Hirzel" <[hidden email]> wrote:
> >> >>> >>
> >> >>> >> > As Casey writes "The hard part is time and energy to do so."
> >> >>> >> >
> >> >>> >> > So what can be done with minimal effort and big impact?
> >> >>> >>
> >> >>> >> Looking around, I see that I could make some simple naming changes
> >> >>> >> for
> >> >>> >> test cases.
> >> >>> >>
> >> >>> >> E.g in package Crypto-Nacl there are test cases (class NaclTests)
> >> >>> >>   testExampleHighlevel -> testNaclHiglLevelAPI
> >> >>> >>   textExampleLowLevel -> testNaclLowLevelAPI
> >> >>> >>
> >> >>> >> We could use other words, but API is shorter than Usage, Example,
> >> >>> >> Interface.
> >> >>> >>
> >> >>> >> Having specific test names end in 'API' would clue me in.
> >> >>> >>
> >> >>> >> Adding 'Nacl' means I can use the existing message finder to
> >> >>> >> search
> >> >>> >> for
> >> >>> >> 'api' and note testNacl*API methods.  This lets me find API tests
> >> >>> >> specific
> >> >>> >> to Nacl.
> >> >>> >>
> >> >>> >> And we can do this as I get the chance.
> >> >>> >>
> >> >>> >> Other suggestions?
> >> >>> >
> >> >>> > I started playing around with a couple of example test cases to see
> >> >>> > what
> >> >>> > I ran into and came up with a distinct class to store all of these
> >> >>> > per
> >> >>> > test category (i.e. under Test-Files a class ApiFile which could
> >> >>> > have
> >> >>> > a
> >> >>> > method testWriteTextFile)  The rationale was that it might make
> >> >>> > sense
> >> >>> > to
> >> >>> > keep these test cases separate from traditional test cases which
> >> >>> > are
> >> >>> > free to make calls that users of the class (i.e. who are calling
> >> >>> > the
> >> >>> > supported API) should not.
> >> >>>
> >> >>> Phil,
> >> >>>
> >> >>> where do you keep this class ApiFile?
> >> >>>
> >> >>
> >> >> It's in Packages/Tests.pck.st package. After loading it will be listed
> >> >> in the Tests-Files category.
> >> >
> >> >
> >> > I loaded the class ApiFile only and I get 3 tests passed and 1 failure
> >> > in Cuis 2.4-2424.
> >> >
> >> > Can you confirm this, Phil?
> >> >
> >> >>> regards
> >> >>>
> >> >>> Hannes
> >> >>>
> >> >>
> >> >> Thanks,
> >> >> Phil
> >> >>
> >> >>
> >> >> _______________________________________________
> >> >> Cuis mailing list
> >> >> [hidden email]
> >> >> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
> >> >>
> >> >
> >> _______________________________________________
> >> Cuis mailing list
> >> [hidden email]
> >> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
> >
> >
> >
> > _______________________________________________
> > Cuis mailing list
> > [hidden email]
> > http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
> >
>
> _______________________________________________
> Cuis mailing list
> [hidden email]
> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org

> On Mon, 2015-07-20 at 08:45 +0000, H. Hirzel wrote:
>> The reason was
>>
>> I only cherry picked the ApiFile class by manual loading.
>>
>> After adding  Graphics-Files-Additional.pck.st manually it worked.
>>
>
> That would do it.  By adding 'manually' do you mean that you didn't have
> the Graphics package downloaded at the time you loaded Tests?  That's
> the only reason I can think of that the package manager wouldn't have
> loaded Graphics for you automatically given the dependency.
>
>> Another interesting test case is to write out a StrikeFont to the file
>> system in the proper
>> canonical export format (bmp and text) with a new name  and read it
>> back as a new font.
>>
>
> That's the sort of thing I was getting at with these tests: if there is
> some useful bit of work that you need to be able to do (especially
> something larger than a unit test and/or crossing package boundaries),
> to create a test for it to help Juan understand the functionality we're
> using.  This was one way I came up with to try to get to some sort of
> agreed upon public API.  In retrospect, I probably put the cart before
> the horse as doing all of these tests will take considerably longer than
> initially flagging methods considered to be public.  But hopefully you
> can see how doing both could complement each other.  To reiterate: this
> work is on us to do up front, Juan would just review and commit.
>
>> --Hannes
>>
>> On 7/20/15, Phil (list) <[hidden email]> wrote:
>> > On Mon, 2015-07-20 at 08:16 +0000, H. Hirzel wrote:
>> >> The file testResults.xml which Juan added has
>> >>
>> >> <testcase classname="ApiFile" name="testImagePNG01WriteToFile"
>> >> status="passed"/>
>> >> <testcase classname="ApiFile" name="testImagePNG02ReadFromFile"
>> >> status="passed"/>
>> >> <testcase classname="ApiFile" name="testText01WriteToFile"
>> >> status="passed"/>
>> >> <testcase classname="ApiFile" name="testText02ReadFromFile"
>> >> status="passed"/>
>> >>
>> >> So there is an issue of test setup which was not fullfilled in my
>> >> case....
>> >> Might be a Linux Mint issue (access rights)
>> >>
>> >
>> > I don't have the test in front of me right now but if I recall, the
>> > setup code exists to create a temp directory in the same directory the
>> > image was launched from to write the test files. (that was the best way
>> > I could come up with to not introduce any platform-specific path
>> > handling issues to the test)
>> >
>> > If the setup code failed, then the test would have probably failed.
>> > Did
>> > you manually do anything or did it just work the 2nd time you ran it?
>> > (if it re-ran successfully without you needing to do anything else, it
>> > definitely sounds like a problem in the test setup.  If it were a
>> > rights
>> > issue, it should have continued to fail.)
>> >
>> > The good news is we're about to change all the file/directory handling
>> > code so it needs to be updated anyway...
>> >
>> >> On 7/20/15, H. Hirzel <[hidden email]> wrote:
>> >> > On 7/20/15, Phil (list) <[hidden email]> wrote:
>> >> >> Hi Hannes,
>> >> >>
>> >> >> On Sun, 2015-07-19 at 23:52 +0000, H. Hirzel wrote:
>> >> >>> On 2/23/15, Phil (list) <[hidden email]> wrote:
>> >> >>> > On Mon, 2015-02-23 at 07:53 -0800, Ken.Dickey wrote:
>> >> >>> >> On Mon, 23 Feb 2015 08:38:44 +0000
>> >> >>> >> "H. Hirzel" <[hidden email]> wrote:
>> >> >>> >>
>> >> >>> >> > As Casey writes "The hard part is time and energy to do so."
>> >> >>> >> >
>> >> >>> >> > So what can be done with minimal effort and big impact?
>> >> >>> >>
>> >> >>> >> Looking around, I see that I could make some simple naming
>> >> >>> >> changes
>> >> >>> >> for
>> >> >>> >> test cases.
>> >> >>> >>
>> >> >>> >> E.g in package Crypto-Nacl there are test cases (class
>> >> >>> >> NaclTests)
>> >> >>> >>   testExampleHighlevel -> testNaclHiglLevelAPI
>> >> >>> >>   textExampleLowLevel -> testNaclLowLevelAPI
>> >> >>> >>
>> >> >>> >> We could use other words, but API is shorter than Usage,
>> >> >>> >> Example,
>> >> >>> >> Interface.
>> >> >>> >>
>> >> >>> >> Having specific test names end in 'API' would clue me in.
>> >> >>> >>
>> >> >>> >> Adding 'Nacl' means I can use the existing message finder to
>> >> >>> >> search
>> >> >>> >> for
>> >> >>> >> 'api' and note testNacl*API methods.  This lets me find API
>> >> >>> >> tests
>> >> >>> >> specific
>> >> >>> >> to Nacl.
>> >> >>> >>
>> >> >>> >> And we can do this as I get the chance.
>> >> >>> >>
>> >> >>> >> Other suggestions?
>> >> >>> >
>> >> >>> > I started playing around with a couple of example test cases to
>> >> >>> > see
>> >> >>> > what
>> >> >>> > I ran into and came up with a distinct class to store all of
>> >> >>> > these
>> >> >>> > per
>> >> >>> > test category (i.e. under Test-Files a class ApiFile which could
>> >> >>> > have
>> >> >>> > a
>> >> >>> > method testWriteTextFile)  The rationale was that it might make
>> >> >>> > sense
>> >> >>> > to
>> >> >>> > keep these test cases separate from traditional test cases which
>> >> >>> > are
>> >> >>> > free to make calls that users of the class (i.e. who are calling
>> >> >>> > the
>> >> >>> > supported API) should not.
>> >> >>>
>> >> >>> Phil,
>> >> >>>
>> >> >>> where do you keep this class ApiFile?
>> >> >>>
>> >> >>
>> >> >> It's in Packages/Tests.pck.st package. After loading it will be
>> >> >> listed
>> >> >> in the Tests-Files category.
>> >> >
>> >> >
>> >> > I loaded the class ApiFile only and I get 3 tests passed and 1
>> >> > failure
>> >> > in Cuis 2.4-2424.
>> >> >
>> >> > Can you confirm this, Phil?
>> >> >
>> >> >>> regards
>> >> >>>
>> >> >>> Hannes
>> >> >>>
>> >> >>
>> >> >> Thanks,
>> >> >> Phil
>> >> >>
>> >> >>
>> >> >> _______________________________________________
>> >> >> Cuis mailing list
>> >> >> [hidden email]
>> >> >> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
>> >> >>
>> >> >
>> >> _______________________________________________
>> >> Cuis mailing list
>> >> [hidden email]
>> >> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
>> >
>> >
>> >
>> > _______________________________________________
>> > Cuis mailing list
>> > [hidden email]
>> > http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
>> >
>>
>> _______________________________________________
>> Cuis mailing list
>> [hidden email]
>> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
>
>
>
> _______________________________________________
> Cuis mailing list
> [hidden email]
> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
>