Smalltalk › Cincom › VisualWorks

Problem with SUnit test suite usage....

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

19 messages Options

Rick Flower

Problem with SUnit test suite usage....

Good evening all.. I'm muddling my way through learning how to use the
SUnit framework specifically for use with Seaside+Glorp on VW7.4..
In particular, I'm trying to use the TestSuite object to create a higher
level test suite of the various test cases I want run and to also use a
resource to setup my database connection,etc..

So, I'm slowly getting somewhere, I just wish I knew where the heck I'm
going.. I've found several odd things that don't seem to make much
sense to me :

1) Defining a testsuite in VW doesn't seem to show up in the provided
TestRunner -- it only looks for code that uses TestCases.. Nothing in
there I believe talks to TestSuites unless you run ALL of the testSuites
which I'd rather not do.. So, this leaves me to using a Workspace to
invoke my testsuite which is fine for now, but I might decide to modify
the TestRunner to be a bit smarter unless someone has a good reason why
it ignores TestSuites or is there some sort of magic potion to make them
appear..?

2) I've subclassed SUnit.TestSuite and have in that class an
initialization method that looks like the following (and the resource
method below that):

MSADBAllTests>>initialize
super initialize.
self addTest: MSADBLocationLookupTest.
self name: 'AAARick Test'.

MSADBAllTests>>resources
^ Array with: MSADBAllResources

I then set a breakpoint in the initialize method and then I execute the
following in the workspace :

tf := MSADBAllTests new.

However, no breakpoint is triggered which coincides with what I see when
I inspect the "tf" object -- no name is set or much of anything else (no
tests either -- the unit test collection is empty). This seems to
indicate that the initialize method is not being called on object
creation. I thought ALL objects call an initialize method in ST if it
was available -- am I mistaken here? (sorry -- newbie question)

Either there's something really wrong with VW or I'm overlooking
something really stupid -- I'm sure it's me and not VW, so perhaps
someone can point me in the right direction here..

Any input would be greatly appreciated!

-- Rick

Alan Knight-2

Re: Problem with SUnit test suite usage....

I think the test runner builds its own suite of all test cases in the image. What I normally do is use the RBSunitExtensions. With that loaded you can select a package or group of package, or class/group of classes, and the same with methods. If they are tests, then you get run/debug/etc buttons at the bottom of the browser window.

The #initialize method is only sent if there's a #new method that calls it. That's a common pattern, but not universal. So you probably need to add a class-side new method as
new
^super new initialize.

Note that if there wasn't a new that did that, then there probably also wasn't a superclass implementation of initialize, so your initialize method won't want to call super.

At 12:50 AM 08/07/2006, Rick Flower wrote:

>Good evening all.. I'm muddling my way through learning how to use the SUnit framework specifically for use with Seaside+Glorp on VW7.4..
>In particular, I'm trying to use the TestSuite object to create a higher level test suite of the various test cases I want run and to also use a resource to setup my database connection,etc..
>
>So, I'm slowly getting somewhere, I just wish I knew where the heck I'm going.. I've found several odd things that don't seem to make much sense to me :
>
>1) Defining a testsuite in VW doesn't seem to show up in the provided TestRunner -- it only looks for code that uses TestCases.. Nothing in there I believe talks to TestSuites unless you run ALL of the testSuites which I'd rather not do.. So, this leaves me to using a Workspace to invoke my testsuite which is fine for now, but I might decide to modify the TestRunner to be a bit smarter unless someone has a good reason why it ignores TestSuites or is there some sort of magic potion to make them appear..?
>
>2) I've subclassed SUnit.TestSuite and have in that class an initialization method that looks like the following (and the resource method below that):
>
>MSADBAllTests>>initialize
> super initialize.
> self addTest: MSADBLocationLookupTest.
> self name: 'AAARick Test'.
>
>MSADBAllTests>>resources
> ^ Array with: MSADBAllResources
>
>I then set a breakpoint in the initialize method and then I execute the following in the workspace :
>
>tf := MSADBAllTests new.
>
>However, no breakpoint is triggered which coincides with what I see when I inspect the "tf" object -- no name is set or much of anything else (no tests either -- the unit test collection is empty). This seems to indicate that the initialize method is not being called on object creation. I thought ALL objects call an initialize method in ST if it was available -- am I mistaken here? (sorry -- newbie question)
>
>Either there's something really wrong with VW or I'm overlooking something really stupid -- I'm sure it's me and not VW, so perhaps someone can point me in the right direction here..
>
>Any input would be greatly appreciated!
>
>-- Rick

--
Alan Knight [|], Cincom Smalltalk Development
[hidden email]
[hidden email]
http://www.cincom.com/smalltalk

"The Static Typing Philosophy: Make it fast. Make it right. Make it run." - Niall Ross

Rick Flower

Re: Problem with SUnit test suite usage....

Alan Knight wrote:
> I think the test runner builds its own suite of all test cases in the image. What I normally do is use the RBSunitExtensions. With that loaded you can select a package or group of package, or class/group of classes, and the same with methods. If they are tests, then you get run/debug/etc buttons at the bottom of the browser window.
>
> The #initialize method is only sent if there's a #new method that calls it. That's a common pattern, but not universal. So you probably need to add a class-side new method as
> new
> ^super new initialize.
>
> Note that if there wasn't a new that did that, then there probably also wasn't a superclass implementation of initialize, so your initialize method won't want to call super.

Thanks.. That did the trick.. I figured it was something rather silly
like that which I was missing.. Once I put the class-side new in place,
as indicated above, my initialize method was called and all is good
now.. Thanks again!

Rick Flower

Re: Problem with SUnit test suite usage....

In reply to this post by Alan Knight-2

Alan Knight wrote:
> I think the test runner builds its own suite of all test cases in the image.
> What I normally do is use the RBSunitExtensions. With that loaded you
can
> select a package or group of package, or class/group of classes, and the
> same with methods. If they are tests, then you get run/debug/etc buttons
> at the bottom of the browser window.

I forgot to ask -- is the version checked into the Cincom repository the
latest version (dated sometime in 2004 if I recall)? I briefly loaded
it last night into an image I had but didn't notice any obvious changes
to the regular system browser. I'll try loading it again and take a
closer look. Thanks!

Terry Raymond

RE: Problem with SUnit test suite usage....

In reply to this post by Rick Flower

Rick

You might want to look at SUnitToo in the public repository.
Travis Griggs has made extensions to SUnit that you might like.

Terry

===========================================================
Terry Raymond Smalltalk Professional Debug Package
Crafted Smalltalk
80 Lazywood Ln.
Tiverton, RI 02878
(401) 624-4517 [hidden email]
<http://www.craftedsmalltalk.com>
===========================================================

> -----Original Message-----
> From: Rick Flower [mailto:[hidden email]]
> Sent: Saturday, July 08, 2006 12:51 AM
> To: Visualworks Mailing List
> Subject: Problem with SUnit test suite usage....
>
> Good evening all.. I'm muddling my way through learning how to use the
> SUnit framework specifically for use with Seaside+Glorp on VW7.4..
> In particular, I'm trying to use the TestSuite object to create a higher
> level test suite of the various test cases I want run and to also use a
> resource to setup my database connection,etc..
>
> So, I'm slowly getting somewhere, I just wish I knew where the heck I'm
> going.. I've found several odd things that don't seem to make much
> sense to me :
>
> 1) Defining a testsuite in VW doesn't seem to show up in the provided
> TestRunner -- it only looks for code that uses TestCases.. Nothing in
> there I believe talks to TestSuites unless you run ALL of the testSuites
> which I'd rather not do.. So, this leaves me to using a Workspace to
> invoke my testsuite which is fine for now, but I might decide to modify
> the TestRunner to be a bit smarter unless someone has a good reason why
> it ignores TestSuites or is there some sort of magic potion to make them
> appear..?
>
> 2) I've subclassed SUnit.TestSuite and have in that class an
> initialization method that looks like the following (and the resource
> method below that):
>
> MSADBAllTests>>initialize
> super initialize.
> self addTest: MSADBLocationLookupTest.
> self name: 'AAARick Test'.
>
> MSADBAllTests>>resources
> ^ Array with: MSADBAllResources
>
> I then set a breakpoint in the initialize method and then I execute the
> following in the workspace :
>
> tf := MSADBAllTests new.
>
> However, no breakpoint is triggered which coincides with what I see when
> I inspect the "tf" object -- no name is set or much of anything else (no
> tests either -- the unit test collection is empty). This seems to
> indicate that the initialize method is not being called on object
> creation. I thought ALL objects call an initialize method in ST if it
> was available -- am I mistaken here? (sorry -- newbie question)
>
> Either there's something really wrong with VW or I'm overlooking
> something really stupid -- I'm sure it's me and not VW, so perhaps
> someone can point me in the right direction here..
>
> Any input would be greatly appreciated!
>
> -- Rick

Terry Raymond

RE: Problem with SUnit test suite usage....

In reply to this post by Rick Flower

Rick

I should have added that Dave Buck just had a blog article about SUnitToo.

http://www.cincomsmalltalk.com/userblogs/buck/blogView?showComments=true&ent
ry=3329680495

Terry

===========================================================
Terry Raymond Smalltalk Professional Debug Package
Crafted Smalltalk
80 Lazywood Ln.
Tiverton, RI 02878
(401) 624-4517 [hidden email]
<http://www.craftedsmalltalk.com>
===========================================================

Alan Knight-2

Re: Problem with SUnit test suite usage....

In reply to this post by Rick Flower

At 03:52 PM 08/07/2006, Rick Flower wrote:
>Alan Knight wrote:
>>I think the test runner builds its own suite of all test cases in the image.
>> What I normally do is use the RBSunitExtensions. With that loaded you can
>> select a package or group of package, or class/group of classes, and the
>> same with methods. If they are tests, then you get run/debug/etc buttons
>> at the bottom of the browser window.
>
>I forgot to ask -- is the version checked into the Cincom repository the latest version (dated sometime in 2004 if I recall)? I briefly loaded it last night into an image I had but didn't notice any obvious changes to the regular system browser. I'll try loading it again and take a closer look. Thanks!

I think that version will do. That's been pretty stable for a while. Some people have made tweaks, but for basic usage they probably aren't important. It's also included in contributed on the CD, so you don't need to go the repository to get it, but can just load it as a parcel.

--
Alan Knight [|], Cincom Smalltalk Development
[hidden email]
[hidden email]
http://www.cincom.com/smalltalk

"The Static Typing Philosophy: Make it fast. Make it right. Make it run." - Niall Ross

Rick Flower

Re: Problem with SUnit test suite usage....

In reply to this post by Terry Raymond

Terry Raymond wrote:
> Rick
>
> You might want to look at SUnitToo in the public repository.
> Travis Griggs has made extensions to SUnit that you might like.

I've got a TestCase/TestSuite question I'm hoping someone can shed some
light on.. If I've got a TestSuite (or TestCase) that has defined a
"resource" method that lists the resources needed to run the tests and
that resource object gets it's "setUp" method invoked -- is there a way
to have the TestCase being executed gain access to the resource object
directly? I've got tests that have a database connection which is
established (and tables created) for my unit tests and ideally, these
tests need to get the db-session from that resource object in order to
talk to the database.. Is there some way to do this?

Initially I thought I could query the TestCase (e.g. self resources) to
get that, but quickly determined that I only get the initial list of
resources I already specified(duh!).. Anyway, is there some way to do
this? I didn't see any obvious way with either the vanilla SUnit or
SUnitToo variant since there are no specific object hierarchy between
say a TestCase --> TestSuite.. Any ideas?

Rick Flower

Re: Problem with SUnit test suite usage....

Nevermind.. Solved my problem..

Rick Flower wrote:

> Terry Raymond wrote:
>> Rick
>>
>> You might want to look at SUnitToo in the public repository.
>> Travis Griggs has made extensions to SUnit that you might like.
>
> I've got a TestCase/TestSuite question I'm hoping someone can shed some
> light on.. If I've got a TestSuite (or TestCase) that has defined a
> "resource" method that lists the resources needed to run the tests and
> that resource object gets it's "setUp" method invoked -- is there a way
> to have the TestCase being executed gain access to the resource object
> directly? I've got tests that have a database connection which is
> established (and tables created) for my unit tests and ideally, these
> tests need to get the db-session from that resource object in order to
> talk to the database.. Is there some way to do this?
>
> Initially I thought I could query the TestCase (e.g. self resources) to
> get that, but quickly determined that I only get the initial list of
> resources I already specified(duh!).. Anyway, is there some way to do
> this? I didn't see any obvious way with either the vanilla SUnit or
> SUnitToo variant since there are no specific object hierarchy between
> say a TestCase --> TestSuite.. Any ideas?
>
>

Steven Kelly

RE: Problem with SUnit test suite usage....

In reply to this post by Rick Flower

Rick,

Hope I understood you correctly. This is what I do:

MyTestCase class>>resources
^Array with: MyTestResource

MyTestResource>>setup
"set up the db and store it in an instance variable"
myTestDB := MyDBConnection new.

MyTestCase>>myTestDB
"grab the db from the resource"
^self resources asArray first myTestDB

MyTestCase>>testDB
self assert: self myTestDB worksFine

If you find yourself writing "self resources as Array first", add this:
TestCase>>resource
^self resources as Array first

HTH,
Steve

> -----Original Message-----
> From: Rick Flower [mailto:[hidden email]]
> Sent: 11 July 2006 01:35
> To: Terry Raymond
> Cc: 'Visualworks Mailing List'
> Subject: Re: Problem with SUnit test suite usage....
>
> Terry Raymond wrote:
> > Rick
> >
> > You might want to look at SUnitToo in the public repository.
> > Travis Griggs has made extensions to SUnit that you might like.
>
> I've got a TestCase/TestSuite question I'm hoping someone can shed

some
> light on.. If I've got a TestSuite (or TestCase) that has defined a
> "resource" method that lists the resources needed to run the tests and
> that resource object gets it's "setUp" method invoked -- is there a
way
> to have the TestCase being executed gain access to the resource object
> directly? I've got tests that have a database connection which is
> established (and tables created) for my unit tests and ideally, these
> tests need to get the db-session from that resource object in order to
> talk to the database.. Is there some way to do this?
>
> Initially I thought I could query the TestCase (e.g. self resources)
to
> get that, but quickly determined that I only get the initial list of
> resources I already specified(duh!).. Anyway, is there some way to do
> this? I didn't see any obvious way with either the vanilla SUnit or
> SUnitToo variant since there are no specific object hierarchy between
> say a TestCase --> TestSuite.. Any ideas?

Ladislav Lenart

Re: Problem with SUnit test suite usage....

Steven Kelly wrote:

> Rick,
>
> Hope I understood you correctly. This is what I do:
>
> MyTestCase class>>resources
> ^Array with: MyTestResource
>
> MyTestResource>>setup
> "set up the db and store it in an instance variable"
> myTestDB := MyDBConnection new.
>
> MyTestCase>>myTestDB
> "grab the db from the resource"
> ^self resources asArray first myTestDB
>
> MyTestCase>>testDB
> self assert: self myTestDB worksFine
>
> If you find yourself writing "self resources as Array first", add this:
> TestCase>>resource
> ^self resources as Array first

Well,

and what about this

TestCase>>resource
^MyResource current

The #resources method is there only for specifying prerequisites that
need to be available in order to run the test. But when they are, they
are all accessible via #current, because they are effectively singletons...

This is the way it works in both SUnit and SUnitToo.

Ladislav Lenart

Rick Flower

Re: Problem with SUnit test suite usage....

Ladislav Lenart wrote:

> Well,
>
> and what about this
>
> TestCase>>resource
> ^MyResource current
>
> The #resources method is there only for specifying prerequisites that
> need to be available in order to run the test. But when they are, they
> are all accessible via #current, because they are effectively singletons...
>
> This is the way it works in both SUnit and SUnitToo.

I was able to get this whole thing behaving after reading a file I found
on the web called "Pelrine on TestResources.pdf" dated 12-July-2005 (he
also has an earlier version (dated in 2003) of this or a related
document that isn't nearly as interesting) and he discussed defining a
'resource' method (as opposed to the required 'resources' class side
method) and I'm using exactly what is shown above, but I'll have to
admit that I don't really follow what it's doing. I didn't know what
'current' was or why/how it can return what appears from my C++ eyes to
be an item from a static class of sorts since "MyResource" wasn't really
instantiated from what I can tell -- at least not from the TestCases
point of view... If anyone can explain how this works, I'd be
interested in learning what it's doing..

Thanks!

Steven Kelly

RE: Problem with SUnit test suite usage....

In reply to this post by Rick Flower

Steven Kelly wrote:
> > TestCase>>resource
> > ^self resources as Array first

From: Ladislav Lenart [mailto:[hidden email]]
> TestCase>>resource
> ^MyResource current
>
> The #resources method is there only for specifying prerequisites that
> need to be available in order to run the test. But when they are, they
> are all accessible via #current, because they are effectively
singletons...

Interesting! I know where my way comes from: inspecting a TestCase and
its methods to try to find how to get from it to its resources. I didn't
like the answer then, and I don't like it much better now. Your way is
neater, but has the minor problem that test cases can then reference
resources that they haven't "declared" in the class side #resources
method. Both have the problem that the resource is a singleton - a case
cannot have two instances of the same resource, nor are resources thread
safe.

I've also noticed that my resources tend to have ivs, for each of which
I have to make accessors in both the resource and the test case. Roughly
speaking, a resource here is a database connection, and each iv is for
some particular persistent thing in the database. Maybe I should be
making each iv into its own resource?

For me, it would make sense if a TestSuite held on to its TestResources
(instances, not classes like it does now), and a TestCase being run
would know which TestSuite it belonged to, so it could access resources
through that. Resources would be named with symbols by their TestCase,
so we could know what we were referring to, and that it must be declared
to be used:

MyTestCase class>>resources
^Dictionary
with: #clientEndpoint -> EndpointResource
with: #serverEndpoint -> EndpointResource

MyTestCase>>test1
self assert: self clientEndpoint isAlive

MyTestCase>>clientEndpoint
self testSuite resources at: #clientEndpoint

But if we're going that far, why have the ugliness of the #resources
method and corresponding method for each resource? The TestSuite will
have to initialize the MyTestCase instance anyway, to set itself into
the testSuite iv. So why not:

MyTestCase>>clientEndpoint
<resource: EndpointResource>
^clientEndpoint

The test suite would find all the methods marked as resource methods,
create and store an instance of the named resource class in a dict,
keyed by the method selector, and then start running tests. When it
creates an instance of MyTestCase, it sets the iv of that name with its
resource of the same name.

This way each resource is declared and used in the same place, and
access to them is simple and standard Smalltalk, direct from the
TestCase. Whilst I'm not a great fan of magic like pragmas, there's no
more magic here than in the current system of initializing and accessing
resources. Those who like more magic could move the declaration of all
resources to one method on the class side, and have doesNotUnderstand:
forward calls to #clientEndpoint etc. to the test suite resources,
getting rid of the need for an iv per resource in the test case.

Have I got the idea of resources all wrong, or is there an easy way in
the current framework to achieve similar results?

Cheers,
Steve

Reinout Heeck-2

Re: Problem with SUnit test suite usage....

Steven Kelly wrote:

> For me, it would make sense if [SUnit suggestions...]

Steven, Joseph Pelrine is the maintainer of SUnit and I have heard that
he is starting work on a new iteration.
I suggest you CC your suggestions directly to him since I don't expect
he'll be following the vwnc list closely. (jpelrine at metaprog dot com)

R
-

Ladislav Lenart

Re: Problem with SUnit test suite usage....

In reply to this post by Steven Kelly

Steven Kelly wrote:

>>> TestCase>>resource
>>> ^self resources as Array first
>
>
>> TestCase>>resource
>> ^MyResource current
>>
>> The #resources method is there only for specifying prerequisites that
>> need to be available in order to run the test. But when they are, they
>> are all accessible via #current, because they are effectively
>> singletons...
>
> Interesting! I know where my way comes from: inspecting a TestCase and
> its methods to try to find how to get from it to its resources. I didn't
> like the answer then, and I don't like it much better now. Your way is
> neater, but has the minor problem that test cases can then reference
> resources that they haven't "declared" in the class side #resources
> method.

Well, you're right but the same problem is for example with package
prerequisites. In the end, you have to specify it by hand (at least
you have to click on compute prereqs and resolve cycles), but no one
prevents you from using classes that are not in the package prerequisites
chain.

> Both have the problem that the resource is a singleton - a case
> cannot have two instances of the same resource, nor are resources thread
> safe.
I don't see the point where thread-safeness would matter, because I only
run one TestSuite at a time and I don't think this is the only problem
that prevents running multiple tests at the same time...

But your first objection (resource is a singleton) - I completely agree.

> I've also noticed that my resources tend to have ivs, for each of which
> I have to make accessors in both the resource and the test case. Roughly
> speaking, a resource here is a database connection, and each iv is for
> some particular persistent thing in the database. Maybe I should be
> making each iv into its own resource?
>
> For me, it would make sense if a TestSuite held on to its TestResources
> (instances, not classes like it does now), and a TestCase being run
> would know which TestSuite it belonged to, so it could access resources
> through that. Resources would be named with symbols by their TestCase,
> so we could know what we were referring to, and that it must be declared
> to be used:
>
> MyTestCase class>>resources
> ^Dictionary
> with: #clientEndpoint -> EndpointResource
> with: #serverEndpoint -> EndpointResource
>
> MyTestCase>>test1
> self assert: self clientEndpoint isAlive
>
> MyTestCase>>clientEndpoint
> self testSuite resources at: #clientEndpoint
>
> But if we're going that far, why have the ugliness of the #resources
> method and corresponding method for each resource? The TestSuite will
> have to initialize the MyTestCase instance anyway, to set itself into
> the testSuite iv. So why not:
>
> MyTestCase>>clientEndpoint
> <resource: EndpointResource>
> ^clientEndpoint
>
> The test suite would find all the methods marked as resource methods,
> create and store an instance of the named resource class in a dict,
> keyed by the method selector, and then start running tests. When it
> creates an instance of MyTestCase, it sets the iv of that name with its
> resource of the same name.

Excellent idea! Though I don't use resources that much this would be
great. But one more thing - I would like to be able to parametrize the
resource upon its creation. For example when I want to use aFileResource
I would like to specify aFilename. In the current implementation, I have
to make two resource classes on two concrete file names...

So I would change it to something like:
MyTestCase>>clientEndPointResource
<testResource: #clientEndPoint>

^EndPointResource new
"You can do whatever you like here to complete its initialization,
"or you can simulate current behavior with EndPointResource current."

And refer to it in tests via
MyTestCase>>clientEndPoint
^self resourceAt: #clientEndPoint.

It is a bit more work, but it is also more flexible...

> This way each resource is declared and used in the same place, and
> access to them is simple and standard Smalltalk, direct from the
> TestCase. Whilst I'm not a great fan of magic like pragmas, there's no
> more magic here than in the current system of initializing and accessing
> resources. Those who like more magic could move the declaration of all
> resources to one method on the class side, and have doesNotUnderstand:
> forward calls to #clientEndpoint etc. to the test suite resources,
> getting rid of the need for an iv per resource in the test case.
As soon as I got the idea of pragmas, I started to like them. Their big
advantage over class side methods or methods relying on super is, that
you can easily extent behavior of classes from other packages without
introducing new method overrides and you can still find all references.
You can even change the order of invocation if you add some order
parameter to the pragma...

Let's hope that Travis will read this and will take the best from it to
improve his SUnitToo... :-)

Ladislav Lenart

Travis Griggs

Re: Problem with SUnit test suite usage....

On Jul 13, 2006, at 1:08, Ladislav Lenart wrote:

Steven Kelly wrote:

TestCase>>resource
^self resources as Array first

TestCase>>resource
^MyResource current

The #resources method is there only for specifying prerequisites that
need to be available in order to run the test. But when they are, they
are all accessible via #current, because they are effectively
singletons...

Interesting! I know where my way comes from: inspecting a TestCase and
its methods to try to find how to get from it to its resources. I didn't
like the answer then, and I don't like it much better now. Your way is
neater, but has the minor problem that test cases can then reference
resources that they haven't "declared" in the class side #resources
method.

Well, you're right but the same problem is for example with package
prerequisites. In the end, you have to specify it by hand (at least
you have to click on compute prereqs and resolve cycles), but no one
prevents you from using classes that are not in the package prerequisites
chain.

Both have the problem that the resource is a singleton - a case
cannot have two instances of the same resource, nor are resources thread
safe.

I don't see the point where thread-safeness would matter, because I only
run one TestSuite at a time and I don't think this is the only problem
that prevents running multiple tests at the same time...

But your first objection (resource is a singleton) - I completely agree.

I've also noticed that my resources tend to have ivs, for each of which
I have to make accessors in both the resource and the test case. Roughly
speaking, a resource here is a database connection, and each iv is for
some particular persistent thing in the database. Maybe I should be
making each iv into its own resource? For me, it would make sense if a TestSuite held on to its TestResources
(instances, not classes like it does now), and a TestCase being run
would know which TestSuite it belonged to, so it could access resources
through that. Resources would be named with symbols by their TestCase,
so we could know what we were referring to, and that it must be declared
to be used:
MyTestCase class>>resources
^Dictionary with: #clientEndpoint -> EndpointResource
with: #serverEndpoint -> EndpointResource
MyTestCase>>test1
self assert: self clientEndpoint isAlive
MyTestCase>>clientEndpoint
self testSuite resources at: #clientEndpoint
But if we're going that far, why have the ugliness of the #resources
method and corresponding method for each resource? The TestSuite will
have to initialize the MyTestCase instance anyway, to set itself into
the testSuite iv. So why not:
MyTestCase>>clientEndpoint
<resource: EndpointResource>
^clientEndpoint
The test suite would find all the methods marked as resource methods,
create and store an instance of the named resource class in a dict,
keyed by the method selector, and then start running tests. When it
creates an instance of MyTestCase, it sets the iv of that name with its
resource of the same name.

Excellent idea! Though I don't use resources that much this would be
great. But one more thing - I would like to be able to parametrize the
resource upon its creation. For example when I want to use aFileResource
I would like to specify aFilename. In the current implementation, I have
to make two resource classes on two concrete file names...

So I would change it to something like:
MyTestCase>>clientEndPointResource
<testResource: #clientEndPoint>

^EndPointResource new
"You can do whatever you like here to complete its initialization,
"or you can simulate current behavior with EndPointResource current."

And refer to it in tests via
MyTestCase>>clientEndPoint
^self resourceAt: #clientEndPoint.

It is a bit more work, but it is also more flexible...

This way each resource is declared and used in the same place, and
access to them is simple and standard Smalltalk, direct from the
TestCase. Whilst I'm not a great fan of magic like pragmas, there's no
more magic here than in the current system of initializing and accessing
resources. Those who like more magic could move the declaration of all
resources to one method on the class side, and have doesNotUnderstand:
forward calls to #clientEndpoint etc. to the test suite resources,
getting rid of the need for an iv per resource in the test case.

As soon as I got the idea of pragmas, I started to like them. Their big
advantage over class side methods or methods relying on super is, that
you can easily extent behavior of classes from other packages without
introducing new method overrides and you can still find all references.
You can even change the order of invocation if you add some order
parameter to the pragma...

Let's hope that Travis will read this and will take the best from it to
improve his SUnitToo... :-)

To be honest, I probably won't. Not because I don't want to. But because I never use TestResources. But I use SUnit nominal alot. So I was in a good position to know where I wanted to go with that.

SUnitToo was born of my frustration with not having an SUnit that seemed to have any sort of reachable stearage, and I got frustrated with all of the xplatform stuff.

Hopefully reading between the lines, you can see what I'd love to see happen: One (or more) of you guys who really knows and understands where the TestResource stuff ought to go, should do it and publish it. I'd love it. SUnitToo is actively maintained in the OR for all too use and benefit from. My only input is that the TestResource "framework" should be light. I believe firmly that SUnit should be something you could put together in a short time, like, in the airports on the way to OOPSLA.

Travis Griggs

Objologist

One man's blue plane is another man's pink plane.

DISCLAIMER: This email is bound by the terms and conditions described at http://www.key.net/disclaimer.htm

Steven Kelly

RE: Problem with SUnit test suite usage....

In reply to this post by Rick Flower

Steven Kelly wrote:

why not:

MyTestCase>>clientEndpoint

<resource: EndpointResource>

^clientEndpoint

The test suite would find all the methods marked as resource methods,

create and store an instance of the named resource class in a dict,

keyed by the method selector, and then start running tests. When it

creates an instance of MyTestCase, it sets the iv of that name with its

resource of the same name.

Ladislave Lenart wrote:

Excellent idea! Though I don't use resources that much this would be

great. But one more thing - I would like to be able to parametrize the

resource upon its creation. For example when I want to use aFileResource

I would like to specify aFilename. In the current implementation, I have

to make two resource classes on two concrete file names...

So I would change it to something like:

MyTestCase>>clientEndPointResource

<testResource: #clientEndPoint>

^EndPointResource new

"You can do whatever you like here to complete its initialization,

"or you can simulate current behavior with EndPointResource current."

And refer to it in tests via

MyTestCase>>clientEndPoint

^self resourceAt: #clientEndPoint.

It is a bit more work, but it is also more flexible...

Let's hope that Travis will read this and will take the best from it to

improve his SUnitToo... :-)

To be honest, I probably won't. Not because I don't want to. But because I never use TestResources. But I use SUnit nominal alot. So I was in a good position to know where I wanted to go with that.

Travis Griggs

Great to hear you say that, Travis – and actually what I guessed (hoped) you'd say. That's how the open repository should work at best.

I suggest that Ladislav and I both mess around in our own images first, and then agree on what to publish. If possible, I'd publish so it wouldn't be in SUnitToo: I don't know how many people use it, but it's certainly not an absolute prerequisite for a better way to handle resources. It would be great if what we published worked in both SUnit and SUnitToo.

My current feeling is that the best way to support what Ladislav suggested is something simple like

MyTestCase>>clientEndPointResource

^self resourceAt: #clientEndPoint ifAbsentPut: [EndPointResource new]

TestCase would have an iv, testSuite, which the TestSuite would set to itself in each TestCase it creates. The TestSuite would hold on to the resources in a resourcesDict iv, lazily initializing them as TestCases require them.

TestCase>>resourceAt: key ifAbsentPut: createBlock

^self testSuite resourceAt: key ifAbsentPut: createBlock

TestSuite>>resourceAt: key ifAbsentPut: createBlock

^self resourcesDict at: key ifAbsentPut: createBlock

having the lazy initialization is nice if you have a big suite with many resources that are costly to create, but only want to run one test from it – only the resources needed by that test are created.

Thoughts? I'd also be interested to hear about how others use resources and their ivs: do you tend to have one resource with several rather disparate things each in its own iv, or make several resources each with one iv?

Steve

DISCLAIMER: This email is bound by the terms and conditions described at http://www.key.net/disclaimer.htm

Ladislav Lenart

Re: Problem with SUnit test suite usage....

Steven Kelly wrote:
> I suggest that Ladislav and I both mess around in our own images first,
> and then agree on what to publish. If possible, I'd publish so it
> wouldn't be in SUnitToo: I don't know how many people use it, but it's
> certainly not an absolute prerequisite for a better way to handle
> resources. It would be great if what we published worked in both SUnit
> and SUnitToo.
Well, I looked a bit at how SUnitToo maintains its resources and
what changes are neccessary to support new behavior. Having own
package for new resources means that we either end up with overrides
in TestSuite, TestCase and TestResource or with subclasses of those
(I vote for subclasses).

And about this working in both SUnit and SUnitToo. I would prefer
to have two separate packages, one for SUnitToo and the second
one for SUnit. And these two packages would have nothing in common
(except the idea of course).

For me personally, I don't need the SUnit package, because each new
TestCase I create is SUnit.TestCase and existing tests work fine...

> My current feeling is that the best way to support what Ladislav
> suggested is something simple like
>
> MyTestCase>>clientEndPointResource
>
> ^self
> resourceAt: #clientEndPoint
> ifAbsentPut: [EndPointResource new]
>
> TestCase would have an iv, testSuite, which the TestSuite would set to
> itself in each TestCase it creates. The TestSuite would hold on to the
> resources in a resourcesDict iv, lazily initializing them as TestCases
> require them.
>
> TestCase>>resourceAt: key ifAbsentPut: createBlock
>
> ^self testSuite
> resourceAt: key
> ifAbsentPut: createBlock
>
> TestSuite>>resourceAt: key ifAbsentPut: createBlock
>
> ^self resourcesDict
> at: key
> ifAbsentPut: createBlock
>
> having the lazy initialization is nice if you have a big suite with many
> resources that are costly to create, but only want to run one test from
> it – only the resources needed by that test are created.
I agree with interface but miss the pragmas and don't get the lazy initialization
thing. The purpose of resources is, that the test need them in order to be able
to run. This means, that prior the TestCase is ran, all its resources must be up
and running. And if some of them is not available, the test won't run at all.
Not mentioning that some resources are not referenced in tests but are still
needed. For example we have UserGuaranteedResource (in old SUnit) that is used
to supress running of some tests on demand. The test itself doesn't use the
resource, it is just in its resources and when it is not available, the test
won't run.

So I would change it to

MyTestCase>>clientEndPointResource
<testResource>

^self
resourceAt: #clientEndPoint
ifAbsentPut: [EndPointResource new].

This way, TestSuite prior to run its tests would gather all necessary resources
by performing #testResource pragma methods in all its tests (and recursively for
their resources).

But there is one problem with #resourceAt:ifAbsentPut: mechanism: in SUnit
even resources can have their resources and this doesn't deal with it. Probably
TestResource will have to know its testSuite as well?

Ladislav Lenart

Steven Kelly

RE: Problem with SUnit test suite usage....

In reply to this post by Rick Flower

From: Ladislav Lenart [mailto:[hidden email]]
> Well, I looked a bit at how SUnitToo maintains its resources and
> what changes are neccessary to support new behavior. Having own
> package for new resources means that we either end up with overrides
> in TestSuite, TestCase and TestResource or with subclasses of those
> (I vote for subclasses).

I'd wait and see before deciding. If our changes make sense to people,
we can just get them into SUnit. If not, I doubt I'd bother.

> For me personally, I don't need the SUnit package, because each new
> TestCase I create is SUnit.TestCase and existing tests work fine...

Sorry, I don't understand that. Do you mean you add test methods to
TestCase itself?

> > My current feeling is that the best way to support what Ladislav
> > suggested is something simple like
> >
> > MyTestCase>>clientEndPointResource
> >
> > ^self
> > resourceAt: #clientEndPoint
> > ifAbsentPut: [EndPointResource new]
> >
> > having the lazy initialization is nice if you have a big suite with

many
> > resources that are costly to create, but only want to run one test
from
> > it - only the resources needed by that test are created.
>
> I agree with interface but miss the pragmas and don't get the lazy
> initialization thing. The purpose of resources is, that the test
> need them in order to be able to run.

OK, I see them differently. MyTestCase is already intended to be able to
be a fixture (right term?), i.e. to hold things it needs to run. The
reason resources exist IIUC is for when creating those things is
expensive, so you don't want to create them before and destroy them
after each test in MyTestCase.

> This means, that prior the TestCase is ran, all its resources must
> be up and running. And if some of them is not available, the test
> won't run at all.

For me it is enough that the test that first tries to use them isn't
marked as failing per se, but raises a different error. Any tests in the
suite before that that passed would still be valid passes for me, and
subsequent tests would not be run.

> Not mentioning that some resources are not referenced in tests but
> are still needed. For example we have UserGuaranteedResource (in old
> SUnit) that is used to supress running of some tests on demand. The
> test itself doesn't use the resource, it is just in its resources
> and when it is not available, the test won't run.

Are there many real examples of that kind of need? It is of course
simple to have the test specify that the resource must be up and running
- just send "self myResource" at the start. And if all tests in a
TestCase class need it, just put "self myResource" in setUp.

Currently, the TestCase _class_ specifies that all the resources must be
there, and they are all initialized even if they are not needed. I think
that's less useful, if the main point of resources is that initializing
them is expensive - I'd rather be able to run one test quickly whenever
I wanted (because it only used one resource out of several), than know
that all resources, even ones my selected tests don't need or use, have
been initialized.

> But there is one problem with #resourceAt:ifAbsentPut: mechanism:
> in SUnit even resources can have their resources and this doesn't
> deal with it.

If I was building SUnit now from scratch, this would be a clear YAGNI.
We need more people's experiences with resources, but at the moment it
looks to me like most people don't use them much, and the way they work
isn't optimal for most people who do use them.

I suppose I should look at existing packages. Pollock's a bad example,
but still:
TestCase allSubclasses size "printit" 320
TestResource allSubclasses size "printit" 4

A quick look at how resources are used there indicates that they are not
actually used by the tests, but just as a way to make sure that the
right LookPolicy is installed before the tests are run. That's similar
to the usage you talk about above, but it doesn't seem to match the name
"Resource". Putting "self myResource" in the TestCase setup would
suffice for Pollock using my suggested resource semantics, but it might
also be better that we'd have a separate TestPrerequisite class for
things that merely set the scene, and use TestResource for things the
#testCase methods actually need to refer to.

Steve