Smalltalk › Squeak › Squeak - Dev

Code coverage

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

14 messages Options

Diego Fernández

Code coverage

Hi, is there any tool to do code coverage? (at method level is enough, i want to do something to check that unit test cover all the methods)

ccrraaiigg

re: Code coverage

Hi Diego--

> Hi, is there any tool to do code coverage? (at method level is enough,
> i want to do something to check that unit test cover all the methods)

It's not available as a tool for current Squeak yet, but for what it's
worth...

One could make such a tool with Spoon by using method activation
marking. Clear all the marks, run the tests, then check the marks. (The
Spoon VM has a distinct object format for methods which have been run.)

-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]

Diego Fernández

Re: Code coverage

Thanks Craig, I would take a look to Spoon.

I want to make a tool to test a package "quality" before saving it in MC, these tests includes:
- Run the unit tests and check that the coverage is ok
- Run Smalllint
- Verify that:
- there is no uncategorized methods
- all the dependencies of the package are declared (for example if I use a TestCase, then the SUnit package must be in the requisites)

That's why I want to do code coverage with the current Squeak VM.
Maybe I can use "method wrappers", has someone tried to do that without modifying the VM?

On 1/16/06, Craig Latta <[hidden email]> wrote:

Hi Diego--

> Hi, is there any tool to do code coverage? (at method level is enough,
> i want to do something to check that unit test cover all the methods)

It's not available as a tool for current Squeak yet, but for what it's
worth...

One could make such a tool with Spoon by using method activation
marking. Clear all the marks, run the tests, then check the marks. (The
Spoon VM has a distinct object format for methods which have been run.)

-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]

Tony Garnock-Jones-2

Re: Code coverage

In reply to this post by ccrraaiigg

Craig Latta wrote:
> (The
> Spoon VM has a distinct object format for methods which have been run.)

(This sounds like the hook for an interesting conversation!) Why?

Tony

ccrraaiigg

re: Code coverage

Hi Tony--

> > (The Spoon VM has a distinct [object header format field value] for
> > methods which have been run.)
>
> (This sounds like the hook for an interesting conversation!) Why?

Oh, just to have a convenient way for the VM to mark a method when
running it, and to check later whether a method has been run. The
original motivation was for detecting methods which hadn't been run, so
they could be discarded.

-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]

Lukas Renggli

Re: Code coverage

In reply to this post by Diego Fernández

> Hi, is there any tool to do code coverage? (at method level is enough, i
> want to do something to check that unit test cover all the methods)

You might want to try something like that in any Squeak image:

tallies := IdentityDictionary new.
prev := [ MAFileModelTest suite run ].
thisContext sender
runSimulated: prev
contextAtEachStep: [ :curr |
curr == prev ifFalse: [
(tallies
at: curr receiver
ifAbsentPut: [ Set new ])
add: curr selector ].
prev := curr ] ].

There are some bugs in the simulator that prevent you from running
tests that containing #should:raise: and #shouldnt:raise:, else this
technique works very well and requires no additional package.

Lukas

--
Lukas Renggli
http://www.lukas-renggli.ch

Tony Garnock-Jones-2

Re: Code coverage

In reply to this post by ccrraaiigg

Craig Latta wrote:
> Oh, just to have a convenient way for the VM to mark a method when
> running it, and to check later whether a method has been run. The
> original motivation was for detecting methods which hadn't been run, so
> they could be discarded.

Sounds like a risky idea :-) Have you abandoned it for something better?
Was it before you invented the module system you're using?

Tony

Diego Fernández

Re: Code coverage

In reply to this post by Lukas Renggli

Thanks Lucas, I will try it.
Yesterday I started to do the code coverage tool using "method wrappers", it worked :)

But my unit tests of the coverage tool some times fails and some times pass (I think that is caused by some initialization made by the method wrappers).
Anyway.. I think that tomorrow I'm going to upload the first version of the package to SqueakMap.

On 1/17/06, Lukas Renggli <[hidden email]> wrote:

> Hi, is there any tool to do code coverage? (at method level is enough, i
> want to do something to check that unit test cover all the methods)

You might want to try something like that in any Squeak image:

tallies := IdentityDictionary new.
prev := [ MAFileModelTest suite run ].
thisContext sender
        runSimulated: prev
        contextAtEachStep: [ :curr |
                curr == prev ifFalse: [
                        (tallies
                                at: curr receiver
                                ifAbsentPut: [ Set new ])
                                        add: curr selector ].
                        prev := curr ] ].

There are some bugs in the simulator that prevent you from running
tests  that containing #should:raise: and #shouldnt:raise:, else this
technique works very well and requires no additional package.

Lukas

--
Lukas Renggli
http://www.lukas-renggli.ch

stéphane ducasse-2

Re: Code coverage

Diego

I suggest you to have a look at the testrunner of lukas, Because it
is real fast and much more usable.
Available in the latest 3.9

stef

On 17 janv. 06, at 15:12, Diego Fernandez wrote:

> Thanks Lucas, I will try it.
> Yesterday I started to do the code coverage tool using "method
> wrappers", it worked :)
>
> But my unit tests of the coverage tool some times fails and some
> times pass (I think that is caused by some initialization made by
> the method wrappers).
> Anyway.. I think that tomorrow I'm going to upload the first
> version of the package to SqueakMap.
>
> On 1/17/06, Lukas Renggli < [hidden email]> wrote:> Hi, is there
> any tool to do code coverage? (at method level is enough, i
> > want to do something to check that unit test cover all the methods)
>
> You might want to try something like that in any Squeak image:
>
> tallies := IdentityDictionary new.
> prev := [ MAFileModelTest suite run ].
> thisContext sender
> runSimulated: prev
> contextAtEachStep: [ :curr |
> curr == prev ifFalse: [
> (tallies
> at: curr receiver
> ifAbsentPut: [ Set new ])
> add: curr selector ].
> prev := curr ] ].
>
> There are some bugs in the simulator that prevent you from running
> tests that containing #should:raise: and #shouldnt:raise:, else this
> technique works very well and requires no additional package.
>
> Lukas
>
> --
> Lukas Renggli
> http://www.lukas-renggli.ch
>
>
>

Serge Stinckwich-4

Re: Code coverage

stéphane ducasse a écrit :
> Diego
>
> I suggest you to have a look at the testrunner of lukas, Because it is
> real fast and much more usable.
> Available in the latest 3.9
>

Looks very nice !!! But i think the interface is a bit disturbing :
selected categories and classes are grayed and "Run All" and "Run
Profiled" also grayed when you can't use them ...

-- oooo
Dr. Serge Stinckwich OOOOOOOO
Université de Caen>CNRS UMR 6072>GREYC>MAD OOESUGOO
http://purl.org/net/SergeStinckwich oooooo
Smalltalkers do: [:it | All with: Class, (And love: it)] \ /
##

ccrraaiigg

re: Code coverage

In reply to this post by Tony Garnock-Jones-2

> > Oh, just to have a convenient way for the VM to mark a method
> > [dictionary] when running it, and to check later whether a method
> > [in it] has been run. The original motivation was for detecting
> > [entire sets of] methods which hadn't been run, so they could be
> > discarded.

(Methods also get marked by the Spoon VM, but the mark is a bit in the
method trailer, not the object header.)

> Sounds like a risky idea :-)

Nah, it's been working fine for three years now. The fact that it
worked for the first three minutes was enough to convince me. :)

> Have you abandoned it for something better?

No, I think it's a good idea. I wanted something that's invisible to
normal object manipulation, and extremely fast.

> Was it before you invented the module system you're using?

No, I thought of it at roughly the same time, and they're independent
concepts.

-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]

Diego Fernández

Re: Code coverage

In reply to this post by stéphane ducasse-2

I can't figure what does the MessageTally class?

On 1/17/06, stéphane ducasse <[hidden email]> wrote:

Diego

I suggest you to have a look at the testrunner of lukas, Because it
is real fast and much more usable.
Available in the latest 3.9

stef

On 17 janv. 06, at 15:12, Diego Fernandez wrote:

> Thanks Lucas, I will try it.
> Yesterday I started to do the code coverage tool using "method
> wrappers", it worked :)
>
> But my unit tests of the coverage tool some times fails and some
> times pass (I think that is caused by some initialization made by
> the method wrappers).
> Anyway.. I think that tomorrow I'm going to upload the first
> version of the package to SqueakMap.
>
> On 1/17/06, Lukas Renggli < [hidden email]> wrote:> Hi, is there
> any tool to do code coverage? (at method level is enough, i
> > want to do something to check that unit test cover all the methods)
>
> You might want to try something like that in any Squeak image:
>
> tallies := IdentityDictionary new.
> prev := [ MAFileModelTest suite run ].
> thisContext sender
>         runSimulated: prev
>         contextAtEachStep: [ :curr |
>                 curr == prev ifFalse: [
>                         (tallies
>                                 at: curr receiver
>                                 ifAbsentPut: [ Set new ])
>                                         add: curr selector ].
>                         prev := curr ] ].
>
> There are some bugs in the simulator that prevent you from running
> tests  that containing #should:raise: and #shouldnt:raise:, else this
> technique works very well and requires no additional package.
>
> Lukas
>
> --
> Lukas Renggli
> http://www.lukas-renggli.ch
>
>
>

Tony Garnock-Jones-2

Re: Code coverage

In reply to this post by ccrraaiigg

Craig Latta wrote:
> > Sounds like a risky idea :-)
>
> Nah, it's been working fine for three years now. The fact that it
> worked for the first three minutes was enough to convince me. :)

I just mean determining the complete behaviour of a piece of code from
the methods it touches during a particular test run. I guess it's the
same objection raised when test suites are compared to static type
systems: one proves correctness, one proves not-incorrectness... :-)

Tony

joshscholar

Re: Code coverage

Well that's where that class reference documentation should come in.

You shouldn't have to prove what methods are needed by what subsystems, that
should be in the documentation.

----- Original Message -----
From: "Tony Garnock-Jones" <[hidden email]>
To: "The general-purpose Squeak developers list"
<[hidden email]>
Sent: Tuesday, January 17, 2006 2:39 PM
Subject: Re: Code coverage

> Craig Latta wrote:
> > > Sounds like a risky idea :-)
> >
> > Nah, it's been working fine for three years now. The fact that it
> > worked for the first three minutes was enough to convince me. :)
>
> I just mean determining the complete behaviour of a piece of code from
> the methods it touches during a particular test run. I guess it's the
> same objection raised when test suites are compared to static type
> systems: one proves correctness, one proves not-incorrectness... :-)
>
> Tony
>
>
>