Code coverage

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Code coverage

Diego Fernández
Hi, is there any tool to do code coverage? (at method level is enough, i want to do something to check that unit test cover all the methods)




Reply | Threaded
Open this post in threaded view
|

re: Code coverage

ccrraaiigg

Hi Diego--

 > Hi, is there any tool to do code coverage? (at method level is enough,
 > i want to do something to check that unit test cover all the methods)

        It's not available as a tool for current Squeak yet, but for what it's
worth...

        One could make such a tool with Spoon by using method activation
marking. Clear all the marks, run the tests, then check the marks. (The
Spoon VM has a distinct object format for methods which have been run.)


-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]



Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

Diego Fernández
Thanks Craig, I would take a look to Spoon.

I want to make a tool to test a package "quality" before saving it in MC, these tests includes:
- Run the unit tests and check that the coverage is ok
- Run Smalllint
- Verify that:
    - there is no uncategorized methods
    - all the dependencies of the package are declared (for example if I use a TestCase, then the SUnit package must be in the requisites)

That's why I want to do code coverage with the current Squeak VM.
Maybe I can use "method wrappers", has someone tried to do that without modifying the VM?


On 1/16/06, Craig Latta <[hidden email]> wrote:

Hi Diego--

> Hi, is there any tool to do code coverage? (at method level is enough,
> i want to do something to check that unit test cover all the methods)

        It's not available as a tool for current Squeak yet, but for what it's
worth...

        One could make such a tool with Spoon by using method activation
marking. Clear all the marks, run the tests, then check the marks. (The
Spoon VM has a distinct object format for methods which have been run.)


-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]






Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

Tony Garnock-Jones-2
In reply to this post by ccrraaiigg
Craig Latta wrote:
> (The
> Spoon VM has a distinct object format for methods which have been run.)

(This sounds like the hook for an interesting conversation!) Why?

Tony

Reply | Threaded
Open this post in threaded view
|

re: Code coverage

ccrraaiigg

Hi Tony--

 > > (The Spoon VM has a distinct [object header format field value] for
 > > methods which have been run.)
 >
 > (This sounds like the hook for an interesting conversation!) Why?

        Oh, just to have a convenient way for the VM to mark a method when
running it, and to check later whether a method has been run. The
original motivation was for detecting methods which hadn't been run, so
they could be discarded.


-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]



Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

Lukas Renggli
In reply to this post by Diego Fernández
> Hi, is there any tool to do code coverage? (at method level is enough, i
> want to do something to check that unit test cover all the methods)

You might want to try something like that in any Squeak image:

tallies := IdentityDictionary new.
prev := [ MAFileModelTest suite run ].
thisContext sender
        runSimulated: prev
        contextAtEachStep: [ :curr |
                curr == prev ifFalse: [
                        (tallies
                                at: curr receiver
                                ifAbsentPut: [ Set new ])
                                        add: curr selector ].
                        prev := curr ] ].

There are some bugs in the simulator that prevent you from running
tests  that containing #should:raise: and #shouldnt:raise:, else this
technique works very well and requires no additional package.

Lukas

--
Lukas Renggli
http://www.lukas-renggli.ch

Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

Tony Garnock-Jones-2
In reply to this post by ccrraaiigg
Craig Latta wrote:
>     Oh, just to have a convenient way for the VM to mark a method when
> running it, and to check later whether a method has been run. The
> original motivation was for detecting methods which hadn't been run, so
> they could be discarded.

Sounds like a risky idea :-) Have you abandoned it for something better?
Was it before you invented the module system you're using?

Tony


Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

Diego Fernández
In reply to this post by Lukas Renggli
Thanks Lucas, I will try it.
Yesterday I started to do the code coverage tool using "method wrappers", it worked :)

But my unit tests of the coverage tool some times fails and some times pass (I think that is caused by some initialization made by the method wrappers).
Anyway.. I think that tomorrow I'm going to upload the first version of the package to SqueakMap.

On 1/17/06, Lukas Renggli <[hidden email]> wrote:
> Hi, is there any tool to do code coverage? (at method level is enough, i
> want to do something to check that unit test cover all the methods)

You might want to try something like that in any Squeak image:

tallies := IdentityDictionary new.
prev := [ MAFileModelTest suite run ].
thisContext sender
        runSimulated: prev
        contextAtEachStep: [ :curr |
                curr == prev ifFalse: [
                        (tallies
                                at: curr receiver
                                ifAbsentPut: [ Set new ])
                                        add: curr selector ].
                        prev := curr ] ].

There are some bugs in the simulator that prevent you from running
tests  that containing #should:raise: and #shouldnt:raise:, else this
technique works very well and requires no additional package.

Lukas

--
Lukas Renggli
http://www.lukas-renggli.ch




Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

stéphane ducasse-2
Diego

I suggest you to have a look at the testrunner of lukas, Because it  
is real fast and much more usable.
Available in the latest 3.9

stef

On 17 janv. 06, at 15:12, Diego Fernandez wrote:

> Thanks Lucas, I will try it.
> Yesterday I started to do the code coverage tool using "method  
> wrappers", it worked :)
>
> But my unit tests of the coverage tool some times fails and some  
> times pass (I think that is caused by some initialization made by  
> the method wrappers).
> Anyway.. I think that tomorrow I'm going to upload the first  
> version of the package to SqueakMap.
>
> On 1/17/06, Lukas Renggli < [hidden email]> wrote:> Hi, is there  
> any tool to do code coverage? (at method level is enough, i
> > want to do something to check that unit test cover all the methods)
>
> You might want to try something like that in any Squeak image:
>
> tallies := IdentityDictionary new.
> prev := [ MAFileModelTest suite run ].
> thisContext sender
>         runSimulated: prev
>         contextAtEachStep: [ :curr |
>                 curr == prev ifFalse: [
>                         (tallies
>                                 at: curr receiver
>                                 ifAbsentPut: [ Set new ])
>                                         add: curr selector ].
>                         prev := curr ] ].
>
> There are some bugs in the simulator that prevent you from running
> tests  that containing #should:raise: and #shouldnt:raise:, else this
> technique works very well and requires no additional package.
>
> Lukas
>
> --
> Lukas Renggli
> http://www.lukas-renggli.ch
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

Serge Stinckwich-4
stéphane ducasse a écrit :
> Diego
>
> I suggest you to have a look at the testrunner of lukas, Because it is
> real fast and much more usable.
> Available in the latest 3.9
>

Looks very nice !!! But i think the interface is a bit disturbing :
selected categories and classes are grayed and "Run All" and "Run
Profiled" also grayed when you can't use them ...


--                                                         oooo
Dr. Serge Stinckwich                                     OOOOOOOO
Université de Caen>CNRS UMR 6072>GREYC>MAD               OOESUGOO
http://purl.org/net/SergeStinckwich                       oooooo
Smalltalkers do: [:it | All with: Class, (And love: it)]   \  /
                                                             ##




Reply | Threaded
Open this post in threaded view
|

re: Code coverage

ccrraaiigg
In reply to this post by Tony Garnock-Jones-2

 > > Oh, just to have a convenient way for the VM to mark a method
 > > [dictionary] when running it, and to check later whether a method
 > > [in it] has been run. The original motivation was for detecting
 > > [entire sets of] methods which hadn't been run, so they could be
 > > discarded.

        (Methods also get marked by the Spoon VM, but the mark is a bit in the
method trailer, not the object header.)

 > Sounds like a risky idea :-)

        Nah, it's been working fine for three years now. The fact that it
worked for the first three minutes was enough to convince me. :)

 > Have you abandoned it for something better?

        No, I think it's a good idea. I wanted something that's invisible to
normal object manipulation, and extremely fast.

 > Was it before you invented the module system you're using?

        No, I thought of it at roughly the same time, and they're independent
concepts.


-C

--
Craig Latta
improvisational musical informaticist
www.netjam.org
Smalltalkers do: [:it | All with: Class, (And love: it)]



Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

Diego Fernández
In reply to this post by stéphane ducasse-2
I can't figure what does the MessageTally class?

On 1/17/06, stéphane ducasse <[hidden email]> wrote:
Diego

I suggest you to have a look at the testrunner of lukas, Because it
is real fast and much more usable.
Available in the latest 3.9

stef

On 17 janv. 06, at 15:12, Diego Fernandez wrote:

> Thanks Lucas, I will try it.
> Yesterday I started to do the code coverage tool using "method
> wrappers", it worked :)
>
> But my unit tests of the coverage tool some times fails and some
> times pass (I think that is caused by some initialization made by
> the method wrappers).
> Anyway.. I think that tomorrow I'm going to upload the first
> version of the package to SqueakMap.
>
> On 1/17/06, Lukas Renggli < [hidden email]> wrote:> Hi, is there
> any tool to do code coverage? (at method level is enough, i
> > want to do something to check that unit test cover all the methods)
>
> You might want to try something like that in any Squeak image:
>
> tallies := IdentityDictionary new.
> prev := [ MAFileModelTest suite run ].
> thisContext sender
>         runSimulated: prev
>         contextAtEachStep: [ :curr |
>                 curr == prev ifFalse: [
>                         (tallies
>                                 at: curr receiver
>                                 ifAbsentPut: [ Set new ])
>                                         add: curr selector ].
>                         prev := curr ] ].
>
> There are some bugs in the simulator that prevent you from running
> tests  that containing #should:raise: and #shouldnt:raise:, else this
> technique works very well and requires no additional package.
>
> Lukas
>
> --
> Lukas Renggli
> http://www.lukas-renggli.ch
>
>
>





Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

Tony Garnock-Jones-2
In reply to this post by ccrraaiigg
Craig Latta wrote:
>  > Sounds like a risky idea :-)
>
>     Nah, it's been working fine for three years now. The fact that it
> worked for the first three minutes was enough to convince me. :)

I just mean determining the complete behaviour of a piece of code from
the methods it touches during a particular test run. I guess it's the
same objection raised when test suites are compared to static type
systems: one proves correctness, one proves not-incorrectness... :-)

Tony


Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

joshscholar
Well that's where that class reference documentation should come in.

You shouldn't have to prove what methods are needed by what subsystems, that
should be in the documentation.

----- Original Message -----
From: "Tony Garnock-Jones" <[hidden email]>
To: "The general-purpose Squeak developers list"
<[hidden email]>
Sent: Tuesday, January 17, 2006 2:39 PM
Subject: Re: Code coverage


> Craig Latta wrote:
> >  > Sounds like a risky idea :-)
> >
> >     Nah, it's been working fine for three years now. The fact that it
> > worked for the first three minutes was enough to convince me. :)
>
> I just mean determining the complete behaviour of a piece of code from
> the methods it touches during a particular test run. I guess it's the
> same objection raised when test suites are compared to static type
> systems: one proves correctness, one proves not-incorrectness... :-)
>
> Tony
>
>
>