Smalltalk › Pharo › Pharo Smalltalk Developers

Proposal: coverage tests

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

8 messages Options

Pavel Krivanek-3

Proposal: coverage tests

Hi,

what about to intrroduce tests that will test coverage of tests? Some
improved version of the code below. Are there some tool for Smalltalk
showing uncovered lines of code? There are such tools in Java world
:-) One little problem is that this code do not work on Linux because
of a VM error that cause test failures on CI server too.

Cheers,
Pavel

| class test trace classTrace result |

class := Time.
test := TimeTest.

trace := Dictionary new.

thisContext runSimulated: [test suite run] contextAtEachStep: [ :current |
| cls sel methods bytecodes |
cls := current method methodClass.
sel := current method selector.
methods := trace at: cls ifAbsentPut: Dictionary new.
bytecodes := methods at: sel ifAbsentPut: Set new.
bytecodes add: current pc.]

trace .

result := String streamContents: [:s |

classTrace := trace at: class ifAbsent: nil.
classTrace
ifNil: [
s nextPutAll: 'Class ', class name, ' not called'; cr. ]
ifNotNil: [
s nextPutAll: class name; cr.
class selectorsDo: [:sel |
| m usedBytecodes allBytecodes |
m := class >> sel.
usedBytecodes := (classTrace at: sel ifAbsent: Set new) size.
allBytecodes := 0.
m symbolicLinesDo: [:pc :lineForPC | allBytecodes := allBytecodes + 1 ]..
s tab; nextPutAll: sel; nextPutAll: ' - '; nextPutAll:
(usedBytecodes / allBytecodes * 100) asFloat asString; nextPutAll:
'%'; cr. ]].
].
result..

Sven Van Caekenberghe

Re: Proposal: coverage tests

Pavel,

This is incredible !

This is why we love Smalltalk, in what other language could you write it like that ?

I love it.

Sven

On 21 Feb 2012, at 16:03, Pavel Krivanek wrote:

> Hi,
>
> what about to intrroduce tests that will test coverage of tests? Some
> improved version of the code below. Are there some tool for Smalltalk
> showing uncovered lines of code? There are such tools in Java world
> :-) One little problem is that this code do not work on Linux because
> of a VM error that cause test failures on CI server too.
>
> Cheers,
> Pavel
>
> | class test trace classTrace result |
>
> class := Time.
> test := TimeTest.
>
> trace := Dictionary new.
>
> thisContext runSimulated: [test suite run] contextAtEachStep: [ :current |
> | cls sel methods bytecodes |
> cls := current method methodClass.
> sel := current method selector.
> methods := trace at: cls ifAbsentPut: Dictionary new.
> bytecodes := methods at: sel ifAbsentPut: Set new.
> bytecodes add: current pc.]
>
> trace .
>
> result := String streamContents: [:s |
>
> classTrace := trace at: class ifAbsent: nil.
> classTrace
> ifNil: [
> s nextPutAll: 'Class ', class name, ' not called'; cr. ]
> ifNotNil: [
> s nextPutAll: class name; cr.
> class selectorsDo: [:sel |
> | m usedBytecodes allBytecodes |
> m := class >> sel.
> usedBytecodes := (classTrace at: sel ifAbsent: Set new) size.
> allBytecodes := 0.
> m symbolicLinesDo: [:pc :lineForPC | allBytecodes := allBytecodes + 1 ]..
> s tab; nextPutAll: sel; nextPutAll: ' - '; nextPutAll:
> (usedBytecodes / allBytecodes * 100) asFloat asString; nextPutAll:
> '%'; cr. ]].
> ].
> result..
>

Mariano Martinez Peck

Re: Proposal: coverage tests

In reply to this post by Pavel Krivanek-3

That is just AMAZINGLY COOL. I cannot believe we can do that in 5 lines of code. WOW.
Why nobody wrote a better test coverage report wich such backend? Alex, does you stuff do something with that?

cheers

On Tue, Feb 21, 2012 at 4:03 PM, Pavel Krivanek <[hidden email]> wrote:

Hi,

what about to intrroduce tests that will test coverage of tests? Some
improved version of the code below. Are there some tool for Smalltalk
showing uncovered lines of code? There are such tools in Java world
:-) One little problem is that this code do not work on Linux because
of a VM error that cause test failures on CI server too.

Cheers,
Pavel

| class test trace classTrace result |

class := Time.
test := TimeTest.

trace := Dictionary new.

thisContext runSimulated: [test suite run] contextAtEachStep: [ :current |
| cls sel methods bytecodes |
cls := current method methodClass.
sel := current method selector.
methods := trace at: cls ifAbsentPut: Dictionary new.
bytecodes := methods at: sel ifAbsentPut: Set new.
bytecodes add: current pc.]

trace .

result := String streamContents: [:s |

classTrace := trace at: class ifAbsent: nil.
classTrace
ifNil: [
s nextPutAll: 'Class ', class name, ' not called'; cr. ]
ifNotNil: [
s nextPutAll: class name; cr.
class selectorsDo: [:sel |
| m usedBytecodes allBytecodes |
m := class >> sel.
usedBytecodes := (classTrace at: sel ifAbsent: Set new) size.
allBytecodes := 0.
m symbolicLinesDo: [:pc :lineForPC | allBytecodes := allBytecodes + 1 ]..
s tab; nextPutAll: sel; nextPutAll: ' - '; nextPutAll:
(usedBytecodes / allBytecodes * 100) asFloat asString; nextPutAll:
'%'; cr. ]].
].
result..

--
Mariano
http://marianopeck.wordpress.com

Guido Stepken

Re: Proposal: coverage tests

In reply to this post by Pavel Krivanek-3

Well done, a real *masterpieche*! :-)

Depreciated (marked as depreciated) classes could / should be moved either into a compatibility layer (sport) or, if only a few "apps" are using it, into the app itself, being slightly renamed for keeping up compatibility.

This should work in a similar way by first identifying concerned apps, automatically moving - while renaming classes - into the app, exporting as new package.

Does that make sense? How would u solve that?

tnx in advance, Guido Stepken

Am 21.02.2012 16:03 schrieb "Pavel Krivanek" <[hidden email]>:

Hi,

what about to intrroduce tests that will test coverage of tests? Some
improved version of the code below. Are there some tool for Smalltalk
showing uncovered lines of code? There are such tools in Java world
:-) One little problem is that this code do not work on Linux because
of a VM error that cause test failures on CI server too.

Cheers,
Pavel

| class test trace classTrace result |

class := Time.
test := TimeTest.

trace := Dictionary new.

thisContext runSimulated: [test suite run] contextAtEachStep: [ :current |
| cls sel methods bytecodes |
cls := current method methodClass.
sel := current method selector.
methods := trace at: cls ifAbsentPut: Dictionary new.
bytecodes := methods at: sel ifAbsentPut: Set new.
bytecodes add: current pc.]

trace .

result := String streamContents: [:s |

classTrace := trace at: class ifAbsent: nil.
classTrace
ifNil: [
s nextPutAll: 'Class ', class name, ' not called'; cr. ]
ifNotNil: [
s nextPutAll: class name; cr.
class selectorsDo: [:sel |
| m usedBytecodes allBytecodes |
m := class >> sel.
usedBytecodes := (classTrace at: sel ifAbsent: Set new) size.
allBytecodes := 0.
m symbolicLinesDo: [:pc :lineForPC | allBytecodes := allBytecodes + 1 ]..
s tab; nextPutAll: sel; nextPutAll: ' - '; nextPutAll:
(usedBytecodes / allBytecodes * 100) asFloat asString; nextPutAll:
'%'; cr. ]].
].
result..

Marcus Denker-4

Re: Proposal: coverage tests

In reply to this post by Pavel Krivanek-3

On Feb 21, 2012, at 10:40 PM, Mariano Martinez Peck wrote:

> That is just AMAZINGLY COOL. I cannot believe we can do that in 5 lines of code. WOW.

But
-> it is interpreted
-> it is very low level

Now imagine we would have instead of bytecode the a high level AST *and* a
way to extend a runtime compiler infrastructure at runtime...

With that you can do it in less lines, and even very trivally :-)

Marcus

--
Marcus Denker -- http://marcusdenker.de

Göran Krampe

Re: Proposal: coverage tests

In reply to this post by Pavel Krivanek-3

Hi all!

I seem to recall a small discussion with Peter Hugosson-Miller at ESUG
in Brest (walking to lunch :)) in which he told me a bit about the tests
they used where he worked. IIRC a LOT of them were indeed "meta tests"
that were testing properties of the source code itself (metrics, lints etc).

Peter, any more ideas in this direction you can share?

If coupled with "background execution" (or hey, in a forked image
through OSProcess so we don't mess up the one we work in) and immediate
feedback in browser tools this is "winner stuff".

regards, Göran

Pavel Krivanek-3

Re: Proposal: coverage tests

In reply to this post by Pavel Krivanek-3

This is the way how to get the the source code that belongs to
uncovered bytecodes (expecting "trace" from the previous code)

class := Time.
selector := #print24:showSeconds:on:.
m := class >> selector.
usedBytecodes := ((trace at: class) at: selector ifAbsent: Set new).
allBytecodes := Set new.
m symbolicLinesDo: [:pc :lineForPC | allBytecodes add: pc ].
unusedBytecodes := allBytecodes copyWithoutAll: usedBytecodes.

dm := m debuggerMap.
source := (class sourceCodeAt: selector) asText.
unusedBytecodes do: [:pc |
range := dm rangeForPC: pc contextIsActiveContext: false.
source makeBoldFrom: range first to: range last.].
source asMorph openInWindow.

Cheers,
-- Pavel

On Tue, Feb 21, 2012 at 4:03 PM, Pavel Krivanek
<[hidden email]> wrote:

abergel

Re: Proposal: coverage tests

In reply to this post by Mariano Martinez Peck

> That is just AMAZINGLY COOL. I cannot believe we can do that in 5 lines of code. WOW.
> Why nobody wrote a better test coverage report wich such backend? Alex, does you stuff do something with that?

Hapao does not support statement coverage yet.
We are currently working on it. We will announce it soon.

Cheers,
Alexandre
--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.