SUnit disappointment

Coming back to Smalltalk from Java, I have to say that I am rather
disappointed that the "grand-daddy" of testing frameworks isn't very helpful
at tracking down why tests fail. I've never spent so much time in a debugger
working out why tests fail, when all of the information that led up to the
error was around but gets buried in a stack trace.

It seems to me that for starters - should:raise: could at least tell you
what exception was raised and what was expected. E.g. I would change the
implemtnation to something like:

should: aBlock raise: anExceptionalEvent
     self should: aBlock raise: anExceptionalEvent description: ''

should: aBlock raise: anErrorClass description: aStringDescription
 | formattedDescripton |
 aStringDescription isEmpty
      ifTrue: [formattedDescripton := '']
      ifFalse: [formattedDescripton := ' (' , aStringDescription , ')'].

 [[aBlock value] sunitOn: anErrorClass do: [:ex | ^true]] sunitOn:
TestResult error
  do:
   [:ex |
       self assert: false
            description: 'Should raise: #' , anErrorClass name , ' but was:
#' , ex class name
              , formattedDescripton].

 self assert: false description: 'Should raise: #' , anErrorClass name ,
formattedDescripton


Furthermore - tests that everyone seems to write like:
    self should: [ object printString = 'test' ]

Should have the infrastructure to allow a much more explicit failure so that
you don't have to restart the relevant #should: method and step through the
block evaluate code to find the problem.

I would propose that SUnit should come bundled with some useful Contraint
objects that can report failures more meaningfully -e.g.
    Equals verify: object printString with: 'test'

// or - add some loose methods to dispatch things and make it more readable:

    object printString shouldEqual: 'test'.

Internally the constraint can generate and internal #assert:description:
with the relevent information to form a decent description when things fail.
e.g. TestFailure signal: 'Contraint error: ',expectedValue, ' not = ',
actualValue

I've played with it for about 15 minutes and its definitely a big time
saver - I just don't understand why this kind of stuff isn't in place
already?

Tim

TimM wrote:

> // or - add some loose methods to dispatch things and make it more
> readable:
>
>     object printString shouldEqual: 'test'.
[...]
> I've played with it for about 15 minutes and its definitely a big time
> saver - I just don't understand why this kind of stuff isn't in place
> already?

Makes sense to me.  I imagine that many people have their own variants on this
(and other similar time savers).   Why not submit your suggestions to the SUnit
people at:
    http://sunit.sourceforge.net/
(where there is an enhancement request page).  Presumably a set of concrete
suggestions accompanied by working code (cross-dialect if possible) would be
accepted more readily.

    -- chris

"TimM" <[hidden email]> wrote in message
news:ddl8g1$qg6$[hidden email]...
> Coming back to Smalltalk from Java, I have to say that I am rather
> disappointed that the "grand-daddy" of testing frameworks isn't very
> helpful at tracking down why tests fail. I've never spent so much time in
> a debugger working out why tests fail, when all of the information that
> led up to the error was around but gets buried in a stack trace.

Well I won't comment on SUnit here as its a community project, except to
point out that the community is rather smaller than you might be used to,
and perhaps somewhat harder pressed. This does mean there is less effort
available to address issues that are probably less painful in Smalltalk if
you work with it. And that is key. If you use the same techniques, you'll
certainly find it less helpful. In static languages you need more help
because there is typically less you can do post mortem.

Here are a couple of techniques I use:
1) I always press the 'Debug' button in the SUnitBrowser when I'm
developing. Mind you this requires a fix that we put in D6 beta 1 so that
the success/failure indicators update in debug runs as well as normal runs.
2) Don't bother trying to work out why something failed by inspection, just
run it again. For example when examining a failure in the debugger rather
than sherlocking around in the stack trace, or even rather than restarting
and stepping through it, just select the text and evaluate it, or parts of
it. If the assertion says it should be equal, evaluate both sides
independently and see what you get.
3) Following on from (2), select the failing assertion expression itself (or
the part of it that you've determined is giving the wrong answer) and debug
through it using another debugger (Debug It command).
4) Following on from (3), I'll often be able to fix the failure in my second
debugger and then return back to the first, restart, and continue the test
run.

>
> It seems to me that for starters - should:raise: could at least tell you
> what exception was raised and what was expected. E.g. I would change the
> implemtnation to something like:
>

Please send any such suggestions to Joseph Pelrine. I'm sure he'll listen.

>... Furthermore - tests that everyone seems to write like:
>    self should: [ object printString = 'test' ]

Why is "everyone"  using that form? The block seems gratuitous. I would
write it:

self assert: object printString = 'test'

Like I say, not enough time and more pressing issues, depending on your
personal itch. If this is something you feel is important, please do go
ahead and put something together and run it by Joseph. If he is no longer
dealing with it, he'll know who is.

Regards

Blair

Tim,

> Like I say, not enough time and more pressing issues, depending on
> your personal itch. If this is something you feel is important,
> please do go ahead and put something together and run it by Joseph.
> If he is no longer dealing with it, he'll know who is.

It seems that Joseph Pelrine is still working with SUnit and, judging
by James Robertsons blog entry for Joseph's ESUG talk, he has added
some enhancements which go at least some way to addressing your
observations.  See here...

http://www.cincomsmalltalk.com/blog/blogView?showComments=true&entry=330
1730651


best regards,


Andy Bower
Dolphin Support
www.object-arts.com