Smalltalk › Pharo › Pharo Smalltalk Developers

[ENH]: Test Failure Browser

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

8 messages Options

Sean P. DeNigris

[ENH]: Test Failure Browser

Administrator

Issue 6457: [ENH]: Test Failure Browser
http://code.google.com/p/pharo/issues/detail?id=6457

This can be included in the Jenkins test images...

Browse or debug test failures serialized as fuel files.

The Jenkins test script could be enhanced to leave an instance open if there are test failures, so that you would have a dashboard to work from just by downloading and launching the image...

Todo:
* Toggle button enabling (commented out right now, seems to be a bug in Spec - always passes nil to #whenSelectionChanged:
* I want the initial width to be big enough for the contents; it seems like that should be the default. For now, I hardcoded an initialExtent
* Layout: how do I specify a toolbar of buttons with static height and that the next row should take up the rest of the height, but not impinge on the toolbar?

Cheers,
Sean

Cheers,
Sean

Sean P. DeNigris

Re: [ENH]: Test Failure Browser

Administrator

Here (on Nabble) is a screenshot of the browser and a test failure context materialized by clicking "Debug"...

Cheers,
Sean

Mariano Martinez Peck

Re: [ENH]: Test Failure Browser

Thanks Sean!!! THIS IS AWESOME.

I was trying to do exactly that yesterday, but I failed.

I like TestsReviver rather than TestsMedic, but that's a detail :)

On Wed, Jul 25, 2012 at 5:31 PM, Sean P. DeNigris <[hidden email]> wrote:

Here (on Nabble) is a screenshot of the browser and a test failure context
materialized by clicking "Debug"...

http://forum.world.st/file/n4641533/Screen_Shot_2012-07-25_at_11.22.08_AM.png

--
View this message in context: http://forum.world.st/ENH-Test-Failure-Browser-tp4641525p4641533.html

Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

--
Mariano
http://marianopeck.wordpress.com

Camillo Bruni-3

Re: [ENH]: Test Failure Browser

nice++

On 2012-07-25, at 17:44, Mariano Martinez Peck wrote:

> Thanks Sean!!! THIS IS AWESOME.
> I was trying to do exactly that yesterday, but I failed.
> I like TestsReviver rather than TestsMedic, but that's a detail :)
>
> On Wed, Jul 25, 2012 at 5:31 PM, Sean P. DeNigris <[hidden email]>wrote:
>
>> Here (on Nabble) is a screenshot of the browser and a test failure context
>> materialized by clicking "Debug"...
>>
>>
>> http://forum.world.st/file/n4641533/Screen_Shot_2012-07-25_at_11.22.08_AM.png
>>
>>
>>
>> --
>> View this message in context:
>> http://forum.world.st/ENH-Test-Failure-Browser-tp4641525p4641533.html
>> Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
>>
>>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com

Mariano Martinez Peck

Re: [ENH]: Test Failure Browser

Hi Sean. I have improved a couple of things in Fuel. Now you can:

a) attach additional objects to the object to serialize (useful for custom metadata)
b) add pre and post materialization actions (clean closures)

Reviving failed unit tests is cool, but we still miss the OTHER information that is in a PharoDebug.log which is not the stack, that is, VM version, image version, timetamp, etc. So what I did in HDTestReport is to store all that info as "additional objects" to the serialization of the test failure.

Then, what I also did is to register a post materialization action that directly opens the debugger :)

So...TestMedic (anyway, I think TestReviver is much cooler) now just need to do:

materialize: fuelFileReference

FLMaterializer materializeFromFileNamed: fuelFileReference fullName.

And...what I want to do (but no time today) is to modify the UI so that when you select a test, on the right side you have a panel that displays the vm version, timestamp and image version of the failed test. To do that you can do:

header := FLMaterializer materializeHeaderFromFileNamed: 'FLBasicSerializationTest-testBitmap.fuel'.

header additionalObjectAt: #VM. "this answers the vm string"

header additionalObjectAt: #Image.

header additionalObjectAt: #Timestamp.

so you can just take that and print it in the UI :)

So...if you think this is useful then I can:

1) create an issue to integrate new version of fuel ( you can find this new code in ss3 last commits).

2) create an issue to integrate new HDTestReport ( I attach here the new version.)

3) add anything else apart from vm, image and timestamp that you feel that would be a good idea.

Cheers,

On Wed, Jul 25, 2012 at 6:16 PM, Camillo Bruni <[hidden email]> wrote:

nice++

On 2012-07-25, at 17:44, Mariano Martinez Peck wrote:

> Thanks Sean!!! THIS IS AWESOME.
> I was trying to do exactly that yesterday, but I failed.
> I like TestsReviver rather than TestsMedic, but that's a detail :)
>
> On Wed, Jul 25, 2012 at 5:31 PM, Sean P. DeNigris <[hidden email]>wrote:
>
>> Here (on Nabble) is a screenshot of the browser and a test failure context
>> materialized by clicking "Debug"...
>>
>>
>> http://forum.world.st/file/n4641533/Screen_Shot_2012-07-25_at_11.22.08_AM.png
>>
>>
>>
>> --
>> View this message in context:
>> http://forum.world.st/ENH-Test-Failure-Browser-tp4641525p4641533.html
>> Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
>>
>>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com

--
Mariano
http://marianopeck.wordpress.com

HDTestReport.st (11K) Download Attachment

Sean P. DeNigris

Re: [ENH]: Test Failure Browser

Administrator

Mariano Martinez Peck wrote

if you think this is useful

It all sounds great; go for it!

Cheers,
Sean

Mariano Martinez Peck

Re: [ENH]: Test Failure Browser

So...I did it. With the latest Pharo update 20238 (with new Fuel version) and my latest commit HudsonBuildTools20-MarianoMartinezPeck.3 to CISupport, we have all what I mentioned here. So all you have to do is:

FLMaterializer materializeFromFileNamed: 'XXXTest-testYYY.fuel'.

I have also updated the job description in: https://ci.lille.inria.fr/pharo/view/Pharo%202.0/job/pharo-2.0-tests/

Finally, for Sean, it would awesome if you can now improve TestMedic to display the debug information of a selected test before debugging it. To get the info, you can do:

| header |

header := FLMaterializer materializeHeaderFromFileNamed: 'FLFullBasicSerializationTest-testDate.fuel'.

header additionalObjectAt: #VM.

header additionalObjectAt: #Image.

header additionalObjectAt: #Timestamp.

those last 3 lines give you the info you would need to show.

I can give it a try, but I suck in UI/Spec and I have some pending things to do with Fuel and Tanker.

Cheers,

On Sun, Jul 29, 2012 at 4:36 PM, Sean P. DeNigris <[hidden email]> wrote:

Mariano Martinez Peck wrote

>
> if you think this is useful

It all sounds great; go for it!

--
View this message in context: http://forum.world.st/ENH-Test-Failure-Browser-tp4641525p4642057.html

Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

--
Mariano
http://marianopeck.wordpress.com

Sean P. DeNigris

Re: [ENH]: Test Failure Browser

Administrator

Mariano Martinez Peck wrote

it would awesome if you can now improve TestMedic to
display the debug information of a selected test before debugging it.

Sounds easy enough...

Cheers,
Sean