[Failing Tests] Cleaned up failing test serialization

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

[Failing Tests] Cleaned up failing test serialization

Marcus Denker-4
Hi,

To make it easy to explore why a test fails on the test server we serialize the stack of falling tests.

With two changes this has now gotten much easier:

1) Drag and Drop support for .fuel file
Just drag the file to the Pharo Desktop --> Debugger opens

2) There where always a lot of fuel files. Reason: Expected failures where serialized, too.
     This is now changed and the only serialized tests are those that failed.

E.g for linux, you can download them here:

        https://ci.inria.fr/pharo/job/Pharo-2.0-Tests/lastCompletedBuild/label=linux/


        Marcus
Reply | Threaded
Open this post in threaded view
|

Re: [Failing Tests] Cleaned up failing test serialization

Ben Coman
Marcus Denker wrote:
> Hi,
>
> To make it easy to explore why a test fails on the test server we serialize the stack of falling tests.
>
> With two changes this has now gotten much easier:
>
> 1) Drag and Drop support for .fuel file
> Just drag the file to the Pharo Desktop --> Debugger opens
>  

Could you advise how to do this from an operational image (that is, not
a CI test image).  I would like to use this to demonstrate Pharo's
outstanding development environment.     A killer feature would be the
debugger having a <File out> button, that a user could use to mail to
support.  Does any other programming system allow this ?

> 2) There where always a lot of fuel files. Reason: Expected failures where serialized, too.
>      This is now changed and the only serialized tests are those that failed.
>
> E.g for linux, you can download them here:
>
> https://ci.inria.fr/pharo/job/Pharo-2.0-Tests/lastCompletedBuild/label=linux/
>
>
> Marcus
>
>  


Reply | Threaded
Open this post in threaded view
|

Re: [Failing Tests] Cleaned up failing test serialization

Camillo Bruni-3

On 2013-02-16, at 14:11, Ben Coman <[hidden email]> wrote:

> Marcus Denker wrote:
>> Hi,
>>
>> To make it easy to explore why a test fails on the test server we serialize the stack of falling tests.
>>
>> With two changes this has now gotten much easier:
>>
>> 1) Drag and Drop support for .fuel file
>> Just drag the file to the Pharo Desktop --> Debugger opens
>>  
>
> Could you advise how to do this from an operational image (that is, not a CI test image).  I would like to use this to demonstrate Pharo's outstanding development environment.     A killer feature would be the debugger having a <File out> button, that a user could use to mail to support.  Does any other programming system allow this ?


you can just select "fileout / serialize stack" in the debugger somewhere.
The you get a local .fuel file with the serialized stack :)
Reply | Threaded
Open this post in threaded view
|

Re: [Failing Tests] Cleaned up failing test serialization

Camillo Bruni-3

On 2013-02-16, at 14:14, Camillo Bruni <[hidden email]> wrote:

>
> On 2013-02-16, at 14:11, Ben Coman <[hidden email]> wrote:
>
>> Marcus Denker wrote:
>>> Hi,
>>>
>>> To make it easy to explore why a test fails on the test server we serialize the stack of falling tests.
>>>
>>> With two changes this has now gotten much easier:
>>>
>>> 1) Drag and Drop support for .fuel file
>>> Just drag the file to the Pharo Desktop --> Debugger opens
>>>
>>
>> Could you advise how to do this from an operational image (that is, not a CI test image).  I would like to use this to demonstrate Pharo's outstanding development environment.     A killer feature would be the debugger having a <File out> button, that a user could use to mail to support.  Does any other programming system allow this ?
>
>
> you can just select "fileout / serialize stack" in the debugger somewhere.
> The you get a local .fuel file with the serialized stack :)

it's the last entry "File out Stack" in the right-click menu of the debugger
Reply | Threaded
Open this post in threaded view
|

Re: [Failing Tests] Cleaned up failing test serialization

Marcus Denker-4
In reply to this post by Camillo Bruni-3

On Feb 16, 2013, at 2:14 PM, Camillo Bruni <[hidden email]> wrote:

>
> On 2013-02-16, at 14:11, Ben Coman <[hidden email]> wrote:
>
>> Marcus Denker wrote:
>>> Hi,
>>>
>>> To make it easy to explore why a test fails on the test server we serialize the stack of falling tests.
>>>
>>> With two changes this has now gotten much easier:
>>>
>>> 1) Drag and Drop support for .fuel file
>>> Just drag the file to the Pharo Desktop --> Debugger opens
>>>
>>
>> Could you advise how to do this from an operational image (that is, not a CI test image).  I would like to use this to demonstrate Pharo's outstanding development environment.     A killer feature would be the debugger having a <File out> button, that a user could use to mail to support.  Does any other programming system allow this ?
>
>
> you can just select "fileout / serialize stack" in the debugger somewhere.
> The you get a local .fuel file with the serialized stack :)

We should

        -> rename it "Fuel out stack"
        -> set a post-dematerialization action

Because right now drag-and-drop does not lead to any result.

        Marcus
Reply | Threaded
Open this post in threaded view
|

Re: [Failing Tests] Cleaned up failing test serialization

Mariano Martinez Peck


On Sat, Feb 16, 2013 at 10:16 AM, Marcus Denker <[hidden email]> wrote:

On Feb 16, 2013, at 2:14 PM, Camillo Bruni <[hidden email]> wrote:

>
> On 2013-02-16, at 14:11, Ben Coman <[hidden email]> wrote:
>
>> Marcus Denker wrote:
>>> Hi,
>>>
>>> To make it easy to explore why a test fails on the test server we serialize the stack of falling tests.
>>>
>>> With two changes this has now gotten much easier:
>>>
>>> 1) Drag and Drop support for .fuel file
>>> Just drag the file to the Pharo Desktop --> Debugger opens
>>>
>>
>> Could you advise how to do this from an operational image (that is, not a CI test image).  I would like to use this to demonstrate Pharo's outstanding development environment.     A killer feature would be the debugger having a <File out> button, that a user could use to mail to support.  Does any other programming system allow this ?
>
>
> you can just select "fileout / serialize stack" in the debugger somewhere.
> The you get a local .fuel file with the serialized stack :)

We should

        -> rename it "Fuel out stack"

+1
 
        -> set a post-dematerialization action


Yes, you can copy it from the HDTestRunner that does exactly this :)

 
Because right now drag-and-drop does not lead to any result.

        Marcus



--
Mariano
http://marianopeck.wordpress.com
Reply | Threaded
Open this post in threaded view
|

Re: [Failing Tests] Cleaned up failing test serialization

Camillo Bruni-3
In reply to this post by Marcus Denker-4
> We should
>
> -> rename it "Fuel out stack"
> -> set a post-dematerialization action
> Because right now drag-and-drop does not lead to any result.



that is strange.. since I remember Mariano added a post action
that opens the debugger:

see HDTestReport>>#serializeTestFailureContext:toFileNamed:
-------------------------------------------------------------------
...
serializer addPostMaterializationAction: [:materialization |
                Smalltalk tools debugger
                        openContext: materialization root
                    label: 'This is the new debugger!'
                    contents: nil
                ].
...
-------------------------------------------------------------------
so there must be something wrong with drag n'drop handler...

Reply | Threaded
Open this post in threaded view
|

Re: [Failing Tests] Cleaned up failing test serialization

Mariano Martinez Peck


On Sat, Feb 16, 2013 at 10:27 AM, Camillo Bruni <[hidden email]> wrote:
> We should
>
>       -> rename it "Fuel out stack"
>       -> set a post-dematerialization action
> Because right now drag-and-drop does not lead to any result.



that is strange.. since I remember Mariano added a post action
that opens the debugger:


But I think Marcus is talking about the button in the debugger, not the HDTestRunner
 
see HDTestReport>>#serializeTestFailureContext:toFileNamed:
-------------------------------------------------------------------
...
serializer addPostMaterializationAction: [:materialization |
                Smalltalk tools debugger
                        openContext: materialization root
                        label: 'This is the new debugger!'
                        contents: nil
                ].
...
-------------------------------------------------------------------
so there must be something wrong with drag n'drop handler...




--
Mariano
http://marianopeck.wordpress.com
Reply | Threaded
Open this post in threaded view
|

Re: [Failing Tests] Cleaned up failing test serialization

Marcus Denker-4

On Feb 16, 2013, at 2:30 PM, Mariano Martinez Peck <[hidden email]> wrote:



On Sat, Feb 16, 2013 at 10:27 AM, Camillo Bruni <[hidden email]> wrote:
> We should
>
>       -> rename it "Fuel out stack"
>       -> set a post-dematerialization action
> Because right now drag-and-drop does not lead to any result.



that is strange.. since I remember Mariano added a post action
that opens the debugger:


But I think Marcus is talking about the button in the debugger, not the HDTestRunner
yes!



Marcus

Reply | Threaded
Open this post in threaded view
|

Re: [Failing Tests] Cleaned up failing test serialization

Camillo Bruni-3

On 2013-02-16, at 14:30, Marcus Denker <[hidden email]> wrote:

>
> On Feb 16, 2013, at 2:30 PM, Mariano Martinez Peck <[hidden email]> wrote:
>
>>
>>
>> On Sat, Feb 16, 2013 at 10:27 AM, Camillo Bruni <[hidden email]> wrote:
>>> We should
>>>
>>>      -> rename it "Fuel out stack"
>>>      -> set a post-dematerialization action
>>> Because right now drag-and-drop does not lead to any result.
>>
>>
>>
>> that is strange.. since I remember Mariano added a post action
>> that opens the debugger:
>>
>>
>> But I think Marcus is talking about the button in the debugger, not the HDTestRunner

sorry, right... if I were fully awake I'd have realized :D


Reply | Threaded
Open this post in threaded view
|

Re: [Failing Tests] Cleaned up failing test serialization

EstebanLM
In reply to this post by Marcus Denker-4
talking about the debugger... yesterday we were talking about putting also a "report bug" button there, for 3.0 :)

Esteban

On Feb 16, 2013, at 2:30 PM, Marcus Denker <[hidden email]> wrote:


On Feb 16, 2013, at 2:30 PM, Mariano Martinez Peck <[hidden email]> wrote:



On Sat, Feb 16, 2013 at 10:27 AM, Camillo Bruni <[hidden email]> wrote:
> We should
>
>       -> rename it "Fuel out stack"
>       -> set a post-dematerialization action
> Because right now drag-and-drop does not lead to any result.



that is strange.. since I remember Mariano added a post action
that opens the debugger:


But I think Marcus is talking about the button in the debugger, not the HDTestRunner
yes!



Marcus


Reply | Threaded
Open this post in threaded view
|

Re: [Failing Tests] Cleaned up failing test serialization

Ben Coman
In reply to this post by Camillo Bruni-3
Camillo Bruni wrote:

> On 2013-02-16, at 14:14, Camillo Bruni <[hidden email]> wrote:
>
>  
>> On 2013-02-16, at 14:11, Ben Coman <[hidden email]> wrote:
>>
>>    
>>> Marcus Denker wrote:
>>>      
>>>> Hi,
>>>>
>>>> To make it easy to explore why a test fails on the test server we serialize the stack of falling tests.
>>>>
>>>> With two changes this has now gotten much easier:
>>>>
>>>> 1) Drag and Drop support for .fuel file
>>>> Just drag the file to the Pharo Desktop --> Debugger opens
>>>>
>>>>        
>>> Could you advise how to do this from an operational image (that is, not a CI test image).  I would like to use this to demonstrate Pharo's outstanding development environment.     A killer feature would be the debugger having a <File out> button, that a user could use to mail to support.  Does any other programming system allow this ?
>>>      
>> you can just select "fileout / serialize stack" in the debugger somewhere.
>> The you get a local .fuel file with the serialized stack :)
>>    
>
> it's the last entry "File out Stack" in the right-click menu of the debugger
>
>  
Brilliant!  I thought the idea was obvious :)

Now that sparks a following thought, what might be interesting for
customer/support interaction would a mechanism packaging together:
* the serialized stack
* a manifest of all loaded ConfigurationOfXXX versions
* the diff of any dirty ConfigurationOfXXX
subject to user review to avoid leaking proprietary information.

cheers, Ben

Reply | Threaded
Open this post in threaded view
|

Re: [Failing Tests] Cleaned up failing test serialization

EstebanLM
yep... something like that :)


On Sat, Feb 16, 2013 at 3:25 PM, Ben Coman <[hidden email]> wrote:
Camillo Bruni wrote:
On 2013-02-16, at 14:14, Camillo Bruni <[hidden email]> wrote:

 
On 2013-02-16, at 14:11, Ben Coman <[hidden email]> wrote:

   
Marcus Denker wrote:
     
Hi,

To make it easy to explore why a test fails on the test server we serialize the stack of falling tests.

With two changes this has now gotten much easier:

1) Drag and Drop support for .fuel file
Just drag the file to the Pharo Desktop --> Debugger opens

       
Could you advise how to do this from an operational image (that is, not a CI test image).  I would like to use this to demonstrate Pharo's outstanding development environment.     A killer feature would be the debugger having a <File out> button, that a user could use to mail to support.  Does any other programming system allow this ?
     
you can just select "fileout / serialize stack" in the debugger somewhere.
The you get a local .fuel file with the serialized stack :)
   

it's the last entry "File out Stack" in the right-click menu of the debugger

 
Brilliant!  I thought the idea was obvious :)

Now that sparks a following thought, what might be interesting for customer/support interaction would a mechanism packaging together:
* the serialized stack
* a manifest of all loaded ConfigurationOfXXX versions
* the diff of any dirty ConfigurationOfXXX
subject to user review to avoid leaking proprietary information.

cheers, Ben


Reply | Threaded
Open this post in threaded view
|

Re: [Failing Tests] Cleaned up failing test serialization

tinchodias
Yes, post materialization should be added in Debugger>>serializeStack
like it's been done in
HDTestReport>>serializeTestFailureContext:toFileNamed:

To avoid code replication, we may create a extension method to
MethodContext, #serializeOn: that receives a stream.

Martin

On Sat, Feb 16, 2013 at 3:31 PM, Esteban Lorenzano <[hidden email]> wrote:

> yep... something like that :)
>
>
> On Sat, Feb 16, 2013 at 3:25 PM, Ben Coman <[hidden email]> wrote:
>>
>> Camillo Bruni wrote:
>>>
>>> On 2013-02-16, at 14:14, Camillo Bruni <[hidden email]> wrote:
>>>
>>>
>>>>
>>>> On 2013-02-16, at 14:11, Ben Coman <[hidden email]> wrote:
>>>>
>>>>
>>>>>
>>>>> Marcus Denker wrote:
>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> To make it easy to explore why a test fails on the test server we
>>>>>> serialize the stack of falling tests.
>>>>>>
>>>>>> With two changes this has now gotten much easier:
>>>>>>
>>>>>> 1) Drag and Drop support for .fuel file
>>>>>> Just drag the file to the Pharo Desktop --> Debugger opens
>>>>>>
>>>>>>
>>>>>
>>>>> Could you advise how to do this from an operational image (that is, not
>>>>> a CI test image).  I would like to use this to demonstrate Pharo's
>>>>> outstanding development environment.     A killer feature would be the
>>>>> debugger having a <File out> button, that a user could use to mail to
>>>>> support.  Does any other programming system allow this ?
>>>>>
>>>>
>>>> you can just select "fileout / serialize stack" in the debugger
>>>> somewhere.
>>>> The you get a local .fuel file with the serialized stack :)
>>>>
>>>
>>>
>>> it's the last entry "File out Stack" in the right-click menu of the
>>> debugger
>>>
>>>
>>
>> Brilliant!  I thought the idea was obvious :)
>>
>> Now that sparks a following thought, what might be interesting for
>> customer/support interaction would a mechanism packaging together:
>> * the serialized stack
>> * a manifest of all loaded ConfigurationOfXXX versions
>> * the diff of any dirty ConfigurationOfXXX
>> subject to user review to avoid leaking proprietary information.
>>
>> cheers, Ben
>>
>