GUI Testing Library

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

GUI Testing Library

Sean P. DeNigris
Administrator
I'm starting to put together some helpers for UI testing. What's out there already?

I started simulating keyboard events. It seems like the place to hook is ActiveHand>>handleEvent:. This way, it will take into account the keyboard focus and route the event the way it would for a real event. Does that sound right?

So far the API adds TestCase>>simulateKeyStrokes: which forwards to ActiveHand>>handleEvent:, so you can put in a test:
testMyMorph
    MyMorph new openInWorld. "MyMorph takes the keyboard focus in #initialize"
    self simulateKeyStrokes: 'abcd'. "keystrokes go to MyMorph instance"

Obviously, there are things missing (like modifier keys), but what are your thoughts on the general approach?

Sean
Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: GUI Testing Library

Sean P. DeNigris
Administrator
Sean P. DeNigris wrote
So far the API adds...
This is fun! I integrated with Keymapping, so now you can write:
        self simulateKeyStrokes: 'Object'.
        self simulateKeyStroke: $s command.

To play, doit:
Gofer new
                squeaksource: 'ShortWays';
                package: 'ConfigurationOfKeymapping';
                load.
(Smalltalk at: #ConfigurationOfKeymapping) project bleedingEdge load.
       
Gofer it
        squeaksource: 'BDDExtensions';
        package: 'BDDExtensions';
        load.
Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: GUI Testing Library

Igor Stasenko
On 23 November 2011 02:09, Sean P. DeNigris <[hidden email]> wrote:

>
> Sean P. DeNigris wrote
>>
>> So far the API adds...
>>
>
> This is fun! I integrated with Keymapping, so now you can write:
>        self simulateKeyStrokes: 'Object'.
>        self simulateKeyStroke: $s command.
>
> To play, doit:
>
Nice!
It would be cool to cover UI stuff with such tests.
Like if you have browser window and press something, it should do
somehting.. etc

>
>> Gofer new
>>               squeaksource: 'ShortWays';
>>               package: 'ConfigurationOfKeymapping';
>>               load.
>> (Smalltalk at: #ConfigurationOfKeymapping) project bleedingEdge load.
>>
>> Gofer it
>>       squeaksource: 'BDDExtensions';
>>       package: 'BDDExtensions';
>>       load.
>>
>
> --
> View this message in context: http://forum.world.st/GUI-Testing-Library-tp4097866p4098126.html
> Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
>
>



--
Best regards,
Igor Stasenko.

Reply | Threaded
Open this post in threaded view
|

Re: GUI Testing Library

Tudor Girba-2
I am quite interested, as well.

I am not sure I understand how to reproduce your approach, though.

Could you provide more information? Or maybe an example?

Cheers,
Doru


On Wed, Nov 23, 2011 at 11:54 AM, Igor Stasenko <[hidden email]> wrote:

> On 23 November 2011 02:09, Sean P. DeNigris <[hidden email]> wrote:
>>
>> Sean P. DeNigris wrote
>>>
>>> So far the API adds...
>>>
>>
>> This is fun! I integrated with Keymapping, so now you can write:
>>        self simulateKeyStrokes: 'Object'.
>>        self simulateKeyStroke: $s command.
>>
>> To play, doit:
>>
> Nice!
> It would be cool to cover UI stuff with such tests.
> Like if you have browser window and press something, it should do
> somehting.. etc
>
>>
>>> Gofer new
>>>               squeaksource: 'ShortWays';
>>>               package: 'ConfigurationOfKeymapping';
>>>               load.
>>> (Smalltalk at: #ConfigurationOfKeymapping) project bleedingEdge load.
>>>
>>> Gofer it
>>>       squeaksource: 'BDDExtensions';
>>>       package: 'BDDExtensions';
>>>       load.
>>>
>>
>> --
>> View this message in context: http://forum.world.st/GUI-Testing-Library-tp4097866p4098126.html
>> Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> Best regards,
> Igor Stasenko.
>
>



--
www.tudorgirba.com
"Every thing has its own flow"

Reply | Threaded
Open this post in threaded view
|

Re: GUI Testing Library

abergel
In reply to this post by Sean P. DeNigris
Hi Sean,

This is very valuable. Currently it is really hard to test GUI in Pharo.

One problem I faced several times is that GUI are not uniformly managed: some GUI widgets use of exceptions (which is cool), some other are blocking the currently thread (which is not cool). For example, I can write:

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
| t |
[self inform: 'blah'] on: Exception do: [:ex | t := ex ].
self assert: t messageText = 'blah'
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Other cool tests:

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
testSupplyAnswerOfFillInTheBlank

        self should: ['blue' = ([UIManager default request: 'Your favorite color?']
                valueSupplyingAnswer: #('Your favorite color?' 'blue'))]
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

However, not all widgets can be tested that way, which is quite problematic. For example, the following code opens a window, which is not one would expect:
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
self should: ['/tmp' = ([UIManager default chooseDirectory]
                valueSupplyingAnswer: #('/tmp'))]
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

I am not sure how easy is to solve this problem, but this is something we need to work on. This is important.

I opened an issue: http://code.google.com/p/pharo/issues/detail?id=5010

Cheers,
Alexandre


On 22 Nov 2011, at 20:27, Sean P. DeNigris wrote:

> I'm starting to put together some helpers for UI testing. What's out there
> already?
>
> I started simulating keyboard events. It seems like the place to hook is
> ActiveHand>>handleEvent:. This way, it will take into account the keyboard
> focus and route the event the way it would for a real event. Does that sound
> right?
>
> So far the API adds TestCase>>simulateKeyStrokes: which forwards to
> ActiveHand>>handleEvent:, so you can put in a test:
> testMyMorph
>    MyMorph new openInWorld. "MyMorph takes the keyboard focus in
> #initialize"
>    self simulateKeyStrokes: 'abcd'. "keystrokes go to MyMorph instance"
>
> Obviously, there are things missing (like modifier keys), but what are your
> thoughts on the general approach?
>
> Sean
>
> --
> View this message in context: http://forum.world.st/GUI-Testing-Library-tp4097866p4097866.html
> Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
>

--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel  http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.






Reply | Threaded
Open this post in threaded view
|

Re: GUI Testing Library

Guillermo Polito
In reply to this post by Sean P. DeNigris

On Tue, Nov 22, 2011 at 10:09 PM, Sean P. DeNigris <[hidden email]> wrote:

This is fun! I integrated with Keymapping, so now you can write:
       self simulateKeyStrokes: 'Object'.
       self simulateKeyStroke: $s command.
 
Haha! great :)

I'll freeze an stable version soon.
Reply | Threaded
Open this post in threaded view
|

Re: GUI Testing Library

Sean P. DeNigris
Administrator
In reply to this post by Tudor Girba-2
Tudor Girba-2 wrote
Could you provide more information? Or maybe an example?
Sure. Thanks for the push to write tests :)

I just uploaded a new version with some simple tests. Here is a snippet:
testSimulateCmdKeystroke

        | ws |
        ws := Workspace open.
        self simulateKeyStrokes: 'var := 3.'.
       
        self simulateKeyStroke: $s command.
       
        self assert: ws hasUnacceptedEdits = false.

I'll be adding functionality as I go because I'm using it to test my own GUIs, but let me know if there's anything specific that you'd really love to see here...

Sean

p.s. only in a live, open, dynamic system like Smalltalk could I do this so easily. I've read books devoted to this topic, but I coded this up in a few minutes.
Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: GUI Testing Library

Tudor Girba-2
Wow, this looks brilliant!

I will definitely give it a try in the following days.

Doru

On Wed, Nov 23, 2011 at 4:10 PM, Sean P. DeNigris <[hidden email]> wrote:

>
> Tudor Girba-2 wrote
>>
>> Could you provide more information? Or maybe an example?
>>
> Sure. Thanks for the push to write tests :)
>
> I just uploaded a new version with some simple tests. Here is a snippet:
> testSimulateCmdKeystroke
>
>        | ws |
>        ws := Workspace open.
>        self simulateKeyStrokes: 'var := 3.'.
>
>        self simulateKeyStroke: $s command.
>
>        self assert: ws hasUnacceptedEdits = false.
>
> I'll be adding functionality as I go because I'm using it to test my own
> GUIs, but let me know if there's anything specific that you'd really love to
> see here...
>
> Sean
>
> p.s. only in a live, open, dynamic system like Smalltalk could I do this so
> easily. I've read books devoted to this topic, but I coded this up in a few
> minutes.
>
> --
> View this message in context: http://forum.world.st/GUI-Testing-Library-tp4097866p4099975.html
> Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
>
>



--
www.tudorgirba.com
"Every thing has its own flow"

Reply | Threaded
Open this post in threaded view
|

Re: GUI Testing Library

Sean P. DeNigris
Administrator
In reply to this post by abergel
Alexandre Bergel-7 wrote
Fix (although I guess there are more widgets to fix?) in 1.4 inbox SLICE-Issue-5010-Testing-GUI-widgets-SeanDeNigris.1

Alexandre Bergel-7 wrote
This is very valuable. Currently it is really hard to test GUI in Pharo.
I think it could be very easy with a little work

Alexandre Bergel-7 wrote
One problem I faced several times is that GUI are not uniformly managed: some GUI widgets use of exceptions (which is cool), some other are blocking the currently thread (which is not cool).
The hook points are in MorphicUIManager's "ui requests" protocol. For each type of interaction (e.g. confirm), in the method that takes the most arguments (e.g. #confirm:label:, not #confirm:), ProvideAnswerNotification is signaled.

Alexandre Bergel-7 wrote
For example, I can write...
Thanks for the examples. I wrote them into Morphic tests (incl. in fix above)

Alexandre Bergel-7 wrote
self should: ['/tmp' = ([UIManager default chooseDirectory]
                valueSupplyingAnswer: #('/tmp'))]
This works now, but the syntax is ...valueSupplyingAnswer: { 'Choose Directory'. FileDirectory on: '/tmp' }.

HTH,
Sean
Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: GUI Testing Library

abergel
I will invite you for a couple of beers at the next esug :-)
I cannot try it right now (bad internet connection), but I checked the source code. It looks good.

Cheers,
Alexandre


On 23 Nov 2011, at 13:12, Sean P. DeNigris wrote:

>
> Alexandre Bergel-7 wrote
>>
>> I opened an issue: http://code.google.com/p/pharo/issues/detail?id=5010
>>
>
> Fix (although I guess there are more widgets to fix?) in 1.4 inbox
> SLICE-Issue-5010-Testing-GUI-widgets-SeanDeNigris.1
>
>
> Alexandre Bergel-7 wrote
>>
>> This is very valuable. Currently it is really hard to test GUI in Pharo.
>>
> I think it could be very easy with a little work
>
>
> Alexandre Bergel-7 wrote
>>
>> One problem I faced several times is that GUI are not uniformly managed:
>> some GUI widgets use of exceptions (which is cool), some other are
>> blocking the currently thread (which is not cool).
>>
> The hook points are in MorphicUIManager's "ui requests" protocol. For each
> type of interaction (e.g. confirm), in the method that takes the most
> arguments (e.g. #confirm:label:, not #confirm:), ProvideAnswerNotification
> is signaled.
>
>
> Alexandre Bergel-7 wrote
>>
>> For example, I can write...
>>
> Thanks for the examples. I wrote them into Morphic tests (incl. in fix
> above)
>
>
> Alexandre Bergel-7 wrote
>>
>> self should: ['/tmp' = ([UIManager default chooseDirectory]
>> valueSupplyingAnswer: #('/tmp'))]
>>
> This works now, but the syntax is ...valueSupplyingAnswer: { 'Choose
> Directory'. FileDirectory on: '/tmp' }.
>
> HTH,
> Sean
>
> --
> View this message in context: http://forum.world.st/GUI-Testing-Library-tp4097866p4100236.html
> Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
>

--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel  http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.






Reply | Threaded
Open this post in threaded view
|

Re: GUI Testing Library

Sean P. DeNigris
Administrator
In reply to this post by Sean P. DeNigris
Starting to add simulated mouse actions...

testSimulateMouseClick

        | textMorph |
        textMorph := TextMorph new contents: ''; openInWorld.
       
        textMorph simulateClick.
       
        self assert: textMorph hasKeyboardFocus.
Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: GUI Testing Library

Sean P. DeNigris
Administrator
In reply to this post by abergel
Alexandre Bergel-7 wrote
I will invite you for a couple of beers at the next esug :-)
Paid in beer?! This is the best job ever ;-)
Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: GUI Testing Library

Ben Coman
In reply to this post by Sean P. DeNigris
Sean P. DeNigris wrote:

> I'm starting to put together some helpers for UI testing. What's out there
> already?
>
> I started simulating keyboard events. It seems like the place to hook is
> ActiveHand>>handleEvent:. This way, it will take into account the keyboard
> focus and route the event the way it would for a real event. Does that sound
> right?
>
> So far the API adds TestCase>>simulateKeyStrokes: which forwards to
> ActiveHand>>handleEvent:, so you can put in a test:
> testMyMorph
>     MyMorph new openInWorld. "MyMorph takes the keyboard focus in
> #initialize"
>     self simulateKeyStrokes: 'abcd'. "keystrokes go to MyMorph instance"
>
> Obviously, there are things missing (like modifier keys), but what are your
> thoughts on the general approach?
>
> Sean
>
> --
> View this message in context: http://forum.world.st/GUI-Testing-Library-tp4097866p4097866.html
> Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
>
>
>  
Perhaps an event recorder and replay could be added.  This may be useful
for instance if a regression GUI bug is introduced between Pharo
versions that I encounter when driving the GUI.  Then in the earlier
version I could record the expected output of what I do in the GUI, then
save that as a test to be loaded into the later version, and also
attached to the issue tracker.

just idle chatter and wild speculation. cheers, Ben

Reply | Threaded
Open this post in threaded view
|

Re: GUI Testing Library

Tudor Girba-2
In reply to this post by Tudor Girba-2
Hi Sean,

I just wrote the first test case for Glamour using the BDDExtensions. This was for a bug that that was present in Glamour since several months, but I just did not know how to test.

With the new facilities, it got as simple as:

testExplicitAcceptDoesNotAffectTextPort
        ...
        textMorph := self find: GLMPluggableTextMorph in: window.
        textMorph simulateClick.
        self simulateKeyStrokes: '4'.
        self assert: (browser panes first port: #text) value = '1234'.
        self simulateKeyStroke: $s command.
        self simulateKeyStrokes: '56'.
        self assert: (browser panes first port: #text) value = '123456'

It's just cool.

Thanks,
Doru


On 23 Nov 2011, at 16:17, Tudor Girba wrote:

> Wow, this looks brilliant!
>
> I will definitely give it a try in the following days.
>
> Doru
>
> On Wed, Nov 23, 2011 at 4:10 PM, Sean P. DeNigris <[hidden email]> wrote:
>>
>> Tudor Girba-2 wrote
>>>
>>> Could you provide more information? Or maybe an example?
>>>
>> Sure. Thanks for the push to write tests :)
>>
>> I just uploaded a new version with some simple tests. Here is a snippet:
>> testSimulateCmdKeystroke
>>
>>        | ws |
>>        ws := Workspace open.
>>        self simulateKeyStrokes: 'var := 3.'.
>>
>>        self simulateKeyStroke: $s command.
>>
>>        self assert: ws hasUnacceptedEdits = false.
>>
>> I'll be adding functionality as I go because I'm using it to test my own
>> GUIs, but let me know if there's anything specific that you'd really love to
>> see here...
>>
>> Sean
>>
>> p.s. only in a live, open, dynamic system like Smalltalk could I do this so
>> easily. I've read books devoted to this topic, but I coded this up in a few
>> minutes.
>>
>> --
>> View this message in context: http://forum.world.st/GUI-Testing-Library-tp4097866p4099975.html
>> Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> www.tudorgirba.com
> "Every thing has its own flow"

--
www.tudorgirba.com

"From an abstract enough point of view, any two things are similar."




Reply | Threaded
Open this post in threaded view
|

Re: GUI Testing Library

Sean P. DeNigris
Administrator
Tudor Girba-2 wrote
It's just cool.
Excellent! The latest package in the repo adds right clicking, choosing from menus, and a bug fix for Morphs that use local coordinates.
Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: GUI Testing Library

Tudor Girba-2
Hi Sean,

Unfortunately, it seems that Keymapping introduces subtle bugs in the other tools.

Would it be possible to simulate keys with modifiers without Keymapping present?

Cheers,
Doru


On 3 Dec 2011, at 17:38, Sean P. DeNigris wrote:

>
> Tudor Girba-2 wrote
>>
>> It's just cool.
>>
>
> Excellent! The latest package in the repo adds right clicking, choosing from
> menus, and a bug fix for Morphs that use local coordinates.
>
> --
> View this message in context: http://forum.world.st/GUI-Testing-Library-tp4097866p4153835.html
> Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
>

--
www.tudorgirba.com

If you can't say why something is relevant,
it probably isn't.


Reply | Threaded
Open this post in threaded view
|

Re: GUI Testing Library

Sean P. DeNigris
Administrator
Tudor Girba-2 wrote
Would it be possible to simulate keys with modifiers without Keymapping present?
I think I only used Keymapping for the modifier key syntax, but looking at its repository, the shortcuts are packaged separately. So you could try only loading Keymapping-Shortcuts, which does not hook into the system (so shouldn't break anything) and will add #command, $shift, etc. Let me know if that works for you.

Sean
Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: GUI Testing Library

Tudor Girba-2
Thanks a lot, Sean!

It works just great with BDDExtensions and Keymapping-Shortcuts.

Cheers,
Doru


On Thu, Dec 8, 2011 at 3:35 PM, Sean P. DeNigris <[hidden email]> wrote:

>
> Tudor Girba-2 wrote
>>
>> Would it be possible to simulate keys with modifiers without Keymapping
>> present?
>>
>
> I think I only used Keymapping for the modifier key syntax, but looking at
> its repository, the shortcuts are packaged separately. So you could try only
> loading Keymapping-Shortcuts, which does not hook into the system (so
> shouldn't break anything) and will add #command, $shift, etc. Let me know if
> that works for you.
>
> Sean
>
> --
> View this message in context: http://forum.world.st/GUI-Testing-Library-tp4097866p4172952.html
> Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
>



--
www.tudorgirba.com

"Every thing has its own flow"

Reply | Threaded
Open this post in threaded view
|

Re: GUI Testing Library

Tudor Girba-2
Btw, I think these should get in the Pharo distribution because they
are just too valuable.

Doru


On Fri, Dec 9, 2011 at 4:07 PM, Tudor Girba <[hidden email]> wrote:

> Thanks a lot, Sean!
>
> It works just great with BDDExtensions and Keymapping-Shortcuts.
>
> Cheers,
> Doru
>
>
> On Thu, Dec 8, 2011 at 3:35 PM, Sean P. DeNigris <[hidden email]> wrote:
>>
>> Tudor Girba-2 wrote
>>>
>>> Would it be possible to simulate keys with modifiers without Keymapping
>>> present?
>>>
>>
>> I think I only used Keymapping for the modifier key syntax, but looking at
>> its repository, the shortcuts are packaged separately. So you could try only
>> loading Keymapping-Shortcuts, which does not hook into the system (so
>> shouldn't break anything) and will add #command, $shift, etc. Let me know if
>> that works for you.
>>
>> Sean
>>
>> --
>> View this message in context: http://forum.world.st/GUI-Testing-Library-tp4097866p4172952.html
>> Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
>>
>
>
>
> --
> www.tudorgirba.com
>
> "Every thing has its own flow"



--
www.tudorgirba.com

"Every thing has its own flow"

Reply | Threaded
Open this post in threaded view
|

Re: GUI Testing Library

Marcus Denker-4
In reply to this post by Tudor Girba-2

On Dec 9, 2011, at 4:08 PM, Tudor Girba wrote:

> Btw, I think these should get in the Pharo distribution because they
> are just too valuable.
>

Yes!

> Doru
>
>
> On Fri, Dec 9, 2011 at 4:07 PM, Tudor Girba <[hidden email]> wrote:
>> Thanks a lot, Sean!
>>
>> It works just great with BDDExtensions and Keymapping-Shortcuts.
>>
>> Cheers,
>> Doru
>>
>>
>> On Thu, Dec 8, 2011 at 3:35 PM, Sean P. DeNigris <[hidden email]> wrote:
>>>
>>> Tudor Girba-2 wrote
>>>>
>>>> Would it be possible to simulate keys with modifiers without Keymapping
>>>> present?
>>>>
>>>
>>> I think I only used Keymapping for the modifier key syntax, but looking at
>>> its repository, the shortcuts are packaged separately. So you could try only
>>> loading Keymapping-Shortcuts, which does not hook into the system (so
>>> shouldn't break anything) and will add #command, $shift, etc. Let me know if
>>> that works for you.
>>>
>>> Sean
>>>
>>> --
>>> View this message in context: http://forum.world.st/GUI-Testing-Library-tp4097866p4172952.html
>>> Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
>>>
>>
>>
>>
>> --
>> www.tudorgirba.com
>>
>> "Every thing has its own flow"
>
>
>
> --
> www.tudorgirba.com
>
> "Every thing has its own flow"
>

--
Marcus Denker -- http://marcusdenker.de


12