Testing a VW GUI
Locked
 
|
Hello everyone,
since this question came up a couple of times when discussing languages and testing: What tools could one use to test VW GUIs, both during development and for final testing of the ready-to-deploy application? _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
Dear Claus,
>What tools could one use to test VW GUIs, both during development and >for final testing of the ready-to-deploy application? > I list the approaches and tools I've encountered. A) Test 'from the back' by subclassing or delegating top-of-model-layer tests to become UI tests: see eXtreme UI Testing, my talk at Smalltalk Solutions 2007. (Scroll to the foot of http://www.esug.org/Conferences to get a link to my write-up of the conference.) In the Cincom open repository, examples of these approaches are in the RBDynamicRefactoringsTests, in RBCustomerBrowserUITests and in the CS... versions of the RBTests . I see all these are only in that repository in such old versions that you'll have to use VW7.5 to run them - sorry. (I have up-to-date versions and will try to get around to making them visible.) The following text is largely an edit of the first part of my talk, where I review alternatives. B) Smalltalk tools: - David Buck’s VWUnit: ApplicationModels are located by test-supplied, test-specific name. Widgets are located by relative location (to others and/or within bounds) and/or name/id. (See his talk at Smalltalk Solutions 2003. Also c.f. some similarities of style with widget location and manipulation in David Shaeffer’s Seaside Testing Framework; see David’s talks at Smalltalk Solutions 2005 and at ESUG 2005.) VWUnit is in the open repository. - Silvermark’s Test Mentor offers click and type test generation for UI’s built from standard VW or VA widgets (it can need adaption to applicationspecific UI frameworks). It locates widgets by name if they are named, or by generated name if possible, or by the path from the enclosing subcanvas or window when duplicate names occur. A trial version is available to be installed on the iso. I do not know the current usage state of either of these tools, or if any further development of them has been done in recent years. C) Non-Smalltalk tools that have been used on Smalltalk. - James Foster wrote Smalltalk-oriented extensions to WinRunner. WinRunner offers click and type test generation for a fixed (inaccurate) view of a Smalltalk application’s UI. Its widget positions are fragile and its widget names are not unique. James’ add-ons help you tweak your application to fit WinRunner’s expectations better (see his talk at Smalltalk Solutions 2004). My experience of using these tools is very limited (limited commercial experience of Silvermark's TestRunner, even more limited hobby/open-source use of VWUnit, none of James Foster’s work) and are now all from some years ago, as I use my own approach (when I do adequate UI testing, which is far from always). Hence the following comments should be read with caution. I find them good for retrofitting UI tests to stable systems, and for automating acceptance-style tests, etc. They are also useful as examples of how to write widget-driving code. But somehow they lack the XP test-driven feel. In ‘A posteriori’ use, the tests tend to be brittle under refactoring. This brittleness effectively discourages ‘A priori’ use: you tend to build a stable UI then retro-fit its tests (and it’s hard to click a not-yet-existing widget to generate a test :-). HTH Niall Ross _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
Hello Niall, thank you a lot for this. This helps me a lot. Judging from the absence of other responses, I find it interesting that not many people seem to use tests like this. -- Claus Kick "Wenn Sie mich suchen: Ich halte mich in der Nähe des Wahnsinns auf. Genauer gesagt auf der schmalen Linie zwischen Wahnsinn und Panik. Gleich um die Ecke von Todesangst, nicht weit weg von Irrwitz und Idiotie." "If you are looking for me: I am somewhere near to lunacy. More clearly, on the narrow path between lunacy and panic. Right around the corner of fear of death, not far away from idiocy and insanity."
Gesendet: Dienstag, 14. Februar 2012 um 14:04 Uhr
Von: "Niall Ross" <[hidden email]> An: "Claus Kick" <[hidden email]> Cc: "VWNC List" <[hidden email]> Betreff: Re: [vwnc] Testing a VW GUI
Dear Claus,
>What tools could one use to test VW GUIs, both during development and >for final testing of the ready-to-deploy application? > I list the approaches and tools I've encountered. A) Test 'from the back' by subclassing or delegating top-of-model-layer tests to become UI tests: see eXtreme UI Testing, my talk at Smalltalk Solutions 2007. (Scroll to the foot of http://www.esug.org/Conferences to get a link to my write-up of the conference.) In the Cincom open repository, examples of these approaches are in the RBDynamicRefactoringsTests, in RBCustomerBrowserUITests and in the CS... versions of the RBTests . I see all these are only in that repository in such old versions that you'll have to use VW7.5 to run them - sorry. (I have up-to-date versions and will try to get around to making them visible.) The following text is largely an edit of the first part of my talk, where I review alternatives. B) Smalltalk tools: - David Buck’s VWUnit: ApplicationModels are located by test-supplied, test-specific name. Widgets are located by relative location (to others and/or within bounds) and/or name/id. (See his talk at Smalltalk Solutions 2003. Also c.f. some similarities of style with widget location and manipulation in David Shaeffer’s Seaside Testing Framework; see David’s talks at Smalltalk Solutions 2005 and at ESUG 2005.) VWUnit is in the open repository. - Silvermark’s Test Mentor offers click and type test generation for UI’s built from standard VW or VA widgets (it can need adaption to applicationspecific UI frameworks). It locates widgets by name if they are named, or by generated name if possible, or by the path from the enclosing subcanvas or window when duplicate names occur. A trial version is available to be installed on the iso. I do not know the current usage state of either of these tools, or if any further development of them has been done in recent years. C) Non-Smalltalk tools that have been used on Smalltalk. - James Foster wrote Smalltalk-oriented extensions to WinRunner. WinRunner offers click and type test generation for a fixed (inaccurate) view of a Smalltalk application’s UI. Its widget positions are fragile and its widget names are not unique. James’ add-ons help you tweak your application to fit WinRunner’s expectations better (see his talk at Smalltalk Solutions 2004). My experience of using these tools is very limited (limited commercial experience of Silvermark's TestRunner, even more limited hobby/open-source use of VWUnit, none of James Foster’s work) and are now all from some years ago, as I use my own approach (when I do adequate UI testing, which is far from always). Hence the following comments should be read with caution. I find them good for retrofitting UI tests to stable systems, and for automating acceptance-style tests, etc. They are also useful as examples of how to write widget-driving code. But somehow they lack the XP test-driven feel. In ‘A posteriori’ use, the tests tend to be brittle under refactoring. This brittleness effectively discourages ‘A priori’ use: you tend to build a stable UI then retro-fit its tests (and it’s hard to click a not-yet-existing widget to generate a test :-). HTH Niall Ross _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
Just as side notes: - HP replaced WinRunner by another solution based on QuickTest in 2010 - at least for ObjectStudio Classic there is Test-Fan http://www.renneisen-informatik.de/TestFan.html (German only). I don’t know whether it has been updated to support ObjectStudio 8 and VisualWorks. Cheers Helge Von: [hidden email] [mailto:[hidden email]] Im Auftrag von Claus Kick Hello Niall, thank you a lot for this. This helps me a lot. Judging from the absence of other responses, I find it interesting that not many people seem to use tests like this. -- Gesendet: Dienstag, 14. Februar 2012 um 14:04 Uhr Dear Claus, _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
In reply to this post by Claus Kick
Hello Claus, We use GUI tests a lot – we have thousands of them and they are a crucial part of the dev cyclus and numerous issues have been detected by them. Everything happens in the image; we wrote our own testing framework in which we can edit fields, catch dialogs, select menuitems etc. The tests are set up as unit tests, running after every integration in sunit together with the “regular” tests. We have built it after using TestMentor a while (10 years ago). Iirc the big disadvantage of the testmentor system was that maintaining the tests there just took too much time. UI tests now are written with the same toolkit as regular tests, and that works very good for us. Greetings, Peter From: [hidden email] [mailto:[hidden email]]
On Behalf Of Claus Kick Hello Niall, thank you a lot for this. This helps me a lot. Judging from the absence of other responses, I find it interesting that not many people seem to use tests like this. -- Gesendet: Dienstag, 14. Februar 2012 um 14:04 Uhr Dear Claus, _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
We use the same approach as Peter describes.
We mix between “driving” the UI and creating objects using Smalltalk statements. In some cases it is easiest to configure an object in code, and then open an UI to use the object created. Tests varies from testing small parts of the UI (like a custom input field), to testing the entire installation of the product and browsing through all screens. Tests often fail in the code that “drives” the UI, but we also have “assert”-statements. All tests are run for each build using Polycephaly: http://blog.epigent.com/2011/03/use-of-visualworks-polycephaly-at.html Runar Jordahl _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
In reply to this post by Nowak, Helge
Another option that I'll throw out there is TestComplete by AutomatedQA. I'm mainly using them to test web apps right now but there is extensive support for Windows GUIs, built in just about anything you can imagine.
As an old WinRunner hacker myself -- once upon a time I was a certified product instructor and specialist in the Mercury suite -- I find TestComplete to be a great bargain in terms of price and functionality. However I have not (yet) tried to use it with a VW GUI on Windows. If someone can point me to a package with a simple GUI app with some plain-vanilla windows controls for testing, something I can quickly install, I can see what TC does if I try to record/replay actions against it. I haven't played with VW GUI's at all yet, being a noob. - Bob On Wed, Feb 15, 2012 at 4:39 PM, Nowak, Helge <[hidden email]> wrote:
_______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
In reply to this post by Claus Kick
I have used SUnit to test GUI's in VW & VA, but I tend to minimize testing via gui - e.g, "does security work", "can I talk to the model", "are the correct objects enabled based on user or condition". Even in these cases, the tests run like:
1. Instantiate the view, setting relevant conditions via either direct setters or setting up data which drive the desired conditions. 2. Send a method that a (for example) button click would send to the instance via the test framework, rather than actually clicking the button via gui. 3. Observe the return from that method, for example, a collection of objects to be displayed, matches expectations, rather than querying the view to get the list of displayed object, though that can be done too with the right accessors on the view |
|
|
In reply to this post by Claus Kick
You might a look at the GUI oriented unit tests in Widgetry.
On Feb 16, 2012, at 1:00 PM, [hidden email] wrote: > Send vwnc mailing list submissions to > [hidden email] > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > or, via email, send a message with subject or body 'help' to > [hidden email] > > You can reach the person managing the list at > [hidden email] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of vwnc digest..." > > > Today's Topics: > > 1. Re: Testing a VW GUI (Bob Calco) > 2. Re: Testing a VW GUI (Steve Cline) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 16 Feb 2012 11:22:01 +0000 > From: Bob Calco <[hidden email]> > Subject: Re: [vwnc] Testing a VW GUI > To: "Nowak, Helge" <[hidden email]> > Cc: VWNC List <[hidden email]> > Message-ID: > <[hidden email]> > Content-Type: text/plain; charset="windows-1252" > > Another option that I'll throw out there is TestComplete by AutomatedQA. > I'm mainly using them to test web apps right now but there is extensive > support for Windows GUIs, built in just about anything you can imagine. > > As an old WinRunner hacker myself -- once upon a time I was a certified > product instructor and specialist in the Mercury suite -- I find > TestComplete to be a great bargain in terms of price and functionality. > > However I have not (yet) tried to use it with a VW GUI on Windows. If > someone can point me to a package with a simple GUI app with some > plain-vanilla windows controls for testing, something I can quickly > install, I can see what TC does if I try to record/replay actions against > it. I haven't played with VW GUI's at all yet, being a noob. > > - Bob > > > On Wed, Feb 15, 2012 at 4:39 PM, Nowak, Helge <[hidden email]> wrote: > >> Just as side notes:**** >> >> - HP replaced WinRunner by another solution based on QuickTest in 2010**** >> >> - at least for ObjectStudio Classic there is Test-Fan >> http://www.renneisen-informatik.de/TestFan.html (German only). I don?t >> know whether it has been updated to support ObjectStudio 8 and VisualWorks. >> **** >> >> ** ** >> >> Cheers**** >> >> Helge**** >> >> ** ** >> >> *Von:* [hidden email] [mailto:[hidden email]] *Im >> Auftrag von *Claus Kick >> *Gesendet:* Mittwoch, 15. Februar 2012 16:48 >> *An:* Niall Ross; Claus Kick >> *Cc:* VWNC List >> >> *Betreff:* Re: [vwnc] Testing a VW GUI**** >> >> ** ** >> >> Hello Niall,**** >> >> ** ** >> >> thank you a lot for this. This helps me a lot.**** >> >> Judging from the absence of other responses, I find it interesting that >> not many people seem to use tests like this.**** >> >> ** ** >> >> -- >> Claus Kick >> >> "Wenn Sie mich suchen: Ich halte mich in der N?he des Wahnsinns auf. >> Genauer gesagt auf der schmalen Linie zwischen Wahnsinn und Panik. >> Gleich um die Ecke von Todesangst, nicht weit weg von Irrwitz und Idiotie." >> >> "If you are looking for me: I am somewhere near to lunacy. >> More clearly, on the narrow path between lunacy and panic. >> Right around the corner of fear of death, >> not far away from idiocy and insanity."**** >> >> ** ** >> >> *Gesendet:* Dienstag, 14. Februar 2012 um 14:04 Uhr >> *Von:* "Niall Ross" <[hidden email]> >> *An:* "Claus Kick" <[hidden email]> >> *Cc:* "VWNC List" <[hidden email]> >> *Betreff:* Re: [vwnc] Testing a VW GUI **** >> >> Dear Claus, >> >>> What tools could one use to test VW GUIs, both during development and >>> for final testing of the ready-to-deploy application? >>> >> I list the approaches and tools I've encountered. >> >> A) Test 'from the back' by subclassing or delegating top-of-model-layer >> tests to become UI tests: see eXtreme UI Testing, my talk at Smalltalk >> Solutions 2007. (Scroll to the foot of http://www.esug.org/Conferences >> to get a link to my write-up of the conference.) >> >> In the Cincom open repository, examples of these approaches are in the >> RBDynamicRefactoringsTests, in RBCustomerBrowserUITests and in the CS... >> versions of the RBTests . I see all these are only in that repository in >> such old versions that you'll have to use VW7.5 to run them - sorry. (I >> have up-to-date versions and will try to get around to making them >> visible.) >> >> The following text is largely an edit of the first part of my talk, >> where I review alternatives. >> >> B) Smalltalk tools: >> >> - David Buck?s VWUnit: ApplicationModels are located by test-supplied, >> test-specific name. Widgets are located by relative location (to others >> and/or within bounds) and/or name/id. (See his talk at Smalltalk >> Solutions 2003. Also c.f. some similarities of style with widget >> location and manipulation in David Shaeffer?s Seaside Testing Framework; >> see David?s talks at Smalltalk Solutions 2005 and at ESUG 2005.) VWUnit >> is in the open repository. >> >> - Silvermark?s Test Mentor offers click and type test generation for >> UI?s built from standard VW or VA widgets (it can need adaption to >> applicationspecific UI frameworks). It locates widgets by name if they >> are named, or by generated name if possible, or by the path from the >> enclosing subcanvas or window when duplicate names occur. A trial >> version is available to be installed on the iso. >> >> I do not know the current usage state of either of these tools, or if >> any further development of them has been done in recent years. >> >> C) Non-Smalltalk tools that have been used on Smalltalk. >> >> - James Foster wrote Smalltalk-oriented extensions to WinRunner. >> WinRunner offers click and type test generation for a fixed (inaccurate) >> view of a Smalltalk application?s UI. Its widget positions are fragile >> and its widget names are not unique. James? add-ons help you tweak your >> application to fit WinRunner?s expectations better (see his talk at >> Smalltalk Solutions 2004). >> >> My experience of using these tools is very limited (limited commercial >> experience of Silvermark's TestRunner, even more limited >> hobby/open-source use of VWUnit, none of James Foster?s work) and are >> now all from some years ago, as I use my own approach (when I do >> adequate UI testing, which is far from always). Hence the following >> comments should be read with caution. >> >> I find them good for retrofitting UI tests to stable systems, and for >> automating acceptance-style tests, etc. They are also useful as examples >> of how to write widget-driving code. But somehow they lack the XP >> test-driven feel. In ?A posteriori? use, the tests tend to be brittle >> under refactoring. This brittleness effectively discourages ?A priori? >> use: you tend to build a stable UI then retro-fit its tests (and it?s >> hard to click a not-yet-existing widget to generate a test :-). >> >> HTH >> Niall Ross**** >> >> _______________________________________________ >> vwnc mailing list >> [hidden email] >> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc >> >> > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: http://lists.cs.uiuc.edu/mailman/private/vwnc/attachments/20120216/1fd7e3e8/attachment-0001.html > > ------------------------------ > > Message: 2 > Date: Thu, 16 Feb 2012 06:32:03 -0800 (PST) > From: Steve Cline <[hidden email]> > Subject: Re: [vwnc] Testing a VW GUI > To: [hidden email] > Message-ID: <[hidden email]> > Content-Type: text/plain; charset=us-ascii > > I have used SUnit to test GUI's in VW & VA, but I tend to minimize testing > via gui - e.g, "does security work", "can I talk to the model", "are the > correct objects enabled based on user or condition". Even in these cases, > the tests run like: > > 1. Instantiate the view, setting relevant conditions via either direct > setters or setting up data which drive the desired conditions. > > 2. Send a method that a (for example) button click would send to the > instance via the test framework, rather than actually clicking the button > via gui. > > 3. Observe the return from that method, for example, a collection of objects > to be displayed, matches expectations, rather than querying the view to get > the list of displayed object, though that can be done too with the right > accessors on the view > > > -- > View this message in context: http://forum.world.st/Testing-a-VW-GUI-tp4380825p4394135.html > Sent from the VisualWorks mailing list archive at Nabble.com. > > > ------------------------------ > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > > End of vwnc Digest, Vol 49, Issue 31 > ************************************ _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
In reply to this post by Nowak, Helge
Hello all, for track back: got a response from the Test-Fan people. They said that support is especially good for MFC-Applications. Which begs the question: As far as I know, ObjectStudio windows appear as MFC. How is this for VW? What are VW windows, from a Windows point of view? -- Claus Kick "Wenn Sie mich suchen: Ich halte mich in der Nähe des Wahnsinns auf. Genauer gesagt auf der schmalen Linie zwischen Wahnsinn und Panik. Gleich um die Ecke von Todesangst, nicht weit weg von Irrwitz und Idiotie." "If you are looking for me: I am somewhere near to lunacy. More clearly, on the narrow path between lunacy and panic. Right around the corner of fear of death, not far away from idiocy and insanity."
Gesendet: Mittwoch, 15. Februar 2012 um 17:39 Uhr
Von: "Nowak, Helge" <[hidden email]> An: "Claus Kick" <[hidden email]>, "Niall Ross" <[hidden email]> Cc: "VWNC List" <[hidden email]> Betreff: AW: [vwnc] Testing a VW GUI Just as side notes: - HP replaced WinRunner by another solution based on QuickTest in 2010 - at least for ObjectStudio Classic there is Test-Fan http://www.renneisen-informatik.de/TestFan.html (German only). I don’t know whether it has been updated to support ObjectStudio 8 and VisualWorks.
Cheers Helge
Von: [hidden email] [mailto:[hidden email]] Im Auftrag von Claus Kick
Hello Niall,
thank you a lot for this. This helps me a lot. Judging from the absence of other responses, I find it interesting that not many people seem to use tests like this.
--
Gesendet: Dienstag, 14. Februar 2012 um 14:04 Uhr Dear Claus, _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
The Windows are native Windows; the contents have no native widgets in them, however. That's why stock GUI testing tools don't tend to work with VW (unless they simply track timing, key presses, and click locations) - they don't find the things they look for.
On Mar 6, 2012, at 5:42 AM, Claus Kick wrote:
James Robertson _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
| Free forum by Nabble | Edit this page |