[squeak-dev] All tests running green

All tests running green


Ken G. Brown wrote:
> I think that the dev images should ship with all tests running green.

Cool. How?

It is a hard task just to get the basic image to run its tests green.
The dev image introduces a lot of packages to each other often for the first time.

Damien does a mighty service by providing fresh integration every month.
This is probably near the limit of the time service he can give to the community.

Are you willing to recruit someone to post
 the results of the tests runs for a dev image each month?
It seems to me that would be the next step.

The reports would be the feedback that lead to green tests.

*** Ken Wrote:
>I'd like to have confidence that everything added to 3.10.2 also includes
>tests for everything added, and that all tests run unattended and everything
>ends green.

+1

>For starters, on Mac 10.5.4, fresh sq3.10.2-7179dev08.07.1 on Squeak
>3.8.18beta1U.app vm, pull out the Test Runner, select all standard tests,
>start to ru

n 2788 tests,  and with manual intervention required several
>times, it only gets to 394 run, 374 passes, 2 expected failures, 5 failures
>13 error, before it quits processing.
***


Cool, what test does it quit on?

What test expect themselves to fail?
And what are the errors and failures?

If you take out those tests how does the rest run?

It would seem to me, just as Damien contributes by producing the images,
 someone needs to contribute by producing the reports
 on the tests from the images.

If someone one is willing, then others will be able to add bug removal.
And the dev images can converge until Ken gets his wish.

Bugs are like litter they can accumulate one at a time
 or be eliminated one at a time.

What the community needs is the accumulated help
 of those who are in the right position to tackle the problems.

I can find bugs and I can puzzle out some good solutions to interesting ones.

I can't also run tests extensively or continuously to discover
 all the possible intergration problems the fixes might lead to.


Sqeuak is 30 years old some of the older parts do not share
 the same assumptions as the newer parts.
There are a lot of integration bugs.
And the more you integrate the more problems you will find.

 ***And later Ken wrote:
>I'm suggesting that anything pre-loaded by you into the dev images, come with
>tests and all tests be green for the release dev image.
>Other stuff can however be showcased by listing in the dev image and be easy
>to load, but not be pre-loaded if they cause any tests to fail.
***from http://lists.squeakfoundation.org/pipermail/squeak-dev/2008-July/129970.html

It will probably be important to have distributable images with those properties.
 But the fastest way to get there is to make the mistakes first.
Damiens dev images are integration betas.
 They are there first and will need to be tested.

Once the tests are made and reported.
Other integration scripts can be written
 s.t. a stable dev image meeting your critriria could be accomplished.
Either by leaving out packages or failing tests.
Progress is made by walking forward.
Walking forward is made by falling
 and catching yourself before you land on your face.

Damien's images are a necessary step.
Someone stepping forward and reporting test results
 is the next necessary step.


Yours in curiosity and service, --Jerome Peace




     

Hi,

> Are you willing to recruit someone to post
>  the results of the tests runs for a dev image each month?

I've discovered Squeak and SmallTalk last week at the RMLL and I want
to contribute to the project. This task seems to be a not-too-hard one
to get involved, so I can do it if you wish. What basic informations
must be in this report ? Is there a way to export this into a text
file ? Is there a tool like CruiseControl for Squeak ?

Regards,

Laurent Laffont

Hi Laurent,

welcome to Squeak and thank you for your interest.

On Mon, Jul 7, 2008 at 9:03 AM, laurent laffont
<[hidden email]> wrote:
>> Are you willing to recruit someone to post
>>  the results of the tests runs for a dev image each month?
>
> I've discovered Squeak and SmallTalk last week at the RMLL and I want
> to contribute to the project. This task seems to be a not-too-hard one
> to get involved, so I can do it if you wish. What basic informations
> must be in this report ? Is there a way to export this into a text
> file ? Is there a tool like CruiseControl for Squeak ?

I think the information we need looks like
http://speedy.cs.uiuc.edu/~johnson/testResults/3.10alpha.7105-none.html.
Ralph Johnson wrote a tool to generate these reports but I don't know
more than that. You can ask him.

--
Damien Cassou
Peter von der Ahé: «I'm beginning to see why Gilad wished us good
luck». (http://blogs.sun.com/ahe/entry/override_snafu)

> I think the information we need looks like
> http://speedy.cs.uiuc.edu/~johnson/testResults/3.10alpha.7105-none.html.
> Ralph Johnson wrote a tool to generate these reports but I don't know
> more than that. You can ask him.

Done.

I think the way to do it is to periodically download last dev image,
run the tests, output the results to a file and upload it to a
webserver. I think we can do this with a simple bash script and a cron
task.  Is it possible to run all the Squeak tests from the command
line ?

Laurent

On Mon, Jul 7, 2008 at 10:56 AM, laurent laffont
<[hidden email]> wrote:
Yes. First create a text file which contains the Smalltalk
instructions to run the tests you want. You might want to look at
classes TestCase and TestRunner to find what to write in this file.

Then, you have to pass the full path of that file as the last
parameter to Squeak.


--
Damien Cassou
Peter von der Ahé: «I'm beginning to see why Gilad wished us good
luck». (http://blogs.sun.com/ahe/entry/override_snafu)

laurent laffont wrote

I think the way to do it is to periodically download last dev image,
run the tests, output the results to a file and upload it to a
webserver. I think we can do this with a simple bash script and a cron
task.  Is it possible to run all the Squeak tests from the command
line ?

Laurent

I'd also be happy to run the full tests once a month on the new images. I'm on Ubuntu 7.10.

There seems to be a bit of work before it can be a one-click or automated process. As I mentioned here:
http://www.nabble.com/-squeak-dev---ANN--Squeak-dev-Images--June-2008-versions-td17674890.html#a17708211
ChildrenToSiblingsTest throws a syntax error, and SemaphoreTest hangs. There are also a number of tests that ask you to click something, such as 'Should not see this message or this test failed!', 'You like Smalltalk?', 'One', 'Two'...

...Stan

stan shepherd wrote

I'd also be happy to run the full tests once a month on the new images. I'm on Ubuntu 7.10.

There seems to be a bit of work before it can be a one-click or automated process. As I mentioned here:
http://www.nabble.com/-squeak-dev---ANN--Squeak-dev-Images--June-2008-versions-td17674890.html#a17708211
ChildrenToSiblingsTest throws a syntax error, and SemaphoreTest hangs. There are also a number of tests that ask you to click something, such as 'Should not see this message or this test failed!', 'You like Smalltalk?', 'One', 'Two'...

Sorry, SemaphoreTest seems to be fine. I think I upgraded my VM since the original message.  
...Stan

Damien Cassou wrote: There is a version of TestRunner in 'SUnit-improved' (however some
needed code needs to be retrieved from the history) that takes
parameters from the command line using Installer-Launcher (part of LPF)
to invoke them.

TestRunner can be configured to report its results using TestReporter
generating results to files, which when served by an appropriately
configured web server (apache + some icons) give you both a progress
status and red/green/yellow pass indication in your web browser and a
script to load into a fileList and re-run tests via doIts.

TestReporter can be run standalone without TestRunner which runs the
tests without an in image UI.

This will all be a standard part of Sake/Bob, an auto image
build/testing tool built upon Sake.  (Bob version 1 was written in ruby,
so the ideas have been tested, we just await the smalltalk version.
There is also a server ready to be put to work on this)

regards

Keith

As i remember there are tool for exporting debug information (stack,
vars and other debug stuff) to HTML with hyperlinks. It can be very
useful for failures and errors.

On Mon, Jul 7, 2008 at 16:35, Keith Hodges <[hidden email]> wrote:

El 7/7/08 8:44 AM, "Andrey Larionov" <[hidden email]> escribió:

> As i remember there are tool for exporting debug information (stack,
> vars and other debug stuff) to HTML with hyperlinks. It can be very
> useful for failures and errors.

Such tool is a must, is called DebugReport-minami.25.mcz and part of my own
dev tools.
You could find a ready to run .sar with all tools I use here
http://wiki.squeak.org/squeak/6056

For the test, I run  the test not once a month in a single system, but each
time we the 3.10 ReleaseTeam (Ralph and me) made a new .mcz. on Mac, Windows
XP and SimplyMepis 6.5 Linux

Ralph made very clear how to contribute guides, we have many help with
excellent reports in Mantis and fixes.

But some test can't be solved to green status and we deliver as the thing
was.

Now I doing a 3.11 as I think should be.

Maybe two year from now people start complaining about tons of mails I
sending to all list now ...


Edgar

laurent laffont a écrit :
Nice to see u here Laurent ! This is the first time, i made a demo of
Smalltalk and then the next week, the guy is in Squeak-dev ;-)

--
Serge Stinckwich
http://blog.doesnotunderstand.org/

On Mon, Jul 7, 2008 at 4:04 PM, Serge Stinckwich
<[hidden email]> wrote:
I think it is at least the second time :-)


--
Damien Cassou
Peter von der Ahé: «I'm beginning to see why Gilad wished us good
luck». (http://blogs.sun.com/ahe/entry/override_snafu)

> Nice to see u here Laurent ! This is the first time, i made a demo of
> Smalltalk and then the next week, the guy is in Squeak-dev ;-)
Your MorphClignotant class is so attractive, it has enlightened me ;)

Damien Cassou a écrit :
Yes sorry Damien. Stef and me met Damien at RMLL'05 (or 04 ?) and now he
is part of the community ;-) If only we could recruit one guy like you
each year, we would be lucky.

--
Serge Stinckwich
http://blog.doesnotunderstand.org/

laurent laffont a écrit :
>> Nice to see u here Laurent ! This is the first time, i made a demo of
>> Smalltalk and then the next week, the guy is in Squeak-dev ;-)
> Your MorphClignotant class is so attractive, it has enlightened me ;)
>

Just to put some context for the others, Stéphane (Ducasse), Cedrick
(Beler) and I met Laurent at the RMLL'08 (a french open-source event) on
the ESUG booth. We spent a lot of time doing a lot of demoes to people
passing on the booth. One of my favorite demo is a small kata called
"Blinking Morph". This is a nice way to show the usual developping
process of a Smalltalker. There is a french explanation about this kata
on my blog :

http://blog.doesnotunderstand.org/2008/05/kata-de-demonstration-smalltalk/

Maybe i need to translate this demo in english.

If you want to promote Smalltalk and Squeak, just try to attend
open-source conference and events all over the world. Ask ESUG for
funding if you need to be helped.

--
Serge Stinckwich
http://blog.doesnotunderstand.org/

Google translate did good enough for this English-only speaker (living
in Kansas, my 1/2 year of 8th grade French has left me long ago):

http://translate.google.com/translate?u=http%3A%2F%2Fblog.doesnotunderstand.org%2F2008%2F05%2Fkata-de-demonstration-smalltalk%2F&hl=en&ie=UTF8&sl=fr&tl=en

Thanks for the great example.

On Mon, Jul 7, 2008 at 9:20 AM, Serge Stinckwich
<[hidden email]> wrote:

On Jul 7, 2008, at 9:03 AM, laurent laffont wrote:

> Hi,
>
>> Are you willing to recruit someone to post
>> the results of the tests runs for a dev image each month?
>
> I've discovered Squeak and SmallTalk last week at the RMLL and I want
> to contribute to the project.

Salut laurent,
I know the cool french guys you met :)
It was cool.

> This task seems to be a not-too-hard one
> to get involved, so I can do it if you wish. What basic informations
> must be in this report ? Is there a way to export this into a text
> file ? Is there a tool like CruiseControl for Squeak ?

 From the pharo mailing-list:

Perhaps out "Package List Tool" may be of help.
(screenshot: http://www.flickr.com/photos/12018791@N06/2616669616/ )

has autoload (of newer packages), auto-recompilation (for previous MC  
1.5
problems) and auto test/spec. Works well with our "staging" server for a
client...

One-clicks upload of changes to packages listed etc.

stef
>
>
> Regards,
>
> Laurent Laffont
>
>

Having a test server and integration server.

What would be nice is to know
        the image
        the vm
        the packages loaded
        the tests
        and may be something per package.

Stef

On Jul 7, 2008, at 9:32 AM, Damien Cassou wrote:

TestCase allSubclasses do: [:each | each run] or something like that.

Stef

Indeed :)
It was a nice RMLL too that year :)

>>
>> Nice to see u here Laurent ! This is the first time, i made a demo of
>> Smalltalk and then the next week, the guy is in Squeak-dev ;-)
>
> I think it is at least the second time :-)