Smalltalk › Squeak › Squeak - Dev

testrun results

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

12 messages Options

laza

testrun results

Ken Causey-3

Re: testrun results

Thanks! These are really developing into a useful tool.

Ken

P.S. Is this Alexander Lazarevic?

On Wed, 2010-04-07 at 03:30 +0200, [hidden email] wrote:

signature.asc (197 bytes) Download Attachment

laza

Re: testrun results

Hi Ken!

2010/4/7 Ken Causey <[hidden email]>

Thanks! These are really developing into a useful tool.

Thanks! Once a day this just updates a trunk image, runs all tests and compares the results with the previous ones. Only if there are any differences it will send out a report.

Even if I'm using an older vm on linux, so that some tests results get reported as failure/error, where they would pass on other platforms with newer VMs, I think it is still useful, because it should only draw attention to tests that have changed results between two runs.

I'll use a newer vm when there will be one available for linux.

P.S. Is this Alexander Lazarevic?

It still is! ;)

Alex

Yanni Chiu

Hudson build server (was: testrun results)

Alexander Lazarević wrote:
>
> Thanks! Once a day this just updates a trunk image, runs all tests and
> compares the results with the previous ones. Only if there are any
> differences it will send out a report.

Check out:
http://hudson.jooshr.org/job/Squeak4.1/

It's currently set up to run:

[Utilities updateFromServer] valueSuppressingAllMessages.

once a day, starting with a Squeak4.1-9850-beta image. Then it runs the
tests using:

HDTestReport runClasses: TestCase withAllSubclasses named:
'AllTestCases'.

HDTestReport is available from:
http://www.squeaksource.com/HudsonBuild.html

The shell scripts and .st scripts are available from:
http://github.com/renggli/builder

There's also a job to load and test Seaside3.0 into the updated Squeak
image, using a Metacello configuration. And, there are some Pharo &
Seaside build jobs too.

I keep up with image releases, and try to make sure the build keeps
running, but not much else. There are some test running
incompatibilities between Squeak and Pharo that need to be fixed, but
the current workaround is to use some older package versions and
customized scripts for Squeak vs. Pharo.

--
Yanni

laza

Re: Hudson build server (was: testrun results)

Hello Yanni!

2010/4/8 Yanni Chiu <[hidden email]>

Check out:
http://hudson.jooshr.org/job/Squeak4.1/

Thanks for the pointer. Looks very sophisticated, but I'm not at all familiar with Hudson.
I use perl scripts to create testbeds, copy image/change files and start/stop the testruns. Some Smalltalk scripts take care of getting the results. These are directly piped into a postgresDB. Then I use queries to create a report (or not).

A nice side effect is, that I keep the updated images and can go easily back in time. (Ok, I need to cleanup some time).

Alex

Bert Freudenberg

Re: testrun results

In reply to this post by laza

On 08.04.2010, at 07:20, Alexander Lazarević wrote:

> Hi Ken!
>
> 2010/4/7 Ken Causey <[hidden email]>
>
>> Thanks! These are really developing into a useful tool.
>
> Thanks! Once a day this just updates a trunk image, runs all tests and compares the results with the previous ones. Only if there are any differences it will send out a report.
>
> Even if I'm using an older vm on linux, so that some tests results get reported as failure/error, where they would pass on other platforms with newer VMs, I think it is still useful, because it should only draw attention to tests that have changed results between two runs.
>
> I'll use a newer vm when there will be one available for linux.

There is version 4.0.2 in svn, it's just not released yet, you have to compile it yourself ...

- Bert -

Bert Freudenberg

Re: Hudson build server (was: testrun results)

In reply to this post by Yanni Chiu

On 08.04.2010, at 09:12, Yanni Chiu wrote:

>
> Alexander Lazarević wrote:
>> Thanks! Once a day this just updates a trunk image, runs all tests and compares the results with the previous ones. Only if there are any differences it will send out a report.
>
> Check out:
> http://hudson.jooshr.org/job/Squeak4.1/
>
> It's currently set up to run:
>
> [Utilities updateFromServer] valueSuppressingAllMessages.
>
> once a day, starting with a Squeak4.1-9850-beta image. Then it runs the tests using:
>
> HDTestReport runClasses: TestCase withAllSubclasses named: 'AllTestCases'.
>
> HDTestReport is available from:
> http://www.squeaksource.com/HudsonBuild.html
>
> The shell scripts and .st scripts are available from:
> http://github.com/renggli/builder
>
> There's also a job to load and test Seaside3.0 into the updated Squeak image, using a Metacello configuration. And, there are some Pharo & Seaside build jobs too.
>
> I keep up with image releases, and try to make sure the build keeps running, but not much else. There are some test running incompatibilities between Squeak and Pharo that need to be fixed, but the current workaround is to use some older package versions and customized scripts for Squeak vs. Pharo.
>
> --
> Yanni

Very nice!

Which VM version are you using for this? At least the BitBltTest failures need a new VM.

Or can we change the test to be an expected failure on an older VMs only?

- Bert -

David T. Lewis

Re: Hudson build server (was: testrun results)

On Thu, Apr 08, 2010 at 11:53:56AM +0200, Bert Freudenberg wrote:

> On 08.04.2010, at 09:12, Yanni Chiu wrote:
> >
> > Alexander Lazarevi?? wrote:
> >> Thanks! Once a day this just updates a trunk image, runs all tests and compares the results with the previous ones. Only if there are any differences it will send out a report.
> >
> > Check out:
> > http://hudson.jooshr.org/job/Squeak4.1/
> >
> > It's currently set up to run:
> >
> > [Utilities updateFromServer] valueSuppressingAllMessages.
> >
> > once a day, starting with a Squeak4.1-9850-beta image. Then it runs the tests using:
> >
> > HDTestReport runClasses: TestCase withAllSubclasses named: 'AllTestCases'.
> >
> > HDTestReport is available from:
> > http://www.squeaksource.com/HudsonBuild.html
> >
> > The shell scripts and .st scripts are available from:
> > http://github.com/renggli/builder
> >
> > There's also a job to load and test Seaside3.0 into the updated Squeak image, using a Metacello configuration. And, there are some Pharo & Seaside build jobs too.
> >
> > I keep up with image releases, and try to make sure the build keeps running, but not much else. There are some test running incompatibilities between Squeak and Pharo that need to be fixed, but the current workaround is to use some older package versions and customized scripts for Squeak vs. Pharo.
> >
> > --
> > Yanni
>
> Very nice!
>
> Which VM version are you using for this? At least the BitBltTest failures need a new VM.
>
> Or can we change the test to be an expected failure on an older VMs only?

The updated unix VM will be published within a few days, so I don't think
the tests should change (after all they are real failures).

Hmm, I think we need a better way to identify VM version at runtime ...

Dave

Levente Uzonyi-2

Re: Hudson build server (was: testrun results)

On Thu, 8 Apr 2010, David T. Lewis wrote:

> On Thu, Apr 08, 2010 at 11:53:56AM +0200, Bert Freudenberg wrote:
>> On 08.04.2010, at 09:12, Yanni Chiu wrote:
>>>
>>> Alexander Lazarevi?? wrote:
>>>> Thanks! Once a day this just updates a trunk image, runs all tests and compares the results with the previous ones. Only if there are any differences it will send out a report.
>>>
>>> Check out:
>>> http://hudson.jooshr.org/job/Squeak4.1/
>>>
>>> It's currently set up to run:
>>>
>>> [Utilities updateFromServer] valueSuppressingAllMessages.
>>>
>>> once a day, starting with a Squeak4.1-9850-beta image. Then it runs the tests using:
>>>
>>> HDTestReport runClasses: TestCase withAllSubclasses named: 'AllTestCases'.
>>>
>>> HDTestReport is available from:
>>> http://www.squeaksource.com/HudsonBuild.html
>>>
>>> The shell scripts and .st scripts are available from:
>>> http://github.com/renggli/builder
>>>
>>> There's also a job to load and test Seaside3.0 into the updated Squeak image, using a Metacello configuration. And, there are some Pharo & Seaside build jobs too.
>>>
>>> I keep up with image releases, and try to make sure the build keeps running, but not much else. There are some test running incompatibilities between Squeak and Pharo that need to be fixed, but the current workaround is to use some older package versions and customized scripts for Squeak vs. Pharo.
>>>
>>> --
>>> Yanni
>>
>> Very nice!
>>
>> Which VM version are you using for this? At least the BitBltTest failures need a new VM.
>>
>> Or can we change the test to be an expected failure on an older VMs only?
>
> The updated unix VM will be published within a few days, so I don't think
> the tests should change (after all they are real failures).
>
> Hmm, I think we need a better way to identify VM version at runtime ...

+1

Levente

>
> Dave
>
>
>

laza

Re: Hudson build server (was: testrun results)

In reply to this post by Bert Freudenberg

2010/4/8 Bert Freudenberg <[hidden email]>

Very nice!

Thanks!

Which VM version are you using for this? At least the BitBltTest failures need a new VM.

I'm using a 3.10-6 #1

Or can we change the test to be an expected failure on an older VMs only?

I'll just would wait until a 4.x vm is available for linux.

Alex

Yanni Chiu

Re: Hudson build server

In reply to this post by David T. Lewis

David T. Lewis wrote:

> On Thu, Apr 08, 2010 at 11:53:56AM +0200, Bert Freudenberg wrote:
>>
>> Which VM version are you using for this? At least the BitBltTest failures need a new VM.
>>
>> Or can we change the test to be an expected failure on an older VMs only?
>
> The updated unix VM will be published within a few days, so I don't think
> the tests should change (after all they are real failures).
>
> Hmm, I think we need a better way to identify VM version at runtime ...

The vm's filename is:

.../Squeak-3.11.3.2135-linux_i386/bin/squeak

But one of the debug log files has:

VM: unix - i686 - linux - Squeak3.10.2 of '5 June 2008' [latest update:
#7179]

Yanni Chiu

Re: Hudson build server

In reply to this post by laza

Alexander Lazarević wrote:
> Thanks for the pointer. Looks very sophisticated, but I'm not at all
> familiar with Hudson.

I don't know much about it's internals either. I just use it, and don't
have to maintain it. It happens to be written in Java, and can integrate
with other programming languages. So it's a counter argument for
Smalltalk being a island on its own.

> I use perl scripts to create testbeds, copy image/change files and
> start/stop the testruns. Some Smalltalk scripts take care of getting the
> results. These are directly piped into a postgresDB. Then I use queries
> to create a report (or not).
>
> A nice side effect is, that I keep the updated images and can go easily
> back in time. (Ok, I need to cleanup some time).

I had initially set the build server to keep the last N=10 builds. It
actually keeps N builds, plus the last successful one, if none of the
last N were successful. I reduced it to 5 builds to reduce disk space
usage; also, I found old builds had little use.