How are you doing your jenkins build scripts?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

How are you doing your jenkins build scripts?

NorbertHartl
I'm close to having ported all of our projects to pharo 2.0. Now I'm asking myself what would be a proper setup to run my jenkins build scripts. With RPackage there are more packages in the system then before and in the jenkins scripts you need to invoke

HDTestReport forPackages: #( …)

In 1.4 I just added them manually. Using 2.0 makes this too cumbersome to deal with. So I need to figure out the amount of tests programmatically. The first I came up with (and that would solve my common use case) is something like


myPrefix := 'Emcee-'
myPackages := RPackage organizer packages select: [:each| each name beginsWith: myPrefix ].
packagesWithTests := myPackages select: [ :package| package classes anySatisfy: [ :cls| cls includesBehavior: TestCase  ] ].
packagesNamedTest := myPackages select: [ :each| each name includesSubstring: '-Tests-' ].
testPackages := packagesWithTests union: packagesNamedTest.
packages := myPackages difference: packagesNamedTest.

HDTestReport runPackages: (testPackages collect: #name).
HDLintReport runPackages: (packages collect: #name)

So I like to ask how you guys are doing it. Thanks in advance for your answers.

Norbert
Reply | Threaded
Open this post in threaded view
|

Re: How are you doing your jenkins build scripts?

Camillo Bruni-3
You use the built-in command line tools and no longer .st files.

Pointers:
=========
curl get.pharo.org | bash
./pharo Pharo.image --help
./pharo Pharo.image --list
./pharo Pharo.image test --help
# everything should be pretty self explaining


A typical script looks like this on jenkins:
==========================================================================
wget --quiet -O - get.pharo.org/20+vm | bash

./pharo Pharo.image save $JOB_NAME --delete-old
./pharo $JOB_NAME.image --version > version.txt

REPO=http://www.squeaksource.com/MetacelloRepository
./pharo $JOB_NAME.image config $REPO ConfigurationOf$JOB_NAME --install=$VERSION --groups='All OS',Tests
./pharo $JOB_NAME.image test --junit-xml-output "$JOB_NAME.*"

zip -r $JOB_NAME.zip $JOB_NAME.image $JOB_NAME.changes
==========================================================================


On 2013-07-25, at 16:36, Norbert Hartl <[hidden email]> wrote:

> I'm close to having ported all of our projects to pharo 2.0. Now I'm asking myself what would be a proper setup to run my jenkins build scripts. With RPackage there are more packages in the system then before and in the jenkins scripts you need to invoke
>
> HDTestReport forPackages: #( …)
>
> In 1.4 I just added them manually. Using 2.0 makes this too cumbersome to deal with. So I need to figure out the amount of tests programmatically. The first I came up with (and that would solve my common use case) is something like
>
>
> myPrefix := 'Emcee-'
> myPackages := RPackage organizer packages select: [:each| each name beginsWith: myPrefix ].
> packagesWithTests := myPackages select: [ :package| package classes anySatisfy: [ :cls| cls includesBehavior: TestCase  ] ].
> packagesNamedTest := myPackages select: [ :each| each name includesSubstring: '-Tests-' ].
> testPackages := packagesWithTests union: packagesNamedTest.
> packages := myPackages difference: packagesNamedTest.
>
> HDTestReport runPackages: (testPackages collect: #name).
> HDLintReport runPackages: (packages collect: #name)
>
> So I like to ask how you guys are doing it. Thanks in advance for your answers.
>
> Norbert


Reply | Threaded
Open this post in threaded view
|

Re: How are you doing your jenkins build scripts?

NorbertHartl
Cami,

thanks for the snippet. The commandline handler for tests is a super replacement for one part of my snippet. But how do you do the lint tests?

Norbert

Am 25.07.2013 um 16:54 schrieb Camillo Bruni <[hidden email]>:

> You use the built-in command line tools and no longer .st files.
>
> Pointers:
> =========
> curl get.pharo.org | bash
> ./pharo Pharo.image --help
> ./pharo Pharo.image --list
> ./pharo Pharo.image test --help
> # everything should be pretty self explaining
>
>
> A typical script looks like this on jenkins:
> ==========================================================================
> wget --quiet -O - get.pharo.org/20+vm | bash
>
> ./pharo Pharo.image save $JOB_NAME --delete-old
> ./pharo $JOB_NAME.image --version > version.txt
>
> REPO=http://www.squeaksource.com/MetacelloRepository
> ./pharo $JOB_NAME.image config $REPO ConfigurationOf$JOB_NAME --install=$VERSION --groups='All OS',Tests
> ./pharo $JOB_NAME.image test --junit-xml-output "$JOB_NAME.*"
>
> zip -r $JOB_NAME.zip $JOB_NAME.image $JOB_NAME.changes
> ==========================================================================
>
>
> On 2013-07-25, at 16:36, Norbert Hartl <[hidden email]> wrote:
>
>> I'm close to having ported all of our projects to pharo 2.0. Now I'm asking myself what would be a proper setup to run my jenkins build scripts. With RPackage there are more packages in the system then before and in the jenkins scripts you need to invoke
>>
>> HDTestReport forPackages: #( …)
>>
>> In 1.4 I just added them manually. Using 2.0 makes this too cumbersome to deal with. So I need to figure out the amount of tests programmatically. The first I came up with (and that would solve my common use case) is something like
>>
>>
>> myPrefix := 'Emcee-'
>> myPackages := RPackage organizer packages select: [:each| each name beginsWith: myPrefix ].
>> packagesWithTests := myPackages select: [ :package| package classes anySatisfy: [ :cls| cls includesBehavior: TestCase  ] ].
>> packagesNamedTest := myPackages select: [ :each| each name includesSubstring: '-Tests-' ].
>> testPackages := packagesWithTests union: packagesNamedTest.
>> packages := myPackages difference: packagesNamedTest.
>>
>> HDTestReport runPackages: (testPackages collect: #name).
>> HDLintReport runPackages: (packages collect: #name)
>>
>> So I like to ask how you guys are doing it. Thanks in advance for your answers.
>>
>> Norbert
>
>


Reply | Threaded
Open this post in threaded view
|

Re: How are you doing your jenkins build scripts?

Camillo Bruni-3

On 2013-07-25, at 17:10, Norbert Hartl <[hidden email]> wrote:

> Cami,
>
> thanks for the snippet. The commandline handler for tests is a super replacement for one part of my snippet. But how do you do the lint tests?

ah sorry, didn't see that last line. I didn't add support for that, so I guess you're stuck with the script,

=======================================================================
./pharo Pharo.image eval "
...
your code goes here
...
"
=======================================================================

We have a hacked version that works from the commandline for the lint rules in 3.0 (might still work for 2.0):

=======================================================================
REPO="http://smalltalkhub.com/mc/ClementBera/code-critic-jenkins/main"
./pharo Pharo.image config $REPO ConfigurationOfCommandLineMaintenanceTool --install=stable

# Run lint rules
./pharo Pharo.image analyse2 >> maintenancePharo2.xml
=======================================================================

but I had the impression that there are too many false positives with the selected rules. To collect the packages you can have a look at the CommandLineHandler subclasses (I don't remember the name for the test runner thingy anymore in 2.0).


> Am 25.07.2013 um 16:54 schrieb Camillo Bruni <[hidden email]>:
>
>> You use the built-in command line tools and no longer .st files.
>>
>> Pointers:
>> =========
>> curl get.pharo.org | bash
>> ./pharo Pharo.image --help
>> ./pharo Pharo.image --list
>> ./pharo Pharo.image test --help
>> # everything should be pretty self explaining
>>
>>
>> A typical script looks like this on jenkins:
>> ==========================================================================
>> wget --quiet -O - get.pharo.org/20+vm | bash
>>
>> ./pharo Pharo.image save $JOB_NAME --delete-old
>> ./pharo $JOB_NAME.image --version > version.txt
>>
>> REPO=http://www.squeaksource.com/MetacelloRepository
>> ./pharo $JOB_NAME.image config $REPO ConfigurationOf$JOB_NAME --install=$VERSION --groups='All OS',Tests
>> ./pharo $JOB_NAME.image test --junit-xml-output "$JOB_NAME.*"
>>
>> zip -r $JOB_NAME.zip $JOB_NAME.image $JOB_NAME.changes
>> ==========================================================================
>>
>>
>> On 2013-07-25, at 16:36, Norbert Hartl <[hidden email]> wrote:
>>
>>> I'm close to having ported all of our projects to pharo 2.0. Now I'm asking myself what would be a proper setup to run my jenkins build scripts. With RPackage there are more packages in the system then before and in the jenkins scripts you need to invoke
>>>
>>> HDTestReport forPackages: #( …)
>>>
>>> In 1.4 I just added them manually. Using 2.0 makes this too cumbersome to deal with. So I need to figure out the amount of tests programmatically. The first I came up with (and that would solve my common use case) is something like
>>>
>>>
>>> myPrefix := 'Emcee-'
>>> myPackages := RPackage organizer packages select: [:each| each name beginsWith: myPrefix ].
>>> packagesWithTests := myPackages select: [ :package| package classes anySatisfy: [ :cls| cls includesBehavior: TestCase  ] ].
>>> packagesNamedTest := myPackages select: [ :each| each name includesSubstring: '-Tests-' ].
>>> testPackages := packagesWithTests union: packagesNamedTest.
>>> packages := myPackages difference: packagesNamedTest.
>>>
>>> HDTestReport runPackages: (testPackages collect: #name).
>>> HDLintReport runPackages: (packages collect: #name)
>>>
>>> So I like to ask how you guys are doing it. Thanks in advance for your answers.
>>>
>>> Norbert
>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: How are you doing your jenkins build scripts?

Clément Béra
2013/7/25 Camillo Bruni <[hidden email]>

On 2013-07-25, at 17:10, Norbert Hartl <[hidden email]> wrote:

> Cami,
>
> thanks for the snippet. The commandline handler for tests is a super replacement for one part of my snippet. But how do you do the lint tests?

ah sorry, didn't see that last line. I didn't add support for that, so I guess you're stuck with the script,

=======================================================================
./pharo Pharo.image eval "
...
your code goes here
...
"
=======================================================================

We have a hacked version that works from the commandline for the lint rules in 3.0 (might still work for 2.0):

=======================================================================
REPO="http://smalltalkhub.com/mc/ClementBera/code-critic-jenkins/main"
./pharo Pharo.image config $REPO ConfigurationOfCommandLineMaintenanceTool --install=stable

# Run lint rules
./pharo Pharo.image analyse2 >> maintenancePharo2.xml
=======================================================================

This tool is a prototype huhum.

The script is:
# install the code critics command line tool
./pharo Pharo.image config $REPO ConfigurationOfCommandLineMaintenanceTool --install=stable

# Run the analysis
./pharo Pharo.image analyse >> maintenancePharo.xml
./pharo Pharo.image analyse --packages='Kernel*' >> maintenanceKernel.xml
./pharo Pharo.image analyse2 >> maintenancePharo2.xml

It uses both the CodeCritics and HDLintReport

Of course now Pharo analysis on this build are not run with Manifest so there is a massive number of false positive.
 

but I had the impression that there are too many false positives with the selected rules. To collect the packages you can have a look at the CommandLineHandler subclasses (I don't remember the name for the test runner thingy anymore in 2.0).


> Am 25.07.2013 um 16:54 schrieb Camillo Bruni <[hidden email]>:
>
>> You use the built-in command line tools and no longer .st files.
>>
>> Pointers:
>> =========
>> curl get.pharo.org | bash
>> ./pharo Pharo.image --help
>> ./pharo Pharo.image --list
>> ./pharo Pharo.image test --help
>> # everything should be pretty self explaining
>>
>>
>> A typical script looks like this on jenkins:
>> ==========================================================================
>> wget --quiet -O - get.pharo.org/20+vm | bash
>>
>> ./pharo Pharo.image save $JOB_NAME --delete-old
>> ./pharo $JOB_NAME.image --version > version.txt
>>
>> REPO=http://www.squeaksource.com/MetacelloRepository
>> ./pharo $JOB_NAME.image config $REPO ConfigurationOf$JOB_NAME --install=$VERSION --groups='All OS',Tests
>> ./pharo $JOB_NAME.image test --junit-xml-output "$JOB_NAME.*"
>>
>> zip -r $JOB_NAME.zip $JOB_NAME.image $JOB_NAME.changes
>> ==========================================================================
>>
>>
>> On 2013-07-25, at 16:36, Norbert Hartl <[hidden email]> wrote:
>>
>>> I'm close to having ported all of our projects to pharo 2.0. Now I'm asking myself what would be a proper setup to run my jenkins build scripts. With RPackage there are more packages in the system then before and in the jenkins scripts you need to invoke
>>>
>>> HDTestReport forPackages: #( …)
>>>
>>> In 1.4 I just added them manually. Using 2.0 makes this too cumbersome to deal with. So I need to figure out the amount of tests programmatically. The first I came up with (and that would solve my common use case) is something like
>>>
>>>
>>> myPrefix := 'Emcee-'
>>> myPackages := RPackage organizer packages select: [:each| each name beginsWith: myPrefix ].
>>> packagesWithTests := myPackages select: [ :package| package classes anySatisfy: [ :cls| cls includesBehavior: TestCase  ] ].
>>> packagesNamedTest := myPackages select: [ :each| each name includesSubstring: '-Tests-' ].
>>> testPackages := packagesWithTests union: packagesNamedTest.
>>> packages := myPackages difference: packagesNamedTest.
>>>
>>> HDTestReport runPackages: (testPackages collect: #name).
>>> HDLintReport runPackages: (packages collect: #name)
>>>
>>> So I like to ask how you guys are doing it. Thanks in advance for your answers.
>>>
>>> Norbert
>>
>>
>
>



Reply | Threaded
Open this post in threaded view
|

Re: How are you doing your jenkins build scripts?

Tudor Girba-2
In reply to this post by NorbertHartl
Hi Norbert,

In Moose, the configuration also contains a little package that comes with development tools. Among these development tools we have a couple of Moose-specific command lines.

You can see a description here:
http://www.tudorgirba.com/blog/moose-4-8-on-jenkins

Cheers,
Doru


On Jul 25, 2013, at 5:10 PM, Norbert Hartl <[hidden email]> wrote:

> Cami,
>
> thanks for the snippet. The commandline handler for tests is a super replacement for one part of my snippet. But how do you do the lint tests?
>
> Norbert
>
> Am 25.07.2013 um 16:54 schrieb Camillo Bruni <[hidden email]>:
>
>> You use the built-in command line tools and no longer .st files.
>>
>> Pointers:
>> =========
>> curl get.pharo.org | bash
>> ./pharo Pharo.image --help
>> ./pharo Pharo.image --list
>> ./pharo Pharo.image test --help
>> # everything should be pretty self explaining
>>
>>
>> A typical script looks like this on jenkins:
>> ==========================================================================
>> wget --quiet -O - get.pharo.org/20+vm | bash
>>
>> ./pharo Pharo.image save $JOB_NAME --delete-old
>> ./pharo $JOB_NAME.image --version > version.txt
>>
>> REPO=http://www.squeaksource.com/MetacelloRepository
>> ./pharo $JOB_NAME.image config $REPO ConfigurationOf$JOB_NAME --install=$VERSION --groups='All OS',Tests
>> ./pharo $JOB_NAME.image test --junit-xml-output "$JOB_NAME.*"
>>
>> zip -r $JOB_NAME.zip $JOB_NAME.image $JOB_NAME.changes
>> ==========================================================================
>>
>>
>> On 2013-07-25, at 16:36, Norbert Hartl <[hidden email]> wrote:
>>
>>> I'm close to having ported all of our projects to pharo 2.0. Now I'm asking myself what would be a proper setup to run my jenkins build scripts. With RPackage there are more packages in the system then before and in the jenkins scripts you need to invoke
>>>
>>> HDTestReport forPackages: #( …)
>>>
>>> In 1.4 I just added them manually. Using 2.0 makes this too cumbersome to deal with. So I need to figure out the amount of tests programmatically. The first I came up with (and that would solve my common use case) is something like
>>>
>>>
>>> myPrefix := 'Emcee-'
>>> myPackages := RPackage organizer packages select: [:each| each name beginsWith: myPrefix ].
>>> packagesWithTests := myPackages select: [ :package| package classes anySatisfy: [ :cls| cls includesBehavior: TestCase  ] ].
>>> packagesNamedTest := myPackages select: [ :each| each name includesSubstring: '-Tests-' ].
>>> testPackages := packagesWithTests union: packagesNamedTest.
>>> packages := myPackages difference: packagesNamedTest.
>>>
>>> HDTestReport runPackages: (testPackages collect: #name).
>>> HDLintReport runPackages: (packages collect: #name)
>>>
>>> So I like to ask how you guys are doing it. Thanks in advance for your answers.
>>>
>>> Norbert
>>
>>
>
>

--
www.tudorgirba.com

"We can create beautiful models in a vacuum.
But, to get them effective we have to deal with the inconvenience of reality."