Continuous Integration Server for Pharo?
Locked
Continuous Integration Server for Pharo?
 
|
Dear all, I come from a Java world and there we use for instance : http://hudson-ci.org/ to do a continuous integration build. Which means all latest code gets updated, all test ran & if build successful, the code is tagged (so you can take the code again from a successful build). It also provides a nice GUI, and lets you inspect previous builds, the results from that build, ... .
In Pharo, I did not directly find an alternative, and therefor, also as an exercise, I tried to build one myself. The primitive, but working result so far is: http://www.squeaksource.com/BuildBot.
This project depends on BBProjectDescription, which defines class BBBuildProjectDescription with defines:
What buildbot then does is using a BBBuildProjectDescription , checkout all latest code, run the tests for all these packages & , if successfull, take the buildsClassToPutVersionsOn and add a new method on it, describing the exact package numbers :
for instance: BuildBotMockProjectBuilds class >> packagesForBuild20091127102654 ^#(('TODO' -> 'TODO-BKBAG.5.mcz') ('BuildBtMockProject' -> 'BuildBtMockProject-BartGauquie.7.mcz') ). Using this information I can easily install this exact version. Furthermore creating a release is then just creating for instance a class BuildBotMockProjectReleases with: BuildBotMockProjectReleasesclass >> release_1_0 ^BuildBotMockProjectBuilds packagesForBuild20091127102654 I am in fact 'tagging' a group packages that belong together and work together. So I can rollout this exact versions of the packages to some image automatically.
However, Today I noticed that there already existed a project: http://www.squeaksource.com/TestServerSimple which more or less does the same thing. The only thing it does not do, or I did not find it, is to version the build, so that you can restore a project again ; a project which is consisting of a number of packages which are compatible and for which all tests ran. Is there a way to do that in TestServerSimple?
Furthermore, are there any other projects that do a similar thing I'm not aware of? Is there any default way in Pharo to build a project which consists of multiple packages that each have different versions?
I've seen MetaCello & Gofer as some other way 2 load groups of packages, but no integration into some kind of ci tool. This kind of automated ci server seems 2 me an essential tool in Pharo.
Thanks for any pointers / ideas ! Kind Regards, Bart -- imagination is more important than knowledge - Albert EinsteinLogic will get you from A to B. Imagination will take you everywhere - Albert Einstein Learn from yesterday, live for today, hope for tomorrow. The important thing is not to stop questioning. - Albert Einstein The true sign of intelligence is not knowledge but imagination. - Albert Einstein Gravitation is not responsible for people falling in love. - Albert Einstein _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: Continuous Integration Server for Pharo?
|
|
> Today I noticed that there already existed a project:
> http://www.squeaksource.com/TestServerSimple which more or less does the > same thing. For Seaside we are in the process of applying Mason, a CI server Colin implemented. It is available from <http://source.wiresong.ca/mc>. > This kind of automated ci server seems 2 me an essential tool in Pharo. Definitely, that's absolutely crucial. Lukas -- Lukas Renggli http://www.lukas-renggli.ch _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: Continuous Integration Server for Pharo?
|
|
In reply to this post by Bart Gauquie
Hi bart
we use some scripts (developed at netstyle) and ESUG will support hernan student to develop one. I know that Jorge started also and should work with them. So we should join forces. The idea is that the server should be able to load metacello configuration. Jorge/Hernan/Simon - what is the status - how could we join forces? Stef On Nov 28, 2009, at 1:23 PM, Bart Gauquie wrote: > Dear all, > > I come from a Java world and there we use for instance : http://hudson-ci.org/ to do a continuous integration build. Which means all latest code gets updated, all test ran & if build successful, the code is tagged (so you can take the code again from a successful build). It also provides a nice GUI, and lets you inspect previous builds, the results from that build, ... . > > In Pharo, I did not directly find an alternative, and therefor, also as an exercise, I tried to build one myself. > The primitive, but working result so far is: http://www.squeaksource.com/BuildBot. > > This project depends on BBProjectDescription, which defines class BBBuildProjectDescription with defines: > • BBBuildProjectDescription>>allProjectPackageNames > • BBBuildProjectDescription>>installLatestWithDependencies > • BBBuildProjectDescription>>buildsClassToPutVersionsOn > What buildbot then does is using a BBBuildProjectDescription , checkout all latest code, run the tests for all these packages & , if successfull, take the buildsClassToPutVersionsOn and add a new method on it, describing the exact package numbers : > for instance: > BuildBotMockProjectBuilds class >> packagesForBuild20091127102654 > ^#(('TODO' -> 'TODO-BKBAG.5.mcz') ('BuildBtMockProject' -> 'BuildBtMockProject-BartGauquie.7.mcz') ). > > Using this information I can easily install this exact version. Furthermore creating a release is then just creating for instance a class BuildBotMockProjectReleases with: > BuildBotMockProjectReleasesclass >> release_1_0 > ^BuildBotMockProjectBuilds packagesForBuild20091127102654 > > I am in fact 'tagging' a group packages that belong together and work together. So I can rollout this exact versions of the packages to some image automatically. > > > However, > Today I noticed that there already existed a project: http://www.squeaksource.com/TestServerSimple which more or less does the same thing. The only thing it does not do, or I did not find it, is to version the build, so that you can restore a project again ; a project which is consisting of a number of packages which are compatible and for which all tests ran. Is there a way to do that in TestServerSimple? > > Furthermore, are there any other projects that do a similar thing I'm not aware of? Is there any default way in Pharo to build a project which consists of multiple packages that each have different versions? > > I've seen MetaCello & Gofer as some other way 2 load groups of packages, but no integration into some kind of ci tool. > > This kind of automated ci server seems 2 me an essential tool in Pharo. > > Thanks for any pointers / ideas ! > > Kind Regards, > > Bart > > -- > imagination is more important than knowledge - Albert Einstein > Logic will get you from A to B. Imagination will take you everywhere - Albert Einstein > Learn from yesterday, live for today, hope for tomorrow. The important thing is not to stop questioning. - Albert Einstein > The true sign of intelligence is not knowledge but imagination. - Albert Einstein > Gravitation is not responsible for people falling in love. - Albert Einstein > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: Continuous Integration Server for Pharo?
|
|
Hi,
I am currently working with Diego Kogan on the Continuous Integration Server for pharo. We will be working heavily the comming two weeks in order to finish the implementation of the basic requirements. Our case studies will be Moose and Seaside. We expect ot have a running version for mid december. Cheers, Jorge On Sun, Nov 29, 2009 at 7:53 AM, Stéphane Ducasse <[hidden email]> wrote: > Hi bart > > we use some scripts (developed at netstyle) > and ESUG will support hernan student to develop one. I know that Jorge started also and should work with them. > So we should join forces. > The idea is that the server should be able to load metacello configuration. > Jorge/Hernan/Simon > - what is the status > - how could we join forces? > > Stef > > > On Nov 28, 2009, at 1:23 PM, Bart Gauquie wrote: > >> Dear all, >> >> I come from a Java world and there we use for instance : http://hudson-ci.org/ to do a continuous integration build. Which means all latest code gets updated, all test ran & if build successful, the code is tagged (so you can take the code again from a successful build). It also provides a nice GUI, and lets you inspect previous builds, the results from that build, ... . >> >> In Pharo, I did not directly find an alternative, and therefor, also as an exercise, I tried to build one myself. >> The primitive, but working result so far is: http://www.squeaksource.com/BuildBot. >> >> This project depends on BBProjectDescription, which defines class BBBuildProjectDescription with defines: >> • BBBuildProjectDescription>>allProjectPackageNames >> • BBBuildProjectDescription>>installLatestWithDependencies >> • BBBuildProjectDescription>>buildsClassToPutVersionsOn >> What buildbot then does is using a BBBuildProjectDescription , checkout all latest code, run the tests for all these packages & , if successfull, take the buildsClassToPutVersionsOn and add a new method on it, describing the exact package numbers : >> for instance: >> BuildBotMockProjectBuilds class >> packagesForBuild20091127102654 >> ^#(('TODO' -> 'TODO-BKBAG.5.mcz') ('BuildBtMockProject' -> 'BuildBtMockProject-BartGauquie.7.mcz') ). >> >> Using this information I can easily install this exact version. Furthermore creating a release is then just creating for instance a class BuildBotMockProjectReleases with: >> BuildBotMockProjectReleasesclass >> release_1_0 >> ^BuildBotMockProjectBuilds packagesForBuild20091127102654 >> >> I am in fact 'tagging' a group packages that belong together and work together. So I can rollout this exact versions of the packages to some image automatically. >> >> >> However, >> Today I noticed that there already existed a project: http://www.squeaksource.com/TestServerSimple which more or less does the same thing. The only thing it does not do, or I did not find it, is to version the build, so that you can restore a project again ; a project which is consisting of a number of packages which are compatible and for which all tests ran. Is there a way to do that in TestServerSimple? >> >> Furthermore, are there any other projects that do a similar thing I'm not aware of? Is there any default way in Pharo to build a project which consists of multiple packages that each have different versions? >> >> I've seen MetaCello & Gofer as some other way 2 load groups of packages, but no integration into some kind of ci tool. >> >> This kind of automated ci server seems 2 me an essential tool in Pharo. >> >> Thanks for any pointers / ideas ! >> >> Kind Regards, >> >> Bart >> >> -- >> imagination is more important than knowledge - Albert Einstein >> Logic will get you from A to B. Imagination will take you everywhere - Albert Einstein >> Learn from yesterday, live for today, hope for tomorrow. The important thing is not to stop questioning. - Albert Einstein >> The true sign of intelligence is not knowledge but imagination. - Albert Einstein >> Gravitation is not responsible for people falling in love. - Albert Einstein >> _______________________________________________ >> Pharo-project mailing list >> [hidden email] >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: Continuous Integration Server for Pharo?
|
|
In reply to this post by Bart Gauquie
> I am currently working with Diego Kogan on the Continuous Integration Server for pharo.
While implementing this in Smalltalk is interesting - have you looked at just using Hudson out of the box? Its a pretty trivial install and would get something running quite quickly to bootstrap features that might be interesting later? I often worry that we don't study/use the competition first to understand all of the good features. For example Hudson/Cruise will monitor for hung builds for you - they also have excellent web dashboards that give you great stats on whats happening. Sure to do it properly you would need a Monticello plugin to detect changes (and that might be a good project) - however to get going its pretty easy to have a timed build that runs every hour. The detail is then to just focus on getting something that will run from the command line - load some script from a base image and then pull in the relavent metacello? lineup - and report all transcript errors to an external file for Hudson to pick up? The second artifact could then be to take the produced image and then run unit tests from it - and again report the results to a file for output. Finally Hudson does a great job of fanning out to multiple machines to run Web Browser tests (which typically can take a bit longer) to get results from those as well. It seems like its potentially easy to focus on the build tool and not the script that it needs to run to test all of the relevant details? Tim _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: Continuous Integration Server for Pharo?
|
|
In reply to this post by Jorge Ressia
Good!
Did you look/based on testServer? Did you check mason? I'm not sure that it was announce or that it is open-source or welcoming contribution? I think that this is important that we get a momentum. Stef On Nov 29, 2009, at 3:23 PM, Jorge Ressia wrote: > Hi, > > I am currently working with Diego Kogan on the Continuous Integration > Server for pharo. We will be working heavily the comming two weeks in > order to finish the implementation of the basic requirements. Our case > studies will be Moose and Seaside. > > We expect ot have a running version for mid december. > > Cheers, > > Jorge > > On Sun, Nov 29, 2009 at 7:53 AM, Stéphane Ducasse > <[hidden email]> wrote: >> Hi bart >> >> we use some scripts (developed at netstyle) >> and ESUG will support hernan student to develop one. I know that Jorge started also and should work with them. >> So we should join forces. >> The idea is that the server should be able to load metacello configuration. >> Jorge/Hernan/Simon >> - what is the status >> - how could we join forces? >> >> Stef >> >> >> On Nov 28, 2009, at 1:23 PM, Bart Gauquie wrote: >> >>> Dear all, >>> >>> I come from a Java world and there we use for instance : http://hudson-ci.org/ to do a continuous integration build. Which means all latest code gets updated, all test ran & if build successful, the code is tagged (so you can take the code again from a successful build). It also provides a nice GUI, and lets you inspect previous builds, the results from that build, ... . >>> >>> In Pharo, I did not directly find an alternative, and therefor, also as an exercise, I tried to build one myself. >>> The primitive, but working result so far is: http://www.squeaksource.com/BuildBot. >>> >>> This project depends on BBProjectDescription, which defines class BBBuildProjectDescription with defines: >>> • BBBuildProjectDescription>>allProjectPackageNames >>> • BBBuildProjectDescription>>installLatestWithDependencies >>> • BBBuildProjectDescription>>buildsClassToPutVersionsOn >>> What buildbot then does is using a BBBuildProjectDescription , checkout all latest code, run the tests for all these packages & , if successfull, take the buildsClassToPutVersionsOn and add a new method on it, describing the exact package numbers : >>> for instance: >>> BuildBotMockProjectBuilds class >> packagesForBuild20091127102654 >>> ^#(('TODO' -> 'TODO-BKBAG.5.mcz') ('BuildBtMockProject' -> 'BuildBtMockProject-BartGauquie.7.mcz') ). >>> >>> Using this information I can easily install this exact version. Furthermore creating a release is then just creating for instance a class BuildBotMockProjectReleases with: >>> BuildBotMockProjectReleasesclass >> release_1_0 >>> ^BuildBotMockProjectBuilds packagesForBuild20091127102654 >>> >>> I am in fact 'tagging' a group packages that belong together and work together. So I can rollout this exact versions of the packages to some image automatically. >>> >>> >>> However, >>> Today I noticed that there already existed a project: http://www.squeaksource.com/TestServerSimple which more or less does the same thing. The only thing it does not do, or I did not find it, is to version the build, so that you can restore a project again ; a project which is consisting of a number of packages which are compatible and for which all tests ran. Is there a way to do that in TestServerSimple? >>> >>> Furthermore, are there any other projects that do a similar thing I'm not aware of? Is there any default way in Pharo to build a project which consists of multiple packages that each have different versions? >>> >>> I've seen MetaCello & Gofer as some other way 2 load groups of packages, but no integration into some kind of ci tool. >>> >>> This kind of automated ci server seems 2 me an essential tool in Pharo. >>> >>> Thanks for any pointers / ideas ! >>> >>> Kind Regards, >>> >>> Bart >>> >>> -- >>> imagination is more important than knowledge - Albert Einstein >>> Logic will get you from A to B. Imagination will take you everywhere - Albert Einstein >>> Learn from yesterday, live for today, hope for tomorrow. The important thing is not to stop questioning. - Albert Einstein >>> The true sign of intelligence is not knowledge but imagination. - Albert Einstein >>> Gravitation is not responsible for people falling in love. - Albert Einstein >>> _______________________________________________ >>> Pharo-project mailing list >>> [hidden email] >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> >> >> _______________________________________________ >> Pharo-project mailing list >> [hidden email] >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: Continuous Integration Server for Pharo?
|
|
In reply to this post by Tim Mackinnon
On Nov 30, 2009, at 5:02 AM, Tim Mackinnon wrote: >> I am currently working with Diego Kogan on the Continuous Integration Server for pharo. > > While implementing this in Smalltalk is interesting - have you looked at just using Hudson out of the box? > > Its a pretty trivial install and would get something running quite quickly to bootstrap features that might be interesting later? I often worry that we don't study/use the competition first to understand all of the good features. For example Hudson/Cruise will monitor for hung builds for you - they also have excellent web dashboards that give you great stats on whats happening. Sure to do it properly you would need a Monticello plugin to detect changes (and that might be a good project) - however to get going its pretty easy to have a timed build that runs every hour. do you know if hudson has a public format for results. Because we could use that? > The detail is then to just focus on getting something that will run from the command line - load some script from a base image and then pull in the relavent metacello? lineup - and report all transcript errors to an external file for Hudson to pick up? The second artifact could then be to take the produced image and then run unit tests from it - and again report the results to a file for output. > > Finally Hudson does a great job of fanning out to multiple machines to run Web Browser tests (which typically can take a bit longer) to get results from those as well. > > It seems like its potentially easy to focus on the build tool and not the script that it needs to run to test all of the relevant details? > > Tim > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: Continuous Integration Server for Pharo?
|
|
In reply to this post by Bart Gauquie
As 'developer by default' of TestServerSimple, I can just: TestServerSimple is just that, something to automatically download the latest version of packages, run the tests, and send a small report by mail. Right now the server works with Metacello configuration, is still not bulletproof to some exceptions (like a TimeOut on a distant repository), and used to work with Flair config or directly with Monticello packages. Now I dont have the time or the energy to invest more in this thing, so I would be happy to beta-test integration servers on Moose. The features you describe are interesting. So once, everybody has settled on some good solution (be it BB, Hudson, ...), just come to me with a good tutorial :) On 28 nov. 2009, at 09:23, Bart Gauquie wrote:
-- Simon _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: Continuous Integration Server for Pharo?
|
|
Lukas, on the Mason angle, I followed the original thread on the
Seaside list but when I looked at the repository I couldn't really figure out where to start. Also it looked like there were two versions. Can you give any pointers on how to setup Mason? cheers, Mike _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: Continuous Integration Server for Pharo?
|
|
Colin, could you provide more information on Mason (maybe the mail you
once sent to me)? Pharo people are interested to investigate the system. Cheers, Lukas 2009/11/30 Michael Roberts <[hidden email]>: > Lukas, on the Mason angle, I followed the original thread on the > Seaside list but when I looked at the repository I couldn't really > figure out where to start. Also it looked like there were two > versions. Can you give any pointers on how to setup Mason? > > cheers, > Mike > > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > -- Lukas Renggli http://www.lukas-renggli.ch _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: Continuous Integration Server for Pharo?
|
|
Colin gave me the following text to forward to this thread:
> Colin, could you provide more information on Mason (maybe the mail you > once sent to me)? Pharo people are interested to investigate the > system. Sure. Here's that email, slightly reworked to bring it up to date with the current state of Mason. This isn't the grand design I started with, it's what's emerging, so it's a bit fuzzy and might be a bit hard to see in the current code. The core model is a hierarchy of targets. Targets represent the artifacts that Mason can produce, such as files, directories, images, packages etc, and the hierarchy represents the desired state, which Mason will work to create. There's a Canvas-like API for creating target hierarchies, and Products are objects that know how to specify targets using that API. A Product is an abstract thing, which represents a high-level concept such as "the latest version of a given package" or "the HTML documentation for a project." The role of a Product is to reduce that abstraction to a set of concrete targets which it specifies using the canvas API. A Product probably has to examine the environment to decide what targets to specify - it might scan a repository to determine the latest version of a package or scan a directory of creole files to determine which HTML files need to be generated. Given a target hierarchy, Mason can generate a Build. A Build contains a number of jobs, which collectively create the artifacts specified by the targets. Jobs essentially MethodObjects, and each do small amounts of work. The Build is responsible for running the jobs and managing dependencies between them. The build runs the jobs in parallel where possible, and since many jobs involve IO or spawning other processes via OSProcess, so there's actually a pretty decent performance advantage. Those are the broad outlines. There's a few issues here still, both architectural and with the current implementation. One is that building a target tree may be an unnecessary intermediate step. It would feasible to have targets just be transient entities the way TagBrushes are in Seaside - Products would specify a Build directly. Another is that I haven't tried to do any optimization along the lines of make - everything is built fresh every time. The targets -> build converter could try to examine the disk and see if it could avoid building things that are already up to date. I have an idea to address both these issues by having a step that walks over the target tree and optimizes it before the Build is created. This could involve identifying and merging common intermediate steps, and then doing the make trick of eliminating any targets that already exist and are up-to-date. For now though, I've been quite happy with even the unoptimized version. I've used it to build distribution SAR archives for OB, MC2 and Filesystem, development images, load-order and package dependency testing etc. Colin -- Lukas Renggli http://www.lukas-renggli.ch _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
«
Return to Pharo Smalltalk Developers
|
1 view|%1 views
| Free forum by Nabble | Edit this page |