[ANN] Mason 1.0

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[ANN] Mason 1.0

Colin Putney
Hi folks,

As I mentioned yesterday, here's the release version of Mason, a Squeak build system. It relies on OSProcess to build images, but it should be possible to build other artifacts, such as directory trees, zip files, SAR files etc without OSProcess. The download is available here:

http://www.wiresong.ca/downloads/Mason-1.0.0.sar

There's no documentation yet, but I've pasted below a brief description that I sent to the Pharo folks a little while ago.

Cheers,

Colin

---

The core model is a hierarchy of targets. Targets represent the artifacts that Mason can produce, such as files, directories, images, packages etc, and the hierarchy represents the desired state, which Mason will work to create. There's a Canvas-like API for creating target hierarchies, and Products are objects that know how to specify targets using that API.

A Product is an abstract thing, which represents a high-level concept such as "the latest version of a given package" or "the HTML documentation for a project." The role of a Product is to reduce that abstraction to a set of concrete targets which it specifies using the canvas API. A Product probably has to examine the environment to decide what targets to specify - it might scan a repository to determine the latest version of a package or scan a directory of creole files to determine which HTML files need to be generated.

Given a target hierarchy, Mason can generate a Build. A Build contains a number of jobs, which collectively create the artifacts specified by the targets. Jobs essentially MethodObjects, and each do small amounts of work. The Build is responsible for running the jobs and managing dependencies between them. The build runs the jobs in parallel where possible, and since many jobs involve IO or spawning other processes via OSProcess, so there's actually a pretty decent performance advantage.

Those are the broad outlines. There's a few issues here still, both architectural and with the current implementation.

One is that building a target tree may be an unnecessary intermediate step. It would feasible to have targets just be transient entities the way TagBrushes are in Seaside - Products would specify a Build directly. Another is that I haven't tried to do any optimization along the lines of make - everything is built fresh every time. The targets -> build converter could try to examine the disk and see if it could avoid building things that are already up to date.

I have an idea to address both these issues by having a step that walks over the target tree and optimizes it before the Build is created. This could involve identifying and merging common intermediate steps, and then doing the make trick of eliminating any targets that already exist and are up-to-date.

For now though, I've been quite happy with even the unoptimized version. I've used it to build distribution SAR archives for OB, MC2 and Filesystem, development images, load-order and package dependency testing etc.



Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Mason 1.0

Andreas.Raab
Hi Colin -

I looked briefly at Mason, but I'm clearly missing a bit of
documentation :-) Could you perhaps make a very brief tutorial that
includes how to load specific packages and how to set up an environment
that builds it? Specifically, I'd be interested in what a Mason setup
would look like that would allow us to automatically build up from a
core to a standard image from MC packages, run all the tests, and
package and save the result (and if supported, how to upload it to
Squeak.org and mail the test results to Squeak-dev ;-)

Thanks!
   - Andreas


Colin Putney wrote:

> Hi folks,
>
> As I mentioned yesterday, here's the release version of Mason, a Squeak build system. It relies on OSProcess to build images, but it should be possible to build other artifacts, such as directory trees, zip files, SAR files etc without OSProcess. The download is available here:
>
> http://www.wiresong.ca/downloads/Mason-1.0.0.sar
>
> There's no documentation yet, but I've pasted below a brief description that I sent to the Pharo folks a little while ago.
>
> Cheers,
>
> Colin
>
> ---
>
> The core model is a hierarchy of targets. Targets represent the artifacts that Mason can produce, such as files, directories, images, packages etc, and the hierarchy represents the desired state, which Mason will work to create. There's a Canvas-like API for creating target hierarchies, and Products are objects that know how to specify targets using that API.
>
> A Product is an abstract thing, which represents a high-level concept such as "the latest version of a given package" or "the HTML documentation for a project." The role of a Product is to reduce that abstraction to a set of concrete targets which it specifies using the canvas API. A Product probably has to examine the environment to decide what targets to specify - it might scan a repository to determine the latest version of a package or scan a directory of creole files to determine which HTML files need to be generated.
>
> Given a target hierarchy, Mason can generate a Build. A Build contains a number of jobs, which collectively create the artifacts specified by the targets. Jobs essentially MethodObjects, and each do small amounts of work. The Build is responsible for running the jobs and managing dependencies between them. The build runs the jobs in parallel where possible, and since many jobs involve IO or spawning other processes via OSProcess, so there's actually a pretty decent performance advantage.
>
> Those are the broad outlines. There's a few issues here still, both architectural and with the current implementation.
>
> One is that building a target tree may be an unnecessary intermediate step. It would feasible to have targets just be transient entities the way TagBrushes are in Seaside - Products would specify a Build directly. Another is that I haven't tried to do any optimization along the lines of make - everything is built fresh every time. The targets -> build converter could try to examine the disk and see if it could avoid building things that are already up to date.
>
> I have an idea to address both these issues by having a step that walks over the target tree and optimizes it before the Build is created. This could involve identifying and merging common intermediate steps, and then doing the make trick of eliminating any targets that already exist and are up-to-date.
>
> For now though, I've been quite happy with even the unoptimized version. I've used it to build distribution SAR archives for OB, MC2 and Filesystem, development images, load-order and package dependency testing etc.
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Re: [ANN] Mason 1.0

Colin Putney

On 2009-12-13, at 10:25 PM, Andreas Raab wrote:

> Hi Colin -
>
> I looked briefly at Mason, but I'm clearly missing a bit of documentation :-) Could you perhaps make a very brief tutorial that includes how to load specific packages and how to set up an environment that builds it? Specifically, I'd be interested in what a Mason setup would look like that would allow us to automatically build up from a core to a standard image from MC packages, run all the tests, and package and save the result (and if supported, how to upload it to Squeak.org and mail the test results to Squeak-dev ;-)

1. create a directory for the tutorial
2. copy a trunk image into the directory, name it 'trunk.image'.
3. make another copy of the trunk image, name it 'mason.image'.

4. In mason.image, create a class called MDStandardImageProduct.
5. Implement the following method:

specifyOn: targets
        | repository |
        repository := MCHttpRepository
                location: 'http://squeak.krampe.se/public/'
                user: ''
                password: ''.

        targets image
                name: 'Squeak-Standard-3-11';
                baseImage: FSLocator image resolve parent / 'trunk.image';
                with: [targets version name: 'Blackfoot-gk.6.mcz' repository: repository].
               
        targets tests
                name: 'results';
                imagePath: 'Squeak-Standard-3-11';
                with: [targets suite package: 'Blackfoot'].
                       
6. In a workspace execute 'MDStandardImageProduct build'.


This should product a directory called something like build-123.1. Inside that
directory will be an image called Squeak-Standard-3-11.image. It will have
Göran's Blackfoot SimpleCGI server loaded.

There will also be an image called results.image and a file named results.txt.
This is the image in which the tests were run, and the result of the tests
respectively.

And that's it. There's currently no support for uploading the image or mailing
the test results, but that could certainly be implemented.

The #specifyOn: method uses a Canvas-style protocol - a bit like a Seaside rendering method. See MDTargetCanvas. The targets it supports right now are: #directory, #file, #image, #script, #suite, #tests and #zip.

Obviously building a complete release image from core would be a bit more involved, but it would boil down to something more or less like this.



Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Mason 1.0

Patrick Shouse
In reply to this post by Colin Putney

Colin Putney wrote
Hi folks,

As I mentioned yesterday, here's the release version of Mason, a Squeak build system. It relies on OSProcess to build images, but it should be possible to build other artifacts, such as directory trees, zip files, SAR files etc without OSProcess. The download is available here:

http://www.wiresong.ca/downloads/Mason-1.0.0.sar

There's no documentation yet, but I've pasted below a brief description that I sent to the Pharo folks a little while ago.

Cheers,

Colin
Here's a patch to Filesystem-Resolvers to get this working on Linux. Tested on Ubuntu 9.10 using the tutorial from this thread.

Thanks,
Patrick

'From Squeak3.11alpha of 16 December 2009 [latest update: #8496] on 18 December 2009 at 6:26:25 pm'!
FSPlatformResolver subclass: #FSUnixResolver
        instanceVariableNames: ''
        classVariableNames: ''
        poolDictionaries: ''
        category: 'Filesystem-Resolvers'!

!FSUnixResolver methodsFor: 'as yet unclassified' stamp: 'pls 12/18/2009 04:52'!
desktop
        ^ self home / 'Desktop'! !

!FSUnixResolver methodsFor: 'as yet unclassified' stamp: 'pls 12/18/2009 04:53'!
documents
        ^ self home / 'Documents'! !

!FSUnixResolver methodsFor: 'as yet unclassified' stamp: 'pls 12/18/2009 04:52'!
home
        ^ (self resolveString: SecurityManager default untrustedUserDirectory)
                parent parent parent parent parent! !

"-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- "!

FSUnixResolver class
        instanceVariableNames: ''!

!FSUnixResolver class methodsFor: 'as yet unclassified' stamp: 'pls 12/18/2009 04:53'!
platformName
        ^  'unix'! !
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Mason 1.0

Colin Putney

On 2009-12-18, at 10:30 AM, Patrick Shouse wrote:

> Here's a patch to Filesystem-Resolvers to get this working on Linux. Tested
> on Ubuntu 9.10 using the tutorial from this thread.

Hi Patrick,

Was this using Gnome or KDE? I guess the code for resolving "the Desktop" on Unix will end up being quite complicated, and sometimes fail and fallback to asking the user. This is a good start, thanks!

Colin