Fwd: Metacello questions

[forwarded from Squeak-dev]

Hi -

I've been trying to wet my feet with Metacello a little by writing a
ConfigurationOfWebClient and in the process I've come across various
things that I don't understand or don't know how to express. The
configuration itself can be found here:

        http://www.squeaksource.com/MetacelloRepository

Here are the questions I'm having:

1) #includes: vs. #requires:
What is the difference between #includes: and #requires:? They sound
entirely interchangeable, in particular after reading the tutorial which
states:

        "When you use the #includes: directive, you are not only specifying
that the listed packages should be loaded when the parent package is
loaded, but that the #included: packages should be loaded _before_ any
packages that require the parent package."

I would expect that precise behavior for required packages, i.e., if Foo
requires: Bar then Bar is loaded before Foo if some other package
requires: Foo. What am I missing?

2) Why is HelpSystem not loaded?
In my configuration, baseline specifies that HelpSystem is required for
WebClient-Help:

                "Documentation doesn't require Core/Tests, but HelpSystem"
                spec package: 'WebClient-Help' with:[spec requires: 'HelpSystem'].

                spec project: 'HelpSystem' with:[
                        spec
                                className: 'ConfigurationOfHelpSystem';
                                repository: 'http://www.squeaksource.com/MetacelloRepository'
                ].

However, when executed, Metacello goes through the motions but doesn't
actually *load* any version of HelpSystem (I can see it 'fetch' the
packages but not load them). It even calls the postLoadDoIt on it (which
fails since nothing was loaded). What am I doing wrong?

3) When exactly is HelpSystem loaded?

What I'm wondering about in the above is that HelpSystem should only be
loaded if WebClient-Help is being loaded. Is this implicitly part of
specifying a 'project' instead of a 'package'? Or is there some other
way of specifying that?

I guess really the question here is what entities get loaded implicitly
simply by declaring them (packages for sure, any others?) and what
entities need to be 'required' in order to be loaded?

4) How does one define dependencies that differ based on platform?

In WebClient, the WebClient-HTTP package requires WebClient-Core and
-only for: #pharo- also WebClient-Pharo. How does one express that
there's an additional dependency for WebClient-HTTP that only exists
for: #pharo? The config currently specifies:

        spec for: #common do:[
                " ... "
                spec package: 'WebClient-HTTP' with:[spec requires: 'WebClient-Core'].
                " ... "
        ].

        spec for: #pharo do:[
                "Just the fact it exists; no requirements here"
                spec package: 'WebClient-Pharo'.
        ].

Which means that WebClient-Pharo will be loaded on Pharo by default but
it's missing the fact that WebClient-Pharo is in fact a requirement for
WebClient-HTTP (but only on that platform). How is that best expressed?

5) Bootstrapping Metacello?

I found that most of the ConfigurationsOfXXX include some code to
'bootstrap' Metacello in some form. There appear to be variants on the
code; is there a "canonical bootstrap method" that should be used?

6) A standard Configuration template?

The Metacello tutorial is very useful as far as the configuration goes
but it falls short of getting into the details of what it means to
provide a 'working' configuration. Testing the configuration isn't
covered (is there a way to get back to a 'clean' state?); bootstrapping
isn't covered; the meaning of #isMetacelloConfig,
#metacelloVesion:loads:, #lastMetacelloVersionLoad and several other
-seemingly random- methods that can be found on most -but not all- of
the configurations. There is no indication on what these do, if they're
required, recommended, or optional. Is there some 'standard'
Configuration template that people can use to avoid guessing about which
parts of the boilerplate to copy from some other configuration?

That's it for now. Thanks for any insights on any of these issues.

Cheers,
   - Andreas

I've replied to question 1) in the Squeak-dev list (http://forum.world.st/Metacello-questions-tt2132073.html#a2133038), but will finish/continue the discussion on the Metacello list (http://forum.world.st/Fwd-Metacello-questions-tt2132927.html#a2132927)...

Dale
----- "Andreas.Raab" <[hidden email]> wrote:

| [forwarded from Squeak-dev]
|
| Hi -
|
| I've been trying to wet my feet with Metacello a little by writing a
| ConfigurationOfWebClient and in the process I've come across various
| things that I don't understand or don't know how to express. The
| configuration itself can be found here:
|
| http://www.squeaksource.com/MetacelloRepository
|
| Here are the questions I'm having:
|
| 1) #includes: vs. #requires:
| What is the difference between #includes: and #requires:? They sound
| entirely interchangeable, in particular after reading the tutorial
| which
| states:
|
| "When you use the #includes: directive, you are not only specifying
| that the listed packages should be loaded when the parent package is
| loaded, but that the #included: packages should be loaded _before_
| any
| packages that require the parent package."
|
| I would expect that precise behavior for required packages, i.e., if
| Foo
| requires: Bar then Bar is loaded before Foo if some other package
| requires: Foo. What am I missing?
|
| 2) Why is HelpSystem not loaded?
| In my configuration, baseline specifies that HelpSystem is required
| for
| WebClient-Help:
|
| "Documentation doesn't require Core/Tests, but HelpSystem"
| spec package: 'WebClient-Help' with:[spec requires: 'HelpSystem'].
|
| spec project: 'HelpSystem' with:[
| spec
| className: 'ConfigurationOfHelpSystem';
| repository: 'http://www.squeaksource.com/MetacelloRepository'
| ].
|
| However, when executed, Metacello goes through the motions but
| doesn't
| actually *load* any version of HelpSystem (I can see it 'fetch' the
| packages but not load them). It even calls the postLoadDoIt on it
| (which
| fails since nothing was loaded). What am I doing wrong?
|
| 3) When exactly is HelpSystem loaded?
|
| What I'm wondering about in the above is that HelpSystem should only
| be
| loaded if WebClient-Help is being loaded. Is this implicitly part of
| specifying a 'project' instead of a 'package'? Or is there some other
| way of specifying that?
|
| I guess really the question here is what entities get loaded
| implicitly
| simply by declaring them (packages for sure, any others?) and what
| entities need to be 'required' in order to be loaded?
|
| 4) How does one define dependencies that differ based on platform?
|
| In WebClient, the WebClient-HTTP package requires WebClient-Core and
| -only for: #pharo- also WebClient-Pharo. How does one express that
| there's an additional dependency for WebClient-HTTP that only exists
| for: #pharo? The config currently specifies:
|
| spec for: #common do:[
| " ... "
| spec package: 'WebClient-HTTP' with:[spec requires:
| 'WebClient-Core'].
| " ... "
| ].
|
| spec for: #pharo do:[
| "Just the fact it exists; no requirements here"
| spec package: 'WebClient-Pharo'.
| ].
|
| Which means that WebClient-Pharo will be loaded on Pharo by default
| but
| it's missing the fact that WebClient-Pharo is in fact a requirement
| for
| WebClient-HTTP (but only on that platform). How is that best
| expressed?
|
| 5) Bootstrapping Metacello?
|
| I found that most of the ConfigurationsOfXXX include some code to
| 'bootstrap' Metacello in some form. There appear to be variants on
| the
| code; is there a "canonical bootstrap method" that should be used?
|
| 6) A standard Configuration template?
|
| The Metacello tutorial is very useful as far as the configuration
| goes
| but it falls short of getting into the details of what it means to
| provide a 'working' configuration. Testing the configuration isn't
| covered (is there a way to get back to a 'clean' state?);
| bootstrapping
| isn't covered; the meaning of #isMetacelloConfig,
| #metacelloVesion:loads:, #lastMetacelloVersionLoad and several other
| -seemingly random- methods that can be found on most -but not all- of
| the configurations. There is no indication on what these do, if
| they're
| required, recommended, or optional. Is there some 'standard'
| Configuration template that people can use to avoid guessing about
| which
| parts of the boilerplate to copy from some other configuration?
|
| That's it for now. Thanks for any insights on any of these issues.
|
| Cheers,
|    - Andreas

----- "Andreas.Raab" <[hidden email]> wrote:

| 3) When exactly is HelpSystem loaded?
|
| What I'm wondering about in the above is that HelpSystem should only
| be
| loaded if WebClient-Help is being loaded. Is this implicitly part of
| specifying a 'project' instead of a 'package'? Or is there some other
| way of specifying that?
|
| I guess really the question here is what entities get loaded
| implicitly
| simply by declaring them (packages for sure, any others?) and what
| entities need to be 'required' in order to be loaded?

Andreas,

Here's the Transcript from a (successful) load of WebClient:

Loading 1.0 of ConfigurationOfWebClient...
Fetched -> ConfigurationOfHelpSystem-tbn.12
Loaded -> ConfigurationOfHelpSystem-tbn.12
Fetched -> WebClient-Core-ar.15
Fetched -> WebClient-Tests-ar.7
Fetched -> WebClient-HTTP-ar.1
Project: HelpSystem
Fetched -> HelpSystem-Core-tbn.40
Fetched -> HelpSystem-Tests-tbn.9
Fetched -> Pharo-Project-Help-tbn.8
Fetched -> WebClient-Help-ar.4
Fetched -> WebClient-Pharo-ar.1
Starting atomic load
        Loaded -> WebClient-Core-ar.15
        Loaded -> WebClient-Tests-ar.7
        Loaded -> WebClient-HTTP-ar.1
Finished atomic load
Loaded -> HelpSystem-Core-tbn.40
Loaded -> HelpSystem-Tests-tbn.9
Loaded -> Pharo-Project-Help-tbn.8
Starting atomic load
        Loaded -> WebClient-Help-ar.4
        Loaded -> WebClient-Pharo-ar.1
Finished atomic load
Evaluated -> 1.1 [ConfigurationOfHelpSystem] >> openHelpBrowser

In your configuration you had specified that WebClient-Help required the Help System, so you can see that the load of the Help System was inserted at that point in the load (which is what triggered the bug, too).

The order of specifying packages/projects implies the order that things will be loaded in. With #requires:, the load order is explicit.

I would recommend that you explicitly use #requires: unless only one or two packages are involved.

The implicit load order is not obvious when you introduce conditional loading (with #squeakCommon #squeak, and #pharo), so if load order matters I would explicitly declare the relationships.

The other aspect besides load order is what you want loaded. By default all packages/projects are loaded, but you can define a 'default' group which would be used to specify a subset of the packages/projects that are loaded by default. This is useful if you want to control whether or not a set of packages (say tests) are loaded or not ...

I don't think I've completely answered your questions so we might want to continue the dialog on this question....

Dale

Andreas,

Your 'self answers' all look good.

| spec
| package:'Foo' with:[spec requires: 'Bar'];
| package:'Foo' with:[spec requires: 'Baz'];
|
| and
| spec
| package: 'Foo' with:[spec requires: #('Bar' 'Baz')].

are indeed equivalent. So the difference pieces of the spec can be included in different #for:do: blocks based on the platform.

To take you a little bit into the internals...You will notice that all packages/projects/groups are named, so for each spec of a given name an ordered list of modifications are recorded while resolving the spec on a given platform.

  1. all #version: pragmas (no import and therefore self-contained) are processed
     first
  2. #version:import: pragmas are processed next (using the imported version as a
     starting point).
  3. within a #version: or #version:import: method, the specs are processed in
     project attribute order (i.e., for Squeak -> #common, #squeakCommon, #squeak).
  4. the individual statements in the #for:do: for a given attribute are processed
     in order.
  5. there are 4 unique operations supported:
     - add
     - copy
     - merge
     - remove

Once the initial processing is complete, the operations are collapsed to form the final named spec. The named specs are collected into the version ....

The #add operation is used by the constructors with 'overrides' in their name. The #add operation replaces the named spec.

The #copy operation is used the by the #project:copyFrom:with: constructor. Note that #copy requires that an #add/#merge operation proceeds it in it's list.

The #merge operation merges it's items into the previous version of the named spec. Not that #merge is equivalent to #add if the name spec doesn't exist yet. Most of the constructors use the #merge operation.

The #remove operation is used by the constructors with 'remove' in their name. The #remove operation removes it's from the previous version of the named spec.

So you can imagine that as you traverse through an attribute list for a particular project, that each of the individual constructor statements for a named spec are processed in a very specific order with each of the #add, #copy, #merge, and #remove operations applied in order...

Hope this helps,

Dale

----- "Andreas Raab" <[hidden email]> wrote:

| On 5/6/2010 5:26 PM, Dale Henrichs wrote:
| > I don't think I've completely answered your questions so we might
| want to continue the dialog on this question....
|
| We're getting close :-) Let me take my own questions and answer them
| in
| my own words to see if I understand the issues correctly:
|
| Q1: #includes: vs. #requires:
|
| If I understand correctly what you're saying then, in heavily
| simplified
| form, I'd say they're both dependencies, except that #requires: is
| loaded before the package it relates to and #includes: afterwords.
| Thus:
|
| spec package: 'B' with:[
| spec requires: 'A'.
| spec includes: 'C'.
| ].
|
| will load A, B, and C in that order. One thing one can do with this is
|
| to have mutually dependent packages, for example if package Foo and
| Bar
| always must be loaded together (Foo before Bar) one would specify:
|
| spec package: 'Foo' with:[spec includes: 'Bar'].
| spec package: 'Bar' with:[spec requires: 'Foo'].
|
| There is now no way by which Foo or Bar could be loaded without its
| companion. Correct?
|
| Q2: Why is HelpSystem not loaded?
|
| A: That's a bug which I can work around by specifying either #version:
|
| or #versionString:.
|
| Q3: When exactly is HelpSystem loaded?
| A: In the context of my configuration HelpSystem is loaded for two
| reasons: a) because it's a dependency for WebClient-Help and b)
| because
| no 'default' group has been specified it would be loaded implicitly.
|
| Q4: How does one define dependencies that differ based on platform?
| A: I'm not really sure about this. My understanding is that I could
| use
| #includes: to model the dependency in the "for: #pharo" branch, but
| one
| question that is still open is whether the following two expressions
| are
| equivalent:
|
| spec
| package:'Foo' with:[spec requires: 'Bar'];
| package:'Foo' with:[spec requires: 'Baz'];
|
| and
| spec
| package: 'Foo' with:[spec requires: #('Bar' 'Baz')].
|
| Are these equivalent?
|
| Cheers,
|    - Andreas

----- "Andreas.Raab" <[hidden email]> wrote:

| 5) Bootstrapping Metacello?
|
| I found that most of the ConfigurationsOfXXX include some code to
| 'bootstrap' Metacello in some form. There appear to be variants on
| the
| code; is there a "canonical bootstrap method" that should be used?
|

There is a canonical bootstrap method that is documented here:

  http://code.google.com/p/metacello/wiki/EnsureMetacello

But the best technique is to copy the class MetacelloConfigTemplate where all of the required methods are already implemented...

For more information see my answer to Question #6 below.

| 6) A standard Configuration template?
|
| The Metacello tutorial is very useful as far as the configuration
| goes
| but it falls short of getting into the details of what it means to
| provide a 'working' configuration. Testing the configuration isn't
| covered (is there a way to get back to a 'clean' state?);
| bootstrapping
| isn't covered; the meaning of #isMetacelloConfig,
| #metacelloVesion:loads:, #lastMetacelloVersionLoad and several other
| -seemingly random- methods that can be found on most -but not all- of
| the configurations. There is no indication on what these do, if
| they're
| required, recommended, or optional. Is there some 'standard'
| Configuration template that people can use to avoid guessing about
| which
| parts of the boilerplate to copy from some other configuration?

There is a FAQ on creating a new Metacello configuration:

 http://code.google.com/p/metacello/wiki/FAQ#How_do_I_create_a_Metacello_Configuration?

but unfortunately the instructions are buried within a series of workspaces in ProfStef ... a Pharo-only package:(

I've just extracted the instructions and added the following link to the FAQ answer:

  http://code.google.com/p/metacello/wiki/CreateMetacelloConfiguration

Hope this helps,

Dale