Metacello Bootstrap

Now that Metacello is becoming integral to many of our workflows, it'd be nice to have the bootstrap be robust, esp. to squeaksource being down...

A few things:
* some feedback when waiting would be great. When sqs is down, evaluating Metacello scripts seems to hang the image
* I'm thinking we should at least add the LatAm mirror to each package/project depending on sqs (e.g. HelpSystem)...
* (mildly related) The bootstrap is loading Help, Toolbox, etc. Bootstrapping feels slow and most users will never want these packages. I think the bootstrap should load the bare-minimum - Metacello-Core, I guess. Also, it seems confusing that 'Core' = 'default', which loads 'Metacello-Core' as well as other package...

Dale, I'll create issues if you are on board...

Sean

On 18 June 2012 17:04, Sean P. DeNigris <[hidden email]> wrote:
> Now that Metacello is becoming integral to many of our workflows, it'd be
> nice to have the bootstrap be robust, esp. to squeaksource being down...

This is partly a Pharo thing. If you look at ConfigurationOfXtreams
you'll see that bootstrapping has a fallover option. In Squeak that's
always worked, but I found that it didn't work in Pharo. I think
that's because Zinc behaves differently to whatever Squeak uses for
HTTP. Something to bear in mind, at least.

frank

Sean,

Good feedback ...

Regarding sqs being down, the Metacello bootstrapping should not be impacted if sqs is down .. regarding having Metacello be a sentinel for sqs problems I'm less inclined to do anything special (uncoditionally checking for sqs being up penalizes folks who are not using sqs ...). The latam mirror works so well, that I'm not sure that Metacello can do better ...

With all of that said, moving forward with 1.0-beta.32, I could change the configuration for Metacello to directly reference the latam mirror for the dependent project configurations (could you submit an issue using the github list[1]?).

Regarding bootstrap contents ... with 1.0-beta.32, I have created a 'batch' group that loads only the bare-minimum needed for loading configurations (no help, tutorial, etc.). And the bootstrap process no loads 'batch' ...

I'm going to add a `help` command[2] to the scripting api that will load the tutorial and help info, so I can probably afford to switch the 'default' group to 'batch' as well (could you add this note to your issue?).

Thanks,

Dale

[1] https://github.com/dalehenrich/metacello-work
[2] https://github.com/dalehenrich/metacello-work/issues/36

----- Original Message -----
| From: "Sean P. DeNigris" <[hidden email]>
| To: [hidden email]
| Sent: Monday, June 18, 2012 9:04:38 AM
| Subject: [Metacello] Metacello Bootstrap
|
| Now that Metacello is becoming integral to many of our workflows,
| it'd be
| nice to have the bootstrap be robust, esp. to squeaksource being
| down...
|
| A few things:
| * some feedback when waiting would be great. When sqs is down,
| evaluating
| Metacello scripts seems to hang the image
| * I'm thinking we should at least add the LatAm mirror to each
| package/project depending on sqs (e.g. HelpSystem)...
| * (mildly related) The bootstrap is loading Help, Toolbox, etc.
| Bootstrapping feels slow and most users will never want these
| packages. I
| think the bootstrap should load the bare-minimum - Metacello-Core, I
| guess.
| Also, it seems confusing that 'Core' = 'default', which loads
| 'Metacello-Core' as well as other package...
|
| Dale, I'll create issues if you are on board...
|
| Sean
|
| --
| View this message in context:
| http://forum.world.st/Metacello-Bootstrap-tp4635359.html
| Sent from the Metacello mailing list archive at Nabble.com.
|

Dale Henrichs wrote

Regarding sqs being down, the Metacello bootstrapping should not be impacted if sqs is down

In Pharo 1.4, the bootstrap loads the default group, which includes Help on squeakCommon, which includes HelpSystem, the config for which uses sqs/MetacelloRepository as the repo

Dale Henrichs wrote

 .. regarding having Metacello be a sentinel for sqs problems I'm less inclined to do anything special (uncoditionally checking for sqs being up penalizes folks who are not using sqs ...).

I was thinking something like stricter timeouts for the http requests...

Dale Henrichs wrote

With all of that said, moving forward with 1.0-beta.32, I could change the configuration for Metacello to directly reference the latam mirror for the dependent project configurations (could you submit an issue using the github list[1]?).

Worth a try... I'll submit an issue...

Dale Henrichs wrote

Regarding bootstrap contents ... with 1.0-beta.32, I have created a 'batch' group that loads only the bare-minimum needed for loading configurations (no help, tutorial, etc.). And the bootstrap process no loads 'batch' ...

MetacelloBaseConfiguration>>ensureMetacello -> "ConfigurationOfMetacello load" -> "self project latestVersion load" = 'default'; should be #load: 'bootstrap', no?

Dale Henrichs wrote

I'm going to add a `help` command[2] to the scripting api that will load the tutorial and help info, so I can probably afford to switch the 'default' group to 'batch' as well (could you add this note to your issue?).

I don't think it's bad having help loaded by default. Someone loading Metacello by itself may very well want to explore. I just don't think dev-related stuff belongs in the bootstrap.

Sean

b.t.w. this all started because I was trying to make a config for MonticelloRedirect -> LatAm mirror, which bootstrapped Metacello, which tried to look on sqs...

Thanks :)
Sean

Embedded replies

----- Original Message -----
| From: "Sean P. DeNigris" <[hidden email]>
| To: [hidden email]
| Sent: Monday, June 18, 2012 10:53:37 AM
| Subject: [Metacello] Re: Metacello Bootstrap
|
|
| Dale Henrichs wrote
| >
| > Regarding sqs being down, the Metacello bootstrapping should not be
| > impacted if sqs is down
| >
| In Pharo 1.4, the bootstrap loads the default group, which includes
| Help on
| squeakCommon, which includes HelpSystem, the config for which uses
| sqs/MetacelloRepository as the repo

yeah, so the HelpSystem config would need to be changed...

|
|
| Dale Henrichs wrote
| >
| >  .. regarding having Metacello be a sentinel for sqs problems I'm
| >  less
| > inclined to do anything special (uncoditionally checking for sqs
| > being up
| > penalizes folks who are not using sqs ...).
| >
| I was thinking something like stricter timeouts for the http
| requests...

Metacello doesn't touch the http server directly, in fact Metacello uses Gofer to load packages...

|
|
| Dale Henrichs wrote
| >
| > With all of that said, moving forward with 1.0-beta.32, I could
| > change the
| > configuration for Metacello to directly reference the latam mirror
| > for the
| > dependent project configurations (could you submit an issue using
| > the
| > github list[1]?).
| >
| Worth a try... I'll submit an issue...
|
|
| Dale Henrichs wrote
| >
| > Regarding bootstrap contents ... with 1.0-beta.32, I have created a
| > 'batch' group that loads only the bare-minimum needed for loading
| > configurations (no help, tutorial, etc.). And the bootstrap process
| > no
| > loads 'batch' ...
| >
| MetacelloBaseConfiguration>>ensureMetacello ->
| "ConfigurationOfMetacello
| load" -> "self project latestVersion load" = 'default'; should be
| #load:
| 'bootstrap', no?

Yes, once the `latestVersion` is 1.0-beta.32...

|
|
| Dale Henrichs wrote
| >
| > I'm going to add a `help` command[2] to the scripting api that will
| > load
| > the tutorial and help info, so I can probably afford to switch the
| > 'default' group to 'batch' as well (could you add this note to your
| > issue?).
| >
| I don't think it's bad having help loaded by default. Someone loading
| Metacello by itself may very well want to explore. I just don't think
| dev-related stuff belongs in the bootstrap.

Yep, I agree.

|
| Sean
|
| b.t.w. this all started because I was trying to make a config for
| MonticelloRedirect -> LatAm mirror, which bootstrapped Metacello,
| which
| tried to look on sqs...

Bootstrapping the bootstrap is always fun ... you know if you load the LatAm mirror code first, then the bootstrap of Metacello will work correctly:)

|
| Thanks :)
| Sean
|
| --
| View this message in context:
| http://forum.world.st/Metacello-Bootstrap-tp4635359p4635371.html
| Sent from the Metacello mailing list archive at Nabble.com.
|

Dale Henrichs wrote

yeah, so the HelpSystem config would need to be changed...

I made a mistake. The problems are two-fold. 1) Help is being loaded in the "minimum" install and 2) ConfigurationOfMetacello is looking for ConfigurationOfHelpSystem in sqs/MetacelloRepository.

If you make the bootstrap load the bootstrap group, and make the LatAm mirror a repo for dependent projects, as we discussed above, it will be fine.

b.t.w. how far out is 1.0-beta.32?

"It is faaaaar out, man!"

I'm getting ready to start introducing a Preview version so that folks can start playing a bit ... within days.

I've got almost 30 open issues before it's ready for a full-fledged preview ... so I'm guessing weeks to a month or so.

I'll give you the keys to the configuration if you'd like to push out an updated config with the changes you are suggesting sooner...

Dale

[1] https://github.com/dalehenrich/metacello-work/issues?milestone=1&sort=updated&state=open

----- Original Message -----
| From: "Sean P. DeNigris" <[hidden email]>
| To: [hidden email]
| Sent: Monday, June 18, 2012 12:19:46 PM
| Subject: [Metacello] Re: Metacello Bootstrap
|
|
| Dale Henrichs wrote
| >
| > yeah, so the HelpSystem config would need to be changed...
| >
|
| I made a mistake. The problems are two-fold. 1) Help is being loaded
| in the
| "minimum" install and 2) ConfigurationOfMetacello is looking for
| ConfigurationOfHelpSystem in sqs/MetacelloRepository.
|
| If you make the bootstrap load the bootstrap group, and make the
| LatAm
| mirror a repo for dependent projects, as we discussed above, it will
| be
| fine.
|
| b.t.w. how far out is 1.0-beta.32?
|
| --
| View this message in context:
| http://forum.world.st/Metacello-Bootstrap-tp4635359p4635378.html
| Sent from the Metacello mailing list archive at Nabble.com.
|

--
Mariano
http://marianopeck.wordpress.com

Mariano,

That's exactly where I'm headed with 1.0-beta.32 ... I've created a 'batch' group that loads just what's needed for loading ... running the scripts ... there will be a script command `help` that will load the Metacello help/tutorial.

Dale

----- Original Message -----
| From: "Mariano Martinez Peck" <[hidden email]>
| To: [hidden email]
| Sent: Monday, June 18, 2012 2:08:53 PM
| Subject: Re: [Metacello] Re: Metacello Bootstrap
|
|
|
|
|
|
| | > Regarding sqs being down, the Metacello bootstrapping should not
| | > be
| | > impacted if sqs is down
| | >
| | In Pharo 1.4, the bootstrap loads the default group, which includes
| | Help on
| | squeakCommon, which includes HelpSystem, the config for which uses
| | sqs/MetacelloRepository as the repo
|
| yeah, so the HelpSystem config would need to be changed...
|
|
|
|
| I think we have discussed this in the past. Once I proposed that
| #ensureMetacello should load the really minimal core needed.
| No tools, no help, no nothing. Just the core in order to be able to
| load stuff.
| If someone wants more than that (as it was the previous Pharo images)
| then such person should explictly load Metacello and not by the
| #ensureMetacello. Then ConfigurationOfMetacello 'default' group can
| load whatevet you want.
| What do you think?
|
|
|
|
| |
| |
| | Dale Henrichs wrote
| | >
| | > .. regarding having Metacello be a sentinel for sqs problems I'm
| | > less
| | > inclined to do anything special (uncoditionally checking for sqs
| | > being up
| | > penalizes folks who are not using sqs ...).
| | >
| | I was thinking something like stricter timeouts for the http
| | requests...
|
| Metacello doesn't touch the http server directly, in fact Metacello
| uses Gofer to load packages...
|
|
| |
| |
| | Dale Henrichs wrote
| | >
| | > With all of that said, moving forward with 1.0-beta.32, I could
| | > change the
| | > configuration for Metacello to directly reference the latam
| | > mirror
| | > for the
| | > dependent project configurations (could you submit an issue using
| | > the
| | > github list[1]?).
| | >
| | Worth a try... I'll submit an issue...
| |
| |
| | Dale Henrichs wrote
| | >
| | > Regarding bootstrap contents ... with 1.0-beta.32, I have created
| | > a
| | > 'batch' group that loads only the bare-minimum needed for loading
| | > configurations (no help, tutorial, etc.). And the bootstrap
| | > process
| | > no
| | > loads 'batch' ...
| | >
| | MetacelloBaseConfiguration>>ensureMetacello ->
| | "ConfigurationOfMetacello
| | load" -> "self project latestVersion load" = 'default'; should be
| | #load:
| | 'bootstrap', no?
|
| Yes, once the `latestVersion` is 1.0-beta.32...
|
|
| |
| |
| | Dale Henrichs wrote
| | >
| | > I'm going to add a `help` command[2] to the scripting api that
| | > will
| | > load
| | > the tutorial and help info, so I can probably afford to switch
| | > the
| | > 'default' group to 'batch' as well (could you add this note to
| | > your
| | > issue?).
| | >
| | I don't think it's bad having help loaded by default. Someone
| | loading
| | Metacello by itself may very well want to explore. I just don't
| | think
| | dev-related stuff belongs in the bootstrap.
|
| Yep, I agree.
|
|
| |
| | Sean
| |
| | b.t.w. this all started because I was trying to make a config for
| | MonticelloRedirect -> LatAm mirror, which bootstrapped Metacello,
| | which
| | tried to look on sqs...
|
| Bootstrapping the bootstrap is always fun ... you know if you load
| the LatAm mirror code first, then the bootstrap of Metacello will
| work correctly:)
|
|
|
| |
| | Thanks :)
| | Sean
| |
| | --
| | View this message in context:
| | http://forum.world.st/Metacello-Bootstrap-tp4635359p4635371.html
| | Sent from the Metacello mailing list archive at Nabble.com.
| |
|
|
|
| --
| Mariano
| http://marianopeck.wordpress.com
|
|

Mariano Martinez Peck wrote

#ensureMetacello should load the really minimal core needed... Then ConfigurationOfMetacello  'default' group can load whatevet you want.

I think you, me and Dale are aligned...

Dale, how do we change this while waiting for 1.0-beta.32? I'm thinking we should make an intermediate version between current stable and 1.0-beta.32, loading the same versions of each package, but using an updated baseline with the batch group and LatAm mirror. If that sounds right (and you can suggest a version number), I'll update the config...

Sean

call it 1.0-beta.31.1.3 (1.0-beta.31.2 has been burned)...

My inclination is to make the smallest change possible which would be using the LatAm mirror for external configs and not monkeying with the 'batch' group and changed default .. but since it's your nickel, go ahead and make the changes that you're comfortable with ... use the config from seaside.gemstone.com/ss/metacello, I'll synch up my GitHub version when you're done ...

Thanks man!

Dale

----- Original Message -----
| From: "Sean P. DeNigris" <[hidden email]>
| To: [hidden email]
| Sent: Monday, June 18, 2012 2:18:39 PM
| Subject: [Metacello] Re: Metacello Bootstrap
|
|
| Mariano Martinez Peck wrote
| >
| > #ensureMetacello should load the really minimal core needed... Then
| > ConfigurationOfMetacello  'default' group can load whatevet you
| > want.
| >
|
| I think you, me and Dale are aligned...
|
| Dale, how do we change this while waiting for 1.0-beta.32? I'm
| thinking we
| should make an intermediate version between current stable and
| 1.0-beta.32,
| loading the same versions of each package, but using an updated
| baseline
| with the batch group and LatAm mirror. If that sounds right (and you
| can
| suggest a version number), I'll update the config...
|
| Sean
|
| --
| View this message in context:
| http://forum.world.st/Metacello-Bootstrap-tp4635359p4635392.html
| Sent from the Metacello mailing list archive at Nabble.com.
|

Dale Henrichs wrote

call it 1.0-beta.31.1.3 (1.0-beta.31.2 has been burned)...

Okay, I'm almost done... quick question: why the group names 'bootstrap' and 'batch'? What does batch mean? And why 'bootstrap' if that's not the group we want to use for the bootstrap - could be confusing?

I'd like to have a logical group for the bootstrap (hint... hint... :)), so that if different dialects have different stable versions, the bootstrap group can be defined in each and do the right thing...

Thanks,
Sean

the 'batch' and 'bootstrap' groups are the same (or should be)... I used the name 'batch' instead of 'bootstrap' (not sure where that name came from) because the group is actually meant to be used whenever you are doing batch-style non-GUI based load processes like using Metacello in a continuous integration environment (Jenkins/travis-ci).

There are actually three phases of bootstrapping Metacello:

  The first phase loads the Gofer package.
  The second phase loads a couple of the basic packages (using Gofer), a
    skeleton load with enough of the system loaded to be able
    to load a Metacello configuration on all platforms - technically
    this would be the 'bootstrap' group, but Metacello isn't
    used to do the bootstrapping.
  The third phase loads the latest version of Metacello ('default' today
    'batch' tomorrow).

Only the third phase does an official Metacello load using groups and that third phase should use the 'batch' group. When done, you should have a standalone Metacello. That functions just fine ... so I don't think distinguishing between 'batch' and 'bootstrap' is necessary.

Besides, I've already started using 'batch' in a number of places in the 1.0-beta.32 code base.

So I would prefer to use the name 'batch' that covers both the 'bootstrapping' that you are focussed on and the more general "batch processing support":), which I've been fretting over...

Dale

----- Original Message -----
| From: "Sean P. DeNigris" <[hidden email]>
| To: [hidden email]
| Sent: Monday, June 18, 2012 5:51:16 PM
| Subject: [Metacello] Re: Metacello Bootstrap
|
|
| Dale Henrichs wrote
| >
| > call it 1.0-beta.31.1.3 (1.0-beta.31.2 has been burned)...
| >
|
| Okay, I'm almost done... quick question: why the group names
| 'bootstrap' and
| 'batch'? What does batch mean? And why 'bootstrap' if that's not the
| group
| we want to use for the bootstrap - could be confusing?
|
| I'd like to have a logical group for the bootstrap (hint... hint...
| :)), so
| that if different dialects have different stable versions, the
| bootstrap
| group can be defined in each and do the right thing...
|
| Thanks,
| Sean
|
| --
| View this message in context:
| http://forum.world.st/Metacello-Bootstrap-tp4635359p4635408.html
| Sent from the Metacello mailing list archive at Nabble.com.
|

Okay, check out these enhancements and see what you think...

Sean P. DeNigris wrote

Dale Henrichs wrote

With all of that said, moving forward with 1.0-beta.32, I could change the configuration for Metacello to directly reference the latam mirror for the dependent project configurations (could you submit an issue using the github list[1]?).

Worth a try... I'll submit an issue...

https://github.com/dalehenrich/metacello-work/issues/53
- updated to link back to attached files

n.b. my fixes are using the 'bootstrap' group, but rereading your post, it looks like you wanted me to use batch... well, take a look and confirm I'm on the right track for now...

Dale,

If you like what I did, lmk. Before you integrate, we have to update the version I created to include my changes to Metacello-Base...

Thanks,
Sean

will do ... I'm going to look at it when I get into the office this morning...

Dale

----- Original Message -----
| From: "Sean P. DeNigris" <[hidden email]>
| To: [hidden email]
| Sent: Tuesday, June 19, 2012 8:31:55 AM
| Subject: [Metacello] Re: Metacello Bootstrap
|
| Dale,
|
| If you like what I did, lmk. Before you integrate, we have to update
| the
| version I created to include my changes to Metacello-Base...
|
| Thanks,
| Sean
|
| --
| View this message in context:
| http://forum.world.st/Metacello-Bootstrap-tp4635359p4635506.html
| Sent from the Metacello mailing list archive at Nabble.com.
|

Sean,

In general things look okay ... the only dicey bit is in MetacelloBaseConfiguration class>>ensureMetacello: calls ConfigurationOfMetacello class>>ensureMetacello:, since the ConfigurationOfMetacello code runs through the bootstrap sequence again ...

I'd say either just do a direct version load at that point, or strip down the ensureDependencies code to just find ConfigurationOfMetacello and then forward the work to ConfigurationOfMetacello...

Dale

----- Original Message -----
| From: "Sean DeNigris" <[hidden email]>
| To: [hidden email]
| Sent: Monday, June 18, 2012 7:18:13 PM
| Subject: Re: [Metacello] Re: Metacello Bootstrap
|
| Okay, check out these enhancements and see what you think...
|
|
|
| Name: Metacello-Base-SeanDeNigris.20
| Enhance Metacello bootstrap (class>>ensureMetacello)
| * Load 'bootstrap' group of ConfigurationOfMetacello
| * Refactor into two steps
| - ensureDependencies handles Gofer and ConfigurationOfMetacello
| - delegate to ConfigurationOfMetacello>>ensureMetacello:
|
|
|
| Name: ConfigurationOfMetacello-SeanDeNigris.695
| * Add Version 1.0-beta.31.1.3
|
| - Add LatAm squeaksource mirror to project dependencies
| - 1.0-beta.31.1.3-baseline has bootstrap group, which is the minimum
| install for loading configs

Okay, I pushed the new config (ConfigurationOfMetacello-SeanDeNigris.695) to http://squeaksource.com/MetacelloRepository and http://seaside.gemstone.com/ss/metacello

The bootstrap should now only load the bare minimum to load configs (Metacello-{Base, Core, MC, Platform}).

If you do explicitly load dependent projects, it will also check the LatAm mirror for them if sqs is down...

Cheers,
Sean

Sean P. DeNigris wrote

The bootstrap should now only load the bare minimum to load configs (Metacello-{Base, Core, MC, Platform}).

Damn! So close, but not there yet. Now I remember why I was calling "ConfigurationOfMetacello class>>ensureMetacello:" and not #load

     ConfigurationOfMetacello class>>load
        "Load a minimum installation of Metacello"
        ^ self project latestVersion load

It seems that sending #project above correctly loads the 'batch' group. But... right after, #load loads the default group, which includes Help and Toolbox :(

Dale, should I change class>>load to "... load: 'batch': or change the 'default' group to = 'batch'. I'm leaning toward option #1, but whatever you think is best...

Sean