loading the baseline / configuration

Hi,

I'm trying to understand how I should be using the scripting API.

In the documentation [1] I read that
Metacello new configuration: 'Sample'; load
does the same work as
Gofer ... load (the config package itself) and ConfigurationOfSample
project ... load (the project spec).

When I have the configuration already loaded, why do I have to do a
get to update it? Why can't the load just do it?

I have to now write a script that looks like this:

(Smalltalk includesKey: 'ConfigurationOfSample')
  ifTrue: [
    Metacello new configuration: 'Sample'; get ].
Metacello new configuration: 'Sample'; load.

To me, this is ugly stuff. Using Gofer directly appears to be as
simple. Also, it is repetitive and silly that I have to create a new
Metacello object with the parameters, one with a get and one with a
load. Or am I missing it here?

Cheers
Otto

[1] https://github.com/dalehenrich/metacello-work/blob/master/docs/MetacelloScriptingAPI.md

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

Hi Otto,

For, the thought comes to mind that I would not want to discard the local changes to a configuration by an automatic get if I try to load the project.
So, the semantics would need to differ when there are local changes to the configuration class?
I’m in favour of the explicit get.

cheers
Johan

On 27 Dec 2013, at 08:22, Otto Behrens <[hidden email]> wrote:

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

> For, the thought comes to mind that I would not want to discard the local changes to a configuration by an automatic get if I try to load the project.
> So, the semantics would need to differ when there are local changes to the configuration class?
> I’m in favour of the explicit get.

But is the problem not the same with updating any package with changes
in the image? If you have local changes in the image, you can either
choose to ignore the warnings or if you are warned, choose not to
proceed. Can this not apply to baseline / configuration packages in
the same way as the packages that you are loading listed in the spec?

Normally when I develop, I don't want to leave the baseline /
configuration dirty and update the packages I am loading. Or perhaps
there's a workflow that I don't get. This is something that I seem to
misunderstand. I have a way of working / workflow that I regard as
good and it appears as if other people work in very different ways.
And I think Metacello tries to cater for all of it and that makes it
quite complex. Perhaps we should chat one day so I can see how you
work.

What would help me is to add a "proceed on package with unsaved
changes warnings" option to the scripting API.

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

Johan, Otto,

Johan has hit the nail on the head here ... The separate `get` is so that you can explicitly control when you want to refresh a configuration or baseline... the `get` will also flush the github repository cache ...

When you do the `get` command you are registering the project with Metacello and Metacello will endeavor to use the registered project at all times .... all of the on*: messages are driven off of the registration...so as such it is important for Metacello where you have decided that the configuration/baseline for a particular project will come from.

Dale


----- Original Message -----
| From: "Johan Brichau" <[hidden email]>
| To: [hidden email]
| Sent: Thursday, December 26, 2013 11:43:53 PM
| Subject: Re: [Metacello] loading the baseline / configuration
|
| Hi Otto,
|
| For, the thought comes to mind that I would not want to discard the
| local changes to a configuration by an automatic get if I try to
| load the project.
| So, the semantics would need to differ when there are local changes
| to the configuration class?
| I’m in favour of the explicit get.
|
| cheers
| Johan
|
| On 27 Dec 2013, at 08:22, Otto Behrens <[hidden email]> wrote:
|
| > Hi,
| >
| > I'm trying to understand how I should be using the scripting API.
| >
| > In the documentation [1] I read that
| > Metacello new configuration: 'Sample'; load
| > does the same work as
| > Gofer ... load (the config package itself) and
| > ConfigurationOfSample
| > project ... load (the project spec).
| >
| > When I have the configuration already loaded, why do I have to do a
| > get to update it? Why can't the load just do it?
| >
| > I have to now write a script that looks like this:
| >
| > (Smalltalk includesKey: 'ConfigurationOfSample')
| >  ifTrue: [
| >    Metacello new configuration: 'Sample'; get ].
| > Metacello new configuration: 'Sample'; load.
| >
| > To me, this is ugly stuff. Using Gofer directly appears to be as
| > simple. Also, it is repetitive and silly that I have to create a
| > new
| > Metacello object with the parameters, one with a get and one with a
| > load. Or am I missing it here?
| >
| > Cheers
| > Otto
| >
| > [1]
| > https://github.com/dalehenrich/metacello-work/blob/master/docs/MetacelloScriptingAPI.md
| >
| > --
| > You received this message because you are subscribed to the Google
| > Groups "Metacello" group.
| > To unsubscribe from this group and stop receiving emails from it,
| > send an email to [hidden email].
| > For more options, visit https://groups.google.com/groups/opt_out.
|
| --
| You received this message because you are subscribed to the Google
| Groups "Metacello" group.
| To unsubscribe from this group and stop receiving emails from it,
| send an email to [hidden email].
| For more options, visit https://groups.google.com/groups/opt_out.
|

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

----- Original Message -----
| From: "Otto Behrens" <[hidden email]>
| To: [hidden email]
| Sent: Thursday, December 26, 2013 11:22:59 PM
| Subject: [Metacello] loading the baseline / configuration
|
| Hi,
|
| I'm trying to understand how I should be using the scripting API.
|
| In the documentation [1] I read that
| Metacello new configuration: 'Sample'; load
| does the same work as
| Gofer ... load (the config package itself) and ConfigurationOfSample
| project ... load (the project spec).
|
| When I have the configuration already loaded, why do I have to do a
| get to update it? Why can't the load just do it?
Answer by Johan...
|
| I have to now write a script that looks like this:
|
| (Smalltalk includesKey: 'ConfigurationOfSample')
|   ifTrue: [
|     Metacello new configuration: 'Sample'; get ].
| Metacello new configuration: 'Sample'; load.
|
| To me, this is ugly stuff.

Ah, you have missed the fact (my poor documentation) that a `load` command _will_ automatically load a configuration/baseline if it is not present in the image ... In you above example it is only necessary to use the `load` command...

  Metacello new configuration: 'Sample'; repository: 'repo'; version: 'xxx'; load.

You need to supply the repo and version when supplying a configuration:) ... but no `get` is needed ... especially if you are only trying to load the configuration/baseline if it is not already loaded ...


| Using Gofer directly appears to be as
| simple. Also, it is repetitive and silly that I have to create a new
| Metacello object with the parameters, one with a get and one with a
| load. Or am I missing it here

I have toyed with simplifying things so that you could do this:

  Metacello new configuration: 'Sample'; get; load.

You are the first person to complain, so I will probably make this change:)

I have been holding off and holding off on officially releasing the Scripting API, because I have been patiently waiting for feedback like yours:)

Load scripts tend to get written one day and used for weeks/months so it hasn't been a priority to make the load script easy to write:)

Dale

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

LOL, I am totally daft. I'm sorry. I really don't get it! (Please be patient)

> | When I have the configuration already loaded, why do I have to do a
> | get to update it? Why can't the load just do it?
> Answer by Johan...

I don't understand the answer. Why not? Because I load the
configuration / baseline from the same place that I load the packages:
my local disk. So if I update (i.e. pull from git / copy from
squeaksource or where ever) the files on disk, I want to load the
configuration / baseline package from disk (the one that is there). If
I have changes in my baseline / configuration package in my image, I
can choose to ignore it or abort the load when the warning that I'm
going to overwrite something appears.

No, I got that from your documentation (which is ok!), and that is
exactly my point! So this is why it is funny, because I must be on
another planet.

In my script, I always want to load / update the configuration first,
whether it is already present or not. So I assumed I have to do a get
if it is present to that it is up to date. If it is not present, the
load will do it. That's why I wrote the script this way. Where am I
missing it?

>
>   Metacello new configuration: 'Sample'; repository: 'repo'; version: 'xxx'; load.
>
> You need to supply the repo and version when supplying a configuration:) ... but no `get` is needed ... especially if you are only trying to load the configuration/baseline if it is not already loaded ...

In this example, if it is already loaded and a new
ConfigurationOfSample is on my disk (where I load from), it won't be
loaded (unless I do a get first).

> | Using Gofer directly appears to be as
> | simple. Also, it is repetitive and silly that I have to create a new
> | Metacello object with the parameters, one with a get and one with a
> | load. Or am I missing it here
>
> I have toyed with simplifying things so that you could do this:
>
>   Metacello new configuration: 'Sample'; get; load.

Yes, I tried that, so the simplification would make sense to me. Will
it work if I do a get even if the project is not present?

> I have been holding off and holding off on officially releasing the Scripting API, because I have been patiently waiting for feedback like yours:)

Thanks, sorry I'm slow and such a pain. Perhaps our workflow article
will help (another email).

> Load scripts tend to get written one day and used for weeks/months so it hasn't been a priority to make the load script easy to write:)

Yes, true. So getting back to them after not looking at them for a
while and seeing a simple script helps. Also, we are 5 people working
on the system. If someone else has to work on the scripts, simplicity
and clarity will be a huge win.

Cheers
Otto

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

----- Original Message -----
| From: "Otto Behrens" <[hidden email]>
| To: [hidden email]
| Sent: Friday, December 27, 2013 12:12:29 AM
| Subject: Re: [Metacello] loading the baseline / configuration
|
| > For, the thought comes to mind that I would not want to discard the
| > local changes to a configuration by an automatic get if I try to
| > load the project.
| > So, the semantics would need to differ when there are local changes
| > to the configuration class?
| > I’m in favour of the explicit get.
|
| But is the problem not the same with updating any package with
| changes
| in the image? If you have local changes in the image, you can either
| choose to ignore the warnings or if you are warned, choose not to
| proceed. Can this not apply to baseline / configuration packages in
| the same way as the packages that you are loading listed in the spec?

I'm not sure what you are complaining about ... if you look at the `get` command, I am simply replacing the Gofer expression with an equivalent Metacello expression ... when you want to refresh a baseline/configuration, use the `get` command instead of Gofer.

At a superficial level they perform the same functionality, but at an api level, you now have a consistent interface. Since the scripting api remembers the project specifications, you can write the following:

  Metacello image baseline: 'xxx'; get.

To `get` all of the baselines updated you can do the following:

  Metacello image baseline: [:spec | true]; get.

These are "raw" capabilities that are built into the scripting api, but I need to get feedback from real users to determine the direction to take these "raw" capabilities.

So I appreciate the fact that you are taking a crack at using the API and casting a critical eye ...

By your comments above you are implying that you may want some sort of automatic configuration/baseline update feature? I recall in other emails that you have mentioned something along the same lines... Or am I reading more into this than necessary?

| Normally when I develop, I don't want to leave the baseline /
| configuration dirty and update the packages I am loading. Or perhaps
| there's a workflow that I don't get. This is something that I seem to
| misunderstand. I have a way of working / workflow that I regard as
| good and it appears as if other people work in very different ways.
| And I think Metacello tries to cater for all of it and that makes it
| quite complex. Perhaps we should chat one day so I can see how you
| work.

This is all about workflow and I would really appreciate explicit descriptions of the various workflows ... When you say "Metacello tries to cater for all of it and that makes it quite complex" you make a good point ... The scripting api is an api and not a workflow ... I expect workflows to be built on top of the scripting api ...

I have been saying over and over that I want feedback, because I need to understand how the scripting api can bes t fit the needs of disparate workflows ...

With my work in tODE, I have created diff/commit/load scripts build on top of Metacello and I have found that these scripts are slightly different depending upon which project I am contributing to. for the projects that I own I have a particular style that I can enforce. For projects that I participate in, I have to conform to the style chosen by the maintainers and that forces me to do things differenly ... with tODE I encode these differences in scripts that I run to support the different workflows ... so from my experience the scripts are the most important thing ... once I encode the workflow I can forget about the details ... but when I am developing the scripts the details are important and it is important that a given level of support is available ...

|
| What would help me is to add a "proceed on package with unsaved
| changes warnings" option to the scripting API.

There is already a MetacelloSkipDirtyPackageLoad, so such an option can be added:)

Dale

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

> Johan has hit the nail on the head here ... The separate `get` is so that you can explicitly control when you want to refresh a configuration or baseline... the `get` will also flush the github repository cache ...

Ah, ok. I think the main issue is that I have a different workflow. I
control when I want to refresh a configuration or baseline by copying
the configuration from somewhere onto my file system or pulling /
checking out a specific / more recent commit from a git repo. I've
actually tried to "disable" the github-cache by creating a read only
directory with that name! (I wish I can do the same with
package-cache)

> When you do the `get` command you are registering the project with Metacello and Metacello will endeavor to use the registered project at all times .... all of the on*: messages are driven off of the registration...so as such it is important for Metacello where you have decided that the configuration/baseline for a particular project will come from.

I think I get what you're doing. But I tell Metacello where I want to
load the project from *every time* I invoke it. Because I invoke it
from a script. I actually still don't get why I need to persist the
information in a registry. Don't let this comment get under your skin
- it is probably due to my lack of understanding.

Cheers
Otto

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

>> Johan has hit the nail on the head here ... The separate `get` is so that you can explicitly control when you want to refresh a configuration or baseline... the `get` will also flush the github repository cache ...
>
> Ah, ok. I think the main issue is that I have a different workflow. I
> control when I want to refresh a configuration or baseline by copying
> the configuration from somewhere onto my file system or pulling /
> checking out a specific / more recent commit from a git repo. I've
> actually tried to "disable" the github-cache by creating a read only
> directory with that name! (I wish I can do the same with
> package-cache)

I just want to add to this that at this point, the file system is
changed and not my image. I then use Metacello to load the changes
from disk into my image. If the image is fresh, some of the
configurations / baselines may not be loaded. I want to load them and
the packages in the spec. If it is reasonably up to date, I just want
to refresh the configs / baselines that changed and load the packages
in the spec.

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

Otto,

Our workflows are more equal than you may think :-)
We also keep a local copy for all repositories we use, except for the git repositories (at this point). There are two reasons for us to keep local copies: independence of third party services for a build and protection from unwanted changes in packages/configurations that might suddenly sneak in when someone updates the #stable symbolic version somewhere.
Metacello is used to load and update the project in as follows:

Metacello new
        configuration: 'Sample';
        version: #released
  repositoryOverrides: listOfLocalRepositories;
        get;
        load: 'Core'

This will always fetch the latest version of the configuration and load it. Wether it was loaded in the image or not.
We also use the conflict handlers, but I leave them out here.
Also: I understood from a previous discussion that the repository overrides do not work for github repositories.
Logically speaking, they are not necessary because a certain commit hash will never change, but for the independence on github for building, it is definitely useful.

For development, if I am testing the configuration, I leave out the 'get'.

So, I'm not seeing the problem you have with just always sending get and load? It seems that fits your workflow as well.

Sorry if I misunderstood, I'm feeling a bit under the weather for the moment so my mind might not have caught up...

Johan



On 27 Dec 2013, at 18:51, Otto Behrens <[hidden email]> wrote:

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

----- Original Message -----
| From: "Otto Behrens" <[hidden email]>
| To: [hidden email]
| Sent: Friday, December 27, 2013 9:37:05 AM
| Subject: Re: [Metacello] loading the baseline / configuration
|
| LOL, I am totally daft. I'm sorry. I really don't get it! (Please be
| patient)
|
| > | When I have the configuration already loaded, why do I have to do
| > | a
| > | get to update it? Why can't the load just do it?
| > Answer by Johan...
|
| I don't understand the answer. Why not? Because I load the
| configuration / baseline from the same place that I load the
| packages:
| my local disk. So if I update (i.e. pull from git / copy from
| squeaksource or where ever) the files on disk, I want to load the
| configuration / baseline package from disk (the one that is there).

So this is an `update` step in your workflow. When you do an `update`, you refresh the local disk copies and you then want to refresh the version that is loaded in your image (or images?).

| If
| I have changes in my baseline / configuration package in my image, I
| can choose to ignore it or abort the load when the warning that I'm
| going to overwrite something appears.

So when you update your image, you want to make a configuration by configuration, package by package decision whether or not you want to override your configuraton/baseline/packages?

I mentioned in my previous email I inferred that an `onDirty:` option could be added and I assume that this option would work for you.

But I also get the feeling that one of the things that you want is to be able to have the configuration/basline automatically loaded when you do a load? So you don't have to put a separate `get` statement into your script?

My incliniation is to allow the overloading of a Matacello statement to allow both a `get` and `load` like the following:

  Metacello new baseline: 'xxx'; repository: 'yyy'; get; load.

Would that work better for you?



| > |
| > | I have to now write a script that looks like this:
| > |
| > | (Smalltalk includesKey: 'ConfigurationOfSample')
| > |   ifTrue: [
| > |     Metacello new configuration: 'Sample'; get ].
| > | Metacello new configuration: 'Sample'; load.
| > |
| > | To me, this is ugly stuff.
| >
| > Ah, you have missed the fact (my poor documentation) that a `load`
| > command _will_ automatically load a configuration/baseline if it
| > is not present in the image ... In you above example it is only
| > necessary to use the `load` command...
|
| No, I got that from your documentation (which is ok!), and that is
| exactly my point! So this is why it is funny, because I must be on
| another planet.
|
| In my script, I always want to load / update the configuration first,
| whether it is already present or not. So I assumed I have to do a get
| if it is present to that it is up to date. If it is not present, the
| load will do it. That's why I wrote the script this way. Where am I
| missing it?

Haha, I misread your code ... in your case, does it hurt to unconditionally do a `get`? It sounds like that is exactly what you want ... load a fresh copy of the configuration before loading the code...
|
| >
| >   Metacello new configuration: 'Sample'; repository: 'repo';
| >   version: 'xxx'; load.
| >
| > You need to supply the repo and version when supplying a
| > configuration:) ... but no `get` is needed ... especially if you
| > are only trying to load the configuration/baseline if it is not
| > already loaded ...
|
| In this example, if it is already loaded and a new
| ConfigurationOfSample is on my disk (where I load from), it won't be
| loaded (unless I do a get first).
|
| > | Using Gofer directly appears to be as
| > | simple. Also, it is repetitive and silly that I have to create a
| > | new
| > | Metacello object with the parameters, one with a get and one with
| > | a
| > | load. Or am I missing it here
| >
| > I have toyed with simplifying things so that you could do this:
| >
| >   Metacello new configuration: 'Sample'; get; load.
|
| Yes, I tried that, so the simplification would make sense to me. Will
| it work if I do a get even if the project is not present?

Yes, `get` always loads the configuration/baseline from disk whether or not it is present in the image.

|
| > I have been holding off and holding off on officially releasing the
| > Scripting API, because I have been patiently waiting for feedback
| > like yours:)
|
| Thanks, sorry I'm slow and such a pain. Perhaps our workflow article
| will help (another email).

Don't be sorry and you are not a pain ... I am very happy that I am getting feedback!

It's not easy to work through these types of issues remotely ... if we were in the same room we could yell at each other for a few minutes and then once we scrapped and scrambled to get to the "same page" we'd be off to the races:)

I hope that you remain patient with me as I come to understand your workflow... I will tell you up front that we'll go  down a couple more dead ends before we find common ground, but I think that the scripting api will be better for the effort ... at least I hope it will:)

|
| > Load scripts tend to get written one day and used for weeks/months
| > so it hasn't been a priority to make the load script easy to
| > write:)
|
| Yes, true. So getting back to them after not looking at them for a
| while and seeing a simple script helps. Also, we are 5 people working
| on the system. If someone else has to work on the scripts, simplicity
| and clarity will be a huge win.

Agreed ... right now there are gaps in the functionality and gaps in the simplicity/clarity that need to be filled ... with patience and peseverence we'll get there:)

Dale

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

oeps... chaining effectively does not work... sorry about that. I am indeed loading the configuration first using gofer...

So...  Dale,I guess that would be the thing to add to Metacello scripting api ;-) ?

Johan

On 27 Dec 2013, at 19:10, Johan Brichau <[hidden email]> wrote:

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

----- Original Message -----
| From: "Otto Behrens" <[hidden email]>
| To: [hidden email]
| Sent: Friday, December 27, 2013 9:51:18 AM
| Subject: Re: [Metacello] loading the baseline / configuration
|
| > Johan has hit the nail on the head here ... The separate `get` is
| > so that you can explicitly control when you want to refresh a
| > configuration or baseline... the `get` will also flush the github
| > repository cache ...
|
| Ah, ok. I think the main issue is that I have a different workflow. I
| control when I want to refresh a configuration or baseline by copying
| the configuration from somewhere onto my file system or pulling /
| checking out a specific / more recent commit from a git repo. I've
| actually tried to "disable" the github-cache by creating a read only
| directory with that name! (I wish I can do the same with
| package-cache)

Let's take some time and look at the github-cache a bit more. I'd like to get feedback from you to refine the way that it works.

By default, the github-cache is updated on demand. I try to avoid hitting github as much as possible. The intent is that the github-cache is not updated unless:

  - there is no entry for the project in the cache
  - you have done a `get` command for the baseline of the project
  - you have done a Monticello Repository flush.

So managing the github-cache was one of the functions tied to the `get` command.

I am curious aobut the use case where you felt it was necessary to further "disable' the github-cache. Perhaps you were using the 'get' command or perhaps there is a bug?

|
| > When you do the `get` command you are registering the project with
| > Metacello and Metacello will endeavor to use the registered
| > project at all times .... all of the on*: messages are driven off
| > of the registration...so as such it is important for Metacello
| > where you have decided that the configuration/baseline for a
| > particular project will come from.
|
| I think I get what you're doing. But I tell Metacello where I want to
| load the project from *every time* I invoke it. Because I invoke it
| from a script. I actually still don't get why I need to persist the
| information in a registry. Don't let this comment get under your skin
| - it is probably due to my lack of understanding.

The registry exists for a couple of reasons, but the main reason it was invented was to eliminate the need to calculate which version of a project is loaded (using #currentVersion). With the registry the version of the project that _is_ loaded is recorded, then whenever Metacello needs to know the "current" version it looks the project up in the registry ...

Dale

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

> I'm not sure what you are complaining about ... if you look at the `get` command, I am simply replacing the Gofer expression with an equivalent Metacello expression ... when you want to refresh a baseline/configuration, use the `get` command instead of Gofer.
>
> At a superficial level they perform the same functionality, but at an api level, you now have a consistent interface. Since the scripting api remembers the project specifications, you can write the following:

The bit that confused me was that a load implies a get, but only if
the baseline was not in the image yet. That felt like complex
semantics. Or I don't get the semantics.


>   Metacello image baseline: 'xxx'; get.
>
> To `get` all of the baselines updated you can do the following:
>
>   Metacello image baseline: [:spec | true]; get.
>
> These are "raw" capabilities that are built into the scripting api, but I need to get feedback from real users to determine the direction to take these "raw" capabilities.

Nice capabilities. I don't see where I could use it though. This is
because I have a script that loads the code from disk. And I would
just call that. (via my World menu item called 'load', or via a ruby
script called 'deploy' in our environment).

> So I appreciate the fact that you are taking a crack at using the API and casting a critical eye ...

I am renowned for being critical. So I'm enjoying someone asking for
it :-) It may be a bit taxing for you though. But please, the context
is that it's meant to help.

> By your comments above you are implying that you may want some sort of automatic configuration/baseline update feature? I recall in other emails that you have mentioned something along the same lines... Or am I reading more into this than necessary?

Automatic to me is that when I want to load new code in from disk, I
want the baselines / configs to be updated as well.

> This is all about workflow and I would really appreciate explicit descriptions of the various workflows ... When you say "Metacello tries to cater for all of it and that makes it quite complex" you make a good point ... The scripting api is an api and not a workflow ... I expect workflows to be built on top of the scripting api ...

Yes. What would be nice is if there is a close mapping between what
the scripting api provides and a workflow that we implement. I did
write up something about our workflows here:
http://finworks.biz/development-workflow-at-finworks/

> With my work in tODE, I have created diff/commit/load scripts build on top of Metacello and I have found that these scripts are slightly different depending upon which project I am contributing to. for the projects that I own I have a particular style that I can enforce. For projects that I participate in, I have to conform to the style chosen by the maintainers and that forces me to do things differenly ... with tODE I encode these differences in scripts that I run to support the different workflows ... so from my experience the scripts are the most important thing ... once I encode the workflow I can forget about the details ... but when I am developing the scripts the details are important and it is important that a given level of support is available ...

Interesting, yes. I've only really worked on the metacello-work git
repo recently. I did not have to change my workflow to do that. And I
don't see that I'd have to change anything in the way that I work for
the other cloned git repos either.

It would be very interesting to understand how these projects differ.
May be a good writeup.

> There is already a MetacelloSkipDirtyPackageLoad, so such an option can be added:)

Yes, that would be nice

Thanks

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

> So this is an `update` step in your workflow. When you do an `update`, you refresh the local disk copies and you then want to refresh the version that is loaded in your image (or images?).

I update the local disk copies by doing a git pull (which could
merge). I then have to resolve merge issues and merge commit. Or I
edit files directly on disk.

The second step is to refresh / load into the image / images, yes.

> | If
> | I have changes in my baseline / configuration package in my image, I
> | can choose to ignore it or abort the load when the warning that I'm
> | going to overwrite something appears.
>
> So when you update your image, you want to make a configuration by configuration, package by package decision whether or not you want to override your configuraton/baseline/packages?

No, not really. It is all or nothing. If I have made changes in the
image, I want to either save it or discard it. I don't differentiate
here if the changes where in a baseline / configuration or in another
package.

> I mentioned in my previous email I inferred that an `onDirty:` option could be added and I assume that this option would work for you.

Yes, would be nice.

> But I also get the feeling that one of the things that you want is to be able to have the configuration/basline automatically loaded when you do a load? So you don't have to put a separate `get` statement into your script?
>
> My incliniation is to allow the overloading of a Matacello statement to allow both a `get` and `load` like the following:
>
>   Metacello new baseline: 'xxx'; repository: 'yyy'; get; load.
>
> Would that work better for you?

Yes, it would, thanks. I would just always do the get; load for all
packages. That's ok. That's my 'semantics'.

> Haha, I misread your code ... in your case, does it hurt to unconditionally do a `get`? It sounds like that is exactly what you want ... load a fresh copy of the configuration before loading the code...

Yes, I'll just always do a get. (But why does a load do a get only if
the baseline / config is not there yet? - this is the bit I did not
understand.)

I tried that previously, but it broke on filetree.

And this opened another can of worms: bootstrapping correctly. I had a
version of Metacello bootstrapped on which the get broke when I tried
to load filetree, which is a dependency of metacello itself, if I load
metacello from the filtree repo.

> It's not easy to work through these types of issues remotely ... if we were in the same room we could yell at each other for a few minutes and then once we scrapped and scrambled to get to the "same page" we'd be off to the races:)

We could try skype / google+ some day.

> I hope that you remain patient with me as I come to understand your workflow... I will tell you up front that we'll go  down a couple more dead ends before we find common ground, but I think that the scripting api will be better for the effort ... at least I hope it will:)

I believe in the idea that more input can result in a better product.
It's harder work (the pain we're going through now). But the result
will be better!

Cheers
Otto

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

> Our workflows are more equal than you may think :-)

Glad to hear our workflow is not *that* strange. It was a bit lonely
out there for a while. haha.

> We also keep a local copy for all repositories we use, except for the git repositories (at this point). There are two reasons for us to keep local copies: independence of third party services for a build and protection from unwanted changes in packages/configurations that might suddenly sneak in when someone updates the #stable symbolic version somewhere.

Yes, exactly, same reasons here! We keep the git repositories as well,
as submodules.

> Metacello new
>         configuration: 'Sample';
>         version: #released
>         repositoryOverrides: listOfLocalRepositories;
>         get;
>         load: 'Core'
>
> This will always fetch the latest version of the configuration and load it. Wether it was loaded in the image or not.

Ok. (I saw your oeps that the cascading on the same metacello object
does not work. And I replied on another mail saying that I'd like this
to work.)

I just had a problem with doing a get on FileTree if
BaselineOfFileTree is not loaded yet, and I think that's because I do
not have a version bootstrapped when I try to load a more recent
version. Will look at that later.

> We also use the conflict handlers, but I leave them out here.

Yes, also have some.

> Also: I understood from a previous discussion that the repository overrides do not work for github repositories.
> Logically speaking, they are not necessary because a certain commit hash will never change, but for the independence on github for building, it is definitely useful.

We use submodules to get the independence. But we found submodules
need their own TLC (tender loving care). Some more work to improve on
that needed...

We found that with all the builds / loads / pulls we do during a day,
even github breaks. So we tried to reduce the number of pulls from
github. git submodule update appears to work (i.e. not create
unnecessary pulls) because it looks at the commit checked out on the
submodule.

> For development, if I am testing the configuration, I leave out the 'get'.

Ah, ok. Are you doing this often? Or is this not perhaps the exceptional case?

> So, I'm not seeing the problem you have with just always sending get and load? It seems that fits your workflow as well.

I can always do a get and load. That's ok. My protest is not all that
significant; it is more about taste and how I would prefer to see an
API.

To me, the normal case is to load from disk and update my image
(including the baseline / config). So I expected load to do that. And
it does that, but only if the baseline / config is not present.

I also do not understand why you guys have to make an exception for
the baseline / config package, where you have a modification in the
image and you load. I understand that in the case where you are
testing your config / baseline, you don't want it overwritten. But in
that case, I'd prefer something like:

Metacello new
        configuration: 'Sample';
        skipDirtyPackageLoad;
        load

where my interpretation / intention with the message 'load' here is
that it does a get as well (i.e. loads ConfigurationOfSample).

Alternatively, a more explicit version of that workflow would be:

Metacello new
        configuration: 'Sample';
        skipConfigurationPackage;
        load

> Sorry if I misunderstood, I'm feeling a bit under the weather for the moment so my mind might not have caught up...

I don't think you misunderstood, I understand better now. Hope you're
better, hope my comments make sense.

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

----- Original Message -----
| From: "Otto Behrens" <[hidden email]>
| To: [hidden email]
| Sent: Saturday, December 28, 2013 11:41:26 PM
| Subject: Re: [Metacello] loading the baseline / configuration
|
| > I'm not sure what you are complaining about ... if you look at the
| > `get` command, I am simply replacing the Gofer expression with an
| > equivalent Metacello expression ... when you want to refresh a
| > baseline/configuration, use the `get` command instead of Gofer.
| >
| > At a superficial level they perform the same functionality, but at
| > an api level, you now have a consistent interface. Since the
| > scripting api remembers the project specifications, you can write
| > the following:
|
| The bit that confused me was that a load implies a get, but only if
| the baseline was not in the image yet. That felt like complex
| semantics. Or I don't get the semantics.

Several forces come into play:

The `load` command was the first idea ... combine the `Gofer...load` and `Configuration...load` into a single command. Just specify the configuration name, version, repository and load list in one command:

  Metacello new
    configuration: 'Seaside3';
    version: #stable;
    repository: 'http://smalltalkhub.com/mc/Seaside/MetacelloConfigurations/main';
    load.

... and Bob's your uncle.

It wasn't too long before I found that I wanted to look at the configuration without doing the load and the `get` command was invented...just leave off the version:

  Metacello new
    configuration: 'Seaside3';
    repository: 'http://smalltalkhub.com/mc/Seaside/MetacelloConfigurations/main';
    get.

In parallel to this work I was endeavoring to eliminate the #currentVersion hack from Metacello ...

In the initial versions of Metacello, I explicitly avoided saving any state at all ... if Metacello needed to know which version of a project was loaded, Metacello would look at the image, see what package versions and dependent projects were actually loaded in the image and "calculate the current version" ... this work reasonably well, but for large projects with lots of dependencies their were a lot of cycles wasted calculating the current version for projects plus the current version calculation were pretty sensitive to the version of the configuration that was loaded in the image and sometimes Metacello would get the calculation wrong ... for the most part the load would end up in the right spot, but every once in awhile, the current version calculation was wrong enough that the wrong thing would be loaded ....

My solution to this was to simply record in the image the exact version of each project that was loaded ... if you loaded Seaside 3.0.5, then I would  record that version in the registry and if Metacello ever needed to know what version was loaded, it would look in the registry and that was it ...

Also in parallel, I wanted to give developers more control over the decisions that Metacello was making during a load ... Metacello would always upgrade a project version and would never downgrade a project version ... in most cases this was okay, but when you didn't want to automatically upgrade a project there was nothing you could do ... so I put in the onUpgrade: and onDowngrade: commands giving you monitor _and_ control the decisions that Metacello was making ...

It was at about this point in the process where I decided that automagically loading the configuration whenever you did a `load` was a bad idea. I wanted the developer to have explicit control over what was going on so I stamped out yet another magical decision and put the control in the hands of the developer with the simple expedient of saying "if the developer wants a fresh copy of the configuration, she can do a `get`" ...

BUT, because I wanted to keep the simplicity of the original single `load` command, I kept the sematics of the `load` command will load a configuration if it is not already in the image ...

I really think that it is important that developers have complete control over all of the decisions that Metacello is making and I really think that I have achieved this ... however, I need to round off some of the sharp corners.

If I am reading your comments correctly, I understand that you would deem the following would work for you:

  Metacello new
    configuration: 'Seaside3';
    repository: 'http://smalltalkhub.com/mc/Seaside/MetacelloConfigurations/main';
    get;
    load

Johan also agrees that this would be useful ... so I will go ahead and add this feature[1] to the Metacello Preview.

[1] https://github.com/dalehenrich/metacello-work/issues/230

|
| >   Metacello image baseline: 'xxx'; get.
| >
| > To `get` all of the baselines updated you can do the following:
| >
| >   Metacello image baseline: [:spec | true]; get.
| >
| > These are "raw" capabilities that are built into the scripting api,
| > but I need to get feedback from real users to determine the
| > direction to take these "raw" capabilities.
|
| Nice capabilities. I don't see where I could use it though. This is
| because I have a script that loads the code from disk. And I would
| just call that. (via my World menu item called 'load', or via a ruby
| script called 'deploy' in our environment).

Yeah, these commands are a bit more useful in tools than in static scripts ...

|
| > So I appreciate the fact that you are taking a crack at using the
| > API and casting a critical eye ...
|
| I am renowned for being critical. So I'm enjoying someone asking for
| it :-) It may be a bit taxing for you though. But please, the context
| is that it's meant to help.
|
| > By your comments above you are implying that you may want some sort
| > of automatic configuration/baseline update feature? I recall in
| > other emails that you have mentioned something along the same
| > lines... Or am I reading more into this than necessary?
|
| Automatic to me is that when I want to load new code in from disk, I
| want the baselines / configs to be updated as well.
|
| > This is all about workflow and I would really appreciate explicit
| > descriptions of the various workflows ... When you say "Metacello
| > tries to cater for all of it and that makes it quite complex" you
| > make a good point ... The scripting api is an api and not a
| > workflow ... I expect workflows to be built on top of the
| > scripting api ...
|
| Yes. What would be nice is if there is a close mapping between what
| the scripting api provides and a workflow that we implement. I did
| write up something about our workflows here:
| http://finworks.biz/development-workflow-at-finworks/
|
| > With my work in tODE, I have created diff/commit/load scripts build
| > on top of Metacello and I have found that these scripts are
| > slightly different depending upon which project I am contributing
| > to. for the projects that I own I have a particular style that I
| > can enforce. For projects that I participate in, I have to conform
| > to the style chosen by the maintainers and that forces me to do
| > things differenly ... with tODE I encode these differences in
| > scripts that I run to support the different workflows ... so from
| > my experience the scripts are the most important thing ... once I
| > encode the workflow I can forget about the details ... but when I
| > am developing the scripts the details are important and it is
| > important that a given level of support is available ...
|
| Interesting, yes. I've only really worked on the metacello-work git
| repo recently. I did not have to change my workflow to do that. And I
| don't see that I'd have to change anything in the way that I work for
| the other cloned git repos either.

By it's very nature, git-based projects would fit nicely with your workflow ... you want to have ownership of all of the projects that you depend upon and that is exactly what git gives you. Things get tricky when you have to work with Monticello-based projects ...

You've "addressed" this by copying all o the configurations and projects to your local disk, but if we're honest about this the solutions that we have to use for the Monticello repositories are not any where near as elegant as those for git-based projects ...

In fact I think we still have some outstanding issues for dealing with Monticello-based repositories ...

|
| It would be very interesting to understand how these projects differ.
| May be a good writeup.

The workflow for git using a BaselineOf is very simple:

  - save package(s) to disk,
  - do git commit
  - done

With a Monticello repository using a ConfigurationOf things are slightly more complicated:

  - save package(s) to disk
  - if no #development version exists in config add new Metacello version
  - for each package saved to disk, update package version in config
  - done

It's the extra maintenance overhead of managing the package versions in the configuration that causes the headaches.

|
| > There is already a MetacelloSkipDirtyPackageLoad, so such an option
| > can be added:)
|
| Yes, that would be nice

So then adding and onDirty: command[2] would make the API complete, as far as your load/update workflow is concerned? The onDirty: would look something like:

  Metacello new
    configuration: 'Seaside3';
    repository: 'http://smalltalkhub.com/mc/Seaside/MetacelloConfigurations/main';
    onDirty: [:ex | ex load ]; "or `ex skip`?"
    get;
    load.

Dale

[2] https://github.com/dalehenrich/metacello-work/issues/231

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

> If I am reading your comments correctly, I understand that you would deem the following would work for you:
>
>   Metacello new
>     configuration: 'Seaside3';
>     repository: 'http://smalltalkhub.com/mc/Seaside/MetacelloConfigurations/main';
>     get;
>     load

Yes, that's fine

> So then adding and onDirty: command[2] would make the API complete, as far as your load/update workflow is concerned? The onDirty: would look something like:
>
>   Metacello new
>     configuration: 'Seaside3';
>     repository: 'http://smalltalkhub.com/mc/Seaside/MetacelloConfigurations/main';
>     onDirty: [:ex | ex load ]; "or `ex skip`?"
>     get;
>     load.

Yes, that can work nicely.

Thanks

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.