github baseline reference in ConfigurationOf + MetacelloPreview load issue?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

github baseline reference in ConfigurationOf + MetacelloPreview load issue?

Johan Brichau-3
Hi there,

When I add a reference to a baseline on github to a ConfigurationOfMyProject like this:

spec
 baseline: 'ProjectOnGithub'
 with: [  spec
                className: 'BaselineOfProjectOnGithub';
                repository: 'github://glassdb/projectongithub:master/repository' ]

I notice that once the github-cache directory has been created, the project will not be updated with the latest packages once the HEAD of the repository is different from the one pointed to.
So, although my ConfigurationOfMyProject explicitly refers to the HEAD of the master branch, it will not try to get the latest packages every time I do a load using the ConfigurationOfMyProject.

I only tried this on Gemstone GLASS 1.0-beta.9. So, the first question is: is this intended behavior? Should I do it in a different way?

I also tried downloading the latest MetacelloPreview too to check, but because my project references GLASS-1.0-beta.9 as a prerequisite, part of the Metacello packages get reloaded with newer versions:

These are the packages loaded with latest MetacelloPreview

--transcript--'Fetched -> Metacello-Base-dkh.101 --- github://dalehenrich/metacello-work:bdb13998bf2399ca7f9027c566d8e5ba9e837d3f/repository --- github://dalehenrich/metacello-work:bdb13998bf2399ca7f9027c566d8e5ba9e837d3f/repository'
--transcript--'Fetched -> Metacello-Core-dkh.489 --- github://dalehenrich/metacello-work:bdb13998bf2399ca7f9027c566d8e5ba9e837d3f/repository --- github://dalehenrich/metacello-work:bdb13998bf2399ca7f9027c566d8e5ba9e837d3f/repository'
--transcript--'Fetched -> Metacello-MC-dkh.662 --- github://dalehenrich/metacello-work:bdb13998bf2399ca7f9027c566d8e5ba9e837d3f/repository --- github://dalehenrich/metacello-work:bdb13998bf2399ca7f9027c566d8e5ba9e837d3f/repository'
--transcript--'Fetched -> Metacello-Platform.gemstone-dkh.26 --- github://dalehenrich/metacello-work:bdb13998bf2399ca7f9027c566d8e5ba9e837d3f/repository --- github://dalehenrich/metacello-work:bdb13998bf2399ca7f9027c566d8e5ba9e837d3f/repository'
--transcript--'Fetched -> Metacello-ToolBox-dkh.127 --- github://dalehenrich/metacello-work:bdb13998bf2399ca7f9027c566d8e5ba9e837d3f/repository --- github://dalehenrich/metacello-work:bdb13998bf2399ca7f9027c566d8e5ba9e837d3f/repository'

These packages get loaded again when I load my project. These versions are newer?

--transcript--'Fetched -> Metacello-Base-dkh.103 --- http://dev.yesplan.be/gsreplication --- http://dev.yesplan.be/gsreplication'
--transcript--'Fetched -> Metacello-Core-dkh.669 --- http://dev.yesplan.be/gsreplication --- http://dev.yesplan.be/gsreplication'
--transcript--'Fetched -> Metacello-MC-dkh.669 --- http://dev.yesplan.be/gsreplication --- http://dev.yesplan.be/gsreplication'
--transcript--'Fetched -> Metacello-Platform.gemstone-dkh.29 --- http://dev.yesplan.be/gsreplication --- http://dev.yesplan.be/gsreplication'
--transcript--'Fetched -> Metacello-ToolBox-dkh.131 --- http://dev.yesplan.be/gsreplication --- http://dev.yesplan.be/gsreplication'

Johan

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: github baseline reference in ConfigurationOf + MetacelloPreview load issue?

Dale Henrichs-3
Johan,

When you use a github repository, the full repository is downloaded as a zip file to your github cache and then unzipped ... Since it is expensive to download the whole repository I don't want Metacello downloading the whole repo everytime that the baseline is referenced ... especially when nothing may have changed in the remote repository (the whole download is required before I can tell whether anything has changed or not and this can make loads run very slow) ...

The second thing that is going on is that Metacello cannot tell by looking at the reference string  'github://glassdb/projectongithub:master/repository'  whether or not the string 'master' refers to a branch or a tag, so it is entirely possible that if 'master' is a tag, the contents of the repository will never change...

The third thing that is going on is that I want to give the developers full control over what is being loaded ... for example you know whether or not 'master' refers to a branch or a tag ... you also know whether or not you want to refresh the load from the github repo or not ... there are times when you may not want to get the latest version from the github repo (like when you have discovered there is a bug in a later version ... or are trying to reproduce a bug that depends upon the version that you are currently using ...


So ... when baselines are involved, the rules that Metacello follows are:

- when you load using a baseline and a version matching the specified repo 
  already exists in the github cache, then reuse the downloaded version.
- if you do a `get` on the baseline, the repo associated with the baseline is flushed 
  and when the next load using that baseline occurs, a fresh version of the github
  repo will be downloaded
- if you use the the MCFileBasedRepository class>>flushAllCaches command 
  (available from the Monticello Browser as 'flush cached versions"), the next load
  will of a github repo will download a fresh version ...

Therefore, if you want a fresh version downloaded every time you load, do a `get` on the baseline right before doing your load ...

I've got to run to a meeting so I'll address  the second half of your question laters...

Dale



On Mon, Feb 10, 2014 at 9:10 AM, Johan Brichau <[hidden email]> wrote:
Hi there,

When I add a reference to a baseline on github to a ConfigurationOfMyProject like this:

spec
 baseline: 'ProjectOnGithub'
 with: [  spec
                className: 'BaselineOfProjectOnGithub';
                repository: 'github://glassdb/projectongithub:master/repository' ]

I notice that once the github-cache directory has been created, the project will not be updated with the latest packages once the HEAD of the repository is different from the one pointed to.
So, although my ConfigurationOfMyProject explicitly refers to the HEAD of the master branch, it will not try to get the latest packages every time I do a load using the ConfigurationOfMyProject.

I only tried this on Gemstone GLASS 1.0-beta.9. So, the first question is: is this intended behavior? Should I do it in a different way?

I also tried downloading the latest MetacelloPreview too to check, but because my project references GLASS-1.0-beta.9 as a prerequisite, part of the Metacello packages get reloaded with newer versions:

These are the packages loaded with latest MetacelloPreview

--transcript--'Fetched -> Metacello-Base-dkh.101 --- github://dalehenrich/metacello-work:bdb13998bf2399ca7f9027c566d8e5ba9e837d3f/repository --- github://dalehenrich/metacello-work:bdb13998bf2399ca7f9027c566d8e5ba9e837d3f/repository'
--transcript--'Fetched -> Metacello-Core-dkh.489 --- github://dalehenrich/metacello-work:bdb13998bf2399ca7f9027c566d8e5ba9e837d3f/repository --- github://dalehenrich/metacello-work:bdb13998bf2399ca7f9027c566d8e5ba9e837d3f/repository'
--transcript--'Fetched -> Metacello-MC-dkh.662 --- github://dalehenrich/metacello-work:bdb13998bf2399ca7f9027c566d8e5ba9e837d3f/repository --- github://dalehenrich/metacello-work:bdb13998bf2399ca7f9027c566d8e5ba9e837d3f/repository'
--transcript--'Fetched -> Metacello-Platform.gemstone-dkh.26 --- github://dalehenrich/metacello-work:bdb13998bf2399ca7f9027c566d8e5ba9e837d3f/repository --- github://dalehenrich/metacello-work:bdb13998bf2399ca7f9027c566d8e5ba9e837d3f/repository'
--transcript--'Fetched -> Metacello-ToolBox-dkh.127 --- github://dalehenrich/metacello-work:bdb13998bf2399ca7f9027c566d8e5ba9e837d3f/repository --- github://dalehenrich/metacello-work:bdb13998bf2399ca7f9027c566d8e5ba9e837d3f/repository'

These packages get loaded again when I load my project. These versions are newer?

--transcript--'Fetched -> Metacello-Base-dkh.103 --- http://dev.yesplan.be/gsreplication --- http://dev.yesplan.be/gsreplication'
--transcript--'Fetched -> Metacello-Core-dkh.669 --- http://dev.yesplan.be/gsreplication --- http://dev.yesplan.be/gsreplication'
--transcript--'Fetched -> Metacello-MC-dkh.669 --- http://dev.yesplan.be/gsreplication --- http://dev.yesplan.be/gsreplication'
--transcript--'Fetched -> Metacello-Platform.gemstone-dkh.29 --- http://dev.yesplan.be/gsreplication --- http://dev.yesplan.be/gsreplication'
--transcript--'Fetched -> Metacello-ToolBox-dkh.131 --- http://dev.yesplan.be/gsreplication --- http://dev.yesplan.be/gsreplication'

Johan

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: github baseline reference in ConfigurationOf + MetacelloPreview load issue?

Johan Brichau-3

On 10 Feb 2014, at 19:30, Dale Henrichs <[hidden email]> wrote:

> Therefore, if you want a fresh version downloaded every time you load, do a `get` on the baseline right before doing your load ...

Hi Dale,

Thanks for the answer. It definitely is a reasonable approach but I'm wondering if there is a solution that makes referencing github repositories less different than referencing monticello repositories.

Would it be possible to store the SHA of the commit that is contained in the downloaded github.zip repo and use that to see if a newer commit exists on the repository using the github api? In that way, when a metacello configuration references the branch HEAD, it would be possible to automatically refresh the contents when a newer commit exists. Am I overseeing anything here?

If I want to stick to a certain version, and prevent automated updates, I can still reference the SHA explictly. In fact, I do that for all projects I reference on github. The situation arose when I was myself developing a project on github and wanted to always load the latest version of that through the ConfigurationOf of another project.

Given that a solution to refresh the referenced repositories exists, this is not an urgent issue.

cheers,
Johan

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: github baseline reference in ConfigurationOf + MetacelloPreview load issue?

Johan Brichau-3

On 10 Feb 2014, at 20:12, Johan Brichau <[hidden email]> wrote:

> Given that a solution to refresh the referenced repositories exists, this is not an urgent issue.

I wanted to say: if you think it's a good idea, make it an issue on the github issue tracker so it can be picked up later.
If not, dump the idea ;-)

Johan

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: github baseline reference in ConfigurationOf + MetacelloPreview load issue?

Dale Henrichs-3
Johan,

go ahead and "make an issue of it":) 

Once we can get Zodiac ported to GemStone, we could start using https to make the necessary Web API calls to github (bitbucket, etc.?) to get the sha for comparison although 

I still  don't like the idea that we'd be hitting the web (requiring that a web connection exists) during every load (probably multiple times) ... with git you are supposed to be able to work completely disconnected (and right now that is  true for github repos that have been pre-downloaded) ...

So ... I suppose we'd want to make this "refresh from web" a load-time option of some sort ... and if we make this a load-time decision, we can arrange to have the repo unconditionally refreshed (once per load) and not have to query the github web api .... 

Dale



On Mon, Feb 10, 2014 at 11:16 AM, Johan Brichau <[hidden email]> wrote:

On 10 Feb 2014, at 20:12, Johan Brichau <[hidden email]> wrote:

> Given that a solution to refresh the referenced repositories exists, this is not an urgent issue.

I wanted to say: if you think it's a good idea, make it an issue on the github issue tracker so it can be picked up later.
If not, dump the idea ;-)

Johan

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.