Saving to local git and "Loading all file names from http://...pharo5/inbox"

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Saving to local git and "Loading all file names from http://...pharo5/inbox"

Holger Freyther
Hi,

every time I save a local package using gitfiletree:// it tries to download from the pharo5 inbox. Is this to be expected? I do not have the inbox associated with that package though? Can the version number resolving be changed?

kind regards
        holger
Reply | Threaded
Open this post in threaded view
|

Re: Saving to local git and "Loading all file names from http://...pharo5/inbox"

Peter Uhnák
On Sat, May 28, 2016 at 10:48:47PM +0200, Holger Freyther wrote:
> Hi,
>
> every time I save a local package using gitfiletree:// it tries to download from the pharo5 inbox. Is this to be expected? I do not have the inbox associated with that package though? Can the version number resolving be changed?

For some reason the package manager is refreshing all packages. I don't know why it happens, and it's quite annoying (because it slows down commits), but it doesn't cause any actual problems, so don't worry about it too much.

Peter

>
> kind regards
> holger

Reply | Threaded
Open this post in threaded view
|

Re: Saving to local git and "Loading all file names from http://...pharo5/inbox"

Sven Van Caekenberghe-2

> On 29 May 2016, at 09:51, Peter Uhnak <[hidden email]> wrote:
>
> On Sat, May 28, 2016 at 10:48:47PM +0200, Holger Freyther wrote:
>> Hi,
>>
>> every time I save a local package using gitfiletree:// it tries to download from the pharo5 inbox. Is this to be expected? I do not have the inbox associated with that package though? Can the version number resolving be changed?
>
> For some reason the package manager is refreshing all packages. I don't know why it happens, and it's quite annoying (because it slows down commits), but it doesn't cause any actual problems, so don't worry about it too much.

As I understand it, what happens is the following: before you commit to your MC repo, you have to find the next version number; a check is then done in all relevant repos; the cached content is not used, but an actual refresh is done. All this is so that my .5 would not conflict with someone else .5 - the chance that this happens is very small, and the check does not really prevent it.

> Peter
>
>>
>> kind regards
>> holger
>


Reply | Threaded
Open this post in threaded view
|

Re: Saving to local git and "Loading all file names from http://...pharo5/inbox"

Holger Freyther

> On 29 May 2016, at 09:58, Sven Van Caekenberghe <[hidden email]> wrote:
>
>
>>
>> For some reason the package manager is refreshing all packages. I don't know why it happens, and it's quite annoying (because it slows down commits), but it doesn't cause any actual problems, so don't worry about it too much.
>
> As I understand it, what happens is the following: before you commit to your MC repo, you have to find the next version number; a check is then done in all relevant repos; the cached content is not used, but an actual refresh is done. All this is so that my .5 would not conflict with someone else .5 - the chance that this happens is very small, and the check does not really prevent it.

I assumed that but can it be limited to the Repositories that are associated with the package? I am afraid that next time I travel I can not commit to my local repository (and ofcourse the speed part). :)

holger
Reply | Threaded
Open this post in threaded view
|

Re: Saving to local git and "Loading all file names from http://...pharo5/inbox"

Sven Van Caekenberghe-2

> On 29 May 2016, at 10:28, Holger Freyther <[hidden email]> wrote:
>
>
>> On 29 May 2016, at 09:58, Sven Van Caekenberghe <[hidden email]> wrote:
>>
>>
>>>
>>> For some reason the package manager is refreshing all packages. I don't know why it happens, and it's quite annoying (because it slows down commits), but it doesn't cause any actual problems, so don't worry about it too much.
>>
>> As I understand it, what happens is the following: before you commit to your MC repo, you have to find the next version number; a check is then done in all relevant repos; the cached content is not used, but an actual refresh is done. All this is so that my .5 would not conflict with someone else .5 - the chance that this happens is very small, and the check does not really prevent it.
>
> I assumed that but can it be limited to the Repositories that are associated with the package? I am afraid that next time I travel I can not commit to my local repository (and ofcourse the speed part). :)

We should go one step further: add an option to totally disable this check to go outside the target repo, it makes little sense. But MC is a complex beast ...

> holger


Reply | Threaded
Open this post in threaded view
|

Re: Saving to local git and "Loading all file names from http://...pharo5/inbox"

Peter Uhnák
All this is so that my .5 would not conflict with someone else .5

How is this a problem? Because it will be "Me.5" and "You.5", so there can't be any conflict.

Peter

On Sun, May 29, 2016 at 10:37 AM, Sven Van Caekenberghe <[hidden email]> wrote:

> On 29 May 2016, at 10:28, Holger Freyther <[hidden email]> wrote:
>
>
>> On 29 May 2016, at 09:58, Sven Van Caekenberghe <[hidden email]> wrote:
>>
>>
>>>
>>> For some reason the package manager is refreshing all packages. I don't know why it happens, and it's quite annoying (because it slows down commits), but it doesn't cause any actual problems, so don't worry about it too much.
>>
>> As I understand it, what happens is the following: before you commit to your MC repo, you have to find the next version number; a check is then done in all relevant repos; the cached content is not used, but an actual refresh is done. All this is so that my .5 would not conflict with someone else .5 - the chance that this happens is very small, and the check does not really prevent it.
>
> I assumed that but can it be limited to the Repositories that are associated with the package? I am afraid that next time I travel I can not commit to my local repository (and ofcourse the speed part). :)

We should go one step further: add an option to totally disable this check to go outside the target repo, it makes little sense. But MC is a complex beast ...

> holger



Reply | Threaded
Open this post in threaded view
|

Re: Saving to local git and "Loading all file names from http://...pharo5/inbox"

Thierry Goubier
Le 29/05/2016 11:15, Peter Uhnák a écrit :
>  > All this is so that my .5 would not conflict with someone else .5
>
> How is this a problem? Because it will be "Me.5" and "You.5", so there
> can't be any conflict.

Me.5 and Me.5 are possible...

Think of numbering your own stuff on two different branches.

More, Metacello depends on Me.5 5 to be greater than You.5 5 for some of
the baselines / configurations to work properly :)

Thierry

>
> Peter
>
> On Sun, May 29, 2016 at 10:37 AM, Sven Van Caekenberghe <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>
>     > On 29 May 2016, at 10:28, Holger Freyther <[hidden email] <mailto:[hidden email]>> wrote:
>     >
>     >
>     >> On 29 May 2016, at 09:58, Sven Van Caekenberghe <[hidden email] <mailto:[hidden email]>> wrote:
>     >>
>     >>
>     >>>
>     >>> For some reason the package manager is refreshing all packages. I don't know why it happens, and it's quite annoying (because it slows down commits), but it doesn't cause any actual problems, so don't worry about it too much.
>     >>
>     >> As I understand it, what happens is the following: before you commit to your MC repo, you have to find the next version number; a check is then done in all relevant repos; the cached content is not used, but an actual refresh is done. All this is so that my .5 would not conflict with someone else .5 - the chance that this happens is very small, and the check does not really prevent it.
>     >
>     > I assumed that but can it be limited to the Repositories that are associated with the package? I am afraid that next time I travel I can not commit to my local repository (and ofcourse the speed part). :)
>
>     We should go one step further: add an option to totally disable this
>     check to go outside the target repo, it makes little sense. But MC
>     is a complex beast ...
>
>      > holger
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Saving to local git and "Loading all file names from http://...pharo5/inbox"

Peter Uhnák
On Sun, May 29, 2016 at 11:21:21AM +0200, Thierry Goubier wrote:

> Le 29/05/2016 11:15, Peter Uhnák a écrit :
> >  > All this is so that my .5 would not conflict with someone else .5
> >
> > How is this a problem? Because it will be "Me.5" and "You.5", so there
> > can't be any conflict.
>
> Me.5 and Me.5 are possible...
>
> Think of numbering your own stuff on two different branches.
>
> More, Metacello depends on Me.5 5 to be greater than You.5 5 for some of the
> baselines / configurations to work properly :)

Can't it use the ancestry to decide?

Because my impression is from this that merges are naive, compared to
git which actually checks the ancestry of both up the common endpoint
and diffs based on that.

Peter

>
> Thierry
>
> >
> > Peter
> >
> > On Sun, May 29, 2016 at 10:37 AM, Sven Van Caekenberghe <[hidden email]
> > <mailto:[hidden email]>> wrote:
> >
> >
> >     > On 29 May 2016, at 10:28, Holger Freyther <[hidden email] <mailto:[hidden email]>> wrote:
> >     >
> >     >
> >     >> On 29 May 2016, at 09:58, Sven Van Caekenberghe <[hidden email] <mailto:[hidden email]>> wrote:
> >     >>
> >     >>
> >     >>>
> >     >>> For some reason the package manager is refreshing all packages. I don't know why it happens, and it's quite annoying (because it slows down commits), but it doesn't cause any actual problems, so don't worry about it too much.
> >     >>
> >     >> As I understand it, what happens is the following: before you commit to your MC repo, you have to find the next version number; a check is then done in all relevant repos; the cached content is not used, but an actual refresh is done. All this is so that my .5 would not conflict with someone else .5 - the chance that this happens is very small, and the check does not really prevent it.
> >     >
> >     > I assumed that but can it be limited to the Repositories that are associated with the package? I am afraid that next time I travel I can not commit to my local repository (and ofcourse the speed part). :)
> >
> >     We should go one step further: add an option to totally disable this
> >     check to go outside the target repo, it makes little sense. But MC
> >     is a complex beast ...
> >
> >      > holger
> >
> >
> >
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Saving to local git and "Loading all file names from http://...pharo5/inbox"

Thierry Goubier
Le 29/05/2016 14:04, Peter Uhnak a écrit :

> On Sun, May 29, 2016 at 11:21:21AM +0200, Thierry Goubier wrote:
>> Le 29/05/2016 11:15, Peter Uhnák a écrit :
>>>   > All this is so that my .5 would not conflict with someone else .5
>>>
>>> How is this a problem? Because it will be "Me.5" and "You.5", so there
>>> can't be any conflict.
>>
>> Me.5 and Me.5 are possible...
>>
>> Think of numbering your own stuff on two different branches.
>>
>> More, Metacello depends on Me.5 5 to be greater than You.5 5 for some of the
>> baselines / configurations to work properly :)
>
> Can't it use the ancestry to decide?

I suggested that to Dale... Disregard version numbers, only consider as
newer if the other is in the ancestry chain.

Would also be a lot more precise, because we would then use the package
version UID instead of the name.version number approximation which can
leads to failures or loading the wrong package in some cases.

> Because my impression is from this that merges are naive, compared to
> git which actually checks the ancestry of both up the common endpoint
> and diffs based on that.

This does not describe the Monticello merges. Monticello merges do walk
the ancestry chain to find a common ancestor and does a three way merge,
as git does.

The problem is that the Monticello ancestry chain is often damaged for
three reasons: removing part of it to reduce memory usage (Done for the
Pharo3 release), removing all or part of it because your package
ancestry chain is too long (I remember seeing some packages doing that
on purpose?), the common ancestor doesn't exist anymore in the
repositories visible to Monticello (long-lived packages that have moved
between repositories, typically).

When Monticello sees a version in the history, it expects the full
package to be available in the repositories; if this package is the
common ancestor of the merge and can't be retrieved, then the merge will
fail.

What GitFileTree/git does there is to ensure that if a version is in the
ancestry, it can be retrieved. Git merge also use that property. Any
GitFileTree-like layer would provide the same property (FossilFileTree,
for example).

Note: in the git documentation, it is stated that git may create virtual
ancestors in some cases, to simplify/reduce the merge work. In such
cases, if set up, the gitfiletree merge driver will be called for all
those merges (i.e. a single merge in git may trigger multiple merges).

Obvious, isn't it ;)

Thierry

> Peter
>
>>
>> Thierry
>>
>>>
>>> Peter
>>>
>>> On Sun, May 29, 2016 at 10:37 AM, Sven Van Caekenberghe <[hidden email]
>>> <mailto:[hidden email]>> wrote:
>>>
>>>
>>>      > On 29 May 2016, at 10:28, Holger Freyther <[hidden email] <mailto:[hidden email]>> wrote:
>>>      >
>>>      >
>>>      >> On 29 May 2016, at 09:58, Sven Van Caekenberghe <[hidden email] <mailto:[hidden email]>> wrote:
>>>      >>
>>>      >>
>>>      >>>
>>>      >>> For some reason the package manager is refreshing all packages. I don't know why it happens, and it's quite annoying (because it slows down commits), but it doesn't cause any actual problems, so don't worry about it too much.
>>>      >>
>>>      >> As I understand it, what happens is the following: before you commit to your MC repo, you have to find the next version number; a check is then done in all relevant repos; the cached content is not used, but an actual refresh is done. All this is so that my .5 would not conflict with someone else .5 - the chance that this happens is very small, and the check does not really prevent it.
>>>      >
>>>      > I assumed that but can it be limited to the Repositories that are associated with the package? I am afraid that next time I travel I can not commit to my local repository (and ofcourse the speed part). :)
>>>
>>>      We should go one step further: add an option to totally disable this
>>>      check to go outside the target repo, it makes little sense. But MC
>>>      is a complex beast ...
>>>
>>>       > holger
>>>
>>>
>>>
>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Saving to local git and "Loading all file names from http://...pharo5/inbox"

stepharo
In reply to this post by Sven Van Caekenberghe-2

>>>> For some reason the package manager is refreshing all packages. I don't know why it happens, and it's quite annoying (because it slows down commits), but it doesn't cause any actual problems, so don't worry about it too much.
>>> As I understand it, what happens is the following: before you commit to your MC repo, you have to find the next version number; a check is then done in all relevant repos; the cached content is not used, but an actual refresh is done. All this is so that my .5 would not conflict with someone else .5 - the chance that this happens is very small, and the check does not really prevent it.
>> I assumed that but can it be limited to the Repositories that are associated with the package? I am afraid that next time I travel I can not commit to my local repository (and ofcourse the speed part). :)
> We should go one step further: add an option to totally disable this check to go outside the target repo,

It would be cool because you imagine when we integrate a change :)

>   it makes little sense. But MC is a complex beast ...
>
>> holger
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Saving to local git and "Loading all file names from http://...pharo5/inbox"

Peter Uhnák
In reply to this post by Thierry Goubier
I don't see anything actionable in this thread.
What can we do to move this forward? Is it only up to Dale to change the
behavior?

Can we have some workaround? E.g. syncing in the background at regular
intervals so when I want to commit I am not blocked by it?

Because it doesn't always check the repos, and I don't see what is the
actual trigger.

Peter

On Sun, May 29, 2016 at 02:28:48PM +0200, Thierry Goubier wrote:

> Le 29/05/2016 14:04, Peter Uhnak a écrit :
> > On Sun, May 29, 2016 at 11:21:21AM +0200, Thierry Goubier wrote:
> > > Le 29/05/2016 11:15, Peter Uhnák a écrit :
> > > >   > All this is so that my .5 would not conflict with someone else .5
> > > >
> > > > How is this a problem? Because it will be "Me.5" and "You.5", so there
> > > > can't be any conflict.
> > >
> > > Me.5 and Me.5 are possible...
> > >
> > > Think of numbering your own stuff on two different branches.
> > >
> > > More, Metacello depends on Me.5 5 to be greater than You.5 5 for some of the
> > > baselines / configurations to work properly :)
> >
> > Can't it use the ancestry to decide?
>
> I suggested that to Dale... Disregard version numbers, only consider as
> newer if the other is in the ancestry chain.
>
> Would also be a lot more precise, because we would then use the package
> version UID instead of the name.version number approximation which can leads
> to failures or loading the wrong package in some cases.
>
> > Because my impression is from this that merges are naive, compared to
> > git which actually checks the ancestry of both up the common endpoint
> > and diffs based on that.
>
> This does not describe the Monticello merges. Monticello merges do walk the
> ancestry chain to find a common ancestor and does a three way merge, as git
> does.
>
> The problem is that the Monticello ancestry chain is often damaged for three
> reasons: removing part of it to reduce memory usage (Done for the Pharo3
> release), removing all or part of it because your package ancestry chain is
> too long (I remember seeing some packages doing that on purpose?), the
> common ancestor doesn't exist anymore in the repositories visible to
> Monticello (long-lived packages that have moved between repositories,
> typically).
>
> When Monticello sees a version in the history, it expects the full package
> to be available in the repositories; if this package is the common ancestor
> of the merge and can't be retrieved, then the merge will fail.
>
> What GitFileTree/git does there is to ensure that if a version is in the
> ancestry, it can be retrieved. Git merge also use that property. Any
> GitFileTree-like layer would provide the same property (FossilFileTree, for
> example).
>
> Note: in the git documentation, it is stated that git may create virtual
> ancestors in some cases, to simplify/reduce the merge work. In such cases,
> if set up, the gitfiletree merge driver will be called for all those merges
> (i.e. a single merge in git may trigger multiple merges).
>
> Obvious, isn't it ;)
>
> Thierry
>
> > Peter
> >
> > >
> > > Thierry
> > >
> > > >
> > > > Peter
> > > >
> > > > On Sun, May 29, 2016 at 10:37 AM, Sven Van Caekenberghe <[hidden email]
> > > > <mailto:[hidden email]>> wrote:
> > > >
> > > >
> > > >      > On 29 May 2016, at 10:28, Holger Freyther <[hidden email] <mailto:[hidden email]>> wrote:
> > > >      >
> > > >      >
> > > >      >> On 29 May 2016, at 09:58, Sven Van Caekenberghe <[hidden email] <mailto:[hidden email]>> wrote:
> > > >      >>
> > > >      >>
> > > >      >>>
> > > >      >>> For some reason the package manager is refreshing all packages. I don't know why it happens, and it's quite annoying (because it slows down commits), but it doesn't cause any actual problems, so don't worry about it too much.
> > > >      >>
> > > >      >> As I understand it, what happens is the following: before you commit to your MC repo, you have to find the next version number; a check is then done in all relevant repos; the cached content is not used, but an actual refresh is done. All this is so that my .5 would not conflict with someone else .5 - the chance that this happens is very small, and the check does not really prevent it.
> > > >      >
> > > >      > I assumed that but can it be limited to the Repositories that are associated with the package? I am afraid that next time I travel I can not commit to my local repository (and ofcourse the speed part). :)
> > > >
> > > >      We should go one step further: add an option to totally disable this
> > > >      check to go outside the target repo, it makes little sense. But MC
> > > >      is a complex beast ...
> > > >
> > > >       > holger
> > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Saving to local git and "Loading all file names from http://...pharo5/inbox"

Dale Henrichs-3
In reply to this post by Thierry Goubier


On 5/29/16 5:28 AM, Thierry Goubier wrote:

> Le 29/05/2016 14:04, Peter Uhnak a écrit :
>> On Sun, May 29, 2016 at 11:21:21AM +0200, Thierry Goubier wrote:
>>> Le 29/05/2016 11:15, Peter Uhnák a écrit :
>>>>   > All this is so that my .5 would not conflict with someone else .5
>>>>
>>>> How is this a problem? Because it will be "Me.5" and "You.5", so there
>>>> can't be any conflict.
>>>
>>> Me.5 and Me.5 are possible...
>>>
>>> Think of numbering your own stuff on two different branches.
>>>
>>> More, Metacello depends on Me.5 5 to be greater than You.5 5 for
>>> some of the
>>> baselines / configurations to work properly :)
>>
>> Can't it use the ancestry to decide?
>
> I suggested that to Dale... Disregard version numbers, only consider
> as newer if the other is in the ancestry chain.
For FileTree repos it is even better to just load the package
(Monticello does a good job of only loading the changed packages and
we're looking at packages already on disk) ...  ... I have an extension
to Metacello for Filetree repos (it's only been around for about 3
years) that ignores version numbers and "loads" all of the packages from
the the repo (I mentioned _this_ to Thierry:) ... which for FileTree
repos is actually the proper behavior ... I happen to be very busy at
the moment so I don't have time to blow the dust off of that extension
... the big gotcha is everyone has to have the extension present in
their image for this kind of thing to work and AFAIK pharo is not kept
up-to-date with the latest versions that I publish on github, so I'm
afraid that much breakage would ensure if only a few individuals started
using this extension ... Seems to be that we're stuck between a rock and
a had place... I wish there was a better way to keep the Metacello
distribution up-to-date in images that I have no control over ...

FWIW, I've found that if I manually flush the cache for each package
before I save it, I don't have to wait the minute or so for all of the
repos to be refreshed ...

Dale

Reply | Threaded
Open this post in threaded view
|

Re: Saving to local git and "Loading all file names from http://...pharo5/inbox"

Peter Uhnák
On Wed, Jun 01, 2016 at 10:28:12PM -0700, Dale Henrichs wrote:
>
> FWIW, I've found that if I manually flush the cache for each package before
> I save it, I don't have to wait the minute or so for all of the repos to be
> refreshed ...
>

How can I do that?

Peter

Reply | Threaded
Open this post in threaded view
|

Re: Saving to local git and "Loading all file names from http://...pharo5/inbox"

otto
In reply to this post by Dale Henrichs-3
> For FileTree repos it is even better to just load the package (Monticello
> does a good job of only loading the changed packages and we're looking at
> packages already on disk) ...  ... I have an extension to Metacello for
> Filetree repos (it's only been around for about 3 years) that ignores
> version numbers and "loads" all of the packages from the the repo (I
> mentioned _this_ to Thierry:) ... which for FileTree repos is actually the
> proper behavior ... I happen to be very busy at the moment so I don't have
> time to blow the dust off of that extension ... the big gotcha is everyone
> has to have the extension present in their image for this kind of thing to
> work and AFAIK pharo is not kept up-to-date with the latest versions that I
> publish on github, so I'm afraid that much breakage would ensure if only a
> few individuals started using this extension ... Seems to be that we're
> stuck between a rock and a had place... I wish there was a better way to
> keep the Metacello distribution up-to-date in images that I have no control
> over ...
>
> FWIW, I've found that if I manually flush the cache for each package before
> I save it, I don't have to wait the minute or so for all of the repos to be
> refreshed ...

Dale, will you please publish this stuff somewhere, with dust on, as
is? (Or tell us where we can find it).

We've just battled sooo much in this area. I wonder if it is worth it
to think again about this whole git / monticello / metacello thing.
For me, it's just too much to comprehend. As soon as I dig into
something here, I get lost within minutes.

I believe the complexity has to do with trying to maintain too many
different ways of solving the problem. We are trying to keep things
(Monticello / Metacello) working with or without git or filetree or
whatever else people are using.

I would really like to do merging in git and just have a fast and
simple way of loading / compiling into an image (and GemStone). I do
not want a package cache, 3 way merging and all the other things,
because it makes what I end up with unpredictable. But you see, other
people want to work in other ways, which I understand. My perception
is that the result is messy.

We should try to strive more towards *fewer* ways of doing things,
without killing innovation to try different ways. It feels as if there
are enormous, yet very incomplete silos of efforts to solve problems.

Reply | Threaded
Open this post in threaded view
|

Re: Saving to local git and "Loading all file names from http://...pharo5/inbox"

Dale Henrichs-3
In reply to this post by Peter Uhnák
In the Monticello Browser, select the repository that you will be saving
your package to and use the "Flush cached versions" menu item ... it
seems that you have to do it before each package save...

Dale

On 6/3/16 12:13 AM, Peter Uhnak wrote:
> On Wed, Jun 01, 2016 at 10:28:12PM -0700, Dale Henrichs wrote:
>> FWIW, I've found that if I manually flush the cache for each package before
>> I save it, I don't have to wait the minute or so for all of the repos to be
>> refreshed ...
>>
> How can I do that?
>
> Peter
>


Reply | Threaded
Open this post in threaded view
|

Re: Saving to local git and "Loading all file names from http://...pharo5/inbox"

Dale Henrichs-3
In reply to this post by otto



On 6/3/16 2:43 AM, Otto Behrens wrote:
For FileTree repos it is even better to just load the package (Monticello
does a good job of only loading the changed packages and we're looking at
packages already on disk) ...  ... I have an extension to Metacello for
Filetree repos (it's only been around for about 3 years) that ignores
version numbers and "loads" all of the packages from the the repo (I
mentioned _this_ to Thierry:) ... which for FileTree repos is actually the
proper behavior ... I happen to be very busy at the moment so I don't have
time to blow the dust off of that extension ... the big gotcha is everyone
has to have the extension present in their image for this kind of thing to
work and AFAIK pharo is not kept up-to-date with the latest versions that I
publish on github, so I'm afraid that much breakage would ensure if only a
few individuals started using this extension ... Seems to be that we're
stuck between a rock and a had place... I wish there was a better way to
keep the Metacello distribution up-to-date in images that I have no control
over ...

FWIW, I've found that if I manually flush the cache for each package before
I save it, I don't have to wait the minute or so for all of the repos to be
refreshed ...
Dale, will you please publish this stuff somewhere, with dust on, as
is? (Or tell us where we can find it).
Well let's see there's the Metacello-Cypress package is in the Metacello repo[1] and the following should load the Cypress support into an image with Metacello already loaded:

  Metacello image
    baseline: 'Metacello';
    load: ' Cypress'

To use the Cypress support, implement a #projectClass method in your BaselineOf that returns MetacelloCypressBaselineProject:

  projectClass
    ^MetacelloCypressBaselineProject

and you should be in business ...

IIRC the MetacelloCypressBaselineProject should be usable with any BaselineOf. The only difference for Cypress is that Monticello versions are ignored when making loading decisions --- at least that's the intent.

Before going full scale, we'd need to add some tests for the Cypress project and of course make sure that the Cypress support gets loaded by default ...

[1] https://github.com/dalehenrich/metacello-work/tree/master/repository/Metacello-Cypress.package
We've just battled sooo much in this area. I wonder if it is worth it
to think again about this whole git / monticello / metacello thing.
For me, it's just too much to comprehend. As soon as I dig into
something here, I get lost within minutes.

I believe the complexity has to do with trying to maintain too many
different ways of solving the problem. We are trying to keep things
(Monticello / Metacello) working with or without git or filetree or
whatever else people are using.

I would really like to do merging in git and just have a fast and
simple way of loading / compiling into an image (and GemStone). I do
not want a package cache, 3 way merging and all the other things,
because it makes what I end up with unpredictable. But you see, other
people want to work in other ways, which I understand. My perception
is that the result is messy.

We should try to strive more towards *fewer* ways of doing things,
without killing innovation to try different ways. It feels as if there
are enormous, yet very incomplete silos of efforts to solve problems.

I basically agree with everything you say here.

I added the Cypress package support several years ago when Richard Sargent wrote a Cypress implementation for the base GemStone[2], so we've had the bits a pieces necessary for a cleaner simpler mechanism for a while at least for GemStone ...

I would like to distinguish between Cypress and FileTree.

FileTree was created (by you:) to support disk-based repos while maintaining full Monticello compatibility. Over the years I've (stubbornly) maintained that the Monticello meta data be preserved in FileTree repositories. By preserving Monticello meta data, the packages can be easily moved between FileTree and SmalltalkHub, etc. while faithfully preserving the Monticello ancestry. Maintaining Monticello meta data is critical for folks to be able to "dip their toe into the git pool" while preserving Monticello ancestry.

When I say "Cypress" (as opposed to FileTree), I think of a stripped down implementation of FileTree where Monticello meta data is not stored on disk and, perhaps more importantly the package loading code, in-image package management and tool support is also stripped down to support a simple model where package history and version numbers are non-existent. This particular model is aimed at projects that spend their entire lives in "git" repos and there is no desire, nor need to keep a parallel ancestry in the Monticello meta data and I think that what you are asking for ... in fact with Cypress package names do not need any package meta data ... the SHA of the commit is all that is needed to uniquely identify packages [in tODE the development environment is focused on the Metacello project and the package is not really the focus, so right now the Metacello registry is capable of tracking the SHA of the commit used to load a _project_, so I don't think that a package needs to have any additional meta data like UUID or version number or and the SHA of the commit at _project_ load time is all that is needed]

For Pharo, I think that GitFileTree provides you with support for a metadata-less disk footprint, but the in-image support is still based on Monticello --- at least that's what I understand about GitFileTree, since up to now I haven't spent any time working with it ... that will change this summer as I am working on a project in Pharo5. GitFileTree is an extension of the "FileTree model" in that it is compatible with the current in-image tools that all pretty much expect Monticello packages --- this approach is needed because it is a big job to incorporate an alternate source code management scheme in the in-image tools across the board ... If one were to switch the pharo tool focus from the _package_ (Monticello Browser) to the the _project_ (Project Browser) there would be additional work to be done: 'save' of project means save all dirty packages in project as well as a `git commit`, 'diff' of project means show changes for all dirty packages in project, and there would be analogs for 'load', 'revert', 'history' etc.

For GemStone, I have written tODE to be git-aware from the beginning and I have built tools to support Cypress package management as well[3] although they _are_ 3 years old and would deserve additional work ... Internally we are seriously considering adding Cypress support to the base image for the GemStone 3.4 release and of course in the process of doing that work, I assume that we'd make the code available for use in 3.2 and 3.3 as well.

In the short term I think that a combination of Cypress package support from Metacello, GitFileTree in Pharo and the updating of the Cypress tools for tODE will allow you to share packages between Pharo and GemStone without needing to create and maintain Monticello meta data ... in the midterm Cypress support is slated for this year and if necessary I can make changes to the current tODE FileTree implementation to allow the writing of metadata-less packages -- I think GitFileTree records whether or not a repository is metadata-less in a repository property file that I can use if necessary ...

Does this make sense?

Dale

[2] https://github.com/rjsargent/CypressReferenceImplementation
[3] https://github.com/dalehenrich/tode/tree/master/repository/Topez-Server-CypressTools.package

Reply | Threaded
Open this post in threaded view
|

Re: Saving to local git and "Loading all file names from http://...pharo5/inbox"

Peter Uhnák
As this is a real nuisance, I might have a partial solution — remove the repos :)

Apart from contributing to pharo (which is done in clean images) I never needed reference to those repositories, so I don't see why I would need them there.

So a startup script to the rescue

StartupPreferencesLoader default executeAtomicItems: {
StartupAction
name: 'remove references to Pharo50 and Pharo50Inbox'
code: [

|rx group|
rx := '([0-9]{5})' asRegex.
(rx matchesPrefix: Smalltalk image shortImageName) ifFalse: [
group := MCRepositoryGroup default.
group repositories detect: [ :each |
each location includesSubstring: 'Pharo/Pharo50Inbox/main'
] ifFound: [ :repo |
group removeRepository: repo
].

"This repo is added twice for whatever reason (once main and once main/ or whatever."
group repositories detect: [ :each |
each location includesSubstring: 'Pharo/Pharo50/main'
] ifFound: [ :repo |
group removeRepository: repo
].

group repositories detect: [ :each |
each location includesSubstring: 'Pharo/Pharo50/main'
] ifFound: [ :repo |
group removeRepository: repo
]
]

]
runOnce: true.
}

Peter

On Fri, Jun 3, 2016 at 8:49 PM, Dale Henrichs <[hidden email]> wrote:



On 6/3/16 2:43 AM, Otto Behrens wrote:
For FileTree repos it is even better to just load the package (Monticello
does a good job of only loading the changed packages and we're looking at
packages already on disk) ...  ... I have an extension to Metacello for
Filetree repos (it's only been around for about 3 years) that ignores
version numbers and "loads" all of the packages from the the repo (I
mentioned _this_ to Thierry:) ... which for FileTree repos is actually the
proper behavior ... I happen to be very busy at the moment so I don't have
time to blow the dust off of that extension ... the big gotcha is everyone
has to have the extension present in their image for this kind of thing to
work and AFAIK pharo is not kept up-to-date with the latest versions that I
publish on github, so I'm afraid that much breakage would ensure if only a
few individuals started using this extension ... Seems to be that we're
stuck between a rock and a had place... I wish there was a better way to
keep the Metacello distribution up-to-date in images that I have no control
over ...

FWIW, I've found that if I manually flush the cache for each package before
I save it, I don't have to wait the minute or so for all of the repos to be
refreshed ...
Dale, will you please publish this stuff somewhere, with dust on, as
is? (Or tell us where we can find it).
Well let's see there's the Metacello-Cypress package is in the Metacello repo[1] and the following should load the Cypress support into an image with Metacello already loaded:

  Metacello image
    baseline: 'Metacello';
    load: ' Cypress'

To use the Cypress support, implement a #projectClass method in your BaselineOf that returns MetacelloCypressBaselineProject:

  projectClass
    ^MetacelloCypressBaselineProject

and you should be in business ...

IIRC the MetacelloCypressBaselineProject should be usable with any BaselineOf. The only difference for Cypress is that Monticello versions are ignored when making loading decisions --- at least that's the intent.

Before going full scale, we'd need to add some tests for the Cypress project and of course make sure that the Cypress support gets loaded by default ...

[1] https://github.com/dalehenrich/metacello-work/tree/master/repository/Metacello-Cypress.package
We've just battled sooo much in this area. I wonder if it is worth it
to think again about this whole git / monticello / metacello thing.
For me, it's just too much to comprehend. As soon as I dig into
something here, I get lost within minutes.

I believe the complexity has to do with trying to maintain too many
different ways of solving the problem. We are trying to keep things
(Monticello / Metacello) working with or without git or filetree or
whatever else people are using.

I would really like to do merging in git and just have a fast and
simple way of loading / compiling into an image (and GemStone). I do
not want a package cache, 3 way merging and all the other things,
because it makes what I end up with unpredictable. But you see, other
people want to work in other ways, which I understand. My perception
is that the result is messy.

We should try to strive more towards *fewer* ways of doing things,
without killing innovation to try different ways. It feels as if there
are enormous, yet very incomplete silos of efforts to solve problems.

I basically agree with everything you say here.

I added the Cypress package support several years ago when Richard Sargent wrote a Cypress implementation for the base GemStone[2], so we've had the bits a pieces necessary for a cleaner simpler mechanism for a while at least for GemStone ...

I would like to distinguish between Cypress and FileTree.

FileTree was created (by you:) to support disk-based repos while maintaining full Monticello compatibility. Over the years I've (stubbornly) maintained that the Monticello meta data be preserved in FileTree repositories. By preserving Monticello meta data, the packages can be easily moved between FileTree and SmalltalkHub, etc. while faithfully preserving the Monticello ancestry. Maintaining Monticello meta data is critical for folks to be able to "dip their toe into the git pool" while preserving Monticello ancestry.

When I say "Cypress" (as opposed to FileTree), I think of a stripped down implementation of FileTree where Monticello meta data is not stored on disk and, perhaps more importantly the package loading code, in-image package management and tool support is also stripped down to support a simple model where package history and version numbers are non-existent. This particular model is aimed at projects that spend their entire lives in "git" repos and there is no desire, nor need to keep a parallel ancestry in the Monticello meta data and I think that what you are asking for ... in fact with Cypress package names do not need any package meta data ... the SHA of the commit is all that is needed to uniquely identify packages [in tODE the development environment is focused on the Metacello project and the package is not really the focus, so right now the Metacello registry is capable of tracking the SHA of the commit used to load a _project_, so I don't think that a package needs to have any additional meta data like UUID or version number or and the SHA of the commit at _project_ load time is all that is needed]

For Pharo, I think that GitFileTree provides you with support for a metadata-less disk footprint, but the in-image support is still based on Monticello --- at least that's what I understand about GitFileTree, since up to now I haven't spent any time working with it ... that will change this summer as I am working on a project in Pharo5. GitFileTree is an extension of the "FileTree model" in that it is compatible with the current in-image tools that all pretty much expect Monticello packages --- this approach is needed because it is a big job to incorporate an alternate source code management scheme in the in-image tools across the board ... If one were to switch the pharo tool focus from the _package_ (Monticello Browser) to the the _project_ (Project Browser) there would be additional work to be done: 'save' of project means save all dirty packages in project as well as a `git commit`, 'diff' of project means show changes for all dirty packages in project, and there would be analogs for 'load', 'revert', 'history' etc.

For GemStone, I have written tODE to be git-aware from the beginning and I have built tools to support Cypress package management as well[3] although they _are_ 3 years old and would deserve additional work ... Internally we are seriously considering adding Cypress support to the base image for the GemStone 3.4 release and of course in the process of doing that work, I assume that we'd make the code available for use in 3.2 and 3.3 as well.

In the short term I think that a combination of Cypress package support from Metacello, GitFileTree in Pharo and the updating of the Cypress tools for tODE will allow you to share packages between Pharo and GemStone without needing to create and maintain Monticello meta data ... in the midterm Cypress support is slated for this year and if necessary I can make changes to the current tODE FileTree implementation to allow the writing of metadata-less packages -- I think GitFileTree records whether or not a repository is metadata-less in a repository property file that I can use if necessary ...

Does this make sense?

Dale

[2] https://github.com/rjsargent/CypressReferenceImplementation
[3] https://github.com/dalehenrich/tode/tree/master/repository/Topez-Server-CypressTools.package


Reply | Threaded
Open this post in threaded view
|

Re: Saving to local git and "Loading all file names from http://...pharo5/inbox"

webwarrior
Updated script for Pharo6:

StartupPreferencesLoader default executeAtomicItems: {
        StartupAction
                name: 'remove references to Pharo60 and Pharo60Inbox'
                code: [

|rx group|
rx := '([0-9]{5})' asRegex.
(rx matchesPrefix: Smalltalk image shortImageName) ifFalse: [
        group := MCRepositoryGroup default.
        group repositories detect: [ :each |
                each location includesSubstring: 'Pharo/Pharo60Inbox/main'
        ] ifFound: [ :repo |
                group removeRepository: repo
        ].

        group repositories detect: [ :each |
                each location includesSubstring: 'Pharo/Pharo60/main'
        ] ifFound: [ :repo |
                group removeRepository: repo
        ].

        group repositories detect: [ :each |
                each location includesSubstring: 'MetaRepoForPharo60/main'
        ] ifFound: [ :repo |
                group removeRepository: repo
        ]
]

                ]
                runOnce: true.
}
Reply | Threaded
Open this post in threaded view
|

Re: Saving to local git and "Loading all file names from http://...pharo5/inbox"

Stephane Ducasse-3
Hi ww

Can you explain what your script do or even better what is the scenario ?
You remove all the pharo60 and 60 inbox from the packages. ok
This because you do not want to do merge? or changes comparison?


Tx for sharing it.

On Fri, Jun 16, 2017 at 2:43 PM, webwarrior <[hidden email]> wrote:

> Updated script for Pharo6:
>
> StartupPreferencesLoader default executeAtomicItems: {
>         StartupAction
>                 name: 'remove references to Pharo60 and Pharo60Inbox'
>                 code: [
>
> |rx group|
> rx := '([0-9]{5})' asRegex.
> (rx matchesPrefix: Smalltalk image shortImageName) ifFalse: [
>         group := MCRepositoryGroup default.
>         group repositories detect: [ :each |
>                 each location includesSubstring: 'Pharo/Pharo60Inbox/main'
>         ] ifFound: [ :repo |
>                 group removeRepository: repo
>         ].
>
>         group repositories detect: [ :each |
>                 each location includesSubstring: 'Pharo/Pharo60/main'
>         ] ifFound: [ :repo |
>                 group removeRepository: repo
>         ].
>
>         group repositories detect: [ :each |
>                 each location includesSubstring: 'MetaRepoForPharo60/main'
>         ] ifFound: [ :repo |
>                 group removeRepository: repo
>         ]
> ]
>
>                 ]
>                 runOnce: true.
> }
>
>
>
> --
> View this message in context: http://forum.world.st/Saving-to-local-git-and-Loading-all-file-names-from-http-pharo5-inbox-tp4897962p4951677.html
> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>

Reply | Threaded
Open this post in threaded view
|

Re: Saving to local git and "Loading all file names from http://...pharo5/inbox"

webwarrior
Well, that is the question to the author of original script - Peter Uhnak.

The goal is to avoid contacting server (takes alot of time and blocks the image) every time you save a package to a filetree repo. As I understand repositories in question are kind of special and are always checked, so removing them solves the problem.

I just modified it to work on Pharo6.
12