Contributing to Pharo

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
104 messages Options
123456
Reply | Threaded
Open this post in threaded view
|

Re: Contributing to Pharo

Thierry Goubier
Le 03/02/2016 22:51, Eliot Miranda a écrit :

>
>
> On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     Hi Eliot,
>
>     Le 02/02/2016 21:54, Eliot Miranda a écrit :
>       ....
>
>
>         No it's /not/ the end of the story.  The essential part of the
>         story is
>         how Monticello remains compatible and interoperable between
>         dialects.  I
>         haven't seen you account for how you maintain that
>         compatibility.  As
>         far as I can tell, you propose replacing the Monticello metadata
>         with
>         that from git.  How do I, as a Squeak user with Monticello, ever
>         get to
>         look at your package again?  As I understand it, moving the metadata
>         from Monticello commit time to git means that the metadata is in a
>         format that git determines, not Monticello.
>
>
>     Yes. See below why.
>
>         So I don't understand how on the one hand you can say "The
>         Monticello
>         metadata in a git repository is redundant and leads to unnecessary
>         commit conflicts -- end of story ....", which implies you want to
>         eliminate the Monticello metadata, and on the other hand you say
>         you're
>         keeping the Monticello metadata.  I'm hopelessly confused.  How
>         does the
>         Monticello metadata get reconstituted if it's been thrown away?
>
>         What happens to the metadata in the following workflow?
>
>         load package P from Monticello repository R into an image
>         change P, commit via git to local git repository G
>         load P from G into an image
>         store P to R via Monticello
>
>
>     It's not a scenario I've specifically worked on, but all the tech is
>     implemented / implementable to do that perfectly.
>
>     The only thing that is problematic there is that the only safe
>     history is the one generated from git... there are so many MC
>     packages with broken history that, on mcz packages, you have to
>     admit that it's not safe to base things on their history.
>
>
> I'm sorry but I don't accept that.  In the Squeak trunk we have history
> in our mczs that is correct.  Certainly in VMMaker.oscog I have history
> that goes back a long time.  If bugs have broken history then efforts
> should be made to repair that history.  But you can't just write off
> Monticello history like that.

I don't. You presuppose.

I write tools that work with Monticello repositories, not just yours. I
have to do with what is given to me. On a general level, as a mcz user,
I'll just have to consider that you are as susceptible to be trusted as
with any other mcz producer. This means not much...

Thierry


Reply | Threaded
Open this post in threaded view
|

Re: Contributing to Pharo

Dale Henrichs-3


On 02/03/2016 02:34 PM, Thierry Goubier wrote:

> Le 03/02/2016 22:51, Eliot Miranda a écrit :
>>
>>
>> On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier
>> <[hidden email] <mailto:[hidden email]>> wrote:
>>
>>     Hi Eliot,
>>
>>     Le 02/02/2016 21:54, Eliot Miranda a écrit :
>>       ....
>>
>>
>>         No it's /not/ the end of the story.  The essential part of the
>>         story is
>>         how Monticello remains compatible and interoperable between
>>         dialects.  I
>>         haven't seen you account for how you maintain that
>>         compatibility.  As
>>         far as I can tell, you propose replacing the Monticello metadata
>>         with
>>         that from git.  How do I, as a Squeak user with Monticello, ever
>>         get to
>>         look at your package again?  As I understand it, moving the
>> metadata
>>         from Monticello commit time to git means that the metadata is
>> in a
>>         format that git determines, not Monticello.
>>
>>
>>     Yes. See below why.
>>
>>         So I don't understand how on the one hand you can say "The
>>         Monticello
>>         metadata in a git repository is redundant and leads to
>> unnecessary
>>         commit conflicts -- end of story ....", which implies you
>> want to
>>         eliminate the Monticello metadata, and on the other hand you say
>>         you're
>>         keeping the Monticello metadata.  I'm hopelessly confused.  How
>>         does the
>>         Monticello metadata get reconstituted if it's been thrown away?
>>
>>         What happens to the metadata in the following workflow?
>>
>>         load package P from Monticello repository R into an image
>>         change P, commit via git to local git repository G
>>         load P from G into an image
>>         store P to R via Monticello
>>
>>
>>     It's not a scenario I've specifically worked on, but all the tech is
>>     implemented / implementable to do that perfectly.
>>
>>     The only thing that is problematic there is that the only safe
>>     history is the one generated from git... there are so many MC
>>     packages with broken history that, on mcz packages, you have to
>>     admit that it's not safe to base things on their history.
>>
>>
>> I'm sorry but I don't accept that.  In the Squeak trunk we have history
>> in our mczs that is correct.  Certainly in VMMaker.oscog I have history
>> that goes back a long time.  If bugs have broken history then efforts
>> should be made to repair that history.  But you can't just write off
>> Monticello history like that.
>
> I don't. You presuppose.
>
> I write tools that work with Monticello repositories, not just yours.
> I have to do with what is given to me. On a general level, as a mcz
> user, I'll just have to consider that you are as susceptible to be
> trusted as with any other mcz producer. This means not much...

... And this is the reason why I am inclined to favor option 3, which
records the package version history as it existed at the point it was
copied into a git repo. When copied back out from the git universe,
create a version history that starts with the original version history
and generates a history of the package in git ....

Correct or not, the Monticello version history should be preserved ....

Dale

Reply | Threaded
Open this post in threaded view
|

Re: Contributing to Pharo

Thierry Goubier
Le 03/02/2016 23:58, Dale Henrichs a écrit :

>
>
> On 02/03/2016 02:34 PM, Thierry Goubier wrote:
>> Le 03/02/2016 22:51, Eliot Miranda a écrit :
>>>
>>>
>>> On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier
>>> <[hidden email] <mailto:[hidden email]>> wrote:
>>>
>>>     Hi Eliot,
>>>
>>>     Le 02/02/2016 21:54, Eliot Miranda a écrit :
>>>       ....
>>>
>>>
>>>         No it's /not/ the end of the story.  The essential part of the
>>>         story is
>>>         how Monticello remains compatible and interoperable between
>>>         dialects.  I
>>>         haven't seen you account for how you maintain that
>>>         compatibility.  As
>>>         far as I can tell, you propose replacing the Monticello metadata
>>>         with
>>>         that from git.  How do I, as a Squeak user with Monticello, ever
>>>         get to
>>>         look at your package again?  As I understand it, moving the
>>> metadata
>>>         from Monticello commit time to git means that the metadata is
>>> in a
>>>         format that git determines, not Monticello.
>>>
>>>
>>>     Yes. See below why.
>>>
>>>         So I don't understand how on the one hand you can say "The
>>>         Monticello
>>>         metadata in a git repository is redundant and leads to
>>> unnecessary
>>>         commit conflicts -- end of story ....", which implies you
>>> want to
>>>         eliminate the Monticello metadata, and on the other hand you say
>>>         you're
>>>         keeping the Monticello metadata.  I'm hopelessly confused.  How
>>>         does the
>>>         Monticello metadata get reconstituted if it's been thrown away?
>>>
>>>         What happens to the metadata in the following workflow?
>>>
>>>         load package P from Monticello repository R into an image
>>>         change P, commit via git to local git repository G
>>>         load P from G into an image
>>>         store P to R via Monticello
>>>
>>>
>>>     It's not a scenario I've specifically worked on, but all the tech is
>>>     implemented / implementable to do that perfectly.
>>>
>>>     The only thing that is problematic there is that the only safe
>>>     history is the one generated from git... there are so many MC
>>>     packages with broken history that, on mcz packages, you have to
>>>     admit that it's not safe to base things on their history.
>>>
>>>
>>> I'm sorry but I don't accept that.  In the Squeak trunk we have history
>>> in our mczs that is correct.  Certainly in VMMaker.oscog I have history
>>> that goes back a long time.  If bugs have broken history then efforts
>>> should be made to repair that history.  But you can't just write off
>>> Monticello history like that.
>>
>> I don't. You presuppose.
>>
>> I write tools that work with Monticello repositories, not just yours.
>> I have to do with what is given to me. On a general level, as a mcz
>> user, I'll just have to consider that you are as susceptible to be
>> trusted as with any other mcz producer. This means not much...
>
> ... And this is the reason why I am inclined to favor option 3, which
> records the package version history as it existed at the point it was
> copied into a git repo. When copied back out from the git universe,
> create a version history that starts with the original version history
> and generates a history of the package in git ....

Which is not very difficult to do given how GitFileTree is implemented.
And I agree this may well be the way to go. But ...

> Correct or not, the Monticello version history should be preserved ....

I wonder about that. The property of the Monticello version history is
that it has value when you can access the versions listed in it. If you
mix repositories like that, unless you maintain a link to the previous
mcz repository, pre-git versions can't be accessed. So, most of the
time, what we do with a project moved under git is to clone the previous
repository, not take just the current head.

(you'll notice, by the way, that vcs usually work that way when moving
from, say, CVS to git - you move the entire repository, not just the
latest version).

In short, the question would really be:
1- should we invest into making that integration of past history a
selling point (but I foresee issues down the road; I've only described
one so far, and I have seen others)
or
2- into making a better "clone" of a package history, timestamps and
everything when moving a complete repository to git?

Honestly, I'd consider 1- to be the easiest to implement. 2- there is
already some code floating around.

Thierry

Reply | Threaded
Open this post in threaded view
|

Re: Contributing to Pharo

Nicolas Cellier
I don't understand broken history either.
Yes there can be .mcz name clashes but UUID history is stored together in metadata no?
If some tool only trust .mcz name without checking UUID, consider it's a bug, and let's correct it (there is some in Monticello Configuration Map)

Or is it the fact that some .mcz could be missing?
Consider this is a feature, MC tools are robust to missing .mcz (it's just that you'll have to redo the merge if you lost a common ancestor).

Or is it the fact that some repository might contain only a slice of history?
This is another feature... You can view all the versions in a collection of repositories without needing to replicate.
You can replicate if you want but it's not mandatory and completely orthogonal.
So yes, this information - the list or repositories you want to consider - has to be stored separetely and this can sound quite unconventional.
But IMO, it's an important feature: it gives much resilience for a very low investment.
And that also mean that you can hardly break things (have unconsistent history).

Maybe when you say broken, you mean not 100% git compatible?

2016-02-04 7:53 GMT+01:00 Thierry Goubier <[hidden email]>:
Le 03/02/2016 23:58, Dale Henrichs a écrit :


On 02/03/2016 02:34 PM, Thierry Goubier wrote:
Le 03/02/2016 22:51, Eliot Miranda a écrit :


On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier
<[hidden email] <mailto:[hidden email]>> wrote:

    Hi Eliot,

    Le 02/02/2016 21:54, Eliot Miranda a écrit :
      ....


        No it's /not/ the end of the story.  The essential part of the
        story is
        how Monticello remains compatible and interoperable between
        dialects.  I
        haven't seen you account for how you maintain that
        compatibility.  As
        far as I can tell, you propose replacing the Monticello metadata
        with
        that from git.  How do I, as a Squeak user with Monticello, ever
        get to
        look at your package again?  As I understand it, moving the
metadata
        from Monticello commit time to git means that the metadata is
in a
        format that git determines, not Monticello.


    Yes. See below why.

        So I don't understand how on the one hand you can say "The
        Monticello
        metadata in a git repository is redundant and leads to
unnecessary
        commit conflicts -- end of story ....", which implies you
want to
        eliminate the Monticello metadata, and on the other hand you say
        you're
        keeping the Monticello metadata.  I'm hopelessly confused.  How
        does the
        Monticello metadata get reconstituted if it's been thrown away?

        What happens to the metadata in the following workflow?

        load package P from Monticello repository R into an image
        change P, commit via git to local git repository G
        load P from G into an image
        store P to R via Monticello


    It's not a scenario I've specifically worked on, but all the tech is
    implemented / implementable to do that perfectly.

    The only thing that is problematic there is that the only safe
    history is the one generated from git... there are so many MC
    packages with broken history that, on mcz packages, you have to
    admit that it's not safe to base things on their history.


I'm sorry but I don't accept that.  In the Squeak trunk we have history
in our mczs that is correct.  Certainly in VMMaker.oscog I have history
that goes back a long time.  If bugs have broken history then efforts
should be made to repair that history.  But you can't just write off
Monticello history like that.

I don't. You presuppose.

I write tools that work with Monticello repositories, not just yours.
I have to do with what is given to me. On a general level, as a mcz
user, I'll just have to consider that you are as susceptible to be
trusted as with any other mcz producer. This means not much...

... And this is the reason why I am inclined to favor option 3, which
records the package version history as it existed at the point it was
copied into a git repo. When copied back out from the git universe,
create a version history that starts with the original version history
and generates a history of the package in git ....

Which is not very difficult to do given how GitFileTree is implemented. And I agree this may well be the way to go. But ...

Correct or not, the Monticello version history should be preserved ....

I wonder about that. The property of the Monticello version history is that it has value when you can access the versions listed in it. If you mix repositories like that, unless you maintain a link to the previous mcz repository, pre-git versions can't be accessed. So, most of the time, what we do with a project moved under git is to clone the previous repository, not take just the current head.

(you'll notice, by the way, that vcs usually work that way when moving from, say, CVS to git - you move the entire repository, not just the latest version).

In short, the question would really be:
1- should we invest into making that integration of past history a selling point (but I foresee issues down the road; I've only described one so far, and I have seen others)
or
2- into making a better "clone" of a package history, timestamps and everything when moving a complete repository to git?

Honestly, I'd consider 1- to be the easiest to implement. 2- there is already some code floating around.

Thierry


Reply | Threaded
Open this post in threaded view
|

Re: Contributing to Pharo

Thierry Goubier
Le 04/02/2016 10:04, Nicolas Cellier a écrit :
> I don't understand broken history either.
> Yes there can be .mcz name clashes but UUID history is stored together
> in metadata no?

Yes.

> If some tool only trust .mcz name without checking UUID, consider it's a
> bug, and let's correct it (there is some in Monticello Configuration Map)

Gofer. In fact, most of Monticello never checks the UUID, only
dependencies do, but this looks like a deprecated feature given how
often it is used.

> Or is it the fact that some .mcz could be missing?
> Consider this is a feature, MC tools are robust to missing .mcz (it's
> just that you'll have to redo the merge if you lost a common ancestor).

What? You really consider that a feature?

> Or is it the fact that some repository might contain only a slice of
> history?
> This is another feature... You can view all the versions in a collection
> of repositories without needing to replicate.

Understandable in theory. Unworkable over time and change (repositories
disappear and die, and this stops working)

> You can replicate if you want but it's not mandatory and completely
> orthogonal.
> So yes, this information - the list or repositories you want to consider
> - has to be stored separetely and this can sound quite unconventional.
> But IMO, it's an important feature: it gives much resilience for a very
> low investment.
> And that also mean that you can hardly break things (have unconsistent
> history).

> Maybe when you say broken, you mean not 100% git compatible?

No, what I say is true mcz inconsistent history (missing versions making
merges very unreliable, basically).

I describe a while ago a case where, thanks to mcz features, I couldn't
merge a small change done to Roassal without generating a ton of
conflicts. I moved the three needed mcz(s) to git (the change ancestor,
the change, and the current head), did git merge and had no conflicts.

If you consider those features, then I disagree.

I'd really like to improve MC and get a better integration. Considering
those as features just make me think that Eliot may well be right in
believing we'll end up throwing away MC completely.

Coping with MC idiosyncrasies is hard as it is, and at least a complete
replacement is being considered for Pharo. In the meantime, Cuis has
completely given up managing packages in Smalltalk: all is done in git,
externally.

Thierry

> 2016-02-04 7:53 GMT+01:00 Thierry Goubier <[hidden email]
> <mailto:[hidden email]>>:
>
>     Le 03/02/2016 23:58, Dale Henrichs a écrit :
>
>
>
>         On 02/03/2016 02:34 PM, Thierry Goubier wrote:
>
>             Le 03/02/2016 22:51, Eliot Miranda a écrit :
>
>
>
>                 On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier
>                 <[hidden email]
>                 <mailto:[hidden email]>
>                 <mailto:[hidden email]
>                 <mailto:[hidden email]>>> wrote:
>
>                      Hi Eliot,
>
>                      Le 02/02/2016 21:54, Eliot Miranda a écrit :
>                        ....
>
>
>                          No it's /not/ the end of the story.  The
>                 essential part of the
>                          story is
>                          how Monticello remains compatible and
>                 interoperable between
>                          dialects.  I
>                          haven't seen you account for how you maintain that
>                          compatibility.  As
>                          far as I can tell, you propose replacing the
>                 Monticello metadata
>                          with
>                          that from git.  How do I, as a Squeak user with
>                 Monticello, ever
>                          get to
>                          look at your package again?  As I understand
>                 it, moving the
>                 metadata
>                          from Monticello commit time to git means that
>                 the metadata is
>                 in a
>                          format that git determines, not Monticello.
>
>
>                      Yes. See below why.
>
>                          So I don't understand how on the one hand you
>                 can say "The
>                          Monticello
>                          metadata in a git repository is redundant and
>                 leads to
>                 unnecessary
>                          commit conflicts -- end of story ....", which
>                 implies you
>                 want to
>                          eliminate the Monticello metadata, and on the
>                 other hand you say
>                          you're
>                          keeping the Monticello metadata.  I'm
>                 hopelessly confused.  How
>                          does the
>                          Monticello metadata get reconstituted if it's
>                 been thrown away?
>
>                          What happens to the metadata in the following
>                 workflow?
>
>                          load package P from Monticello repository R
>                 into an image
>                          change P, commit via git to local git repository G
>                          load P from G into an image
>                          store P to R via Monticello
>
>
>                      It's not a scenario I've specifically worked on,
>                 but all the tech is
>                      implemented / implementable to do that perfectly.
>
>                      The only thing that is problematic there is that
>                 the only safe
>                      history is the one generated from git... there are
>                 so many MC
>                      packages with broken history that, on mcz packages,
>                 you have to
>                      admit that it's not safe to base things on their
>                 history.
>
>
>                 I'm sorry but I don't accept that.  In the Squeak trunk
>                 we have history
>                 in our mczs that is correct.  Certainly in VMMaker.oscog
>                 I have history
>                 that goes back a long time.  If bugs have broken history
>                 then efforts
>                 should be made to repair that history.  But you can't
>                 just write off
>                 Monticello history like that.
>
>
>             I don't. You presuppose.
>
>             I write tools that work with Monticello repositories, not
>             just yours.
>             I have to do with what is given to me. On a general level,
>             as a mcz
>             user, I'll just have to consider that you are as susceptible
>             to be
>             trusted as with any other mcz producer. This means not much...
>
>
>         ... And this is the reason why I am inclined to favor option 3,
>         which
>         records the package version history as it existed at the point
>         it was
>         copied into a git repo. When copied back out from the git universe,
>         create a version history that starts with the original version
>         history
>         and generates a history of the package in git ....
>
>
>     Which is not very difficult to do given how GitFileTree is
>     implemented. And I agree this may well be the way to go. But ...
>
>         Correct or not, the Monticello version history should be
>         preserved ....
>
>
>     I wonder about that. The property of the Monticello version history
>     is that it has value when you can access the versions listed in it.
>     If you mix repositories like that, unless you maintain a link to the
>     previous mcz repository, pre-git versions can't be accessed. So,
>     most of the time, what we do with a project moved under git is to
>     clone the previous repository, not take just the current head.
>
>     (you'll notice, by the way, that vcs usually work that way when
>     moving from, say, CVS to git - you move the entire repository, not
>     just the latest version).
>
>     In short, the question would really be:
>     1- should we invest into making that integration of past history a
>     selling point (but I foresee issues down the road; I've only
>     described one so far, and I have seen others)
>     or
>     2- into making a better "clone" of a package history, timestamps and
>     everything when moving a complete repository to git?
>
>     Honestly, I'd consider 1- to be the easiest to implement. 2- there
>     is already some code floating around.
>
>     Thierry
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Contributing to Pharo

Henrik Sperre Johansen

> On 04 Feb 2016, at 10:26 , Thierry Goubier <[hidden email]> wrote:
>
> Le 04/02/2016 10:04, Nicolas Cellier a écrit :
>> I don't understand broken history either.
>> Yes there can be .mcz name clashes but UUID history is stored together
>> in metadata no?
>
> Yes.
>
>> If some tool only trust .mcz name without checking UUID, consider it's a
>> bug, and let's correct it (there is some in Monticello Configuration Map)
>
> Gofer. In fact, most of Monticello never checks the UUID, only dependencies do, but this looks like a deprecated feature given how often it is used.
>
>> Or is it the fact that some .mcz could be missing?
>> Consider this is a feature, MC tools are robust to missing .mcz (it's
>> just that you'll have to redo the merge if you lost a common ancestor).
>
> What? You really consider that a feature?
>
>> Or is it the fact that some repository might contain only a slice of
>> history?
>> This is another feature... You can view all the versions in a collection
>> of repositories without needing to replicate.
>
> Understandable in theory. Unworkable over time and change (repositories disappear and die, and this stops working)
>
>> You can replicate if you want but it's not mandatory and completely
>> orthogonal.
>> So yes, this information - the list or repositories you want to consider
>> - has to be stored separetely and this can sound quite unconventional.
>> But IMO, it's an important feature: it gives much resilience for a very
>> low investment.
>> And that also mean that you can hardly break things (have unconsistent
>> history).
>
>> Maybe when you say broken, you mean not 100% git compatible?
>
> No, what I say is true mcz inconsistent history (missing versions making merges very unreliable, basically).
>
> I describe a while ago a case where, thanks to mcz features, I couldn't merge a small change done to Roassal without generating a ton of conflicts. I moved the three needed mcz(s) to git (the change ancestor, the change, and the current head), did git merge and had no conflicts.
>
> If you consider those features, then I disagree.
Also, the ancestry of all packages in Pharo Core was truncated a few years back, which also screwed up merging*.
I don't think anyone looked into why it happened.

Cheers,
Henry

*http://forum.world.st/FogBugz-Case-Issue-12776-Tools-Cancelling-quot-Previous-Contents-quot-in-a-Workspace-clears-the-currs-td4753381.html

signature.asc (859 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Contributing to Pharo

Stephan Eggermont-3
On 04-02-16 13:09, Henrik Johansen wrote:
> Also, the ancestry of all packages in Pharo Core was truncated a few years back, which also screwed up merging*.
> I don't think anyone looked into why it happened.

My experiment in creating one file for a whole project would be a
suitable starting point for fixing that.

Stephan



Reply | Threaded
Open this post in threaded view
|

Re: Contributing to Pharo

Ben Coman
In reply to this post by Thierry Goubier
On Thu, Feb 4, 2016 at 5:26 PM, Thierry Goubier
<[hidden email]> wrote:
> In the meantime, Cuis has completely given up managing packages in
> Smalltalk: all is done in git externally.

Thats very interesting to hear - since 2012 even...
http://www.jvuletich.org/Cuis/CuisAndGitHub.html

and interesting to compare different approaches...
* https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-Dev/blob/master/Documentation/CuisAndGitHub.md

* https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-Dev/blob/master/Documentation/CodeManagementInCuis.md

cheers -ben

Reply | Threaded
Open this post in threaded view
|

Re: Contributing to Pharo

Thierry Goubier
In reply to this post by Henrik Sperre Johansen
Le 04/02/2016 13:09, Henrik Johansen a écrit :
>
> Also, the ancestry of all packages in Pharo Core was truncated a few
> years back, which also screwed up merging*. I don't think anyone
> looked into why it happened.

It was done on purpose for the Pharo 3 release to reduce the amount of
memory used by the MCVersionInfo instances.

For Pharo 4, a lazy loading of the MCVersionInfo was implemented
instead. We should have recreated the lost history then, but it seemed
not to have been important.

Thierry

Reply | Threaded
Open this post in threaded view
|

Re: Contributing to Pharo

Nicolas Cellier
In reply to this post by Thierry Goubier


2016-02-04 10:26 GMT+01:00 Thierry Goubier <[hidden email]>:
Le 04/02/2016 10:04, Nicolas Cellier a écrit :
I don't understand broken history either.
Yes there can be .mcz name clashes but UUID history is stored together
in metadata no?

Yes.

If some tool only trust .mcz name without checking UUID, consider it's a
bug, and let's correct it (there is some in Monticello Configuration Map)

Gofer. In fact, most of Monticello never checks the UUID, only dependencies do, but this looks like a deprecated feature given how often it is used.

Or is it the fact that some .mcz could be missing?
Consider this is a feature, MC tools are robust to missing .mcz (it's
just that you'll have to redo the merge if you lost a common ancestor).

What? You really consider that a feature?

Sure. Consider it's somehow like a git squash...

 

Or is it the fact that some repository might contain only a slice of
history?
This is another feature... You can view all the versions in a collection
of repositories without needing to replicate.

Understandable in theory. Unworkable over time and change (repositories disappear and die, and this stops working)

But that's the same with over vcs, repositories disappear and die (google code etc...).
Knowing that you can take usual defensive decisions (replicate what you depend on and not trust too much).
 

You can replicate if you want but it's not mandatory and completely
orthogonal.
So yes, this information - the list or repositories you want to consider
- has to be stored separetely and this can sound quite unconventional.
But IMO, it's an important feature: it gives much resilience for a very
low investment.
And that also mean that you can hardly break things (have unconsistent
history).

Maybe when you say broken, you mean not 100% git compatible?

No, what I say is true mcz inconsistent history (missing versions making merges very unreliable, basically).

Loosing a .mcz on which you branched is a bit more than unfortunate, it's a lack of understanding of the tool...
I say give git squash and rebase to unexperimented/uninformed people and that will be far worse.
 
I describe a while ago a case where, thanks to mcz features, I couldn't merge a small change done to Roassal without generating a ton of conflicts. I moved the three needed mcz(s) to git (the change ancestor, the change, and the current head), did git merge and had no conflicts.


Do you mean that git succeeded in merging text because different lines were changed in different branches?
I agree that it can solve some basic refactoring (renaming...)
But couldn't it lead to syntactically incorrect code? Traditional Smalltalk IDE are not well equipped for handling this...

The biggest grief I have with MC is more about package delimitation, when code is moved from 1 package to another.
For example, this effectively prevent merging very distant branches like Squeak and Pharo
(but we do not want to merge, just cherry pick in fact).

 
If you consider those features, then I disagree.

I'd really like to improve MC and get a better integration. Considering those as features just make me think that Eliot may well be right in believing we'll end up throwing away MC completely.


As long as it's as efficient, and well integrated in IDE, but again, it's against the small steps approach...
For me small steps would be to replace mcz name indexing by UUID indexing and just keep name as a façade.
I'm quite sure most work is at server side. How difficult would it really be?
No one will consider this maybe, because it's against the stream ;)
 
Coping with MC idiosyncrasies is hard as it is, and at least a complete replacement is being considered for Pharo. In the meantime, Cuis has completely given up managing packages in Smalltalk: all is done in git, externally.


But Cuis vcs come from dinosaure era (change sets) so anything else is already a progress.
Doing that in Squeak would feel like a loss of tool/functionality/integration.
Why didn't Pharo took the same path?

Thierry

2016-02-04 7:53 GMT+01:00 Thierry Goubier <[hidden email]
<mailto:[hidden email]>>:

    Le 03/02/2016 23:58, Dale Henrichs a écrit :



        On 02/03/2016 02:34 PM, Thierry Goubier wrote:

            Le 03/02/2016 22:51, Eliot Miranda a écrit :



                On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier
                <[hidden email]
                <mailto:[hidden email]>
                <mailto:[hidden email]

                <mailto:[hidden email]>>> wrote:

                     Hi Eliot,

                     Le 02/02/2016 21:54, Eliot Miranda a écrit :
                       ....


                         No it's /not/ the end of the story.  The
                essential part of the
                         story is
                         how Monticello remains compatible and
                interoperable between
                         dialects.  I
                         haven't seen you account for how you maintain that
                         compatibility.  As
                         far as I can tell, you propose replacing the
                Monticello metadata
                         with
                         that from git.  How do I, as a Squeak user with
                Monticello, ever
                         get to
                         look at your package again?  As I understand
                it, moving the
                metadata
                         from Monticello commit time to git means that
                the metadata is
                in a
                         format that git determines, not Monticello.


                     Yes. See below why.

                         So I don't understand how on the one hand you
                can say "The
                         Monticello
                         metadata in a git repository is redundant and
                leads to
                unnecessary
                         commit conflicts -- end of story ....", which
                implies you
                want to
                         eliminate the Monticello metadata, and on the
                other hand you say
                         you're
                         keeping the Monticello metadata.  I'm
                hopelessly confused.  How
                         does the
                         Monticello metadata get reconstituted if it's
                been thrown away?

                         What happens to the metadata in the following
                workflow?

                         load package P from Monticello repository R
                into an image
                         change P, commit via git to local git repository G
                         load P from G into an image
                         store P to R via Monticello


                     It's not a scenario I've specifically worked on,
                but all the tech is
                     implemented / implementable to do that perfectly.

                     The only thing that is problematic there is that
                the only safe
                     history is the one generated from git... there are
                so many MC
                     packages with broken history that, on mcz packages,
                you have to
                     admit that it's not safe to base things on their
                history.


                I'm sorry but I don't accept that.  In the Squeak trunk
                we have history
                in our mczs that is correct.  Certainly in VMMaker.oscog
                I have history
                that goes back a long time.  If bugs have broken history
                then efforts
                should be made to repair that history.  But you can't
                just write off
                Monticello history like that.


            I don't. You presuppose.

            I write tools that work with Monticello repositories, not
            just yours.
            I have to do with what is given to me. On a general level,
            as a mcz
            user, I'll just have to consider that you are as susceptible
            to be
            trusted as with any other mcz producer. This means not much...


        ... And this is the reason why I am inclined to favor option 3,
        which
        records the package version history as it existed at the point
        it was
        copied into a git repo. When copied back out from the git universe,
        create a version history that starts with the original version
        history
        and generates a history of the package in git ....


    Which is not very difficult to do given how GitFileTree is
    implemented. And I agree this may well be the way to go. But ...

        Correct or not, the Monticello version history should be
        preserved ....


    I wonder about that. The property of the Monticello version history
    is that it has value when you can access the versions listed in it.
    If you mix repositories like that, unless you maintain a link to the
    previous mcz repository, pre-git versions can't be accessed. So,
    most of the time, what we do with a project moved under git is to
    clone the previous repository, not take just the current head.

    (you'll notice, by the way, that vcs usually work that way when
    moving from, say, CVS to git - you move the entire repository, not
    just the latest version).

    In short, the question would really be:
    1- should we invest into making that integration of past history a
    selling point (but I foresee issues down the road; I've only
    described one so far, and I have seen others)
    or
    2- into making a better "clone" of a package history, timestamps and
    everything when moving a complete repository to git?

    Honestly, I'd consider 1- to be the easiest to implement. 2- there
    is already some code floating around.

    Thierry





Reply | Threaded
Open this post in threaded view
|

Re: Contributing to Pharo

Thierry Goubier
Le 04/02/2016 14:32, Nicolas Cellier a écrit :
>
>
> 2016-02-04 10:26 GMT+01:00 Thierry Goubier <[hidden email]
> <mailto:[hidden email]>>:
>
>     Le 04/02/2016 10:04, Nicolas Cellier a écrit :
>
>
[... cut ]

>         Or is it the fact that some .mcz could be missing?
>         Consider this is a feature, MC tools are robust to missing .mcz
>         (it's
>         just that you'll have to redo the merge if you lost a common
>         ancestor).
>
>
>     What? You really consider that a feature?
>
>
> Sure. Consider it's somehow like a git squash...

I consider history rewriting a very bad idea...

But, if we are at that, I just consider that the removed versions do not
exist anymore, and are not visible (should not be visible) in the
history, and that you have perfectly predictable correct behavior, not
can't do this because package version X is missing.

>         Or is it the fact that some repository might contain only a slice of
>         history?
>         This is another feature... You can view all the versions in a
>         collection
>         of repositories without needing to replicate.
>
>
>     Understandable in theory. Unworkable over time and change
>     (repositories disappear and die, and this stops working)
>
>
> But that's the same with over vcs, repositories disappear and die
> (google code etc...).
> Knowing that you can take usual defensive decisions (replicate what you
> depend on and not trust too much).

I mean that VCS recreate a working history and log when you are in such
a situation. MC just let you believe you have all and fail when you
attain the holes.

>         You can replicate if you want but it's not mandatory and completely
>         orthogonal.
>         So yes, this information - the list or repositories you want to
>         consider
>         - has to be stored separetely and this can sound quite
>         unconventional.
>         But IMO, it's an important feature: it gives much resilience for
>         a very
>         low investment.
>         And that also mean that you can hardly break things (have
>         unconsistent
>         history).
>
>
>         Maybe when you say broken, you mean not 100% git compatible?
>
>
>     No, what I say is true mcz inconsistent history (missing versions
>     making merges very unreliable, basically).
>
> Loosing a .mcz on which you branched is a bit more than unfortunate,
> it's a lack of understanding of the tool...

Not true. MC let you believe it will cope and it doesn't, which is not
exactly the same.

> I say give git squash and rebase to unexperimented/uninformed people and
> that will be far worse.

Agreed.

>     I describe a while ago a case where, thanks to mcz features, I
>     couldn't merge a small change done to Roassal without generating a
>     ton of conflicts. I moved the three needed mcz(s) to git (the change
>     ancestor, the change, and the current head), did git merge and had
>     no conflicts.
>
>
> Do you mean that git succeeded in merging text because different lines
> were changed in different branches?

No, in fact it was a clean merge because the changes in the branch were
on methods untouched in the main... git recreated a proper history and
could merge, whereas MC with the mcz history generated conflicts without
any reason.

> I agree that it can solve some basic refactoring (renaming...)
> But couldn't it lead to syntactically incorrect code? Traditional
> Smalltalk IDE are not well equipped for handling this...

Yes, this is partly why we're looking at more integration (merge tool)
and we may have to do more (a smalltalk syntax checker before commit
could be nice; CI does wonders on that).

> The biggest grief I have with MC is more about package delimitation,
> when code is moved from 1 package to another.
> For example, this effectively prevent merging very distant branches like
> Squeak and Pharo
> (but we do not want to merge, just cherry pick in fact).

Ok. Merge with multiple packages, right?

>
>     If you consider those features, then I disagree.
>
>     I'd really like to improve MC and get a better integration.
>     Considering those as features just make me think that Eliot may well
>     be right in believing we'll end up throwing away MC completely.
>
>
> As long as it's as efficient, and well integrated in IDE, but again,
> it's against the small steps approach...
> For me small steps would be to replace mcz name indexing by UUID
> indexing and just keep name as a façade.

I'd agree with that, as long as you don't ask too much of
compatibilities with some of MC features (:)).

> I'm quite sure most work is at server side. How difficult would it
> really be?

Why server side? I don't think it is.

> No one will consider this maybe, because it's against the stream ;)

No, in fact we're on it. Git integration is showing us where MC is weak,
and we're trying to fit it with MC.

>     Coping with MC idiosyncrasies is hard as it is, and at least a
>     complete replacement is being considered for Pharo. In the meantime,
>     Cuis has completely given up managing packages in Smalltalk: all is
>     done in git, externally.
>
>
> But Cuis vcs come from dinosaure era (change sets) so anything else is
> already a progress.
> Doing that in Squeak would feel like a loss of
> tool/functionality/integration.

I agree.

> Why didn't Pharo took the same path?

Than Cuis?

Thierry

>     Thierry
>
>         2016-02-04 7:53 GMT+01:00 Thierry Goubier
>         <[hidden email] <mailto:[hidden email]>
>         <mailto:[hidden email]
>         <mailto:[hidden email]>>>:
>
>              Le 03/02/2016 23:58, Dale Henrichs a écrit :
>
>
>
>                  On 02/03/2016 02:34 PM, Thierry Goubier wrote:
>
>                      Le 03/02/2016 22:51, Eliot Miranda a écrit :
>
>
>
>                          On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier
>                          <[hidden email]
>         <mailto:[hidden email]>
>                          <mailto:[hidden email]
>         <mailto:[hidden email]>>
>                          <mailto:[hidden email]
>         <mailto:[hidden email]>
>
>                          <mailto:[hidden email]
>         <mailto:[hidden email]>>>> wrote:
>
>                               Hi Eliot,
>
>                               Le 02/02/2016 21:54, Eliot Miranda a écrit :
>                                 ....
>
>
>                                   No it's /not/ the end of the story.  The
>                          essential part of the
>                                   story is
>                                   how Monticello remains compatible and
>                          interoperable between
>                                   dialects.  I
>                                   haven't seen you account for how you
>         maintain that
>                                   compatibility.  As
>                                   far as I can tell, you propose
>         replacing the
>                          Monticello metadata
>                                   with
>                                   that from git.  How do I, as a Squeak
>         user with
>                          Monticello, ever
>                                   get to
>                                   look at your package again?  As I
>         understand
>                          it, moving the
>                          metadata
>                                   from Monticello commit time to git
>         means that
>                          the metadata is
>                          in a
>                                   format that git determines, not
>         Monticello.
>
>
>                               Yes. See below why.
>
>                                   So I don't understand how on the one
>         hand you
>                          can say "The
>                                   Monticello
>                                   metadata in a git repository is
>         redundant and
>                          leads to
>                          unnecessary
>                                   commit conflicts -- end of story
>         ....", which
>                          implies you
>                          want to
>                                   eliminate the Monticello metadata, and
>         on the
>                          other hand you say
>                                   you're
>                                   keeping the Monticello metadata.  I'm
>                          hopelessly confused.  How
>                                   does the
>                                   Monticello metadata get reconstituted
>         if it's
>                          been thrown away?
>
>                                   What happens to the metadata in the
>         following
>                          workflow?
>
>                                   load package P from Monticello
>         repository R
>                          into an image
>                                   change P, commit via git to local git
>         repository G
>                                   load P from G into an image
>                                   store P to R via Monticello
>
>
>                               It's not a scenario I've specifically
>         worked on,
>                          but all the tech is
>                               implemented / implementable to do that
>         perfectly.
>
>                               The only thing that is problematic there
>         is that
>                          the only safe
>                               history is the one generated from git...
>         there are
>                          so many MC
>                               packages with broken history that, on mcz
>         packages,
>                          you have to
>                               admit that it's not safe to base things on
>         their
>                          history.
>
>
>                          I'm sorry but I don't accept that.  In the
>         Squeak trunk
>                          we have history
>                          in our mczs that is correct.  Certainly in
>         VMMaker.oscog
>                          I have history
>                          that goes back a long time.  If bugs have
>         broken history
>                          then efforts
>                          should be made to repair that history.  But you
>         can't
>                          just write off
>                          Monticello history like that.
>
>
>                      I don't. You presuppose.
>
>                      I write tools that work with Monticello
>         repositories, not
>                      just yours.
>                      I have to do with what is given to me. On a general
>         level,
>                      as a mcz
>                      user, I'll just have to consider that you are as
>         susceptible
>                      to be
>                      trusted as with any other mcz producer. This means
>         not much...
>
>
>                  ... And this is the reason why I am inclined to favor
>         option 3,
>                  which
>                  records the package version history as it existed at
>         the point
>                  it was
>                  copied into a git repo. When copied back out from the
>         git universe,
>                  create a version history that starts with the original
>         version
>                  history
>                  and generates a history of the package in git ....
>
>
>              Which is not very difficult to do given how GitFileTree is
>              implemented. And I agree this may well be the way to go.
>         But ...
>
>                  Correct or not, the Monticello version history should be
>                  preserved ....
>
>
>              I wonder about that. The property of the Monticello version
>         history
>              is that it has value when you can access the versions
>         listed in it.
>              If you mix repositories like that, unless you maintain a
>         link to the
>              previous mcz repository, pre-git versions can't be
>         accessed. So,
>              most of the time, what we do with a project moved under git
>         is to
>              clone the previous repository, not take just the current head.
>
>              (you'll notice, by the way, that vcs usually work that way when
>              moving from, say, CVS to git - you move the entire
>         repository, not
>              just the latest version).
>
>              In short, the question would really be:
>              1- should we invest into making that integration of past
>         history a
>              selling point (but I foresee issues down the road; I've only
>              described one so far, and I have seen others)
>              or
>              2- into making a better "clone" of a package history,
>         timestamps and
>              everything when moving a complete repository to git?
>
>              Honestly, I'd consider 1- to be the easiest to implement.
>         2- there
>              is already some code floating around.
>
>              Thierry
>
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Contributing to Pharo

Thierry Goubier
In reply to this post by Stephan Eggermont-3
Le 04/02/2016 13:21, Stephan Eggermont a écrit :
> On 04-02-16 13:09, Henrik Johansen wrote:
>> Also, the ancestry of all packages in Pharo Core was truncated a few
>> years back, which also screwed up merging*.
>> I don't think anyone looked into why it happened.
>
> My experiment in creating one file for a whole project would be a
> suitable starting point for fixing that.

Yes. I think we need a MC version history recreation / garbage collect.
Something that removes dead versions and compress history to make MC
merge work correctly.

(unless you move to git on metadataless mode and back...).

Thierry

> Stephan
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Contributing to Pharo

Dale Henrichs-3
In reply to this post by Stephan Eggermont-3
Stephan,

Have you described this anywhere?

Dale

On 2/4/16 4:21 AM, Stephan Eggermont wrote:

> On 04-02-16 13:09, Henrik Johansen wrote:
>> Also, the ancestry of all packages in Pharo Core was truncated a few
>> years back, which also screwed up merging*.
>> I don't think anyone looked into why it happened.
>
> My experiment in creating one file for a whole project would be a
> suitable starting point for fixing that.
>
> Stephan
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Contributing to Pharo

Dale Henrichs-3
In reply to this post by Thierry Goubier


On 2/3/16 10:53 PM, Thierry Goubier wrote:

> Le 03/02/2016 23:58, Dale Henrichs a écrit :
>>
>>
>> On 02/03/2016 02:34 PM, Thierry Goubier wrote:
>>> Le 03/02/2016 22:51, Eliot Miranda a écrit :
>>>>
>>>>
>>>> On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier
>>>> <[hidden email] <mailto:[hidden email]>> wrote:
>>>>
>>>>     Hi Eliot,
>>>>
>>>>     Le 02/02/2016 21:54, Eliot Miranda a écrit :
>>>>       ....
>>>>
>>>>
>>>>         No it's /not/ the end of the story.  The essential part of the
>>>>         story is
>>>>         how Monticello remains compatible and interoperable between
>>>>         dialects.  I
>>>>         haven't seen you account for how you maintain that
>>>>         compatibility.  As
>>>>         far as I can tell, you propose replacing the Monticello
>>>> metadata
>>>>         with
>>>>         that from git.  How do I, as a Squeak user with Monticello,
>>>> ever
>>>>         get to
>>>>         look at your package again?  As I understand it, moving the
>>>> metadata
>>>>         from Monticello commit time to git means that the metadata is
>>>> in a
>>>>         format that git determines, not Monticello.
>>>>
>>>>
>>>>     Yes. See below why.
>>>>
>>>>         So I don't understand how on the one hand you can say "The
>>>>         Monticello
>>>>         metadata in a git repository is redundant and leads to
>>>> unnecessary
>>>>         commit conflicts -- end of story ....", which implies you
>>>> want to
>>>>         eliminate the Monticello metadata, and on the other hand
>>>> you say
>>>>         you're
>>>>         keeping the Monticello metadata.  I'm hopelessly confused.  
>>>> How
>>>>         does the
>>>>         Monticello metadata get reconstituted if it's been thrown
>>>> away?
>>>>
>>>>         What happens to the metadata in the following workflow?
>>>>
>>>>         load package P from Monticello repository R into an image
>>>>         change P, commit via git to local git repository G
>>>>         load P from G into an image
>>>>         store P to R via Monticello
>>>>
>>>>
>>>>     It's not a scenario I've specifically worked on, but all the
>>>> tech is
>>>>     implemented / implementable to do that perfectly.
>>>>
>>>>     The only thing that is problematic there is that the only safe
>>>>     history is the one generated from git... there are so many MC
>>>>     packages with broken history that, on mcz packages, you have to
>>>>     admit that it's not safe to base things on their history.
>>>>
>>>>
>>>> I'm sorry but I don't accept that.  In the Squeak trunk we have
>>>> history
>>>> in our mczs that is correct.  Certainly in VMMaker.oscog I have
>>>> history
>>>> that goes back a long time.  If bugs have broken history then efforts
>>>> should be made to repair that history.  But you can't just write off
>>>> Monticello history like that.
>>>
>>> I don't. You presuppose.
>>>
>>> I write tools that work with Monticello repositories, not just yours.
>>> I have to do with what is given to me. On a general level, as a mcz
>>> user, I'll just have to consider that you are as susceptible to be
>>> trusted as with any other mcz producer. This means not much...
>>
>> ... And this is the reason why I am inclined to favor option 3, which
>> records the package version history as it existed at the point it was
>> copied into a git repo. When copied back out from the git universe,
>> create a version history that starts with the original version history
>> and generates a history of the package in git ....
>
> Which is not very difficult to do given how GitFileTree is
> implemented. And I agree this may well be the way to go. But ...
>
>> Correct or not, the Monticello version history should be preserved ....
>
> I wonder about that. The property of the Monticello version history is
> that it has value when you can access the versions listed in it. If
> you mix repositories like that, unless you maintain a link to the
> previous mcz repository, pre-git versions can't be accessed. So, most
> of the time, what we do with a project moved under git is to clone the
> previous repository, not take just the current head.
>
> (you'll notice, by the way, that vcs usually work that way when moving
> from, say, CVS to git - you move the entire repository, not just the
> latest version).
>
> In short, the question would really be:
> 1- should we invest into making that integration of past history a
> selling point (but I foresee issues down the road; I've only described
> one so far, and I have seen others)
> or
> 2- into making a better "clone" of a package history, timestamps and
> everything when moving a complete repository to git?
>
> Honestly, I'd consider 1- to be the easiest to implement. 2- there is
> already some code floating around.
Thierry,

My take on this is that we are not trying to "fix" Monticello.

The developers who prefer to use Monticello should continue to use
Monticello and the existing Monticello tools should continue to be
available. Any new formats should provide a migration path from
Monticello and any new tools like a `project list` tool should
accommodate projects based on both git and Monticello repositories.

This particular branch of the conversation has been aimed at trying to
resolve the tension that has been created by including Monticello
metadata in FileTree repositories and I think that "option 3": including
past history in the FileTree repository, but not updating the version
history for each git commit is a workable compromise --- there are
always "issues down the road" whether or not we see them and that
shouldn't keep us from trying to evolve:)

FileTree was invented (props to Otto Behrens) to smoothly integrate the
use of disk-based repositories into a Monticello development environment
dominated by Monticello tools.

Thierry your work building on the original FileTree has gone a long way
towards making git a viable development option for Pharo.

I think that you'd agree that the next step of evolution for "git-based"
development involves improved tool support and I would think that we
should turn our focus in that direction...

Dale

Reply | Threaded
Open this post in threaded view
|

Re: Contributing to Pharo

Stephan Eggermont-3
In reply to this post by Dale Henrichs-3
On 04-02-16 17:22, Dale Henrichs wrote:
> Stephan,
>
> Have you described this anywhere?

Not very much

http://forum.world.st/Overhead-in-mczs-MCVersionInfo-td4869374.html

MCHttpRepository
        location:
'http://smalltalkhub.com/mc/StephanEggermont/MonticelloProjects/main'
        user: ''
        password: ''

Jan 5. on Slack

Stephan Eggermont
20:21 So I deduplicated the ancestry, putting the UUID of missing
ancestors in place of the MCVersionInfo
20:25
123 MB instead of 1.2 GB, compressed 70.8 MB
20:26
The whole history of VMMaker
Thierry Goubier
20:34 With all versions?
Stephan Eggermont
20:34 missing 2,
20:35
and only those on source.squeak.org
Stephan Eggermont
20:35
CompactVMMaker.fl.tar.gz

67MB Binary Click to download
Thierry Goubier
20:37 I'd be interested to see how large would be the git repository
containing that. Maybe I'll make a try.
Stephan Eggermont
20:42 Nice compression factor. package cache of 2725 items, 6.1 GB
Thierry Goubier
20:44 !
Thierry Goubier
21:21 Then I can't test the git thing. Not enough HD space on the
Chromebook.


Reply | Threaded
Open this post in threaded view
|

Re: Contributing to Pharo

Ben Coman
In reply to this post by Dale Henrichs-3
On Fri, Feb 5, 2016 at 1:08 AM, Dale Henrichs
<[hidden email]> wrote:

>
>
> On 2/3/16 10:53 PM, Thierry Goubier wrote:
>>
>> Le 03/02/2016 23:58, Dale Henrichs a écrit :
>>>
>>>
>>>
>>> On 02/03/2016 02:34 PM, Thierry Goubier wrote:
>>>>
>>>> Le 03/02/2016 22:51, Eliot Miranda a écrit :
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier
>>>>> <[hidden email] <mailto:[hidden email]>> wrote:
>>>>>
>>>>>     Hi Eliot,
>>>>>
>>>>>     Le 02/02/2016 21:54, Eliot Miranda a écrit :
>>>>>       ....
>>>>>
>>>>>
>>>>>         No it's /not/ the end of the story.  The essential part of the
>>>>>         story is
>>>>>         how Monticello remains compatible and interoperable between
>>>>>         dialects.  I
>>>>>         haven't seen you account for how you maintain that
>>>>>         compatibility.  As
>>>>>         far as I can tell, you propose replacing the Monticello
>>>>> metadata
>>>>>         with
>>>>>         that from git.  How do I, as a Squeak user with Monticello,
>>>>> ever
>>>>>         get to
>>>>>         look at your package again?  As I understand it, moving the
>>>>> metadata
>>>>>         from Monticello commit time to git means that the metadata is
>>>>> in a
>>>>>         format that git determines, not Monticello.
>>>>>
>>>>>
>>>>>     Yes. See below why.
>>>>>
>>>>>         So I don't understand how on the one hand you can say "The
>>>>>         Monticello
>>>>>         metadata in a git repository is redundant and leads to
>>>>> unnecessary
>>>>>         commit conflicts -- end of story ....", which implies you
>>>>> want to
>>>>>         eliminate the Monticello metadata, and on the other hand you
>>>>> say
>>>>>         you're
>>>>>         keeping the Monticello metadata.  I'm hopelessly confused.  How
>>>>>         does the
>>>>>         Monticello metadata get reconstituted if it's been thrown away?
>>>>>
>>>>>         What happens to the metadata in the following workflow?
>>>>>
>>>>>         load package P from Monticello repository R into an image
>>>>>         change P, commit via git to local git repository G
>>>>>         load P from G into an image
>>>>>         store P to R via Monticello
>>>>>
>>>>>
>>>>>     It's not a scenario I've specifically worked on, but all the tech
>>>>> is
>>>>>     implemented / implementable to do that perfectly.
>>>>>
>>>>>     The only thing that is problematic there is that the only safe
>>>>>     history is the one generated from git... there are so many MC
>>>>>     packages with broken history that, on mcz packages, you have to
>>>>>     admit that it's not safe to base things on their history.
>>>>>
>>>>>
>>>>> I'm sorry but I don't accept that.  In the Squeak trunk we have history
>>>>> in our mczs that is correct.  Certainly in VMMaker.oscog I have history
>>>>> that goes back a long time.  If bugs have broken history then efforts
>>>>> should be made to repair that history.  But you can't just write off
>>>>> Monticello history like that.
>>>>
>>>>
>>>> I don't. You presuppose.
>>>>
>>>> I write tools that work with Monticello repositories, not just yours.
>>>> I have to do with what is given to me. On a general level, as a mcz
>>>> user, I'll just have to consider that you are as susceptible to be
>>>> trusted as with any other mcz producer. This means not much...
>>>
>>>
>>> ... And this is the reason why I am inclined to favor option 3, which
>>> records the package version history as it existed at the point it was
>>> copied into a git repo. When copied back out from the git universe,
>>> create a version history that starts with the original version history
>>> and generates a history of the package in git ....
>>
>>
>> Which is not very difficult to do given how GitFileTree is implemented.
>> And I agree this may well be the way to go. But ...
>>
>>> Correct or not, the Monticello version history should be preserved ....
>>
>>
>> I wonder about that. The property of the Monticello version history is
>> that it has value when you can access the versions listed in it. If you mix
>> repositories like that, unless you maintain a link to the previous mcz
>> repository, pre-git versions can't be accessed. So, most of the time, what
>> we do with a project moved under git is to clone the previous repository,
>> not take just the current head.
>>
>> (you'll notice, by the way, that vcs usually work that way when moving
>> from, say, CVS to git - you move the entire repository, not just the latest
>> version).
>>
>> In short, the question would really be:
>> 1- should we invest into making that integration of past history a selling
>> point (but I foresee issues down the road; I've only described one so far,
>> and I have seen others)
>> or
>> 2- into making a better "clone" of a package history, timestamps and
>> everything when moving a complete repository to git?
>>
>> Honestly, I'd consider 1- to be the easiest to implement. 2- there is
>> already some code floating around.
>
> Thierry,
>
> My take on this is that we are not trying to "fix" Monticello.
>
> The developers who prefer to use Monticello should continue to use
> Monticello and the existing Monticello tools should continue to be
> available. Any new formats should provide a migration path from Monticello
> and any new tools like a `project list` tool should accommodate projects
> based on both git and Monticello repositories.
>
> This particular branch of the conversation has been aimed at trying to
> resolve the tension that has been created by including Monticello metadata
> in FileTree repositories and I think that "option 3": including past history
> in the FileTree repository, but not updating the version history for each
> git commit is a workable compromise --- there are always "issues down the
> road" whether or not we see them and that shouldn't keep us from trying to
> evolve:)
>
> FileTree was invented (props to Otto Behrens) to smoothly integrate the use
> of disk-based repositories into a Monticello development environment
> dominated by Monticello tools.
>
> Thierry your work building on the original FileTree has gone a long way
> towards making git a viable development option for Pharo.
>
> I think that you'd agree that the next step of evolution for "git-based"
> development involves improved tool support and I would think that we should
> turn our focus in that direction...
>
> Dale
>

Pertinent is Joel's description of how Microsoft Excel's tipping point
[1] in competing against Lotus123 was when it became able to export to
Lotu123 format.  Making it easy for people to move from git back to
Monticello lower the barrier for entry for people to use git.

[1] Scroll down to the second "tipping point"
http://www.joelonsoftware.com/articles/fog0000000052.html

cheers -ben

Reply | Threaded
Open this post in threaded view
|

Re: Contributing to Pharo

Esteban A. Maringolo
2016-02-04 14:39 GMT-03:00 Ben Coman <[hidden email]>:

> Pertinent is Joel's description of how Microsoft Excel's tipping point
> [1] in competing against Lotus123 was when it became able to export to
> Lotu123 format.  Making it easy for people to move from git back to
> Monticello lower the barrier for entry for people to use git.

I always remember this article, but regarding Pharo and Git (or any
other file based SCM) I always thought about it the other way, it
is... the tipping point could be a seamless/smooth adoption of file
based SCMs, with git being the most prominent case.

Esteban A. Maringolo

Reply | Threaded
Open this post in threaded view
|

Re: Contributing to Pharo

Dale Henrichs-3
In reply to this post by Ben Coman


On 2/4/16 9:39 AM, Ben Coman wrote:

> On Fri, Feb 5, 2016 at 1:08 AM, Dale Henrichs
> <[hidden email]> wrote:
>>
>> On 2/3/16 10:53 PM, Thierry Goubier wrote:
>>> Le 03/02/2016 23:58, Dale Henrichs a écrit :
>>>>
>>>>
>>>> On 02/03/2016 02:34 PM, Thierry Goubier wrote:
>>>>> Le 03/02/2016 22:51, Eliot Miranda a écrit :
>>>>>>
>>>>>>
>>>>>> On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier
>>>>>> <[hidden email] <mailto:[hidden email]>> wrote:
>>>>>>
>>>>>>      Hi Eliot,
>>>>>>
>>>>>>      Le 02/02/2016 21:54, Eliot Miranda a écrit :
>>>>>>        ....
>>>>>>
>>>>>>
>>>>>>          No it's /not/ the end of the story.  The essential part of the
>>>>>>          story is
>>>>>>          how Monticello remains compatible and interoperable between
>>>>>>          dialects.  I
>>>>>>          haven't seen you account for how you maintain that
>>>>>>          compatibility.  As
>>>>>>          far as I can tell, you propose replacing the Monticello
>>>>>> metadata
>>>>>>          with
>>>>>>          that from git.  How do I, as a Squeak user with Monticello,
>>>>>> ever
>>>>>>          get to
>>>>>>          look at your package again?  As I understand it, moving the
>>>>>> metadata
>>>>>>          from Monticello commit time to git means that the metadata is
>>>>>> in a
>>>>>>          format that git determines, not Monticello.
>>>>>>
>>>>>>
>>>>>>      Yes. See below why.
>>>>>>
>>>>>>          So I don't understand how on the one hand you can say "The
>>>>>>          Monticello
>>>>>>          metadata in a git repository is redundant and leads to
>>>>>> unnecessary
>>>>>>          commit conflicts -- end of story ....", which implies you
>>>>>> want to
>>>>>>          eliminate the Monticello metadata, and on the other hand you
>>>>>> say
>>>>>>          you're
>>>>>>          keeping the Monticello metadata.  I'm hopelessly confused.  How
>>>>>>          does the
>>>>>>          Monticello metadata get reconstituted if it's been thrown away?
>>>>>>
>>>>>>          What happens to the metadata in the following workflow?
>>>>>>
>>>>>>          load package P from Monticello repository R into an image
>>>>>>          change P, commit via git to local git repository G
>>>>>>          load P from G into an image
>>>>>>          store P to R via Monticello
>>>>>>
>>>>>>
>>>>>>      It's not a scenario I've specifically worked on, but all the tech
>>>>>> is
>>>>>>      implemented / implementable to do that perfectly.
>>>>>>
>>>>>>      The only thing that is problematic there is that the only safe
>>>>>>      history is the one generated from git... there are so many MC
>>>>>>      packages with broken history that, on mcz packages, you have to
>>>>>>      admit that it's not safe to base things on their history.
>>>>>>
>>>>>>
>>>>>> I'm sorry but I don't accept that.  In the Squeak trunk we have history
>>>>>> in our mczs that is correct.  Certainly in VMMaker.oscog I have history
>>>>>> that goes back a long time.  If bugs have broken history then efforts
>>>>>> should be made to repair that history.  But you can't just write off
>>>>>> Monticello history like that.
>>>>>
>>>>> I don't. You presuppose.
>>>>>
>>>>> I write tools that work with Monticello repositories, not just yours.
>>>>> I have to do with what is given to me. On a general level, as a mcz
>>>>> user, I'll just have to consider that you are as susceptible to be
>>>>> trusted as with any other mcz producer. This means not much...
>>>>
>>>> ... And this is the reason why I am inclined to favor option 3, which
>>>> records the package version history as it existed at the point it was
>>>> copied into a git repo. When copied back out from the git universe,
>>>> create a version history that starts with the original version history
>>>> and generates a history of the package in git ....
>>>
>>> Which is not very difficult to do given how GitFileTree is implemented.
>>> And I agree this may well be the way to go. But ...
>>>
>>>> Correct or not, the Monticello version history should be preserved ....
>>>
>>> I wonder about that. The property of the Monticello version history is
>>> that it has value when you can access the versions listed in it. If you mix
>>> repositories like that, unless you maintain a link to the previous mcz
>>> repository, pre-git versions can't be accessed. So, most of the time, what
>>> we do with a project moved under git is to clone the previous repository,
>>> not take just the current head.
>>>
>>> (you'll notice, by the way, that vcs usually work that way when moving
>>> from, say, CVS to git - you move the entire repository, not just the latest
>>> version).
>>>
>>> In short, the question would really be:
>>> 1- should we invest into making that integration of past history a selling
>>> point (but I foresee issues down the road; I've only described one so far,
>>> and I have seen others)
>>> or
>>> 2- into making a better "clone" of a package history, timestamps and
>>> everything when moving a complete repository to git?
>>>
>>> Honestly, I'd consider 1- to be the easiest to implement. 2- there is
>>> already some code floating around.
>> Thierry,
>>
>> My take on this is that we are not trying to "fix" Monticello.
>>
>> The developers who prefer to use Monticello should continue to use
>> Monticello and the existing Monticello tools should continue to be
>> available. Any new formats should provide a migration path from Monticello
>> and any new tools like a `project list` tool should accommodate projects
>> based on both git and Monticello repositories.
>>
>> This particular branch of the conversation has been aimed at trying to
>> resolve the tension that has been created by including Monticello metadata
>> in FileTree repositories and I think that "option 3": including past history
>> in the FileTree repository, but not updating the version history for each
>> git commit is a workable compromise --- there are always "issues down the
>> road" whether or not we see them and that shouldn't keep us from trying to
>> evolve:)
>>
>> FileTree was invented (props to Otto Behrens) to smoothly integrate the use
>> of disk-based repositories into a Monticello development environment
>> dominated by Monticello tools.
>>
>> Thierry your work building on the original FileTree has gone a long way
>> towards making git a viable development option for Pharo.
>>
>> I think that you'd agree that the next step of evolution for "git-based"
>> development involves improved tool support and I would think that we should
>> turn our focus in that direction...
>>
>> Dale
>>
> Pertinent is Joel's description of how Microsoft Excel's tipping point
> [1] in competing against Lotus123 was when it became able to export to
> Lotu123 format.  Making it easy for people to move from git back to
> Monticello lower the barrier for entry for people to use git.
>
> [1] Scroll down to the second "tipping point"
> http://www.joelonsoftware.com/articles/fog0000000052.html
>
That was the original rationale for including the Monticello metadata.

Unfortunately the implementation I chose interferes with the natural use
of git, so an alternate solution is needed. I think the technology is
present to be able to implement "option 3" in relatively short order and
we can look forward to this in a future release of FileTree[1].

But we also need to focus on improving the "git" support by the tools so
that developers can leverage some of the advantages that "git" brings to
the tble... developers need to be able to see version history at the
package, class, and method level; they need to be able perform the
"standard" git operations (commit, checkout, branch, diff, push, pull
...) without leaving the image; and they need to be able to do git
merges in the image ....

At least that's my opinion:)

Dale

[1] https://github.com/dalehenrich/filetree/issues/177

Reply | Threaded
Open this post in threaded view
|

Re: Contributing to Pharo

Dale Henrichs-3
In reply to this post by Esteban A. Maringolo


On 2/4/16 9:48 AM, Esteban A. Maringolo wrote:

> 2016-02-04 14:39 GMT-03:00 Ben Coman <[hidden email]>:
>
>> Pertinent is Joel's description of how Microsoft Excel's tipping point
>> [1] in competing against Lotus123 was when it became able to export to
>> Lotu123 format.  Making it easy for people to move from git back to
>> Monticello lower the barrier for entry for people to use git.
> I always remember this article, but regarding Pharo and Git (or any
> other file based SCM) I always thought about it the other way, it
> is... the tipping point could be a seamless/smooth adoption of file
> based SCMs, with git being the most prominent case.
>
> Esteban A. Maringolo
>
... and both are important:)

Reply | Threaded
Open this post in threaded view
|

Re: Contributing to Pharo

Nicolas Cellier
In reply to this post by Henrik Sperre Johansen


2016-02-04 13:09 GMT+01:00 Henrik Johansen <[hidden email]>:

> On 04 Feb 2016, at 10:26 , Thierry Goubier <[hidden email]> wrote:
>
> Le 04/02/2016 10:04, Nicolas Cellier a écrit :
>> I don't understand broken history either.
>> Yes there can be .mcz name clashes but UUID history is stored together
>> in metadata no?
>
> Yes.
>
>> If some tool only trust .mcz name without checking UUID, consider it's a
>> bug, and let's correct it (there is some in Monticello Configuration Map)
>
> Gofer. In fact, most of Monticello never checks the UUID, only dependencies do, but this looks like a deprecated feature given how often it is used.
>
>> Or is it the fact that some .mcz could be missing?
>> Consider this is a feature, MC tools are robust to missing .mcz (it's
>> just that you'll have to redo the merge if you lost a common ancestor).
>
> What? You really consider that a feature?
>
>> Or is it the fact that some repository might contain only a slice of
>> history?
>> This is another feature... You can view all the versions in a collection
>> of repositories without needing to replicate.
>
> Understandable in theory. Unworkable over time and change (repositories disappear and die, and this stops working)
>
>> You can replicate if you want but it's not mandatory and completely
>> orthogonal.
>> So yes, this information - the list or repositories you want to consider
>> - has to be stored separetely and this can sound quite unconventional.
>> But IMO, it's an important feature: it gives much resilience for a very
>> low investment.
>> And that also mean that you can hardly break things (have unconsistent
>> history).
>
>> Maybe when you say broken, you mean not 100% git compatible?
>
> No, what I say is true mcz inconsistent history (missing versions making merges very unreliable, basically).
>
> I describe a while ago a case where, thanks to mcz features, I couldn't merge a small change done to Roassal without generating a ton of conflicts. I moved the three needed mcz(s) to git (the change ancestor, the change, and the current head), did git merge and had no conflicts.
>
> If you consider those features, then I disagree.

Also, the ancestry of all packages in Pharo Core was truncated a few years back, which also screwed up merging*.
I don't think anyone looked into why it happened.

Cheers,
Henry

*http://forum.world.st/FogBugz-Case-Issue-12776-Tools-Cancelling-quot-Previous-Contents-quot-in-a-Workspace-clears-the-currs-td4753381.html

Henry, you perfectly nailed it...
Break it first/throw away/replace is not allways a winning strategy.
We are ready to re-integrate source code in image at the price of a few MBytes, but it was urgent to throw history away BEFORE we even get a working replacement...
Here is a mail i did not send because I didn't want to be to start the year too negativily:

----------------------------

"snip... was about a regexp bug that I tried to bisect... (a post of january 2015)
So I wanted to be informed of the other changes going on since plenty of interesting things are going on in Pharo.

Ah hem, but the package has been renamed and the 'version history' menu is there only to show me that history has been purposely expunged from image... It became such a useless menu now, that i think you could as well remove it.

Spawning the 4.0 repository and using the History button, reveals a bit of history this time, sounds like the 3.0 bits...
Notice in attachment how the versions 22 and 23 which are in the 3.0 repository seems to be missing...
Ah, no this is a brand new strategy for presenting the merged branches... without any care for a common ancestor.
Notice in other attachment how it's not allways been so.

And what if I want to look a bit further behind?
Shall I go outside the image and browse github now?
The web UI did absolutely not work for me, the 3.0 history is cut there too, and blame only reveals me the name of Jenkins...
Git command line might have done a better work, but anyway, doesn't the I in IDE stands for Integrated?
I won't know because at this stage i gave up.

snip... that's enough ranting "

Regex-Core-MarcusDenker.33 History.png (96K) Download Attachment
Regex-Core-MarcusDenker.33 History viewed from Squeak.png (21K) Download Attachment
123456