[Pharo-dev] Versions Browser over gitfiletree://

Or ... A first step towards Pharo without a changeset.

Thierry

I like what I see! :)


Esteban A. Maringolo


2013/6/26 GOUBIER Thierry <[hidden email]>:
> Or ... A first step towards Pharo without a changeset.
>
> Thierry

Cool! I plan to integrate your work soon and I expect to merge your work on the Pharo3.0 branch as well...

Dale

----- Original Message -----
| From: "GOUBIER Thierry" <[hidden email]>
| To: "Pharo Development List ‎[[hidden email]]‎" <[hidden email]>
| Sent: Wednesday, June 26, 2013 2:28:55 PM
| Subject: [Pharo-dev] Versions Browser over gitfiletree://
|
| Or ... A first step towards Pharo without a changeset.
|
| Thierry

> Or ... A first step towards Pharo without a changeset.


without Monticello file back-end no?

Stef
>
> Thierry<Capture du 2013-06-26 23:14:08.png>

Le 27/06/2013 08:48, Stéphane Ducasse a écrit :
>
>
>> Or ... A first step towards Pharo without a changeset.
>
>
> without Monticello file back-end no?

No, in fact it's very much dependent on Monticello filetree back end.
This backend has the granularity which allow a use of git querying /
fetching facilities to the level associated with changesets.

I could implement a use of a gitfiletree:// repo as a complete
replacement of a live changeset + monticello repository in one, if needed.

Still wondering if this is the way to go.

Thierry

> Stef
>>
>> Thierry<Capture du 2013-06-26 23:14:08.png>
>
>
>

--
Thierry Goubier
CEA list
Laboratoire des Fondations des Systèmes Temps Réel Embarqués
91191 Gif sur Yvette Cedex
France
Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

Hi Dale,

should I merge this work into the last pull request (#85) I made? I've
corrected a few bugs and solved some performance issues, in addition to
adding the API for querying and loading parts of a package. It's
currently on a separate branch.

Thierry


Le 27/06/2013 01:26, Dale K. Henrichs a écrit :
--
Thierry Goubier
CEA list
Laboratoire des Fondations des Systèmes Temps Réel Embarqués
91191 Gif sur Yvette Cedex
France
Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

>
> No, in fact it's very much dependent on Monticello filetree back end. This backend has the granularity which allow a use of git querying / fetching facilities to the level associated with changesets.
>
> I could implement a use of a gitfiletree:// repo as a complete replacement of a live changeset + monticello repository in one, if needed.
>
> Still wondering if this is the way to go.

I do not know.

Stef

Le 27/06/2013 09:10, Stéphane Ducasse a écrit :
>>
>> No, in fact it's very much dependent on Monticello filetree back end. This backend has the granularity which allow a use of git querying / fetching facilities to the level associated with changesets.
>>
>> I could implement a use of a gitfiletree:// repo as a complete replacement of a live changeset + monticello repository in one, if needed.
>>
>> Still wondering if this is the way to go.
>
> I do not know.

It's tempting because it seems to solve long running irritations I had
with Smalltalk development (from many years ago); the need to keep the
same image (saving and saving again) to maintain the development
context, and being at the mercy of memory leaks, lack of stability,
updates trashing my development image, etc :(

With that kind of git support, I can change my workflow to: save
frequently my packages, rm the image, package-cache and all the junk,
and rebuilt everything from get.pharo.org and the git repos... Thanks to
the pharo command line improvements as well :)

Thierry
--
Thierry Goubier
CEA list
Laboratoire des Fondations des Systèmes Temps Réel Embarqués
91191 Gif sur Yvette Cedex
France
Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

On Jun 27, 2013, at 9:24 AM, Goubier Thierry <[hidden email]> wrote:

while I think we need to replace monticello eventually (it as became aged and accumulated patches over patches), I have to say that the workflow you are describing is perfectly doable with the state of the art versioning system, no need of git or whatever :)

Esteban

>> without Monticello file back-end no?
>
> No, in fact it's very much dependent on Monticello filetree back end. This backend has the granularity which allow a use of git querying / fetching facilities to the level associated with changesets.
>
> I could implement a use of a gitfiletree:// repo as a complete replacement of a live changeset + monticello repository in one, if needed.
>
> Still wondering if this is the way to go.

I think it is a very good start :),
eventually we have to come up with second version of the filetree format as
the current one has a major limitiation: categories are stored in the method source

Impact:
- md5 hashes of the pure ST method sources do not correspond the md5 hashes in the
  git repository
- each file-in requires another strip-the-category-path

of course there are workarounds for that. Additionally I would love to have a
filetree version that doesn't store all this MC metadata. The changes in these
files generate so much noise that is it hard to see the actual changes.

Anyway, most of it is future talk, I guess the current approach will work :)

Though I wonder what we will do once we have custom slots :/. I guess we need
some hacks to make that run on top of Monticello.

> It's tempting because it seems to solve long running irritations I had with Smalltalk development (from many years ago); the need to keep the same image (saving and saving again) to maintain the development context, and being at the mercy of memory leaks, lack of stability, updates trashing my development image, etc :(
>
> With that kind of git support, I can change my workflow to: save frequently my packages, rm the image, package-cache and all the junk, and rebuilt everything from get.pharo.org and the git repos... Thanks to the pharo command line improvements as well :)

But I save regularly on MC server or locally (after each new read and green tests is a good moment)
Because having git as a backend will not give you micro commit.

Then we are working on a really nice code modification logging named EPICEA to replace the old cs.
Because if you crash your image between two git commit.

Stef

On Jun 27, 2013, at 9:32 AM, Camillo Bruni <[hidden email]> wrote:

yes
:)
before it stays there with JSON for the next 30 years.


> Impact:
> - md5 hashes of the pure ST method sources do not correspond the md5 hashes in the
>  git repository
> - each file-in requires another strip-the-category-path

what is that?

>
> of course there are workarounds for that. Additionally I would love to have a
> filetree version that doesn't store all this MC metadata. The changes in these
> files generate so much noise that is it hard to see the actual changes.
>
> Anyway, most of it is future talk, I guess the current approach will work :)
>
> Though I wonder what we will do once we have custom slots :/. I guess we need
> some hacks to make that run on top of Monticello.

Le 27/06/2013 09:26, Esteban Lorenzano a écrit :
Hum. Redoing / redesigning a state of the art versionning system is so
large a task that I wouldn't start it unless I really have no choice :(

Improving the infrastructure for supporting / interacting with
versionning systems, yes! (I'm a bit fed up having to deal of a mix of
RG / MC / ChangeRecords / RPackage concepts where there should be only
one... Make it RG, if its complete enough). But trashing Monticello
because it looks old, I don't see the point.

In the meantime, git support allows me to study a few use cases we can't
do any other way with the current infrastructure, so that's interesting
(and make my life as a developper better :)).

Thierry
--
Thierry Goubier
CEA list
Laboratoire des Fondations des Systèmes Temps Réel Embarqués
91191 Gif sur Yvette Cedex
France
Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

Le 27/06/2013 09:39, Stéphane Ducasse a écrit :
>
>> It's tempting because it seems to solve long running irritations I had with Smalltalk development (from many years ago); the need to keep the same image (saving and saving again) to maintain the development context, and being at the mercy of memory leaks, lack of stability, updates trashing my development image, etc :(
>>
>> With that kind of git support, I can change my workflow to: save frequently my packages, rm the image, package-cache and all the junk, and rebuilt everything from get.pharo.org and the git repos... Thanks to the pharo command line improvements as well :)
>
> But I save regularly on MC server or locally (after each new read and green tests is a good moment)
> Because having git as a backend will not give you micro commit.

Yes and no. You could have micro commit, it's dead easy to implement.

> Then we are working on a really nice code modification logging named EPICEA to replace the old cs.
> Because if you crash your image between two git commit.

Micro-commit can be implemented on MC over git. It's a easy deal if you
need it.

I'm sure EPICEA will improve over the old cs format. Will that make it
better than improving the package infrastructure? I'm not so sure.

Thierry
--
Thierry Goubier
CEA list
Laboratoire des Fondations des Systèmes Temps Réel Embarqués
91191 Gif sur Yvette Cedex
France
Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

Le 27/06/2013 09:32, Camillo Bruni a écrit :
But, knowing the filetree format, you could rebuilt the correct md5, no?

Oups, finally, I'm not impressed by your idea of trying to second guess
git hashing :(

I like the fact that I can do git log --pretty="%H" path_to_method_file,
and I have all the hashes I need.

> - each file-in requires another strip-the-category-path

?

> of course there are workarounds for that. Additionally I would love to have a
> filetree version that doesn't store all this MC metadata. The changes in these
> files generate so much noise that is it hard to see the actual changes.

The version file for me is a bit strange, yes, and it tend to creates
conflict when merging with git. I don't see the problem for the rest of
the metadata changes, but I don't use gitk to look at the changes; I
build a Pharo GUI to filter that noise out.

> Anyway, most of it is future talk, I guess the current approach will work :)

It works surprisingly well. I even prefer it to smalltalkhub at the
moment: simpler authentification, more powerfull, probably smaller on
disk, better offline behavior.

> Though I wonder what we will do once we have custom slots :/. I guess we need
> some hacks to make that run on top of Monticello.

We will certainly. But I don't see that as a problem now that I've put
my hands into the Monticello filetree reader/writer.

Thierry
--
Thierry Goubier
CEA list
Laboratoire des Fondations des Systèmes Temps Réel Embarqués
91191 Gif sur Yvette Cedex
France
Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

yeah but you have the "wrong" hash as the source includes the category :)
I prefer storing the methods in subfolders which represent categories, so I keep
the methods sources as clean as possible. For instance this way a recategorization
is detected as a move by standard git tools. Now that I say, we should do the same
for the classes :P.

Anyway, right now this is mostly a detail, more importantly we need make sure the
image-side workflow is good :), so I am really happy that you push it

>> - each file-in requires another strip-the-category-path
>
> ?

=> typo, should be, strip-the-category pass :)

Le 27/06/2013 10:14, Camillo Bruni a écrit :
>>>>
>>
>> I like the fact that I can do git log --pretty="%H" path_to_method_file, and I have all the hashes I need.
>
> yeah but you have the "wrong" hash as the source includes the category :)
> I prefer storing the methods in subfolders which represent categories, so I keep
> the methods sources as clean as possible. For instance this way a recategorization
> is detected as a move by standard git tools. Now that I say, we should do the same
> for the classes :P.

In this case, how would you query git for all the versions of a method?
If it is recategorized, you would not be able to see through the move, no?

> Anyway, right now this is mostly a detail, more importantly we need make sure the
> image-side workflow is good :), so I am really happy that you push it

I think it is good that we try to see all the use cases and possible
changes at the format as we use it and improve the workflow around it.
There is a lot of experience to be gained there.

Thierry

>>> - each file-in requires another strip-the-category-path
>>
>> ?
>
> => typo, should be, strip-the-category pass :)
>
>

--
Thierry Goubier
CEA list
Laboratoire des Fondations des Systèmes Temps Réel Embarqués
91191 Gif sur Yvette Cedex
France
Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

On 2013-06-27, at 10:32, Goubier Thierry <[hidden email]> wrote:

if you store the category in the method you won't see a move of course, otherwise yes:

https://github.com/pharo-project/pharo-core/commit/4383816cd432229a9977cfbe7bf508690dcf495e

On 27 June 2013 08:32, Camillo Bruni <[hidden email]> wrote:
*cough* *cough* You mean SHA-1, I hope. But also you have a good point
re comparing hashes, but let's also not forget that whitespace changes
will affect the hash. You're right: storing the metadata of the thing
separate from the thing itself allows you to add to the _description_
of a thing without touching the thing itself.

frank

> of course there are workarounds for that. Additionally I would love to have a
> filetree version that doesn't store all this MC metadata. The changes in these
> files generate so much noise that is it hard to see the actual changes.
>
> Anyway, most of it is future talk, I guess the current approach will work :)
>
> Though I wonder what we will do once we have custom slots :/. I guess we need
> some hacks to make that run on top of Monticello.

On 2013-06-27, at 10:37, Frank Shearar <[hidden email]> wrote:

ups, right :D

> But also you have a good point
> re comparing hashes, but let's also not forget that whitespace changes
> will affect the hash. You're right: storing the metadata of the thing
> separate from the thing itself allows you to add to the _description_
> of a thing without touching the thing itself.
>
> frank