Smalltalk › Pharo › Pharo Smalltalk Users

Loading and saving packages to filetree repos

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

9 messages Options

Luke Gorrie

Loading and saving packages to filetree repos

Hoi,

I want to have a quick "cheat mode" for loading and saving the Smalltalk packages in my project. This is to make life easy for newbies who are not very familiar with Monticello and Metacello.

The "cheat" is to assume that there is one filetree:// repository that contains all of the relevant packages, and all we need to do is load or save each of those packages in that repository.

I have the loading part working already:

repo := MCFileTreeRepository new directory: '/foo/bar/baz' asFileReference.

repo allFileNames do: [ :file |

(repo versionFromFileNamed: file) load.

but now I am wondering how to do the saving part? That is, given a path to a filetree repo like '/foo/bar/baz', how do I save each package in that repo i.e. export the code in the image?

Ideally I would like the same operation to skip metadata that is likely to cause conflicts when the code is checked into Git later e.g. package timestamps and versions.

Tips would be appreciated :).

Thierry Goubier

Re: Loading and saving packages to filetree repos

Hi Luke,

if you use gitfiletree with AltBrowser and configurations/baselines,
then you'll see that you have a command to do the writing for you,
without metadata and with a single git commit.

Regards,

Thierry

Le 17/08/2017 à 13:25, Luke Gorrie a écrit :

> Hoi,
>
> I want to have a quick "cheat mode" for loading and saving the Smalltalk
> packages in my project. This is to make life easy for newbies who are
> not very familiar with Monticello and Metacello.
>
> The "cheat" is to assume that there is one filetree:// repository that
> contains all of the relevant packages, and all we need to do is load or
> save each of those packages in that repository.
>
> I have the loading part working already:
>
> repo := MCFileTreeRepository new directory: '/foo/bar/baz'
> asFileReference.
> repo allFileNames do: [ :file |
> (repo versionFromFileNamed: file) load.
> ].
>
> but now I am wondering how to do the saving part? That is, given a path
> to a filetree repo like '/foo/bar/baz', how do I save each package in
> that repo i.e. export the code in the image?
>
> Ideally I would like the same operation to skip metadata that is likely
> to cause conflicts when the code is checked into Git later e.g. package
> timestamps and versions.
>
> Tips would be appreciated :).
>
>
>

kilon.alios

Re: Loading and saving packages to filetree repos

In reply to this post by Luke Gorrie

First lets get a basic understanding of git, filetree and metadata.

The process of using both filetree and git are simple and short , the only problem is Monticello with a GUI that is not well designed.

With filetree you do what you would have done anyway

1) You pick a package

2) You create a filetree repo for it

3) You tell to filetree where to save your files

4) You edit/add code

5) You tell filetree to save your files

In another language you would have done something similar using an IDE

1) Create a project

2) Tell to the IDE where to save your project

3) Create new source files

4) edit code in those source files

5) save those source files

Git is also very simple , git is a version control system which basically what it does is create a history of changes (= versions) which calls commits , it also allows you to have versions of your history which calls branches which are basically variations of the same project and you end up with a nice history tree

Git has 3 basic actions , push, pull and commit

The first thing is pull which what it does is that it pull from another git repo, usually this is a github or other online repo used as the main repo but it can be also local too. In a pull git does again what you would expect , it takes a look at the history of the other repo and the history of the repo you want to pull to and first makes sure that the repos do not contain different histories. If they do then git ask 2 logical questions

1) Are these commits just additional commits that the receiver repo does not have ? If yes then just bring this commits and add them to the receiving repo history

2) Are these commits not only additional but also in between , which means they can be the commits for example no 1, 6 , 23 , 45 and 64 ? if yes then I do not know what to do , alert the user there are conflicts, which commits do conflict and exactly which code and which source files

git commit just add commits to your repo history.

git push does the opposite of git pull its sends your commits to the other repo but again it asks those two questions

There is also git clone which basically creates a repo and git pulls from another repo

This means that conflicts cannot exist , under usual circumstances, for single user repos. Because a user cannot create randomly diffirent commits in his/her own repo unless she works from different computers and forgets to git pull and thus creates different histories.

Commits appear when you have multiple people with their own repos pushing to the same repo, in that case its much easier to create conflicts. However it may surprise people that if you remove the metadata you will not greatly reduce git conflicts. What greatly reduces conflicts is first to always remember to git pull BEFORE adding new commits , this way there is less chance for histories to go out of sync and of course having small teams. The less people the less chance to create conflicts.

A way around this is to make branches or forks, it does not actually solve the problem. A branch as we said is a version of the history of a repo and it evolves separately. A fork is basically another repo that has its own history and also of course its own branches. Both branches and forks can merge with another history at any point along the way, still they will have conflicts. The big diffirence here is that you deal with those conflicts only when you do that merge (in the case of a fork we call it "pull request" because you request from another repo to pull your commits). So basically what you do is delay the inevitable.

If you work alone you don't need to worry about conflicts and thus not need to worry about metadata. There is very little chance to cause conflicts , close to zero.

if you work as a team, conflicts will be more and more frequent , the less properly your teammates use git, aka forget to git pull and push often, thus metadata will end up being the lest of your concerns. Basically because metadata are just information about your code and your project, there is nothing really complex about it and thus a conflict with metadata will be a minor concern.

A conflict on the code on the other hand is a big deal, if your teammate forgets to git push and pull often , there is also a big chance he does an even bigger mistakes and commits less often. This is a huge deal because that means his commits will contain huge chunks of code which in return will make conflict resolution a nightmare.

Again the problem does not originate from git or filetree , the problem is understanding the code and making sure that what you merge which basically means you decide which code is rejected and which is accepted inside the history is code that works, is reliable and does not create a stream of bugs and mostly does not take your project backward instead of forward.

So complexity is multiplied much more by the human factor and much less from the technology factor.

A work around this is to use powerful git tools that make some common decisions for you and help you understand those conflicts , but in the end conflict resolution is an art and a science by itself and its directly related on how well you understand not only the code of the project itself but also the code others provide.

Why all this lengthy post ?

I am not saying you do not know all this, I have no means to know what you do or what you do not know . My purpose is to make sure people reading this post realise this is not a metadata, git or filetree problem . The problem is the actual code conflicts. It does not matter so much if metadata is removed from the equation if your or anyone else reading does not understands the nature of those code conflicts.

Where this leaves you with your initial question how to do this easier for newcomers ?

DONT

Because any means to simplify a already simple process is hiding important steps of that process which means the confusion your will try to avoid know will multiply later for the newcomer when he end up not only having to learn all you tried to help me avoid to learn but also having to resolve an army of problems he generates because of his or her ignorance.

Or to put this entire lengthy post in one sentence

Laziness always comes with a high price to pay ;)

On the matter of auto save, sure you can do that but again its not that necessary because any change you make to the code is already saved to changes files. But yes sure you can make filetree auto save to files , that could make things a tiny bit easier.

Hoi,

I want to have a quick "cheat mode" for loading and saving the Smalltalk packages in my project. This is to make life easy for newbies who are not very familiar with Monticello and Metacello.

The "cheat" is to assume that there is one filetree:// repository that contains all of the relevant packages, and all we need to do is load or save each of those packages in that repository.

I have the loading part working already:

repo := MCFileTreeRepository new directory: '/foo/bar/baz' asFileReference.
repo allFileNames do: [ :file |
(repo versionFromFileNamed: file) load.
].

but now I am wondering how to do the saving part? That is, given a path to a filetree repo like '/foo/bar/baz', how do I save each package in that repo i.e. export the code in the image?

Ideally I would like the same operation to skip metadata that is likely to cause conflicts when the code is checked into Git later e.g. package timestamps and versions.

Tips would be appreciated :).

Luke Gorrie

Re: Loading and saving packages to filetree repos

In reply to this post by Thierry Goubier

Hi Thierry,

I am really looking for a "low-tech" solution here - five line Monticello code snippet - that will be easy for me to understand (as a newbie) and also easy for new contributors to my project to understand (who will have no prior exposure to Pharo.)

I don't know what AltBrowser is, but just as a user I need to get a bit more "winnage" before I am ready to take another iteration at learning more tools. The tools currently represent barriers in the way of solving my problems. I am sure this will change over time with more gradual learning and experience.

On 18 August 2017 at 16:28, Thierry Goubier <[hidden email]> wrote:

Hi Luke,

if you use gitfiletree with AltBrowser and configurations/baselines, then you'll see that you have a command to do the writing for you, without metadata and with a single git commit.

Regards,

Thierry

Le 17/08/2017 à 13:25, Luke Gorrie a écrit :

Hoi,

I want to have a quick "cheat mode" for loading and saving the Smalltalk packages in my project. This is to make life easy for newbies who are not very familiar with Monticello and Metacello.

The "cheat" is to assume that there is one filetree:// repository that contains all of the relevant packages, and all we need to do is load or save each of those packages in that repository.

I have the loading part working already:

repo := MCFileTreeRepository new directory: '/foo/bar/baz' asFileReference.
repo allFileNames do: [ :file |
(repo versionFromFileNamed: file) load.
].

but now I am wondering how to do the saving part? That is, given a path to a filetree repo like '/foo/bar/baz', how do I save each package in that repo i.e. export the code in the image?

Ideally I would like the same operation to skip metadata that is likely to cause conflicts when the code is checked into Git later e.g. package timestamps and versions.

Tips would be appreciated :).

Thierry Goubier

Re: Loading and saving packages to filetree repos

Hi Luke,

ok, it's a high tech solution, this is more because I could not on the
spot remember what you would need as a script (and also because your 'no
metadata' requirement makes the filetree solution problematic).

But I'd suggest still one bit of advice that is implicit in my solution:
create at least a baseline for your project (covering say all packages
in your repository); instead of hacking around with Monticello and
FileTree, it will allow you to use a simple Metacello command to load
everything and prepare correct, long term use of your project.

And you will have the save everything as well.

Regards,

Thierry

Le 20/08/2017 à 10:36, Luke Gorrie a écrit :

> Hi Thierry,
>
> I am really looking for a "low-tech" solution here - five line
> Monticello code snippet - that will be easy for me to understand (as a
> newbie) and also easy for new contributors to my project to understand
> (who will have no prior exposure to Pharo.)
>
> I don't know what AltBrowser is, but just as a user I need to get a bit
> more "winnage" before I am ready to take another iteration at learning
> more tools. The tools currently represent barriers in the way of solving
> my problems. I am sure this will change over time with more gradual
> learning and experience.
>
>
> On 18 August 2017 at 16:28, Thierry Goubier <[hidden email]
> <mailto:[hidden email]>> wrote:
>
> Hi Luke,
>
> if you use gitfiletree with AltBrowser and configurations/baselines,
> then you'll see that you have a command to do the writing for you,
> without metadata and with a single git commit.
>
> Regards,
>
> Thierry
>
>
> Le 17/08/2017 à 13:25, Luke Gorrie a écrit :
>
> Hoi,
>
> I want to have a quick "cheat mode" for loading and saving the
> Smalltalk packages in my project. This is to make life easy for
> newbies who are not very familiar with Monticello and Metacello.
>
> The "cheat" is to assume that there is one filetree://
> repository that contains all of the relevant packages, and all
> we need to do is load or save each of those packages in that
> repository.
>
> I have the loading part working already:
>
> repo := MCFileTreeRepository new directory: '/foo/bar/baz'
> asFileReference.
> repo allFileNames do: [ :file |
> (repo versionFromFileNamed: file) load.
> ].
>
> but now I am wondering how to do the saving part? That is, given
> a path to a filetree repo like '/foo/bar/baz', how do I save
> each package in that repo i.e. export the code in the image?
>
> Ideally I would like the same operation to skip metadata that is
> likely to cause conflicts when the code is checked into Git
> later e.g. package timestamps and versions.
>
> Tips would be appreciated :).
>
>
>
>
>
>

Stephan Eggermont-3

Re: Loading and saving packages to filetree repos

In reply to this post by Luke Gorrie

On 17-08-17 13:25, Luke Gorrie wrote:
> I want to have a quick "cheat mode" for loading and saving the Smalltalk
> packages in my project. This is to make life easy for newbies who are
> not very familiar with Monticello and Metacello.

I'm afraid you might do newbies a disservice by adding yet another way
of loading and saving combinations of packages. Dependency management is
a 5-dimensional problem, so there are no solutions that are both simple
and cover the needed variation [1]. In the current discussions on
version management on this list the focus seems to be on the solving of
simple problems, whilst avoiding the question of how to deal with many
repositories. Metacello might not be ideal there, but it provides a
shared solution.

Stephan

[1] Peter van den Hamer & Kees Lepoeter (1996) Managing Design Data: The
Five Dimensions of CAD Frameworks, Configuration Management, and Product
Data Management

Luke Gorrie

Re: Loading and saving packages to filetree repos

I wish that somebody would take my original question at face value.

It is frustrating to be told that I don't really understand the problem. I actually believe that the solution I am looking for is what I need in this project at this time. I don't want to write an essay-length justification for this position though.

Luke Gorrie

Re: Loading and saving packages to filetree repos

In reply to this post by Stephan Eggermont-3

On 20 August 2017 at 12:50, stephan <[hidden email]> wrote:

I'm afraid you might do newbies a disservice by adding yet another way of loading and saving combinations of packages.

I have been reflecting on this point a bit. I think there are two relevant classes of newbies here.

One is people who are learning to be Pharo hackers and joining this community. They need to "get with the program" and learn to do things the same way as everybody else. So they need to learn Smalltalk, Monticello, Metacello, Iceberg, FogBugz, etc.

The other is people who are using a Pharo-based application and want to make an extension. For example, imagine a Dr. Geo user who wants to contribute support for a new geometric function that they need for their work. This person needs to take the shortest path to solving their problem: reuse their existing skills (e.g. command-line git) and make use of examples (e.g. copy-paste-tweak some existing code.)

I'm concerned about the latter group. I'm building an application that I want users to be able to extend, and I am assuming that most of them are not aspiring Smalltalk hackers (yet.) So I am wondering how to minimize the barriers to entry for people to contribute their first extension.

My model is Emacs. Lots of people use Emacs to edit text files. Many of them become power users by copy-paste-tweak from each others' configuration files. Only a few take the next step and learn Lisp e.g. to understand what "lambda" actually means. Everybody manages to participate and share.

So, coming back to point, this is why I imagine now is to have a simple "Save all..." item in the World menu that saves all of the relevant-to-this-application packages back into a directory on the local disk (which will be a Git checkout that can be committed, pushed, and Pull Requested.) Then I don't need to force people to learn Metacello, etc, just to become "power users" of the application.

Maybe I am still on the wrong track anyway? Maybe others have been down this specific trail before, too?

Peter Uhnak

Re: Loading and saving packages to filetree repos

Apart from the things I mentioned to you on discord (fileout through FileWriter, etc)... another idea came into my mind that could be possibly interesting.

Every time a thing (Class, Method, ...) is created/changed/deleted, an event is announced in the system that you can watch for. (e.g. Nautilus uses this so when you add a method in one window, all Nautilus windows are updated).

And then you do an immediate fileout of the changed item to the disk.

a (hopefully still working) example:

```

ReflectiveWriter compile: 'methodChanged: aMethodAnnouncement

| method dir |

method := aMethodAnnouncement method.

dir := FileLocator temp / ''outside-code'' / method methodClass name.

dir ensureCreateDirectory.

(dir / method selector)

ensureDelete;

writeStreamDo: [ :stream |

"because line endings are #cr-azy by default"

stream wantsLineEndConversion: true.

stream << method sourceCode ]'.

```

I don't know how can Pharo watch for file changes in the system, but the opposite is also be doable in some manner.

(and of course you could filter to write changes only when the change is in particular class, or particular package, or whatever).

You would have to implement it yourself (extra time of coding, testing, debugging, ...), but in principle it should be doable within reasonable effort.

Peter

On Fri, Aug 25, 2017 at 4:14 PM, Luke Gorrie <[hidden email]> wrote:

On 20 August 2017 at 12:50, stephan <[hidden email]> wrote:
I'm afraid you might do newbies a disservice by adding yet another way of loading and saving combinations of packages.

I have been reflecting on this point a bit. I think there are two relevant classes of newbies here.

One is people who are learning to be Pharo hackers and joining this community. They need to "get with the program" and learn to do things the same way as everybody else. So they need to learn Smalltalk, Monticello, Metacello, Iceberg, FogBugz, etc.

The other is people who are using a Pharo-based application and want to make an extension. For example, imagine a Dr. Geo user who wants to contribute support for a new geometric function that they need for their work. This person needs to take the shortest path to solving their problem: reuse their existing skills (e.g. command-line git) and make use of examples (e.g. copy-paste-tweak some existing code.)

I'm concerned about the latter group. I'm building an application that I want users to be able to extend, and I am assuming that most of them are not aspiring Smalltalk hackers (yet.) So I am wondering how to minimize the barriers to entry for people to contribute their first extension.

My model is Emacs. Lots of people use Emacs to edit text files. Many of them become power users by copy-paste-tweak from each others' configuration files. Only a few take the next step and learn Lisp e.g. to understand what "lambda" actually means. Everybody manages to participate and share.

So, coming back to point, this is why I imagine now is to have a simple "Save all..." item in the World menu that saves all of the relevant-to-this-application packages back into a directory on the local disk (which will be a Git checkout that can be committed, pushed, and Pull Requested.) Then I don't need to force people to learn Metacello, etc, just to become "power users" of the application.

Maybe I am still on the wrong track anyway? Maybe others have been down this specific trail before, too?