git workflow from development side? (not as consumer)

embedded ...

----- Original Message -----
| To: [hidden email]
| Sent: Tuesday, July 31, 2012 3:20:49 AM
| Subject: Re: [Metacello] git workflow from development side? (not as consumer)
|
| Reflection:
| To me the convention of placing the monticello packages in
| repository/ is a confusing naming convention - too many repositories
| given it's already inside a git repository (well, when using
| filetree and git together at least).
|
|
| My suggestion is to change the convention to "packages". More
| descriptive to me and avoids the multiple references to repository
| meaning different things.
|
|
| What do you think?

I think that's a fine suggestion ... conventions need a starting point and `repository` is serving it's purpose as a starting point ...

If you want to edit the GettingStarted... doc and issue a pull request I'll gladly merge it in...

Patrik,

Excellent feedback... keep it coming!

BTW, will you be attending ESUG (in Ghent) this year? If you were to attend, we'd have a chance to talk in person.

more comments embedded

----- Original Message -----
| To: [hidden email]
| Sent: Tuesday, July 31, 2012 5:45:13 AM
| Subject: Re: [Metacello] git workflow from development side? (not as consumer)
|
| I created a test project for myself. Got everything working - it's
| cool! Very promising start.

Excellent!

|
|
| Reflections:
|
|
| - I tried the config branch approach and also the separate repo for
| ConfigOf approach. Neither felt good. I found it painful to have to
| mess with switching branches to of the same filetree in order to
| work with packages that are all loaded in the image. I can also see
| scope for problems if automating this stuff from within the image
| (chances of leaving git repo in the wrong branch, trying to save the
| package belonging to the other branch etc).

To be frank, nothing has felt completely good ... so far. Putting the ConfigOf in a configuration branch is the least of the evils (note that the separate branch is only really required when you end up with multiple platform branches ... which don't necessarily feel good either).

I understand the bit about switching branches and I've settled (for the time being) a directory structure that looks like this:

+-opt
  +-git
    +-configurations
    | +-filetree
    +-filetree

Where I've got separate git repositories for the configurations and for the working branch ... I'm almost getting to the point where I'm thinking that I want separate git repos for each image ... which I think is headed towards what Bundler does ... is that right?

In some senses it will be easier to automate in the image, because we will have a spec that tells the image exactly which project/branch we are dealing with and which packages are associated with that repository ...

This is why I am interested in limited use of the git/github features right now ... we need to work through these issues and try out different approaches ... I'm settling into a comfortable pattern, but we need multiple instances of "comfortable pattern" to determine where the convention should be ...

|
|
| - For my purposes I don't really need particular SHA's for releases,
| I'll always use git tags if I need a release (and I tend to run on
| branches and not releases, usually don't have a concept of a release
| in the way I work). So I can update it all in one repo, use a tag in
| ConfigOf, commit, then create the git tag, then push it and it'll
| all be self-consistent on each push. So far that works for me - at
| least so far and in my mental model.

Tags are a really nice way to make a human readable SHA and when it comes to best practices (start making notes:) this would be a good recommendation.

|
| - Having separate Metacello commits and git commits makes no sense to
| me, so I'll always be creating them in sync. The different comments
| etc is obviously annoying. A tool that deals with that from within
| the image on "Save" (will take me a while to get used to it not
| being commit..) and uses the same commit comments etc doesn't seem
| that complicated to create. May have a crack at that once I'm happy
| with workflow.

Totally agree. Before striking off on your own with a git interface, take a look at the work of Max Leske and Cami Bruni on FSGit[1]. Their project provides an in-image api for git.

I think that both Cami and Max will be at ESUG and I plan on spending some time talking with them about how to integrate their work into with Metacello and FileTree...
 

[1] https://github.com/dalehenrich/FSGit

|
|
| - The API is nice in general.
|
|
| - One thing that tends to be important to me is to be able to develop
| several "parts" of my system at the same time. The different parts
| live in different git repos. In this case that translates to
| developing things coming from several ConfigOf's at the same time,
| and several git repos. I think I'll be able to create development
| specs that overrides github:// URIs to filetree:// URIs for the
| various pieces that I am developing at the same time (obv can't be
| done generically since relying on local path names, but still good
| and teams can have conventions on where to check things out etc).
| Then I can load things via Metacello, still have "2-way" ability to
| save changes to some of the dependencies of the main project into
| their filetree:// repos and manage the various git repos
| independently. That's my next thing to try with the API, check that
| such a workflow is feasible.[1]

Two things:

  - the configuration (ConfigurationOf) is intended for your end users
    whether that be other developers or other projects of your own that
    depend upon the project
  - the baseline (BaselineOf) is intended for developers (just as you
    have discovered)

The way to use a baseline is exactly as you've described (perhaps the GettingStarted... doc should be updated to make this more clear).
Create :filetree// repos for each of your git repositories and load the baseline directly:

  Metacello image
    baseline: 'Sample';
    get.
  Metacello image
    baseline: 'Sample';
    load.

The `get` refreshes the BaselineOf in the event that it may have changed and the `load` refreshes your image with the changed packages. If you have multiple projects (baselines) then you use the following script:

  Metacello image
    baseline: #('Sample' 'Other');
    get.
  Metacello image
    baseline: #('Sample' 'Other');
    load.

I only update the ConfigurationOf with a new SHA or tag when I'm ready for other folks/projects to use the code ... this makes the overhead of keeping a separate git repo for configs less onerous, because it doesn't change that frequently.
 

|
|
| - It seems generally very useful to me to be able to specify
| filetree:// URIs relative to home directory. I'll take a look at
| that. Makes collaborating a bit easier if one can check in dev
| versions referring to say "$HOME/development/projectA" rather than
| "/home/xxx/development/projectA".

I tend to use the :github in my project refs in configurations/baselines and I use the :filetree variant in the image for development only ... I also "centrally" locate all of my git repositories in `/opt/git` so that my various images and common scripts work (on my work and home computers)...so I guess in my case an absolute path the is constant for all images and working environments works well for me ...

With that said, I don't want to discourage experiments ... as I've said before we're in the process of discovery here (I'm saying that to remind myself as much as anyone else:)

|
|
| My test repo is here: https://github.com/sundbp/PatriksImageTools

Ah, you've based your work on Mariano's stuff[1]. I'll be interested if you find places where a) the scripting api _can_ be used and b) the scripting api _should_ be used

[1] http://marianopeck.wordpress.com/2011/11/19/loading-projects-and-building-your-own-images-with-metacello/

Thanks for the good comments!

Dale

On 14 August 2012 20:14, Patrik Sundberg <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="tLeOla-0YR4J">patrik....@...> wrote:
> Just a short FYI: I've decided to wait a while longer to dive into doing
> some projects in pharo. I can't stand the mouse intensity of it and before
> the Keymapping stuff is further along and one can start operating more via
> keyboard it makes me an unhappy person. Too much friction. So been extending
> my ruby+vim+pry+debugger setup to be more smalltalk like for the time being
> instead. My dream would be smalltalk backend with vim frontend like slime
> and emacs for lisp..

GNU Smalltalk. Although I can't say I've actually used gst, they
usually use vim or emacs.

I'm interested in supplying alternative ways of talking to Smalltalk
images, like running new UIs in browsers or (and there are folk in
Pharo doing this now) a REPL, or something like SLIME.

All of which is rather off-topic, but I guess I'm just saying that
there are folks willing to try keep the good bits of Smalltalk and
import the good bits of other environments.

frank

> Still reading the mailing lists and hope to help with some feedback. Hoping
> to later on do some actual projects in it but the mouse friction needs to go
> down before it makes sense for me - no use fighting deeply set habits like
> that :/

> On Tuesday, July 31, 2012 8:15:58 PM UTC+1, Patrik Sundberg wrote:
>>
>> On Tuesday, July 31, 2012 6:26:18 PM UTC+1, Dale wrote:
>>>
>>> Patrik,
>>>
>>> Excellent feedback... keep it coming!
>>>
>>> BTW, will you be attending ESUG (in Ghent) this year? If you were to
>>> attend, we'd have a chance to talk in person.
>>>
>>
>> No idea - I've never gone to any conference ever so the chances aren't
>> great :( I'll never say never though.
>>
>>>
>>> more comments embedded
>>>
>>> ----- Original Message -----
>>> | To: <a href="javascript:" target="_blank" gdf-obfuscated-mailto="tLeOla-0YR4J">meta...@...
>>> | Sent: Tuesday, July 31, 2012 5:45:13 AM
>>> | Subject: Re: [Metacello] git workflow from development side? (not as
>>> consumer)
>>> |
>>> | I created a test project for myself. Got everything working - it's
>>> | cool! Very promising start.
>>>
>>> Excellent!
>>>
>>> |
>>> |
>>> | Reflections:
>>> |
>>> |
>>> | - I tried the config branch approach and also the separate repo for
>>> | ConfigOf approach. Neither felt good. I found it painful to have to
>>> | mess with switching branches to of the same filetree in order to
>>> | work with packages that are all loaded in the image. I can also see
>>> | scope for problems if automating this stuff from within the image
>>> | (chances of leaving git repo in the wrong branch, trying to save the
>>> | package belonging to the other branch etc).
>>>
>>> To be frank, nothing has felt completely good ... so far. Putting the
>>> ConfigOf in a configuration branch is the least of the evils (note that the
>>> separate branch is only really required when you end up with multiple
>>> platform branches ... which don't necessarily feel good either).
>>>
>>> I understand the bit about switching branches and I've settled (for the
>>> time being) a directory structure that looks like this:
>>>
>>> +-opt
>>>   +-git
>>>     +-configurations
>>>     | +-filetree
>>>     +-filetree
>>>
>>> Where I've got separate git repositories for the configurations and for
>>> the working branch ... I'm almost getting to the point where I'm thinking
>>> that I want separate git repos for each image ... which I think is headed
>>> towards what Bundler does ... is that right?
>>>
>>
>> Ah, 2 working copies. Yeah, that's another way to do it. Hadn't thought of
>> that.
>>
>> There's no image in ruby so there's no straight forward equivalence. In
>> bundler a running ruby process only has one active bundle, but that bundle
>> can depend on any number of gems, some of which can be living in git repos
>> (that are cloned as read only storage for that project), local files, etc.
>> If two different processes, that use 2 different bundles, happen to refer to
>> the same revision of the same git repo they'll share that cache the way
>> bundler works. However, that git clone is only used in a read only way.
>>
>> So in development if I am developing two different "parts" with their own
>> Gemfiles/bundles, then I'll have a main project that refers to the checked
>> out git repos (just as normal files) for development, but for deployment
>> they use the git/github repos. So then I can move around the various
>> directories and commit/push/pull as I want to for the different parts.
>>
>>>
>>> In some senses it will be easier to automate in the image, because we
>>> will have a spec that tells the image exactly which project/branch we are
>>> dealing with and which packages are associated with that repository ...
>>>
>>
>> Yeah, Metacello has the potential of being more powerful in this sense
>> than bundler since it's "alive" and has all the info as you develop. bundles
>> in ruby only live at runtime, so it's got an edge on that side.
>>
>>>
>>> This is why I am interested in limited use of the git/github features
>>> right now ... we need to work through these issues and try out different
>>> approaches ... I'm settling into a comfortable pattern, but we need multiple
>>> instances of "comfortable pattern" to determine where the convention should
>>> be ...
>>>
>>
>> I'm anything but comfortable given I'm still finding my keyboard shortcuts
>> :) Agreed there's a lot of exploring to be done to tease out best practices.
>>
>>
>>> |
>>> |
>>> | - For my purposes I don't really need particular SHA's for releases,
>>> | I'll always use git tags if I need a release (and I tend to run on
>>> | branches and not releases, usually don't have a concept of a release
>>> | in the way I work). So I can update it all in one repo, use a tag in
>>> | ConfigOf, commit, then create the git tag, then push it and it'll
>>> | all be self-consistent on each push. So far that works for me - at
>>> | least so far and in my mental model.
>>>
>>> Tags are a really nice way to make a human readable SHA and when it comes
>>> to best practices (start making notes:) this would be a good recommendation.
>>>
>>
>> They have the benefit of not affecting SHA's as well, which is why I can
>> update a version in the main repo to be a specific tag (that doesn't yet
>> exist), then commit it (along with the changes to the real project), and
>> then tag it. At the following push the tag in the configof will exist at the
>> same time as the update to configof is available for consumption.
>>
>>> |
>>> | - Having separate Metacello commits and git commits makes no sense to
>>> | me, so I'll always be creating them in sync. The different comments
>>> | etc is obviously annoying. A tool that deals with that from within
>>> | the image on "Save" (will take me a while to get used to it not
>>> | being commit..) and uses the same commit comments etc doesn't seem
>>> | that complicated to create. May have a crack at that once I'm happy
>>> | with workflow.
>>>
>>> Totally agree. Before striking off on your own with a git interface, take
>>> a look at the work of Max Leske and Cami Bruni on FSGit[1]. Their project
>>> provides an in-image api for git.
>>>
>>> I think that both Cami and Max will be at ESUG and I plan on spending
>>> some time talking with them about how to integrate their work into with
>>> Metacello and FileTree...
>>>
>>>
>>> [1] https://github.com/dalehenrich/FSGit
>>
>>
>> My naive thinking was to just shell out and invoke the correct git command
>> line. Nothing fancy, just relying on my usual git command line tools.
>>
>> I'll have a peek at FSGit.
>>
>>>
>>> |
>>> |
>>> | - The API is nice in general.
>>> |
>>> |
>>> | - One thing that tends to be important to me is to be able to develop
>>> | several "parts" of my system at the same time. The different parts
>>> | live in different git repos. In this case that translates to
>>> | developing things coming from several ConfigOf's at the same time,
>>> | and several git repos. I think I'll be able to create development
>>> | specs that overrides github:// URIs to filetree:// URIs for the
>>> | various pieces that I am developing at the same time (obv can't be
>>> | done generically since relying on local path names, but still good
>>> | and teams can have conventions on where to check things out etc).
>>> | Then I can load things via Metacello, still have "2-way" ability to
>>> | save changes to some of the dependencies of the main project into
>>> | their filetree:// repos and manage the various git repos
>>> | independently. That's my next thing to try with the API, check that
>>> | such a workflow is feasible.[1]
>>>
>>> Two things:
>>>
>>>   - the configuration (ConfigurationOf) is intended for your end users
>>>     whether that be other developers or other projects of your own that
>>>     depend upon the project
>>>   - the baseline (BaselineOf) is intended for developers (just as you
>>>     have discovered)
>>>
>>> The way to use a baseline is exactly as you've described (perhaps the
>>> GettingStarted... doc should be updated to make this more clear).
>>> Create :filetree// repos for each of your git repositories and load the
>>> baseline directly:
>>>
>>>   Metacello image
>>>     baseline: 'Sample';
>>>     get.
>>>   Metacello image
>>>     baseline: 'Sample';
>>>     load.
>>>
>>> The `get` refreshes the BaselineOf in the event that it may have changed
>>> and the `load` refreshes your image with the changed packages. If you have
>>> multiple projects (baselines) then you use the following script:
>>>
>>>   Metacello image
>>>     baseline: #('Sample' 'Other');
>>>     get.
>>>   Metacello image
>>>     baseline: #('Sample' 'Other');
>>>     load.
>>>
>>> I only update the ConfigurationOf with a new SHA or tag when I'm ready
>>> for other folks/projects to use the code ... this makes the overhead of
>>> keeping a separate git repo for configs less onerous, because it doesn't
>>> change that frequently.
>>>
>>
>>
>> I was talking of bootstrapping my setup from just one configof/baselineof.
>> I.e. I check out say 3 git repos locally into well defined places (say
>> $HOME/pharo/projectA, $HOME/pharo/projectB, etc). A uses B and C. I load the
>> A baseline, which loads the B and C - from their filetree:// locations. Then
>> I go one doing my work and can commit/push/pull A, B and C as it makes
>> sense.
>>
>> To get going developing form a fresh image I don't want to load A, B and C
>> separately and I want to load them in such a way that they are setup for
>> development (i.e. filetree://). So my releases would point dependencies to
>> github, but my development would point to filetree if you see what I mean.
>> It'd be great to predefine both of those and just load one baseline and be
>> done. Later one one could see how no manual checkout is needed, it could be
>> done by Metacello for say a git:// URI. I couldn't do that with bundler.
>>
>>>
>>> |
>>> |
>>> | - It seems generally very useful to me to be able to specify
>>> | filetree:// URIs relative to home directory. I'll take a look at
>>> | that. Makes collaborating a bit easier if one can check in dev
>>> | versions referring to say "$HOME/development/projectA" rather than
>>> | "/home/xxx/development/projectA".
>>>
>>> I tend to use the :github in my project refs in configurations/baselines
>>> and I use the :filetree variant in the image for development only ... I also
>>> "centrally" locate all of my git repositories in `/opt/git` so that my
>>> various images and common scripts work (on my work and home computers)...so
>>> I guess in my case an absolute path the is constant for all images and
>>> working environments works well for me ...
>>>
>>> With that said, I don't want to discourage experiments ... as I've said
>>> before we're in the process of discovery here (I'm saying that to remind
>>> myself as much as anyone else:)
>>>
>>
>> Right. Comments above hopefully clarifies why I think it'd be neat to have
>> filetree:// (and later git://) in baselines for dev purposes.
>>
>> And then $HOME or similar is better to use if one wants to cater to people
>> that develop on machines without full permissions.
>>
>>>
>>> |
>>> |
>>> | My test repo is here: https://github.com/sundbp/PatriksImageTools
>>>
>>> Ah, you've based your work on Mariano's stuff[1]. I'll be interested if
>>> you find places where a) the scripting api _can_ be used and b) the
>>> scripting api _should_ be used
>>>
>>
>> Yes, it's completely "stolen" from Mariano. End goal is to take a fresh
>> pharo download and issue one command on the command line to initialize it as
>> a full development environment with all settings and stuff for a given
>> project. So far it just uses Metacello new ... load. and that does all I
>> need.
>>
>> I have some shell scripts that starts the VM with a minimal boot script
>> looking like:
>>
>> " first we set the HTTP proxy "
>> NetworkSystemSettings
>>   useHTTPProxy: true;
>>   httpProxyServer: '127.0.0.1';
>>   httpProxyPort: 8123.
>>
>> " next we get PatriksImageTools "
>> Metacello new
>>   configuration: 'PatriksImageTools';
>>   repository: 'github://sundbp/PatriksImageTools:master/packages';
>>   fetch. " Could use load as well, but it's done by my buildXXXImage later
>> so just doing fetch "
>>
>> ConfigurationOfPatriksImageTools buildCommanderImage
>>
>>>
>>> [1]
>>> http://marianopeck.wordpress.com/2011/11/19/loading-projects-and-building-your-own-images-with-metacello/
>>>
>>> Thanks for the good comments!
>>>
>>> Dale