feedback on using JSON to specify baselines for git repository

Frank,

Good point...

Dale

----- Original Message -----
| From: "Frank Shearar" <[hidden email]>
| To: [hidden email]
| Sent: Wednesday, February 8, 2012 1:31:36 PM
| Subject: Re: [Metacello] feedback on using JSON to specify baselines for git repository
|
| On 8 February 2012 21:23, Dale Henrichs <[hidden email]> wrote:
| > Stef,
| >
| > I can only think is that you must be _assuming_ that the literal
| > array is consumed by a parser that does not accept standard
| > smalltalk sytax, but is written to parse literal arrays of strings
| > and literal arrays.
| >
| > If you expect the Smalltalk compiler to compile and evaluate an
| > expression to produce an array from the literal array string, then
| > you are open to the security hole.
| >
| > If you postulate that someone were to write a portable literal
| > array parser, then we have come to an understanding...and I will
| > patiently wait for one to appear.
|
| There's another advantage to JSON over a Smalltalk literal array:
| loads of other languages have JSON parsers, so may parse these specs
| and do interesting things.
|
| (I do this with ruby and Maven's pom.xml files. It's really, really
| handy.)
|
| frank
|
| > Dale
| >
| > ----- Original Message -----
| > | From: "stephane ducasse" <[hidden email]>
| > | To: [hidden email]
| > | Sent: Wednesday, February 8, 2012 12:12:01 PM
| > | Subject: Re: [Metacello] feedback on using JSON to specify
| > | baselines for git repository
| > |
| > | >
| > | > If I expect to execute an expression using the literal array
| > | > then
| > | > it is a problem.
| > | >
| > | >  SmalltalkImage current shutdown.
| > | >  ^#()
| > | >
| > | > is still a problem if one is expected to EXECUTE the statements
| > | > whether it be a doit or a method or a string that is compiled
| > | > and
| > | > executed ... The security hole exists if one is expected to
| > | > EXECUTE a Smalltalk expression, because there is no way to
| > | > guarantee that the expression is safe.
| > | >
| > | > You do understand that the current mechanism we use in
| > | > Metacello is
| > | > to send the #project message to a class:
| > | >
| > | >  (Smalltalk at: #ConfigurationOfXXX) project
| > | >
| > | > and there are ZERO guarantees that the #project message won't
| > | > do
| > | > something nasty ... One cannot even load the code from an
| > | > arbitrary mcz file on a server, to create the class before
| > | > getting
| > | > a chance to send the #project message, because the #initialize
| > | > message is AUTOMATICALLY executed. As it stands today, there is
| > | > no
| > | > (safe) way for code executing on a server to reason about a
| > | > Metacello configration without executing arbitrary Smalltalk
| > | > code.
| > | >
| > | > Executing arbitrary Smalltalk expressions in a server is a
| > | > security
| > | > hole. It is the moral equivalent of giving a hacker a login on
| > | > your system and your server is not safe. I cannot run a server
| > | > exposed to the outside world in a VMWare data center that has
| > | > this
| > | > security hole.
| > |
| > | yes that I understand.
| > |
| > | but tell me why returning a JSON Strings would be any different
| > | than
| > | a literal array?
| > | because the problem is not the literal array of the JSON but the
| > | way
| > | you get you hand on it.
| > |
| > | If your literal array parser parsers a string then you should get
| > | a
| > | stream of literals.
| > | the fact that you parse
| > |       #(#baseline
| > |               #(project '3.6')
| > |               )
| > |
| > | will return a stream composed of literals.
| > | I think that this is exactly the same as for a JSON expression.
| > | Now if a hack add
| > |
| > |       SmalltalkImage killtherest
| > |       #(#baseline
| > |               #(project '3.6')
| > |               )
| > |
| > | then you semantics analysis will see that this is not an element:
| > | first not an array, second that SmalltalkImage is not
| > | an authorized keywords like baseline, repository or whatever.
| > |
| > | So the parsing is not the problem or there is something I do not
| > | understand.
| > |
| > |
| > |
| > |
| > | > If you would just acknowledge that this is true I will read the
| > | > rest of your email message ...
| > | >
| > | > If you don't acknowledge this point, there is no point in
| > | > trying to
| > | > talk about anything else on this subject.
| > | >
| > | > Dale
| > |
| > |
|

For me I tend to avoid to rely on other languages to manage my own system.
In gemstone you do not have a parser class?

Stef

Sure. After all, that's why we're working on Pharo/Squeak in the first place.

Sometimes a language can express a solution more clearly, or has
better library support, or whatever, and you can then solve the
problem at hand freeing you up to do more important work!

frank

On 9 February 2012 13:14, stephane ducasse <[hidden email]> wrote:

Yes I imagine.

Indeed
I was also thinking that chaining definition over multiple methods can be a problem
when you start to get a lot of versions.
>
>     Monticello doesn't have this problem, because it stores
>     the serialized meta model. It is trivial to have as many
>     different versions of the meta model for a package as one
>     needs.
>
>     With a structured file format like JSON, it becomes trivial
>     to load the Metacello version information for multiple
>     project versions into memory at once.

Yes if you do not have dependencies because if I understand correctly how metacello works
executing multiple method to get one configuration is a problem (cf above) and this is
why I tried to only have one baseline and one version.
>
>     For mcz-based ConfigurationOfXXX files, it is still an open
>     issue as to where to stash the structured (PARSABLE) data
>     file, but having a structured file format gives us one less
>     problem to solve in this area.

We want to add a class holding metadata (false positive for rules) so
this class could be the package to hold that but it would just be a replacement for a ConfigurationOfXXX.
In any case we need a way to store package metadata.

>
>  4. In a directory-based SCM (git or SVN) the entire directory
>     is versioned. Only the Metacello baseline specification
>     applies to a single version of the directory.

This is what I tried to understand when discussing with camillo but I'm still unclear on it.



I never understood why we need to keep the old baseline around.
Now I do not fully get why we would only need the baseline and not the version number.
Somewhere we need to have the hash of the commit or the tag.

> So the net-net is that a single baseline
>     specification is all that is needed as part of the directory.
>
>     It is true, though, that there is information like the symbolic
>     version information in the ConfigurationOf file that needs to
>     be versioned separately from the project packages.

I guess that (according to what I understood discussing with camillo) GIT tags solve this problem.

> This is the
>     meta information that tells you which git (or SVN) version you
>     should use for GemStone or Pharo1.1, or Squeak.

I think that there is a real difference between svn and git and that git is the way to go.

> This meta  information will still be kept in the new ConfigurationOf that
>     is versioned on a separate cycle than the embedded baseline
>     information.
>
>     In the end then, I will be using the JSON (or literal array)
>     format only to store the per project version baseline information
>     in a file currently called metacello.json[1].
>
> Does this help clear things up?

Kind of
Thanks for the time you spend writing it.
To really understand the point 4 which to me is the most important (because the rest is about syntax)
I guess that I will way that you show us an example.

to summarize my state of mind, as I said I'm not bound to MC or the way it is managed now.
I'm just a bit relunctant to lose the possibility to change merge semantics from inside our system but this is ok
too. My point is that if Git really support clever three way merging at the method level then why not.
I simply hope that it will not block us in the future if we want to invent something different.
I hope that Git will run on any platform. Some people think that we are idiot to want to have all our tools
in our system - the problem is that following this path we would end up coding in emacs and this can be boring.
I think that MC is really superior to SVN to manage our code but it seems that Git is superior.
Now what I see with GITFS is that it maps entities to file to get the right granularity because GIT is not
well done for Smalltalk. But this is ok if with some twists we can take advantage of the Git infrastructure.
Now I'm not convinced that I want to use Unix tools to manage my merges.


Stef

Nope, the compiler and parser are written in C...

Dale

----- Original Message -----
| From: "stephane ducasse" <[hidden email]>
| To: [hidden email]
| Sent: Thursday, February 9, 2012 5:14:57 AM
| Subject: Re: [Metacello] feedback on using JSON to specify baselines for git repository
|
| For me I tend to avoid to rely on other languages to manage my own
| system.
| In gemstone you do not have a parser class?
|
| Stef
|
|
|
|
|
|
| >
| > Dale
| >
| > ----- Original Message -----
| > | From: "Frank Shearar" <[hidden email]>
| > | To: [hidden email]
| > | Sent: Wednesday, February 8, 2012 1:31:36 PM
| > | Subject: Re: [Metacello] feedback on using JSON to specify
| > | baselines for git repository
| > |
| > | On 8 February 2012 21:23, Dale Henrichs <[hidden email]>
| > | wrote:
| > | > Stef,
| > | >
| > | > I can only think is that you must be _assuming_ that the
| > | > literal
| > | > array is consumed by a parser that does not accept standard
| > | > smalltalk sytax, but is written to parse literal arrays of
| > | > strings
| > | > and literal arrays.
| > | >
| > | > If you expect the Smalltalk compiler to compile and evaluate an
| > | > expression to produce an array from the literal array string,
| > | > then
| > | > you are open to the security hole.
| > | >
| > | > If you postulate that someone were to write a portable literal
| > | > array parser, then we have come to an understanding...and I
| > | > will
| > | > patiently wait for one to appear.
| > |
| > | There's another advantage to JSON over a Smalltalk literal array:
| > | loads of other languages have JSON parsers, so may parse these
| > | specs
| > | and do interesting things.
| > |
| > | (I do this with ruby and Maven's pom.xml files. It's really,
| > | really
| > | handy.)
| > |
| > | frank
| > |
| > | > Dale
| > | >
| > | > ----- Original Message -----
| > | > | From: "stephane ducasse" <[hidden email]>
| > | > | To: [hidden email]
| > | > | Sent: Wednesday, February 8, 2012 12:12:01 PM
| > | > | Subject: Re: [Metacello] feedback on using JSON to specify
| > | > | baselines for git repository
| > | > |
| > | > | >
| > | > | > If I expect to execute an expression using the literal
| > | > | > array
| > | > | > then
| > | > | > it is a problem.
| > | > | >
| > | > | >  SmalltalkImage current shutdown.
| > | > | >  ^#()
| > | > | >
| > | > | > is still a problem if one is expected to EXECUTE the
| > | > | > statements
| > | > | > whether it be a doit or a method or a string that is
| > | > | > compiled
| > | > | > and
| > | > | > executed ... The security hole exists if one is expected to
| > | > | > EXECUTE a Smalltalk expression, because there is no way to
| > | > | > guarantee that the expression is safe.
| > | > | >
| > | > | > You do understand that the current mechanism we use in
| > | > | > Metacello is
| > | > | > to send the #project message to a class:
| > | > | >
| > | > | >  (Smalltalk at: #ConfigurationOfXXX) project
| > | > | >
| > | > | > and there are ZERO guarantees that the #project message
| > | > | > won't
| > | > | > do
| > | > | > something nasty ... One cannot even load the code from an
| > | > | > arbitrary mcz file on a server, to create the class before
| > | > | > getting
| > | > | > a chance to send the #project message, because the
| > | > | > #initialize
| > | > | > message is AUTOMATICALLY executed. As it stands today,
| > | > | > there is
| > | > | > no
| > | > | > (safe) way for code executing on a server to reason about a
| > | > | > Metacello configration without executing arbitrary
| > | > | > Smalltalk
| > | > | > code.
| > | > | >
| > | > | > Executing arbitrary Smalltalk expressions in a server is a
| > | > | > security
| > | > | > hole. It is the moral equivalent of giving a hacker a login
| > | > | > on
| > | > | > your system and your server is not safe. I cannot run a
| > | > | > server
| > | > | > exposed to the outside world in a VMWare data center that
| > | > | > has
| > | > | > this
| > | > | > security hole.
| > | > |
| > | > | yes that I understand.
| > | > |
| > | > | but tell me why returning a JSON Strings would be any
| > | > | different
| > | > | than
| > | > | a literal array?
| > | > | because the problem is not the literal array of the JSON but
| > | > | the
| > | > | way
| > | > | you get you hand on it.
| > | > |
| > | > | If your literal array parser parsers a string then you should
| > | > | get
| > | > | a
| > | > | stream of literals.
| > | > | the fact that you parse
| > | > |       #(#baseline
| > | > |               #(project '3.6')
| > | > |               )
| > | > |
| > | > | will return a stream composed of literals.
| > | > | I think that this is exactly the same as for a JSON
| > | > | expression.
| > | > | Now if a hack add
| > | > |
| > | > |       SmalltalkImage killtherest
| > | > |       #(#baseline
| > | > |               #(project '3.6')
| > | > |               )
| > | > |
| > | > | then you semantics analysis will see that this is not an
| > | > | element:
| > | > | first not an array, second that SmalltalkImage is not
| > | > | an authorized keywords like baseline, repository or whatever.
| > | > |
| > | > | So the parsing is not the problem or there is something I do
| > | > | not
| > | > | understand.
| > | > |
| > | > |
| > | > |
| > | > |
| > | > | > If you would just acknowledge that this is true I will read
| > | > | > the
| > | > | > rest of your email message ...
| > | > | >
| > | > | > If you don't acknowledge this point, there is no point in
| > | > | > trying to
| > | > | > talk about anything else on this subject.
| > | > | >
| > | > | > Dale
| > | > |
| > | > |
| > |
|
|

stephane ducasse wrote:
> Now I'm not convinced that I want to use Unix tools to manage my merges.
>  
Down the track you might not need to to ...
http://developer.github.com/v3/git/
http://develop.github.com/p/libraries.html

One side benefit to using git is that it lowers the perceived difficulty
of open source developers from other languages transferring to
Smalltalk.  The best way to lead people into something new for something
familiar to also be there.

You could even go so far as to turn this right around.  If Pharo ends up
developing good git api client gui (eg http://www.git-tower.com/) for
our own use, then others might find it useful to manage their
non-Smalltalk projects.  Some of the perceived issues people consider
against Smalltalk for deployable projects don't apply to internal tools
for managing code.  So once they are a user of this tool, they then need
to add a small modification, and once the initial "barrier of the
unknown" is broken down you've then got another person hooked. /*wishful
thinking*/

cheers, -ben

comments embedded below...

----- Original Message -----
| From: "stephane ducasse" <[hidden email]>
| To: [hidden email]
| Sent: Thursday, February 9, 2012 9:44:13 AM
| Subject: Re: [Metacello] feedback on using JSON to specify baselines for git repository
|
|
| > BTW, I consider the current method-based format to be declarative
| > syntax as well, but I digress...
| >
| > To start with, I covered most of the reasons for considering using
| > a structured file format (as opposed to a Smalltalk source format)
| > in my first post and it was the sum of all of these issues that
| > lead me to consider JSON:
| >
| >  1. The security issue, of course.
| >
| >  2. Last Friday, I hit the 255 literal limit for a method
| >     in the Seaside30 3.0.6 version method. A year ago I hit
| >     the literal limit for the 3.0.0-alpha5.5-baseline
| >     specification. The implication is that using method-
| >     based or doit-based specifications will not scale and
| >     in the case of Seaside30, has already hit the limits.
|
| Yes I imagine.
|
| >  3. When doing a merge of a configuration it is necessary
| >     to have the Metacello project information for three
| >     different versions of the ConfigurationOfXXX class:
| >     common ancestor, configuration version A, and
| >     configuration version B (just like Monticello).
| >     Unfortunately, with the current system that necessitates
| >     loading each version of the ConfigurationOfXXX class,
| >     executing the #project method and then stashing the result
| >     while going about loading the other versions of the class.
| >     If we are LOADING classes what happens if the class has
| >     been modified in the current image, how does one get the
| >     other two versions of the classes needed to perform the
| >     merge?
|
| Indeed
| I was also thinking that chaining definition over multiple methods
| can be a problem
| when you start to get a lot of versions.

Yes. the #version:imports: pragma can be used to chain multiple methods together but it is not a clean way to do things, especially when the tools are expected to try to make sense of these "synthetic versions" ... on version / method is definitely the preferred technique ...

| >
| >     Monticello doesn't have this problem, because it stores
| >     the serialized meta model. It is trivial to have as many
| >     different versions of the meta model for a package as one
| >     needs.
| >
| >     With a structured file format like JSON, it becomes trivial
| >     to load the Metacello version information for multiple
| >     project versions into memory at once.
|
| Yes if you do not have dependencies because if I understand correctly
| how metacello works
| executing multiple method to get one configuration is a problem (cf
| above) and this is
| why I tried to only have one baseline and one version.

agreed.

| >
| >     For mcz-based ConfigurationOfXXX files, it is still an open
| >     issue as to where to stash the structured (PARSABLE) data
| >     file, but having a structured file format gives us one less
| >     problem to solve in this area.
|
| We want to add a class holding metadata (false positive for rules) so
| this class could be the package to hold that but it would just be a
| replacement for a ConfigurationOfXXX.
| In any case we need a way to store package metadata.

One idea that I've entertained is to add an additional "directory" to the mcz file to contain Metacello meta data for the ConfigurationOfXXX packages...but I'm not sure that this technique would work in the wild ...

|
| >
| >  4. In a directory-based SCM (git or SVN) the entire directory
| >     is versioned. Only the Metacello baseline specification
| >     applies to a single version of the directory.
|
| This is what I tried to understand when discussing with camillo but
| I'm still unclear on it.

Yes this is one of the things that I'm still wrapping my brain around as I continue to work through the issues ...

|
|
|
| > The directory
| >     will contain the correct version of each package for the
| >     project, so there is no need to specify any package
| >     versions - only the name is needed and the name is already
| >     in the baseline. You need package dependencies and required
| >     projects, but these are also specified in the baseline. If
| >     you add or remove a package, you will edit the baseline to
| >     reflect the changes to the structure of the project, then
| >     when you commit there is no need to keep the old baseline
| >     version around.
|
| I never understood why we need to keep the old baseline around.
| Now I do not fully get why we would only need the baseline and not
| the version number.
| Somewhere we need to have the hash of the commit or the tag.

This is actually one of the "sticky wickets" that I'm having to think about right now ... [I can't think about svn and git at the same time for this level of problem, so I'm just thining about git now].... at the moment my working hypothesis is that you will add a Monticello repository for git that includes the name and SHA for the commit (or tag or branch name or ...) so one will be opening a version of a Monticello repository ... you won't store more than one version of package in a git repository, so there will only be one "mcz file" per package in the repository.

I think this model will actually work, but I have to work through the implications for Metacello ... I have some ideas, but nothing concrete yet ....


|
| > So the net-net is that a single baseline
| >     specification is all that is needed as part of the directory.
| >
| >     It is true, though, that there is information like the symbolic
| >     version information in the ConfigurationOf file that needs to
| >     be versioned separately from the project packages.
|
| I guess that (according to what I understood discussing with camillo)
| GIT tags solve this problem.

At one level, I agree. With git you can tag a particular commit as '1.0-stable-Pharo1.3', another as '1.1-stable-Pharo1.3', and another commit as '1.0-stable-GemStone', etc. However, I still think that stable versions will have a place to answer the question "which version of the project should I use on Pharo1.3?"....In the absence of enforced naming conventions for tags, I think that we will always require some level of mapping (symbolic versions are an enforced naming convention)...

|
| > This is the
| >     meta information that tells you which git (or SVN) version you
| >     should use for GemStone or Pharo1.1, or Squeak.
|
| I think that there is a real difference between svn and git and that
| git is the way to go.

I agree that git should be the focus for enhanced tool support from within an image (ala FSGit), but the filetree repository format (which I am thinking we'll use for git) can be used with SVN or any other directory-based SCM (i.e., cvs).

|
| > This meta  information will still be kept in the new
| > ConfigurationOf that
| >     is versioned on a separate cycle than the embedded baseline
| >     information.
| >
| >     In the end then, I will be using the JSON (or literal array)
| >     format only to store the per project version baseline
| >     information
| >     in a file currently called metacello.json[1].
| >
| > Does this help clear things up?
|
| Kind of
| Thanks for the time you spend writing it.
| To really understand the point 4 which to me is the most important
| (because the rest is about syntax)
| I guess that I will way that you show us an example.

That is the particular problem that I have set for myself and I have made good progress. I am a day or two away from hooking things up with FSGit which will be a nice milestone...

|
| to summarize my state of mind, as I said I'm not bound to MC or the
| way it is managed now.
| I'm just a bit relunctant to lose the possibility to change merge
| semantics from inside our system but this is ok
| too. My point is that if Git really support clever three way merging
| at the method level then why not.
| I simply hope that it will not block us in the future if we want to
| invent something different.
| I hope that Git will run on any platform. Some people think that we
| are idiot to want to have all our tools
| in our system - the problem is that following this path we would end
| up coding in emacs and this can be boring.
| I think that MC is really superior to SVN to manage our code but it
| seems that Git is superior.
| Now what I see with GITFS is that it maps entities to file to get the
| right granularity because GIT is not
| well done for Smalltalk. But this is ok if with some twists we can
| take advantage of the Git infrastructure.
| Now I'm not convinced that I want to use Unix tools to manage my
| merges.

Agreed.... My basic thought process has been that the git branching model[1] (this is a nice article with good graphics) is very nice and maps well to distributed open source development projects. Branches are not first-class objects in Monticello and I think that that makes using and managing branches very difficult with Monticello. OTOH git embraces the notion of branches and very complex branch trees can actually be managed effectively ...

FSGit provides direct access to the git database, which means that over time, sophisticated git tools can be built in Smalltalk, so eventually we can have our cake and eat it too.

|
|
| Stef

Dale

[1] http://nvie.com/posts/a-successful-git-branching-model/

On Feb 9, 2012, at 7:12 PM, Dale Henrichs wrote:

> Nope, the compiler and parser are written in C…

amazing…
I did not know that.

>
> Yes. the #version:imports: pragma can be used to chain multiple methods together but it is not a clean way to do things, especially when the tools are expected to try to make sense of these "synthetic versions" ... on version / method is definitely the preferred technique …

Yes this is why I always avoided it because I always felt like I do not want to execute multiple methods (since I need two in fact current baseline and versions max per stable/dev…).

| We want to add a class holding metadata (false positive for rules) so
> | this class could be the package to hold that but it would just be a
> | replacement for a ConfigurationOfXXX.
> | In any case we need a way to store package metadata.
>
> One idea that I've entertained is to add an additional "directory" to the mcz file to contain Metacello meta data for the ConfigurationOfXXX packages...but I'm not sure that this technique would work in the wild …

This is why we opted for something ortho to MC = one class with pure metadata.
We are really thinking about how to handle metadata in general (at various levels package/class…) and so far
this is ok. We will continue to experiment with it.

> | I never understood why we need to keep the old baseline around.
> | Now I do not fully get why we would only need the baseline and not
> | the version number.
> | Somewhere we need to have the hash of the commit or the tag.
>
> This is actually one of the "sticky wickets" that I'm having to think about right now ... [I can't think about svn and git at the same time for this level of problem, so I'm just thining about git now].... at the moment my working hypothesis is that you will add a Monticello repository for git that includes the name and SHA for the commit (or tag or branch name or ...) so one will be opening a version of a Monticello repository ... you won't store more than one version of package in a git repository, so there will only be one "mcz file" per package in the repository.
>
> I think this model will actually work, but I have to work through the implications for Metacello ... I have some ideas, but nothing concrete yet ….

Ok I think that camillo has some ideas on that. I should sit with him one of these days but he should focus on his phd :).

> At one level, I agree. With git you can tag a particular commit as '1.0-stable-Pharo1.3', another as '1.1-stable-Pharo1.3', and another commit as '1.0-stable-GemStone', etc. However, I still think that stable versions will have a place to answer the question "which version of the project should I use on Pharo1.3?"....In the absence of enforced naming conventions for tags, I think that we will always require some level of mapping (symbolic versions are an enforced naming convention)…

yes

>
> I agree that git should be the focus for enhanced tool support from within an image (ala FSGit), but the filetree repository format (which I am thinking we'll use for git) can be used with SVN or any other directory-based SCM (i.e., cvs).

ok I have no idea about file tree but no need to explain. I will learn it when needed.

----- Original Message -----
| From: "stephane ducasse" <[hidden email]>
| To: [hidden email]
| Sent: Friday, February 10, 2012 12:04:32 PM
| Subject: Re: [Metacello] feedback on using JSON to specify baselines for git repository
|
| >
| > Yes. the #version:imports: pragma can be used to chain multiple
| > methods together but it is not a clean way to do things,
| > especially when the tools are expected to try to make sense of
| > these "synthetic versions" ... on version / method is definitely
| > the preferred technique …
|
| Yes this is why I always avoided it because I always felt like I do
| not want to execute multiple methods (since I need two in fact
| current baseline and versions max per stable/dev…).
|
| | We want to add a class holding metadata (false positive for rules)
| | so
| > | this class could be the package to hold that but it would just be
| > | a
| > | replacement for a ConfigurationOfXXX.
| > | In any case we need a way to store package metadata.
| >
| > One idea that I've entertained is to add an additional "directory"
| > to the mcz file to contain Metacello meta data for the
| > ConfigurationOfXXX packages...but I'm not sure that this technique
| > would work in the wild …
|
| This is why we opted for something ortho to MC = one class with pure
| metadata.
| We are really thinking about how to handle metadata in general (at
| various levels package/class…) and so far
| this is ok. We will continue to experiment with it.

Is this the RING work or something else?

|
| > | I never understood why we need to keep the old baseline around.
| > | Now I do not fully get why we would only need the baseline and
| > | not
| > | the version number.
| > | Somewhere we need to have the hash of the commit or the tag.
| >
| > This is actually one of the "sticky wickets" that I'm having to
| > think about right now ... [I can't think about svn and git at the
| > same time for this level of problem, so I'm just thining about git
| > now].... at the moment my working hypothesis is that you will add
| > a Monticello repository for git that includes the name and SHA for
| > the commit (or tag or branch name or ...) so one will be opening a
| > version of a Monticello repository ... you won't store more than
| > one version of package in a git repository, so there will only be
| > one "mcz file" per package in the repository.
| >
| > I think this model will actually work, but I have to work through
| > the implications for Metacello ... I have some ideas, but nothing
| > concrete yet ….
|
| Ok I think that camillo has some ideas on that. I should sit with him
| one of these days but he should focus on his phd :).

I would like to hear what Camillo is thinking about as well ... any level of detail is fine ... now is the time to bring the interesting ideas into the open ... even if an idea is not used immediately it is important to consider the implications so that we can keep our options open ...

The "house" is just being built and at this point in time, I have no qualms about kicking the house down and starting over from scratch, that's why I'm soliciting feedback now rather than waiting until I have a "finished product" ... I am feeling my way along and learning both about git and how to integrate it into the existing systems we have with as little  change as possible ... I am also using the system to develop the code... I have been committing my work using git for about the last week week now and as of yesterday I am able to use Metacello scripts to load from the git repository (using filetree not fsgit at the moment). As a result I can see and feel how things ought to work as opposed to what I have to do right now:) and I'm using that to base my next round of ideas/development ...

|
| > At one level, I agree. With git you can tag a particular commit as
| > '1.0-stable-Pharo1.3', another as '1.1-stable-Pharo1.3', and
| > another commit as '1.0-stable-GemStone', etc. However, I still
| > think that stable versions will have a place to answer the
| > question "which version of the project should I use on
| > Pharo1.3?"....In the absence of enforced naming conventions for
| > tags, I think that we will always require some level of mapping
| > (symbolic versions are an enforced naming convention)…
|
| yes
|
| >
| > I agree that git should be the focus for enhanced tool support from
| > within an image (ala FSGit), but the filetree repository format
| > (which I am thinking we'll use for git) can be used with SVN or
| > any other directory-based SCM (i.e., cvs).
|
| ok I have no idea about file tree but no need to explain. I will
| learn it when needed.
|
| > That is the particular problem that I have set for myself and I
| > have made good progress. I am a day or two away from hooking
| > things up with FSGit which will be a nice milestone...
| >
| >
| > Agreed.... My basic thought process has been that the git branching
| > model[1] (this is a nice article with good graphics) is very nice
| > and maps well to distributed open source development projects.
| > Branches are not first-class objects in Monticello and I think
| > that that makes using and managing branches very difficult with
| > Monticello. OTOH git embraces the notion of branches and very
| > complex branch trees can actually be managed effectively ...
| >
| > FSGit provides direct access to the git database, which means that
| > over time, sophisticated git tools can be built in Smalltalk, so
| > eventually we can have our cake and eat it too.
| >
| > |
| > |
| > | Stef
| >
| > Dale
| >
| > [1] http://nvie.com/posts/a-successful-git-branching-model/
|
|

>
> | This is why we opted for something ortho to MC = one class with pure
> | metadata.
> | We are really thinking about how to handle metadata in general (at
> | various levels package/class…) and so far
> | this is ok. We will continue to experiment with it.
>
> Is this the RING work or something else?

Something else. Ring is for declarative manipulation of code (to replace pseudo class, methodReference) to be able to browse remote code….
Build better tools: right now Nautilus can manipulate real class or off image classes.
Now what we need is a way to say which are the false positive when running small lint (in fact funnily this topic was raised in the VW mailing-list too).
this tagging is important if we want to have a certification process of package quality for real.
so now we add per package a ManifestXXX class and put metadata for SmallLint. I imagine that other tools could benefit from it.

We will do some real use of that before pushing it in the system.
I want to run SmallLint on the complete image and go over all the warning one by one.


> | one of these days but he should focus on his phd :).
>
> I would like to hear what Camillo is thinking about as well ... any level of detail is fine ... now is the time to bring the interesting ideas into the open ... even if an idea is not used immediately it is important to consider the implications so that we can keep our options open …

yes may be you should have a Skype meeting.

> The "house" is just being built and at this point in time, I have no qualms about kicking the house down and starting over from scratch, that's why I'm soliciting feedback now rather than waiting until I have a "finished product" ... I am feeling my way along and learning both about git and how to integrate it into the existing systems we have with as little  change as possible ... I am also using the system to develop the code... I have been committing my work using git for about the last week week now and as of yesterday I am able to use Metacello scripts to load from the git repository (using filetree not fsgit at the moment). As a result I can see and feel how things ought to work as opposed to what I have to do right now:) and I'm using that to base my next round of ideas/development ...

I have a though head but I try to listen :) and camillo convinced me to retry git for papers :) and that this is a good idea for Pharo
so I'm curious to see that happening.

Stef

Ah, on the one hand, I am thinking that your MetaData work should be folded into Metacello, but my head explodes immediately after:)

Keep in mind that the directory structure for git Monticello packages (filetree) is very flexible. It is versioned so each package can have a different structure if necessary, so new information can be introduced without much pain (at least so far:)...

Yes skype would be a good idea ... I'd need a days warning to be sure that I bring my laptop into work when I want to use Skype...

By allowing developers to choose git if they want, I think there will be lots of benefits beyond the purely technical improvements in our own development processes...

Dale

----- Original Message -----
| From: "stephane ducasse" <[hidden email]>
| To: [hidden email]
| Cc: "Camillo Bruni" <[hidden email]>
| Sent: Friday, February 10, 2012 12:42:56 PM
| Subject: Re: [Metacello] feedback on using JSON to specify baselines for git repository
|
| >
| > | This is why we opted for something ortho to MC = one class with
| > | pure
| > | metadata.
| > | We are really thinking about how to handle metadata in general
| > | (at
| > | various levels package/class…) and so far
| > | this is ok. We will continue to experiment with it.
| >
| > Is this the RING work or something else?
|
| Something else. Ring is for declarative manipulation of code (to
| replace pseudo class, methodReference) to be able to browse remote
| code….
| Build better tools: right now Nautilus can manipulate real class or
| off image classes.
| Now what we need is a way to say which are the false positive when
| running small lint (in fact funnily this topic was raised in the VW
| mailing-list too).
| this tagging is important if we want to have a certification process
| of package quality for real.
| so now we add per package a ManifestXXX class and put metadata for
| SmallLint. I imagine that other tools could benefit from it.
|
| We will do some real use of that before pushing it in the system.
| I want to run SmallLint on the complete image and go over all the
| warning one by one.
|
|
| > | one of these days but he should focus on his phd :).
| >
| > I would like to hear what Camillo is thinking about as well ... any
| > level of detail is fine ... now is the time to bring the
| > interesting ideas into the open ... even if an idea is not used
| > immediately it is important to consider the implications so that
| > we can keep our options open …
|
| yes may be you should have a Skype meeting.
|
| > The "house" is just being built and at this point in time, I have
| > no qualms about kicking the house down and starting over from
| > scratch, that's why I'm soliciting feedback now rather than
| > waiting until I have a "finished product" ... I am feeling my way
| > along and learning both about git and how to integrate it into the
| > existing systems we have with as little  change as possible ... I
| > am also using the system to develop the code... I have been
| > committing my work using git for about the last week week now and
| > as of yesterday I am able to use Metacello scripts to load from
| > the git repository (using filetree not fsgit at the moment). As a
| > result I can see and feel how things ought to work as opposed to
| > what I have to do right now:) and I'm using that to base my next
| > round of ideas/development ...
|
| I have a though head but I try to listen :) and camillo convinced me
| to retry git for papers :) and that this is a good idea for Pharo
| so I'm curious to see that happening.
|
| Stef
|
|

> Ah, on the one hand, I am thinking that your MetaData work should be folded into Metacello, but my head explodes immediately after:)

I was thinking about that too. May be metacello declarative data can just be hold on the ManifestConfigurationOfXXX class.

Is there a project where I can look at examples of the manifest-style classes?

----- Original Message -----
| From: "stephane ducasse" <[hidden email]>
| To: [hidden email]
| Cc: "Camillo Bruni" <[hidden email]>
| Sent: Friday, February 10, 2012 1:02:00 PM
| Subject: Re: [Metacello] feedback on using JSON to specify baselines for git repository
|
| > Ah, on the one hand, I am thinking that your MetaData work should
| > be folded into Metacello, but my head explodes immediately after:)
|
| I was thinking about that too. May be metacello declarative data can
| just be hold on the ManifestConfigurationOfXXX class.
|
| > Keep in mind that the directory structure for git Monticello
| > packages (filetree) is very flexible. It is versioned so each
| > package can have a different structure if necessary, so new
| > information can be introduced without much pain (at least so
| > far:)...
| >
| > Yes skype would be a good idea ... I'd need a days warning to be
| > sure that I bring my laptop into work when I want to use Skype...
| >
| > By allowing developers to choose git if they want, I think there
| > will be lots of benefits beyond the purely technical improvements
| > in our own development processes...
| >
| > Dale
| >
| > ----- Original Message -----
| > | From: "stephane ducasse" <[hidden email]>
| > | To: [hidden email]
| > | Cc: "Camillo Bruni" <[hidden email]>
| > | Sent: Friday, February 10, 2012 12:42:56 PM
| > | Subject: Re: [Metacello] feedback on using JSON to specify
| > | baselines for git repository
| > |
| > | >
| > | > | This is why we opted for something ortho to MC = one class
| > | > | with
| > | > | pure
| > | > | metadata.
| > | > | We are really thinking about how to handle metadata in
| > | > | general
| > | > | (at
| > | > | various levels package/class…) and so far
| > | > | this is ok. We will continue to experiment with it.
| > | >
| > | > Is this the RING work or something else?
| > |
| > | Something else. Ring is for declarative manipulation of code (to
| > | replace pseudo class, methodReference) to be able to browse
| > | remote
| > | code….
| > | Build better tools: right now Nautilus can manipulate real class
| > | or
| > | off image classes.
| > | Now what we need is a way to say which are the false positive
| > | when
| > | running small lint (in fact funnily this topic was raised in the
| > | VW
| > | mailing-list too).
| > | this tagging is important if we want to have a certification
| > | process
| > | of package quality for real.
| > | so now we add per package a ManifestXXX class and put metadata
| > | for
| > | SmallLint. I imagine that other tools could benefit from it.
| > |
| > | We will do some real use of that before pushing it in the system.
| > | I want to run SmallLint on the complete image and go over all the
| > | warning one by one.
| > |
| > |
| > | > | one of these days but he should focus on his phd :).
| > | >
| > | > I would like to hear what Camillo is thinking about as well ...
| > | > any
| > | > level of detail is fine ... now is the time to bring the
| > | > interesting ideas into the open ... even if an idea is not used
| > | > immediately it is important to consider the implications so
| > | > that
| > | > we can keep our options open …
| > |
| > | yes may be you should have a Skype meeting.
| > |
| > | > The "house" is just being built and at this point in time, I
| > | > have
| > | > no qualms about kicking the house down and starting over from
| > | > scratch, that's why I'm soliciting feedback now rather than
| > | > waiting until I have a "finished product" ... I am feeling my
| > | > way
| > | > along and learning both about git and how to integrate it into
| > | > the
| > | > existing systems we have with as little  change as possible ...
| > | > I
| > | > am also using the system to develop the code... I have been
| > | > committing my work using git for about the last week week now
| > | > and
| > | > as of yesterday I am able to use Metacello scripts to load from
| > | > the git repository (using filetree not fsgit at the moment). As
| > | > a
| > | > result I can see and feel how things ought to work as opposed
| > | > to
| > | > what I have to do right now:) and I'm using that to base my
| > | > next
| > | > round of ideas/development ...
| > |
| > | I have a though head but I try to listen :) and camillo convinced
| > | me
| > | to retry git for papers :) and that this is a good idea for Pharo
| > | so I'm curious to see that happening.
| > |
| > | Stef
| > |
| > |
|
|

Simon where do you publish Manifest code?

Stef

On Feb 10, 2012, at 10:41 PM, Dale Henrichs wrote:

but where do you publish your code?
And reply to all!!!!
Stef