Source code management - my vision. (Was: Re: Why a package management system)

> Hi!
>
> Germán Arduino wrote:
>> Hi Göran:
>>
>> Sorry by my intromision here, but want to say somethings :)
>>
>> 2010/2/23 Göran Krampe <[hidden email] <mailto:[hidden email]>>
> [SNIP]
>>     What I have seen so far in Metacello is indeed nothing new - the
>>     senarios and discussions about dependencies etc go loooong way back.
>>
>>
>> I'm on Squeak from old 3.2 times. And yes, a lot of discussions about
>> packages,
>> etc, but only discussions on most of cases. I think Metacello solves the
>> dependencies
>> questions very well and the other tools not (with exception of Universes
>> probably, but
>> abandoned now). Also Metacello provides all the features explained on
>> the excellent
>> mail of Torsten, assuring that the configurations will work with the
>> time not breaking
>> stuffs.
>>
>> The solution of using Metacello+Monticello is, imho, very simple, easy
>> and productive
>> and is here, available to us. :)
>
> I never claimed otherwise! I am just saying that there is nothing new -
> and frankly, though I admit not having read carefully, I didn't find the
> example so... impressive. Perhaps I need to read it again.
>
> Again though - I am not saying anything negative about Metacello.
>
> [SNIP]
>>     Anyway, "common exchange mechanism" - I am working on Deltastreams which
>>     I even presented at Brest, though unfortunately colliding with the
>>     Seaside tutorial. Deltas and Deltastreams are indeed focused on EXACTLY
>>     this problem (fork interchange), and it was born in 2007:
>>
>>     http://lists.squeakfoundation.org/pipermail/squeak-dev/2007-August/119471.html
>>
>>     I and Igor are the ones working on it - and after Brest we have had very
>>     little time to move it forward. I still think it is a very important
>>     piece of the puzzle though.
>>
>> But Deltastreams are ready to use on production systems?
>
> Nope, not yet, although very close. But since the topic came up I wanted
> to mention it. But AFAIK there is no other tool/project aimed at that
> use case. Again, I may be wrong :)
>
> regards, Göran
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

> Since Goran mentioned DeltaStreams, i'd like to share one of my ideas,
> which came into my mind, when i complete coding the method trailers.
>
> I made a method trailers open for future extensions.
> A first step would be to introduce an abstract 'source management'
> layer, which will be responsible for delivering
> a method's and classes sources from some abstract data source. The
> next step is actually implement such data sources.
>
> So, the idea is to abandon hardwiring with .sources and .changes files.
> What new opportunities it gives to us?
> Suppose that instead of storing sources into file, you putting them
> directly to database , such as CouchDB (yes! :).
>
> Now if you imagine that anyone could access to this database on some
> public location, this means that we automagically made a source code
> exchange hub, and could do a very cool things with it.
>
> Suppose that ALL source code is put into a single database. Now
> different users creating own accounts and makind own 'forks' from
> original. What we getting is a number of update streams (methods,
> doits etc etc), which lying in those databases,
> and if one user wants to load/merge code from one of the sources he
> simply points to it and it get loaded into his image.
> We could organize things very closely to what github currently is:
> everyone could create any number of new projects, or forks of existing
> ones. Freely exchange the code, do a cherry picking etc etc.
> And what is most important, that with such organization you can track
> any methods, classes in the image and their evolutionary history back
> to its origin. As well as, you can query database, how an when a
> concrete method/class get deleted (everything is recorded, not just
> current state).
>
> Also, it would allow us to manipulate the code at the lowest level of
> granularity (individual methods/classes/doits, same as DS does), not
> at a package level.
> There are a lot of synergy with DeltaStreams - it could serve as an
> 'offline' format, in same way as git storing the files locally, but
> can reconnect with remote storage (which can be a public database).
> One of the things, which could be made is to allow stashing/popping
> code from stash. Suppose two projects started from a single starting
> point (Squeak 3.9). Now suppose that by clicking a single button you
> could get either trunk image, or Pharo 1.0 image.
> But that's not all! With DeltaStreams, you could then click once more,
> and revert things back from , say Pharo 1.0 to trunk or vice versa :)
> Of course it would be stupid to do things at such large scale, but if
> we put some common 'root' stuff into database, then we could very
> easily track all changes & compare them. This would make cherry
> picking and code exchange much more easier.
>
> This could get us on the next level of code management and system
> organization and organization of community efforts.
>
>
>
> 2010/2/23 Göran Krampe <[hidden email]>:
>> Hi!
>>
>> Germán Arduino wrote:
>>> Hi Göran:
>>>
>>> Sorry by my intromision here, but want to say somethings :)
>>>
>>> 2010/2/23 Göran Krampe <[hidden email] <mailto:[hidden email]>>
>> [SNIP]
>>> What I have seen so far in Metacello is indeed nothing new - the
>>> senarios and discussions about dependencies etc go loooong way back.
>>>
>>>
>>> I'm on Squeak from old 3.2 times. And yes, a lot of discussions about
>>> packages,
>>> etc, but only discussions on most of cases. I think Metacello solves the
>>> dependencies
>>> questions very well and the other tools not (with exception of Universes
>>> probably, but
>>> abandoned now). Also Metacello provides all the features explained on
>>> the excellent
>>> mail of Torsten, assuring that the configurations will work with the
>>> time not breaking
>>> stuffs.
>>>
>>> The solution of using Metacello+Monticello is, imho, very simple, easy
>>> and productive
>>> and is here, available to us. :)
>>
>> I never claimed otherwise! I am just saying that there is nothing new -
>> and frankly, though I admit not having read carefully, I didn't find the
>> example so... impressive. Perhaps I need to read it again.
>>
>> Again though - I am not saying anything negative about Metacello.
>>
>> [SNIP]
>>> Anyway, "common exchange mechanism" - I am working on Deltastreams which
>>> I even presented at Brest, though unfortunately colliding with the
>>> Seaside tutorial. Deltas and Deltastreams are indeed focused on EXACTLY
>>> this problem (fork interchange), and it was born in 2007:
>>>
>>> http://lists.squeakfoundation.org/pipermail/squeak-dev/2007-August/119471.html
>>>
>>> I and Igor are the ones working on it - and after Brest we have had very
>>> little time to move it forward. I still think it is a very important
>>> piece of the puzzle though.
>>>
>>> But Deltastreams are ready to use on production systems?
>>
>> Nope, not yet, although very close. But since the topic came up I wanted
>> to mention it. But AFAIK there is no other tool/project aimed at that
>> use case. Again, I may be wrong :)
>>
>> regards, Göran
>>
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project 

> Some kind of graph-based GUI would certainly help in this scenario to
> visualise the dependencies/changes... just a thought.
>
> Regards, Gary
>
> ----- Original Message -----
> From: "Igor Stasenko" <[hidden email]>
> To: <[hidden email]>
> Sent: Tuesday, February 23, 2010 4:02 PM
> Subject: [Pharo-project] Source code management - my vision. (Was:  
> Re: Why
> apackage management system)
>
>
>> Since Goran mentioned DeltaStreams, i'd like to share one of my  
>> ideas,
>> which came into my mind, when i complete coding the method trailers.
>>
>> I made a method trailers open for future extensions.
>> A first step would be to introduce an abstract 'source management'
>> layer, which will be responsible for delivering
>> a method's and classes sources from some abstract data source. The
>> next step is actually implement such data sources.
>>
>> So, the idea is to abandon hardwiring with .sources and .changes  
>> files.
>> What new opportunities it gives to us?
>> Suppose that instead of storing sources into file, you putting them
>> directly to database , such as CouchDB (yes! :).
>>
>> Now if you imagine that anyone could access to this database on some
>> public location, this means that we automagically made a source code
>> exchange hub, and could do a very cool things with it.
>>
>> Suppose that ALL source code is put into a single database. Now
>> different users creating own accounts and makind own 'forks' from
>> original. What we getting is a number of update streams (methods,
>> doits etc etc), which lying in those databases,
>> and if one user wants to load/merge code from one of the sources he
>> simply points to it and it get loaded into his image.
>> We could organize things very closely to what github currently is:
>> everyone could create any number of new projects, or forks of  
>> existing
>> ones. Freely exchange the code, do a cherry picking etc etc.
>> And what is most important, that with such organization you can track
>> any methods, classes in the image and their evolutionary history back
>> to its origin. As well as, you can query database, how an when a
>> concrete method/class get deleted (everything is recorded, not just
>> current state).
>>
>> Also, it would allow us to manipulate the code at the lowest level of
>> granularity (individual methods/classes/doits, same as DS does), not
>> at a package level.
>> There are a lot of synergy with DeltaStreams - it could serve as an
>> 'offline' format, in same way as git storing the files locally, but
>> can reconnect with remote storage (which can be a public database).
>> One of the things, which could be made is to allow stashing/popping
>> code from stash. Suppose two projects started from a single starting
>> point (Squeak 3.9). Now suppose that by clicking a single button you
>> could get either trunk image, or Pharo 1.0 image.
>> But that's not all! With DeltaStreams, you could then click once  
>> more,
>> and revert things back from , say Pharo 1.0 to trunk or vice versa :)
>> Of course it would be stupid to do things at such large scale, but if
>> we put some common 'root' stuff into database, then we could very
>> easily track all changes & compare them. This would make cherry
>> picking and code exchange much more easier.
>>
>> This could get us on the next level of code management and system
>> organization and organization of community efforts.
>>
>>
>>
>> 2010/2/23 Göran Krampe <[hidden email]>:
>>> Hi!
>>>
>>> Germán Arduino wrote:
>>>> Hi Göran:
>>>>
>>>> Sorry by my intromision here, but want to say somethings :)
>>>>
>>>> 2010/2/23 Göran Krampe <[hidden email] <mailto:[hidden email]>>
>>> [SNIP]
>>>> What I have seen so far in Metacello is indeed nothing new - the
>>>> senarios and discussions about dependencies etc go loooong way  
>>>> back.
>>>>
>>>>
>>>> I'm on Squeak from old 3.2 times. And yes, a lot of discussions  
>>>> about
>>>> packages,
>>>> etc, but only discussions on most of cases. I think Metacello  
>>>> solves the
>>>> dependencies
>>>> questions very well and the other tools not (with exception of  
>>>> Universes
>>>> probably, but
>>>> abandoned now). Also Metacello provides all the features  
>>>> explained on
>>>> the excellent
>>>> mail of Torsten, assuring that the configurations will work with  
>>>> the
>>>> time not breaking
>>>> stuffs.
>>>>
>>>> The solution of using Metacello+Monticello is, imho, very simple,  
>>>> easy
>>>> and productive
>>>> and is here, available to us. :)
>>>
>>> I never claimed otherwise! I am just saying that there is nothing  
>>> new -
>>> and frankly, though I admit not having read carefully, I didn't  
>>> find the
>>> example so... impressive. Perhaps I need to read it again.
>>>
>>> Again though - I am not saying anything negative about Metacello.
>>>
>>> [SNIP]
>>>> Anyway, "common exchange mechanism" - I am working on  
>>>> Deltastreams which
>>>> I even presented at Brest, though unfortunately colliding with the
>>>> Seaside tutorial. Deltas and Deltastreams are indeed focused on  
>>>> EXACTLY
>>>> this problem (fork interchange), and it was born in 2007:
>>>>
>>>> http://lists.squeakfoundation.org/pipermail/squeak-dev/2007-August/119471.html
>>>>
>>>> I and Igor are the ones working on it - and after Brest we have  
>>>> had very
>>>> little time to move it forward. I still think it is a very  
>>>> important
>>>> piece of the puzzle though.
>>>>
>>>> But Deltastreams are ready to use on production systems?
>>>
>>> Nope, not yet, although very close. But since the topic came up I  
>>> wanted
>>> to mention it. But AFAIK there is no other tool/project aimed at  
>>> that
>>> use case. Again, I may be wrong :)
>>>
>>> regards, Göran
>>>
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

> Regards, Gary
>
> ----- Original Message -----
> From: "Igor Stasenko" <[hidden email]>
> To: <[hidden email]>
> Sent: Tuesday, February 23, 2010 4:02 PM
> Subject: [Pharo-project] Source code management - my vision. (Was: Re: Why
> apackage management system)
>
>
>> Since Goran mentioned DeltaStreams, i'd like to share one of my ideas,
>> which came into my mind, when i complete coding the method trailers.
>>
>> I made a method trailers open for future extensions.
>> A first step would be to introduce an abstract 'source management'
>> layer, which will be responsible for delivering
>> a method's and classes sources from some abstract data source. The
>> next step is actually implement such data sources.
>>
>> So, the idea is to abandon hardwiring with .sources and .changes files.
>> What new opportunities it gives to us?
>> Suppose that instead of storing sources into file, you putting them
>> directly to database , such as CouchDB (yes! :).
>>
>> Now if you imagine that anyone could access to this database on some
>> public location, this means that we automagically made a source code
>> exchange hub, and could do a very cool things with it.
>>
>> Suppose that ALL source code is put into a single database. Now
>> different users creating own accounts and makind own 'forks' from
>> original. What we getting is a number of update streams (methods,
>> doits etc etc), which lying in those databases,
>> and if one user wants to load/merge code from one of the sources he
>> simply points to it and it get loaded into his image.
>> We could organize things very closely to what github currently is:
>> everyone could create any number of new projects, or forks of existing
>> ones. Freely exchange the code, do a cherry picking etc etc.
>> And what is most important, that with such organization you can track
>> any methods, classes in the image and their evolutionary history back
>> to its origin. As well as, you can query database, how an when a
>> concrete method/class get deleted (everything is recorded, not just
>> current state).
>>
>> Also, it would allow us to manipulate the code at the lowest level of
>> granularity (individual methods/classes/doits, same as DS does), not
>> at a package level.
>> There are a lot of synergy with DeltaStreams - it could serve as an
>> 'offline' format, in same way as git storing the files locally, but
>> can reconnect with remote storage (which can be a public database).
>> One of the things, which could be made is to allow stashing/popping
>> code from stash. Suppose two projects started from a single starting
>> point (Squeak 3.9). Now suppose that by clicking a single button you
>> could get either trunk image, or Pharo 1.0 image.
>> But that's not all! With DeltaStreams, you could then click once more,
>> and revert things back from , say Pharo 1.0 to trunk or vice versa :)
>> Of course it would be stupid to do things at such large scale, but if
>> we put some common 'root' stuff into database, then we could very
>> easily track all changes & compare them. This would make cherry
>> picking and code exchange much more easier.
>>
>> This could get us on the next level of code management and system
>> organization and organization of community efforts.
>>
>>
>>
>> 2010/2/23 Göran Krampe <[hidden email]>:
>>> Hi!
>>>
>>> Germán Arduino wrote:
>>>> Hi Göran:
>>>>
>>>> Sorry by my intromision here, but want to say somethings :)
>>>>
>>>> 2010/2/23 Göran Krampe <[hidden email] <mailto:[hidden email]>>
>>> [SNIP]
>>>> What I have seen so far in Metacello is indeed nothing new - the
>>>> senarios and discussions about dependencies etc go loooong way back.
>>>>
>>>>
>>>> I'm on Squeak from old 3.2 times. And yes, a lot of discussions about
>>>> packages,
>>>> etc, but only discussions on most of cases. I think Metacello solves the
>>>> dependencies
>>>> questions very well and the other tools not (with exception of Universes
>>>> probably, but
>>>> abandoned now). Also Metacello provides all the features explained on
>>>> the excellent
>>>> mail of Torsten, assuring that the configurations will work with the
>>>> time not breaking
>>>> stuffs.
>>>>
>>>> The solution of using Metacello+Monticello is, imho, very simple, easy
>>>> and productive
>>>> and is here, available to us. :)
>>>
>>> I never claimed otherwise! I am just saying that there is nothing new -
>>> and frankly, though I admit not having read carefully, I didn't find the
>>> example so... impressive. Perhaps I need to read it again.
>>>
>>> Again though - I am not saying anything negative about Metacello.
>>>
>>> [SNIP]
>>>> Anyway, "common exchange mechanism" - I am working on Deltastreams which
>>>> I even presented at Brest, though unfortunately colliding with the
>>>> Seaside tutorial. Deltas and Deltastreams are indeed focused on EXACTLY
>>>> this problem (fork interchange), and it was born in 2007:
>>>>
>>>> http://lists.squeakfoundation.org/pipermail/squeak-dev/2007-August/119471.html
>>>>
>>>> I and Igor are the ones working on it - and after Brest we have had very
>>>> little time to move it forward. I still think it is a very important
>>>> piece of the puzzle though.
>>>>
>>>> But Deltastreams are ready to use on production systems?
>>>
>>> Nope, not yet, although very close. But since the topic came up I wanted
>>> to mention it. But AFAIK there is no other tool/project aimed at that
>>> use case. Again, I may be wrong :)
>>>
>>> regards, Göran
>>>
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

> Few more words.
>
> Deltas/packages.
>
> All we need for Delta is to identify the starting point (an image
> state, from where you started changing it). In contrast to
> DeltaStreams model, where it tries to capture both an 'old' and 'new'
> state, so it can be reverted, we don't need it anymore,
> since we can directly see what was the image state before we applied
> the changes to it. Having all of the history preserved and tracked is
> very powerful thing!
> So, Delta only needs to reference to starting point (URI or some ID,
> which identifies it, and in case of need can be easily found on the
> web).
>
> What is cool about it?
> Suppose, initially, all images started from some release (say Pharo
> 1.0), so all users sharing the same sources/image state.
> Now what happens when you start forking or making changes?
> Users creating a deltas and storing them in the database , and their
> images now knowing that they are derived from Pharo 1.0 point but also
> has some changes stored in separate branch or 'fork'.
>
> Now, what happens if user wants to load the changes made by another
> user: system can easily find the common ancestral point (since all
> deltas knowing its own ancestral point).
> And there is two use cases:
> a) you doing a full synchronization with some remote source. This is
> useful, when all of us want to use the very same image. This is
> primarily a good thing for team-based development or Releases(R).
> b) you doing a partial synchronization (or merge) with remote source.
> In this case you creating own fork, which derived from , say Pharo 1.0
> + some changes from user Foo, Bar, Baz etc and getting own, unique
> version of image.
>
> So no matter in what way you hacking/mixing the changes, you never
> lost track of the system history, which means that for any two images
> which derived from some common ancestral point we could could always
> automatically generate a diff between them. It means that no matter
> how far you went in your hacking - others could always load or
> cherry-pick your code!
>
> Packages? Since model, i describing, operating at lower granularity
> level (individual changes to methods/classes/doits), which means that
> any package can be represented as a set of those
> methods/classes/doits, there should be no problem
> integrating/interacting at a package level.
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

> Stef
>
> On Feb 24, 2010, at 8:36 AM, Igor Stasenko wrote:
>
>> Few more words.
>>
>> Deltas/packages.
>>
>> All we need for Delta is to identify the starting point (an image
>> state, from where you started changing it). In contrast to
>> DeltaStreams model, where it tries to capture both an 'old' and 'new'
>> state, so it can be reverted, we don't need it anymore,
>> since we can directly see what was the image state before we applied
>> the changes to it. Having all of the history preserved and tracked is
>> very powerful thing!
>> So, Delta only needs to reference to starting point (URI or some ID,
>> which identifies it, and in case of need can be easily found on the
>> web).
>>
>> What is cool about it?
>> Suppose, initially, all images started from some release (say Pharo
>> 1.0), so all users sharing the same sources/image state.
>> Now what happens when you start forking or making changes?
>> Users creating a deltas and storing them in the database , and their
>> images now knowing that they are derived from Pharo 1.0 point but also
>> has some changes stored in separate branch or 'fork'.
>>
>> Now, what happens if user wants to load the changes made by another
>> user: system can easily find the common ancestral point (since all
>> deltas knowing its own ancestral point).
>> And there is two use cases:
>> a) you doing a full synchronization with some remote source. This is
>> useful, when all of us want to use the very same image. This is
>> primarily a good thing for team-based development or Releases(R).
>> b) you doing a partial synchronization (or merge) with remote source.
>> In this case you creating own fork, which derived from , say Pharo 1.0
>> + some changes from user Foo, Bar, Baz etc and getting own, unique
>> version of image.
>>
>> So no matter in what way you hacking/mixing the changes, you never
>> lost track of the system history, which means that for any two images
>> which derived from some common ancestral point we could could always
>> automatically generate a diff between them. It means that no matter
>> how far you went in your hacking - others could always load or
>> cherry-pick your code!
>>
>> Packages? Since model, i describing, operating at lower granularity
>> level (individual changes to methods/classes/doits), which means that
>> any package can be represented as a set of those
>> methods/classes/doits, there should be no problem
>> integrating/interacting at a package level.
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>