Folks -
I truly love this discussion and I think it has generated enough insight that we can try pushing a few things. As a first step, I would like to propose a slightly different development model for Squeak, because I think that is one of the fundamental bits that is wrong. Here we go: 1) Agree on MC as the core development tool/model. This is painful and consequently the decision has to be made consciously. However, from my perspective there is really no suitable alternative [1]. 2) Decide on the canonical repositories. I would suggest that http://source.squeakfoundation.org/trunk is a fine starting point [2]. 3) Decide on who gets commit rights. I would suggest that the board decides that (equally for eventual access removals). The "masters" list from Squeak-people could act as a starting point. 4) Implement an update model for MC. I would suggest to adopt a similar model to what we've been using for Tweak in the past: Use MC config maps to mark points in the update process where order is important and otherwise update the packages in a well-defined order[3]. These four steps are intended to achieve the following: I can propose to the board that Eliot should be given access to the kernel repository so that he can commit any fixes to the compiler or other changes required for Cog directly without the need for an intermediate. I can propose to the board that Nicolas gets commit rights to the kernel repositories so that he can commit any fixes for numeric stuff he has or comes up with in the future. If the board agrees, they can do that, period. If they screw up or abuse their powers the board can retract commit rights. The main difference to what we've done in the past is simply that we have an inclusive list of committers instead of a select few and that we do understand the image as truly being "work in progress" inbetween the releases. In other words, be more active, change things faster, make it possible to report, devise, submit, and deploy a change in a matter of minutes. For people who don't have commit rights, there is both Mantis or commit-by-proxy which reinstates the SqC model of having a champion for your change. I.e., to commit something you can ask someone else to commit your change but that person implicitly takes responsibility for your change and will be the person who gets blamed for it and is expected to take care of it (and they may loose their commit rights if the change they commit is complete crap). To me, that is a useful intermediate state because the proxy acts as a mentor to teach the original author about how to provide changes, make sure it's okay etc. For Mantis, Keith just added a bunch of wonderful scripts that allow us to pull changes from it and build images (and hopefully soon run tests like the integration tests that I've been working on earlier) which means that we can build images with those changes included, test them, and if they pass those tests (and possibly other review), commit them to trunk. Again, I think having a champion who does the commit would be a good model. Obviously there is still a manual element involved here but the idea is that the regular committers would be able to go straight to the repository without requiring further approval. For users of Squeak there would be an update process which (although much slower than the update stream of the days of yore) would allow people to keep track of the ongoing changes and (for forks) merge those changes as desired along the way using Monticello. What do people think about this as a straw-man? Obviously, I want to enable people to contribute in the most straightforward way possible. Allow development to be more fluid, make mistakes faster, fix them faster. It does not address the issue of controversial / strategic changes but I think we're mostly loosing out on the most basic level and if we want to fix the processes we must make simple things simple and complex things possible, not the other way around. Cheers, - Andreas More detail on some of the issues above: [1] Why I think update streams don't cut it: When you do distributed development it is necessary to know if there are conflicts with what you are working on. If you only get to spend two hours a week on development you need to be able to quickly find out if anything changed in an area that you've been working on since last week. If you want to provide backwards compatibility you need to be able to reflect about what has changed, etc. Monticello is pretty bad but it's still the best we got. [2] http://source.squeakfoundation.org/trunk: The idea is to use the version specific directories (Squeak39, Squeak310, Squeak311 etc) as branches for released versions. The "current" version is always on trunk, the specific ones provide updates after a version was released. [3] MC based updates: The basic idea is that the config maps define load order and intermediate stages where required. There is an update-<version>.mcm in every repository that is consulted by the update loader process. The update process loads the update.mcms in order and uses the order in the last loaded update.mcm to update all the remaining packages. In other words, as long as the packages load with the last defined order, we change nothing. If Bob finds a problem we issue a (series of) configurations that get past that update bump. Yes, there is a certain element of risk to it but if Bob runs test builds of the image for us we can simply update to the last version which Bob was able to build for himself (and Bob can notify us that there's a problem). |
2009/6/30 Andreas Raab <[hidden email]>:
> Folks - > > I truly love this discussion and I think it has generated enough insight > that we can try pushing a few things. As a first step, I would like to > propose a slightly different development model for Squeak, because I think > that is one of the fundamental bits that is wrong. Here we go: > > 1) Agree on MC as the core development tool/model. This is painful and > consequently the decision has to be made consciously. However, from my > perspective there is really no suitable alternative [1]. > > 2) Decide on the canonical repositories. I would suggest that > http://source.squeakfoundation.org/trunk is a fine starting point [2]. > > 3) Decide on who gets commit rights. I would suggest that the board decides > that (equally for eventual access removals). The "masters" list from > Squeak-people could act as a starting point. > > 4) Implement an update model for MC. I would suggest to adopt a similar > model to what we've been using for Tweak in the past: Use MC config maps to > mark points in the update process where order is important and otherwise > update the packages in a well-defined order[3]. > A side note: at this point , i would consult with Keith on his vision and how his creation (Bob) and rest of infrastructure he built could help us in avoiding doing boring stuff. (but ignore this comment, i see you mention about it down below). > These four steps are intended to achieve the following: I can propose to the > board that Eliot should be given access to the kernel repository so that he > can commit any fixes to the compiler or other changes required for Cog > directly without the need for an intermediate. I can propose to the board > that Nicolas gets commit rights to the kernel repositories so that he can > commit any fixes for numeric stuff he has or comes up with in the future. If > the board agrees, they can do that, period. If they screw up or abuse their > powers the board can retract commit rights. > I just thought about the similar thing last night, and even considering to write a new topic with 'Squeak privatization' title, when idea will settle down. +1 on that. Lets privatize different areas of Squeak and let the people become landlords of those areas. If something went wrong - we always have a board to revoke the rights and burn heretics :) But lets not get down to a team model - if owner (responsible person) don't have time to react on others contributions of his areas of interest, or sleeps 23 hours a day - this should not become the bottleneck of contribution process. A more better will be just to step down the position and declare it free. Or even do this in a more autocratic fashion: initially the landlord having 10 points - but for each contribution not revised by him or which get past his attention, he gets -1. When counter goes to zero - he's get fired :) > The main difference to what we've done in the past is simply that we have an > inclusive list of committers instead of a select few and that we do > understand the image as truly being "work in progress" inbetween the > releases. In other words, be more active, change things faster, make it > possible to report, devise, submit, and deploy a change in a matter of > minutes. > > For people who don't have commit rights, there is both Mantis or > commit-by-proxy which reinstates the SqC model of having a champion for your > change. I.e., to commit something you can ask someone else to commit your > change but that person implicitly takes responsibility for your change and > will be the person who gets blamed for it and is expected to take care of it > (and they may loose their commit rights if the change they commit is > complete crap). To me, that is a useful intermediate state because the proxy > acts as a mentor to teach the original author about how to provide changes, > make sure it's okay etc. > > For Mantis, Keith just added a bunch of wonderful scripts that allow us to > pull changes from it and build images (and hopefully soon run tests like the > integration tests that I've been working on earlier) which means that we can > build images with those changes included, test them, and if they pass those > tests (and possibly other review), commit them to trunk. Again, I think > having a champion who does the commit would be a good model. Obviously there > is still a manual element involved here but the idea is that the regular > committers would be able to go straight to the repository without requiring > further approval. > > For users of Squeak there would be an update process which (although much > slower than the update stream of the days of yore) would allow people to > keep track of the ongoing changes and (for forks) merge those changes as > desired along the way using Monticello. > > What do people think about this as a straw-man? Obviously, I want to enable > people to contribute in the most straightforward way possible. Allow > development to be more fluid, make mistakes faster, fix them faster. It does > not address the issue of controversial / strategic changes but I think we're > mostly loosing out on the most basic level and if we want to fix the > processes we must make simple things simple and complex things possible, not > the other way around. > I think this is not a surprise, that model what is you proposing having a lot of synergy with 3.11 team vision on how community past 3.11 release could contribute to squeak. > Cheers, > - Andreas > > More detail on some of the issues above: > > [1] Why I think update streams don't cut it: When you do distributed > development it is necessary to know if there are conflicts with what you are > working on. If you only get to spend two hours a week on development you > need to be able to quickly find out if anything changed in an area that > you've been working on since last week. If you want to provide backwards > compatibility you need to be able to reflect about what has changed, etc. > Monticello is pretty bad but it's still the best we got. > > [2] http://source.squeakfoundation.org/trunk: The idea is to use the version > specific directories (Squeak39, Squeak310, Squeak311 etc) as branches for > released versions. The "current" version is always on trunk, the specific > ones provide updates after a version was released. > > [3] MC based updates: The basic idea is that the config maps define load > order and intermediate stages where required. There is an > update-<version>.mcm in every repository that is consulted by the update > loader process. The update process loads the update.mcms in order and uses > the order in the last loaded update.mcm to update all the remaining > packages. In other words, as long as the packages load with the last > defined order, we change nothing. If Bob finds a problem we issue a (series > of) configurations that get past that update bump. Yes, there is a certain > element of risk to it but if Bob runs test builds of the image for us we can > simply update to the last version which Bob was able to build for himself > (and Bob can notify us that there's a problem). > > > -- Best regards, Igor Stasenko AKA sig. |
On Tue, 30 Jun 2009 06:29:05 +0200, Igor Stasenko wrote:
... > But lets not get down to a team model - if owner (responsible person) > don't have time to react on others contributions of his areas of > interest, or sleeps 23 hours a day - this should not become the > bottleneck of contribution process. In a "normal" project, I would rule out "master" people who have _not_ committed a bug fix within the last 12 months (even if that would rule out me, of course ;) If this community restarts, it can equally well restart from where every newbee starts, to show how it works, to demonstrate committment, to everyone who cares. But I doubt that my proposal will be implemented. /Klaus -- "If at first, the idea is not absurd, then there is no hope for it". Albert Einstein |
In reply to this post by Andreas.Raab
2009/6/29 Andreas Raab <[hidden email]>:
> 1) Agree on MC as the core development tool/model. This is painful and > consequently the decision has to be made consciously. However, from my > perspective there is really no suitable alternative [1]. Monticello is an acceptable choice to me. MC sounds like something we can use right here, right now. And as long as we can easily contribute, we can bring improvements to the development process and tools in our way. > 2) Decide on the canonical repositories. I would suggest that > http://source.squeakfoundation.org/trunk is a fine starting point [2]. Yes, I support this idea. > 3) Decide on who gets commit rights. I would suggest that the board decides > that (equally for eventual access removals). The "masters" list from > Squeak-people could act as a starting point. Yes, I support on the board decides who is approved and revoked. Could you point out the masters list mentioned here? > 4) Implement an update model for MC. I would suggest to adopt a similar > model to what we've been using for Tweak in the past: Use MC config maps to > mark points in the update process where order is important and otherwise > update the packages in a well-defined order[3]. What would be the incidence of such an update model, with well-defined order, on the eventual maintenance a component manager (someone who has commit access) has to carry on? Isn't just plain revision number? > The main difference to what we've done in the past is simply that we have an > inclusive list of committers instead of a select few and that we do > understand the image as truly being "work in progress" inbetween the > releases. In other words, be more active, change things faster, make it > possible to report, devise, submit, and deploy a change in a matter of > minutes. Woot! Sounds exciting. =) > What do people think about this as a straw-man? Obviously, I want to enable > people to contribute in the most straightforward way possible. Allow > development to be more fluid, make mistakes faster, fix them faster. It does > not address the issue of controversial / strategic changes but I think we're > mostly loosing out on the most basic level and if we want to fix the > processes we must make simple things simple and complex things possible, not > the other way around. On the account of controversial changes, I think that Squeak Oversight Board should clearly state what changes will be acceptable without thorough review and acceptation process by the board/community. For example, community could commit quickly tests (any), fixes (only on what is tested), minor improvements (which kind?) with tests, etc. A clear statement should avoid misunderstandings, in general at least. Andreas, I would like some clarification on how someone part of the community, with no special rights, can contribute using your process, method and tool. I'd be glad if you could provide a use case/scenario. Your solution does seem leaving the little man outside the loop, sent back to mantis or something along this line. That's hardly something interesting. We as a community need instant feedback on our work and be well integrated in the process. Do not set us apart! It is important to renew the excitement of contributing to Squeak within the community and not just for few selected committers. Considering continuous integration would really help on that matter, I think it's possible to push updates for review from any body to the committers. I would really prefer a community repository rather than yet another elitist process; but I am always fine with the idea that the selected elites will review, accept, modify and/or reject community committed content. Or did I get it all wrong? Regards, Ian -- http://mecenia.blogspot.com/ |
In reply to this post by Andreas.Raab
For someone like Elliot, the current proposal is as follows.
Lets say his aim is to develop an image with closures in it. We will call this image 3.10-closures He would take a fixed starting point... preferably 3.x-build This image is 3.x with the latest version of all tools (MC with atomic loading etc) and a couple of additional packages namely Packages and Tasks. Packages defines the packages which may be loaded that are managed external to the kernel, and the relationships between them. This could include the compiler and other kernel packages, if they are maintained as a common package that all forks may adopt. Tasks defines actions that may be applied to the kernel image to acheive particular aims, such as loading all tests, loading fixes, and preparing the image for release. Elliot would then develop a "Task" which may combine the loading of updated packages, and change sets, and fixes to the image. The task is essentially a script, though if defined as a task it can define pre-requisites. Essentially all I am saying is that I just dont think that an mcm will cut it, I think that each task needs to be crafted to a known fixed point. While Elliot is developing 3.10-closures. Bob is also able to make test builds of 3.10-closures+mantis fixes+tests So the next release becomes a selection of packages and tasks. Keith |
In reply to this post by Andreas.Raab
On 6/30/09 12:58 AM, "Andreas Raab" <[hidden email]> wrote: > 1) Agree on MC as the core development tool/model. This is painful and > consequently the decision has to be made consciously. However, from my > perspective there is really no suitable alternative [1]. Seems 3.9 an 3.10 don't teach MC as is now is not suited for a .image develop. And which MC ? Maybe MC2 could solve all MC don't , but it's used for this kind of job ? Edgar |
In reply to this post by Andreas.Raab
On Mon, Jun 29, 2009 at 08:58:18PM -0700, Andreas Raab wrote:
> > For people who don't have commit rights, there is both Mantis or > commit-by-proxy which reinstates the SqC model of having a champion for > your change. I.e., to commit something you can ask someone else to > commit your change but that person implicitly takes responsibility for > your change and will be the person who gets blamed for it and is > expected to take care of it (and they may loose their commit rights if > the change they commit is complete crap). To me, that is a useful > intermediate state because the proxy acts as a mentor to teach the > original author about how to provide changes, make sure it's okay etc. This sounds like a good approach. It provides a way to get things done, and a process for ensuring that things get done right. As someone who has occasionally submitted both good patches and complete crap, I value both ;) I also think that it is much easier to put crap in than to take it out, so controlling this well is important. > For users of Squeak there would be an update process which (although > much slower than the update stream of the days of yore) would allow > people to keep track of the ongoing changes and (for forks) merge those > changes as desired along the way using Monticello. As a consumer of the update process, I like the update stream (based on change sets) a *lot*. As a developer, Monticello is indispensable. Is there some simple way we could get both, such that each new commit to a Monticello repository can produce a change set relative to its predecessor, with the comment text from the Monticello version forming the preamble of the change set? Perhaps this is something that already exists (or maybe it's an itch I should go scratch?). > What do people think about this as a straw-man? I like it. Dave |
In reply to this post by Klaus D. Witzel
Klaus D. Witzel wrote:
> ... > In a "normal" project, I would rule out "master" people who have _not_ > committed a bug fix within the last 12 months (even if that would rule > out me, of course ;) > > If this community restarts, it can equally well restart from where > every newbee starts, to show how it works, to demonstrate committment, > to everyone who cares. > > But I doubt that my proposal will be implemented. > > /Klaus Tried to get to the masters list. www.squeak.org/Community points to people.squeakfoundation.org But it is no longer there... Can anybody advise? Anyway, last time I read it, it included a lot of people who hasn't been in this community for over 5 years, and people who never contributed a single piece of code. On the other hand, many hard working and valuable people are at the journeyer level. We need a better list of valuable contributors. Cheers, Juan Vuletich |
In reply to this post by David T. Lewis
On 6/30/09 8:53 AM, "David T. Lewis" <[hidden email]> wrote: > Is there some simple way we could get both, such that each new > commit to a Monticello repository can produce a change set relative > to its predecessor, with the comment text from the Monticello version > forming the preamble of the change set? Perhaps this is something > that already exists (or maybe it's an itch I should go scratch?). Well , this was 3.10 and I have many headaches trying to get the work done. And learn Monticello , as we know now, could not be used for a very complex task like building a .image. Cs can and we don't have yet IMHO some better. Eras ago I volunteer to test both DeltaStreams and MC2 , but nobody wish risk and we take the Blue Pill (remember?) But maybe some found the silver bullet and I was the only disbelieving that... Edgar |
Hi!
Edgar J. De Cleene wrote: > Eras ago I volunteer to test both DeltaStreams and MC2 , but nobody wish > risk and we take the Blue Pill (remember?) Ah! DeltaStreams! Just waited for it to get mentioned. :) I actually sat last night trying to get it slightly into shape for Igor to join in. Quick recap and status: DeltaStreams is an idea (and code) that I came up with a while back when trying to figure out "new ways" to live in a forked world. The idea consisted of two parts - Deltas and Streams of them. :) A Delta is simply a "ChangeSet DeLuxe". Main differences off the top of my head: - Is in fact an *ordered* sequence of changes. (A ChangeSet is a *Set* - no ordering) - Each change in a Delta is much more fine granular and has *all* information captured about the change. - Each change can produce its "anti" change. - A Delta is a *standalone* object. It does not refer to anything in the image, each change contains all its information in its own object graph. This standalone property means that you first *load* a Delta into the image and then *apply* it if you want to. Thus you can load one from disk and look at it *without* applying it! You could harvest the Internet for Deltas, suck hundreds of them into an image, and then inspect them. The anti bit above also means that a Delta can be reverted, it just applies all its anti changes in reverse order! And yes, this shit actually works. :) The DeltaStreams code uses SystemEditor from Colin Putney - the new mechanism to *atomically* apply changes to the image. In fact, our work on DS contributed several fixes to SE because DS was first to use it seriously. I thin MC1.6 maybe uses SE too? Deltas are "recording" just like ChangeSets are, but since they record all changes in order - they do double duty as a changelog too. Matthew Fulmer started out implementing a clever hack so that Deltas could be saved as .cs files and actually "work" as a ChangeSet if filed in as one. Now that I am diving into the code again I have decoupled that file format (but still there as an extra package) and instead will implement a format using Tirade. I have blogged about Tirade in 3 articles so you can read about it there. It is a readable format that looks like Smalltalk code, but it is very fast to parse and secure. Status: The domain code is perhaps 90% covered. There are some small things to take care of and since I am now hooking in Tirade I am refactoring some things a bit. A lot of tests are green - but not all. I really want to move to a fully green state. Experts are more than welcome to help out! An example of a "problem": If I have a method change that moves a method to a new method category - and then I *revert* that change, should the new method category (if now empty) be removed? tricksy. The Tirade hookup is ongoing but not hard work, it should be doable in a couple of nights. Then I wish to implement a DualDeltaSorter :) - simply a "clone" of dual changesorter that works with Deltas. If anyone would like to start that work, or implement some other cool thing like: - ChangeSet -> Delta converter. Filing in a ChangeSet should simply create a Delta instead of a ChangeSet. - MC diff -> Delta converter. Given two MCs we can produce a Delta. Just holler! Usecase: Let's say the Pharo people fixes a bug in Pharo. By the look of it the bug should exist in Squeak.org too. Let's say we produce a Delta either from the Pharo changeset or from the MC snapshot that contained the fix. The Delta is filed out from Pharo in Tirade format. It is easily readable in a text editor just like a ChangeSet is. You fire up Squeak.org and open up the DualDeltaSorter and *load* the Delta into the image. At this point *no code* has been touched in the image, we just deserialized the Tirade file and created the DSDelta object. We can now look at the actual changes it contains, perhaps 3 new methods, 4 modified methods, some ivars added - all over the image. A Delta is like a "semantic patch". Since it knows more about itself it can reason about its own applicability. For example, it separates "modified method" and "added method". And it contains both the "before" and "after" state of each change. For example a modified method contains both code before and the new code! We thus ask the Delta if it can be "cleanly applied". Cleanly means that the DSDelta checks if the state of the image concerning the changes is the same as it was in the Pharo image. For example, regarding the modified methods - do they look like they did in Pharo *before* they were modified? If yes, then we are good to go. Can we add the ivars? Are all expected classes in the image? Did any of the new methods perhaps already exist? Let's say it claims to apply cleanly (there are levels of "cleanliness" here, for example a method may already be in the "new state", which is probably fine but not perfectly clean). We apply it (done by SystemEditor atomically). Then we run tests or whatever, if it seems to have broken something we could decide to just back it out with "revert". We could also set the Delta to record and then try to fix it. Conclusion: DeltaStreams are not here yet - but pretty darn close to be able to replace ChangeSets at least! It just needs a push. This means I agree with Andreas to use MC currently, but I also would like us to dig in our heels and get DS ready. It should bring us: - Speed. - Lots of flexibility. regards, Göran |
> The DeltaStreams code uses SystemEditor from Colin Putney - the new > mechanism to *atomically* apply changes to the image. In fact, our > work on DS contributed several fixes to SE because DS was first to use > it seriously. I thin MC1.6 maybe uses SE too? Hi Göran, MC1.6 is pretty much primed to use SE, and its 3x faster than MC1.5 Traits are not yet supported AFAIK, matthew was working on it. If you manage to recruit some helpers, perhaps you could have a further look SystemEditor. Recently a bug was introduced which results in obsolete classes. Keith |
In reply to this post by Göran Krampe
2009/6/30 Göran Krampe <[hidden email]>:
> Hi! > > Edgar J. De Cleene wrote: >> >> Eras ago I volunteer to test both DeltaStreams and MC2 , but nobody wish >> risk and we take the Blue Pill (remember?) > > Ah! DeltaStreams! Just waited for it to get mentioned. :) > > I actually sat last night trying to get it slightly into shape for Igor to > join in. Quick recap and status: > > DeltaStreams is an idea (and code) that I came up with a while back when > trying to figure out "new ways" to live in a forked world. The idea > consisted of two parts - Deltas and Streams of them. :) > > A Delta is simply a "ChangeSet DeLuxe". Main differences off the top of my > head: > > - Is in fact an *ordered* sequence of changes. (A ChangeSet is a *Set* - no > ordering) > - Each change in a Delta is much more fine granular and has *all* > information captured about the change. > - Each change can produce its "anti" change. > - A Delta is a *standalone* object. It does not refer to anything in the > image, each change contains all its information in its own object graph. > > This standalone property means that you first *load* a Delta into the image > and then *apply* it if you want to. Thus you can load one from disk and look > at it *without* applying it! You could harvest the Internet for Deltas, suck > hundreds of them into an image, and then inspect them. > > The anti bit above also means that a Delta can be reverted, it just applies > all its anti changes in reverse order! And yes, this shit actually works. :) > Goran, this is great work I'd like to support. But I can only believe that it works in most cases... More exactly, I suspect it works for code, but not on lively instances: If I remove an inst var, all instances of the class and subclasses will loose some information. Do you store that information in the delta so that reversion restore the state? Otherwise, I know of some anti-refactoring that will kill my image nicely. If I remove a class, how will orphaned instances be restored? Do DeltaStream address something like refactoring ScaledDecimal from something like: Number subclass: #ScaledDecimal instanceVariables: 'fraction precision'. to: Fraction subclass: #ScaledDecimal instanceVariables: 'precision'. This shit is real (MIT refactoring applied in Pharo) and requires some other shit like: Create class ScaledDecimalTemp Mutate allInstances of ScaledDecimal to ScaledDecimalTemp remove ScaledDecimal rename ScaledDecimalTemp to ScaledDecimal I'm very interested how would you handle such mutation and anti-mutation in DeltaStream? It's doable with user crafted pre/post scripts without system validation... Is that the solution? Nicolas > The DeltaStreams code uses SystemEditor from Colin Putney - the new > mechanism to *atomically* apply changes to the image. In fact, our work on > DS contributed several fixes to SE because DS was first to use it seriously. > I thin MC1.6 maybe uses SE too? > > Deltas are "recording" just like ChangeSets are, but since they record all > changes in order - they do double duty as a changelog too. > > Matthew Fulmer started out implementing a clever hack so that Deltas could > be saved as .cs files and actually "work" as a ChangeSet if filed in as one. > Now that I am diving into the code again I have decoupled that file format > (but still there as an extra package) and instead will implement a format > using Tirade. I have blogged about Tirade in 3 articles so you can read > about it there. It is a readable format that looks like Smalltalk code, but > it is very fast to parse and secure. > > > Status: > > The domain code is perhaps 90% covered. There are some small things to take > care of and since I am now hooking in Tirade I am refactoring some things a > bit. A lot of tests are green - but not all. I really want to move to a > fully green state. Experts are more than welcome to help out! > > An example of a "problem": If I have a method change that moves a method to > a new method category - and then I *revert* that change, should the new > method category (if now empty) be removed? tricksy. > > The Tirade hookup is ongoing but not hard work, it should be doable in a > couple of nights. > > Then I wish to implement a DualDeltaSorter :) - simply a "clone" of dual > changesorter that works with Deltas. If anyone would like to start that > work, or implement some other cool thing like: > > - ChangeSet -> Delta converter. Filing in a ChangeSet should simply create a > Delta instead of a ChangeSet. > - MC diff -> Delta converter. Given two MCs we can produce a Delta. > > Just holler! > > Usecase: > > Let's say the Pharo people fixes a bug in Pharo. By the look of it the bug > should exist in Squeak.org too. Let's say we produce a Delta either from the > Pharo changeset or from the MC snapshot that contained the fix. > > The Delta is filed out from Pharo in Tirade format. It is easily readable in > a text editor just like a ChangeSet is. You fire up Squeak.org and open up > the DualDeltaSorter and *load* the Delta into the image. At this point *no > code* has been touched in the image, we just deserialized the Tirade file > and created the DSDelta object. > > We can now look at the actual changes it contains, perhaps 3 new methods, 4 > modified methods, some ivars added - all over the image. > > A Delta is like a "semantic patch". Since it knows more about itself it can > reason about its own applicability. For example, it separates "modified > method" and "added method". And it contains both the "before" and "after" > state of each change. For example a modified method contains both code > before and the new code! > > We thus ask the Delta if it can be "cleanly applied". Cleanly means that the > DSDelta checks if the state of the image concerning the changes is the same > as it was in the Pharo image. For example, regarding the modified methods - > do they look like they did in Pharo *before* they were modified? If yes, > then we are good to go. Can we add the ivars? Are all expected classes in > the image? Did any of the new methods perhaps already exist? > > Let's say it claims to apply cleanly (there are levels of "cleanliness" > here, for example a method may already be in the "new state", which is > probably fine but not perfectly clean). > > We apply it (done by SystemEditor atomically). Then we run tests or > whatever, if it seems to have broken something we could decide to just back > it out with "revert". We could also set the Delta to record and then try to > fix it. > > Conclusion: > > DeltaStreams are not here yet - but pretty darn close to be able to replace > ChangeSets at least! It just needs a push. This means I agree with Andreas > to use MC currently, but I also would like us to dig in our heels and get DS > ready. It should bring us: > > - Speed. > - Lots of flexibility. > > regards, Göran > > > |
Hi!
Nicolas Cellier wrote: > 2009/6/30 Göran Krampe <[hidden email]>: > Goran, this is great work I'd like to support. :) Thanks! > But I can only believe that it works in most cases... > More exactly, I suspect it works for code, but not on lively instances: > If I remove an inst var, all instances of the class and subclasses > will loose some information. > Do you store that information in the delta so that reversion restore the state? Nope, we only deal with source. If you want image level kind of reversion then I can only recommend... image snapshots :). Or some kind of Gemstone-ish memory. > Otherwise, I know of some anti-refactoring that will kill my image nicely. > > If I remove a class, how will orphaned instances be restored? Again, we only deal with code. Like ChangeSets do. > Do DeltaStream address something like refactoring ScaledDecimal from > something like: > Number subclass: #ScaledDecimal instanceVariables: 'fraction precision'. > to: > Fraction subclass: #ScaledDecimal instanceVariables: 'precision'. > > This shit is real (MIT refactoring applied in Pharo) and requires some > other shit like: > Create class ScaledDecimalTemp > Mutate allInstances of ScaledDecimal to ScaledDecimalTemp > remove ScaledDecimal > rename ScaledDecimalTemp to ScaledDecimal > > I'm very interested how would you handle such mutation and > anti-mutation in DeltaStream? Ouch! :) Again, no, we have not looked at instance migration issues. > It's doable with user crafted pre/post scripts without system > validation... Is that the solution? You mean in Deltas? Well, a Delta is meant to be able to do everything a Changeset can - so we need to support scripts (it is just another change class called "DSDoit"). Since the changes are ordered the "doits" can appear not only before or after but in the middle too :). So... well, I presume "surgery" gets a tad simpler to do with Deltas since they simply record all the steps you perform. Thus, if you perform the steps you describe above - in sequence - then you have a DSDelta that will work. Slightly simpler than editing the Delta in order to add pre/post scripts etc. This also means that we typically want to "prune" doits from Deltas, since they are normally not really wanted. regards, Göran |
2009/6/30 Göran Krampe <[hidden email]>:
> Hi! > > Nicolas Cellier wrote: >> >> 2009/6/30 Göran Krampe <[hidden email]>: >> Goran, this is great work I'd like to support. > > :) Thanks! > >> But I can only believe that it works in most cases... >> More exactly, I suspect it works for code, but not on lively instances: >> If I remove an inst var, all instances of the class and subclasses >> will loose some information. >> Do you store that information in the delta so that reversion restore the >> state? > > Nope, we only deal with source. If you want image level kind of reversion > then I can only recommend... image snapshots :). Or some kind of > Gemstone-ish memory. > >> Otherwise, I know of some anti-refactoring that will kill my image nicely. >> >> If I remove a class, how will orphaned instances be restored? > > Again, we only deal with code. Like ChangeSets do. > >> Do DeltaStream address something like refactoring ScaledDecimal from >> something like: >> Number subclass: #ScaledDecimal instanceVariables: 'fraction precision'. >> to: >> Fraction subclass: #ScaledDecimal instanceVariables: 'precision'. >> >> This shit is real (MIT refactoring applied in Pharo) and requires some >> other shit like: >> Create class ScaledDecimalTemp >> Mutate allInstances of ScaledDecimal to ScaledDecimalTemp >> remove ScaledDecimal >> rename ScaledDecimalTemp to ScaledDecimal >> >> I'm very interested how would you handle such mutation and >> anti-mutation in DeltaStream? > > Ouch! :) Again, no, we have not looked at instance migration issues. > >> It's doable with user crafted pre/post scripts without system >> validation... Is that the solution? > > You mean in Deltas? Well, a Delta is meant to be able to do everything a > Changeset can - so we need to support scripts (it is just another change > class called "DSDoit"). Since the changes are ordered the "doits" can appear > not only before or after but in the middle too :). > > So... well, I presume "surgery" gets a tad simpler to do with Deltas since > they simply record all the steps you perform. Thus, if you perform the steps > you describe above - in sequence - then you have a DSDelta that will work. > Slightly simpler than editing the Delta in order to add pre/post scripts > etc. > > This also means that we typically want to "prune" doits from Deltas, since > they are normally not really wanted. > > regards, Göran > > Thanks, that's what I thought. I just wanted to point out that in some cases, special doIts are necessary. In theses cases, anti-delta generally won't work, and should better not be attempted. OK for me if tools cover 90%, atomic loading is just another example not covering 100%. Just think of providing enough hooks for the remaining 10% in the overall process. This kind of refactoring will likely happen to anyone touching Process, Graphics, user interaction like Sensor, Compiler and other Kernel beasts... That's not that rare. Ask Igor, Andreas, Eliot, Michael, Marcus, etc... Nicolas |
In reply to this post by Andreas.Raab
The SqueakPeople Masters and Journeyers as of 12/1/2008 when the site
was closed: Masters Kazuhiro Abe Germán Arduino Jecel Assumpção Jr Bruce Badger Avi Bryant Damien Cassou Ward Cunningham Edgar De Cleene Cees de Groot Marcus Denker Stéphane Ducasse Tansel Ersavas Brad Fowlow Bert Freudenberg Diego Gomez Deck Dale Henrichs Jon Hylands Daniel Ingalls Ralph Johnson Ted Kaehler Bryce Kampjes Ned Konz Göran Krampe Pavel Krivanek Craig Latta Adrian Lienhard John M McIntosh Janko Mivsek Chris Muller Luciano Notarfrancesco Yoshiki Ohshima Stephen Pair Ian Piumarta Colin Putney Andreas Raab Lukas Renggli Kim Rose Tim Rowledge Michael Rueger Lex Spoon Masato Sumi Andrew Tween Masashi Umezawa Daniel Vainsencher Scott Wallace Doug Way Stephan B. Wessels Roel Wuyts Takashi Yamamiya Journeyers Cedrick Beler Alexandre Bergel Torsten Bergmann Andrew Black Todd Blanchard Noury Bouraqadi Julien Bourdon Brian Brown Luca Bruno Chris Burkert Blaine Buxton Frank Caggiano Gwenael Casaccio Andrew Catton Ken Causey nicolas cellier Gary Chambers Yanni Chiu Giovanni Corriga Eddie Cottongim Ramiro Diaz Trepat Pierre-Andre Dreyfuss Jason Dufair Dave Faught Hilaire Fernandes Julian Fitzell James Foster Rita Freudenberg Brad Fuller Matthew Fulmer Thushar G R Markus Gaelli Boris Gaertner Joshua Gargus Giovanni Giorgi Sean Glazier Matthew Hamrick William Harford Norbert Hartl Michael Haupt Tetsuya HAYASHI Keith Hodges Ragnar Hojland Espinosa Simon Holland Dwight Hughes Eiichiro Ito SeungBum Kim Simon Kirk Tom Koenig Jan B. Krejci Martin Kuball tsuyoshi kuwatani Alexander Lazarevic Ramon Leon Dave Lewis Alan Lovejoy Adrian Manrique Norberto Manzanos Mark McCahill Martin McClure Webb McDonald Simon Michael Kazuki Minamitani Masaaki Mitsuhashi Toshio Miyasaka Jens Moenig Peter Moore German Morales Hans-Martin Mosner Philippe Mougin Javier Musa Oscar Nierstrasz Bruce O'Neel Emilio Oca Tomohiro Oda DongHee Park Jerome Peace Nicolas Petton John Pierce Brent Pinkney Alain Plantec Damien Pollet Reza Razavi José L. Redrejo Rodríguez Trygve Reenskaug Aaron Reichow Brian Rice Gerardo Richarte Romain Robbes Stéphane Rollandin Dan Rozenfarb Stephan Rudlof Randal Schwartz Mark Schwenk David Shaffer Frank Shearar Rik Smoody Adam Spitz Igor Stasenko Serge Stinckwich Mathieu Suen Gen Suzuki Steven Swerling Hisashi Tagawa Makoto TAKAHASHI Toshiyuki Takeda Ron Teitelbaum Hernán Tylim Michael van der Gulik Adriaan van Os Brent Vukmer Juan Vuletich Rob Withers Klaus D. Witzel Koji Yokokawa Yakov Zaytsev signature.asc (196 bytes) Download Attachment |
In reply to this post by Nicolas Cellier
Hi!
Nicolas Cellier wrote: > 2009/6/30 Göran Krampe <[hidden email]>: >>> It's doable with user crafted pre/post scripts without system >>> validation... Is that the solution? >> You mean in Deltas? Well, a Delta is meant to be able to do everything a >> Changeset can - so we need to support scripts (it is just another change >> class called "DSDoit"). Since the changes are ordered the "doits" can appear >> not only before or after but in the middle too :). >> >> So... well, I presume "surgery" gets a tad simpler to do with Deltas since >> they simply record all the steps you perform. Thus, if you perform the steps >> you describe above - in sequence - then you have a DSDelta that will work. >> Slightly simpler than editing the Delta in order to add pre/post scripts >> etc. >> >> This also means that we typically want to "prune" doits from Deltas, since >> they are normally not really wanted. >> >> regards, Göran > > Thanks, that's what I thought. > I just wanted to point out that in some cases, special doIts are necessary. > In theses cases, anti-delta generally won't work, and should better > not be attempted. I gully agree. :) > OK for me if tools cover 90%, atomic loading is just another example > not covering 100%. > Just think of providing enough hooks for the remaining 10% in the > overall process. > This kind of refactoring will likely happen to anyone touching > Process, Graphics, user interaction like Sensor, Compiler and other > Kernel beasts... That's not that rare. Ask Igor, Andreas, Eliot, > Michael, Marcus, etc... I agree. This is actually one of the first inspirations of Deltas - I wanted a tool that could do relatively nice merging/conflict detection etc *without* advanced history. We all like the concreteness of ChangeSets - but we could still have a modern implementation. :) In turn this was also inspired by git - not git itself, but by the philosophy of "let the tool be less smart and more predictable", it will still cover 80-90% and will be enough. And you get fewer surprises. regards, Göran |
In reply to this post by Edgar J. De Cleene
2009/6/30 Edgar J. De Cleene <[hidden email]>:
> Seems 3.9 an 3.10 don't teach MC as is now is not suited for a .image > develop. > And which MC ? > > Maybe MC2 could solve all MC don't , but it's used for this kind of job ? > > Edgar Good question, Edgar. By the way, let's not forget that we could use Andreas' ConflictFinder as well. Every bit of tool we can have for a better continuous integration. Ian. -- http://mecenia.blogspot.com/ |
In reply to this post by Göran Krampe
> I gully agree. :)
Beware of guano :) Stef |
In reply to this post by Göran Krampe
On 30-Jun-09, at 6:58 AM, Göran Krampe wrote: > DeltaStreams are not here yet - but pretty darn close to be able to > replace ChangeSets at least! It just needs a push. This means I > agree with Andreas to use MC currently, but I also would like us to > dig in our heels and get DS ready. MC2 is in this position as well. It's available and useful, but it hasn't had the kind of long-term and widespread usage that MC1 has. As the interaction and collaboration patterns that MC2 favors become clear, and the code base matures, it will become suitable for a large- scale project like Squeak. But it's not there yet. Colin |
In reply to this post by Göran Krampe
On 6/30/09 10:58 AM, "Göran Krampe" <[hidden email]> wrote: > Conclusion: > > DeltaStreams are not here yet - but pretty darn close to be able to > replace ChangeSets at least! It just needs a push. This means I agree > with Andreas to use MC currently, but I also would like us to dig in our > heels and get DS ready. It should bring us: > > - Speed. > - Lots of flexibility. > > regards, Göran I confident in you so I wish someone use DeltaStreams instead MC as I know MC "as is now" don't work for a release. If needed, I can do the dirty work. Edgar |
Free forum by Nabble | Edit this page |