Very bad about Squeak in blogosfere

> Hi all!
>
> (long post, sorry!)
>
> Andreas Raab <[hidden email]> wrote:
>  
>> Colin Putney wrote:
>>    
>>> It's stated a bit harshly, but yeah, that sounds basically accurate. The
>>> amazing thing is that, in spite of all that, Squeak is still such a
>>> wonderful platform to work with. I do use Squeak in production, and
>>> there are very few things I would trade it for.
>>>      
>> Well, yes, but you can't deny that the guy's got a point. The
>> frustration he's expressing is something that everyone has felt over the
>> years. And while there are various plain invalid points in his post
>> (like the fact that Squeak has bugs - I'm *shocked* to hear that of
>> course and would have never started three products if I'd known that ;-)
>> the main emerging point is valid: The lack of quality and maintenance.
>> The problems he cites are all known, some of them even have fixes but
>> there isn't enough traction in the community to make this all come
>>    
> ...
>  
>> together. And of course the forks don't exactly help because we still
>> haven't figured out how to share code across the forks and consequently
>>    
>
> And Andreas continues to describe the problem that I have been thinking
> a bit about during my vacation.
>
> Last night I wrote it up and intended to bounce it around "privately"
> first on selected people, but reading this thread it is so inviting to
> share the idea so what the hell :) Forgive me if this post is long - but
> I want to paint my picture as best I can.
>
> IMO these are our major problems today (we have lots of minor ones too
> and the problems intermix):
>
> 1. More or less anarchy when it comes to leadership of Squeak-dev. The
> board is not taking charge, for a whole range of possible reasons. One
> obvious reason is that the guys elected are really good developers who
> tend to be very busy. Another reason may be that we have different
> expectations on what they should/could be doing. IMPORTANT: I am *not*
> placing any blame though - we are all in this together.
>
> 2. We are now "de facto" living in a highly forked world (Croquet,
> Sophie, Squeak-dev, Spoon, Squeakland, OLPC, yaddayadda...) but we don't
> have tools that support such a world!
>
> 3. The core Squeak is not evolving properly mainly due to the fact that
> most of the core codebase is "abandoned" - or in other words, noone
> feels responsible and/or authoritive to bring it forward.
>
> In short we have "leadership paralysis", a "forked world" and a largely
> "abandoned core". Other problems I missed? Of course! ...but lets ignore
> those for this post.
>
>
> Considering a few measures then:
>
> - Make a new fork with a strong leadership? Hmmm, could help with #1 and
> #3 above but does nothing to improve #2 and would probably be tons of
> work to succeed, and I personally don't have the required time to spend.
> Plus I am a hard core "squeak-dev" member - I don't *want* to fork. :)
>
> - Reshape the organisation of Squeak-dev? Nah, I am sick and tired of
> pulling such work, done just too much of that stuff already over the
> years, and it would possibly help with #1 and #3 (strong leadership) but
> again would not help at all with #2.
>
> So ok, forking is out and reshaping the organisation from the inside is
> out, even though I really would want much more initiative from the board
> in the future - no doubt about that.
>
>
> Hmmm. But what could be done then? Well, after seeing over and over
> again how really good TOOLS can change the way we interact in our
> community (Monticello, SqueakMap, SqueakSource, Universes etc) I tried
> to imagine a new infrastructure that would help with the three problems
> above, in a *natural* way.
>
> Let's look at #1 - the anarchy problem. If we don't try to solve it but
> instead try to *live with it* - what could that mean? The piece that
> suffers mostly from the anarchy is of course the base image. All our
> external packages prosper just fine anyway (kinda). So could we put some
> kind of "supertool" in place to maintain the base image that could be
> made to work *without* strong leadership? I think we can.
>
> What about #2 - the forking problem? Well, imagine this "supertool"
> being written in such a way that it can be used in all major forks - if
> its good and easy to adapt/install it would most probably also *be*
> used. Monticello has already showed this can be done.
>
> So instead of trying *not* to fork, we try to make it very easy to fork
> - or rather *easy to live* in a forked world of Squeak
> images/communities.
>
> But #3 - the rotting core problem? Again, if it was very easy to fix
> stuff in the core, publish these changes without having to ask anyone
> for permission and very easy to cherry-pick such fixes from other people
> - then hopefully the core would again start moving forward. As Andreas
> notes - fixes are sitting inside Croquet and my bet is that they are
> mainly sitting there because it is a bit "too much work" to push them
> out. I am aware of fixes sitting in Gjallar (at least a few) just
> because I didn't feel I had time to fix/prepare them sufficiently etc.
>
>
> So my "daft" idea is:
>
> Let's pull together a good team and make a Supertool that is written to
> be used in all forks that... well, what will it actually *do*? :)
>
>
> 1. Introduce a new kind of source unit - let's call it a Delta. The guys
> working with MC2/Monticello may have lots of noteworthy stuff to share
> about how this kind of thing could look, but I simply want a "changeset"
> for the 21st century. A patch. But a smarter one! The philosophy here
> being that changesets are nice given their simplicity and malleability -
> but they are not so nice in many other respects. But the concept of
> communicating with other developers using the natural "work unit" - a
> "commit" - is quite nice. In MC at least I tend to make too large
> snapshots because let's face it - MC is too slow for small commits -
> given how it works. We just don't suffer because it is so darn slick at
> merging :)
>
> 2. Create the notion of "delta streams". Yes, we had the update stream
> earlier - a sequential flow of changesets. It was kinda nifty, but
> imagine having tons of these streams originating both from larger
> cooperative projects (Croquet, Sophie, Seaside, Gjallar etc) and from
> individual developers ("Andreas Raab's kernel fixes", "Juan's
> refactorings of Morphic" etc). A Delta could appear in several streams
> so Croquet could have a single stream for each release AND multiple
> streams for different kinds of Deltas, like "base image fixes for
> Hedgehog" etc).
>
> 3. Make an efficient but simple storage model for deltas and delta
> streams. Here I am thinking KISS. If a delta could be represented as a
> single gzipped file (enabling dumb servers just like MC does) and a
> stream can be represented by file naming conventions (by a number
> suffix) - then a stream is just a directory or a zip of a bunch of delta
> files. Yes, very similar to gzipped changesets and the update stream. :)
> This enables us to use all "vanilla" tools available for dealing with
> files (http servers, ftp, rsync, email etc).
>
> 4. Make a simple but efficient transport. Let's say we set up a public
> server that syncs such directories of delta files (streams) from tons of
> places. And then offers it to all of us as a simple rsync. Each
> developer (or team) could then use rsync to mirror that mega tree of
> delta files onto our laptops/servers and the tool inside Squeak would
> just need to bother with reading the local filesystem - which makes it
> much more portable across Squeak dialects (and possibly even Smalltalk
> dialects - but let's not go there just yet). This also means we don't
> suffer from servers being down.
>
> 4. Make a simple model of Delta and DeltaStream in Squeak and let it
> mirror the filesystem. Add a simple API to the model á la Keith's
> Installer. Or hey, just enable Installer work with this model.
>
> 5. Make a great Morphic tool to work with them. Perhaps we could even
> rework the changesorter family of tools thus superceding changesets?
>
>
> Ok, imagine this supertool in your image. Imagine that all public
> streams are listed directly in the tool and new ones just "pop up" as
> they are added. Imagine that you can add your own private streams just
> like you can add repositories to sources.list in apt-get (you Debian
> people know what I mean). You can have purely private local streams on
> your harddrive or streams on a shared fileserver in your company/dev
> group, etc.
>
> Now you have this huge flood of Deltas in hundreds of streams at your
> fingertips. How would you use it?
>
> 1. Subscribe to some major streams and configure it to automatically
> load deltas whenever they appear when you press "update".
>
> 2. Set some rules that govern if the tool should just try to load - or
> ask for confirmation based on characteristics of the delta.
>
> 3. Autosubscribe to categories of streams. For example, if a new bug-fix
> stream appears for a package you use a lot - you might want to get it
> autosusbcribed - or again, have the tool ask you.
>
> 4. Of course, easily publish your own streams. We are talking single
> button, no getting permissions etc.
>
> 5. Push fixes to other people's packages as deltas onto a personal
> public fixes-stream. This means it does not matter what package you are
> bug fixing - you can *always* push the fix to your personal public
> fixes-stream and don't need to bother with getting permissions on
> SqueakSource or wherever it came from. This is a very IMPORTANT feature
> and should hopefully put an end to "lost fixes" or "fixes sitting inside
> images".
>
> We have too many fixes out there that just never get published because
> of the "hassle", and sure, someone says "it is very easy to upload to
> Mantis!" - but fixes on Mantis sets expectations on quality,
> documentation, follow up etc. A personal fixes stream like above sets no
> such expectations - it is there for the taking - but that is all.
>
> 6. Pull deltas. Selective cherry picking etc.
>
>
> Important features of Deltas:
>
> - Atomic load of a Delta. Either it loads cleanly or it does not load at
> all, and it ensures this by checking FIRST that everything is in place
> for it to be able to apply cleanly. This should prevent "failed loads"
> and broken images. The actual low level atomicity is another story but
> SystemEditor from MC2 perhaps? I dunno.
>
> - Revertable, if *possible*. A delta can be analyzed to see if it is
> revertable. If it includes doits or class initializations it is in
> theory not revertable, it may be in practice though! It can also be
> marked as DefinitelyNonRevertable.
>
> - A Delta is declarative. Another class should be responsible for
> actually applying them.
>
> - When a Delta is applied to an image we generate a reverse Delta which
> (when applied) will reverse the effects. Since the image may be in
> different states this reverse Delta needs to be constructed when the
> Delta is applied!
>
>
> A Delta contains:
>
> - A Preamble similar to ChangeSets with info fields
> - Developer (name, id, signature etc whatever)
> - Original stream (URL)
> - UUID
> - DefinitelyNotRevertable flag
> - Test doit (a non destructive, non interactive doit that should throw
> an exception in order to prevent loading!)
> - An ordered sequence of Actions, see below.
>
> An example list of different types of Actions are:
>
> - Change method (class name, old method src, new method src)
> - Add method (class name, method src, category name)
> - Remove method (class name, method src, category name)
> - Categorize method (class name, method name, old category name, new
> category name)
>
> - Create class (class name, super class name, definition, category)
> - Delete class (class name)
> - Change superclass of class (class name, old super class name, new
> super class name)
> - Rename class (old class name, new class name)
> - Categorize class (class name, old category name, new category name)
> - Change definition (class name, new definition, old definition)
> - Class comment change (new comment, old comment)
>
> - Class initialization
> - Doit (marked as revertable or not by author!)
>
> Note how these Actions contain "more" info than a ChangeSet - it
> contains information about what it was "before" in the image it
> originates from! The idea is to make Deltas "rich" with info so that we
> can apply them with more smarts.
>
> When applied to an image the Delta is copied to a directory with the
> same name as the image followed by "-applied-deltas". It is also logged
> in changesfile. If the Delta can not be cleanly reverted (based solely
> on its own contents) we generate reverse Deltas and store them in the
> same directory prefixed with "reverse-".
>
> Applying a Delta:
>
> 1. Verify that it can be applied (or signal CanNotApplyException). Do
> this by analyzing Actions and running any Test and see if it throws an
> Exception.
> - It can not be applied *cleanly* if something is "different" from
> expected.
> - It can be applied *dirty* if all changes can be applied in "some
> fashion". For example, a delete operation and the thing to delete is
> already deleted. The reverse Delta will then show this so that a revert
> will not put the thing back in.
> - It can be applied *partially* if just a subset of Actions are applied
> (by choice or since some just can't be applied). The reverse Delta will
> then show what was applied or not.
>
> 2. Verify that it can be reverted cleanly (or signal
> CanNotRevertCleanlyException). Do this by analyzing Actions.
>
> 3. Apply and generate the reverse Delta if needed.
>
> 4. Copy delta and any reverse delta to applied dir.
>
> 5. Log to changes file.
>
> 6. Report package touched (for running unit tests afterward!)
>
>
> What about merging then? Well, the concept here is to be much more
> "loose" when it comes to merging compared to say MC. It was inspired by
> something I read about git (Linus' SCM tool) - it doesn't try to be
> smart - it just tries to do the right thing when it is obvious. In other
> situations it just asks the developer. I think I like this approach. The
> Delta model should be easy enough for all developers to understand. It
> is "just" a changeset/patch after all.
>
> So if I am trying to apply a Delta (or several) and the "before state"
> is not as the Delta expects - we can either cop out, or let it be
> "smart", or just ask the user to decide what to do. BUT... I am not an
> SCM implementor - the MC/MC2 guys can clearly explain to me why/how this
> will not work or amend the idea so that it can work. I gladly admit my
> ignorance. ;)
>
>
> ...ok, enough blabbering. :) Does this sound plausible, useful or just
> plain dumb? Is MC2 already this and much more?
>
> ciao, Göran
>
>
>  

>
>
>
> El 8/13/07 5:47 AM, "[hidden email]" <[hidden email]> escribió:
>
> > Hi all!
> >
> > (long post, sorry!)
>
> Gorän:
> I cut you post for not doing more long.
> I read and re-read carefully and you put very clear my own in the works
> ideas about the problems you describe and give a possible solution.
>
> Only I wish you know I ready for work with you or any masters joining to the
> Supertool quest.
>
> Awaiting orders !
>
> Cheers.
>
> Edgar
>
>
>
>

> -----Original Message-----
> From: Craig Latta
>
>
> Hi Ivan--
>
> > Correct me if I am wrong, but the original was not about the people
> > who were developing Squeak. It was about Squeak as such, not the good
> > or bad will of the people who selflessly work on it.
>
>      Actually, I think the comment that "the maintainer of the main VM
> code publically stated he will do shit unless someone pays him" is
> pretty clearly a personal attack.
>
>
> -C
>
> --
> Craig Latta
> improvisational musical informaticist
> www.netjam.org
> Smalltalkers do: [:it | All with: Class, (And love: it)]
>

> Hi Goran,
>
> I'd love to have such a tool! The only thing I'd like to stress is that
> in order to be usable by all the forks, it should not rely on too new or
> advanced features. And those requeriments should be well understood. For
> me to be able to use it, it should not require Monticello, Traits or any
> of the advanced browsers. If it turns out to be incompatible with
> Morphic 3.0 I'd be glad to adapt or rewrite the UI.
>
> Maybe other forks could have some other restrictions. For example, can
> Croquet run Morphic UIs (besides Tweak)?
>
> Cheers,
> Juan Vuletich
> www.jvuletich.org
>
> [hidden email] wrote:
> > Hi all!
> >
> > (long post, sorry!)
> >
> > Andreas Raab <[hidden email]> wrote:
> >
> >> Colin Putney wrote:
> >>
> >>> It's stated a bit harshly, but yeah, that sounds basically accurate. The
> >>> amazing thing is that, in spite of all that, Squeak is still such a
> >>> wonderful platform to work with. I do use Squeak in production, and
> >>> there are very few things I would trade it for.
> >>>
> >> Well, yes, but you can't deny that the guy's got a point. The
> >> frustration he's expressing is something that everyone has felt over the
> >> years. And while there are various plain invalid points in his post
> >> (like the fact that Squeak has bugs - I'm *shocked* to hear that of
> >> course and would have never started three products if I'd known that ;-)
> >> the main emerging point is valid: The lack of quality and maintenance.
> >> The problems he cites are all known, some of them even have fixes but
> >> there isn't enough traction in the community to make this all come
> >>
> > ...
> >
> >> together. And of course the forks don't exactly help because we still
> >> haven't figured out how to share code across the forks and consequently
> >>
> >
> > And Andreas continues to describe the problem that I have been thinking
> > a bit about during my vacation.
> >
> > Last night I wrote it up and intended to bounce it around "privately"
> > first on selected people, but reading this thread it is so inviting to
> > share the idea so what the hell :) Forgive me if this post is long - but
> > I want to paint my picture as best I can.
> >
> > IMO these are our major problems today (we have lots of minor ones too
> > and the problems intermix):
> >
> > 1. More or less anarchy when it comes to leadership of Squeak-dev. The
> > board is not taking charge, for a whole range of possible reasons. One
> > obvious reason is that the guys elected are really good developers who
> > tend to be very busy. Another reason may be that we have different
> > expectations on what they should/could be doing. IMPORTANT: I am *not*
> > placing any blame though - we are all in this together.
> >
> > 2. We are now "de facto" living in a highly forked world (Croquet,
> > Sophie, Squeak-dev, Spoon, Squeakland, OLPC, yaddayadda...) but we don't
> > have tools that support such a world!
> >
> > 3. The core Squeak is not evolving properly mainly due to the fact that
> > most of the core codebase is "abandoned" - or in other words, noone
> > feels responsible and/or authoritive to bring it forward.
> >
> > In short we have "leadership paralysis", a "forked world" and a largely
> > "abandoned core". Other problems I missed? Of course! ...but lets ignore
> > those for this post.
> >
> >
> > Considering a few measures then:
> >
> > - Make a new fork with a strong leadership? Hmmm, could help with #1 and
> > #3 above but does nothing to improve #2 and would probably be tons of
> > work to succeed, and I personally don't have the required time to spend.
> > Plus I am a hard core "squeak-dev" member - I don't *want* to fork. :)
> >
> > - Reshape the organisation of Squeak-dev? Nah, I am sick and tired of
> > pulling such work, done just too much of that stuff already over the
> > years, and it would possibly help with #1 and #3 (strong leadership) but
> > again would not help at all with #2.
> >
> > So ok, forking is out and reshaping the organisation from the inside is
> > out, even though I really would want much more initiative from the board
> > in the future - no doubt about that.
> >
> >
> > Hmmm. But what could be done then? Well, after seeing over and over
> > again how really good TOOLS can change the way we interact in our
> > community (Monticello, SqueakMap, SqueakSource, Universes etc) I tried
> > to imagine a new infrastructure that would help with the three problems
> > above, in a *natural* way.
> >
> > Let's look at #1 - the anarchy problem. If we don't try to solve it but
> > instead try to *live with it* - what could that mean? The piece that
> > suffers mostly from the anarchy is of course the base image. All our
> > external packages prosper just fine anyway (kinda). So could we put some
> > kind of "supertool" in place to maintain the base image that could be
> > made to work *without* strong leadership? I think we can.
> >
> > What about #2 - the forking problem? Well, imagine this "supertool"
> > being written in such a way that it can be used in all major forks - if
> > its good and easy to adapt/install it would most probably also *be*
> > used. Monticello has already showed this can be done.
> >
> > So instead of trying *not* to fork, we try to make it very easy to fork
> > - or rather *easy to live* in a forked world of Squeak
> > images/communities.
> >
> > But #3 - the rotting core problem? Again, if it was very easy to fix
> > stuff in the core, publish these changes without having to ask anyone
> > for permission and very easy to cherry-pick such fixes from other people
> > - then hopefully the core would again start moving forward. As Andreas
> > notes - fixes are sitting inside Croquet and my bet is that they are
> > mainly sitting there because it is a bit "too much work" to push them
> > out. I am aware of fixes sitting in Gjallar (at least a few) just
> > because I didn't feel I had time to fix/prepare them sufficiently etc.
> >
> >
> > So my "daft" idea is:
> >
> > Let's pull together a good team and make a Supertool that is written to
> > be used in all forks that... well, what will it actually *do*? :)
> >
> >
> > 1. Introduce a new kind of source unit - let's call it a Delta. The guys
> > working with MC2/Monticello may have lots of noteworthy stuff to share
> > about how this kind of thing could look, but I simply want a "changeset"
> > for the 21st century. A patch. But a smarter one! The philosophy here
> > being that changesets are nice given their simplicity and malleability -
> > but they are not so nice in many other respects. But the concept of
> > communicating with other developers using the natural "work unit" - a
> > "commit" - is quite nice. In MC at least I tend to make too large
> > snapshots because let's face it - MC is too slow for small commits -
> > given how it works. We just don't suffer because it is so darn slick at
> > merging :)
> >
> > 2. Create the notion of "delta streams". Yes, we had the update stream
> > earlier - a sequential flow of changesets. It was kinda nifty, but
> > imagine having tons of these streams originating both from larger
> > cooperative projects (Croquet, Sophie, Seaside, Gjallar etc) and from
> > individual developers ("Andreas Raab's kernel fixes", "Juan's
> > refactorings of Morphic" etc). A Delta could appear in several streams
> > so Croquet could have a single stream for each release AND multiple
> > streams for different kinds of Deltas, like "base image fixes for
> > Hedgehog" etc).
> >
> > 3. Make an efficient but simple storage model for deltas and delta
> > streams. Here I am thinking KISS. If a delta could be represented as a
> > single gzipped file (enabling dumb servers just like MC does) and a
> > stream can be represented by file naming conventions (by a number
> > suffix) - then a stream is just a directory or a zip of a bunch of delta
> > files. Yes, very similar to gzipped changesets and the update stream. :)
> > This enables us to use all "vanilla" tools available for dealing with
> > files (http servers, ftp, rsync, email etc).
> >
> > 4. Make a simple but efficient transport. Let's say we set up a public
> > server that syncs such directories of delta files (streams) from tons of
> > places. And then offers it to all of us as a simple rsync. Each
> > developer (or team) could then use rsync to mirror that mega tree of
> > delta files onto our laptops/servers and the tool inside Squeak would
> > just need to bother with reading the local filesystem - which makes it
> > much more portable across Squeak dialects (and possibly even Smalltalk
> > dialects - but let's not go there just yet). This also means we don't
> > suffer from servers being down.
> >
> > 4. Make a simple model of Delta and DeltaStream in Squeak and let it
> > mirror the filesystem. Add a simple API to the model á la Keith's
> > Installer. Or hey, just enable Installer work with this model.
> >
> > 5. Make a great Morphic tool to work with them. Perhaps we could even
> > rework the changesorter family of tools thus superceding changesets?
> >
> >
> > Ok, imagine this supertool in your image. Imagine that all public
> > streams are listed directly in the tool and new ones just "pop up" as
> > they are added. Imagine that you can add your own private streams just
> > like you can add repositories to sources.list in apt-get (you Debian
> > people know what I mean). You can have purely private local streams on
> > your harddrive or streams on a shared fileserver in your company/dev
> > group, etc.
> >
> > Now you have this huge flood of Deltas in hundreds of streams at your
> > fingertips. How would you use it?
> >
> > 1. Subscribe to some major streams and configure it to automatically
> > load deltas whenever they appear when you press "update".
> >
> > 2. Set some rules that govern if the tool should just try to load - or
> > ask for confirmation based on characteristics of the delta.
> >
> > 3. Autosubscribe to categories of streams. For example, if a new bug-fix
> > stream appears for a package you use a lot - you might want to get it
> > autosusbcribed - or again, have the tool ask you.
> >
> > 4. Of course, easily publish your own streams. We are talking single
> > button, no getting permissions etc.
> >
> > 5. Push fixes to other people's packages as deltas onto a personal
> > public fixes-stream. This means it does not matter what package you are
> > bug fixing - you can *always* push the fix to your personal public
> > fixes-stream and don't need to bother with getting permissions on
> > SqueakSource or wherever it came from. This is a very IMPORTANT feature
> > and should hopefully put an end to "lost fixes" or "fixes sitting inside
> > images".
> >
> > We have too many fixes out there that just never get published because
> > of the "hassle", and sure, someone says "it is very easy to upload to
> > Mantis!" - but fixes on Mantis sets expectations on quality,
> > documentation, follow up etc. A personal fixes stream like above sets no
> > such expectations - it is there for the taking - but that is all.
> >
> > 6. Pull deltas. Selective cherry picking etc.
> >
> >
> > Important features of Deltas:
> >
> > - Atomic load of a Delta. Either it loads cleanly or it does not load at
> > all, and it ensures this by checking FIRST that everything is in place
> > for it to be able to apply cleanly. This should prevent "failed loads"
> > and broken images. The actual low level atomicity is another story but
> > SystemEditor from MC2 perhaps? I dunno.
> >
> > - Revertable, if *possible*. A delta can be analyzed to see if it is
> > revertable. If it includes doits or class initializations it is in
> > theory not revertable, it may be in practice though! It can also be
> > marked as DefinitelyNonRevertable.
> >
> > - A Delta is declarative. Another class should be responsible for
> > actually applying them.
> >
> > - When a Delta is applied to an image we generate a reverse Delta which
> > (when applied) will reverse the effects. Since the image may be in
> > different states this reverse Delta needs to be constructed when the
> > Delta is applied!
> >
> >
> > A Delta contains:
> >
> > - A Preamble similar to ChangeSets with info fields
> >       - Developer (name, id, signature etc whatever)
> >       - Original stream (URL)
> >       - UUID
> >       - DefinitelyNotRevertable flag
> >       - Test doit (a non destructive, non interactive doit that should throw
> > an exception in order to prevent loading!)
> > - An ordered sequence of Actions, see below.
> >
> > An example list of different types of Actions are:
> >
> > - Change method (class name, old method src, new method src)
> > - Add method (class name, method src, category name)
> > - Remove method (class name, method src, category name)
> > - Categorize method (class name, method name, old category name, new
> > category name)
> >
> > - Create class (class name, super class name, definition, category)
> > - Delete class (class name)
> > - Change superclass of class (class name, old super class name, new
> > super class name)
> > - Rename class (old class name, new class name)
> > - Categorize class (class name, old category name, new category name)
> > - Change definition (class name, new definition, old definition)
> > - Class comment change (new comment, old comment)
> >
> > - Class initialization
> > - Doit (marked as revertable or not by author!)
> >
> > Note how these Actions contain "more" info than a ChangeSet - it
> > contains information about what it was "before" in the image it
> > originates from! The idea is to make Deltas "rich" with info so that we
> > can apply them with more smarts.
> >
> > When applied to an image the Delta is copied to a directory with the
> > same name as the image followed by "-applied-deltas". It is also logged
> > in changesfile. If the Delta can not be cleanly reverted (based solely
> > on its own contents) we generate reverse Deltas and store them in the
> > same directory prefixed with "reverse-".
> >
> > Applying a Delta:
> >
> > 1. Verify that it can be applied (or signal CanNotApplyException). Do
> > this by analyzing Actions and running any Test and see if it throws an
> > Exception.
> >       - It can not be applied *cleanly* if something is "different" from
> > expected.
> >       - It can be applied *dirty* if all changes can be applied in "some
> > fashion". For example, a delete operation and the thing to delete is
> > already deleted. The reverse Delta will then show this so that a revert
> > will not put the thing back in.
> >       - It can be applied *partially* if just a subset of Actions are applied
> > (by choice or since some just can't be applied). The reverse Delta will
> > then show what was applied or not.
> >
> > 2. Verify that it can be reverted cleanly (or signal
> > CanNotRevertCleanlyException). Do this by analyzing Actions.
> >
> > 3. Apply and generate the reverse Delta if needed.
> >
> > 4. Copy delta and any reverse delta to applied dir.
> >
> > 5. Log to changes file.
> >
> > 6. Report package touched (for running unit tests afterward!)
> >
> >
> > What about merging then? Well, the concept here is to be much more
> > "loose" when it comes to merging compared to say MC. It was inspired by
> > something I read about git (Linus' SCM tool) - it doesn't try to be
> > smart - it just tries to do the right thing when it is obvious. In other
> > situations it just asks the developer. I think I like this approach. The
> > Delta model should be easy enough for all developers to understand. It
> > is "just" a changeset/patch after all.
> >
> > So if I am trying to apply a Delta (or several) and the "before state"
> > is not as the Delta expects - we can either cop out, or let it be
> > "smart", or just ask the user to decide what to do. BUT... I am not an
> > SCM implementor - the MC/MC2 guys can clearly explain to me why/how this
> > will not work or amend the idea so that it can work. I gladly admit my
> > ignorance. ;)
> >
> >
> > ...ok, enough blabbering. :) Does this sound plausible, useful or just
> > plain dumb? Is MC2 already this and much more?
> >
> > ciao, Göran
> >
> >
> >
>
>
>

> -----Original Message-----
> From: Brad Fuller
>
> Göran,
>
> I'm surprised with the lack of response to your post. Maybe people are
> letting
> it sink in. I think it's a great idea. FMPOV, the tool would be a clean
> way
> for the developer and user to accept bug fixes into his image. Why not
> make
> bug fixes easily available to everyone? It's also a nice way to put a
> method
> around all the fixes just lying there in various forms and places. And, it
> seems a nice way to update released images on ftp.squeak.org with proper
> fixes - at least the fairly new ones.
>
> The concept and high-level usage is a no-brainer. The tough part is
> getting it
> done.  Since a lot of the work is creating the tool description, and you
> have
> a great handle on that already, I wouldn't be surprised if you've already
> started coding.
>
> brad

> Hi Brad,
>
> It's not just sinking in that is an issue.  I'm also considering the
> practical aspects.  Also I'm not exactly sure how this all fits with
> Andreas' suggestion of managing packages outside the context of an official
> image.
>
> I can't help seeing the down side to all of this, and I wonder what the
> impact will be for those starting out with Squeak.
>
> The issue with too much freedom is that anyone is allowed to break things.
> We have enough trouble as it is with class clutter.  Even I tried
> (unsuccessfully) to add to Collection.
>
> So let's say we go with this path and anyone can start a stream.  How does a
> user decide which stream to use?  If we have two very good streams (like
> Sophie and Croquet) which are changing classes in an incompatible way then
> someone's changes are still lost.
>

> Hi all!
>
> (long post, sorry!)
>
> Andreas Raab <[hidden email]> wrote:
>> Colin Putney wrote:
>>> It's stated a bit harshly, but yeah, that sounds basically accurate. The
>>> amazing thing is that, in spite of all that, Squeak is still such a
>>> wonderful platform to work with. I do use Squeak in production, and
>>> there are very few things I would trade it for.
>> Well, yes, but you can't deny that the guy's got a point. The
>> frustration he's expressing is something that everyone has felt over the
>> years. And while there are various plain invalid points in his post
>> (like the fact that Squeak has bugs - I'm *shocked* to hear that of
>> course and would have never started three products if I'd known that ;-)
>> the main emerging point is valid: The lack of quality and maintenance.
>> The problems he cites are all known, some of them even have fixes but
>> there isn't enough traction in the community to make this all come
> ...
>> together. And of course the forks don't exactly help because we still
>> haven't figured out how to share code across the forks and consequently
>
> And Andreas continues to describe the problem that I have been thinking
> a bit about during my vacation.
>
> Last night I wrote it up and intended to bounce it around "privately"
> first on selected people, but reading this thread it is so inviting to
> share the idea so what the hell :) Forgive me if this post is long - but
> I want to paint my picture as best I can.
>
> IMO these are our major problems today (we have lots of minor ones too
> and the problems intermix):
>
> 1. More or less anarchy when it comes to leadership of Squeak-dev. The
> board is not taking charge, for a whole range of possible reasons. One
> obvious reason is that the guys elected are really good developers who
> tend to be very busy. Another reason may be that we have different
> expectations on what they should/could be doing. IMPORTANT: I am *not*
> placing any blame though - we are all in this together.
>
> 2. We are now "de facto" living in a highly forked world (Croquet,
> Sophie, Squeak-dev, Spoon, Squeakland, OLPC, yaddayadda...) but we don't
> have tools that support such a world!
>
> 3. The core Squeak is not evolving properly mainly due to the fact that
> most of the core codebase is "abandoned" - or in other words, noone
> feels responsible and/or authoritive to bring it forward.
>
> In short we have "leadership paralysis", a "forked world" and a largely
> "abandoned core". Other problems I missed? Of course! ...but lets ignore
> those for this post.
>
>
> Considering a few measures then:
>
> - Make a new fork with a strong leadership? Hmmm, could help with #1 and
> #3 above but does nothing to improve #2 and would probably be tons of
> work to succeed, and I personally don't have the required time to spend.
> Plus I am a hard core "squeak-dev" member - I don't *want* to fork. :)
>
> - Reshape the organisation of Squeak-dev? Nah, I am sick and tired of
> pulling such work, done just too much of that stuff already over the
> years, and it would possibly help with #1 and #3 (strong leadership) but
> again would not help at all with #2.
>
> So ok, forking is out and reshaping the organisation from the inside is
> out, even though I really would want much more initiative from the board
> in the future - no doubt about that.
>
>
> Hmmm. But what could be done then? Well, after seeing over and over
> again how really good TOOLS can change the way we interact in our
> community (Monticello, SqueakMap, SqueakSource, Universes etc) I tried
> to imagine a new infrastructure that would help with the three problems
> above, in a *natural* way.
>
> Let's look at #1 - the anarchy problem. If we don't try to solve it but
> instead try to *live with it* - what could that mean? The piece that
> suffers mostly from the anarchy is of course the base image. All our
> external packages prosper just fine anyway (kinda). So could we put some
> kind of "supertool" in place to maintain the base image that could be
> made to work *without* strong leadership? I think we can.
>
> What about #2 - the forking problem? Well, imagine this "supertool"
> being written in such a way that it can be used in all major forks - if
> its good and easy to adapt/install it would most probably also *be*
> used. Monticello has already showed this can be done.
>
> So instead of trying *not* to fork, we try to make it very easy to fork
> - or rather *easy to live* in a forked world of Squeak
> images/communities.
>
> But #3 - the rotting core problem? Again, if it was very easy to fix
> stuff in the core, publish these changes without having to ask anyone
> for permission and very easy to cherry-pick such fixes from other people
> - then hopefully the core would again start moving forward. As Andreas
> notes - fixes are sitting inside Croquet and my bet is that they are
> mainly sitting there because it is a bit "too much work" to push them
> out. I am aware of fixes sitting in Gjallar (at least a few) just
> because I didn't feel I had time to fix/prepare them sufficiently etc.
>
>
> So my "daft" idea is:
>
> Let's pull together a good team and make a Supertool that is written to
> be used in all forks that... well, what will it actually *do*? :)
>
>
> 1. Introduce a new kind of source unit - let's call it a Delta. The guys
> working with MC2/Monticello may have lots of noteworthy stuff to share
> about how this kind of thing could look, but I simply want a "changeset"
> for the 21st century. A patch. But a smarter one! The philosophy here
> being that changesets are nice given their simplicity and malleability -
> but they are not so nice in many other respects. But the concept of
> communicating with other developers using the natural "work unit" - a
> "commit" - is quite nice. In MC at least I tend to make too large
> snapshots because let's face it - MC is too slow for small commits -
> given how it works. We just don't suffer because it is so darn slick at
> merging :)
>
> 2. Create the notion of "delta streams". Yes, we had the update stream
> earlier - a sequential flow of changesets. It was kinda nifty, but
> imagine having tons of these streams originating both from larger
> cooperative projects (Croquet, Sophie, Seaside, Gjallar etc) and from
> individual developers ("Andreas Raab's kernel fixes", "Juan's
> refactorings of Morphic" etc). A Delta could appear in several streams
> so Croquet could have a single stream for each release AND multiple
> streams for different kinds of Deltas, like "base image fixes for
> Hedgehog" etc).
>
> 3. Make an efficient but simple storage model for deltas and delta
> streams. Here I am thinking KISS. If a delta could be represented as a
> single gzipped file (enabling dumb servers just like MC does) and a
> stream can be represented by file naming conventions (by a number
> suffix) - then a stream is just a directory or a zip of a bunch of delta
> files. Yes, very similar to gzipped changesets and the update stream. :)
> This enables us to use all "vanilla" tools available for dealing with
> files (http servers, ftp, rsync, email etc).
>
> 4. Make a simple but efficient transport. Let's say we set up a public
> server that syncs such directories of delta files (streams) from tons of
> places. And then offers it to all of us as a simple rsync. Each
> developer (or team) could then use rsync to mirror that mega tree of
> delta files onto our laptops/servers and the tool inside Squeak would
> just need to bother with reading the local filesystem - which makes it
> much more portable across Squeak dialects (and possibly even Smalltalk
> dialects - but let's not go there just yet). This also means we don't
> suffer from servers being down.
>
> 4. Make a simple model of Delta and DeltaStream in Squeak and let it
> mirror the filesystem. Add a simple API to the model á la Keith's
> Installer. Or hey, just enable Installer work with this model.
>
> 5. Make a great Morphic tool to work with them. Perhaps we could even
> rework the changesorter family of tools thus superceding changesets?
>
>
> Ok, imagine this supertool in your image. Imagine that all public
> streams are listed directly in the tool and new ones just "pop up" as
> they are added. Imagine that you can add your own private streams just
> like you can add repositories to sources.list in apt-get (you Debian
> people know what I mean). You can have purely private local streams on
> your harddrive or streams on a shared fileserver in your company/dev
> group, etc.
>
> Now you have this huge flood of Deltas in hundreds of streams at your
> fingertips. How would you use it?
>
> 1. Subscribe to some major streams and configure it to automatically
> load deltas whenever they appear when you press "update".
>
> 2. Set some rules that govern if the tool should just try to load - or
> ask for confirmation based on characteristics of the delta.
>
> 3. Autosubscribe to categories of streams. For example, if a new bug-fix
> stream appears for a package you use a lot - you might want to get it
> autosusbcribed - or again, have the tool ask you.
>
> 4. Of course, easily publish your own streams. We are talking single
> button, no getting permissions etc.
>
> 5. Push fixes to other people's packages as deltas onto a personal
> public fixes-stream. This means it does not matter what package you are
> bug fixing - you can *always* push the fix to your personal public
> fixes-stream and don't need to bother with getting permissions on
> SqueakSource or wherever it came from. This is a very IMPORTANT feature
> and should hopefully put an end to "lost fixes" or "fixes sitting inside
> images".
>
> We have too many fixes out there that just never get published because
> of the "hassle", and sure, someone says "it is very easy to upload to
> Mantis!" - but fixes on Mantis sets expectations on quality,
> documentation, follow up etc. A personal fixes stream like above sets no
> such expectations - it is there for the taking - but that is all.
>
> 6. Pull deltas. Selective cherry picking etc.
>
>
> Important features of Deltas:
>
> - Atomic load of a Delta. Either it loads cleanly or it does not load at
> all, and it ensures this by checking FIRST that everything is in place
> for it to be able to apply cleanly. This should prevent "failed loads"
> and broken images. The actual low level atomicity is another story but
> SystemEditor from MC2 perhaps? I dunno.
>
> - Revertable, if *possible*. A delta can be analyzed to see if it is
> revertable. If it includes doits or class initializations it is in
> theory not revertable, it may be in practice though! It can also be
> marked as DefinitelyNonRevertable.
>
> - A Delta is declarative. Another class should be responsible for
> actually applying them.
>
> - When a Delta is applied to an image we generate a reverse Delta which
> (when applied) will reverse the effects. Since the image may be in
> different states this reverse Delta needs to be constructed when the
> Delta is applied!
>
>
> A Delta contains:
>
> - A Preamble similar to ChangeSets with info fields
> - Developer (name, id, signature etc whatever)
> - Original stream (URL)
> - UUID
> - DefinitelyNotRevertable flag
> - Test doit (a non destructive, non interactive doit that should throw
> an exception in order to prevent loading!)
> - An ordered sequence of Actions, see below.
>
> An example list of different types of Actions are:
>
> - Change method (class name, old method src, new method src)
> - Add method (class name, method src, category name)
> - Remove method (class name, method src, category name)
> - Categorize method (class name, method name, old category name, new
> category name)
>
> - Create class (class name, super class name, definition, category)
> - Delete class (class name)
> - Change superclass of class (class name, old super class name, new
> super class name)
> - Rename class (old class name, new class name)
> - Categorize class (class name, old category name, new category name)
> - Change definition (class name, new definition, old definition)
> - Class comment change (new comment, old comment)
>
> - Class initialization
> - Doit (marked as revertable or not by author!)
>
> Note how these Actions contain "more" info than a ChangeSet - it
> contains information about what it was "before" in the image it
> originates from! The idea is to make Deltas "rich" with info so that we
> can apply them with more smarts.
>
> When applied to an image the Delta is copied to a directory with the
> same name as the image followed by "-applied-deltas". It is also logged
> in changesfile. If the Delta can not be cleanly reverted (based solely
> on its own contents) we generate reverse Deltas and store them in the
> same directory prefixed with "reverse-".
>
> Applying a Delta:
>
> 1. Verify that it can be applied (or signal CanNotApplyException). Do
> this by analyzing Actions and running any Test and see if it throws an
> Exception.
> - It can not be applied *cleanly* if something is "different" from
> expected.
> - It can be applied *dirty* if all changes can be applied in "some
> fashion". For example, a delete operation and the thing to delete is
> already deleted. The reverse Delta will then show this so that a revert
> will not put the thing back in.
> - It can be applied *partially* if just a subset of Actions are applied
> (by choice or since some just can't be applied). The reverse Delta will
> then show what was applied or not.
>
> 2. Verify that it can be reverted cleanly (or signal
> CanNotRevertCleanlyException). Do this by analyzing Actions.
>
> 3. Apply and generate the reverse Delta if needed.
>
> 4. Copy delta and any reverse delta to applied dir.
>
> 5. Log to changes file.
>
> 6. Report package touched (for running unit tests afterward!)
>
>
> What about merging then? Well, the concept here is to be much more
> "loose" when it comes to merging compared to say MC. It was inspired by
> something I read about git (Linus' SCM tool) - it doesn't try to be
> smart - it just tries to do the right thing when it is obvious. In other
> situations it just asks the developer. I think I like this approach. The
> Delta model should be easy enough for all developers to understand. It
> is "just" a changeset/patch after all.
>
> So if I am trying to apply a Delta (or several) and the "before state"
> is not as the Delta expects - we can either cop out, or let it be
> "smart", or just ask the user to decide what to do. BUT... I am not an
> SCM implementor - the MC/MC2 guys can clearly explain to me why/how this
> will not work or amend the idea so that it can work. I gladly admit my
> ignorance. ;)
>
>
> ...ok, enough blabbering. :) Does this sound plausible, useful or just
> plain dumb? Is MC2 already this and much more?
>
> ciao, Göran
>
>