[squeak-dev] The role of Bob, Installer & Co.

Folks -

I just had a very good (off-list) discussion where it was brought to my
attention that one might look at the proposal from the point of view
that it replaces Installer, Bob & Co. in situ and that there is no role
for these tools in the proposed process. Nothing could be further from
the truth.

What I am imagining goes along these lines: Consider Bob running
daily/weekly builds and then checks if any new fixes have appeared on
Mantis. If there is anything new, it loads the fix, runs the tests
inside the image, runs the tests attached to the bug. If this works out
Bob puts his "stamp of approval" on the fix or commits it directly to
the repository. In effect, Bob becomes our main harvester doing all the
boring integration stuff that we can reasonably automate.

This is actually quite similar to what Bob does today - except that Bob
isn't producing an image but rather commits to the trunk. In any case,
from my perspective there is a lot of room for Bob and Installer - be
that in providing up-to-date images from weekly builds, running
automated test suites, and so on.

I have no intention to get rid of Bob and Installer. I hope that with
Keith's help we can make maximum use of both since these are valuable
tools in a successful software development project.

Cheers,
   - Andreas

Andreas Raab wrote: Bob generates and maintains a repository with each build target that it
makes.

Given that there may be 5 of us running our own bob servers why bother
with a centralised (read slow) repository as part of the process.  If we
run a build once a day, the repository with that build target tracks the
changes to that build target day by day. The tricky part is putting
meaningful comments into the repository package comments.

ftp://squeak:[hidden email]/bob/output/BuildBase/Squeak3.10.2-build/_source/

A centralised repository can be used for packages that have been pulled
out of the kernel, to be managed externally. We have that repository
already. squeaksource/311

The whole point of the way the process works is that there would be
several projects going on at the same time in parallel. i.e. 3.10 +
closures is a project going on at the same time as 3.10 + mantis fixes.
Each project starts with a fixed starting point. There isn't one
repository that represents the final output, until that final output,
the integration of the chosen projects e.g. 3.10 + closures + fixes is
generated.

It is a point of policy that the build of 3.11 has to be made up of bits
that could be applied to other images. The task that builds 3.11
collects the knowledge of what went in to 3.11, and in doing so makes it
available for other forks to see what we did and follow suit. They can
do this by subclassing our 3.11 generating task and adapting it there.
When they do this, we get to see what knowledge they captured in their
subclass.

You suggest that bob's output should be to commit things to trunk... but
bob is supposed to build images. The reason for this is that he not only
builds the 3.11 image, but he also simultaneously builds all of the
derived images (e.g. the developer image, the full image etc), which you
need to test against.

It looks to me that you are still looking at the production of the final
image as an incremental process. Its not supposed to be an incremental
process. It is supposed to be an atomic process. However it may take
several goes (one per day for a month) to get that atomic process right.
Technically we should be able to make new releases once a month with
ease, but practically the demand for that number of releases per year
would not be there.

Keith
_______________________________________________
Release mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/release

Before:

The procedure for 3.10 was - upload your fix to mantis and it will be
integrated by the build server whose name used to be Edgar, but is now
Bob. i.e. there were 2 inputs.

1. the design of the image (which for 3.10 included the removal of a
couple of packages)
2. the fixes from mantis integrated manually.

After Bob:

1. the design of the image, a task which loads the latest packages,
reorganises the whole image into a DIFFERENT package structure. (how are
you going to support that with mcms?)
2. the fixes from mantis integrated automatically.

Your scheme:

1. People can contribute to trunk...
2. there is no image design because bob is not building images anymore.
3. People can contribute to mantis and those fixes get integrated into
trunk as you go along. So in your scheme the mantis fixes are now being
applied to a moving target. Which means that other people who want to
see what good a fix is for to see if it might be useful to them, have no
idea what state 3.11 trunk was in when the fix was added to it.

So basically the whole commit to trunk idea is not tenable.. for the
following reasons.

1. There is no design of the final image published and coded in advance.
It is now a free for all for the committers that have been let loose on
trunk

2. The packages are going to be reorganised in the process of the image
design.

a) "System" needs to be broken up
b) The Implementation and Tests need to be logically and tidily
organsied for once.

Keith

On Sat, Jul 04, 2009 at 11:42:31PM +0100, Keith Hodges wrote:
Bob does not "design images". neither do you

> 3. People can contribute to mantis and those fixes get integrated into
> trunk as you go along. So in your scheme the mantis fixes are now being
> applied to a moving target. Which means that other people who want to
> see what good a fix is for to see if it might be useful to them, have no
> idea what state 3.11 trunk was in when the fix was added to it.

You want every contribution to have to flow thru the bottleneck
of mantis? So, instead of

1. fix a bug
2. merge in any changes that may have happened since I started
3. press save in Monticello

You want us to

1. fix a bug
2. file out a changeset
3. Grab the latest image from Bob
4. Reaply the changeset to that image, making sure it works
5. upload that changeset to mantis5.

We have a great tool already for keeping track of code that has
changed. It's called Monticello. You use it all the time. Bob
can load .mcm or .mcz packages into an image just as easily as
it can load changesets from mantis.

Bob is not a taskmaster who controls the image, demeaning our
jobs to just submitting changesets for his approval. We, the
community, build the image, and bob relieves us of the drudgery
of making sure the packages load order is reproducable, and from
waiting 5 hours for the unit tests to finish.

> So basically the whole commit to trunk idea is not tenable.. for the
> following reasons.
>
> 1. There is no design of the final image published and coded in advance.
> It is now a free for all for the committers that have been let loose on
> trunk

if there was a final image published and coded in advance, there
would be no reason to be bothering about a release, since it had
already happened. This makes no sense.

> 2. The packages are going to be reorganised in the process of the image
> design.

they are? first I've heard about it.

> a) "System" needs to be broken up
> b) The Implementation and Tests need to be logically and tidily
> organsied for once.

Bob is not going to do that. We are.

--
Matthew Fulmer -- http://mtfulmer.wordpress.com/

Hmm. What keith is saying is something I wrote, with his help, 9
months ago: http://installer.pbworks.com/Squeak311Proposal

I have to decide if I still agree with that proposal we came up
with. My arguments below seem to indicate that I find serious
flaws in it now.

On Sat, Jul 04, 2009 at 07:15:57PM -0400, Matthew Fulmer wrote:
--
Matthew Fulmer -- http://mtfulmer.wordpress.com/

Matthew Fulmer wrote: I write a task which specifies in code what the design of the output
image is to be. What packages are loaded what packages are unloaded and
what classes are in which package. After that it is merely a process of
filling in the steps of the design, of which "apply fixes" is one step
which tends to appear in every iteration, just as deprecateAPI and
update version number.

Building the image is the bit that needs automating, because most of it
is the same every time. If we want to release early and often, then we
automate the building of the image. The only things that change are the
selection of which packages are included and the selection of which bugs
fixes to apply.

Bob relentlessly and ignorantly applies this task until it happens to
work, if all tests pass then in theory you can ding the extreme release
bell.
>> 3. People can contribute to mantis and those fixes get integrated into
>> trunk as you go along. So in your scheme the mantis fixes are now being
>> applied to a moving target. Which means that other people who want to
>> see what good a fix is for to see if it might be useful to them, have no
>> idea what state 3.11 trunk was in when the fix was added to it.
>>    
>
> You want every contribution to have to flow thru the bottleneck
> of mantis?
Abso-bloody-lutely.

Unless it is an externally managed package.

Or it is a specific project (e.g. migrate image to use closures) whose
result is specified as a task (changesets/DS/scripts or MC packages to
merge) that bob can reliably apply.
>  So, instead of
>
> 1. fix a bug
> 2. merge in any changes that may have happened since I started
> 3. press save in Monticello
>  
And you have helped anyone but yourself or your own fork how?

Who knows what your starting point was?
Who knows or records what exactly you did?  (ok mc does, but I mean in a
way that other people can use in their images)
Who has any opportunity to comment and provide feedback on what you have
done in the context of what you have done.
You are a law unto yourself, and there is no pier review taking place.
Your actions are not repeatable, nor are they undoable, because by the
time you want to undo them, someone else will have merged in your change
to their latest.
Your result cannot be used by anyone else until the release is
published. But we want projects to be able to happen in parralel, these
projects need to be able to pick up the fixes that they need. When a
project is published as a task, it specifies the fixes it needs as
prerequisites.

Your steps are describing the sort of process that you are used to when
you are hacking away at your own code. When we build an image we are not
hacking, we are carefully constructing pre-specified result out of
(hopefully) carefully crafted and documented pieces.

Making trunk open to all committers is saying come on in lets have a
hack fest, this is nothing to do with building a release image.

You can do your hacking in your own time, hack away all you like at the
image, use monticello to manage your code, but when you have the result
you desire you need to find a way to publish it as one of those "well
crafted and documented pieces" I just mentioned. If changesets dont
float your boat then you might have to wait for DS. But MC is no good at
publishing a specific change for someone to load into a 3.8 image, and
it does nothing to document of capture the knowledge or design intent of
what exactly transpired.
> You want us to
>
> 1. fix a bug
> 2. file out a changeset
> 3. Grab the latest image from Bob
>  
No need, use your own starting image, you are working to a known fixed
point remember. 3.10.2 or more sensibly 3.10.2-build (since that is what
bob is starting with on every run)
> 4. Reaply the changeset to that image, making sure it works
>  
Bob will apply the fix to bob's images, and hopefully testing will
reveal if there are any conflicts when the fix is integrated with other
fixes.
> 5. upload that changeset to mantis5.
>  
Yes, because mantis is a tool for discussing fixes and having that
dialog. That's what it is designed for.
MC is a tool for managing code, but it doesnt have any means for
providing feedback, it is not a tool for managing fixes. Once you commit
a fix from mantis to MC it is out of mantis' hands discussion is over,
the code then slips under the radar, it merges into the blur of the
package. It also possibly breaks another fix that someone submitted ages
ago.

Fixes that are on the table for integration into a release should remain
on the table until the release is tested and out of the door.
If a test fails, bob could automatically apply the fixes one by one to
find out what caused the problem, he could reorder the application of
the fixes, or try different combinations, or a binary search to find the
problem.

It is also tied to the image you are using. When you use mantis to
publish a fix you are also publishing your fix for ALL forks of squeak
to adopt. A fix could be tagged as "Applies to Squeak 3.8" "Applies to
Gjallar0.1" so that people can pick it up for their own projects that
are unfortunately not able to track the latest image (i.e. most projects
out there of any size)

If you take mantis out of the equation you are not performing bug fixing
you are doing coding, and if you are doing coding you might as well go
off and spend a month rewriting the whole pacakge properly in order to
give the community something really cool to integerate and use when you
are done.
> We have a great tool already for keeping track of code that has
> changed.
Yes changed in one image.
> It's called Monticello. You use it all the time. Bob
> can load .mcm or .mcz packages into an image just as easily as
> it can load changesets from mantis.
>  
Sure you can, but Monticello is not a bug tracker. In this case we are
applying bug fixes and discussing them.

If you have decided that a package is to be managed externally, e.g.
Network, then you might want to perform a radical refactoring or a
redesign. Then you are coding and for coding a code management tool is a
good move. We are not interested in what you do to Network until you
have finished, in which case being an externally managed package we
simply load the latest Network as the second step in the release that is
due to adopt it. The first step would be to load any fixes needed in
order to load it, but hopefully you thought about backwards
compatability when you published and that shouldn't be a problem.

to be contd.

Keith

On Sun, Jul 05, 2009 at 01:58:24AM +0100, Keith Hodges wrote:
I have made the code available. Any other fork can quickly and
easily see and load just my changes by pressing the MERGE
button, if Monticello were being used properly, which it isn't.

> Who knows what your starting point was?

It's called ancestors in MC

> Who knows or records what exactly you did?  (ok mc does, but I mean in a
> way that other people can use in their images)

anybody who presses the merge botton

> Who has any opportunity to comment and provide feedback on what you have
> done in the context of what you have done.

anybody on the mantis page related to the code I just added to
monticello

> You are a law unto yourself, and there is no pier review taking place.
> Your actions are not repeatable, nor are they undoable, because by the
> time you want to undo them, someone else will have merged in your change
> to their latest.

Peer review happens after the code exists, not before. I would
then announce my change on mantis and ask for comments, say if
the bug is fixed, and note any problems I anticipate.

> Your result cannot be used by anyone else until the release is
> published. But we want projects to be able to happen in parralel, these
> projects need to be able to pick up the fixes that they need. When a
> project is published as a task, it specifies the fixes it needs as
> prerequisites.

Which fixes. Why should I care? I want all the fixes. I want
everything to work right. Thats why I upgraded from Squeak 1.1.

> Your steps are describing the sort of process that you are used to when
> you are hacking away at your own code. When we build an image we are not
> hacking, we are carefully constructing pre-specified result out of
> (hopefully) carefully crafted and documented pieces.

Of course. SqueakSource is not the image. the image is not
squeaksource. The image is built from the best working subset of
the code in squeaksource, by loading a known-working version
that has passed some review standards and been nicely released
and packaged and published somewhere other than squeaksource,
such as squeakmap, universes, or Packages, or .mcm.

> Making trunk open to all committers is saying come on in lets have a
> hack fest, this is nothing to do with building a release image.

Of course not.

yes.

> MC is a tool for managing code, but it doesnt have any means for
> providing feedback,

nor should it.

> it is not a tool for managing fixes.

nor should it be.

> Once you commit
> a fix from mantis to MC it is out of mantis' hands discussion is over,
> the code then slips under the radar, it merges into the blur of the
> package.

Wrong. the discussion continues. I announce that the fix is
ready in monticello, ask people to load and test it, and they
review it.

> It also possibly breaks another fix that someone submitted ages
> ago.

Hopefully that comes up in testing, and it will be noted in
mantis.

> Fixes that are on the table for integration into a release should remain
> on the table until the release is tested and out of the door.
> If a test fails, bob could automatically apply the fixes one by one to
> find out what caused the problem, he could reorder the application of
> the fixes, or try different combinations, or a binary search to find the
> problem.

This is bogus. if it worked last time and it doesn't this time,
the problem is obvious. One of the small set of new changes that
got added in the few hours since bob tried last. Bob knows what
those changes are, and so does everybody else, because bob
watches the repository and lists what's new in his report.

> It is also tied to the image you are using. When you use mantis to
> publish a fix you are also publishing your fix for ALL forks of squeak
> to adopt. A fix could be tagged as "Applies to Squeak 3.8" "Applies to
> Gjallar0.1" so that people can pick it up for their own projects that
> are unfortunately not able to track the latest image (i.e. most projects
> out there of any size)

The image is far too course an entity to patch against. I don't
want to fix squeak or pharo. I want to make Rio-1.2.29 better
than Rio 1.2.28.

Any project should be able to understand which versions of
packages it can afford to upgrade and which it can't. Changing
the core from underneath a project is never a great idea.
upgrading it's support libraries thru Universes or Packages
often is.

> If you take mantis out of the equation you are not performing bug fixing
> you are doing coding, and if you are doing coding you might as well go
> off and spend a month rewriting the whole pacakge properly in order to
> give the community something really cool to integerate and use when you
> are done.

I don't want to take Mantis out of the equation. If I did, that
just means I don't care about the community.

> > We have a great tool already for keeping track of code that has
> > changed.
> Yes changed in one image.

One package

> > It's called Monticello. You use it all the time. Bob
> > can load .mcm or .mcz packages into an image just as easily as
> > it can load changesets from mantis.
> >  
> Sure you can, but Monticello is not a bug tracker. In this case we are
> applying bug fixes and discussing them.

of course it's not a bug tracker.

> If you have decided that a package is to be managed externally, e.g.
> Network, then you might want to perform a radical refactoring or a
> redesign. Then you are coding and for coding a code management tool is a
> good move. We are not interested in what you do to Network until you
> have finished, in which case being an externally managed package we
> simply load the latest Network as the second step in the release that is
> due to adopt it. The first step would be to load any fixes needed in
> order to load it, but hopefully you thought about backwards
> compatability when you published and that shouldn't be a problem.

everything is a package, whether it is unloadable or not. There
will be one big one called the kernel. This should be versioned
like any other package. Linux has Linux 2.6, glibc 2.8, gcc 4.4,
and X.org 7.2. Thinking in terms of the image is just plain
wrong.

--
Matthew Fulmer -- http://mtfulmer.wordpress.com/

> I have made the code available. Any other fork can quickly and
> easily see and load just my changes by pressing the MERGE
> button, if Monticello were being used properly, which it isn't.
>
>  
1. You are assuming that the packages have the same boundaries. So for
example in 3.11 the tests would be in a different package to the main
code. In older images the tests might be in the same package as the
implementation.

The untidiness of the image when viewed in the browser is one of the
major irks I have with squeak, please dont condemn me to have to have
the same organisations and same class names forever.

2. But if I understand this rightly, if they push merge then they don't
just get your fix, they get  all your fixes, every difference between
your package and theirs. What if they dont want all of this change, how
do they work out what is relevant to that fix? what if all the changes
present are not appropriate to them?

3. How many significant integration fixes are only restricted to one
package?

4. What if the other project doesn't want to manually cherry pick every
method. Perhaps they just want a fix for semaphores. They add that fix
to their list of fixes for their bob to integrate, how does he know what
part of the MC package is relevant to the fix that they want.

5. This has worked marvelously for 2 years since "Installer mantis" was
invented, why throw it away.
>> Who knows what your starting point was?
>>    
>
> It's called ancestors in MC
>  
I didnt mean your starting point within that one package. If you change
a message, its not the message that is the important thing, its all the
senders of that message. Those dont get recorded in the MC package.
Thats what people need to see if they are going to evaluate your fix for
use in another context.

Anyway ancestors only works if the history is intact, in which case we
are doomed to keep the same package structure for evermore
> anybody on the mantis page related to the code I just added to
> monticello
>  

> Peer review happens after the code exists, not before. I would
> then announce my change on mantis and ask for comments, say if
> the bug is fixed, and note any problems I anticipate.
>  
How are they supposed to test your bug fix in contexts other than the
one you assumed when you wrote the fix.
AFAIK merge is only useful if your branches are pretty close. cool.... lets all run

Installer mantis select: [ :ea | ea ensureFix ].

then we can all go home. That is fine for externally managed packages.
>> Making trunk open to all committers is saying come on in lets have a
>> hack fest, this is nothing to do with building a release image.
>>    
> Of course not.
>  

Are you agreeing or disagreeing here? Absolutely it should. Have a look in squeaksource/MagmaTester and try
and work out what order to load packages in, or even what packages are
old. Chris often publishes a new package version but then if you look
closely he is still actually using an older version. MC gives no clues,
some feedback would enable Chris to tag a package version with, "I am
using this version at the moment". Then there is which version works in
which version of squeak, again no clues are given.
>> it is not a tool for managing fixes.
>>    
>
> nor should it be.
>
>  
Right so why code a fix and publish it in MC, MC is for managing
packages not fixes. It ceases to be a discrete piece of code that can optionally be applied.
You have applied it to the trunk package that is the same as saying it
is now released.

Bob's building of a release is conceptually a planned atomic change to
the image. You are turning it into a slow incrementally arrived at  and
pondered result. 3.10 was done like that and how long did that take. Bob loads externally managed packages from repositories, that is a given.

If he loads kernel packages from the repository then he is providing a
moving target to the fixers who are supplying fixes to mantis. The fixer
may have done the fix a year before, they dont want to have to be
continually reviewing their fix to cope with your changes. If you can
tell people that they are fixing a particular image then everyone knows
exactly where they stand.

The first time bob runs it will probably apply 100 fixes in one go.

We promised that every fix on mantis that is marked as "testing" is
going to be loaded and tested in to an image automatically. Four people
could change four fixes at once.

We are automatically integrating fixes, not manually dragging through
mantis and committing changes to a central repo like edgar had to. Thats an externally managed package, talk to the maintainer.

> Any project should be able to understand which versions of
> packages it can afford to upgrade and which it can't. Changing
> the core from underneath a project is never a great idea.
>  
Ok so fine, every fork keeps its core exactly as it is. None of the
forks get any code in common, and they remain forks forever.

If you can upgrade your core in small manageable pieces then why not try
it? Your tests should tell you if it works or not, otherwise you need
more test coverage. If 3.11 consists of 400 documented pieces, and the
etoys maintainer discovers that they can use 200 of them, then lo and
behold the two images are several hundred methods, and several API's
closer together.
Then what are we talking about Mantis manges changesets, not MC
packages. You cant reliably use "Installer mantis" on an MC unless it is
a fairly standalone bit. The ONLY time I have ever used MC was for
loading fixes to MonticelloConfigurations, and even then I performed the
integration manually, and published a new release of MC1.5/LPF
>>> We have a great tool already for keeping track of code that has
>>> changed.
>>>      
>> Yes changed in one image.
>>    
>
> One package
>  
The code you changed only makes sense in the context of the
users/callers that you were thinking about when you changed it. I.e. the
patch you made only communicates the message of design intent you want
it to communicate in your one image. So better to make that image one
that people are familiar with, i.e a fixed known starting point. Not when you are integrating. When integrating and testing you should be
thinking in terms of all of SqueakMap not just the image.

When we are building a release we are integrating and testing a
deliverable. That deliverable is the image.
A set of packages that work together is pretty irrelevant, until it is
possible to treat them as packages, i.e remove, replace etc. We are not
yet at that stage.

Keith

****Randal accused me of not delivering anything in a year. 3 years ago
can you remember how hard it was to cleanUp an image? Now it is easy,
and it works for all LPF users. "SmalltalkImage current cleanUpAll."
There have been many tangible benefits like this that the community has
been able to take advantage of.

So what if you havent seen a new image release in a while. Just think on
this, Pharo users are still using FileDirectory, you dont have to use
FileDirectory any more, you can load Rio, surely that is worth
something!!!!  (btw. Rio was developed for Bob)

======

I have had enough of trying to fight my corner, it is exhausting and I
have lots of other things to do. I will make an attempt to compare and
contrast the two ideas, and you can simply make your choice.

Option 1
======
The facts remain, Bob is an image builder, whose job is to integrate the
latest working and tested externally maintained packages, fixes that are
supplied to Mantis, and to tidy up a bit****

Having performed his integration the output Bob provides is a repository
with all the changed packages in it. The packages (trunk if you like)
are an expected and complete **output**

The 3.11 plan involved reorganising the image so that more parts of it
could be treated as external packages **in the future**, and sensibly
handed over to maintainers. But that was to happen a bit at a time. i.e.
after some work had been done on improving modularity. Example: When all
collections fixes have been integrated, then perhaps collections could
be spun out to tender.

If someone does want to do coding, for example to rewrite collections
then that is a separate project that has nothing to do with the release
team or bob. This project which would be done offline and incorporated
into a release plan for  integration into a subsequent releaase e.g.
3.12.  The patches needed to integrate the newly coded Collections
package into the image, would be uploaded to mantis, with the tag along
these lines "this fix to be be applied to "3.11 + NewCollections".

The release 3.12 would be 3.11 + NewCollectionsPatchesFor3.11 +
NewCollections + deprecate OldCollections compatibility methods +
(optionally loadable OldCollections?!)
The release 3.13 would be 3.12 - deprecated methods.

Option 2
======
Andreas' proposal is to throw every package within the kernel and the
image as a whole out to be treated as an "external package" to be hacked
on by any approved person who fancies a go.

The result can be viewed and used by anyone by publishing to MC and
subscribing via an updates mechanism.

The image will probably be a bit unstable for a bit, but we cant go to
the extent of breaking it ever, because we are not distibuting a new
image, just updates, everything must continue to work seamlessly. So we
cant make radical changes like to replace morphic, or to change the
object layout, or to replace the changes/source file mechanism etc.
Everything has to be done in small evolutionary steps as required by an
updates mechanism. There is no mechanism for removing things from the
image, or for changing the image structure. You will have to clear out
stale package references, repository references from your image
yourself. (In other words go and use pharo because this is exactly what
they are doing)

Then it is supposedly up to Bob to take these packages as an **input**
putting the image into an unpredictable state.
Then Bob is expected to apply some fixes from mantis which were perhaps
submited a year ago and were definitely not targeted at the image in
which bob has to get them to work. Bob then uploads the resulting
packages up to the trunk repo,
Bob is now an automated contributor to the single trunk repository.

Bob also helps out by running tests in the hope that at some point in 2
years time, we get to a point where all the enthusiastic contributors to
trunk have calmed down and have arrived at a reasonably stable working
image. At which point a release will be asssembled manually by someone,
probably Andreas, who will manually update the versions number, and
manually ftp the new image up to ftp.squeak.org or publish an mcm for
people to use.

Is that about right?

Keith

>>>>> "Keith" == Keith Hodges <[hidden email]> writes:

Keith> ****Randal accused me of not delivering anything in a year.

Please stop lying about me.

No *release* in a year is not the same as "not delivering anything
in a year".

--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<[hidden email]> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion

Keith Hodges wrote:
> Andreas' proposal is to throw every package within the kernel and the
> image as a whole out to be treated as an "external package" to be hacked
> on by any approved person who fancies a go.

Really, the fundamental idea of the proposal to have a shared repository
where I can collaborate with other Squeak developers on core packages.

> The result can be viewed and used by anyone by publishing to MC and
> subscribing via an updates mechanism.

Correct.

> The image will probably be a bit unstable for a bit, but we cant go to
> the extent of breaking it ever, because we are not distibuting a new
> image, just updates, everything must continue to work seamlessly.

Not necessarily. If there is a need to distribute a new image, we can
certainly do that. I'm just not in favor of requiring a new image any
time you want to do anything. Keep in mind that in previous messages
I've pointed out Bob's role in building images based on the trunk
packages. That's useful because people can fetch the latest version for
playing with it which is right in line with the idea of reducing hurdles
since the need to go through a lengthy update process can be just as
bothersome as the need to download a new image every week.

In short: Updates and images go hand in hand. Yes, there will be images,
and I would like Bob to play a role in building these automatically (say
on a weekly basis).

> Everything has to be done in small evolutionary steps as required by an
> updates mechanism. There is no mechanism for removing things from the
> image, or for changing the image structure. You will have to clear out
> stale package references, repository references from your image
> yourself.

All of this can be done automatically. Removals and doIts can be
deployed (for example) via package preambles/postscripts and since the
MCM update mechanism can be instructed to load precisely those version
with the scripts attached to it you can utilize the process in a similar
manner as preambles/postscripts in change sets.

> Then it is supposedly up to Bob to take these packages as an **input**
> putting the image into an unpredictable state.

Actually, you start with some base image (like the previous release)
which I think we can agree on as being called stable. Then you run the
package updater process just like any client would. Not much difference
to a SakeTask (which could be produced from the MCM automatically) and
it leaves the image in a no more unpredicable state than loading any
other code would.

> Then Bob is expected to apply some fixes from mantis which were perhaps
> submited a year ago and were definitely not targeted at the image in
> which bob has to get them to work. Bob then uploads the resulting
> packages up to the trunk repo,
> Bob is now an automated contributor to the single trunk repository.

That's the point where I would hope that Bob can help us to drastically
reduce the turnaround times. A fix produced a year ago should not stall
for a year in Mantis. If Bob identifies the fix within a week or two, if
Bob can prove that the fix doesn't break an existing test and does allow
the provided tests to run then there should be little need to wait for a
year. The point being that Bob's stamp of approval can help us screen
tests that "look ready" much, much faster.

> Bob also helps out by running tests in the hope that at some point in 2
> years time, we get to a point where all the enthusiastic contributors to
> trunk have calmed down and have arrived at a reasonably stable working
> image. At which point a release will be asssembled manually by someone,
> probably Andreas, who will manually update the versions number, and
> manually ftp the new image up to ftp.squeak.org or publish an mcm for
> people to use.

Something along those lines, yes. I think that having an ongoing release
team would be useful, for example to work out which demos should be
included, how the contents is presented, which extra parts (fonts,
media, etc) to include and so on.

Cheers,
   - Andreas

On Sun, Jul 05, 2009 at 07:46:14AM -0700, Andreas Raab wrote:
> Keith Hodges wrote:
>
> >The image will probably be a bit unstable for a bit, but we cant go to
> >the extent of breaking it ever, because we are not distibuting a new
> >image, just updates, everything must continue to work seamlessly.
>
> Not necessarily. If there is a need to distribute a new image, we can
> certainly do that. I'm just not in favor of requiring a new image any
> time you want to do anything.

I want to mention that from the point of view of someone who maintains
external packages, it is *hugely* helpful to me to have the update
stream in addition to complete images. I've been maintaining some of
these packages for a long time, so I speak from experience and it
matters a lot to me.

Dave

David T. Lewis wrote: Thats what Bob is for.

Bob will keep continuously building and testing your package into all
images that you want to maintain it in. It will also allow others who
have forks you have never even heard of to do the same. You wont have to
manually fight with updates or images.

Keith

> h a lengthy update process can be just as bothersome as the need to
> download a new image every week.
>
> In short: Updates and images go hand in hand. Yes, there will be
> images, and I would like Bob to play a role in building these
> automatically (say on a weekly basis).
Use a cron task... Sake has Sake-Scheduler, but with cron you dont even
have to run an image.

You really dont get it do you...

If I publish a bit of knowledge, lets say I fix the changes/source
issue. Bob's job is to build that improvement into every fork so you
test and contribute your improvement as widely as possible.

If you dont capture the knowledge in a form that other people can use
(and bob pretends to be all the other people) then you are just
satisfying yourself otherwise known as ________.

I do not see any need for another fork of squeak, even if it is called
squeak.

Keith

On Sun, Jul 05, 2009 at 07:34:42PM +0100, Keith Hodges wrote:
Yes, I understand. I'm speaking about my own experience. I am an
individual contributor working with a small laptop computer supporting
a number of external packages. So I am just pointing out that for my
purposes the update stream is extremely helpful.

Dave

Keith Hodges wrote:
> You really dont get it do you...

I think that's pretty obvious by now, isn't it. What I'm trying to do is
to enable developers to work together. I can't possibly imagine why
anyone would be so violently against that, so clearly I'm not getting
the point.

I think our main practical difference is that you "abso-bloody-lutely"
want  people to go through Mantis. Which is okay, you can use Mantis for
anything you'd like. But I myself, and other developers who understand
the MC workflow, would rather collaborate that way. If Bob can't support
workflows that use collaboration through shared MC repositories then
it's not suited for the task of image building. And if it does support
these workflows, then I don't understand why we're even having this
discussion.

Cheers,
   - Andreas

On 5-Jul-09, at 5:59 PM, Andreas Raab wrote:

> Keith Hodges wrote:
>> You really dont get it do you...
>
> I think that's pretty obvious by now, isn't it. What I'm trying to  
> do is to enable developers to work together. I can't possibly  
> imagine why anyone would be so violently against that, so clearly  
> I'm not getting the point.

I think you guys are at cross purposes here. Andreas is trying to  
minimize the overhead of contributing to the squeak.org distribution  
of Squeak. Keith is trying to maximize the value of each contribution  
across all distributions. The technical details don't matter much if  
you don't agree on the goals.

Colin

Colin Putney wrote: Exactly,

I wish I could have put it so succinctly.

every contribution you make to your fork without offering it to other
forks makes it difficult for them to catch up later.
Edgar made SqueakLightII over a year ago (?) and we havent even been
able to harvest that work into a useful form yet.

As far as I am concerned Andreas has just announced another fork of
squeak, and my head is spinning with what we have already

Keith

2009/7/6 Colin Putney <[hidden email]>:
Interesting point Colin. Then, the discussion should be about this. At
a first view
seems really very difficult to take care of ALL the forks (In terms of
compatibility
of the code contributed to squeak.org).

On 7/6/09 9:23 AM, "Keith Hodges" <[hidden email]> wrote:

> Edgar made SqueakLightII over a year ago (?) and we havent even been
> able to harvest that work into a useful form yet.

I should take the same road wiser Squeaker take before.
I also feels "kicked in the ass without any good reasons" where I was the
facto fired from Release Team.
Now I wish going into main stream (if I could agree on something like going
smaller an modular and have ProcustesEnd for code + picts + sound + morph +
etc into packages)
That means a new PackageInfo and a modified Monticello.

When Ralph pick me I said he could be good have a "Monticello summit" and
anyway we should use .cs
We end 3.10 with .cs and many who said was wise decision don't raise his
voice now.

Maybe you could modify Bob for writing in the updates folder ?
First "fake .cs" as we have in 3.10 should be some similar to

7068AdvanceToThreeDotTenAlpha.cs

Then  some similar

7069ConvertTo3dot10.cs

And then you could go as wild you wish and Mighty Board bless, use .mcz, .cs
, DS, MC2 , sar , .pr, .morph , .sqz (I hope no miss some kind of code
extension which could be feeded into Squeak)

All could hit our squeak updates button and see how our number rise, our
..image go where no Squeaker go before...

At this point some Java - Windoze comes and tell you he knows more but his
time is too precious too bother change the color of screen he don't like.

Edgar