A process proposal for 3.10

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
132 messages Options
1234 ... 7
Reply | Threaded
Open this post in threaded view
|

A process proposal for 3.10

Giovanni Corriga
Hi all,

since we've started discussing 3.10/4.0, I'd like to relay some thoughts
that I've expressed some times ago on #squeak about a possible
development process for Squeak 3.10.

This proposal is based on two points:
- decentralization of development
- fixed duration of iterations ("timeboxing")

"Decentralization of development" means moving most of the
responsibility of development from the release team to the single
package mantainers (stewards). The release team should just be an
integrator of the code blessed by the stewards, and should facilitate
coordination between the stewards of different packages.
"Timeboxing" means having a fixed length release cycle, at the end of
which a new stable version is released. This means that if there are
some problems with a new feature, the deadline doesn't get postponed;
instead it's the feature that gets dropped from the current development
release and rescheduled for the next release.

How would this work? Here's a more detailed description.
-----------------------------------------------------------------------
PHASE 0: RELEASE TEAM FORMATION
The SqF board will appoint a release team for 3.10. This team will have
3-7 members; 5 would be the sweet spot.
The first task for the release team will be to find a mantainer for each
one of the packages currently in the image. If they can't find a
mantainer for a package, than said package _will get dropped from the
image_ without any further discussion.

PHASE 1: RELEASE PLAN
After having found mantainers for every package, there's a package
selection phase, during which the team and the maintainers:
- decide which packages currently in the image should get removed from
the image and turned into SqueakMap-loadable packages.
- decide which new packages can be added to the image or replace other
packages currently in the image.
Input from the community and the stakeholders will be taken into
account.

After deciding which packages will be in the 3.10 image, the release
team, the mantainers and the community will work on creating the release
plan.
Each package mantainer will announce what it plans to do for 3.10. The
release team will integrate these proposals into a coherent release
plan, solving possible conflicts and coordination problems. The
community will be able to make comments and/or requests.

After all these steps, which shouldn't take more than a month, the
development of 3.10 can start.

PHASE 2: DEVELOPMENT
Development will have three stages:
- a 3 months alpha stage
- a 2 months beta stage
- a 1 month gamma/RC stage.
The deadline for each stage will be decided at the beginning of the
iteration.
Every week during each stage the release team will integrate the new
releases of each package blessed by the package mantainers.

Package mantainers will be the main responsible for development of their
package and they'll be free to manage development as they like; ideally
they should create a development team.

If at the end of the alpha stage some of the work items of the plan
aren't near completion, _these items and the corresponding code will get
dropped_. If this is not possible, the code will be kept in the image,
but it will have not to break the other code in the image.

During the beta and gamma stages, no new features will get added to the
image; only bug fixes can go in. During the gamma stage, only
showstopper bugs will be fixed; other bug fixes (eg. cosmetical fixes)
will be scheduled for 3.10.1.

During the gamma stage of 3.10, the release team for 3.11 (or 4.1 or
whatever) will get appointed and will work with the package mantainers
to form the release plan for the next version.

EXAMPLE
Here's a an example of how things could possibly work.

Release plan:
During phase 1, the various package mantainers propose work items. So
let's pretend that...
  - VMMaker team will want to:
    * integrate the async event patches useful for SqueakGtk.
    * integrate the Ffenestri code in all VMs, including the Unix one.
    * work with the Kernel team/maintainer on CompiledMethod cleanup.
  - Nebraska team will work on removing it from the image.
  - Pavel will want to work with various mantainers to integrate his
patches for his KernelImage system.
  - Tools team will want to:
    * better integrate AST and the Refactoring services.
    * make Omnibrowser the default browser.

Now let's pretend that at the end of the alpha stage, the Tools team
still hasn't produced an Omnibrowser based debugger. Since it's the end
of the alpha stage, the OBDebugger will be punted to the next release,
and 3.10 will have a class browser based on OB but still the old
debugger.
-----------------------------------------------------------------------

I believe a process such as the one I'm proposing can work and help
produce good quality releases.

I'd like to hear comments on this, even if they just say that my
proposal sucks.

        Ciao,

                Giovanni


Reply | Threaded
Open this post in threaded view
|

Re: A process proposal for 3.10

Cees De Groot
On 10/16/06, Giovanni Corriga <[hidden email]> wrote:
> This proposal is based on two points:
> - decentralization of development
> - fixed duration of iterations ("timeboxing")
>
Maybe superfluous, but I support this proposal 100%.

Reply | Threaded
Open this post in threaded view
|

Re: Re: A process proposal for 3.10

Lukas Renggli
> This proposal is based on two points:
> - decentralization of development
> - fixed duration of iterations ("timeboxing")

Sounds like a good proposal, however I see a couple of problems that
you don't adress:

- How to manage the code and the changes? Most changes related to
Compiler, CompiledMethod, Behavior, Traits, etc. cannot be loaded
using Monticello.

- What to do with code that is not properly packaged yet? How to pass
code over to another package?

- How to manage fixes that go across multiple packages?

- What to do with essential packages that do not have a maintainer?
Kernel, Collection, ...

- What to do with essential packages that have a maintainer but that
do not get maintained? No insults, therefor no examples ...

Cheers,
Lukas

--
Lukas Renggli
http://www.lukas-renggli.ch

Reply | Threaded
Open this post in threaded view
|

Re: Re: A process proposal for 3.10

Pavel Krivanek
On 10/16/06, Lukas Renggli <[hidden email]> wrote:

> > This proposal is based on two points:
> > - decentralization of development
> > - fixed duration of iterations ("timeboxing")
>
> Sounds like a good proposal, however I see a couple of problems that
> you don't adress:
>
> - How to manage the code and the changes? Most changes related to
> Compiler, CompiledMethod, Behavior, Traits, etc. cannot be loaded
> using Monticello.
>
> - What to do with code that is not properly packaged yet? How to pass
> code over to another package?
>
> - How to manage fixes that go across multiple packages?
>
> - What to do with essential packages that do not have a maintainer?
> Kernel, Collection, ...
>
> - What to do with essential packages that have a maintainer but that
> do not get maintained? No insults, therefor no examples ...
>
> Cheers,
> Lukas
>

Very good notes. This problems are described in the thread with Stef's
postmortem analysis
(http://lists.squeakfoundation.org/pipermail/squeak-dev/2006-October/109807.html)

Without solution of this problems the release team will have to do
much more than only collect fixes from package maintainers and the
releasing will be as painful as in case of version 3.9.

Maybe we have to create new packaging system and the basic
infrastructure like test server and then plan next version.

-- Pavel

Reply | Threaded
Open this post in threaded view
|

Re: Re: A process proposal for 3.10

Giovanni Corriga
In reply to this post by Lukas Renggli
Il giorno lun, 16/10/2006 alle 10.55 +0200, Lukas Renggli ha scritto:

> > This proposal is based on two points:
> > - decentralization of development
> > - fixed duration of iterations ("timeboxing")
>
> Sounds like a good proposal, however I see a couple of problems that
> you don't adress:
>
> - How to manage the code and the changes? Most changes related to
> Compiler, CompiledMethod, Behavior, Traits, etc. cannot be loaded
> using Monticello.

How did these things get managed in 3.9? By changeset and DoIts? Will it
be possible to keep doing this for 3.10, until we develop better tools?
(MC2 maybe?)
On the other hand, I know this was pretty ugly for Marcus and Stef.
Would reducing the scope of the changes to manageable bites solve this,
at least in part?

> - What to do with code that is not properly packaged yet? How to pass
> code over to another package?

Isn't all code in the image segmented into packages? If not we should do
this.
As for moving code from a package to another, we could have a little
temporary duplication of code by having the same methods in two
different packages.

> - How to manage fixes that go across multiple packages?

Package mantainers will have to coordinate for this, each applying the
fixes to his package(s).

> - What to do with essential packages that do not have a maintainer?
> Kernel, Collection, ...

We should find a mantainer for those.

> - What to do with essential packages that have a maintainer but that
> do not get maintained? No insults, therefor no examples ...

This depends on circumstances. In the best case, we should be able to
find a substitute mantainer. In the worst case, we would be forced to
fork (I don't like the idea, but sometimes it's necessary). Witness what
happened with XFree86 in the Linux world: XFree86's core team became
constantly unresponsive to external inputs, until someone decided to
fork the code and create the X.org server, which was adopted by almost
all distros.

        Giovanni


Reply | Threaded
Open this post in threaded view
|

Re: Re: A process proposal for 3.10

Giovanni Corriga
In reply to this post by Pavel Krivanek
Il giorno lun, 16/10/2006 alle 11.19 +0200, Pavel Krivanek ha scritto:

> On 10/16/06, Lukas Renggli <[hidden email]> wrote:
> > > This proposal is based on two points:
> > > - decentralization of development
> > > - fixed duration of iterations ("timeboxing")
>
> Very good notes. This problems are described in the thread with Stef's
> postmortem analysis
> (http://lists.squeakfoundation.org/pipermail/squeak-dev/2006-October/109807.html)
>
> Without solution of this problems the release team will have to do
> much more than only collect fixes from package maintainers and the
> releasing will be as painful as in case of version 3.9.

My opinion is that then we should reduce the scope of the release until
it's manageable. Removing many packages from the image should ease this
problems until we have better tools, too.

        Giovanni


Reply | Threaded
Open this post in threaded view
|

Re: Re: A process proposal for 3.10

Giovanni Giorgi-3
In reply to this post by Giovanni Corriga
I think the proposed workflow is good.


About these  things:
> - How to manage the code and the changes? Most changes related to
> Compiler, CompiledMethod, Behavior, Traits, etc. cannot be loaded
> using Monticello.

I think we must talk about "application package workflow".
When You need to touch the Kernel or the SmallTalk compier, we need an "ad hoc" approach.
If something cannot be loaded I see only two solution:
A) De-couple the package so it can be loaded via Monticello
or
B) Act with an ad hoc approach

> - What to do with code that is not properly packaged yet? How to pass
> code over to another package?
It must be packaged, at least with a primary release
.
SmallTalk tends to create very immersive packages:
You often find yourself modifiyng core classes.
This is not always the best option, but is also a Strong Feature of this language.


> - How to manage fixes that go across multiple packages?

Package mantainers will have to coordinate for this, each applying the
fixes to his package(s).

If you like a more nice solution, I suggest the "deprecated" way of life of Java.
In Java when an API is obsolate is NOT removed at the next relased, but it is marked as "deprecated" in some way.
For instance sending a message to 
"Object deprecated"
can be a fair soltution to track them.
The deprecated method can do nothing and returns, and be used to track these things.

This point is quite important.
I will stress it.
If, for instance, we change the Socket class we cannot throw away the old API, or even Seaside or SmallWiki will have problems and "derived bugs"
We cannot force developer team to instantly adapt to core changes, even if these changes are small and important.
The only exception is for security or stability holes.
In Java you have a release to fix things. For instance a deprecated method in JVM 1.2 is removed in JVM 1.3, but you can leave it.
So code written for JVM 1.1 will run nicely until JVM 1.3.

In PHP is quite the opposite: they have broken compatibility every time.
And for this reason PHP relases are not done so often, because they must give time to developers to migrate,  I suppose.

So I think we must provide also a "deprecation" micro-workflow.





--
"Just Design It" -- GG
Software Architect
http://www.objectsroot.com/

Reply | Threaded
Open this post in threaded view
|

Re: A process proposal for 3.10

keith1y
In reply to this post by Pavel Krivanek
Some notes:

Three variables, 1. time, 2. functionality 3. quality. When you fix two,
the third must be fluid. Choose the first of these to fix. I would
choose quality, aka and to this end testing is important.
 
Assuming the testing story is worked out. Then I would hope for 2
development streams.

1. Incremental: new feature, new tests, all tests pass -> release
frequently. (no time constraints) no release schedule.
2. Core - Discrete Managed change. Again, new feature implement,
integrate, and migrate/merge back into main stream.

am I being too naive?

Keith



               
___________________________________________________________
All New Yahoo! Mail – Tired of Vi@gr@! come-ons? Let our SpamGuard protect you. http://uk.docs.yahoo.com/nowyoucan.html

Reply | Threaded
Open this post in threaded view
|

Re: Re: A process proposal for 3.10

Philippe Marschall
In reply to this post by Giovanni Corriga
2006/10/16, Giovanni Corriga <[hidden email]>:

> Il giorno lun, 16/10/2006 alle 10.55 +0200, Lukas Renggli ha scritto:
> > > This proposal is based on two points:
> > > - decentralization of development
> > > - fixed duration of iterations ("timeboxing")
> >
> > Sounds like a good proposal, however I see a couple of problems that
> > you don't adress:
> >
> > - How to manage the code and the changes? Most changes related to
> > Compiler, CompiledMethod, Behavior, Traits, etc. cannot be loaded
> > using Monticello.
>
> How did these things get managed in 3.9? By changeset and DoIts? Will it
> be possible to keep doing this for 3.10, until we develop better tools?
> (MC2 maybe?)
> On the other hand, I know this was pretty ugly for Marcus and Stef.
> Would reducing the scope of the changes to manageable bites solve this,
> at least in part?
>
> > - What to do with code that is not properly packaged yet? How to pass
> > code over to another package?
>
> Isn't all code in the image segmented into packages?

In theory yes. In praxis no. You can't just go there and unload eToys.

> If not we should do  this.

Sure, but it's not a simple task. Do you volunteer?

> As for moving code from a package to another, we could have a little
> temporary duplication of code by having the same methods in two
> different packages.
>
> > - How to manage fixes that go across multiple packages?
>
> Package mantainers will have to coordinate for this, each applying the
> fixes to his package(s).
>
> > - What to do with essential packages that do not have a maintainer?
> > Kernel, Collection, ...
>
> We should find a mantainer for those.

And if we don't find one. Drop Kernel? Somewho doesn't like a realistic threat.

Philippe

> > - What to do with essential packages that have a maintainer but that
> > do not get maintained? No insults, therefor no examples ...
>
> This depends on circumstances. In the best case, we should be able to
> find a substitute mantainer. In the worst case, we would be forced to
> fork (I don't like the idea, but sometimes it's necessary). Witness what
> happened with XFree86 in the Linux world: XFree86's core team became
> constantly unresponsive to external inputs, until someone decided to
> fork the code and create the X.org server, which was adopted by almost
> all distros.
>
>         Giovanni
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Re: A process proposal for 3.10

Giovanni Corriga
Il giorno lun, 16/10/2006 alle 12.50 +0200, Philippe Marschall ha
scritto:

> 2006/10/16, Giovanni Corriga <[hidden email]>:
> > Il giorno lun, 16/10/2006 alle 10.55 +0200, Lukas Renggli ha scritto:
> > > > This proposal is based on two points:
> > > > - decentralization of development
> > > > - fixed duration of iterations ("timeboxing")
> > >
> > > Sounds like a good proposal, however I see a couple of problems that
> > > you don't adress:
> > >
> > > - How to manage the code and the changes? Most changes related to
> > > Compiler, CompiledMethod, Behavior, Traits, etc. cannot be loaded
> > > using Monticello.
> >
> > How did these things get managed in 3.9? By changeset and DoIts? Will it
> > be possible to keep doing this for 3.10, until we develop better tools?
> > (MC2 maybe?)
> > On the other hand, I know this was pretty ugly for Marcus and Stef.
> > Would reducing the scope of the changes to manageable bites solve this,
> > at least in part?
> >
> > > - What to do with code that is not properly packaged yet? How to pass
> > > code over to another package?
> >
> > Isn't all code in the image segmented into packages?
>
> In theory yes. In praxis no. You can't just go there and unload eToys.

if we want eToys to be maintained, then the first task of the eToys team
will be to make eToys unloadable.
If we don't find anyone for maintaining eToys, we delete them from the
system, without worring if they were packaged or not.

> > If not we should do  this.
>
> Sure, but it's not a simple task. Do you volunteer?

I would if I had time, but I have a graduation to prepare.

> > > - What to do with essential packages that do not have a maintainer?
> > > Kernel, Collection, ...
> >
> > We should find a mantainer for those.
>
> And if we don't find one. Drop Kernel? Somewho doesn't like a realistic threat.

I think it will be unlikely not to find anyone for Kernel and/or
Collection mantainance.
The alternative would be keeping them in the image, but letting them
rot.

        Giovanni


Reply | Threaded
Open this post in threaded view
|

Re: Re: A process proposal for 3.10

Philippe Marschall
2006/10/16, Giovanni Corriga <[hidden email]>:

> Il giorno lun, 16/10/2006 alle 12.50 +0200, Philippe Marschall ha
> scritto:
> > 2006/10/16, Giovanni Corriga <[hidden email]>:
> > > Il giorno lun, 16/10/2006 alle 10.55 +0200, Lukas Renggli ha scritto:
> > > > > This proposal is based on two points:
> > > > > - decentralization of development
> > > > > - fixed duration of iterations ("timeboxing")
> > > >
> > > > Sounds like a good proposal, however I see a couple of problems that
> > > > you don't adress:
> > > >
> > > > - How to manage the code and the changes? Most changes related to
> > > > Compiler, CompiledMethod, Behavior, Traits, etc. cannot be loaded
> > > > using Monticello.
> > >
> > > How did these things get managed in 3.9? By changeset and DoIts? Will it
> > > be possible to keep doing this for 3.10, until we develop better tools?
> > > (MC2 maybe?)
> > > On the other hand, I know this was pretty ugly for Marcus and Stef.
> > > Would reducing the scope of the changes to manageable bites solve this,
> > > at least in part?
> > >
> > > > - What to do with code that is not properly packaged yet? How to pass
> > > > code over to another package?
> > >
> > > Isn't all code in the image segmented into packages?
> >
> > In theory yes. In praxis no. You can't just go there and unload eToys.
>
> if we want eToys to be maintained, then the first task of the eToys team
> will be to make eToys unloadable.

And if they don't?

> If we don't find anyone for maintaining eToys, we delete them from the
> system, without worring if they were packaged or not.

This will break the system.

> > > If not we should do  this.
> >
> > Sure, but it's not a simple task. Do you volunteer?
>
> I would if I had time, but I have a graduation to prepare.
>
> > > > - What to do with essential packages that do not have a maintainer?
> > > > Kernel, Collection, ...
> > >
> > > We should find a mantainer for those.
> >
> > And if we don't find one. Drop Kernel? Somewho doesn't like a realistic threat.
>
> I think it will be unlikely not to find anyone for Kernel and/or
> Collection mantainance.

Whom?

> The alternative would be keeping them in the image, but letting them
> rot.

You see writing mails full of "we should" is always easy. But when it
comes down to doing stuff,....

Philippe

Reply | Threaded
Open this post in threaded view
|

Re: Re: A process proposal for 3.10

Giovanni Giorgi-3
Philippe,
 Giovanni is working hard to give us a workflow.
I think you are posing the problems in the wrong way.
A package without mantainer is in a very sick state.

If we want to continue developing in Squeak, we must find a solution together.
If you have a better idea please expose it, but critizing without a suggestion is useless.



On 10/16/06, Philippe Marschall <[hidden email]> wrote:
2006/10/16, Giovanni Corriga <[hidden email]>:

> Il giorno lun, 16/10/2006 alle 12.50 +0200, Philippe Marschall ha
> scritto:
> > 2006/10/16, Giovanni Corriga <[hidden email]>:
> > > Il giorno lun, 16/10/2006 alle 10.55 +0200, Lukas Renggli ha scritto:
> > > > > This proposal is based on two points:
> > > > > - decentralization of development
> > > > > - fixed duration of iterations ("timeboxing")
> > > >
> > > > Sounds like a good proposal, however I see a couple of problems that
> > > > you don't adress:
> > > >
> > > > - How to manage the code and the changes? Most changes related to
> > > > Compiler, CompiledMethod, Behavior, Traits, etc. cannot be loaded
> > > > using Monticello.
> > >
> > > How did these things get managed in 3.9? By changeset and DoIts? Will it
> > > be possible to keep doing this for 3.10, until we develop better tools?
> > > (MC2 maybe?)
> > > On the other hand, I know this was pretty ugly for Marcus and Stef.
> > > Would reducing the scope of the changes to manageable bites solve this,
> > > at least in part?
> > >
> > > > - What to do with code that is not properly packaged yet? How to pass
> > > > code over to another package?
> > >
> > > Isn't all code in the image segmented into packages?
> >
> > In theory yes. In praxis no. You can't just go there and unload eToys.
>
> if we want eToys to be maintained, then the first task of the eToys team
[...]

--
"Just Design It" -- GG
Software Architect
http://www.objectsroot.com/

Reply | Threaded
Open this post in threaded view
|

Re: Re: A process proposal for 3.10

Giovanni Corriga
In reply to this post by Philippe Marschall
Il giorno lun, 16/10/2006 alle 13.16 +0200, Philippe Marschall ha
scritto:

> 2006/10/16, Giovanni Corriga <[hidden email]>:
> > Il giorno lun, 16/10/2006 alle 12.50 +0200, Philippe Marschall ha
> > scritto:
> > > 2006/10/16, Giovanni Corriga <[hidden email]>:
> > > > Il giorno lun, 16/10/2006 alle 10.55 +0200, Lukas Renggli ha scritto:
> > > > > > This proposal is based on two points:
> > > > > > - decentralization of development
> > > > > > - fixed duration of iterations ("timeboxing")
> > > > >
> > > > > Sounds like a good proposal, however I see a couple of problems that
> > > > > you don't adress:
> > > > >
> > > > > - How to manage the code and the changes? Most changes related to
> > > > > Compiler, CompiledMethod, Behavior, Traits, etc. cannot be loaded
> > > > > using Monticello.
> > > >
> > > > How did these things get managed in 3.9? By changeset and DoIts? Will it
> > > > be possible to keep doing this for 3.10, until we develop better tools?
> > > > (MC2 maybe?)
> > > > On the other hand, I know this was pretty ugly for Marcus and Stef.
> > > > Would reducing the scope of the changes to manageable bites solve this,
> > > > at least in part?
> > > >
> > > > > - What to do with code that is not properly packaged yet? How to pass
> > > > > code over to another package?
> > > >
> > > > Isn't all code in the image segmented into packages?
> > >
> > > In theory yes. In praxis no. You can't just go there and unload eToys.
> >
> > if we want eToys to be maintained, then the first task of the eToys team
> > will be to make eToys unloadable.
>
> And if they don't?
>
> > If we don't find anyone for maintaining eToys, we delete them from the
> > system, without worring if they were packaged or not.
>
> This will break the system.

Will it? Then we'll have to fix the breakage. This can be done, for
example by moving methods to other packages and slimming down eToys
before removing. This may also mean that we would have to plan removal
of eToys on a larger timeframe (a part in 3.10, then some more in 3.11,
then the rest later).

> > > > If not we should do  this.
> > >
> > > Sure, but it's not a simple task. Do you volunteer?
> >
> > I would if I had time, but I have a graduation to prepare.
> >
> > > > > - What to do with essential packages that do not have a maintainer?
> > > > > Kernel, Collection, ...
> > > >
> > > > We should find a mantainer for those.
> > >
> > > And if we don't find one. Drop Kernel? Somewho doesn't like a realistic threat.
>>
> > I think it will be unlikely not to find anyone for Kernel and/or
> > Collection mantainance.
>
> Whom?

I don't know now. But why do you think that some won't volunteer, if we
decide to go this route?

> > The alternative would be keeping them in the image, but letting them
> > rot.
>
> You see writing mails full of "we should" is always easy. But when it
> comes down to doing stuff,....

That's why this is a proposal. Let's discuss it as we're doing, decide
if we want to adopt it, and see were this leads us.

So, aside from these points, what do you think?

        Giovanni


Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: A process proposal for 3.10

Ralph Johnson
In reply to this post by Giovanni Giorgi-3
Many packages in Squeak cannot be deleted easily.  This is because
Squeak is a monolithic system without clear modules.

So, the rule "If they can't find a mantainer for a package, than said
package _will get dropped from the image_ without any further
discussion" must be changed.

I propose "If a package doesn't have a maintainer and it is possible
to remove it from the image without breaking the image then it will be
removed."

By the way, what is a package in Squeak?  One kind of package is
something available from SqueakMap.  These are easy to understand.
They are not the kind of packages that cause problems.  You are
talking about things like the compiler and collections as being
packages.  Is there a list of packages in Squeak?

-Ralph Johnson

Reply | Threaded
Open this post in threaded view
|

Re: Re: A process proposal for 3.10

Philippe Marschall
In reply to this post by Giovanni Giorgi-3
2006/10/16, Giovanni Giorgi <[hidden email]>:
> Philippe,
>  Giovanni is working hard to give us a workflow.
> I think you are posing the problems in the wrong way.

I'm just saying Squeak is extremely limited on development resources.
There is a lot of great stuff that we could do if we had more
development resources but we have to work with what we have got.
Sometimes stuff just doesn't get maintained because the maintainer is
extremely busy.

Sure it would be cool if "someone" could do this.

Also all suggestions have to orient themselves on the reality. Sure it
would be great if we just could drop package X because it is not
maintained. But often this is not simple and requires some effort. How
should do this if there is not even a maintainer for it?

Sure it would be cool if "someone" could do this.

Forking Xf86 worked fine because there were a lot of development
resources. A lot of the original team (except the leader) could be
taken over. There was work from fd.o to integrate. And finally there
is commercial interest in X11. People are getting paid for developing
this kind of stuff.

> A package without mantainer is in a very sick state.

Agreed.

> If we want to continue developing in Squeak, we must find a solution
> together.

Agreed.

Philippe

> On 10/16/06, Philippe Marschall < [hidden email]> wrote:
> >
> > 2006/10/16, Giovanni Corriga < [hidden email]>:
> > > Il giorno lun, 16/10/2006 alle 12.50 +0200, Philippe Marschall ha
> > > scritto:
> > > > 2006/10/16, Giovanni Corriga <[hidden email] >:
> > > > > Il giorno lun, 16/10/2006 alle 10.55 +0200, Lukas Renggli ha
> scritto:
> > > > > > > This proposal is based on two points:
> > > > > > > - decentralization of development
> > > > > > > - fixed duration of iterations ("timeboxing")
> > > > > >
> > > > > > Sounds like a good proposal, however I see a couple of problems
> that
> > > > > > you don't adress:
> > > > > >
> > > > > > - How to manage the code and the changes? Most changes related to
> > > > > > Compiler, CompiledMethod, Behavior, Traits, etc. cannot be loaded
> > > > > > using Monticello.
> > > > >
> > > > > How did these things get managed in 3.9? By changeset and DoIts?
> Will it
> > > > > be possible to keep doing this for 3.10, until we develop better
> tools?
> > > > > (MC2 maybe?)
> > > > > On the other hand, I know this was pretty ugly for Marcus and Stef.
> > > > > Would reducing the scope of the changes to manageable bites solve
> this,
> > > > > at least in part?
> > > > >
> > > > > > - What to do with code that is not properly packaged yet? How to
> pass
> > > > > > code over to another package?
> > > > >
> > > > > Isn't all code in the image segmented into packages?
> > > >
> > > > In theory yes. In praxis no. You can't just go there and unload eToys.
> > >
> > > if we want eToys to be maintained, then the first task of the eToys team
> > [...]
>
> --
> "Just Design It" -- GG
> Software Architect
> http://www.objectsroot.com/
>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: A process proposal for 3.10

Giovanni Corriga
In reply to this post by Ralph Johnson
Il giorno lun, 16/10/2006 alle 07.49 -0500, Ralph Johnson ha scritto:

> Many packages in Squeak cannot be deleted easily.  This is because
> Squeak is a monolithic system without clear modules.
>
> So, the rule "If they can't find a mantainer for a package, than said
> package _will get dropped from the image_ without any further
> discussion" must be changed.
>
> I propose "If a package doesn't have a maintainer and it is possible
> to remove it from the image without breaking the image then it will be
> removed."

Sounds reasonable.

> By the way, what is a package in Squeak?  One kind of package is
> something available from SqueakMap.  These are easy to understand.
> They are not the kind of packages that cause problems.  You are
> talking about things like the compiler and collections as being
> packages.  Is there a list of packages in Squeak?

What we're talking about is Monticello packages. A Monticello is a set
of classes with the same "major" class category, plus a set of
extensions - methods defined by the package in classes that don't belong
to the package. those extensions methods have a method cathegory that
starts with a star (*) and the package name.

Example: if in the latest Squeak release you browse the instance methods
of class Behavior, you'll see a method cathegory named
"*system-support". Those methods are extensions belonging to the System
package.


The problem with things like Etoys is that they're highly coupled with
the rest of the system, and part of the system depend on it. For
example, trying to remove Etoys triggers an emergency evaluator in
Squeak 3.9RC2. As I see it, if we want to remove Etoys (and either let
it rot in absence of a mantainer or turn it into an installable
package), those couplings would become bugs for the other package
mantainers.

        Giovanni


Reply | Threaded
Open this post in threaded view
|

Re: Re: A process proposal for 3.10

Giovanni Corriga
In reply to this post by Philippe Marschall
Il giorno lun, 16/10/2006 alle 15.26 +0200, Philippe Marschall ha
scritto:

> 2006/10/16, Giovanni Giorgi <[hidden email]>:
> > Philippe,
> >  Giovanni is working hard to give us a workflow.
> > I think you are posing the problems in the wrong way.
>
> I'm just saying Squeak is extremely limited on development resources.
> There is a lot of great stuff that we could do if we had more
> development resources but we have to work with what we have got.
> Sometimes stuff just doesn't get maintained because the maintainer is
> extremely busy.

But that's my point too. We have limited resources, so we can't afford
to have stuff in the image that we can't manage, that is, stuff without
a mantainer but that we depend upon.

> Also all suggestions have to orient themselves on the reality. Sure it
> would be great if we just could drop package X because it is not
> maintained. But often this is not simple and requires some effort. How
> should do this if there is not even a maintainer for it?

On the other hand, can we keep depending on stuff that is not mantained?

        Giovanni


Reply | Threaded
Open this post in threaded view
|

Re: A process proposal for 3.10

Yanni Chiu
Giovanni Corriga wrote:

> Il giorno lun, 16/10/2006 alle 15.26 +0200, Philippe Marschall ha
>>I'm just saying Squeak is extremely limited on development resources.
>>There is a lot of great stuff that we could do if we had more
>>development resources but we have to work with what we have got.
>>Sometimes stuff just doesn't get maintained because the maintainer is
>>extremely busy.
>
>
> But that's my point too. We have limited resources, so we can't afford
> to have stuff in the image that we can't manage, that is, stuff without
> a mantainer but that we depend upon.

The process that was described sounds ideal, and should
be the goal. But the reality is that it has a fatal flaw
in that there just aren't people that step up to do the
work (for lots of good reasons). Dropping packages that
don't attract maintainers won't work if the stuff can't be
dropped.

So let's play to our strengths, instead of trying to shore
up weaknesses - the major weakness being not enough people
stepping up to maintain "excess" packages. The strengths
are that people are working on important features, without
any prodding. If we focus on coordinating and helping the
people working on:

- kernal/minimal image (Pavel's work, and Craig's Spoon)
- OmniBrowser
- Monticello2

then we'll have a basic development platform (4.0?) that can
be the target for the "excess" packages. I suspect it would
be a lot easier to build a team to port a package from 3.9,
than it would be to find a team willing to "maintain" that
package for 3.10. Even if we still don't find maintenance teams
for all packages, there's no work to drop the packages (because
they're already gone).

I'd like to see a 4.0 plan start immediately, starting with
Pavel's image, then adding compiler fixes/changes, changes/sources
file changes, new image format, etc. If Spoon, OmniBrowser,
Monticello2 can fit within the 4.0 timeline, then they'll be
included, but they're not showstoppers. Although, MC2 may be
crucial to the 4.1 phase where we'd attempt to re-package
important 3.9 packages. I think the timeline for a 4.0 release
should be 3 months, since Pavel's work is there already,
and we would be focusing only on a few pieces of code.


Reply | Threaded
Open this post in threaded view
|

Re: A process proposal for 3.10

stephane ducasse-2
In reply to this post by Giovanni Corriga
I like the process you describe but we have to consider time and  
energy too.

>> - What to do with code that is not properly packaged yet? How to pass
>> code over to another package?
>
> Isn't all code in the image segmented into packages? If not we  
> should do
> this.

Been in package does not mean that you can manage it.
We got problems with MC sometimes.

> As for moving code from a package to another, we could have a little
> temporary duplication of code by having the same methods in two
> different packages.

This is not a question of duplication this is a question of slices  
actually
been done at the ***same time***




>> - How to manage fixes that go across multiple packages?
>
> Package mantainers will have to coordinate for this, each applying the
> fixes to his package(s).

Look at the fix andreas proposed for TTC he will not include in Graphics
because he is not the owne of multilangual.

I think that the solution is simple:
        - the release team tries to minimize changes to packages it does not  
control
        as much as possible but not less.
        On certain occasion it publishes/changes maintained package and the  
maintainer
        can react and we can roll back.


>> - What to do with essential packages that do not have a maintainer?
>> Kernel, Collection, ...
>
> We should find a mantainer for those.

The granularity of the packages can kill us.

>> - What to do with essential packages that have a maintainer but that
>> do not get maintained? No insults, therefor no examples ...
>
> This depends on circumstances. In the best case, we should be able to
> find a substitute mantainer. In the worst case, we would be forced to
> fork (I don't like the idea, but sometimes it's necessary). Witness  
> what
> happened with XFree86 in the Linux world: XFree86's core team became
> constantly unresponsive to external inputs, until someone decided to
> fork the code and create the X.org server, which was adopted by almost
> all distros.

Indeed

>
> Giovanni
>
>


Reply | Threaded
Open this post in threaded view
|

Re: A process proposal for 3.10

Andreas.Raab
In reply to this post by Giovanni Corriga
Hi Giovanni -

I fully agree with the meta goals of your proposal, namely to enable
people to do their own development and to limit the amount of time spend
on each individual part of the release.

That said, I'm in almost complete disagreement with the rest of your
proposal ;-) The fundamental flaw in it is that you are trying to commit
other people's time. Starting from "the board appoints a release team"
(phase 0) over "the teams announce the development plans" (phase 1) and
"code will get dropped" (phase 2) up until "the VMMaker team will want
to..." (the example) you're happy committing other people's time. In my
experience this has *always* been a fatal error in a volunteer open
source project.

The *only* time you can commit is your own. And given that, how much of
your proposal works if the only time that you can count on is your own?
Even if we want to change the release process we cannot change some
fundamentals about the environment we're in. And one of those
fundamentals is that (on a scale where it matters) you can't tell people
what to do. They're doing it voluntarily, they're doing it for their own
reasons and trying to change that simply won't work.  To give a more
concrete example, the maintenance work I do for FFI, Graphics, Balloon,
and Compression packages is purely a volunteer effort. If you ask me
about my "development plans" for the next six months, all I can say is
"uhm ... ". Most importantly, because I have a fulltime job in a startup
I for example couldn't possibly commit to a six months active
development plan. It's simply not doable and even if it were I'd reject
it on the grounds that unless you pay me for it I'm not going to treat
this serious enough to make six month plans ;-)

Both Stef and Markus understand that basic point. They both understand
that in reality, all they can count on is their own time and they have
reacted by investing more and more of their own time and effort in the
process. And I would like us to change that process from one that leaves
people burned out into one that can be done in a reasonable amount of
time with a sustainable amount of energy.

That change starts by admitting that we're all volunteers. Whether
something will happen depends on so many factors that it's basically
useless to try to do serious upfront planning for it. And again this is
meant on scale - for each individual team it may make perfect sense but
on scale I think you really can't.

To emphasize that point even more I'll give you two recent examples:
First, we have been talking for *years* about changing the source
pointer in the methods. That it happens *now* is (from my point of view)
exclusively due to the fact that we got a new sources file in 3.9 and
lost the entire previous history for the single reason of the source
limitations. Could anyone really have planned for that? I don't think
so. Second, the SSL implementation we got recently. A while ago we
started looking at SSL ourselves and decided to go with OpenSSL - we
talked about Crypto team and their work but honestly, I would have
called anyone a nut who would have told me that this stuff will be ready
by now (I probably would have just laughed them out of the room). At
least for me, this project came out of the blue, completely unforeseen
and again nothing you could have possibly planned for.

My point here is (in case you haven't guessed it by now) that in a
volunteer community "planning" on the scale you are looking at is in
many ways a futile exercise. Things happen for their own reasons at
their own speed and not because you want them. And unless you want to
put in endless amounts of energy by developing all the stuff that "your
release" needs, you better find a way of dealing with that.

Which brings me to the proposal I made earlier. The main goal of that
proposal is really to get away from the upfront feature planning process
and deal with the chaotic reality of open source development by
*embracing* it. So instead of feature list of things that we SHOULD do,
I'd like us to discuss things we HAVE done. Instead of wishful thinking
I'd like us to talk about all the cool stuff that is actually happening!

I think that this may be the only way to make sure we can survive the
release process in a volunteer community like ours and it's surprisingly
similar to what we did at SqC - the main difference is that SqC (instead
of the community) picked which of the contributions got adopted and
which ones didn't. At SqC (or any other commercial entity that I have
been involved in) we have always been aware of error 33 and therefore
never relied on the open source effort to produce specific results in a
specific time frame. That's what companies do, that's why they take
money to do it. However, anything that happened was always welcome and
often a surprise. And I can say for sure that during those times nobody
got burned out by the process.  And I think that process can be recreated.

Cheers,
   - Andreas

Giovanni Corriga wrote:

> Hi all,
>
> since we've started discussing 3.10/4.0, I'd like to relay some thoughts
> that I've expressed some times ago on #squeak about a possible
> development process for Squeak 3.10.
>
> This proposal is based on two points:
> - decentralization of development
> - fixed duration of iterations ("timeboxing")
>
> "Decentralization of development" means moving most of the
> responsibility of development from the release team to the single
> package mantainers (stewards). The release team should just be an
> integrator of the code blessed by the stewards, and should facilitate
> coordination between the stewards of different packages.
> "Timeboxing" means having a fixed length release cycle, at the end of
> which a new stable version is released. This means that if there are
> some problems with a new feature, the deadline doesn't get postponed;
> instead it's the feature that gets dropped from the current development
> release and rescheduled for the next release.
>
> How would this work? Here's a more detailed description.
> -----------------------------------------------------------------------
> PHASE 0: RELEASE TEAM FORMATION
> The SqF board will appoint a release team for 3.10. This team will have
> 3-7 members; 5 would be the sweet spot.
> The first task for the release team will be to find a mantainer for each
> one of the packages currently in the image. If they can't find a
> mantainer for a package, than said package _will get dropped from the
> image_ without any further discussion.
>
> PHASE 1: RELEASE PLAN
> After having found mantainers for every package, there's a package
> selection phase, during which the team and the maintainers:
> - decide which packages currently in the image should get removed from
> the image and turned into SqueakMap-loadable packages.
> - decide which new packages can be added to the image or replace other
> packages currently in the image.
> Input from the community and the stakeholders will be taken into
> account.
>
> After deciding which packages will be in the 3.10 image, the release
> team, the mantainers and the community will work on creating the release
> plan.
> Each package mantainer will announce what it plans to do for 3.10. The
> release team will integrate these proposals into a coherent release
> plan, solving possible conflicts and coordination problems. The
> community will be able to make comments and/or requests.
>
> After all these steps, which shouldn't take more than a month, the
> development of 3.10 can start.
>
> PHASE 2: DEVELOPMENT
> Development will have three stages:
> - a 3 months alpha stage
> - a 2 months beta stage
> - a 1 month gamma/RC stage.
> The deadline for each stage will be decided at the beginning of the
> iteration.
> Every week during each stage the release team will integrate the new
> releases of each package blessed by the package mantainers.
>
> Package mantainers will be the main responsible for development of their
> package and they'll be free to manage development as they like; ideally
> they should create a development team.
>
> If at the end of the alpha stage some of the work items of the plan
> aren't near completion, _these items and the corresponding code will get
> dropped_. If this is not possible, the code will be kept in the image,
> but it will have not to break the other code in the image.
>
> During the beta and gamma stages, no new features will get added to the
> image; only bug fixes can go in. During the gamma stage, only
> showstopper bugs will be fixed; other bug fixes (eg. cosmetical fixes)
> will be scheduled for 3.10.1.
>
> During the gamma stage of 3.10, the release team for 3.11 (or 4.1 or
> whatever) will get appointed and will work with the package mantainers
> to form the release plan for the next version.
>
> EXAMPLE
> Here's a an example of how things could possibly work.
>
> Release plan:
> During phase 1, the various package mantainers propose work items. So
> let's pretend that...
>   - VMMaker team will want to:
>     * integrate the async event patches useful for SqueakGtk.
>     * integrate the Ffenestri code in all VMs, including the Unix one.
>     * work with the Kernel team/maintainer on CompiledMethod cleanup.
>   - Nebraska team will work on removing it from the image.
>   - Pavel will want to work with various mantainers to integrate his
> patches for his KernelImage system.
>   - Tools team will want to:
>     * better integrate AST and the Refactoring services.
>     * make Omnibrowser the default browser.
>
> Now let's pretend that at the end of the alpha stage, the Tools team
> still hasn't produced an Omnibrowser based debugger. Since it's the end
> of the alpha stage, the OBDebugger will be punted to the next release,
> and 3.10 will have a class browser based on OB but still the old
> debugger.
> -----------------------------------------------------------------------
>
> I believe a process such as the one I'm proposing can work and help
> produce good quality releases.
>
> I'd like to hear comments on this, even if they just say that my
> proposal sucks.
>
> Ciao,
>
> Giovanni
>
>
>


1234 ... 7