Community Supported Packages

Hi,

I tried to follow the discussion on both lists regarding supported packages and how/where to manage them by using Metacello. I can not give complete feedback
since I go with Göran here who said "I need to read your thoughts once more
before giving proper feedback." and I think we all should take the time to
think about all the implications the different approaches have.

With this discussion we set up the base for the future of package maintenance
but also for collaboration amongst the communities, how images are built, ...!

I'm sure we all want to blur the line between in and out of the (whatever) image
and share/reuse as many code/packages as possible within the broader community
(Squeak, Pharo, Cuis, Gemstone, ...). So:

Goal Nr. 1 is: if it is not in my environment then just load it.
Goal Nr. 2 is: if it is loaded it should work in my environment since it brings
               in anything it needs to work

Currently the discussion and actions seem to focus on where to put the
configurations (into the image vs. in repositories per version). I'm in favour
of the latter since that is easier to handle and share between platforms
while having them in the image would limit people who work on various
platforms, it would not scale and eat more resources for maintenance.

My fear is also that if you have the configurations within the image then
we all will end up with something similar to the unmaintained SqueakMap
registrations. With external repos we can also feed CI-Tools (Hudson build
server, ...)

The only benefit from the "maintining configs in-image" would be that
it would be easy to build a UI similar to SqueakMap/Universe Browser.
This is something that has to be solved - especially for Pharo since loading/browsing
it is not very "user friendly" today, but AFAIK work already started with the
"Loader" project. Agree with Andreas that Metacello needs more work and
a good browser of available configs would be a plus.

But thats not the main question. Even if Squeak decides to have the configurations
"in-image" and Pharo decides for external configs one can mix them:
An in-image ConfigurationOfMyApp just loads the ConfigurationOfMyAppExternal
where the last one is maintained in an external repo and the internal
config is for convenience to just load it.

However - I have to reread all the arguments and think about all this.

<warning>
But with a better acceptance of Metacello there will be more to come and
to discuss!
</warning>

Lets think about the implications for anything above the virtual machine:

Squeak is currently maintained as a "full image" using the trunk process
while Pharo already switched to a "small core + external packages" model.

The "pharo core" image (which is not Pharo!) is getting stable and hopefully
smaller and one can already create a Pharo image right from the "ConfigurationOfPharo". That is there and working.

I'm not saying that Squeak is not getting stable (I'm still a Squeak AND Pharo guy)
but it is a different approach to build the end result that we later announce on
our websites.

If Squeak jumps on the Metacello config bandwagon it may also have to think about
jumping onto the "small core + external packages" model. At least to me
this would be the next logical step.

This leads to several questions:

  - could this small core could be the same for all of us - especially since
    we currently fix stuff in Cuis, Squeak and Pharo mostly the same way

  - could this small core be reduced to a minimum set of objects and a loader
    that is able just suck in the things defined in package configurations
    of packages that work together
 
  - would any specific assembled/deployed image not just be a special distro
    of the "small kernel" + selected packages (similar to the linux world)

So "Squeak" may later be the distro with media/fun/experimental packages
or etoys and "Pharo" may later be the distro with packages oriented
towards commercial development.

However then the lines are blurred then and if you have the common core on
your disk you can even create your own distro with Seaside and Etoys ...

Pharo will follow the path "kernel" + selected packages" and external
config path anyway since this is already discusssed and prepared.

>From what I understand so far a repo for Pharo 1.1. will appear soon
with freezed/working configs and a build structure will be set up.
So lets see how this works out and if successfull if Squeak will
adapt it too.

Just my 2 cents. Expect additionally cents after getting more time to
think about all this ...

Bye
T.
       

--
GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT!
Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01

On 5/17/10 9:14 AM, "Torsten Bergmann" <[hidden email]> wrote:

> The "pharo core" image (which is not Pharo!) is getting stable and hopefully
> smaller and one can already create a Pharo image right from the
> "ConfigurationOfPharo". That is there and working.
>
> I'm not saying that Squeak is not getting stable


But is .

Again I beg going to SqueakCore .image as default....



Edgar

Torsten

On 5/17/10, Torsten Bergmann <[hidden email]> wrote:
> Hi,
>
> I tried to follow the discussion on both lists regarding supported packages
> and how/where to manage them by using Metacello. I can not give complete
> feedback
> since I go with Göran here who said "I need to read your thoughts once more
> before giving proper feedback." and I think we all should take the time to
> think about all the implications the different approaches have.

Yes, I think this is good. At the same time it does not exclude to go
one with a trail Configuration Package within the Squeak image and see
how it works.

Today Colin P. put a Configuration of OmniBrowser with refactoring
into the Inbox.
I checked it out and it looks good. I did not thorough testing though.
http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-May/150721.html


>
> With this discussion we set up the base for the future of package
> maintenance
> but also for collaboration amongst the communities, how images are built,
> ...!

Yes

> I'm sure we all want to blur the line between in and out of the (whatever)
> image
> and share/reuse as many code/packages as possible within the broader
> community
> (Squeak, Pharo, Cuis, Gemstone, ...). So:

Great that you mention Cuis and Gemstone in addition.
This will open up a lot of opportunities.

> Goal Nr. 1 is: if it is not in my environment then just load it.
> Goal Nr. 2 is: if it is loaded it should work in my environment since it
> brings in anything it needs to work

Thanks for reminding us of the basics. And that there is a need for
doing so is food for thought.... :-)

> Currently the discussion and actions seem to focus on where to put the
> configurations (into the image vs. in repositories per version). I'm in
> favour
> of the latter since that is easier to handle and share between platforms
> while having them in the image would limit people who work on various
> platforms, it would not scale and eat more resources for maintenance.

I think we need both. A few (6...12) high profile
ConfigurationOfThePackageName (Or as Dale H. suggested and Andreas
proposed  'ThePackageNameInstaller') within the image and
for the rest the Metacello Repository.

For the ones within the image there should be some kind of checks that
they work together when everything is loaded together as well as that
all the tests (with a few exceptions) are green.

> My fear is also that if you have the configurations within the image then
> we all will end up with something similar to the unmaintained SqueakMap
> registrations.

I share this and for this reason I suggest that we limit the number
for the time being.

 With external repos we can also feed CI-Tools (Hudson build
> server, ...)

Yes.

> The only benefit from the "maintining configs in-image" would be that
> it would be easy to build a UI similar to SqueakMap/Universe Browser.

Yes, and that is quite important for a newcomer.

Iin analogy we have here the 'Synaptic' vs. apt-get install discussion.
In the long-run however it pays off to learn 'apt-get'.

> This is something that has to be solved - especially for Pharo since
> loading/browsing
> it is not very "user friendly" today, but AFAIK work already started with
> the
> "Loader" project. Agree with Andreas that Metacello needs more work and
> a good browser of available configs would be a plus.


> But thats not the main question. Even if Squeak decides to have the
> configurations
> "in-image" and Pharo decides for external configs one can mix them:
> An in-image ConfigurationOfMyApp just loads the ConfigurationOfMyAppExternal

An excellent idea! I like this. It says. The core developer have
approved a certain configuration for a certain package.

> where the last one is maintained in an external repo and the internal
> config is for convenience to just load it.

Yes.

> However - I have to reread all the arguments and think about all this.

Sure.



> I'm not saying that Squeak is not getting stable (I'm still a Squeak AND
> Pharo guy)
> but it is a different approach to build the end result that we later
> announce on
> our websites.


Actually Squeak is in the process of getting smaller and more stable
as well.....


> If Squeak jumps on the Metacello config bandwagon it may also have to think
> about
> jumping onto the "small core + external packages" model. At least to me
> this would be the next logical step.
>
> This leads to several questions:
>
>   - could this small core could be the same for all of us - especially since
>     we currently fix stuff in Cuis, Squeak and Pharo mostly the same way


Sure, a good mid-term goal.

>   - could this small core be reduced to a minimum set of objects and a
> loader
>     that is able just suck in the things defined in package configurations
>     of packages that work together


Maybe.

>   - would any specific assembled/deployed image not just be a special distro
>     of the "small kernel" + selected packages (similar to the linux world)


Yes.

> So "Squeak" may later be the distro with media/fun/experimental packages
> or etoys and "Pharo" may later be the distro with packages oriented
> towards commercial development.

Yes, a good vision.

> However then the lines are blurred then and if you have the common core on
> your disk you can even create your own distro with Seaside and Etoys ...


> Pharo will follow the path "kernel" + selected packages" and external
> config path anyway since this is already discusssed and prepared.
>
> >From what I understand so far a repo for Pharo 1.1. will appear soon
> with freezed/working configs and a build structure will be set up.
> So lets see how this works out and if successfull if Squeak will
> adapt it too.

I cannot comment on this because I do not follow Pharo closely for
time constraint reasons.

> Just my 2 cents. Expect additionally cents after getting more time to
> think about all this ...

This was more than just 2 cents. A carefully written mail.

Thank you

--Hannes

Hi Torsten -

Thanks for the thoughtful response. It'll take a little to respond to
the points individually, so apologies for the length of the response.

First of all, nothing is "decided" yet. I've made a proposal, people
appear to like it, so we can start a phase of experimentation. If the
experimentation works out, we can add this for 4.2 and if not, we'll see
what's lacking and how to fix it.

On 5/17/2010 5:14 AM, Torsten Bergmann wrote:
> I tried to follow the discussion on both lists regarding supported packages and how/where to manage them by using Metacello. I can not give complete feedback
> since I go with Göran here who said "I need to read your thoughts once more
> before giving proper feedback." and I think we all should take the time to
> think about all the implications the different approaches have.

Absolutely. One thing that took me by surprise is the response in the
Pharo world. Given that this is a proposal for Squeak I thought that the
response would at most be "see, we told you so". Instead it appears to
be ... well, I'm actually not exactly sure. Seems like a mixture of
feeling threatened and NIH.

If you look at it objectively, my proposal is extremely similar to what
Pharo uses, and I've explicitly mentioned that it was inspired by
Pharo's use of MetacelloRepository. Outside of the community packages
aspect, pretty much the only differences are:
1) Whether to use one configuration per package or not.
2) Whether configurations are preloaded in the image.
That's about it. I really fail to see what the big hiatus is about.

If you do decide to answer the above in a particular way (namely the way
I've proposed it) you also gain a few additional advantages such as
knowing that the code you have is trustworthy and such as having an
easier time building a UI. But that's about it.

 From my perspective that definition of the problem is a bit
superficial. There's a reason why I called this post "community
supported packages" and not just "package management" because we also
need to work out a way to support packages in the long term. I agree
that the above two goals are intrinsic part of what I'm talking about
but there are other aspects to it which are just as important.

We are a small community so we do need to set an artificial boundary on
what we support. If we support "everything" we end up supporting
nothing, which is what we see in Universes and Squeakmap. So an
inseparable part of my proposal is an explicit community contract; there
are certain requirements and benefits that we aim to gain by
coordinating on the package maintenance. I *want* to separate projects
like Omnibrowser and HelpSysstem from random garbage so commonly found
on Squeakmap and Universes. It's because the people around it care and
it's because of that that I want to have an explicit contract that
basically says "you care, we care". No it's not free to become a
community supported package, but it's *because* it's not free that it's
also valuable.

(there are several other aspects to it which I just deleted because
otherwise this post will get positively exhaustive and really all I'm
saying is that for me there's a bit more to it then the individual's
desire to install some code ;-)

> Currently the discussion and actions seem to focus on where to put the
> configurations (into the image vs. in repositories per version). I'm in favour
> of the latter since that is easier to handle and share between platforms
> while having them in the image would limit people who work on various
> platforms, it would not scale and eat more resources for maintenance.

Nothing prevents us from doing just that if we find it preferable. I
agree that updating is a little easier if you do it externally but then
again, I feel this is part of what the community contract implies. Is it
really asked too much to post an update to the inbox if you make a new
version?

On the other hand, having it in-image makes it easier to build a UI and
it also provides implicit structuring simply because we can utilize
system categories to provide a first level of structure. So under
"Configurations-Network" you would find the WebClient config next to
other network related configs. Simple and straightforward.

> My fear is also that if you have the configurations within the image then
> we all will end up with something similar to the unmaintained SqueakMap
> registrations. With external repos we can also feed CI-Tools (Hudson build
> server, ...)

The issues in both Squeakmap and Universes originated from people using
them for many different versions. The same appears to be happening in
Pharo's Metacello repository. There are some configs that work only in
Pharo 1.0 and some that work only in 1.1. That is the root cause for the
problems both Squeakmap and Universes share. And as Squeakmap has
proven, the other problem is that not all software is created equally.
At some point you need to make a decision of what you consider
"supported" (and consequently try to make work for the release) and what
you consider "third party" and none of your business.

There are of course other ways of dealing with that from what I am
proposing. Having it in-image however, makes the status very explicit.
There's no doubt about what is considered supported and when one
achieves (or loses) that status. If it's in, it's supported, if it's
out, it's not. Plus it uses our community processes for development so
if there's an update for a package you'll find out simply by virtue of
seeing the commit message.

> But thats not the main question. Even if Squeak decides to have the configurations
> "in-image" and Pharo decides for external configs one can mix them:
> An in-image ConfigurationOfMyApp just loads the ConfigurationOfMyAppExternal
> where the last one is maintained in an external repo and the internal
> config is for convenience to just load it.

Yes, that would be another fine option.

> Lets think about the implications for anything above the virtual machine:
>
[... snip ...]
>
> If Squeak jumps on the Metacello config bandwagon it may also have to think about
> jumping onto the "small core + external packages" model. At least to me
> this would be the next logical step.

Heh, where have you been over the last year? :-) If there's one thing
where there has been absolute agreement on it's just that. But it's a
matter of when to do it and how to do it - we don't just want to create
bit rot, we want to keep supporting the packages we choose to support.
And that requires some agreement on how the larger picture of
"supporting" packages looks like. And that's where we get into community
supported packages and the reason why it's not "just" a Metacello issue.

> This leads to several questions:
>
>    - could this small core could be the same for all of us - especially since
>      we currently fix stuff in Cuis, Squeak and Pharo mostly the same way

Interesting thought. Is that perhaps the root of feeling threatened? In
any case, it's an interesting question that deserves some thought. I'd
say there are three questions to answer:

* Does such a process requires some (or all?) participants to give up
their original identity?

* What would the development process look like?

* How would a conflict resolution process look like? (and by conflicts I
don't mean package conflicts :-)

If anyone is seriously thinking about this (I'm not) you might want to
approach it by staking out an area that's part of all the images and
that has relatively few changes, like the Graphics package. I'd be
curious what people come with about how to address the resulting questions.

Cheers,
   - Andreas

Hi andreas


>> Absolutely. One thing that took me by surprise is the response in the Pharo world.

I did not see it so can you indicate me where you see a response?

>> Given that this is a proposal for Squeak I thought that the response would at most be "see, we told you so".

who are we to give lesson to squeakers? People can bash metacello in the past and change their mind.
This is since we managed Squeak3.9 with packages (and in fact even before - our ENVY experience back in 99) that we know that
package without a map are not working well.

>> Instead it appears to be ... well, I'm actually not exactly sure. Seems like a mixture of feeling threatened and NIH.

From my point of view I do not really care. If squeak use metacello good for you. We know what we want to try and we will try it.
We worked and discussed with dale since ESUG last year about that and now that 1.1 is getting out this is the right time
to pass in production mode.


> If you look at it objectively, my proposal is extremely similar to what Pharo uses, and I've explicitly mentioned that it was inspired by Pharo's use of MetacelloRepository. Outside of the community packages aspect, pretty much the only differences are:
> 1) Whether to use one configuration per package or not.
> 2) Whether configurations are preloaded in the image.
> That's about it. I really fail to see what the big hiatus is about.