Community Supported Packages

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Community Supported Packages

Andreas.Raab
Folks -

I've finally gotten some thoughts together for how to address the
'package issues' in Squeak. This is going to take a while so I apologize
beforehand for the length of this message :-)

First, let's talk about the goals. What I'm trying to achieve is
basically to blur the line between what is 'in' the image and what is
'out' of the image. What I'm trying to achieve is to enable people who
write applications and libraries for Squeak to be seen as direct
contributors to Squeak. What I'm trying to achieve is to drastically
grow Squeak by being much more inclusive while at the same time
drastically shrinking it by having many fewer packages 'preloaded'.

How can we do this? Over the last months I've spent some time looking at
options for package management. I don't really like any of the available
options but since I'm not going to write something new, I've
concentrated on the one choice that appears to be actively supported -
Metacello. It has its weaknesses but contrary to all the other options
it's the one choice that is being supported and I appreciate that.

While looking into Metacello I also spent some time at the Pharo
Metacello repository on Squeaksource. There are many interesting things
to be said about it, both good and bad. Good things about it are that
it's community maintained. Everyone can update everyone else's
contributions and although that's not generally done, it offers a
community approach to making sure things work. Something that for
example both Universes and Squeakmap lack completely. What's problematic
about the repository on the other hand is that for one thing one needs
to unconditionally trust the code even before one has even decided to
install it (this by far my biggest complaint about Metacello). Also, I
don't think that a single repository is a feasible option in the long
term - it seems to me that supporting the cross-product of image and
package versions is going to be problematic before too long. And of
course browsing repositories with several dozen of packages to find what
you need isn't exactly fun either.

However, the problems can be turned on their head, and that's actually
pretty insightful. Here's the basic idea: What if, instead of having a
single repository for all image versions with separate packages for each
config, we would instead have a single package simply called
Configurations in the trunk?

That Configurations package would contain all the "supported"
configurations for Squeak (I'll get to that in a minute). The package
would be community-maintained, just like any other package in trunk
development process. This addresses the trust issues with Metacello,
since the package has the same level of trust that all the other
packages in a Squeak release have; it has been developed using the same
principles.

What that gives us is a mechanism by which we suddenly can include
Omnibrowser, Seaside, Magma, FFI and anything else people can come up
with in Squeak without having to preload it in the shipping image. What
it also means is that we can start removing packages from the image,
replacing them with the proper configurations for how to load them back.
What it means is that somebody with a new application (be that
Stephane's Muo app, Hillaire's Dr. Geo, or Josh's OpenCL demo) can ask
for the configuration to be part of the next Squeak release, thereby
contributing *directly* to that Squeak version.

Most importantly, however, is that we as the community can decide what
we feel we want to support and what not. By being explicit about the
'house rules' we can make sure that the model is sustainable moving forward.

To do that, I'm proposing to create a status of "Community Supported
Package" in Squeak. What is that? A community supported package is a
piece of software where we feel this is an important/interesting/novel
package that should ship with this Squeak version. A community supported
package can be loaded in a 'one-click' process directly from within the
image. A community supported package is a package where we provide
assurance that the package has been tested for this release.

However, that status does not come for free. In return for achieving
this status, the software needs follow the rules for community supported
packages, which (at least) should include the following:

* No conflicts. Neither class names nor -override categories. The
rationale is that all community supported packages can be loaded
together without creating conflicts.

* Tests mandatory. A community supported package MUST have tests. The
rationale is that since we provide the guarantee that the package has
been tested, we need to ensure that testing can be automated to the
maximum extent possible. It doesn't mean 100% coverage but *some*
meaningful set of tests need to be provided or else we can't say if we
broke it in the latest update.

There may be some other guidelines that we may want to establish but I'm
trying to keep things simple and only the above two are truly crucial -
what they allow is to ensure that during ongoing trunk development we
will always be able to load all supported packages, run the tests and
see if we broke something horribly.

So the goal of the process is that if we do this right, we should be
able to tell how good (or how bad) a shape we're in regarding the
supported packages when we get to ship date. We should be able to
encourage people who write libraries or apps with, for, and in Squeak,
to contribute directly to the next release by providing a configuration,
their code, and the tests so that we can keep things running. In return,
we'll help with porting, provide backwards compatibility as required etc.

Code that we want to move out of the default image would be moved into a
configuration so people can load it if desired, and it remains a part of
Squeak instead of rotting away. Since the Configuration package is small
and has no dependencies, there's no problem to include that in
core/basic images and allow people to tailor custom images from there.

Concretely, I'm seeing the following steps as necessary:

1. Establish the status and the rules for "Community Supported Packages".

2. Create the "Configurations" package and start populating it with some
examples (Omnibrowser, FFI, Magma ...) to see how that 'feels' like.

3. Encourage people to bring their packages up to 'community standards'
and include them in the Configurations.

 From here, there are a few more things that we should do before we can
ship the next Squeak version:

4. Provide a better installer than these awful doIts for Metacallo.

5. Establish a test server to automatically verify the community
standards for supported packages (i.e., load packages randomly, ensure
no conflicts, run tests, report results).

At this point, there is another clearly defined way to contribute to
Squeak even if you're not a core developer: Develop your software,
provide a configuration, ensure your software adheres to the community
standards. That's it. You've just made the next Squeak release a little
better :-)

Cheers,
   - Andreas

Reply | Threaded
Open this post in threaded view
|

Re: Community Supported Packages

ching
terrific!!

On Sat, May 15, 2010 at 11:43 AM, Andreas Raab <[hidden email]> wrote:
Folks -

I've finally gotten some thoughts together for how to address the 'package issues' in Squeak. This is going to take a while so I apologize beforehand for the length of this message :-)

First, let's talk about the goals. What I'm trying to achieve is basically to blur the line between what is 'in' the image and what is 'out' of the image. What I'm trying to achieve is to enable people who write applications and libraries for Squeak to be seen as direct contributors to Squeak. What I'm trying to achieve is to drastically grow Squeak by being much more inclusive while at the same time drastically shrinking it by having many fewer packages 'preloaded'.

How can we do this? Over the last months I've spent some time looking at options for package management. I don't really like any of the available options but since I'm not going to write something new, I've concentrated on the one choice that appears to be actively supported - Metacello. It has its weaknesses but contrary to all the other options it's the one choice that is being supported and I appreciate that.

While looking into Metacello I also spent some time at the Pharo Metacello repository on Squeaksource. There are many interesting things to be said about it, both good and bad. Good things about it are that it's community maintained. Everyone can update everyone else's contributions and although that's not generally done, it offers a community approach to making sure things work. Something that for example both Universes and Squeakmap lack completely. What's problematic about the repository on the other hand is that for one thing one needs to unconditionally trust the code even before one has even decided to install it (this by far my biggest complaint about Metacello). Also, I don't think that a single repository is a feasible option in the long term - it seems to me that supporting the cross-product of image and package versions is going to be problematic before too long. And of course browsing repositories with several dozen of packages to find what you need isn't exactly fun either.

However, the problems can be turned on their head, and that's actually pretty insightful. Here's the basic idea: What if, instead of having a single repository for all image versions with separate packages for each config, we would instead have a single package simply called Configurations in the trunk?

That Configurations package would contain all the "supported" configurations for Squeak (I'll get to that in a minute). The package would be community-maintained, just like any other package in trunk development process. This addresses the trust issues with Metacello, since the package has the same level of trust that all the other packages in a Squeak release have; it has been developed using the same principles.

What that gives us is a mechanism by which we suddenly can include Omnibrowser, Seaside, Magma, FFI and anything else people can come up with in Squeak without having to preload it in the shipping image. What it also means is that we can start removing packages from the image, replacing them with the proper configurations for how to load them back. What it means is that somebody with a new application (be that Stephane's Muo app, Hillaire's Dr. Geo, or Josh's OpenCL demo) can ask for the configuration to be part of the next Squeak release, thereby contributing *directly* to that Squeak version.

Most importantly, however, is that we as the community can decide what we feel we want to support and what not. By being explicit about the 'house rules' we can make sure that the model is sustainable moving forward.

To do that, I'm proposing to create a status of "Community Supported Package" in Squeak. What is that? A community supported package is a piece of software where we feel this is an important/interesting/novel package that should ship with this Squeak version. A community supported package can be loaded in a 'one-click' process directly from within the image. A community supported package is a package where we provide assurance that the package has been tested for this release.

However, that status does not come for free. In return for achieving this status, the software needs follow the rules for community supported packages, which (at least) should include the following:

* No conflicts. Neither class names nor -override categories. The rationale is that all community supported packages can be loaded together without creating conflicts.

* Tests mandatory. A community supported package MUST have tests. The rationale is that since we provide the guarantee that the package has been tested, we need to ensure that testing can be automated to the maximum extent possible. It doesn't mean 100% coverage but *some* meaningful set of tests need to be provided or else we can't say if we broke it in the latest update.

There may be some other guidelines that we may want to establish but I'm trying to keep things simple and only the above two are truly crucial - what they allow is to ensure that during ongoing trunk development we will always be able to load all supported packages, run the tests and see if we broke something horribly.

So the goal of the process is that if we do this right, we should be able to tell how good (or how bad) a shape we're in regarding the supported packages when we get to ship date. We should be able to encourage people who write libraries or apps with, for, and in Squeak, to contribute directly to the next release by providing a configuration, their code, and the tests so that we can keep things running. In return, we'll help with porting, provide backwards compatibility as required etc.

Code that we want to move out of the default image would be moved into a configuration so people can load it if desired, and it remains a part of Squeak instead of rotting away. Since the Configuration package is small and has no dependencies, there's no problem to include that in core/basic images and allow people to tailor custom images from there.

Concretely, I'm seeing the following steps as necessary:

1. Establish the status and the rules for "Community Supported Packages".

2. Create the "Configurations" package and start populating it with some examples (Omnibrowser, FFI, Magma ...) to see how that 'feels' like.

3. Encourage people to bring their packages up to 'community standards' and include them in the Configurations.

>From here, there are a few more things that we should do before we can ship the next Squeak version:

4. Provide a better installer than these awful doIts for Metacallo.

5. Establish a test server to automatically verify the community standards for supported packages (i.e., load packages randomly, ensure no conflicts, run tests, report results).

At this point, there is another clearly defined way to contribute to Squeak even if you're not a core developer: Develop your software, provide a configuration, ensure your software adheres to the community standards. That's it. You've just made the next Squeak release a little better :-)

Cheers,
 - Andreas




Reply | Threaded
Open this post in threaded view
|

Re: Community Supported Packages

Sean P. DeNigris
Administrator
Having it in image will be great, knowing that the latest package version for the correct version/fork will be loaded - no more hunting around the web, installing different package versions.

Sean
Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: Community Supported Packages

Igor Stasenko
In reply to this post by Andreas.Raab
On 15 May 2010 06:43, Andreas Raab <[hidden email]> wrote:
[snip]

Hmm.. it reminds me the Sake/Packages model.

http://www.squeaksource.com/Packages/Packages-Library-kph.97

resembles closely same idea.
Just take a look at its structure (browse the code w/o installing).
So, except from using different backend (Metacello, which is maintained)
it seems like same thing.

So, leaving our dim past aside, i just wanna say that liked this idea,
from a first time
i heard about it from Keith.
And it would be good to employ it in one reincarnation or another :)

> 4. Provide a better installer than these awful doIts for Metacallo.
>
Installer CSP install: 'Magma'

:)


--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: Community Supported Packages

Hannes Hirzel
In reply to this post by ching
Andreas,

Having a package called 'Configurations' in the trunk makes sense for
me. It will contain the configurations for loadable packages which
work for the particular version of the image.

Why not just give it a test drive by working it out for

* Omnibrowser
* Seaside
* Magma
* FFI
* Regex
* ...
* ...

I think we could do this within two weeks.

If the approach fails not much is lost. However currently I do not see
why it should fail.

Really worth trying out and see how it works.

--Hannes

On 5/15/10, Ching de la Serna <[hidden email]> wrote:

> terrific!!
>
> On Sat, May 15, 2010 at 11:43 AM, Andreas Raab <[hidden email]> wrote:
>
>> Folks -
>>
>> I've finally gotten some thoughts together for how to address the 'package
>> issues' in Squeak. This is going to take a while so I apologize beforehand
>> for the length of this message :-)
>>
>> First, let's talk about the goals. What I'm trying to achieve is basically
>> to blur the line between what is 'in' the image and what is 'out' of the
>> image. What I'm trying to achieve is to enable people who write
>> applications
>> and libraries for Squeak to be seen as direct contributors to Squeak. What
>> I'm trying to achieve is to drastically grow Squeak by being much more
>> inclusive while at the same time drastically shrinking it by having many
>> fewer packages 'preloaded'.
>>
>> How can we do this? Over the last months I've spent some time looking at
>> options for package management. I don't really like any of the available
>> options but since I'm not going to write something new, I've concentrated
>> on
>> the one choice that appears to be actively supported - Metacello. It has
>> its
>> weaknesses but contrary to all the other options it's the one choice that
>> is
>> being supported and I appreciate that.
>>
>> While looking into Metacello I also spent some time at the Pharo Metacello
>> repository on Squeaksource. There are many interesting things to be said
>> about it, both good and bad. Good things about it are that it's community
>> maintained. Everyone can update everyone else's contributions and although
>> that's not generally done, it offers a community approach to making sure
>> things work. Something that for example both Universes and Squeakmap lack
>> completely. What's problematic about the repository on the other hand is
>> that for one thing one needs to unconditionally trust the code even before
>> one has even decided to install it (this by far my biggest complaint about
>> Metacello). Also, I don't think that a single repository is a feasible
>> option in the long term - it seems to me that supporting the cross-product
>> of image and package versions is going to be problematic before too long.
>> And of course browsing repositories with several dozen of packages to find
>> what you need isn't exactly fun either.
>>
>> However, the problems can be turned on their head, and that's actually
>> pretty insightful. Here's the basic idea: What if, instead of having a
>> single repository for all image versions with separate packages for each
>> config, we would instead have a single package simply called
>> Configurations
>> in the trunk?
>>
>> That Configurations package would contain all the "supported"
>> configurations for Squeak (I'll get to that in a minute). The package
>> would
>> be community-maintained, just like any other package in trunk development
>> process. This addresses the trust issues with Metacello, since the package
>> has the same level of trust that all the other packages in a Squeak
>> release
>> have; it has been developed using the same principles.
>>
>> What that gives us is a mechanism by which we suddenly can include
>> Omnibrowser, Seaside, Magma, FFI and anything else people can come up with
>> in Squeak without having to preload it in the shipping image. What it also
>> means is that we can start removing packages from the image, replacing
>> them
>> with the proper configurations for how to load them back. What it means is
>> that somebody with a new application (be that Stephane's Muo app,
>> Hillaire's
>> Dr. Geo, or Josh's OpenCL demo) can ask for the configuration to be part
>> of
>> the next Squeak release, thereby contributing *directly* to that Squeak
>> version.
>>
>> Most importantly, however, is that we as the community can decide what we
>> feel we want to support and what not. By being explicit about the 'house
>> rules' we can make sure that the model is sustainable moving forward.
>>
>> To do that, I'm proposing to create a status of "Community Supported
>> Package" in Squeak. What is that? A community supported package is a piece
>> of software where we feel this is an important/interesting/novel package
>> that should ship with this Squeak version. A community supported package
>> can
>> be loaded in a 'one-click' process directly from within the image. A
>> community supported package is a package where we provide assurance that
>> the
>> package has been tested for this release.
>>
>> However, that status does not come for free. In return for achieving this
>> status, the software needs follow the rules for community supported
>> packages, which (at least) should include the following:
>>
>> * No conflicts. Neither class names nor -override categories. The
>> rationale
>> is that all community supported packages can be loaded together without
>> creating conflicts.
>>
>> * Tests mandatory. A community supported package MUST have tests. The
>> rationale is that since we provide the guarantee that the package has been
>> tested, we need to ensure that testing can be automated to the maximum
>> extent possible. It doesn't mean 100% coverage but *some* meaningful set
>> of
>> tests need to be provided or else we can't say if we broke it in the
>> latest
>> update.
>>
>> There may be some other guidelines that we may want to establish but I'm
>> trying to keep things simple and only the above two are truly crucial -
>> what
>> they allow is to ensure that during ongoing trunk development we will
>> always
>> be able to load all supported packages, run the tests and see if we broke
>> something horribly.
>>
>> So the goal of the process is that if we do this right, we should be able
>> to tell how good (or how bad) a shape we're in regarding the supported
>> packages when we get to ship date. We should be able to encourage people
>> who
>> write libraries or apps with, for, and in Squeak, to contribute directly
>> to
>> the next release by providing a configuration, their code, and the tests
>> so
>> that we can keep things running. In return, we'll help with porting,
>> provide
>> backwards compatibility as required etc.
>>
>> Code that we want to move out of the default image would be moved into a
>> configuration so people can load it if desired, and it remains a part of
>> Squeak instead of rotting away. Since the Configuration package is small
>> and
>> has no dependencies, there's no problem to include that in core/basic
>> images
>> and allow people to tailor custom images from there.
>>
>> Concretely, I'm seeing the following steps as necessary:
>>
>> 1. Establish the status and the rules for "Community Supported Packages".
>>
>> 2. Create the "Configurations" package and start populating it with some
>> examples (Omnibrowser, FFI, Magma ...) to see how that 'feels' like.
>>
>> 3. Encourage people to bring their packages up to 'community standards'
>> and
>> include them in the Configurations.
>>
>> From here, there are a few more things that we should do before we can
>> ship
>> the next Squeak version:
>>
>> 4. Provide a better installer than these awful doIts for Metacallo.
>>
>> 5. Establish a test server to automatically verify the community standards
>> for supported packages (i.e., load packages randomly, ensure no conflicts,
>> run tests, report results).
>>
>> At this point, there is another clearly defined way to contribute to
>> Squeak
>> even if you're not a core developer: Develop your software, provide a
>> configuration, ensure your software adheres to the community standards.
>> That's it. You've just made the next Squeak release a little better :-)
>>
>> Cheers,
>>  - Andreas
>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Community Supported Packages

Casey Ransberger-2
Somewhere in the middle between CPAN and APT. Like it. We should be able to use the same system/metaphor for loading and unloading documentation, which seems like a nice "for free" win in the model.

If we have Monticello at the center of our development model, and Metacello can solve the packaging problem, then the package server can be the same kind of server (e.g., SqueakSource) as the development server.

I suspect that most of us would agree that Monticello, Metacello, and SqueakSource are a long way from ideal. They work in complimentary ways, though, and I wonder if investing in any one of them might exert a positive force on the other two. 


On Fri, May 14, 2010 at 9:40 PM, Hannes Hirzel <[hidden email]> wrote:
Andreas,

Having a package called 'Configurations' in the trunk makes sense for
me. It will contain the configurations for loadable packages which
work for the particular version of the image.

Why not just give it a test drive by working it out for

* Omnibrowser
* Seaside
* Magma
* FFI
* Regex
* ...
* ...

I think we could do this within two weeks.

If the approach fails not much is lost. However currently I do not see
why it should fail.

Really worth trying out and see how it works.

--Hannes

On 5/15/10, Ching de la Serna <[hidden email]> wrote:
> terrific!!
>
> On Sat, May 15, 2010 at 11:43 AM, Andreas Raab <[hidden email]> wrote:
>
>> Folks -
>>
>> I've finally gotten some thoughts together for how to address the 'package
>> issues' in Squeak. This is going to take a while so I apologize beforehand
>> for the length of this message :-)
>>
>> First, let's talk about the goals. What I'm trying to achieve is basically
>> to blur the line between what is 'in' the image and what is 'out' of the
>> image. What I'm trying to achieve is to enable people who write
>> applications
>> and libraries for Squeak to be seen as direct contributors to Squeak. What
>> I'm trying to achieve is to drastically grow Squeak by being much more
>> inclusive while at the same time drastically shrinking it by having many
>> fewer packages 'preloaded'.
>>
>> How can we do this? Over the last months I've spent some time looking at
>> options for package management. I don't really like any of the available
>> options but since I'm not going to write something new, I've concentrated
>> on
>> the one choice that appears to be actively supported - Metacello. It has
>> its
>> weaknesses but contrary to all the other options it's the one choice that
>> is
>> being supported and I appreciate that.
>>
>> While looking into Metacello I also spent some time at the Pharo Metacello
>> repository on Squeaksource. There are many interesting things to be said
>> about it, both good and bad. Good things about it are that it's community
>> maintained. Everyone can update everyone else's contributions and although
>> that's not generally done, it offers a community approach to making sure
>> things work. Something that for example both Universes and Squeakmap lack
>> completely. What's problematic about the repository on the other hand is
>> that for one thing one needs to unconditionally trust the code even before
>> one has even decided to install it (this by far my biggest complaint about
>> Metacello). Also, I don't think that a single repository is a feasible
>> option in the long term - it seems to me that supporting the cross-product
>> of image and package versions is going to be problematic before too long.
>> And of course browsing repositories with several dozen of packages to find
>> what you need isn't exactly fun either.
>>
>> However, the problems can be turned on their head, and that's actually
>> pretty insightful. Here's the basic idea: What if, instead of having a
>> single repository for all image versions with separate packages for each
>> config, we would instead have a single package simply called
>> Configurations
>> in the trunk?
>>
>> That Configurations package would contain all the "supported"
>> configurations for Squeak (I'll get to that in a minute). The package
>> would
>> be community-maintained, just like any other package in trunk development
>> process. This addresses the trust issues with Metacello, since the package
>> has the same level of trust that all the other packages in a Squeak
>> release
>> have; it has been developed using the same principles.
>>
>> What that gives us is a mechanism by which we suddenly can include
>> Omnibrowser, Seaside, Magma, FFI and anything else people can come up with
>> in Squeak without having to preload it in the shipping image. What it also
>> means is that we can start removing packages from the image, replacing
>> them
>> with the proper configurations for how to load them back. What it means is
>> that somebody with a new application (be that Stephane's Muo app,
>> Hillaire's
>> Dr. Geo, or Josh's OpenCL demo) can ask for the configuration to be part
>> of
>> the next Squeak release, thereby contributing *directly* to that Squeak
>> version.
>>
>> Most importantly, however, is that we as the community can decide what we
>> feel we want to support and what not. By being explicit about the 'house
>> rules' we can make sure that the model is sustainable moving forward.
>>
>> To do that, I'm proposing to create a status of "Community Supported
>> Package" in Squeak. What is that? A community supported package is a piece
>> of software where we feel this is an important/interesting/novel package
>> that should ship with this Squeak version. A community supported package
>> can
>> be loaded in a 'one-click' process directly from within the image. A
>> community supported package is a package where we provide assurance that
>> the
>> package has been tested for this release.
>>
>> However, that status does not come for free. In return for achieving this
>> status, the software needs follow the rules for community supported
>> packages, which (at least) should include the following:
>>
>> * No conflicts. Neither class names nor -override categories. The
>> rationale
>> is that all community supported packages can be loaded together without
>> creating conflicts.
>>
>> * Tests mandatory. A community supported package MUST have tests. The
>> rationale is that since we provide the guarantee that the package has been
>> tested, we need to ensure that testing can be automated to the maximum
>> extent possible. It doesn't mean 100% coverage but *some* meaningful set
>> of
>> tests need to be provided or else we can't say if we broke it in the
>> latest
>> update.
>>
>> There may be some other guidelines that we may want to establish but I'm
>> trying to keep things simple and only the above two are truly crucial -
>> what
>> they allow is to ensure that during ongoing trunk development we will
>> always
>> be able to load all supported packages, run the tests and see if we broke
>> something horribly.
>>
>> So the goal of the process is that if we do this right, we should be able
>> to tell how good (or how bad) a shape we're in regarding the supported
>> packages when we get to ship date. We should be able to encourage people
>> who
>> write libraries or apps with, for, and in Squeak, to contribute directly
>> to
>> the next release by providing a configuration, their code, and the tests
>> so
>> that we can keep things running. In return, we'll help with porting,
>> provide
>> backwards compatibility as required etc.
>>
>> Code that we want to move out of the default image would be moved into a
>> configuration so people can load it if desired, and it remains a part of
>> Squeak instead of rotting away. Since the Configuration package is small
>> and
>> has no dependencies, there's no problem to include that in core/basic
>> images
>> and allow people to tailor custom images from there.
>>
>> Concretely, I'm seeing the following steps as necessary:
>>
>> 1. Establish the status and the rules for "Community Supported Packages".
>>
>> 2. Create the "Configurations" package and start populating it with some
>> examples (Omnibrowser, FFI, Magma ...) to see how that 'feels' like.
>>
>> 3. Encourage people to bring their packages up to 'community standards'
>> and
>> include them in the Configurations.
>>
>> From here, there are a few more things that we should do before we can
>> ship
>> the next Squeak version:
>>
>> 4. Provide a better installer than these awful doIts for Metacallo.
>>
>> 5. Establish a test server to automatically verify the community standards
>> for supported packages (i.e., load packages randomly, ensure no conflicts,
>> run tests, report results).
>>
>> At this point, there is another clearly defined way to contribute to
>> Squeak
>> even if you're not a core developer: Develop your software, provide a
>> configuration, ensure your software adheres to the community standards.
>> That's it. You've just made the next Squeak release a little better :-)
>>
>> Cheers,
>>  - Andreas
>>
>>
>




--
Casey Ransberger


Reply | Threaded
Open this post in threaded view
|

Re: Community Supported Packages

Edgar De Cleene
In reply to this post by Andreas.Raab



On 5/15/10 12:43 AM, "Andreas Raab" <[hidden email]> wrote:

> Folks -
>
> I've finally gotten some thoughts together for how to address the
> 'package issues' in Squeak. This is going to take a while so I apologize
> beforehand for the length of this message :-)
>
> First, let's talk about the goals. What I'm trying to achieve is
> basically to blur the line between what is 'in' the image and what is
> 'out' of the image. What I'm trying to achieve is to enable people who
> write applications and libraries for Squeak to be seen as direct
> contributors to Squeak. What I'm trying to achieve is to drastically
> grow Squeak by being much more inclusive while at the same time
> drastically shrinking it by having many fewer packages 'preloaded'.
>
> How can we do this? Over the last months I've spent some time looking at
> options for package management. I don't really like any of the available
> options but since I'm not going to write something new, I've
> concentrated on the one choice that appears to be actively supported -
> Metacello. It has its weaknesses but contrary to all the other options
> it's the one choice that is being supported and I appreciate that.
>
> While looking into Metacello I also spent some time at the Pharo
> Metacello repository on Squeaksource. There are many interesting things
> to be said about it, both good and bad. Good things about it are that
> it's community maintained. Everyone can update everyone else's
> contributions and although that's not generally done, it offers a
> community approach to making sure things work. Something that for
> example both Universes and Squeakmap lack completely. What's problematic
> about the repository on the other hand is that for one thing one needs
> to unconditionally trust the code even before one has even decided to
> install it (this by far my biggest complaint about Metacello). Also, I
> don't think that a single repository is a feasible option in the long
> term - it seems to me that supporting the cross-product of image and
> package versions is going to be problematic before too long. And of
> course browsing repositories with several dozen of packages to find what
> you need isn't exactly fun either.
>
> However, the problems can be turned on their head, and that's actually
> pretty insightful. Here's the basic idea: What if, instead of having a
> single repository for all image versions with separate packages for each
> config, we would instead have a single package simply called
> Configurations in the trunk?
>
> That Configurations package would contain all the "supported"
> configurations for Squeak (I'll get to that in a minute). The package
> would be community-maintained, just like any other package in trunk
> development process. This addresses the trust issues with Metacello,
> since the package has the same level of trust that all the other
> packages in a Squeak release have; it has been developed using the same
> principles.
>
> What that gives us is a mechanism by which we suddenly can include
> Omnibrowser, Seaside, Magma, FFI and anything else people can come up
> with in Squeak without having to preload it in the shipping image. What
> it also means is that we can start removing packages from the image,
> replacing them with the proper configurations for how to load them back.
> What it means is that somebody with a new application (be that
> Stephane's Muo app, Hillaire's Dr. Geo, or Josh's OpenCL demo) can ask
> for the configuration to be part of the next Squeak release, thereby
> contributing *directly* to that Squeak version.
>
> Most importantly, however, is that we as the community can decide what
> we feel we want to support and what not. By being explicit about the
> 'house rules' we can make sure that the model is sustainable moving forward.
>
> To do that, I'm proposing to create a status of "Community Supported
> Package" in Squeak. What is that? A community supported package is a
> piece of software where we feel this is an important/interesting/novel
> package that should ship with this Squeak version. A community supported
> package can be loaded in a 'one-click' process directly from within the
> image. A community supported package is a package where we provide
> assurance that the package has been tested for this release.
>
> However, that status does not come for free. In return for achieving
> this status, the software needs follow the rules for community supported
> packages, which (at least) should include the following:
>
> * No conflicts. Neither class names nor -override categories. The
> rationale is that all community supported packages can be loaded
> together without creating conflicts.
>
> * Tests mandatory. A community supported package MUST have tests. The
> rationale is that since we provide the guarantee that the package has
> been tested, we need to ensure that testing can be automated to the
> maximum extent possible. It doesn't mean 100% coverage but *some*
> meaningful set of tests need to be provided or else we can't say if we
> broke it in the latest update.
>
> There may be some other guidelines that we may want to establish but I'm
> trying to keep things simple and only the above two are truly crucial -
> what they allow is to ensure that during ongoing trunk development we
> will always be able to load all supported packages, run the tests and
> see if we broke something horribly.
>
> So the goal of the process is that if we do this right, we should be
> able to tell how good (or how bad) a shape we're in regarding the
> supported packages when we get to ship date. We should be able to
> encourage people who write libraries or apps with, for, and in Squeak,
> to contribute directly to the next release by providing a configuration,
> their code, and the tests so that we can keep things running. In return,
> we'll help with porting, provide backwards compatibility as required etc.
>
> Code that we want to move out of the default image would be moved into a
> configuration so people can load it if desired, and it remains a part of
> Squeak instead of rotting away. Since the Configuration package is small
> and has no dependencies, there's no problem to include that in
> core/basic images and allow people to tailor custom images from there.
>
> Concretely, I'm seeing the following steps as necessary:
>
> 1. Establish the status and the rules for "Community Supported Packages".
>
> 2. Create the "Configurations" package and start populating it with some
> examples (Omnibrowser, FFI, Magma ...) to see how that 'feels' like.
>
> 3. Encourage people to bring their packages up to 'community standards'
> and include them in the Configurations.
>
>  From here, there are a few more things that we should do before we can
> ship the next Squeak version:
>
> 4. Provide a better installer than these awful doIts for Metacallo.
>
> 5. Establish a test server to automatically verify the community
> standards for supported packages (i.e., load packages randomly, ensure
> no conflicts, run tests, report results).
>
> At this point, there is another clearly defined way to contribute to
> Squeak even if you're not a core developer: Develop your software,
> provide a configuration, ensure your software adheres to the community
> standards. That's it. You've just made the next Squeak release a little
> better :-)
>
> Cheers,
>    - Andreas
>

Great idea.
And I ask again for a .image closer to SqueakCore or with less packages if
was possible.
Count me on for test, build images, etc.

Edgar



Reply | Threaded
Open this post in threaded view
|

Re: Community Supported Packages

Dale
In reply to this post by Andreas.Raab
Andreas,

The package of Configurations sounds like a very good way to clearly identify (and manage) the set of supported configurations.

As I think about, you may want to a use a different naming convention for the configurations so that there is a reduced chance of the supported configuration clashing with the names of configurations loaded from the "wild".

Regarding tools, I am trying to stay out of the tool business for Metacello. Not that tools aren't needed but that maintaining Metacello is a big enough chunk to bite off without having to get into the tools business:)

With that said I am more than willing to expand the API as needed to support the development of tools ...

Dale
----- "Andreas Raab" <[hidden email]> wrote:

| Folks -
|
| I've finally gotten some thoughts together for how to address the
| 'package issues' in Squeak. This is going to take a while so I
| apologize
| beforehand for the length of this message :-)
|
| First, let's talk about the goals. What I'm trying to achieve is
| basically to blur the line between what is 'in' the image and what is
|
| 'out' of the image. What I'm trying to achieve is to enable people who
|
| write applications and libraries for Squeak to be seen as direct
| contributors to Squeak. What I'm trying to achieve is to drastically
| grow Squeak by being much more inclusive while at the same time
| drastically shrinking it by having many fewer packages 'preloaded'.
|
| How can we do this? Over the last months I've spent some time looking
| at
| options for package management. I don't really like any of the
| available
| options but since I'm not going to write something new, I've
| concentrated on the one choice that appears to be actively supported -
|
| Metacello. It has its weaknesses but contrary to all the other options
|
| it's the one choice that is being supported and I appreciate that.
|
| While looking into Metacello I also spent some time at the Pharo
| Metacello repository on Squeaksource. There are many interesting
| things
| to be said about it, both good and bad. Good things about it are that
|
| it's community maintained. Everyone can update everyone else's
| contributions and although that's not generally done, it offers a
| community approach to making sure things work. Something that for
| example both Universes and Squeakmap lack completely. What's
| problematic
| about the repository on the other hand is that for one thing one needs
|
| to unconditionally trust the code even before one has even decided to
|
| install it (this by far my biggest complaint about Metacello). Also, I
|
| don't think that a single repository is a feasible option in the long
|
| term - it seems to me that supporting the cross-product of image and
| package versions is going to be problematic before too long. And of
| course browsing repositories with several dozen of packages to find
| what
| you need isn't exactly fun either.
|
| However, the problems can be turned on their head, and that's actually
|
| pretty insightful. Here's the basic idea: What if, instead of having a
|
| single repository for all image versions with separate packages for
| each
| config, we would instead have a single package simply called
| Configurations in the trunk?
|
| That Configurations package would contain all the "supported"
| configurations for Squeak (I'll get to that in a minute). The package
|
| would be community-maintained, just like any other package in trunk
| development process. This addresses the trust issues with Metacello,
| since the package has the same level of trust that all the other
| packages in a Squeak release have; it has been developed using the
| same
| principles.
|
| What that gives us is a mechanism by which we suddenly can include
| Omnibrowser, Seaside, Magma, FFI and anything else people can come up
|
| with in Squeak without having to preload it in the shipping image.
| What
| it also means is that we can start removing packages from the image,
| replacing them with the proper configurations for how to load them
| back.
| What it means is that somebody with a new application (be that
| Stephane's Muo app, Hillaire's Dr. Geo, or Josh's OpenCL demo) can ask
|
| for the configuration to be part of the next Squeak release, thereby
| contributing *directly* to that Squeak version.
|
| Most importantly, however, is that we as the community can decide what
|
| we feel we want to support and what not. By being explicit about the
| 'house rules' we can make sure that the model is sustainable moving
| forward.
|
| To do that, I'm proposing to create a status of "Community Supported
| Package" in Squeak. What is that? A community supported package is a
| piece of software where we feel this is an important/interesting/novel
|
| package that should ship with this Squeak version. A community
| supported
| package can be loaded in a 'one-click' process directly from within
| the
| image. A community supported package is a package where we provide
| assurance that the package has been tested for this release.
|
| However, that status does not come for free. In return for achieving
| this status, the software needs follow the rules for community
| supported
| packages, which (at least) should include the following:
|
| * No conflicts. Neither class names nor -override categories. The
| rationale is that all community supported packages can be loaded
| together without creating conflicts.
|
| * Tests mandatory. A community supported package MUST have tests. The
|
| rationale is that since we provide the guarantee that the package has
|
| been tested, we need to ensure that testing can be automated to the
| maximum extent possible. It doesn't mean 100% coverage but *some*
| meaningful set of tests need to be provided or else we can't say if we
|
| broke it in the latest update.
|
| There may be some other guidelines that we may want to establish but
| I'm
| trying to keep things simple and only the above two are truly crucial
| -
| what they allow is to ensure that during ongoing trunk development we
|
| will always be able to load all supported packages, run the tests and
|
| see if we broke something horribly.
|
| So the goal of the process is that if we do this right, we should be
| able to tell how good (or how bad) a shape we're in regarding the
| supported packages when we get to ship date. We should be able to
| encourage people who write libraries or apps with, for, and in Squeak,
|
| to contribute directly to the next release by providing a
| configuration,
| their code, and the tests so that we can keep things running. In
| return,
| we'll help with porting, provide backwards compatibility as required
| etc.
|
| Code that we want to move out of the default image would be moved into
| a
| configuration so people can load it if desired, and it remains a part
| of
| Squeak instead of rotting away. Since the Configuration package is
| small
| and has no dependencies, there's no problem to include that in
| core/basic images and allow people to tailor custom images from
| there.
|
| Concretely, I'm seeing the following steps as necessary:
|
| 1. Establish the status and the rules for "Community Supported
| Packages".
|
| 2. Create the "Configurations" package and start populating it with
| some
| examples (Omnibrowser, FFI, Magma ...) to see how that 'feels' like.
|
| 3. Encourage people to bring their packages up to 'community
| standards'
| and include them in the Configurations.
|
|  From here, there are a few more things that we should do before we
| can
| ship the next Squeak version:
|
| 4. Provide a better installer than these awful doIts for Metacallo.
|
| 5. Establish a test server to automatically verify the community
| standards for supported packages (i.e., load packages randomly, ensure
|
| no conflicts, run tests, report results).
|
| At this point, there is another clearly defined way to contribute to
| Squeak even if you're not a core developer: Develop your software,
| provide a configuration, ensure your software adheres to the community
|
| standards. That's it. You've just made the next Squeak release a
| little
| better :-)
|
| Cheers,
|    - Andreas

Reply | Threaded
Open this post in threaded view
|

Re: Community Supported Packages

Sean P. DeNigris
Administrator
What if we had Configurations-Supported and Configurations-Unsupported, so that, even if we were not vouching for them, or maintaining them, the community would have a place to share what they learned about getting other projects to work?

For example, I've been porting interesting projects (SVI, KeyBinder, Applescript,...) to both Pharo and Squeak trunk.  They are nowhere near the standard, and may never get there, but it would be great for users to know how to load them if they so choose, without leaving the image.

Sean
Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: Community Supported Packages

Göran Krampe
In reply to this post by Andreas.Raab
Hi!

A quick reply, I only skimmed your post and I need to be making breakfast :)

- It reminds me of the positive side of "Lunar Linux" a source distro.
In Lunar all "modules" are simply build scripts held in a CVS tree. So
all developers with a commit bit can add/modify/update the "universe" of
modules. The effects are numerous:
        - Very easy to contribute
        - Everyone has the same "picture" at the same time
        - You get an automatic "universe" effect, but even "over time", since
the repo evolves "hand in hand" with the base code base and the packages
also evolve hand in hand with their dependencies.

- Regarding having to trust code; Deltas are loaded first, and then
installed. IIRC Henrik Gedenryds module system implemented atomic
loading by first compiling classes and then in a last big become
operation they were "swapped in place". Before the last step the classes
were "hanging on the side" again, IIRC.

- The various ideas (Metacello, Sake etc) that use/abuse the SCM tools
for maintaining meta information are both "neat" and "disturbing" at the
same time. :)

Anyway, I need to read your thoughts once more before giving proper
feedback.

regards, Göran

Reply | Threaded
Open this post in threaded view
|

Re: Community Supported Packages

Stéphane Rollandin
In reply to this post by Andreas.Raab
sounds very good !

Stef



Reply | Threaded
Open this post in threaded view
|

Re: Community Supported Packages

Andreas.Raab
In reply to this post by Dale
On 5/15/2010 11:56 AM, Dale Henrichs wrote:
> Andreas,
>
> The package of Configurations sounds like a very good way to clearly identify (and manage) the set of supported configurations.
>
> As I think about, you may want to a use a different naming convention for the configurations so that there is a reduced chance of the supported configuration clashing with the names of configurations loaded from the "wild".

That's good advice. Anyone having opinions about that? I'm thinking that
perhaps <package>Installer might be a good choice, i.e.,

        FFIInstaller load. "Loads the latest FFI"
        OmniBrowserInstaller load: 'default'. "Loads the default group"
        Seaside28Installer load: 'All'. "Loads everything"


> Regarding tools, I am trying to stay out of the tool business for Metacello. Not that tools aren't needed but that maintaining Metacello is a big enough chunk to bite off without having to get into the tools business:)
>
> With that said I am more than willing to expand the API as needed to support the development of tools ...

I'm hoping to keep it simple for starters - what I really want is
something roughly resembling the SqueakMap UI, that is a categorized
list (based on system categories for the installers) that can be
searched and have a one-page description (perhaps listing various load
options). Shouldn't be very hard to mock up.

Cheers,
   - Andreas

> ----- "Andreas Raab"<[hidden email]>  wrote:
>
> | Folks -
> |
> | I've finally gotten some thoughts together for how to address the
> | 'package issues' in Squeak. This is going to take a while so I
> | apologize
> | beforehand for the length of this message :-)
> |
> | First, let's talk about the goals. What I'm trying to achieve is
> | basically to blur the line between what is 'in' the image and what is
> |
> | 'out' of the image. What I'm trying to achieve is to enable people who
> |
> | write applications and libraries for Squeak to be seen as direct
> | contributors to Squeak. What I'm trying to achieve is to drastically
> | grow Squeak by being much more inclusive while at the same time
> | drastically shrinking it by having many fewer packages 'preloaded'.
> |
> | How can we do this? Over the last months I've spent some time looking
> | at
> | options for package management. I don't really like any of the
> | available
> | options but since I'm not going to write something new, I've
> | concentrated on the one choice that appears to be actively supported -
> |
> | Metacello. It has its weaknesses but contrary to all the other options
> |
> | it's the one choice that is being supported and I appreciate that.
> |
> | While looking into Metacello I also spent some time at the Pharo
> | Metacello repository on Squeaksource. There are many interesting
> | things
> | to be said about it, both good and bad. Good things about it are that
> |
> | it's community maintained. Everyone can update everyone else's
> | contributions and although that's not generally done, it offers a
> | community approach to making sure things work. Something that for
> | example both Universes and Squeakmap lack completely. What's
> | problematic
> | about the repository on the other hand is that for one thing one needs
> |
> | to unconditionally trust the code even before one has even decided to
> |
> | install it (this by far my biggest complaint about Metacello). Also, I
> |
> | don't think that a single repository is a feasible option in the long
> |
> | term - it seems to me that supporting the cross-product of image and
> | package versions is going to be problematic before too long. And of
> | course browsing repositories with several dozen of packages to find
> | what
> | you need isn't exactly fun either.
> |
> | However, the problems can be turned on their head, and that's actually
> |
> | pretty insightful. Here's the basic idea: What if, instead of having a
> |
> | single repository for all image versions with separate packages for
> | each
> | config, we would instead have a single package simply called
> | Configurations in the trunk?
> |
> | That Configurations package would contain all the "supported"
> | configurations for Squeak (I'll get to that in a minute). The package
> |
> | would be community-maintained, just like any other package in trunk
> | development process. This addresses the trust issues with Metacello,
> | since the package has the same level of trust that all the other
> | packages in a Squeak release have; it has been developed using the
> | same
> | principles.
> |
> | What that gives us is a mechanism by which we suddenly can include
> | Omnibrowser, Seaside, Magma, FFI and anything else people can come up
> |
> | with in Squeak without having to preload it in the shipping image.
> | What
> | it also means is that we can start removing packages from the image,
> | replacing them with the proper configurations for how to load them
> | back.
> | What it means is that somebody with a new application (be that
> | Stephane's Muo app, Hillaire's Dr. Geo, or Josh's OpenCL demo) can ask
> |
> | for the configuration to be part of the next Squeak release, thereby
> | contributing *directly* to that Squeak version.
> |
> | Most importantly, however, is that we as the community can decide what
> |
> | we feel we want to support and what not. By being explicit about the
> | 'house rules' we can make sure that the model is sustainable moving
> | forward.
> |
> | To do that, I'm proposing to create a status of "Community Supported
> | Package" in Squeak. What is that? A community supported package is a
> | piece of software where we feel this is an important/interesting/novel
> |
> | package that should ship with this Squeak version. A community
> | supported
> | package can be loaded in a 'one-click' process directly from within
> | the
> | image. A community supported package is a package where we provide
> | assurance that the package has been tested for this release.
> |
> | However, that status does not come for free. In return for achieving
> | this status, the software needs follow the rules for community
> | supported
> | packages, which (at least) should include the following:
> |
> | * No conflicts. Neither class names nor -override categories. The
> | rationale is that all community supported packages can be loaded
> | together without creating conflicts.
> |
> | * Tests mandatory. A community supported package MUST have tests. The
> |
> | rationale is that since we provide the guarantee that the package has
> |
> | been tested, we need to ensure that testing can be automated to the
> | maximum extent possible. It doesn't mean 100% coverage but *some*
> | meaningful set of tests need to be provided or else we can't say if we
> |
> | broke it in the latest update.
> |
> | There may be some other guidelines that we may want to establish but
> | I'm
> | trying to keep things simple and only the above two are truly crucial
> | -
> | what they allow is to ensure that during ongoing trunk development we
> |
> | will always be able to load all supported packages, run the tests and
> |
> | see if we broke something horribly.
> |
> | So the goal of the process is that if we do this right, we should be
> | able to tell how good (or how bad) a shape we're in regarding the
> | supported packages when we get to ship date. We should be able to
> | encourage people who write libraries or apps with, for, and in Squeak,
> |
> | to contribute directly to the next release by providing a
> | configuration,
> | their code, and the tests so that we can keep things running. In
> | return,
> | we'll help with porting, provide backwards compatibility as required
> | etc.
> |
> | Code that we want to move out of the default image would be moved into
> | a
> | configuration so people can load it if desired, and it remains a part
> | of
> | Squeak instead of rotting away. Since the Configuration package is
> | small
> | and has no dependencies, there's no problem to include that in
> | core/basic images and allow people to tailor custom images from
> | there.
> |
> | Concretely, I'm seeing the following steps as necessary:
> |
> | 1. Establish the status and the rules for "Community Supported
> | Packages".
> |
> | 2. Create the "Configurations" package and start populating it with
> | some
> | examples (Omnibrowser, FFI, Magma ...) to see how that 'feels' like.
> |
> | 3. Encourage people to bring their packages up to 'community
> | standards'
> | and include them in the Configurations.
> |
> |  From here, there are a few more things that we should do before we
> | can
> | ship the next Squeak version:
> |
> | 4. Provide a better installer than these awful doIts for Metacallo.
> |
> | 5. Establish a test server to automatically verify the community
> | standards for supported packages (i.e., load packages randomly, ensure
> |
> | no conflicts, run tests, report results).
> |
> | At this point, there is another clearly defined way to contribute to
> | Squeak even if you're not a core developer: Develop your software,
> | provide a configuration, ensure your software adheres to the community
> |
> | standards. That's it. You've just made the next Squeak release a
> | little
> | better :-)
> |
> | Cheers,
> |    - Andreas
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Community Supported Packages

Andreas.Raab
In reply to this post by Sean P. DeNigris
On 5/15/2010 12:21 PM, Sean P. DeNigris wrote:
> What if we had Configurations-Supported and Configurations-Unsupported, so
> that, even if we were not vouching for them, or maintaining them, the
> community would have a place to share what they learned about getting other
> projects to work?
>
> For example, I've been porting interesting projects (SVI, KeyBinder,
> Applescript,...) to both Pharo and Squeak trunk.  They are nowhere near the
> standard, and may never get there, but it would be great for users to know
> how to load them if they so choose, without leaving the image.

We can discuss this, but I'd like to point out that *because* it's
unsupported, there needs to be a fence that you gotta hop over before
you can install these unsupported packages. The issue being that the one
of the biggest complaints about the existing solutions (Universes and
SqueakMap) are that between all of the unsupported (non-working) stuff
it's too hard to find the pieces that do work.

Really, I'd rather encourage you to have your SVI, KeyBinder,
AppleScript as a *supported* packages because it means you're providing
a conflict-free, tested set of packages that we can automatically test.
I don't see "supported" packages only to be those that virtually
everyone uses - I think "supported" packages should be inclusive.

Cheers,
   - Andreas

Reply | Threaded
Open this post in threaded view
|

Re: Community Supported Packages

Sean P. DeNigris
Administrator
Teleplacer wrote
there needs to be a fence that you gotta hop over before
you can install these unsupported packages. The issue being that the one
of the biggest complaints about the existing solutions (Universes and
SqueakMap) are that between all of the unsupported (non-working) stuff
it's too hard to find the pieces that do work.
Definitely, and could the status of these be automatically marked by the testing server?  Maybe packages that lapse re the "supported" standard could be bumped over the fence until they clean up their act, instead of disappearing.

Teleplacer wrote
Really, I'd rather encourage you to have your SVI, KeyBinder,
AppleScript as a *supported* packages because it means you're providing
a conflict-free, tested set of packages that we can automatically test.
I definitely agreed ideally.  And, these are not my projects, I just shared the porting I did during research for the community's benefit - adding tests, etc. to get them up to standard is not likely; so an "unsupported but easily available" status seems like a good balance.

Sean
Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: Community Supported Packages

Sean P. DeNigris
Administrator
I eagerly read all the discussions and offer my humble contribution (posting to Pharo too, because I think the benefit is cross-platform):

I'm overjoyed that we are digging in and making packages easily loadable (or clear that they are not).  I don't know what I'll do with all the hours I currently spend fishing for working versions ;-)

Overall the goals seem to be (put best in a post from Mariano Peck):
* have a specific catalog of working and tested projects for each platform (Pharo 1.0, Pharo 1.1, Squeak 4.0, Squeak 4.1...)
* Be able to have a hudson server running and testing such configurations.
* Unified way to load a project.
* (I added this one) have the least amount of effort for the developer

The Pharo and Squeak versions both sound intriguing.  I suggest one addition that would magnify all of the above, and eliminate the accidental complexity that versioning is adding to the mix:

***Extend Metacello specs to account for platform version (e.g. Pharo 1.0 vs. 1.1)***

In effect, since we are distilling the platforms down to the pure good stuff - each version is a whole different platform; from a developer's perspective, it's the same process porting to different forks and porting to new versions of the same fork - either way, incompatible interfaces.

A hypothetical developer perspective:
        - Porting from Squeak to Pharo: I'm a developer with a project that works on Squeak 4.0.  Now, I decide to port it to Pharo 1.0.  So I crank up a new image and... wait, how do I load it?  I don't have a ConfigOf.  So I either alter the Config #squeak to #squeakCommon, or load manually.  Now, it doesn't load because there are classes that don't exist in Pharo, so one by one, I move the platform-specific code into Platform.squeak and Platform.pharo.
        - Pharo 1.0 to Pharo 1.1: Now, Pharo 1.1 is released, with even fewer classes - tight core, yay!  So I go through the same process, but now I have *three platforms* - Pharo 1.0, 1.1, and Squeak 4.0.

User perspective: simple - I want to have *one* place to go where I know that all the packages there will work in my image, and cut out all the junk.  And, if I'm feeling adventurous, I'd like *one* place to see what else is out there, but very clear that it's not up to community standards - for inspiration, to port to my platform, or to steal code :)

Once you have Metacello handling the versions
* the developers go from fork*version configs to one config
* the version problem is handled
* all other features of the current plans are preserved

This is what I envision at the intersection of what's been proposed and Metacello versioning:
* Configs are either:
        - in one central location; it won't matter because we went from projects*forks*versions Configs to just 1*projects.
        - or, kept with the projects they reference e.g. pick squeaksource.com, and every project repository keeps its own ConfigurationOf (might diminish the community involvement if read-only repos)
        >Andreas (http://forum.world.st/Community-Supported-Packages-td2217473.html#a2217473):
        >Good things about it are that
        >it's community maintained. Everyone can update everyone else's
        >contributions and although that's not generally done, it offers a
        >community approach to making sure things work.
* Each platform (fork + version combo) (via hudson server?  I don't know much about them) tries to load each Config from the central repository.  If it meets the community standards (no conflicts, tests, etc.), either the Config is copied to that platform/fork's list of compatible Configs, or a reference is added.
* If it doesn't load, another list could be kept with the result of the attempt to load, so anyone who wants to use or port it has a heads up on the status.

So:
* we don't need a separate Config for fork*version platforms.   Right now, it seems the implementation and behavior are collapsed - we don't care where the actual Configs are (at least in the Pharo plan), as long as we know which ones work.  Which resolves Andreas's concern with a single repository:
        > Andreas (http://forum.world.st/Community-Supported-Packages-td2217473.html#a2217473):
        > Also, I don't think that a single repository is a feasible option in the long
        > term - it seems to me that supporting the cross-product of image and
        > package versions is going to be problematic before too long. And of
        > course browsing repositories with several dozen of packages to find what
        > you need isn't exactly fun either.
and eliminated manual fiddling with the Config file:
        > Pharo proposal @ http://groups.google.com/group/metacello/browse_thread/thread/42c088d58acfb363
        > [developers will copy configs that work in Pharo x.y to] PharoxyMetacelloRepository, then add class>>load for Pharo 1.0 version.
* there is still one place to install for a user's platform
* you really do get "a standard way to load the packages on such repository: just send the message load" because the spec already knows whether it will work on that platform
* you still get a Pharo [and Squeak] release with the correct repository as default (already added in Monticello Browser) - this is your list you copied over

Thanks.
Sean
Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: Community Supported Packages

Andreas.Raab
Hi Sean -

Actually, I had already figured that at some point Metacello would need
to provide more explicit version designators :-) But this doesn't
address my concerns about the cross-product of package and system
versions. Quite the opposite: My concern is more that when you have five
potentially supported image versions there is a high probability that
just for simplicity people will only support certain combinations.

To give a concrete example, just yesterday there was a discussion on the
Seaside list about Seaside 2.8 on Pharo 1.1. Apparently, it's no longer
supported, and that's a fine choice for the Seaside developers. But it's
not quite as clear-cut if it's an equally fine choice for the Pharo (or
Squeak) users of Seaside 2.8. And here's the issue: If you assume that
the Seaside configuration comes from the Seaside folks (stored as
ConfigurationOfSeaside in the external repository) then you're
completely buying into that particular set of supported configurations.

If, however, we wanted to support Seaside 2.8 in Squeak 4.2, we could
start with an older configuration, simply provide a few required patches
for it (in a separate repository if required), and have that be part of
Squeak 4.2 and later versions. No support from the Seaside folks is
required, we can decide as community if and how we want to continue to
support it.

That is the system / package version cross-product that I'm concerned
about. And a version designator doesn't do much to address it. The point
is that if there's a sufficient number of users in our community who say
"this is important for us, we want it to work" they should be able to
support that software as part of a community effort regardless of
whether the original project developers 'officially' support it or not.

Cheers,
   - Andreas

On 5/16/2010 5:05 PM, Sean P. DeNigris wrote:

>
> I eagerly read all the discussions and offer my humble contribution (posting
> to Pharo too, because I think the benefit is cross-platform):
>
> I'm overjoyed that we are digging in and making packages easily loadable (or
> clear that they are not).  I don't know what I'll do with all the hours I
> currently spend fishing for working versions ;-)
>
> Overall the goals seem to be (put best in a post from Mariano Peck):
> * have a specific catalog of working and tested projects for each platform
> (Pharo 1.0, Pharo 1.1, Squeak 4.0, Squeak 4.1...)
> * Be able to have a hudson server running and testing such configurations.
> * Unified way to load a project.
> * (I added this one) have the least amount of effort for the developer
>
> The Pharo and Squeak versions both sound intriguing.  I suggest one addition
> that would magnify all of the above, and eliminate the accidental complexity
> that versioning is adding to the mix:
>
> ***Extend Metacello specs to account for platform version (e.g. Pharo 1.0
> vs. 1.1)***
>
> In effect, since we are distilling the platforms down to the pure good stuff
> - each version is a whole different platform; from a developer's
> perspective, it's the same process porting to different forks and porting to
> new versions of the same fork - either way, incompatible interfaces.
>
> A hypothetical developer perspective:
> - Porting from Squeak to Pharo: I'm a developer with a project that works
> on Squeak 4.0.  Now, I decide to port it to Pharo 1.0.  So I crank up a new
> image and... wait, how do I load it?  I don't have a ConfigOf.  So I either
> alter the Config #squeak to #squeakCommon, or load manually.  Now, it
> doesn't load because there are classes that don't exist in Pharo, so one by
> one, I move the platform-specific code into Platform.squeak and
> Platform.pharo.
> - Pharo 1.0 to Pharo 1.1: Now, Pharo 1.1 is released, with even fewer
> classes - tight core, yay!  So I go through the same process, but now I have
> *three platforms* - Pharo 1.0, 1.1, and Squeak 4.0.
>
> User perspective: simple - I want to have *one* place to go where I know
> that all the packages there will work in my image, and cut out all the junk.
> And, if I'm feeling adventurous, I'd like *one* place to see what else is
> out there, but very clear that it's not up to community standards - for
> inspiration, to port to my platform, or to steal code :)
>
> Once you have Metacello handling the versions
> * the developers go from fork*version configs to one config
> * the version problem is handled
> * all other features of the current plans are preserved
>
> This is what I envision at the intersection of what's been proposed and
> Metacello versioning:
> * Configs are either:
> - in one central location; it won't matter because we went from
> projects*forks*versions Configs to just 1*projects.
> - or, kept with the projects they reference e.g. pick squeaksource.com, and
> every project repository keeps its own ConfigurationOf (might diminish the
> community involvement if read-only repos)
> >Andreas
> (http://forum.world.st/Community-Supported-Packages-td2217473.html#a2217473):
> >Good things about it are that
> >it's community maintained. Everyone can update everyone else's
> >contributions and although that's not generally done, it offers a
> >community approach to making sure things work.
> * Each platform (fork + version combo) (via hudson server?  I don't know
> much about them) tries to load each Config from the central repository.  If
> it meets the community standards (no conflicts, tests, etc.), either the
> Config is copied to that platform/fork's list of compatible Configs, or a
> reference is added.
> * If it doesn't load, another list could be kept with the result of the
> attempt to load, so anyone who wants to use or port it has a heads up on the
> status.
>
> So:
> * we don't need a separate Config for fork*version platforms.   Right now,
> it seems the implementation and behavior are collapsed - we don't care where
> the actual Configs are (at least in the Pharo plan), as long as we know
> which ones work.  Which resolves Andreas's concern with a single repository:
> >  Andreas
> (http://forum.world.st/Community-Supported-Packages-td2217473.html#a2217473):
> >  Also, I don't think that a single repository is a feasible option in the
> long
> >  term - it seems to me that supporting the cross-product of image and
> >  package versions is going to be problematic before too long. And of
> >  course browsing repositories with several dozen of packages to find what
> >  you need isn't exactly fun either.
> and eliminated manual fiddling with the Config file:
> >  Pharo proposal @
> http://groups.google.com/group/metacello/browse_thread/thread/42c088d58acfb363
> >  [developers will copy configs that work in Pharo x.y to]
> PharoxyMetacelloRepository, then add class>>load for Pharo 1.0 version.
> * there is still one place to install for a user's platform
> * you really do get "a standard way to load the packages on such repository:
> just send the message load" because the spec already knows whether it will
> work on that platform
> * you still get a Pharo [and Squeak] release with the correct repository as
> default (already added in Monticello Browser) - this is your list you copied
> over
>
> Thanks.
> Sean


Reply | Threaded
Open this post in threaded view
|

Re: Community Supported Packages

Sean P. DeNigris
Administrator
Teleplacer wrote
The point is that if there's a sufficient number of users in our community who say
"this is important for us, we want it to work" they should be able to
support that software as part of a community effort regardless of
whether the original project developers 'officially' support it or not.
I agree completely.  And, this will only apply to some projects.  Maybe have both - default to pointing to the community version unless there is a reason to physically copy and take control of it.  From the image perspective, the user wouldn't have to know if they were loading a Squeak-controlled Config or a central one.

Teleplacer wrote
version designators... doesn't
address my concerns about the cross-product of package and system
versions. Quite the opposite: My concern is more that when you have five
potentially supported image versions there is a high probability that
just for simplicity people will only support certain combinations.
* How versioning helps: For simple cases, just adding versioning to the current central repo, a specific fork/version could just add a for: fork/version do: [] to the shared Config and a Xxx-Platform package.
* A project inbox could be cool: I really liked you inbox idea for non-core projects, but I would like to have a cross-fork project inbox where the community could contribute.  For example, when I ported ExternalWebBrowser, I didn't have write access to SqS, and the port was to have it work on Pharo on Macs, so I had to break it into Core and Platform packages.  But now, I had a Platform.pharo and Platform.squeak, so I really needed a project inbox without the artificial division of forks.  I ended up creating my own SqS repo, and not knowing if I should upload the Config to the central repo because it only worked with my unapproved altered project, which confused people.  A project inbox would allow people to easily find and choose my version or the project owner's...
* Metacello spec could be a limited guarantee - The other thing that I forgot in my other post, is what if there was an option in Metacello to install the closest? or any? version if the current fork/version combo is not available (kind of like Squeak Map's loader)?  So the fork/version Metacello spec would be a guarantee to work, and not a restriction where the Config will only do anything if the current platform is listed.  Now the automated testing/compatibility server iterates through the Configs, and either:
    - files an issue, or somehow announces that this fork/version can be safely added to the Config
    - if the project meets the standard, adds the Config to that list
    - if it doesn't meet the standard, adds the Config to the unsupported list, with a category of the result e.g. "installs but fails tests," "does not load," etc.

Teleplacer wrote
To give a concrete example, just yesterday there was a discussion on the
Seaside list about Seaside 2.8 on Pharo 1.1. Apparently, it's no longer
supported, and that's a fine choice for the Seaside developers. But it's
not quite as clear-cut if it's an equally fine choice for the Pharo (or
Squeak) users of Seaside 2.8. And here's the issue: If you assume that
the Seaside configuration comes from the Seaside folks (stored as
ConfigurationOfSeaside in the external repository) then you're
completely buying into that particular set of supported configurations.
This would be one instance where copying and taking control of the project would be the way to go.  I just think it adds unnecessary complication to have this be the default.


Cheers,
Sean
Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: Community Supported Packages

Andreas.Raab
Hi Sean -

Correct me if I'm wrong but what you're saying sounds as if we're in
violent agreement :-) Yes, having a more specific version info for
Metacello is needed and a Good Thing (tm), no doubt about it. OTOH, the
specific version info doesn't solve all the problems when dealing with
the issue of package and image versions (as you're admitting below). So
it seems we're in complete agreement here.

Cheers,
   - Andreas

On 5/16/2010 6:31 PM, Sean P. DeNigris wrote:

> Teleplacer wrote:
>>
>> The point is that if there's a sufficient number of users in our community
>> who say
>> "this is important for us, we want it to work" they should be able to
>> support that software as part of a community effort regardless of
>> whether the original project developers 'officially' support it or not.
>>
>
> I agree completely.  And, this will only apply to some projects.  Maybe have
> both - default to pointing to the community version unless there is a reason
> to physically copy and take control of it.  From the image perspective, the
> user wouldn't have to know if they were loading a Squeak-controlled Config
> or a central one.
>
>
> Teleplacer wrote:
>>
>> version designators... doesn't
>> address my concerns about the cross-product of package and system
>> versions. Quite the opposite: My concern is more that when you have five
>> potentially supported image versions there is a high probability that
>> just for simplicity people will only support certain combinations.
>>
>
> * How versioning helps: For simple cases, just adding versioning to the
> current central repo, a specific fork/version could just add a for:
> fork/version do: [] to the shared Config and a Xxx-Platform package.
> * A project inbox could be cool: I really liked you inbox idea for non-core
> projects, but I would like to have a cross-fork project inbox where the
> community could contribute.  For example, when I ported ExternalWebBrowser,
> I didn't have write access to SqS, and the port was to have it work on Pharo
> on Macs, so I had to break it into Core and Platform packages.  But now, I
> had a Platform.pharo and Platform.squeak, so I really needed a project inbox
> without the artificial division of forks.  I ended up creating my own SqS
> repo, and not knowing if I should upload the Config to the central repo
> because it only worked with my unapproved altered project, which confused
> people.  A project inbox would allow people to easily find and choose my
> version or the project owner's...
> * Metacello spec could be a limited guarantee - The other thing that I
> forgot in my other post, is what if there was an option in Metacello to
> install the closest? or any? version if the current fork/version combo is
> not available (kind of like Squeak Map's loader)?  So the fork/version
> Metacello spec would be a guarantee to work, and not a restriction where the
> Config will only do anything if the current platform is listed.  Now the
> automated testing/compatibility server iterates through the Configs, and
> either:
>      - files an issue, or somehow announces that this fork/version can be
> safely added to the Config
>      - if the project meets the standard, adds the Config to that list
>      - if it doesn't meet the standard, adds the Config to the unsupported
> list, with a category of the result e.g. "installs but fails tests," "does
> not load," etc.
>
>
> Teleplacer wrote:
>>
>> To give a concrete example, just yesterday there was a discussion on the
>> Seaside list about Seaside 2.8 on Pharo 1.1. Apparently, it's no longer
>> supported, and that's a fine choice for the Seaside developers. But it's
>> not quite as clear-cut if it's an equally fine choice for the Pharo (or
>> Squeak) users of Seaside 2.8. And here's the issue: If you assume that
>> the Seaside configuration comes from the Seaside folks (stored as
>> ConfigurationOfSeaside in the external repository) then you're
>> completely buying into that particular set of supported configurations.
>>
>
> This would be one instance where copying and taking control of the project
> would be the way to go.  I just think it adds unnecessary complication to
> have this be the default.
>
>
> Cheers,
> Sean


Reply | Threaded
Open this post in threaded view
|

Re: Community Supported Packages

Colin Putney
In reply to this post by Andreas.Raab
Ok, this sounds like a good plan to me. I've been wrestling with the problem of how to distribute OmniBrowser. The base image should be getting smaller and simpler, and OB is too big and complex to be part of it. On the other hand, distributing OB as a SAR file isn't working out very well. Perhaps this will be a better mechanism.

To get things rolling, I've created a configuration for OmniBrowser and uploaded it to the inbox repository. I think OB probably meets the criteria for a community package:

        - It uses the prefix 'OB' for all class- and package-names to avoid name conflict
        - It doesn't override any methods in the base image or other packages
        - It has tests - not 100% coverage, but a lot of the important stuff is covered

If we come up with other criteria, I'll make sure that OmniBrowser continues to meet them.

I've structured the Configuration package with a superclass that includes the general Metacello bootstrapping code, and a ConfigurationOfOmniBrowser class to handle the specifics of this release. For the moment, I've included all the packages that OmniBrowser depends on, but ideally some of these should be factored out into separate configurations and made independently loadable - Kom, VBRegex, and perhaps the refactoring engine.

So, take a look, try it out, tell me what you think.

Colin
Reply | Threaded
Open this post in threaded view
|

Re: Community Supported Packages

Sean P. DeNigris
Administrator
In reply to this post by Andreas.Raab
Teleplacer wrote
Correct me if I'm wrong but what you're saying sounds as if we're in
violent agreement :-)
Ha ha, I was on a roll!

Teleplacer wrote
 Yes, having a more specific version info for
Metacello is needed and a Good Thing (tm), no doubt about it. OTOH, the
specific version info doesn't solve all the problems when dealing with
the issue of package and image versions (as you're admitting below). So
it seems we're in complete agreement here.
Yes, my tweak to the great proposals is - for each project:
1. use a fork-specific ConfigurationOf (as suggested by both Pharo and Squeak), but only when projects become, well... forked
2. and, default to one shared ConfigurationOf to make life easier for cross-fork developers, until the project takes on a life of it's own in a particular platform; using Metacello's built-in (if versioning was included) platform management.

Sean
Cheers,
Sean
12