[squeak-dev] Squeak Package Management

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Squeak Package Management

Tapple Gao
Hi all. A discussion started on IRC about how difficult it can
be to install packages in Squeak, and about how confusing the
current situation can look to a newcomer. All of our solutions
seem to have problems:

SqueakMap: Packages are out-of-date, because the security model
does not allow anyone to maintain the package; also, there are
no dependencies

Universes addressed the dependency issue, but still does not
allow widely distributed maintenance of the packages. So far,
it is kept mostly up-to-date, due to a few motivated
individuals.

Sake/Packages is an attempt to address both of the above
concerns. It as yet has no UI. Unlike in SM or Universes, the
entire distribution is versioned as a whole, rather than the
individual packages, so editing one package means making a new
distribution version. This makes it easy to specify exactly what
version of the database your image is up-to-date with, but does
not scale if there are many contributions.

--
Matthew Fulmer -- http://mtfulmer.wordpress.com/

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Squeak Package Management

Damien Pollet
On Thu, Jun 26, 2008 at 9:28 PM, Matthew Fulmer <[hidden email]> wrote:

> SqueakMap: Packages are out-of-date, because the security model
> does not allow anyone to maintain the package; also, there are
> no dependencies
>
> Universes addressed the dependency issue, but still does not
> allow widely distributed maintenance of the packages. So far,
> it is kept mostly up-to-date, due to a few motivated
> individuals.
>
> Sake/Packages is an attempt to address both of the above
> concerns. It as yet has no UI. Unlike in SM or Universes, the
> entire distribution is versioned as a whole, rather than the
> individual packages, so editing one package means making a new
> distribution version. This makes it easy to specify exactly what
> version of the database your image is up-to-date with, but does
> not scale if there are many contributions.


I don't see the point of having completely separate tools/websites for
code management (SqueakSource/MC) and package management (all others).
I want to configure an image as a user, then decide that I want to
contribute on such package, and upgrade it to the devel snapshot with
the same tool.

Package management and code version control are really the same thing,
just at different levels of granularity.

As a developer, you want the bleeding edge version of the code most of
the time, but also need flexibility (load older versions, branches,
bypass dependancies). You do need dependancies if only for the
documentation value (it's always a pain when code is split in several
packages and you have to checkout each of them manually).

As a user, you want no hassles, ie. a complete and up-to-date package
set with dependancies between stable blessed versions that work. Also,
feature-oriented rather than code-oriented packages, ie. Seaside
instead of all the dozen actual class categories.

Of course these are just two points somewhere in a continuum: paranoid
users will only use bare major release images, power-users will want
more recent packages at the price of more frequent updates and less
stability. Even developers are only developers for a subset of their
image and users for all the rest.

So for me, SqueakSource with practical dependancies and the
possibility to make actual releases would be the best. Maybe the
dependancy/release handling can be done by merging SM  or
Sake-Packages metadata into MC snapshots, but I do believe that it
should be a single integrated application.

--
Damien Pollet
type less, do more [ | ] http://people.untyped.org/damien.pollet

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Squeak Package Management

keith1y
In reply to this post by Tapple Gao
Matthew wrote:
> Sake/Packages is an attempt to address both of the above
> concerns. It as yet has no UI. Unlike in SM or Universes, the
> entire distribution is versioned as a whole, rather than the
>  
This is a feature. If you want an old version of the s/p package map,
then everything is as it was on that past date. (if certain criteria are
adhered to)
> individual packages, so editing one package means making a new
> distribution version. This makes it easy to specify exactly what
> version of the database your image is up-to-date with, but does
> not scale if there are many contributions.
>  
Currently s/p uses mcz files to store its package definitions as a body
of code.

I do not know of any squeak project that currently has scalability
issues with too many contributors to the code repo. s/p is a body of
code like any other.

If a scalability problem was to occur which I doubt very much indeed,
there are numerous solutions.

1. split the packages up into smaller granularity, i.e. WebDevelopment
packages, Dev Tools packages

2. adopt an editorial policy informally - i.e. load scripts only load
the latest "approved" version.
(seaside users do this frequently, loading Lukas or Phillippe's versions
in preference to unknown contributors.)

3. adopt an editorial policy formally - close the repo to a small number
of approved editors.

4. spilt the package up, so that some packages load further package
definitions as needed.

5. adopt a different server from the basic mcz server, such as Magma
with more intelligent actions/merging capbability.

6. write a front end tool for publishing versions with some form of
process management, aka, Gjallar / map.squeak.org / SqueakMap3

7. Split the package definitions master repo up into smaller repo's
arbitrarily as needed. e.g. Database packages, Seaside, Kernel, etc.  
Use sake/tasks to auto generate combinations for specific squeak images
based upon metadata. i.e. collect all definitions with the flag
#worksInCroquet set into PackagesCroquet10

So yes it is scalable, because the actual nuts and bolts of the
implementation of sake/packages is fairly orthogonal to how it is used.
If the usage pattern turns out not to be scalable then we change the
usage pattern.

i.e. Sake/Package definitons could be distributed via DeltaStreams
rather than Monticello.

Randal wrote:
> Notice about 50 uploads a *day*.  And everything *works* in an
> automated way, except for the "CPAN gods" exceptions listed above
As I said before if the current solution cannot cope with 50 uploads a
day, then it is not the fault of s/p, this is a server issue or a usage
patterns issue. Squeaksource is not the only mcz server out there, and
it is not the only solution for distributing/managing code. mcz can
store diffs, magma might make a more intelligent back end and mc2 is out
there somewhere too. This issue is orthogonal to s/p.

The s/p model is intended to be like that of wikipedia etc. relying upon
open editorial policies. They cope with scalability issues somehow.

In addition those 50 uploads a day is quite high bandwidth. But I think
that this would consist of people writing their packages, and every time
they bump a single version, they have to republish so that their users
can keep up.

Not so with s/p.

s/p allows publishers to publish 'NON-SPECIFIC' "the latest version that
I have in my repo whatever it is". So package writers do not have to
constantly worry about publishing their work for their bleeding edge
users. This is a wonderful feature, you just publish ONCE even at the
beginning of your project, you dont even need to wait until it is ready
for release!

Users or publishers can write definitions that are more specific such as
"the latest version that is authored by 'the master coder'. "

or even more specific such as "the latest version that is published on
SqueakMap."

It's only when users find that a specific version is needed for a
specific image that the definition needs to be updated with a "SPECIFIC"
definition.  "Squeak3.10 definitely works with version 522 of Seaside2.8."

So...

A developer can publish a package, generating 100 versions as he goes,
as it reaches some level of maturity the users add the definition that
says version 100 definitely works in Squeak1.1.

Beta users who want the latest all the time, use

Packages beta load: 'ThePackageName'.

Normal users what the definitions that have been conformed to work BY
OTHER USERS. They use..

Packages current load: 'ThePackageName'.

So... I doubt very much that the bandwidth for s/p will be anything like
CPAN for the same amount of business.
> Sake/Packages - single point of update, making publishing hard
Single point of update, making management easy! And putting the power in
the USERS hands, not the publishers.

The publisher will not have used the package in the specific images that
the users have tested it in, so actually the Users collectively have
more knowledge about the use of the package than the publisher. This
model allows the user to publish his knowledge in the appropriate place.
i.e A fix for Squeak 3.7 does not need to contaminate everyone elses
definitions.

I would like to  know how CPAN publishes version 23 of the database
driver for use on this version of Linux, with this Database loaded and
this specialised file system. I beleive the database driver publishes
his work and it is the USERS which decide how it will be used. S/P is
primarily a means by which USERS share "What Works" information.
SqueakMap and SqueakSource is where publishers can publish "What is
Available".

Randal wrote:
> recent Squeak releases for installation, so we have a bootstrapping problem.
>  
I dont think s/p has a bootstrapping problem, since if your image is
capable of loading code, s/p is only code and can be loaded any which
way.  I surmse that s/p will have negligable bootstrapping problems when
compared with the SqueakMap problems of the past. Furthermore
LevelPlayingField is present to help with any bootstrapping problems we
may have for any package in the future.

Randal wrote:
> And all of these systems are missing dependency and automatic upgrade
> management.
>  
s/p models dependencies to whatever depth you wish.

s/p has a rudimentary upgrade capability AND package UNLOAD capability...

If I want to unload Seaside it will unload in a sensible order every
package that has been loaded which depended upon Seaside, prior to
unloading Seaside (well that is the theory).

Damien Wrote:

> I don't see the point of having completely separate tools/websites for
> code management (SqueakSource/MC) and package management (all others).
> I want to configure an image as a user, then decide that I want to
> contribute on such package, and upgrade it to the devel snapshot with
> the same tool.
>
> Package management and code version control are really the same thing,
> just at different levels of granularity.
>  
Amen Brother!

If you fit the paradigm to the task it makes things much easier I think.

Let a coder use code to determine what and how his code is loaded into
his image, and it will be a lot simpler than....

Using a web front end to a database of meta data, describing code files
that are to be replicated all over the world for users that are still
using 1980's modems. etc etc.

90% of software engineering is solving the right problem. The problem is
actually very simple,

"collect the necessary data, what works where and with what dependencies."

s/p collects that data in an executable form (and all of the metadata is
optional anyway), I dont see how it can get much simpler than that.

Randal wrote:
> * CPAN installation is easy
> * CPAN publishing is easy
s/p publishing is probably easier than CPAN, since the package writer
only has to publish once ever, and it is easy to browse every use case
of a particular package...

load:  Installer install: 'PacakgesAllVersions'.

To find all versions of squeak which can load versions of seaside, do
"implementers of #Seaside."
To find all packages that are dependant upon Seaside..

Packages current Seaside allDependants.

> Lowering this barrier in Squeak is important
>
> Merik

It may not be perfect but it is lowered to the point of actually being
usable.

Ralph worte:
> A "feature" of Squeak that might differentiate it from perl is that
> there are many versions of Squeak.  Packages that work with one image
> might not work with another.  So, if you are using 3.8 then you do not
> necessarily want the latest version of some package if that package
> only works in 3.9 or 3.10.   PU addresses this problem by having a
> different Universe for each version of the image, though I think there
> is currently a Universe only for 3.9 and 3.10.  Even if the code for
> PU could run in earlier images, it isn't useful until someone builds a
> Universe for it.
Got it exactly. CPAN does not address the real problems that we actually
face. i.e "WHAT works WHERE." as seen by package users.  And this is
exactly why it is that Users of a particular image that need to manage
the package definitions for that image. The package writers are often
behind the enlightened users who are putting their package to new and
varied uses in different contexts.

> Maybe PU is the closest to CPAN we have.  So the question is, do we keep
> mutating PU into what I'm looking for?
No, because PU will be no use in an image that doesnt have any Network
code, doesnt have XML parser, doesnt have Morphic, so if you do improve
PU someone will have to develop another solution for the more stripped
down image. If future squeaks are based upon more stripped down images
then we end up with two solutions to the same problem.
>  Or do we first need to agree that the
> CPAN might be a pretty darn good model?
>  
Its not a good model because it is solving a problem that we don't
currently have. We are not distributing tar files for users around the
world that are using modems to access the internet at 1200 baud, for use
in a language with no reflexive capabilities. i.e. unix bash/csh etc

(btw: Our packages do not have to be restricted to code, they could
include objects, projects, or anything loadable or creatable by code,
this includes mantis fixes.)

Our problem is not with publishing packages, our problem is enabling
users to load what THEY want into their images and to share their
feedback as to what works with other users in a way that can be used.

Ralph wrote:
> CPAN is probably a good model.  However, a model that works well for
> one language does not always work well for another.  How would you
> think a CPAN-like system would handle the fact that people are using
> 3.6, 3.7, 3.8, 3.9 and 3.10 and that some packages work for all of
> these, but some packages are specialized for a particular image?
>  
Exactly, that's why CPAN is not a darn good model for this problem. I
can only envisage that it would be a darn big complicated unmanageable
pain. The proposal is made with blinkers on, since Randal refused to
even look at s/p because of his concerns over "scalability", which as I
have described above is a complete red-herring-issue.

Randal wrote:
> We do need something that says package X works with squeak Y and package X
> depends on package Z.  so it's two dimensions, not just one.
>  
Yes we do... and which omnipotent package publisher knows the answer to
that question in every context? Answer none, problem is solvable with a
user feedback mechanism, not a package publishing mechanism.

Merik wrote:
> And what about patches?
>
> When an Seaside website needs a patch of an Integer method, for example.
Yes indeed , some images need patching before a particular package will
load. Sake/Packages handles this, it can load patches from mantis using
Installer ensureFix: 1234.  OR It can specify a dependency upon the
Sake/Tasks load mantis fix task, which documents the fixes loaded as it
goes.

so Randal why you wont look at Sake/Packages is beyond me.

regards

Keith
























Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Squeak Package Management

stephane ducasse
In reply to this post by Damien Pollet
I agree
For now within my app I use
        MyApp-All
                MyApp-Core
                MyApp....

And I load several such maps
So I should certainly use Configuration for that.

        Config
                MyApp-All
                OtherApp-All
               
Stef


On Jun 26, 2008, at 10:07 PM, Damien Pollet wrote:

> On Thu, Jun 26, 2008 at 9:28 PM, Matthew Fulmer <[hidden email]>  
> wrote:
>> SqueakMap: Packages are out-of-date, because the security model
>> does not allow anyone to maintain the package; also, there are
>> no dependencies
>>
>> Universes addressed the dependency issue, but still does not
>> allow widely distributed maintenance of the packages. So far,
>> it is kept mostly up-to-date, due to a few motivated
>> individuals.
>>
>> Sake/Packages is an attempt to address both of the above
>> concerns. It as yet has no UI. Unlike in SM or Universes, the
>> entire distribution is versioned as a whole, rather than the
>> individual packages, so editing one package means making a new
>> distribution version. This makes it easy to specify exactly what
>> version of the database your image is up-to-date with, but does
>> not scale if there are many contributions.
>
>
> I don't see the point of having completely separate tools/websites for
> code management (SqueakSource/MC) and package management (all others).
> I want to configure an image as a user, then decide that I want to
> contribute on such package, and upgrade it to the devel snapshot with
> the same tool.
>
> Package management and code version control are really the same thing,
> just at different levels of granularity.
>
> As a developer, you want the bleeding edge version of the code most of
> the time, but also need flexibility (load older versions, branches,
> bypass dependancies). You do need dependancies if only for the
> documentation value (it's always a pain when code is split in several
> packages and you have to checkout each of them manually).
>
> As a user, you want no hassles, ie. a complete and up-to-date package
> set with dependancies between stable blessed versions that work. Also,
> feature-oriented rather than code-oriented packages, ie. Seaside
> instead of all the dozen actual class categories.
>
> Of course these are just two points somewhere in a continuum: paranoid
> users will only use bare major release images, power-users will want
> more recent packages at the price of more frequent updates and less
> stability. Even developers are only developers for a subset of their
> image and users for all the rest.
>
> So for me, SqueakSource with practical dependancies and the
> possibility to make actual releases would be the best. Maybe the
> dependancy/release handling can be done by merging SM  or
> Sake-Packages metadata into MC snapshots, but I do believe that it
> should be a single integrated application.
>
> --
> Damien Pollet
> type less, do more [ | ] http://people.untyped.org/damien.pollet
>
>


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Squeak Package Management

stephane ducasse
In reply to this post by keith1y
Hi keith

This is several weeks that I want to seriously look at Sake.
Will you attend ESUG? Participate to the camp?
Because this would be occasion to do something?

Marcus will visit us from 10 to 23 of August. We could organize a  
Squeak day at Lille?

Stef


On Jun 27, 2008, at 5:21 AM, Keith Hodges wrote:

> Matthew wrote:
>> Sake/Packages is an attempt to address both of the above
>> concerns. It as yet has no UI. Unlike in SM or Universes, the
>> entire distribution is versioned as a whole, rather than the
>>
> This is a feature. If you want an old version of the s/p package  
> map, then everything is as it was on that past date. (if certain  
> criteria are adhered to)
>> individual packages, so editing one package means making a new
>> distribution version. This makes it easy to specify exactly what
>> version of the database your image is up-to-date with, but does
>> not scale if there are many contributions.
>>
> Currently s/p uses mcz files to store its package definitions as a  
> body of code.
>
> I do not know of any squeak project that currently has scalability  
> issues with too many contributors to the code repo. s/p is a body of  
> code like any other.
>
> If a scalability problem was to occur which I doubt very much  
> indeed, there are numerous solutions.
>
> 1. split the packages up into smaller granularity, i.e.  
> WebDevelopment packages, Dev Tools packages
>
> 2. adopt an editorial policy informally - i.e. load scripts only  
> load the latest "approved" version.
> (seaside users do this frequently, loading Lukas or Phillippe's  
> versions in preference to unknown contributors.)
>
> 3. adopt an editorial policy formally - close the repo to a small  
> number of approved editors.
>
> 4. spilt the package up, so that some packages load further package  
> definitions as needed.
>
> 5. adopt a different server from the basic mcz server, such as Magma  
> with more intelligent actions/merging capbability.
>
> 6. write a front end tool for publishing versions with some form of  
> process management, aka, Gjallar / map.squeak.org / SqueakMap3
>
> 7. Split the package definitions master repo up into smaller repo's  
> arbitrarily as needed. e.g. Database packages, Seaside, Kernel,  
> etc.  Use sake/tasks to auto generate combinations for specific  
> squeak images based upon metadata. i.e. collect all definitions with  
> the flag #worksInCroquet set into PackagesCroquet10
>
> So yes it is scalable, because the actual nuts and bolts of the  
> implementation of sake/packages is fairly orthogonal to how it is  
> used. If the usage pattern turns out not to be scalable then we  
> change the usage pattern.
>
> i.e. Sake/Package definitons could be distributed via DeltaStreams  
> rather than Monticello.
>
> Randal wrote:
>> Notice about 50 uploads a *day*.  And everything *works* in an
>> automated way, except for the "CPAN gods" exceptions listed above
> As I said before if the current solution cannot cope with 50 uploads  
> a day, then it is not the fault of s/p, this is a server issue or a  
> usage patterns issue. Squeaksource is not the only mcz server out  
> there, and it is not the only solution for distributing/managing  
> code. mcz can store diffs, magma might make a more intelligent back  
> end and mc2 is out there somewhere too. This issue is orthogonal to  
> s/p.
>
> The s/p model is intended to be like that of wikipedia etc. relying  
> upon open editorial policies. They cope with scalability issues  
> somehow.
>
> In addition those 50 uploads a day is quite high bandwidth. But I  
> think that this would consist of people writing their packages, and  
> every time they bump a single version, they have to republish so  
> that their users can keep up.
>
> Not so with s/p.
>
> s/p allows publishers to publish 'NON-SPECIFIC' "the latest version  
> that I have in my repo whatever it is". So package writers do not  
> have to constantly worry about publishing their work for their  
> bleeding edge users. This is a wonderful feature, you just publish  
> ONCE even at the beginning of your project, you dont even need to  
> wait until it is ready for release!
>
> Users or publishers can write definitions that are more specific  
> such as "the latest version that is authored by 'the master coder'. "
>
> or even more specific such as "the latest version that is published  
> on SqueakMap."
>
> It's only when users find that a specific version is needed for a  
> specific image that the definition needs to be updated with a  
> "SPECIFIC" definition.  "Squeak3.10 definitely works with version  
> 522 of Seaside2.8."
>
> So...
>
> A developer can publish a package, generating 100 versions as he  
> goes, as it reaches some level of maturity the users add the  
> definition that says version 100 definitely works in Squeak1.1.
>
> Beta users who want the latest all the time, use
>
> Packages beta load: 'ThePackageName'.
>
> Normal users what the definitions that have been conformed to work  
> BY OTHER USERS. They use..
>
> Packages current load: 'ThePackageName'.
>
> So... I doubt very much that the bandwidth for s/p will be anything  
> like CPAN for the same amount of business.
>> Sake/Packages - single point of update, making publishing hard
> Single point of update, making management easy! And putting the  
> power in the USERS hands, not the publishers.
>
> The publisher will not have used the package in the specific images  
> that the users have tested it in, so actually the Users collectively  
> have more knowledge about the use of the package than the publisher.  
> This model allows the user to publish his knowledge in the  
> appropriate place. i.e A fix for Squeak 3.7 does not need to  
> contaminate everyone elses definitions.
>
> I would like to  know how CPAN publishes version 23 of the database  
> driver for use on this version of Linux, with this Database loaded  
> and this specialised file system. I beleive the database driver  
> publishes his work and it is the USERS which decide how it will be  
> used. S/P is primarily a means by which USERS share "What Works"  
> information. SqueakMap and SqueakSource is where publishers can  
> publish "What is Available".
>
> Randal wrote:
>> recent Squeak releases for installation, so we have a bootstrapping  
>> problem.
>>
> I dont think s/p has a bootstrapping problem, since if your image is  
> capable of loading code, s/p is only code and can be loaded any  
> which way.  I surmse that s/p will have negligable bootstrapping  
> problems when compared with the SqueakMap problems of the past.  
> Furthermore LevelPlayingField is present to help with any  
> bootstrapping problems we may have for any package in the future.
>
> Randal wrote:
>> And all of these systems are missing dependency and automatic upgrade
>> management.
>>
> s/p models dependencies to whatever depth you wish.
>
> s/p has a rudimentary upgrade capability AND package UNLOAD  
> capability...
>
> If I want to unload Seaside it will unload in a sensible order every  
> package that has been loaded which depended upon Seaside, prior to  
> unloading Seaside (well that is the theory).
>
> Damien Wrote:
>
>> I don't see the point of having completely separate tools/websites  
>> for
>> code management (SqueakSource/MC) and package management (all  
>> others).
>> I want to configure an image as a user, then decide that I want to
>> contribute on such package, and upgrade it to the devel snapshot with
>> the same tool.
>>
>> Package management and code version control are really the same  
>> thing,
>> just at different levels of granularity.
>>
> Amen Brother!
>
> If you fit the paradigm to the task it makes things much easier I  
> think.
>
> Let a coder use code to determine what and how his code is loaded  
> into his image, and it will be a lot simpler than....
>
> Using a web front end to a database of meta data, describing code  
> files that are to be replicated all over the world for users that  
> are still using 1980's modems. etc etc.
>
> 90% of software engineering is solving the right problem. The  
> problem is actually very simple,
>
> "collect the necessary data, what works where and with what  
> dependencies."
>
> s/p collects that data in an executable form (and all of the  
> metadata is optional anyway), I dont see how it can get much simpler  
> than that.
>
> Randal wrote:
>> * CPAN installation is easy
>> * CPAN publishing is easy
> s/p publishing is probably easier than CPAN, since the package  
> writer only has to publish once ever, and it is easy to browse every  
> use case of a particular package...
>
> load:  Installer install: 'PacakgesAllVersions'.
>
> To find all versions of squeak which can load versions of seaside,  
> do "implementers of #Seaside."
> To find all packages that are dependant upon Seaside..
>
> Packages current Seaside allDependants.
>
>> Lowering this barrier in Squeak is important
>>
>> Merik
>
> It may not be perfect but it is lowered to the point of actually  
> being usable.
>
> Ralph worte:
>> A "feature" of Squeak that might differentiate it from perl is that
>> there are many versions of Squeak.  Packages that work with one image
>> might not work with another.  So, if you are using 3.8 then you do  
>> not
>> necessarily want the latest version of some package if that package
>> only works in 3.9 or 3.10.   PU addresses this problem by having a
>> different Universe for each version of the image, though I think  
>> there
>> is currently a Universe only for 3.9 and 3.10.  Even if the code for
>> PU could run in earlier images, it isn't useful until someone  
>> builds a
>> Universe for it.
> Got it exactly. CPAN does not address the real problems that we  
> actually face. i.e "WHAT works WHERE." as seen by package users.  
> And this is exactly why it is that Users of a particular image that  
> need to manage the package definitions for that image. The package  
> writers are often behind the enlightened users who are putting their  
> package to new and varied uses in different contexts.
>
>> Maybe PU is the closest to CPAN we have.  So the question is, do we  
>> keep
>> mutating PU into what I'm looking for?
> No, because PU will be no use in an image that doesnt have any  
> Network code, doesnt have XML parser, doesnt have Morphic, so if you  
> do improve PU someone will have to develop another solution for the  
> more stripped down image. If future squeaks are based upon more  
> stripped down images then we end up with two solutions to the same  
> problem.
>> Or do we first need to agree that the
>> CPAN might be a pretty darn good model?
>>
> Its not a good model because it is solving a problem that we don't  
> currently have. We are not distributing tar files for users around  
> the world that are using modems to access the internet at 1200 baud,  
> for use in a language with no reflexive capabilities. i.e. unix bash/
> csh etc
>
> (btw: Our packages do not have to be restricted to code, they could  
> include objects, projects, or anything loadable or creatable by  
> code, this includes mantis fixes.)
>
> Our problem is not with publishing packages, our problem is enabling  
> users to load what THEY want into their images and to share their  
> feedback as to what works with other users in a way that can be used.
>
> Ralph wrote:
>> CPAN is probably a good model.  However, a model that works well for
>> one language does not always work well for another.  How would you
>> think a CPAN-like system would handle the fact that people are using
>> 3.6, 3.7, 3.8, 3.9 and 3.10 and that some packages work for all of
>> these, but some packages are specialized for a particular image?
>>
> Exactly, that's why CPAN is not a darn good model for this problem.  
> I can only envisage that it would be a darn big complicated  
> unmanageable pain. The proposal is made with blinkers on, since  
> Randal refused to even look at s/p because of his concerns over  
> "scalability", which as I have described above is a complete red-
> herring-issue.
>
> Randal wrote:
>> We do need something that says package X works with squeak Y and  
>> package X
>> depends on package Z.  so it's two dimensions, not just one.
>>
> Yes we do... and which omnipotent package publisher knows the answer  
> to that question in every context? Answer none, problem is solvable  
> with a user feedback mechanism, not a package publishing mechanism.
>
> Merik wrote:
>> And what about patches?
>>
>> When an Seaside website needs a patch of an Integer method, for  
>> example.
> Yes indeed , some images need patching before a particular package  
> will load. Sake/Packages handles this, it can load patches from  
> mantis using Installer ensureFix: 1234.  OR It can specify a  
> dependency upon the Sake/Tasks load mantis fix task, which documents  
> the fixes loaded as it goes.
>
> so Randal why you wont look at Sake/Packages is beyond me.
>
> regards
>
> Keith
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Squeak Package Management

Chris Muller-3
In reply to this post by Tapple Gao
I think one thing that would go a long way toward making things easier
is for the standard packaging tools (Monticello, Monticello
Configurations, Universes, SqueakMap) to be delivered in **stable,
working conditions** in base Squeaks downloaded from squeak.org.

The packaging systems are how Squeak users take the base environment
and enrich it with other packages.  Therefore, it is crucial that
these systems at least be able to LOAD things.  Unfortunately, this
did not occur for a time in public (squeak.org) Squeak releases.

But please help me understand something:

> SqueakMap: Packages are out-of-date, because the security model
> does not allow anyone to maintain the package; also, there are
> no dependencies

As far as being out-of-date that's a human issue, not a software one.
SqueakMap has a list of "Co-maintainers:".  So what is the security
model limitation you are referring to?

Also, dependencies are not really needed with SqueakMap because it is
superceded by SAR processing.  A SAR can do anything, like check if a
certain dependent package is loaded, ask if you want a newer version,
etc.  For Magma, its as simple as loading all of the dependent
packages automatically from within the SAR script.

SAR allows something to be deployed exactly as the creator wants it to
be.  The install script can pop up starting screens, documentation,
etc.

I'll add, SAR works in 3.7, 3.8, 3.9, (3.10?), Croquet, etc.  Do the
other packaging systems?

> Universes addressed the dependency issue, but still does not
> allow widely distributed maintenance of the packages. So far,
> it is kept mostly up-to-date, due to a few motivated
> individuals.

By "addressed" do you mean it put a UI on it?  I did that too by
simply adding a button to Monticello Configurations called, "Build
SAR".  You make your configuration and, in one click, you get a SAR
file deployable on SqueakMap.

Universes tries to make a "model" out of the dependencies is that
right?  It portends to keep a record of "what works with what else"
but that question is really ambiguous and impossible to answer
universally or correctly isn't it?  Besides, running a successful
*Test Suite* is my indication that packages are still working after
I've loaded another package, not the official declaration by a package
tool "works with XYZ".

Also, does Universes support, say, calling out to an external Internet
site during package installation to grab latest resource files like
SAR does?

My other gripe about Universes is the UI; it puts the software in
charge of the user, forcing me expand and scroll a tree, manually
hunting for a package whose name I already know.  Isn't there a way to
*search* for a package?

> Sake/Packages is an attempt to address both of the above
> concerns. It as yet has no UI. Unlike in SM or Universes, the
> entire distribution is versioned as a whole, rather than the
> individual packages, so editing one package means making a new
> distribution version. This makes it easy to specify exactly what
> version of the database your image is up-to-date with, but does
> not scale if there are many contributions.

But "many contributions" would simply be accumulated before versioing
up an entire distribution wouldn't it?  Just like Magma's process
today; I post multiple individual packages to squeaksource multiple
times before posting another Configuration..?

ddp
Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Re: Squeak Package Management

ddp
In reply to this post by Tapple Gao
As a relative newcomer here, I want to echo this sentiment.  It is
entirely unclear how SqueakMap, Monticello, and Package Universe Browser
relate to each other evolutionarily or operationally.  I really have no
idea what constitutes the authoritative source for any given package.
And reading this post, I'm not even sure if "Sake/Packages" is the same
as one of these or yet something else...

In contrast to, dare I say, ASDF-INSTALL, which is at least wired to
Cliki, or CPAN, or pkg_install (FreeBSD), or MacPorts, all of which just
"know" where to go to get packages and have at least rudimentary query
capabilities.  MacPorts in particular tracks both dependencies and has a
notion of versioning, though admittedly the version problem is much more
easily solved for shared libraries than it is for Smalltalk.

I do think you need to consider this as one of the most pressing
problems you face in garnering new adoption and new developers.  I'd
also suggest taking a look at the Python and Ruby communities, both of
which are thriving with new code and shared development.

Matthew Fulmer wrote:
> A discussion started on IRC about how difficult it can
> be to install packages in Squeak, and about how confusing the
> current situation can look to a newcomer. All of our solutions
> seem to have problems:


Reply | Threaded
Open this post in threaded view
|

RE: [squeak-dev] Squeak Package Management

Gary Chambers-4
In reply to this post by Chris Muller-3
Fair comment but these things need to be balanced with an ease of deployment...
SARs are not there for that, Universes are close but require some discipline and has a lack of flexibility, similarly to MCs...

A decent packaging system would make it single-click for end users and not far from that for deployers.

Each of SqueakMap/Universes/MC has their own merit but none bring it all together at the moment, IMHO.

Package integration is a complex problem, each of the above addresses some of the issues. It would be *nice* to have something unified (with end-user interfaces,  more friendly than Installer, perhaps).

Two quids worth (a penny doesn't go far these days :-) ).

Gary.

> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]]On Behalf Of Chris
> Muller
> Sent: 27 June 2008 6:57 PM
> To: The general-purpose Squeak developers list
> Subject: Re: [squeak-dev] Squeak Package Management
>
>
> I think one thing that would go a long way toward making things easier
> is for the standard packaging tools (Monticello, Monticello
> Configurations, Universes, SqueakMap) to be delivered in **stable,
> working conditions** in base Squeaks downloaded from squeak.org.
>
> The packaging systems are how Squeak users take the base environment
> and enrich it with other packages.  Therefore, it is crucial that
> these systems at least be able to LOAD things.  Unfortunately, this
> did not occur for a time in public (squeak.org) Squeak releases.
>
> But please help me understand something:
>
> > SqueakMap: Packages are out-of-date, because the security model
> > does not allow anyone to maintain the package; also, there are
> > no dependencies
>
> As far as being out-of-date that's a human issue, not a software one.
> SqueakMap has a list of "Co-maintainers:".  So what is the security
> model limitation you are referring to?
>
> Also, dependencies are not really needed with SqueakMap because it is
> superceded by SAR processing.  A SAR can do anything, like check if a
> certain dependent package is loaded, ask if you want a newer version,
> etc.  For Magma, its as simple as loading all of the dependent
> packages automatically from within the SAR script.
>
> SAR allows something to be deployed exactly as the creator wants it to
> be.  The install script can pop up starting screens, documentation,
> etc.
>
> I'll add, SAR works in 3.7, 3.8, 3.9, (3.10?), Croquet, etc.  Do the
> other packaging systems?
>
> > Universes addressed the dependency issue, but still does not
> > allow widely distributed maintenance of the packages. So far,
> > it is kept mostly up-to-date, due to a few motivated
> > individuals.
>
> By "addressed" do you mean it put a UI on it?  I did that too by
> simply adding a button to Monticello Configurations called, "Build
> SAR".  You make your configuration and, in one click, you get a SAR
> file deployable on SqueakMap.
>
> Universes tries to make a "model" out of the dependencies is that
> right?  It portends to keep a record of "what works with what else"
> but that question is really ambiguous and impossible to answer
> universally or correctly isn't it?  Besides, running a successful
> *Test Suite* is my indication that packages are still working after
> I've loaded another package, not the official declaration by a package
> tool "works with XYZ".
>
> Also, does Universes support, say, calling out to an external Internet
> site during package installation to grab latest resource files like
> SAR does?
>
> My other gripe about Universes is the UI; it puts the software in
> charge of the user, forcing me expand and scroll a tree, manually
> hunting for a package whose name I already know.  Isn't there a way to
> *search* for a package?
>
> > Sake/Packages is an attempt to address both of the above
> > concerns. It as yet has no UI. Unlike in SM or Universes, the
> > entire distribution is versioned as a whole, rather than the
> > individual packages, so editing one package means making a new
> > distribution version. This makes it easy to specify exactly what
> > version of the database your image is up-to-date with, but does
> > not scale if there are many contributions.
>
> But "many contributions" would simply be accumulated before versioing
> up an entire distribution wouldn't it?  Just like Magma's process
> today; I post multiple individual packages to squeaksource multiple
> times before posting another Configuration..?
>


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Squeak Package Management

Randal L. Schwartz
In reply to this post by ddp
>>>>> "Derrell" == Derrell Piper <[hidden email]> writes:

Derrell> I do think you need to consider this as one of the most pressing
Derrell> problems you face in garnering new adoption and new developers.  I'd
Derrell> also suggest taking a look at the Python and Ruby communities, both
Derrell> of which are thriving with new code and shared development.

And Perl.

And this is why I bring this up.  The whole mishmash of SM/SS/PU/$nextthing
makes it hard for both producers *and* consumers.  I'm hoping this dialogue
brings out a roadmap of what to do and who will do it and what the result will
look like so people can get inspired to action.

--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<[hidden email]> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Squeak Package Management

Edgar J. De Cleene
In reply to this post by Chris Muller-3



El 6/27/08 2:57 PM, "Chris Muller" <[hidden email]> escribió:

> Also, dependencies are not really needed with SqueakMap because it is
> superceded by SAR processing.  A SAR can do anything, like check if a
> certain dependent package is loaded, ask if you want a newer version,
> etc.  For Magma, its as simple as loading all of the dependent
> packages automatically from within the SAR script.
>
> SAR allows something to be deployed exactly as the creator wants it to
> be.  The install script can pop up starting screens, documentation,
> etc.
>
> I'll add, SAR works in 3.7, 3.8, 3.9, (3.10?), Croquet, etc.  Do the
> other packaging systems?


At least one sharing my old claming on the desert for use Sar.
And I tell you I put some minor enhancement for Sar build and managment into
3.10

But some people insist in reinvent the wheel...

Edgar



Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Squeak Package Management

Damien Pollet
In reply to this post by Chris Muller-3
On Fri, Jun 27, 2008 at 7:57 PM, Chris Muller <[hidden email]> wrote:
> SAR allows something to be deployed exactly as the creator wants it to
> be.  The install script can pop up starting screens, documentation,
> etc.

Which is wrong if you want to automate package installation and the
popups can't be blocked in some way. As far as I understand, the SAR
can execute arbitrary code when it's loaded. Instead, a package should
only provide stuff, and let the installer from the deal with it
however it wants.

I don't want to a package to change the background color in my image,
popup windows, ask me questions interactively, delete my home
directory, or fail because it tries to phone back to the developer's
web site to count installations and I'm in the train.

Of course the irony is you can do that with class initialization
anyway… but then this is a problem of Smalltalk, not the package
system.

--
Damien Pollet
type less, do more [ | ] http://people.untyped.org/damien.pollet