Personalized systems with Treated

To support Magma 1.2, I introduced some changes to OrderedCollection
into the Squeak trunk some time ago.  To deliver Magma to Pharo users,
I needed to have these same enhancements in Pharo 1.1 and 1.2.  But
they were already released, so how was this solved?

My first instinct was to deliver a change-set; the classical way to
modify the system.  But I know dirty packages are unattractive, and
besides that the only way to achieve one-click load of something
involving a change-set + MC-packages in Pharo is with a SAR - overkill
since Magma is just MC packages; no resources.  I really wanted to
release Magma 1.2 to Pharo with just a Gofer script.

So, I decided to import the same changes I made to Squeak's
OrderedCollection into Pharo's and then save that new version to the
Inbox and adjusted my Gofer script for loading Magma to merge that
version of Collections from repositories:  Pharo, Inbox, TreatedInbox.

I knew they would accept it, thank you Pharo, BUT it's nice to know
that I could actually deliver my application without having to wait
for it to be integrated, or worry even if it didn't get accepted; that
chain of repositories should work immediately and forever, no matter
the outcome of acceptance.

Thanks to MC merging, this is as unintrusive as it can be; in case
someone runs with their own "personalizations" to the Collections
package, loading Magma won't clobber them; nor will updating with
official Pharo-released patches, because those are merged too.

With the context of this experience, I came to view the "Treated"
repository with a new respect; enabling a non-core developer to
personalize system packages for my application..

 - Chris

This workaround is valid although not less a workaround.

The problem with those other repositories is that they are for specific
uses, as Stef said in other post, the Treated repository is for packages
integrated in current Pharo. So, of course you could post some package
there but it wont belongs to there really.

This workaround it could be better implemented if the package with
specific changes will be pushed to some dedicated compatibility
repository where everyone can push every package and use it in their
install procedures. That would be better.

Now, lets remember that for Pharo at lease, backwards compatibility is a
"great to have" feature not a "must have at all means" feature. So if
sometime, with some release, current or future, a break with the past is
to be made, it will be made. That is for sure. There can be a few people
that will be affected by this (in this case that they can't install
something just like they could yesterday, but hey, this is life, as
Markus said, no progress is to be dead already) but the majority won't
care/will be happier because it is always better for a few persons to
fix some problems than the entire project to stall waiting to be all for
everyone and always.

The changes of core libraries, (collections, kernel, etc) almost sure be
ported to pharo from squeak and if some package release can't work
because of a missing library in Squeak not yet in Pharo that won't be a
big problem. Next release will work, when Pharo has the fixes. We don't
worry, one step a time. And nobody has died by waiting a little for some
wanted feature.

So in the end, I think that this is a non-issue, just a minor effect of
the vision of Pharo that is solved with a little of time and patience.

Other thing, we really don't like changesets or gofer scripts for
installing anything on Pharo (even if we really like gofer). Why?
Because they aren't a package management system. The end result could be
the same: the packages are installed in the system, that is granted, but
the capability of Metacello of declaring which other configurations it
depends on and the one of querying the configuration to see the full
list of packages to be installed is something that no gofer script,
installer script or SqueakMap feature can handle right now without
downloading everything to check.

Regards

El mié, 04-05-2011 a las 21:35 -0500, Chris Muller escribió:
--
Miguel Cobá
http://twitter.com/MiguelCobaMtz
http://miguel.leugim.com.mx

Well, I'm 0-2 so far from the replies understanding the point of my
story..  Oh well.

> The problem with those other repositories is that they are for specific
> uses, as Stef said in other post, the Treated repository is for packages
> integrated in current Pharo. So, of course you could post some package
> there but it wont belongs to there really.

I don't see getting more use out of something than for what it was
originally intended as a "problem".  I was just getting the work done
to quell the complaining for Magma coming from that side..

> This workaround it could be better implemented if the package with
> specific changes will be pushed to some dedicated compatibility
> repository where everyone can push every package and use it in their
> install procedures. That would be better.

Sure, that could be an incremental step but I wasn't in a position at
the time to go lobbying to the Pharo group for a new repository; I
just needed to get Magma done for Pharo'ites.

Question:  When is anything in "Treated" ever "consumed"?  If it's not
ever consumed, it's wasting space, so why not delete it?

> Now, lets remember that for Pharo at lease, backwards compatibility is a
> "great to have" feature not a "must have at all means" feature. So if

Actually that describes Squeak's philosophy; they just do it in a
slower, more-incremental nature.

> Other thing, we really don't like changesets or gofer scripts for
> installing anything on Pharo (even if we really like gofer). Why?
> Because they aren't a package management system.

Exactly; philosophical instead of practical..

>The end result could be the same: the packages are installed in the system, that is granted, but
> the capability of Metacello of declaring which other configurations it
> depends on and the one of querying the configuration to see the full
> list of packages to be installed is something that no gofer script,
> installer script or SqueakMap feature can handle right now without
> downloading everything to check.

Not true.  SqueakMap load scripts specify which other prerequisite
configurations are required and merges them, as needed, from SqueakMap
itself.  See Magma or Maui as an example.  Those prerequisite configs
can be browsed before loading, if desired.

I'm more interested to know the answers to these questions:

  - How does one deploy an application on Pharo that requires a
one-line change to a core system method that is not acceptable to the
Pharo gods?

  - How does one deploy an application on Pharo that loads code from
external web-sites?

  - How does one deploy an application on Pharo that requires external
graphics, videos, XML or other resources?

  - How does one deploy an application on Pharo that requires an
object-graph to be loaded internally into the image that was pre-built
using another tool (e.g., NOT generated via code)?

These questions are much more important to me than "it isn't a package
management system".  If Metacello could do all of these things, I
might be interested..

Regards,
  Chris

El jue, 05-05-2011 a las 13:02 -0500, Chris Muller escribió:
For the same reason that the Treated repository from Squeak isn't
deleted, for historical references. That doesn't means that we should
put there what one wants. (well, not that rude, just that as is so easy
to create our own repo, isn't well seen if we put packages in others'
repos)

More than philosophical I think. A script can reflect over itself A
metacello configuration class can. Text vs class. Unless you put a
parser to work, that is extra work.

Yes, I saw the scripts for Magma and I see check for specific classes to
test if they exist in the image (if they are installed):

[Smalltalk at: #MagmaTestCase] ifFalse: [ "install other mcz" ]

>
> I'm more interested to know the answers to these questions:
>
>   - How does one deploy an application on Pharo that requires a
> one-line change to a core system method that is not acceptable to the
> Pharo gods?

The same tha squeak for changes that aren't accepted/belongs to/are too
specific to an application. You include them as an *mypackage extension
in some package of yours and then when the mcz is installed, the
extension is installed too. That is just monticello as always.


>
>   - How does one deploy an application on Pharo that loads code from
> external web-sites?

Using doIts if necessary. Or specifying some other repo to fetch mcz
things from. A metacello configuration can have any nnumber of origin
repositories to fetch mcz from and then install them in the image. For
resources like images and so, the doIts can have HTTPClient to download
and serialize to the image as needed. Smalltalk at the point of your
fingers.

>
>   - How does one deploy an application on Pharo that requires external
> graphics, videos, XML or other resources?

Idem.

>
>   - How does one deploy an application on Pharo that requires an
> object-graph to be loaded internally into the image that was pre-built
> using another tool (e.g., NOT generated via code)?

Don't understand the example
But for loading external data that is in a suitable form to load you can
use a postLoad doit to open (or download the data) and then bring it
into the image, creating objects on the fly. You can load binary data if
you want (SIXX, whatever) or text data (XML,JSON,etc) at the end is
smalltalk in a doIt doing the work.

>
> These questions are much more important to me than "it isn't a package
> management system".  If Metacello could do all of these things, I
> might be interested..

I think it does those things, maybe a second look at the existings
configurations can clear a lot of doubts.

I have a doubt. the metacello configuration can have groups that install
specific mcz of the full list of mcz that Seaside have to install just
what you need. How will that be made with scripts and SqueakMap?

Regards
>
> Regards,
>   Chris

--
Miguel Cobá
http://twitter.com/MiguelCobaMtz
http://miguel.leugim.com.mx

I'm talking about an override not an extension.

>>   - How does one deploy an application on Pharo that loads code from
>> external web-sites?
>
> Using doIts if necessary.

Right, dropping down into straight load-script and doing it.  Got it!  :)

>>   - How does one deploy an application on Pharo that requires external
>> graphics, videos, XML or other resources?
>
> Idem.

Often specific _versions_ of resources go with each specific version
of an application.  Now I am maintaining those separately, in separate
places using separate tools.

I want something to expand the metaphor of an object being
"encapsulated behavior + state" into my deployment system, so it will
be "encapsulated behaviors + resources".  Thankfully, even Pharo 1.3
still supports .SAR files, but no nice GUI to load them, just the
file-manager.  :(

>>   - How does one deploy an application on Pharo that requires an
>> object-graph to be loaded internally into the image that was pre-built
>> using another tool (e.g., NOT generated via code)?
>
> Don't understand the example

A better example.  What if I just want something that strips my image
down of lots of code and objects, without leaving any new code loaded?

It's a rhetorical question, I already know the answer:  Create a
Metacello config to host only a load-script (doIt).  But I wonder
whether said doIt would be able to "unload itself" when it was done?
Probably could.

> I think it does those things, maybe a second look at the existings
> configurations can clear a lot of doubts.

The picture seems to be, as long as my app is all just "clean-loading
MC packages", then I can use Metacello within its normal bounds. (OR,
I could just use the standard "lighter-weight" SCM tools; Monticello +
MonticelloConfigurations + Gofer).

However, if an application requires anything additional:  overrides,
resources or special-configurations, then I drop down to the "ultimate
flexbility" afforded by doIt's - which is just load-scripting.  Even
then, the resources are not encapsulated as I already said.

SqueakMap is a repository that hosts load-scripts encasulated with or
without resources.  It's perfectly complimentary to all of the other
tools.

I'm exhausted of talking about this..

Regards,
  Chris

El jue, 05-05-2011 a las 14:17 -0500, Chris Muller escribió:
Can't see the difference. Your package just have a method that overrides
one method in a core system class. Or am I missing something?

>
> >>   - How does one deploy an application on Pharo that loads code from
> >> external web-sites?
> >
> > Using doIts if necessary.
>
> Right, dropping down into straight load-script and doing it.  Got it!  :)

Yes, because is the last resort. But most scripts are just pulling
dependencies, not resources so if you advocate the scripts just for the
fraction of special uses instead of a reified tool, that is a personal
decision not really an argument.

>
> >>   - How does one deploy an application on Pharo that requires external
> >> graphics, videos, XML or other resources?
> >
> > Idem.
>
> Often specific _versions_ of resources go with each specific version
> of an application.  Now I am maintaining those separately, in separate
> places using separate tools.

You can have as many doits as you want (they are methods) and each
version loaded by the configuration can invoke a given doit to load
specific version of your resources.

This is a very specific scenario that most surely is not needed by the
90% of users that metacello is aiming to be useful to. The 10% that need
some very restricted loading scenario can have a script that does
everything that need.

>
> > I think it does those things, maybe a second look at the existings
> > configurations can clear a lot of doubts.
>
> The picture seems to be, as long as my app is all just "clean-loading
> MC packages", then I can use Metacello within its normal bounds. (OR,
> I could just use the standard "lighter-weight" SCM tools; Monticello +
> MonticelloConfigurations + Gofer).

Yes, but for most maintainers that "clean-loading MC packages" is what
we need. For example magma. Is a complex project and even so it doesn't
use any of the  spoken features so metacello is good enought to load it
in Pharo.

This argument about scripts vs metacello is like saying that just
because you can do the same with assembler when iterating over a list of
string you don't need a OrderedCollection. yes you can do the same, but
the advantages are greater when working with full objects that with
plain text.

Ok, then lets rest it now.

Regards
--
Miguel Cobá
http://twitter.com/MiguelCobaMtz
http://miguel.leugim.com.mx