SqueakMap crashes in 3.10 beta.7158

Giovanni Corriga wrote: I have a feature request here:
It would be nice to browse squeaksource.com and other squeaksource
repositories from within Squeak.
I find I must use a external webbrowser to get project / url info most
of the time. There seem to be something missing between
Universes/SqueakMap and the monticello browser.
Karl

SqueakMap supports SAR files, which can do *anything*.  Therefore it,
too, supports dependencies.

To me, the process of building an image is something that always needs
to be done with care.  A tool that tries to pick "the most recent
versions" of prerequisite packages, to me, is "gunslinger" style of
building an image.

What exactly does "guaranteed to work together" mean?  This is a
notion I never really followed about Universes.  Does it simply mean
what SqueaMap already tells us?  That neither of the packages does any
modifications to the base image?  "Work" has different means for
different people depending on the needs.  If loading one package
causes a slow-down in the performance of the other that matters to me,
how can it be said "guaranteed to work?"

This is why I used the strong word "misguided".  It is not intended as
a criticism of the technology, maybe more of the marketing..

Stef wrote:

> the intention being universes is to have a kind of certified release
> which is really valuable.

By "certiifed" do you mean an assessment of the level of quality?  If
so, SqueakMap has this too ("Solid as a rock" ... "Full of bugs for
developers only).

> Since lot of squeakmap packages do not indicate really if they work
> on not in a given version.

A given version of Squeak?  Yes it does.  They all indicate what
versions of Squeak they are for, and when you load it you get a
warning if your version of Squeak does not match, but still allowing
to proceed at your risk.





On 10/31/07, Jason Johnson <[hidden email]> wrote:

Hi Giovanni!

Giovanni Corriga <[hidden email]> wrote:
I am interested and will gladly chat with you. :) I have tons of "SM3
writeups" laying around with ideas and concepts.

regards, Göran

Hi,

> People want a smaller image (less packages in base) as we have now ?
> And this smaller image could load any Squeak code from past and future ?

Yes, this is a good vision.  But the base 3.9 Squeak has 4077 classes,
of which 40, 40!, are for SqueakMap.  This is less than 1%.

So, what do we pay for that 1% savings in number of classes in the
image?  SqueakMap.

This is not a bargain!  There are many old packages which are not
present on Universes which have a chance of resurrection if they are
visible.

> If the answer is not,  people should download ready to use images , like the
> excellent of Damien for 3.10.

Oh, does Damiens 3.10 have SqueakMap ready to go?  If so, he should be
able to help you get into the standard 3.10.

> Or 3.8.2 full if this suits his needs.

But that's not 3.10,which is what we're talking about.  There are
those of us who want the *possibility* of benefitting from all of your
hard work.

> If the answer is yes, people should encourage guys trying to reach this.
> Meaning some sacrifices are needed.

I do encourage you, but the whole point of a tiny image is to be able
to build it to what I need.  SqueakMap, for the cost of a mere 40
classes, gives me the ability to take my image in 100 million
additional directions than I could/would with ONLY Universes.

So it's more than worth it.  Purge other stuff, leave SqueakMap for now.

> Today you could use SqueakMap , Universes , Installer and Monticello as ways
> to load code into 3.10.

The code-loading tools should all be included *standard* in the 3.10
image.  Purge other stuff if you need smaller.

> I hope in futures Squeak you don't have any (image could don't have this 4
> packages) .

Absolutely.  To me this is the Spoon dream.

> In the distant future, Squeak become Spoon, all you need is type "I want
> mp3" for cite some people request recently and   not more into 3.10.

We're together man.  Just disagreeing about *this point in time* of
evolution for Squeak.  I think it's too early to purge SqueakMap from
the standard image.

 - Chris

On Oct 31, 2007, at 7:51 PM, Chris Muller wrote:

Hi Chris - I think you are missing one of the main points of the  
Universe model. Universes do not pick the "most recent version of  
anything. A particular universe has pointers to specific code  
packages, i.e. specific mcz files which represent one and only one  
version of that piece of code. There is nothing dynamic about it, so  
it is really like a freebsd port or debian package - you have to have  
specific version of the dependencies that port or package needs, not  
just the "latest".

"Guaranteed to work together" means that the author of the Universe  
has ostensibly tested that the specific package versions in that  
Universe do in fact work together. Since it is NOT grabbing "the  
latest code", that is a claim a Universe can make.

Of course, nothing prevents someone from creating a Universe and  
putting incompatible packages in it and not testing it; the purpose,  
however, is define a cohesive set of code that does work together.

I think the fact that most Universes published have a great deal of  
code in them is a bit misleading as to the purpose of Universes - for  
example, if I have a seaside web app that I want to deploy, building a  
universe that contains the specific version of Kom, Seaside,  
Scriptaculous, Magritte, etc. that is a working, stable application  
makes great sense. Someone can load that Universe and expect it to work.

Later I can create a newer version of Universe with newer packages in  
it - someone can load the new one or the old one, it's there choice.

Brian Brown wrote:
> Hi Chris - I think you are missing one of the main points of the
> Universe model. Universes do not pick the "most recent version of
> anything. A particular universe has pointers to specific code packages,
> i.e. specific mcz files which represent one and only one version of that
> piece of code. There is nothing dynamic about it, so it is really like a
> freebsd port or debian package - you have to have specific version of
> the dependencies that port or package needs, not just the "latest".

That is *definitely* not true. The code in
UUniverseBrowser>>allPackagesNeededToInstall:orIfImpossible: explicitly
uses the *latest* version of a package; the basic loop goes like this:

   package depends do: [ :depName |
     "... some code omitted for brevity ..."
    (universe hasPackageNamed: depName) ifTrue: [
      packagesToConsider add: (universe newestPackageNamed: depName)
    ].

and #newestPackageNamed: does just what it says:

newestPackageNamed: name
   | potentials sorted |
   potentials _ self packagesNamed: name.
   sorted _ potentials asSortedCollection: [ :p1 :p2 | p1 version < p2
version ].
   ^sorted last

> "Guaranteed to work together" means that the author of the Universe has
> ostensibly tested that the specific package versions in that Universe do
> in fact work together. Since it is NOT grabbing "the latest code", that
> is a claim a Universe can make.

Given the above, how can it?

Cheers,
   - Andreas

On Nov 1, 2007, at 12:26 AM, Andreas Raab wrote:

Well, obviously a gross misunderstanding on my part, based on Lex's  
initial discussions and further clarifications on what Universes are  
supposed to be. I did not look at the code to verify that, but was  
arguing from what I had gathered from the discussion.

In light of the actual code, I have to retract what I said, with  
apologies.

Thanks for pointing this out!


>
>
>> "Guaranteed to work together" means that the author of the Universe  
>> has ostensibly tested that the specific package versions in that  
>> Universe do in fact work together. Since it is NOT grabbing "the  
>> latest code", that is a claim a Universe can make.
>
> Given the above, how can it?
>

Agreed ;)

> Cheers,
>  - Andreas
>

On Nov 1, 2007, at 7:43 , Brian Brown wrote:


You're forgetting there are multiple Universes. Inside one Universe,  
you get the latest version. Changes that break backwards-
compatibility have to go into the next Universe.

- Bert -

On Wed, Oct 31, 2007 at 11:26:27PM -0700, Andreas Raab wrote:
Certainly, but this refers to the handling of multiple versions of a
package within the universe (which should generally be discouraged
BTW). On the other hand, if you have versions 2 and 3 of FooPackage in
your universe, and next year someone adds versions 4, 5, and 6 of FooPackage
to SqueakMap, your universe will still point at version 3. This is
probably exactly what you want, particularly if the new and improved
FooPackage V6.0 has been updated to use some annoying new feature that
breaks your image and that you didn't want in the first place ;)

Dave

David T. Lewis wrote:
> Certainly, but this refers to the handling of multiple versions of a
> package within the universe (which should generally be discouraged
> BTW). On the other hand, if you have versions 2 and 3 of FooPackage in
> your universe, and next year someone adds versions 4, 5, and 6 of FooPackage
> to SqueakMap, your universe will still point at version 3. This is
> probably exactly what you want, particularly if the new and improved
> FooPackage V6.0 has been updated to use some annoying new feature that
> breaks your image and that you didn't want in the first place ;)

I don't get it. What you say above is in every way true for Squeakmap.
First, I don't think you actually meant what you said above because it's
trivially true in reverse: When putting v2 and v3 on SM and v4...v6 in
Universes then SM would (obviously) still point at v3.

The only way this makes sense is if you say that when you put v2 and v3
into universe A and v4 ... v6 into universe B that universe A still
points to v3. Okay. SM does that by using version tags - if you mark v2
and v3 for vX.y and v4...v6 for vX.z then you'll load v3 if you are in
vX.y and v6 in vX.z. Where is the difference?

Cheers,
   - Andreas

On 10/31/07, Andreas Raab <[hidden email]> wrote:
>
> For me, Universes are useless. Since I don't run the latest Squeak
> versions I always go to SM when I need to find a particular package.
> Since it has a web interface I (or Google) can find things there. Since
> it lists the versions for which a package is available I can choose one
> that is closest to what I need for porting. Universes are no replacement
> for this.

True, SM is certainly the best option for older images.  Universes
have the version of the package as well, not that does you any good.
:)

> You must be new here ;-)

I've only got around 10 years in the field, but I've never fully
understood this emotional part. :)

>If people were less attached to their tools,
> then Universes would have been built *on top of* SM so that it actually
> increases the value of SM as a global repository of "all things squeak"
> instead of having isolated, inaccessible repositories that decrease the
> value of SM.

Good point, I haven't heard/read why it was chosen to be a separate
thing, but I would guess it was the usual:  it seemed easier to make
something from scratch then add functionality to an existing system.

Hi!

"Jason Johnson" <[hidden email]> wrote:
You would have to ask Lex. I have always argued for merging them and as
I hinted in some post - I and Brian proposed it too at one point. I can
not see either why we couldn't have done Universes "on top of" SM, but I
am partial in all this.

regards, Göran

The misunderstanding here is that it is the latest version of the
*configured* package.  So if someone releases a new version of a
dependency you have, you have to explicitly make a pointer to the new
version.  If you don't, the Universe wont load it because it doesn't
know about it.

On 11/1/07, Andreas Raab <[hidden email]> wrote:

Jason Johnson wrote:
> The misunderstanding here is that it is the latest version of the
> *configured* package.  So if someone releases a new version of a
> dependency you have, you have to explicitly make a pointer to the new
> version.  If you don't, the Universe wont load it because it doesn't
> know about it.

I don't understand this. What does "configured package" mean? The way I
understand it, it goes like this:
* You post a couple of versions of the Graphics package into the 3.10
universe.
* I post a couple of versions of the Balloon package into the 3.10
universe. The Balloon package depends on Graphics, but since I use the
Graphics package there is a good chance that I've tested it so it should
be okay.
* Later, you post a new version of the Graphics package. Since you don't
depend on the Balloon package there is absolutely no guarantee that
you've ever tested it together (and how could you - every second package
will depend on Graphics).

The way I read the code, my Balloon package will *automatically* pick up
the latest version of the Graphics package, without any further explicit
intervention.

If I'm mistaken, please correct me.

Cheers,
   - Andreas

On 11/1/07, Chris Muller <[hidden email]> wrote:
> SqueakMap supports SAR files, which can do *anything*.  Therefore it,
> too, supports dependencies.

By writing scripts?  In this case I think we can do better.  I mean,
if someone wanted to modify SM to expose some interface for adding
dependencies and then implement that interface by creating a script
inside the SAR then that's good enough.  Just not the ad hoc scripts
that are done today that fight with each other and so on.

Having full power is good, but in these cases when a common use case
is identified we should "DSL-ize" it imo, and that is basically what
you get with the Universes.

And by the way, I got in this thread only in reaction to your earlier
comment.  I don't want SM to be removed from the standard image
either.  There are plenty of other things that would be higher
priority imo.

> To me, the process of building an image is something that always needs
> to be done with care.  A tool that tries to pick "the most recent
> versions" of prerequisite packages, to me, is "gunslinger" style of
> building an image.

As explained in another mail, it doesn't do this.  If you add a
package to a universe, then add an upgraded version of that package to
the universe later, of course it picks the most up to date version
that you have claimed works.

I suppose you could argue that the act of adding the new version
should remove the old one from the universe, but I believe it is there
for the "upgrade" functionality.

> What exactly does "guaranteed to work together" mean?  This is a
> notion I never really followed about Universes.  Does it simply mean
> what SqueaMap already tells us?  That neither of the packages does any
> modifications to the base image?  "Work" has different means for
> different people depending on the needs.  If loading one package
> causes a slow-down in the performance of the other that matters to me,
> how can it be said "guaranteed to work?"

It's the same concept as they have with the Debian apt-get system,
though obviously there are more people, and therefor more rigor in the
Debian system.

> By "certiifed" do you mean an assessment of the level of quality?  If
> so, SqueakMap has this too ("Solid as a rock" ... "Full of bugs for
> developers only).

Doesn't everyone just pick "Full of bugs" so they don't get blamed if
it breaks? :)

> A given version of Squeak?  Yes it does.  They all indicate what
> versions of Squeak they are for, and when you load it you get a
> warning if your version of Squeak does not match, but still allowing
> to proceed at your risk.

But the issue is that no one is updating and saying "yes this does
work in 3.9/3.10".  I have seen plenty of times where I read about
some package I should get, it's the latest version, so I go to SM and
it says the package is for 3.7.

The "advantage" of Universes is that nothing can be loaded until it's
put in the Universe, so no one can tell me to go load some package
without setting it up to be in 3.9/etc.

On 11/1/07, Andreas Raab <[hidden email]> wrote:
>
> I don't understand this. What does "configured package" mean?

Configured for that specific Universe.  Each universe has it's own
separate configuration about the packages it contains.  Just creating
a package and putting it on Squeak Source wont make the Universe aware
of it.

Well, this *can* be a way it works but this is not how it is
*supposed* to work.  The only people who are supposed to publish
packages to a Universe are the package maintainers, and they are not
supposed test everything when they bring a new package in.  It was my
understanding that this was part of the goal of having an "all tests
green" 3.10.  To make it faster verifying packages in the 3.10
Universe.

> The way I read the code, my Balloon package will *automatically* pick up
> the latest version of the Graphics package, without any further explicit
> intervention.

Yes, that's true.  If someone comes along and adds a new version of a
package to the universe that doesn't work with other packages that
will cause a breakage.  But this is using the system in a way it
wasn't intended.

> If I'm mistaken, please correct me.

You are technically correct.  There is a social aspect required with
Universes to make them work, but the benefit is that when I go get the
latest "Seaside" universe, and I work in it for 3 weeks off line, then
I get back on the internet and click "update" it can automatically
upgrade every package that is behind and I *should* be able to have
confidence that these upgrades wont break existing packages.  Provided
people didn't do what you just described. :)

On 11/1/07, Jason Johnson <[hidden email]> wrote:
>
> The only people who are supposed to publish
> packages to a Universe are the package maintainers, and they are not
> supposed test everything when they bring a new package in.

Ugh, what an embarrassing typo.  I need to upgrade my proof reader! :)

Dear All,

Knowing that the base image is going to get smaller, I thought that we
had already established that the release base image is not supposed to
be the end user image, but a basis for making published images for end
use-cases. Of which squeak-dev is but one example and Edgars promised
FunSqueak would be another.

If I am putting together a release image tailored to my needs it becomes
difficult if I have to first remove the packages that I dont want. I
think that this could improve eventually, but right now, removing is
harder than adding. So the image should be distributed as minimally as
possible. The whole point of pulling things out of 3.10 is to make it
smaller. I think 40 classes is quite a lot, 1% is a lot. Its only 1%
because there is so much other stuff still to be removed.

If I am wanting to deploy a minimal image I do not want to duplicate
functionality, SqueakMap is duplicate functionality, because Installer
can load from squeakmap and Installer is one class!

Ok it may not be the best design or code out there but making it one
class was an initial goal for this reason. On the basis that other
larger-code would be removed in order to minimize the image footprint,
Installer should be small enough to remain in a minimal image while
having sufficient functionality to install whatever you might like from
whatever source.

cheers

Keith

p.s. The biggest problem with Universes that I see is a social one,
rather than a technical one. I think that they need to be in the control
of one "body" of maintainers. Either they need to have one master to be
in charge, or they need to be open so that the community can perform the
role colaboratively. Having different parts of the universe under
different control cultures makes it harder to make a coherent story.

Hi!

Keith Hodges <[hidden email]> wrote:
> If I am wanting to deploy a minimal image I do not want to duplicate
> functionality, SqueakMap is duplicate functionality, because Installer
> can load from squeakmap and Installer is one class!
>
> Ok it may not be the best design or code out there but making it one
> class was an initial goal for this reason. On the basis that other

Note that the reason SM is larger is because you get the full domain
model of SM inside your image.
This means that you can easily write snippets of code interrogating it,
you can use it offline (yes you can) etc etc.

AFAIK Installer (while being a nice concept) operates by web scraping,
which is fine BUT it thus "works around" the SM model and does not work
offline and does not offer the domain model and does not give you client
side caching (?) and does also not have code to use the server side
cache (I am guessing).

So even though I like Installer, don't simplify it for people who know
less - it is NOT a drop in replacement of the SM code.

But yes, if you WANT to skip all the above parts - then fine, it works.

regards, Göran

Jason Johnson wrote:
> On 11/1/07, Jason Johnson <[hidden email]> wrote:
>> The only people who are supposed to publish
>> packages to a Universe are the package maintainers, and they are not
>> supposed test everything when they bring a new package in.
>
> Ugh, what an embarrassing typo.  I need to upgrade my proof reader! :)

I figured it was a typo ;-) So the basic difference between SM and
Universes is that the latter uses a community policy (you "ought" not to
put bad things there) right? This sounds reasonable but can we then
please drop the silly nonsense about "guaranteed to work together"? If
you need a catchy marketing phrase, how about "community tested"? (in
the best of all worlds this is just what it would be)

Also, this only seems to emphasize the complementary nature of Universes
and SM. If, say, Universes would use SM as their "backend-storage" you
could have your cake and eat it too: Sets of community-tested packages
that are available for one-click install, and "all code ever written for
Squeak" available on SM for historical or porting purposes.

Cheers,
   - Andreas