[squeak-dev] Re: worst crash yet trying to load stuff from Packages Universe in 3.10.2....

[squeak-dev] worst crash yet trying to load stuff from Packages Universe in 3.10.2....

Hi Greg,

I like your spirit of adventure.
My bug finding/creating assistant Puck thinks you do good work. :)

Sure you wouldn't like to become a mantis reporter?


***
Ok. From the evidence so far.
You have gotten to the emergency evaluator
 because the debugger blew up on whatever was the original error.

The debugger blew up probably because of a misdirected message.
People tend to put in:
answer
when they meant:
^ answer.
In this case something in the Pinesoft ui enhancements
 looks like it left one out in a LabelMorph class message.
Instead of answering an instance it returned the class itself
 which had no idea what to do with an message to its instance.

Congratulations you found a bug.

Puck would like to know exactly how to recreate this failure.
 (So he can try it out on some of his "friends" :).

My curiosity would like to know why you are so ambitious as to load so much.
And I sympathize with you wanting it all to work right away. Out of the box.

What you may be slowly realizing is that all this is Beta software at best.
Test coverage in squeak is a recent thought.
Most of the software is released with prayers for its survival.
First time heavy users find the integration bugs.
That would be you.
Oldtimers, by instinct born of experience, tend to treat squeak tenderly.
As if they are walking on eggs that might break at any instant
 and give off a rotten stentch.

A sad state of affairs to be sure,
but the alternative would be a lot of unpaid work.
And the person who would take that on
 would have to have a powerful personal reason for doing so.

So far in this small community no one has shown up with that need.  
Still you never know.

Leaving aside the who. Do you or anyone else out there
 have an idea of how that testing could be done?

Yours in curiosity and service, --Jerome Peace



     

On 15-Dec-2008, at 1:46 AM, Jerome Peace wrote:
>
> Sure you wouldn't like to become a mantis reporter?

If I can send an e-mail that will create and/or append to a report,  
then sure!


> Puck would like to know exactly how to recreate this failure.
> (So he can try it out on some of his "friends" :).

Looks like all I had to do was try to load SmallDEVS from the default  
package universe.

> My curiosity would like to know why you are so ambitious as to load  
> so much.

I want to start with everything I want, either functionality or as  
reference classes.

Partly I want to start this way because I have VERY quickly learned  
that I can accidentally break an image just by trying to file  
something in that someone else has published, even if it has  
supposedly been "blessed" in some way by being included in the default  
package universe for the release I'm using.  I'm not quite so  
disciplined that I will remember to be very careful with image copies  
later on so I'm trying to create a stable baseline image with  
everything I think I'll want for now.

> And I sympathize with you wanting it all to work right away. Out of  
> the box.

To quote from <URL:http://wiki.squeak.org/squeak/5918>:

        "The Stable 3.10 universe is a package universe for Squeak 3.10, akin  
to the ones done for 3.7 and 3.9. It includes 214 optional packages  
that have all been verified to at least load into Squeak 3.10."


I don't exactly know if what it refers to is the same thing that I get  
when I press the "Package Universe" button on the initial World in the  
3.10.2 release image I'm using, but I've now proven at least three or  
more of the packages from the default PU for 3.10.2 won't even file in  
cleanly, and I'm just picking from what I would consider to be a  
_very_ conservative list of things I'm interested in.


> What you may be slowly realizing is that all this is Beta software  
> at best.

"beta"?  3.10.2 isn't even a ".0" release (though it is an even  
number...)

Perhaps I'm confusing a nice big button everyone says to press if I  
want stuff back that was taken out of old-time Squeak releases with a  
part of the actual release, but then again there's that claim above  
I'm quoting.


> Test coverage in squeak is a recent thought.

Perhaps, but I didn't have anywhere near this much trouble with 2.8 or  
3.0.  Squeak's QA has completely disintegrated, at least if you  
consider the default PU button to be a part of Squeak.

> Most of the software is released with prayers for its survival.

 From what I've seen of PU, and it's apparent thousands of steps  
backwards from the 3.0 days, I'm not even sure I see any evidence of  
such prayers being made for it!

> First time heavy users find the integration bugs.
> That would be you.

Hah!  If I'm a "heavy user" then I can't imagine how you would  
describe someone who really wants to dive into Squeak and do something  
major.  I'm still just learning here.  I'm not even interested in  
doing any of the really fancy database, web, or image maker stuff.

> Oldtimers, by instinct born of experience, tend to treat squeak  
> tenderly.
> As if they are walking on eggs that might break at any instant
> and give off a rotten stentch.

Having played with 2.8 and 3.0 in days gone by I'd have to say that a  
great deal of this stench is quite recent.

--
                                        Greg A. Woods; Planix, Inc.
                                        <[hidden email]>

[squeak-dev] Re: worst crash yet trying to load stuff from Packages Universe in 3.10.2....
>Greg A. Woods; Planix, Inc. woods at planix.ca
>Mon Dec 15 19:54:06 UTC 2008
At puck's insistence I loaded the SmallDEVS package (by itself) into a 3.10.2 image. I didn't run into the trouble you described. So SmallDEVS alone is not the problem. I looked at the 3 class side methods for LabelMorph and they all look correct as far as answering an instance. I do not have the Pinesoft UI Enhancements in my image so I still suspect that they have a buggy overide somewhere.

Unreproduceable results are not worth tracking. I usually insist on a mantis report first. I made an exception here. I'll be happy to look into it again if you can describe a recipe for reproducing the bug and put it into a mantis report.

Mantis does not have a email to append feature yet. In their discussions about it, they seemed to think that would lead to spam.

The effort to make a good report acts as a good filter. It doesn't guarantee a report will get attention. But if someone will go through that much work, maybe it will be worth other peoples time to play with it.

Fair warning. This is a volunteer community. That usually means the person something most annoys gets to work on it first.



>> My curiosity would like to know why you are so ambitious as to load  
>> so much.
>
>I want to start with everything I want, either functionality or as  
>reference classes.
>
Okay. So the question is would there be a way to do that, that would work?
Then we create your ideal starting image and share it with others.
Advantage: It will take less time to download a 20Meg pre-made image than universe load the pieces from around the world.

Why not make out your list and see if someone with similar interest would be willing to help build the image.



*/>Partly I want to start this way because I have VERY quickly learned  
>that I can accidentally break an image just by trying to file  
>something in that someone else has published, even if it has  
>supposedly been "blessed" in some way by being included in the default  
>package universe for the release I'm using.  I'm not quite so  
>disciplined that I will remember to be very careful with image copies  
>later on so I'm trying to create a stable baseline image with  
>everything I think I'll want for now.
>
Ok. So making the portmanteau image first, saves having to remember what you left out later. You feel it is less risky than trying to read a module in later and breaking an image you have invested your development effort on. Fair enough.


>> And I sympathize with you wanting it all to work right away. Out of  
>> the box.
>
>To quote from <URL:http://wiki.squeak.org/squeak/5918>:
>
> "The Stable 3.10 universe is a package universe for Squeak 3.10, akin  
>to the ones done for 3.7 and 3.9. It includes 214 optional packages  
>that have all been verified to at least load into Squeak 3.10."
>
In theory, theory and practice are the same. In practice they are different.


Yes. The release process has been struggling. The maintanence choices got people into trouble making bugs much harder to correct. In the end deadlines, not quality dictated what got released. Somebody with authority has to care before this will change. I wonder if the will is out there.
Still you never know.

Um, the problem you are describing is there. I would wonder how much is due to laxer quality controls vs. the exponential explosion of combinations that need to be tested. There are a lot more options now than eight to ten years ago.

People have tried to deal with it by striping away pieces and trying for a slimmer basic image in hopes that the quality of that would improve.

Except that there have been major additions.
3.8 added I18n enhancements. That introduced much more complexity especially with fonts and character sets.
3.9 went on for two years. It attempted to add so much from different sources integration problems abound. I've documented what I personally have found on mantis. There is much more that others have come across too.
3.10 was supposed to last six months. It went on for a year. It was headed by Ralph Johnson in Illinois with the workhorse load shouldered by Edgar, who lives in Rosario, Argentina. There were communication problems to put things mildly.

And there were struggles with the method of image maintanence which had shifted at 3.9.  Most people will agree that using MC to maintain an image was a problematic choice. My take is that MC is an out of sequence tool. Requiring decisions about packaging to be made too early in the process of writing methods.
And patching bugs often cut accross packages. Add that to the fact that MC does not scale well. (Longer and longer times are taken to load larger and larger packages.) And Edgar's time was spent like yours. Learning how to make mistakes and then how to recover from them.
Your ambitious loading project puts you in the "heavy use" category. Your other posts and your naive expectations of what squeak will be capable of made me think "first time". Your having used squeak back when it worked better doesn't quite nullify that. It does explain some of your persistence and adventurousness.

You at least know and believe it can BE better than the current state of affairs.


>I'm still just learning here.  I'm not even interested in  
>doing any of the really fancy database, web, or image maker stuff.

Yeah. Me too.
>
>> Oldtimers, by instinct born of experience, tend to treat squeak  
>> tenderly.
>> As if they are walking on eggs that might break at any instant
>> and give off a rotten stentch.
>
>Having played with 2.8 and 3.0 in days gone by I'd have to say that a  
>great deal of this stench is quite recent.
>
Yeah. Puck says: " Ain't progress grand?"

Yours in curiosity and service, --Jerome Peace


     

Greg A. Woods; Planix, Inc. wrote:
>> Puck would like to know exactly how to recreate this failure.
>> (So he can try it out on some of his "friends" :).
>
> Looks like all I had to do was try to load SmallDEVS from the default
> package universe.

No surprise. Polymorph as a class called LabelMorph. SmallDEVS has a
class called LabelMorph. They don't agree. So much for not having name
spaces.

>> Test coverage in squeak is a recent thought.
>
> Perhaps, but I didn't have anywhere near this much trouble with 2.8 or
> 3.0.  Squeak's QA has completely disintegrated, at least if you consider
> the default PU button to be a part of Squeak.

Well, that's both true and not. A thing to keep in mind here is there
wasn't even remotely as much stuff available at your fingertips in those
days. SqueakMap did not exist in 2.8, neither did Universes. There were
a few better known projects that you could easily find, everything else
was basically inaccessible. And with variety come all the packaging and
scaling problems that were never addressed in Squeak.

>> First time heavy users find the integration bugs.
>> That would be you.
>
> Hah!  If I'm a "heavy user" then I can't imagine how you would describe
> someone who really wants to dive into Squeak and do something major.  
> I'm still just learning here.  I'm not even interested in doing any of
> the really fancy database, web, or image maker stuff.

There are misunderstandings on both ends here. As a first-time user you
naturally expect those things to "just work" so calling you a "heavy
user" is not a good characterization for what you're doing. I would call
this "exploratory loading" of packages which is probably the worst kind
for integrators to deal with unless they have automated tools for
detecting those package conflicts.

As for doing something major: The first thing you leave at home is the
idea of loading random untested code from the net and expect it to work
in your project. You start with a nice stable base image that you expect
to use for several years and build on top of that.

When you need something, you look at the options, see if one suits your
task, review the code to see whether it's good enough to reuse or
whether you're better off rewriting it from scratch, and then you port
it into your environment. I've done this many times and if you start
with the expectation that you *will* have to actually port this into
your environment you know that this will be a sizable investment but
that at the end of the process you have something that works and that
you can support over time.

I have done this many times. It works quite well if you are realistic
about the investment you are making.

>> Oldtimers, by instinct born of experience, tend to treat squeak tenderly.
>> As if they are walking on eggs that might break at any instant
>> and give off a rotten stentch.
>
> Having played with 2.8 and 3.0 in days gone by I'd have to say that a
> great deal of this stench is quite recent.

I think yours is a bit of a case of selective memory ;-) I've been
playing in those days too and I do not recall even five projects that
one could easily find on the net.

While I agree that there is some completely needless recent breakage
that makes things significantly worse than they have to be, I would say
that the majority of the problems you are seeing are the result of
having more options available at your fingertips.

And the best way I can think about addressing them would be by having
automated tests which can be run all the time and which test which
packages load together and which ones fail. This information could then
be used to inform the user about the conflict and avoid random crashes.

Cheers,
   - Andreas

Thanks for your comments Jerome!  Your insight is quite useful!

I want to add one comment in reply to one of the things you mention....

On 15-Dec-2008, at 9:36 PM, Jerome Peace wrote: I wouldn't expect every combination to be tested.

However there is one fool-proof and extremely obvious test that should  
always be done in this kind of situation, but which clearly was never  
ever done:  LOAD EVERYTHING!

It is, and I think it _should_ be, my expectation that _everything_ in  
the default Package Universe for an official release should load  
together cleanly without even a hint of problems or conflicts.

Furthermore _everything_ should be tested minimally in this full-load  
image.  If there are unit tests that can be run then they should _all_  
run green.  There's no excuse whatsoever for not doing this minimal  
amount of testing.  This kind of QA testing could even be done  
headless and totally automated.  Obviously finding the cause of  
breakage and eliminating it from the PU is much more difficult, though  
with the right tools even that can be done automatically.  There are  
tons of examples in other system domains of doing this kind of  
automated basic QA testing:  Mozilla Tinderbox, NetBSD nightly builds,  
pkgsrc bulk builds, and so on.  Why it's not done for Package  
Universes for Squeak, especially official release PUs, I cannot even  
begin to imagine.

This lack of basic QA in other Squeak-related images isn't limited to  
the official Squeak+PU either of course.  I've noted that in the most  
recent Pharo image there are things that are _trivial_ to test but  
which throw up debugger windows instantly when I try them in a fresh  
image.  I.e. some things obviously were not even looked at, never mind  
tested in any proper way.  For example the basic Alice demo blows  
chunks.

There are even problems it seems in the latest VisualWorks (7.6) that  
I've tried too, though nothing quite so drastic as the problems I've  
had with even the most minimal stuff from the 3.10 PU.


> People have tried to deal with it by striping away pieces and trying  
> for a slimmer basic image in hopes that the quality of that would  
> improve.

I don't think that's a valid approach in any way, shape, or form.

Sure, the basic image might be clean and stable, but it's useless.  
There's no fun stuff for beginners and kids, and there's no useful  
stuff for developers.

I would try one of the "dev" images, but they include stuff I'm sure I  
_don't_ want.  Sigh.

--
                                        Greg A. Woods; Planix, Inc.
                                        <[hidden email]>

On 16-Dec-2008, at 12:00 AM, Andreas Raab wrote:
>
> There are misunderstandings on both ends here. As a first-time user  
> you naturally expect those things to "just work" so calling you a  
> "heavy user" is not a good characterization for what you're doing. I  
> would call this "exploratory loading" of packages which is probably  
> the worst kind for integrators to deal with unless they have  
> automated tools for detecting those package conflicts.

Indeed, I do fully expect everything in the default "release" version  
of the Package Universe to actually work with the release it is  
intended to work with.

The kinds of conflicts I've found are _TRIVIAL_ to find.  Just LOAD  
EVERYTHING.  If you can't then something MUST be kicked out of the  
PU.  Someone can fix the conflict later if they want the official  
"release" PU to include that something.

This level of QA testing for PU is trivial -- there's no valid excuse  
for not doing it.  It doesn't even have to be automated -- just select  
everything and hit the install button.  If it blows up then the PU is  
broken and useless and you go back a step and decide whether the thing  
causing the conflict is worth the effort of fixing, or whether the  
thing it might be conflicting with is really a higher priority to  
include or not.  The next level of running all available unit tests  
should also be trivial and required for release PUs.

I would expect what you say to be true of SqueakMap, even a SqueakMap  
with proper dependency tracking and conflict management.  SqueakMap  
does claim to be a one-size one-stop shop for everything after all.  
However I really do not expect the same problems of official "release"  
Package Universes, especially when the button is there in the default  
official release and everyone is told to push it and every official  
looking bit of documentation says it's been tested successfully to  
work with the intended release.

--
                                        Greg A. Woods; Planix, Inc.
                                        <[hidden email]>

>
> The kinds of conflicts I've found are _TRIVIAL_ to find.  Just LOAD
> EVERYTHING.  If you can't then something MUST be kicked out of the
> PU.  Someone can fix the conflict later if they want the official
> "release" PU to include that something.
Unfortunately that's an unrealistic test for Package Universes.

There are sometimes two versions of the same or similar item, e.g 3
different Magma installations.
Sometimes more than two versions of the same thing are present for
various reasons.
And finally you cant load 3 different Monticello's with Monticello.

Keith

 > This level of QA testing for PU is trivial -- there's no valid excuse
 > for not doing it.

In a volunteer community, yes there absolutely is. The valid excuse is
"lack of resources" and it can only be fixed by someone (that means you)
actually putting in the time and effort of doing it. How about it? You
could start loading all the packages in order and see what the first one
is that breaks? That'd be a start.

Cheers,
   - Andreas


Greg A. Woods; Planix, Inc. wrote:

On 16-Dec-2008, at 4:23 PM, Keith Hodges wrote:
>
>>
>> The kinds of conflicts I've found are _TRIVIAL_ to find.  Just LOAD
>> EVERYTHING.  If you can't then something MUST be kicked out of the
>> PU.  Someone can fix the conflict later if they want the official
>> "release" PU to include that something.
> Unfortunately that's an unrealistic test for Package Universes.

Hopefully you mean this only in the context of what you say below.

> There are sometimes two versions of the same or similar item, e.g 3
> different Magma installations.
> Sometimes more than two versions of the same thing are present for
> various reasons.
> And finally you cant load 3 different Monticello's with Monticello.


Well as I read the design goals for PU, there were not supposed to be  
multiple versions of things in a given PU now where there.  Indeed  
various comments in the archives about the failure to meet this goal.

Practically though what I really meant was obviously to select just  
the most recent version of everything.  That's time consuming with the  
current interface, especially with the pollution of so many deprecated  
versions of most packages, but not incredibly that hard to do, and I'm  
sure someone who knows the PU implementation could automate it in  
nearly a heartbeat.

I'm absolutely stunned that this most basic test has not been done as  
the minimal go/no-go test for every official "release" PU.  Given the  
emphasis in the community on unit tests it's stunning that they are  
also not a go/no-go requirement for any official release too (image &  
PU, of course).

The tools are there!  Why aren't they being used?

--
                                        Greg A. Woods; Planix, Inc.
                                        <[hidden email]>

On 16-Dec-2008, at 4:28 PM, Andreas Raab wrote:

> > This level of QA testing for PU is trivial -- there's no valid  
> excuse
> > for not doing it.
>
> In a volunteer community, yes there absolutely is. The valid excuse  
> is "lack of resources" and it can only be fixed by someone (that  
> means you) actually putting in the time and effort of doing it. How  
> about it? You could start loading all the packages in order and see  
> what the first one is that breaks? That'd be a start.

Sorry, no that is _not_ a valid excuse.

Someone made the official 3.10 PU.  Someone even made the claim that  
everything in it had been tested to load (without even adding the  
qualifier "on its own").  Sadly they did not also do the obvious thing  
of trying to load the most recent version of everything in that PU.  
It is a simple, obvious, and easy test to do and would clearly have  
avoided all the troubles I have encountered.  Running all unit tests  
is two or three clicks more.

Were I the person responsible for publishing that official PU I would  
never have done it in the first place if I couldn't have managed to  
take the time to do the basic QA testing I'm suggesting should have  
been done.  Don't start something you can't finish and sure as heck  
don't publish a half-finished mess and make outrageous claims about it!

I think the real problem here isn't the lack of QA testing.  The real  
problem is the attitude of how things apparently got selected to be  
included in the official release PU.  It (for 3.10) has ended up being  
effectively just a much less usable collection of what seems to be a  
vast majority of what's also in SqueakMap, and with the only advantage  
in that it automatically selects known dependencies and sorts the  
installs topologically.  Maybe this is useful enough for experts who  
know everything about what they know they want, but it sure as heck  
isn't something for anyone even at my level, let alone any true  
beginner.

The FunSqueak image might be better for beginners, but it has similar  
breakage as Pharo -- stuff might be loaded, but it still doesn't  
work.  Running all SUnit tests in FunSqueak shows even more breakage  
everywhere in the image -- doing so actually core-dumps the VM on OSX  
not far into the over 7000 tests.

In an ideal world something like FunSqueak should be the product of a  
full load of everything in the official "release" PU, and it should be  
something in which all the SUnit tests run green after the load.

At least the 3.10.2-final-7179 image does pass most of the 2254 tests  
it includes (I get 2 failures on OSX)

--
                                        Greg A. Woods; Planix, Inc.
                                        <[hidden email]>

Greg A. Woods; Planix, Inc. wrote: Are you actually using the 3.10 PU? Or the 'development' universe.

Open universe editor to find out.

Keith

On 16-Dec-2008, at 5:43 PM, Keith Hodges wrote:
> Are you actually using the 3.10 PU? Or the 'development' universe.
>
> Open universe editor to find out.

Hah!  First I've heard of that thing!

It says "Developement" in the top field.

If that's not right then I'd say the official image is broken.

How do I change it to whatever it should be for the official 3.10  
release?  I.e. what do I type or paste into that field to replace the  
word "Development"?

Is someone going to fix 3.10.3 so that it uses the right PU?

--
                                        Greg A. Woods; Planix, Inc.
                                        <[hidden email]>

Greg A. Woods; Planix, Inc. wrote: No, please don't.

Development is the universe that is currently being updated with the
latest stuff. Most things in there are tested against 3.9 and 3.10,
since thats what developers are using, so it is the most maintained.

Since 3.10 is the current released image then it is in the Development
Universes that package load dependencies stands the best chance of being
fixed.

This is what is called a compromise, and we know what needs to be done,
and that is being done, just not in 3.10

Keith

On 16-Dec-2008, at 6:13 PM, Keith Hodges wrote:

> Greg A. Woods; Planix, Inc. wrote:
>>
>> How do I change it to whatever it should be for the official 3.10
>> release?  I.e. what do I type or paste into that field to replace the
>> word "Development"?

So, I still don't know how to get to a proper "release" version of the  
PU to open when I press the "Package Universe Browser" button....

>> Is someone going to fix 3.10.3 so that it uses the right PU?
> No, please don't.
>
> Development is the universe that is currently being updated with the
> latest stuff. Most things in there are tested against 3.9 and 3.10,
> since thats what developers are using, so it is the most maintained.

Sorry, but that's just not reflective of the facts at the moment.

The "Development" PU is stinky peee-yooo!  It's clearly not properly  
tested with even a minimum of attention to conflict avoidance.  It's  
full of old useless garbage (i.e. tons of unnecessary old versions of  
almost every package).  It's got some old junk that won't even work in  
any way at all on 3.10.  It's a worse mess than SqueakMap because  
there's no hint as to what's been actually tested against the release  
being used.

It is of no real use to any beginner -- it's just a way to get people  
to give up and go away.

> Since 3.10 is the current released image then it is in the Development
> Universes that package load dependencies stands the best chance of  
> being
> fixed.
>
> This is what is called a compromise, and we know what needs to be  
> done,
> and that is being done, just not in 3.10


So the compromise is to keep what outsiders see as the one true  
official release full of untested garbage?

I'd say that's more of an outright failure instead of a compromise.

You don't put new users on the bleeding edge.  That's just suicide.

If someone could tell me how to change my "Package Universe Browser"  
button to open a PU browser on the official 3.10 PU so that I could  
see what the official release PU should look like, and give it a go to  
see if I have any better luck with it, then maybe I could have a  
better opinion on whether or not it really would be better to set the  
default PU back to the 3.10 official release PU or not.

If the only workable compromise really is just to try to go forward  
then the best solution for helping new users in the immediate future  
is to do the following:

        1. remove the default "Package Universe Browser" button from the  
3.10.2 and subsequent patch release/updated images.

        2. tell new users (on the main squeak.org page and the downloads page  
and any and all relevant Swiki pages) that they can get all the fun  
stuff from some image like the FunSqueak image, but be careful to only  
recommend the most stable and well tested version, not the bleeding  
edge where just running the unit tests will core dump the whole VM.

        3. set up an email gateway so that the "mail report" button in the  
debugger actually goes somewhere where someone who can do something  
about PU problems will take notice and do something to actually make  
this forward progress.  If I (or anyone else in my position) want to  
help and I use the "mail report" button to try to help by reporting  
problems I encounter, and then as a result I'm told that what I did is  
useless because my report didn't go to the right place then I'm just  
as likely to go away and leave you guys to your own mess.

--
                                        Greg A. Woods; Planix, Inc.
                                        <[hidden email]>

Dear Greg,

some of us have spent a lot of time trying to improve things in work
that has been made available SINCE 3.10's release.

However you are basically ignoring what people are trying to tell you,
and insist on being
negative and non-constructive. So I wont be replying any further.

If you want to be constructive and help please try 3.10.2bc as mentioned
in a  recent email, and contribute to the package definitions in the
Packages-Library category there.

regards

Keith