[ANN] 7061 = Squeak 3.9 final

Adrian Lienhard wrote:

> On Sep 25, 2006, at 21:11 , Brad Fuller wrote:

>> Brad Fuller wrote:
>>> tim Rowledge wrote:

>>> Wouldn't the VM Crash, that I and others reported, be a reason not to
>>> release until fixed? Mantis issue# 0005056

>>> brad
>> No comment on my question?

> I think, the current VM issues would be a good reason to pospone the
> release.

> Unfortunately, as it seems to me, the VM is quite poorly maintained.
> Probably the main reason being that the maintainers don't have enough
> time (or, in other words, because they are not paid to fix the
> problems). Another difficulty is that there are only few people that
> actually work on the VM (why?).

Here are my excuses:

1. Linux on my system currently has a major major bug that causes my
machine to be unusable for all but the most trivial tasks. linux blows
donkies.

2. The current Slang > C compiler is very hackish and behaves in very
counterintuitive ways, especially with regards to the VM.

3. It is very difficult to understand how the Slang compiler interacts
with the system compiler so it would be very very difficult for anyone
except the most elite Squeak programmer to work on it. -- One gripe I
have with the current (recient?) system compiler is that it doesn't
provide information about the scope of a variable (method context versus
block context)

4. Because of how the current compiler works, it is impossible to get a
working VM that isn't inlined and therefore doing normal code
coverage/profiling becomes many times more difficult.

5. Mere mortals don't have access to the current version. Any hacker
trying to get the "development" version can't (or at least couldn't last
time I checked). You COULD develop based off of what's on the website or
try to get the SVN source working (VERY HARD/IMPOSSIBLE -- used to be
very reliable), but then you'd have to completely re-do your changes
when you finally get a chance to sync with the mainline version. =(


--
If you tell a linux advocate that you saw Bigfoot, he'd believe you.
If you tell him that you saw linux crash, he won't.

Adrian Lienhard wrote:
> I think, the current VM issues would be a good reason to pospone the
> release.

I disagree. The issue in question is nothing new and it's unlikely to be
fixed any time soon. Mostly because we don't have a reliable way of
recreating (and therefore debugging) the problem.

> Unfortunately, as it seems to me, the VM is quite poorly maintained.

I don't think it's fair to draw this conclusion based on the sample size
you are using in particular given that you don't seem understand the
problem well enough to see why it's so hard to fix.

> Probably the main reason being that the maintainers don't have enough
> time (or, in other words, because they are not paid to fix the
> problems). Another difficulty is that there are only few people that
> actually work on the VM (why?).

Because it's hard. In this case, the problem is largely that you can't
debug this stuff in the simulator because the simulator still uses
Squeak semantics (i.e., unlimited integer arithmetic). For example, if
you have code saying "a + b < c ifTrue:[...]" then in C this may have
some very odd results (depending on the types of a, b, and c, as well as
their values) whereas in the simulator the result is always the same
(since we don't type variables).

> Apart from the bug you mention
> (http://bugs.impara.de/view.php?id=0005056) there are others. For example:
> - http://bugs.impara.de/view.php?id=4709 (VM blocks after memory
> consumtion exceeds ~120MB)

John has answered that one already. There is a simple application-level
workaround for it.

> - http://bugs.impara.de/view.php?id=4882 (VM lockup)

John has replied to that one as well (and not being a Unix guy I have
nothing to add).

Regards,
   - Andreas

Well I've just downloaded this and I'm very disappointed. I  
appreciate how much work is involved and how many sleepless nights it  
must have caused Marcus and Stef but....

This simply isn't an image to release to the general public. The two  
Workspaces need to be edited and de-typo'd.

There are now two preferences tools, neither of which handle font  
settings or display setings. If one is going to replace an old tool  
with a new one it ought to noticeably improve upon it. And, of  
course, actually *replace* it.

Why is a package of omnibrowser included but not, so far as I can  
tell, hooked up to use? Not to mention it appears to be non-
functional; on my machine (a very fast dualcore mac) I opened an  
OBSystemBrowser with #openOnClass: Object, pressed the 'hierarchy'  
button and got bored after more than a minute of waiting for a new  
browser.

What are
        Morphic-Models
        39Deprecated
        FlexibleVocabularies-Info
        ScriptLoader
there for?

It may be a good start but it really isn't ready to be a new release.  
Just pushing something out because you're bored or tired or along  
time has passed is not sensible. All it does is make more, nastier,  
work later.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
29A, the hexadecimal of the Beast.

On 2006 September 25 17:26, Giovanni Corriga wrote:
> Il giorno lun, 25/09/2006 alle 13.42 -0700, Brad Fuller ha scritto:
> > stephane ducasse wrote:
<<snip>>
> >
> > The code is in the mantis defect that is 100% reproducible on my PC. I
> > think it's easy to duplicate or make other PCs crash.
>
> I can reproduce it on my system, (Fedora Core 5, 3.9-7 vm, 3.8 image).

Guys,

Could you paste the exact output of "squeak -version"? - The reason I am
saying it is that this _does_ work on my SuSE 9.3. I think there are multiple
3.9-7 VMs, and we already saw significant performance difference between 3.9
VM version built on RedHat (i think) and Ian's version (on another thread
this week named something like Squeak is slooow):

My VM version:
$ squeak -version
3.9-7 #5 Mon Apr 24 20:07:28 PDT 2006 gcc 3.3.5
Squeak3.9alpha of 4 July 2005 [latest update: #7021]
Linux vps.piumarta.com 2.4.20-021stab028.18.777-enterprise #1 SMP Wed Sep 14
19:34:46 MSD 2005 i686 GNU/Linux
default plugin location: /usr/local/lib/squeak/3.9-7/*.so

| a b |
 a := FloatArray new: (16 * 1024*1024).
 b := FloatArray new: (16 * 1024*1024).
 [a += b] timeToRun. "runs OK in 270 ms"

So it may be completely reproducible on one system, but not all - I have no
idea, but maybe the exact VM version will give some clue,

Milan

On Mon, Sep 25, 2006 at 09:33:08PM +0200, Adrian Lienhard wrote:
Unless there is some reason to believe that this issue is related to
the image, it should have no bearing on the question of whether the
Squeak 3.9 image should or should not be released. The fact that
someone happened to have been running a 3.9 image when they observed
the problem is probably irrelevant (but it would be helpful if someone
who is experiencing the problem could confirm or deny this by testing
for the bug on a 3.8 image).

> Unfortunately, as it seems to me, the VM is quite poorly maintained.
> Probably the main reason being that the maintainers don't have enough
> time (or, in other words, because they are not paid to fix the
> problems). Another difficulty is that there are only few people that
> actually work on the VM (why?).

Actually, it's kind of fun, and anybody with an interest in the topic
can do it. As for myself, none of the computers I have access to exhibit
the problem you are discussing, so it's not very far up the list of
itches I want to scratch. But if your system is crashing, you are in
an ideal position to have a look at it.

It's certainly possible that a bit of funding might lubricate the
maintenance process, but in the mean time there is nothing stopping
you or me or anybody else from working on any of these issues.
If you wait for "somebody else" to work on it, you may be waiting
a long time.

Folks, this is open source. Including the VM.

Dave

2006/9/25, [hidden email] <[hidden email]>:
Pick one. If none of the other shows up, you can be sure an EOCD error
pops up. Then I usally download and install the file manually.
I don't know why SqueakMap can't do that for me.

Philippe

> I am aware of a few current bugs or "inconveniences" but generally I
> think SM works ok in 3.8+.
>
> regards, Göran
>
>

tim Rowledge a écrit :
Yes you are right, this is not a polished version, but i see more this
version as a good start for Squeak distributions (squeak-land,
small-land, s...) who could add more bells and whisles (documentation,
specific examples and tools, ...). This is not for end users. No end
users download the last Linux kernel, only developpers or real hackers
would like to do that.

Look for example as the excellent work of Damien with the squeak-dev
image for developers here :
http://damiencassou.seasidehosting.st/seaside/pier

--                                                         oooo
Dr. Serge Stinckwich                                     OOOOOOOO
Université de Caen>CNRS UMR 6072>GREYC>MAD               OOESUGOO
http://purl.org/net/SergeStinckwich                       oooooo
Smalltalkers do: [:it | All with: Class, (And love: it)]   \  /
                                                             ##

Hi tim

This is good that you woke up. Luckily we were not looking for your  
feedback earlier.!
Sorry but you really pissed me off.

So if you want to fix what you think are the problems, please do.
Because from my side this is done. Now if you want to do RC2 you are  
my guest.

Stef

On 26 sept. 06, at 01:22, tim Rowledge wrote:

My point is simple I take responsibility for what I know I can do or  
try to do and I do "impose" (because noone can impose
work on other) any stuff on others.

So sometimes I have argument with andreas and this is normal because  
we have some strong views on the world
but from the VM perspective I respect the guys doing the job there.  
So if they do not shout and block the release
then this is ok for me. Else I should be smart enough to really  
contribute.

Stef


On 25 sept. 06, at 22:42, Brad Fuller wrote:

On 26.09.2006, at 01:22, tim Rowledge wrote:

> Well I've just downloaded this and I'm very disappointed. I  
> appreciate how much work is involved and how many sleepless nights  
> it must have caused Marcus and Stef but....

If those things are the one you find to be very disappointed, I take  
it as a compliment: The changes in 3.9 are so huge that I, honestly,  
am happy if these
are the problems you find!

> This simply isn't an image to release to the general public. The  
> two Workspaces need to be edited and de-typo'd.
>

Please do.

> There are now two prefer
> ences tools, neither of which handle font settings or display  
> setings. If one is going to replace an old tool with a new one it  
> ought to noticeably improve upon it. And, of course, actually  
> *replace* it.

Please submit a patch. I once looked for some time into removing the  
old Preferences. Sadly the code is so bad that even removing it is *a  
lot* of work. The new
preferences are an improvement on the old ones, so I fail to see why  
we should not have added them. Isn't this the old Squeak pattern of  
"doing nothing because
something more perfect is thinkable" at work?

The new Preferences e.g. integrate the HTTP-Proxy preference. That  
alone makes them worth *a lot*. No code to be executed for proxy  
settings...

> Why is a package of omnibrowser included but not, so far as I can  
> tell, hooked up to use? Not to mention it appears to be non-
> functional; on my machine (a very fast dualcore mac) I opened an  
> OBSystemBrowser with #openOnClass: Object, pressed the 'hierarchy'  
> button and got bored after more than a minute of waiting for a new  
> browser.
>

It is not default because it's new. If people start to use it more,  
fix bugs, integrate it a bit more, then we can remove the old one.  
Don't you think you would be more
pissed if there would only be OB then the state now? People do use  
OmniBrowser, though. The way to do it is to set it as the Default  
Browser (see window
menu of the Browser. I works quite well, but OB needs to be used a  
lot more before we can remove the old Browser.

> What are
> Morphic-Models
> 39Deprecated
> FlexibleVocabularies-Info

This is a package from Connectors that was a pre-requisit for adding  
the squeakland changesets. I did once take the time to rename all  
categories
(as this package makes no sense, it's a patch that Ned managed as an  
mcz package). But at some point it crept back in (I think the Morpic  
Teams
big first "make morphic to packages" changes, or even maybe one of  
the >150 SqueakLand changesets).

Someone should have re-invested the time to move all methods from  
this package into the morphic packages. I didn't, you did not. So is  
it my fault?

Morphic-Models is an auto-generated category by some strange code  
that should be removed.

39Deprecated contains all methods that where deprecated in 3.9a, it's  
supposed to be there in 3.9 and removed in 3.10a


> ScriptLoader

Scriptloader contains the machinary to update from the repository.  
This was how it was done in 3.9, for 3.10, I think people will
again have a look at the tools that Bert did at Impara and retire  
Scriptloader. So why should this be  removed from 3.9? Don't
we want to do a 3.9.1?

     Marcus

Il giorno lun, 25/09/2006 alle 20.13 -0400, Milan Zimmermann ha scritto:
Ok, here's mine:

[gcorriga@rincewind ~]$ squeak -version
3.9-7 #1 Thu May 18 12:06:33 CEST 2006 gcc 4.1.0
Squeak3.9alpha of 4 July 2005 [latest update: #7021]
Linux rincewind 2.6.16-1.2111_FC5.stk16 #1 Mon May 8 10:49:23 EDT 2006
i686 i686 i386 GNU/Linux
default plugin location: /usr/lib/squeak/3.9-7/*.so

could this be related to the use of gcc 4.x?

        Giovanni

Hi!

"Philippe Marschall" <[hidden email]> wrote:
Pick one? What do you mean?

> If none of the other shows up, you can be sure an EOCD error
> pops up.

Yes, this is an unfortunate situation that I need to go through. The
problem is that after I introduced the server side cache and SHA
checksumming apparently lots of releases either were non-available at
the time I initially populated the cache (which lead to crap files in
the server cache, and to make things worse - crappy checksums!) OR have
been unavailable when releases have been registered. I think.

At least that is the current state. If you open a transcript when trying
to install/download something and it complains about the checksum and
"trying server cache" then there may very well be a problem. It compared
the faulty checksum with the correct (presumably) data from the original
URL and then decides to use the server side cache instead which contains
faulty data. Not very helpful ;).

> Then I usally download and install the file manually.
> I don't know why SqueakMap can't do that for me.

Because there is no such menu choice today (one that ignores the SHA
checksum and just downloads anyway). I can add one though.

The real medicine is probably to clean the server side cache and/or
figure out why it got filled with bad data in the first place.

And we could also thirdly add a button somewhere in the map web UI to
"repopulate server cache and recalculcate SHA checksum". That is done
automatically whenever you modify the URL of a release today - but it
could be helpful to have such a button to "repair" entries.

> Philippe

regards, Göran

Am 26.09.2006 um 00:28 schrieb Alan Grimes:

> 1. Linux on my system currently has a major major bug that causes my
> machine to be unusable for all but the most trivial tasks. linux blows
> donkies.

We know by now, and we could not care less. Please stop bitching  
about operating systems on Squeak lists. There are better places for  
that.

> 2. The current Slang > C compiler is very hackish and behaves in very
> counterintuitive ways, especially with regards to the VM.

Actually, the translation is quite straight-forward, in particular  
compared to similar translation projects in other languages. If there  
is something specific you want to know, just ask.

> 3. It is very difficult to understand how the Slang compiler interacts
> with the system compiler so it would be very very difficult for anyone
> except the most elite Squeak programmer to work on it. -- One gripe I
> have with the current (recient?) system compiler is that it doesn't
> provide information about the scope of a variable (method context  
> versus
> block context)

What do you mean by "system compiler"? The (old) Squeak compiler? It  
indeed does handle block-local variables as method temps. Why is this  
a problem for Slang?

> 4. Because of how the current compiler works, it is impossible to  
> get a
> working VM that isn't inlined and therefore doing normal code
> coverage/profiling becomes many times more difficult.

That would be a bug, I think, but it would not be high on the  
priority list. Would be a nice little project to clean that up I guess.

> 5. Mere mortals don't have access to the current version. Any hacker
> trying to get the "development" version can't (or at least couldn't  
> last
> time I checked). You COULD develop based off of what's on the  
> website or
> try to get the SVN source working (VERY HARD/IMPOSSIBLE -- used to be
> very reliable), but then you'd have to completely re-do your changes
> when you finally get a chance to sync with the mainline version. =(

Not true anymore, at least for the Unix VM. SVN head works perfectly  
for me.

- Bert -

Am 26.09.2006 um 10:23 schrieb Giovanni Corriga:

AFAIK it's not related to the Squeak binary at all but just depends  
on the system. The problem started to pop up only recently (a year or  
maybe two ago). Maybe it's kernel 2.6 related or something else that  
changed in Linux recently - or simply that people have more than 2 GB  
virtual memory nowadays.

The problem always occurs when Squeak is loaded partially around the  
2 GB limit. The fix for the problem at hand would be a code audit in  
the FloatArray plugin, converting ints to unsigned where necessary.

- Bert -

On 2006 September 26 04:23, Giovanni Corriga wrote:
I think Bert said in another reply, this problem is likely due to the kernel
and/or memory over 2Gb. On this system (where it works) I have 1Gb memory and
something like 2Gb swap, my kernel=vmlinuz-2.6.11.4-21.13 so I guess it makes
sense it works here from memory perspectivenot sure about 2.6 kernel.
Giovanni, do you have over 2GB?

Milan

>
> Giovanni

Hi,

> AFAIK it's not related to the Squeak binary at all but just depends  
> on the system. The problem started to pop up only recently (a year or  
> maybe two ago). Maybe it's kernel 2.6 related or something else that  
> changed in Linux recently - or simply that people have more than 2 GB  
> virtual memory nowadays.
>
> The problem always occurs when Squeak is loaded partially around the  
> 2 GB limit. The fix for the problem at hand would be a code audit in  
> the FloatArray plugin, converting ints to unsigned where necessary.


I'm being working around this problem since months with the parrmeter:


 -mmap 512M


Cheers,

-- Diego

-mmap size[mk]
     requests that a variable heap of at most size bytes be  
allocated. (The suffixes are as described for the '-memory' option.)  
squeak will initially allocate a heap that is large enough to hold  
the image, with a small amount of headroom. If at any time Squeak  
requires more memory for its image then additional space will be  
allocated dynamically. Likewise, when memory is no longer needed it  
will deallocated and returned to the system. The size argument places  
an upper limit on how big the heap can grow in this fashion. squeak  
uses a dynamic heap by default with the maximum size set to 75% of  
the available virtual memory or 1 gigabyte, whichever is smaller.
-------------

In my past experiences which certainly depends on the operating  
system/version viewpoint, I found that on os-x setting the mmap size  
to 1GB gave a much higher probability that the memory mapped would  
start at or cross the 2GB boundary.  Which is why out of the box the  
carbon VM is set to mmap at 512MB.

On 26-Sep-06, at 6:59 AM, Diego Gomez Deck wrote:

--
========================================================================
===
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

On 26-Sep-06, at 12:45 AM, Marcus Denker wrote:

Exactly; that's how you *should* see it. You've done huge amounts of  
work and deserve the firm pat on the back that I would love to give  
you. I've done this same job many times and I know how painful it is.

>
>> This simply isn't an image to release to the general public. The  
>> two Workspaces need to be edited and de-typo'd.
>>
>
> Please do.

I'll give it a go as soon as I can make time.

It can end up seeming that way. We could at least remove user  
confusion by simply removing the old preferences tool from easy  
access and thereby make it seem to disappear.

>
> The new Preferences e.g. integrate the HTTP-Proxy preference. That  
> alone makes them worth *a lot*. No code to be executed for proxy  
> settings...

Good indeed. So I claim that it is worth thinking very hard about  
trying to make the new tool more complete before final release; we  
could surely remove a lot of new and recently-new user confusion by  
having all the settings available in a single place.

OK; introducing a new browser system is all very well but if I had  
been able to take much part recently I would have suggested that it  
ought to have been kept as a package until it is at least close to  
being a good replacement. If it *is* to be included it really ought  
to be easy enough to access that there is a good chance of people  
trying it out. Buried in a menu that I bet a substantial fraction of  
people don't ever look at, named 'OBSystemBrowserAdaptor' is not  
going to attract a lot of testers.


No, it's our collective fault combined with the depressing reality of  
time pressure. But explanations don't alter the facts (unless you're  
a member of some religion or other) and we have to try to work with  
the facts as much as possible.

>
> Morphic-Models is an auto-generated category by some strange code  
> that should be removed.

In the image I'm looking at, the category 'Morphic-Models' includes a  
single class that has no methods at all, which makes me think that it  
ought not exist. It does however have a single instance (a  
MorphicModle1(2710) which is pointed to by a PasteUpMorph(1622) which  
seems to claim to be the world. This seems like some nasty crap left  
over from ancient times, maybe?
>
> 39Deprecated contains all methods that where deprecated in 3.9a,  
> it's supposed to be there in 3.9 and removed in 3.10a

Sigh, yes, we still have that horrible class. Oh well.
>
>
>> ScriptLoader
>
> Scriptloader contains the machinary to update from the repository.  
> This was how it was done in 3.9, for 3.10, I think people will
> again have a look at the tools that Bert did at Impara and retire  
> Scriptloader. So why should this be  removed from 3.9?

What I forgot was to point out that at least partly it is the  
category names that cause potential confusion. Nit-picking I suppose  
but I'd suggest at least renaming to 'System-ScriptLoader', 'System-
DeprecatedIn39' (and probably move the class MorphicModel1 into it),  
'Tools-PreferenceBuilder', 'Tools-ReleaseBuilder', 'PackageInfo-
FlexibleVocabularies' etc.

Well, that was fun; I just went through to make a changeset for you  
with those category changes and the changeset is empty. How nice,  
work lost. I'll try again and see if does it again....


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
There can never be a computer language in which you cannot write a  
bad program.

Hi goran

I know that this is a responsibility to be the father of a success. ;)
But indeed this would be good if we could get SM working full speed.
So if you could allocate some time on that this would be great.

Stef

On 26-Sep-06, at 12:09 PM, tim Rowledge wrote:


>
> Well, that was fun; I just went through to make a changeset for you  
> with those category changes and the changeset is empty. How nice,  
> work lost. I'll try again and see if does it again....
Yup, editing categories and even editing the system organisation  
results in no input to the changeset. Ooh, even back to 3.8-6665.  
What fun. And 3.7. And 3.6. Well, that's puzzling since I was pretty  
sure we've filed out SystemOrganization edits before now. And it  
doesn't even get into the changes log either. Very odd.


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Hex dump:  Where witches put used curses...