Re: [ANN] 7061 = Squeak 3.9 final

On Sep 25, 2006, at 21:11 , Brad Fuller wrote:

> Brad Fuller wrote:
>> tim Rowledge wrote:
>>
>> Wouldn't the VM Crash, that I and others reported, be a reason not to
>> release until fixed? Mantis issue# 0005056
>>
>> brad
> No comment on my question?

I think, the current VM issues would be a good reason to pospone the  
release.

Unfortunately, as it seems to me, the VM is quite poorly maintained.  
Probably the main reason being that the maintainers don't have enough  
time (or, in other words, because they are not paid to fix the  
problems). Another difficulty is that there are only few people that  
actually work on the VM (why?).

Apart from the bug you mention (http://bugs.impara.de/view.php?
id=0005056) there are others. For example:
- http://bugs.impara.de/view.php?id=4709 (VM blocks after memory  
consumtion exceeds ~120MB)
- http://bugs.impara.de/view.php?id=4882 (VM lockup)

Cheers,
Adrian

PS: cc to vm-dev

On 25-Sep-06, at 12:33 PM, Adrian Lienhard wrote:

$ (or lack there of)

> Apart from the bug you mention (http://bugs.impara.de/view.php?
> id=0005056) there are others.

a pointer to memory is an unsigned value, not a signed integer. Tim  
will gladly accept lots of $$ to fix that.


> For example:
> - http://bugs.impara.de/view.php?id=4709 (VM blocks after memory  
> consumtion exceeds ~120MB)

I added a note about http://minnow.cc.gatech.edu/squeak/3710, since  
likely the problem is hitting the wall when attempting to grow the  
image.

> - http://bugs.impara.de/view.php?id=4882 (VM lockup)

See added notes about setVMStatsTraceMessageSendLevels:
The Mac carbon VM and unix VMs all share the same code for aioPoll  
and socket logic.
Since all the source code *is* available others are welcome to look  
at the logic and try to figure out what is going on.

>
> Cheers,
> Adrian
>
> PS: cc to vm-dev
>
>

--
========================================================================
===
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

Adrian Lienhard wrote:

> On Sep 25, 2006, at 21:11 , Brad Fuller wrote:

>> Brad Fuller wrote:
>>> tim Rowledge wrote:

>>> Wouldn't the VM Crash, that I and others reported, be a reason not to
>>> release until fixed? Mantis issue# 0005056

>>> brad
>> No comment on my question?

> I think, the current VM issues would be a good reason to pospone the
> release.

> Unfortunately, as it seems to me, the VM is quite poorly maintained.
> Probably the main reason being that the maintainers don't have enough
> time (or, in other words, because they are not paid to fix the
> problems). Another difficulty is that there are only few people that
> actually work on the VM (why?).

Here are my excuses:

1. Linux on my system currently has a major major bug that causes my
machine to be unusable for all but the most trivial tasks. linux blows
donkies.

2. The current Slang > C compiler is very hackish and behaves in very
counterintuitive ways, especially with regards to the VM.

3. It is very difficult to understand how the Slang compiler interacts
with the system compiler so it would be very very difficult for anyone
except the most elite Squeak programmer to work on it. -- One gripe I
have with the current (recient?) system compiler is that it doesn't
provide information about the scope of a variable (method context versus
block context)

4. Because of how the current compiler works, it is impossible to get a
working VM that isn't inlined and therefore doing normal code
coverage/profiling becomes many times more difficult.

5. Mere mortals don't have access to the current version. Any hacker
trying to get the "development" version can't (or at least couldn't last
time I checked). You COULD develop based off of what's on the website or
try to get the SVN source working (VERY HARD/IMPOSSIBLE -- used to be
very reliable), but then you'd have to completely re-do your changes
when you finally get a chance to sync with the mainline version. =(


--
If you tell a linux advocate that you saw Bigfoot, he'd believe you.
If you tell him that you saw linux crash, he won't.

Adrian Lienhard wrote:
> I think, the current VM issues would be a good reason to pospone the
> release.

I disagree. The issue in question is nothing new and it's unlikely to be
fixed any time soon. Mostly because we don't have a reliable way of
recreating (and therefore debugging) the problem.

> Unfortunately, as it seems to me, the VM is quite poorly maintained.

I don't think it's fair to draw this conclusion based on the sample size
you are using in particular given that you don't seem understand the
problem well enough to see why it's so hard to fix.

> Probably the main reason being that the maintainers don't have enough
> time (or, in other words, because they are not paid to fix the
> problems). Another difficulty is that there are only few people that
> actually work on the VM (why?).

Because it's hard. In this case, the problem is largely that you can't
debug this stuff in the simulator because the simulator still uses
Squeak semantics (i.e., unlimited integer arithmetic). For example, if
you have code saying "a + b < c ifTrue:[...]" then in C this may have
some very odd results (depending on the types of a, b, and c, as well as
their values) whereas in the simulator the result is always the same
(since we don't type variables).

> Apart from the bug you mention
> (http://bugs.impara.de/view.php?id=0005056) there are others. For example:
> - http://bugs.impara.de/view.php?id=4709 (VM blocks after memory
> consumtion exceeds ~120MB)

John has answered that one already. There is a simple application-level
workaround for it.

> - http://bugs.impara.de/view.php?id=4882 (VM lockup)

John has replied to that one as well (and not being a Unix guy I have
nothing to add).

Regards,
   - Andreas

Hi Andreas,

Sorry, I'm not familiar with the Squeak VM, but why does Slang translate
into untagged C arithmetic?  To ensure the same semantic subset, why can't
you generate C that uses tagged arithmetic that traps on overflow, rather
than enlarging?  Low-bit tags are not that slow!  You could still represent
up to 512MB with 2 tag bits and signed arithmetic.  And if you need more you
just generate 64bit tagged integers, which would be fast on all new
machines.

Then you should get the same semantics in Slang and in the simulator, modulo
traps.

-Dave

David Griswold wrote:
> Sorry, I'm not familiar with the Squeak VM, but why does Slang translate
> into untagged C arithmetic?  

Because Slang is a subset that has been specifically chosen for
translation to C. In other words it was choice made way back when the
first version of the translator was written.

> To ensure the same semantic subset, why can't
> you generate C that uses tagged arithmetic that traps on overflow, rather
> than enlarging?  Low-bit tags are not that slow!  You could still represent
> up to 512MB with 2 tag bits and signed arithmetic.  And if you need more you
> just generate 64bit tagged integers, which would be fast on all new
> machines.
>
> Then you should get the same semantics in Slang and in the simulator, modulo
> traps.

That would be a fun project (I wonder how different/similar it is to
Ian's statically compiled idst version) but nothing I'm too eager to
take on ;-)

Cheers,
   - Andreas

On 25-Sep-06, at 5:57 PM, David Griswold wrote:

>
> Sorry, I'm not familiar with the Squeak VM, but why does Slang  
> translate
> into untagged C arithmetic?
Dave, Slang _is_ the simulator effectively. It's just the Smalltalk  
code written in a pidgin sorta-C that can be executed to simulate the  
VM. Since it's just Smalltalk (albeit rather uglified Smalltalk) we  
just get to use the obvious integer overflow capabilities without  
thinking about it. Well, much. It does bop us on the nose  
occasionally when we forget that it will be turned into C.

Last case I can remember is when trying to make it possible to have  
64 bit filepointers but still manage on OSs that don't use them.

We use the bottom bit as the tag for SmallInt and leave the bottom  
two 0 for oops so there is no fiddling to extract addresses. There  
have been occasional attempts to use the second tag bit for an extra  
immediate but so far no change has been adopted. The latest VM code  
is (we think) clean for 64bit machines but a number of the vm plugins  
are not yet reworked and some platform specific code may still assume  
32bit machines.

I have to doubt that 64bit  integers would be fast on all new  
machines. A substantial subset of 'all new machines' use ARM  
processors and they're still 32bit...... :-)


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Attitudes are contagious. Mine might kill you

On Mon, Sep 25, 2006 at 09:33:08PM +0200, Adrian Lienhard wrote:
Unless there is some reason to believe that this issue is related to
the image, it should have no bearing on the question of whether the
Squeak 3.9 image should or should not be released. The fact that
someone happened to have been running a 3.9 image when they observed
the problem is probably irrelevant (but it would be helpful if someone
who is experiencing the problem could confirm or deny this by testing
for the bug on a 3.8 image).

> Unfortunately, as it seems to me, the VM is quite poorly maintained.
> Probably the main reason being that the maintainers don't have enough
> time (or, in other words, because they are not paid to fix the
> problems). Another difficulty is that there are only few people that
> actually work on the VM (why?).

Actually, it's kind of fun, and anybody with an interest in the topic
can do it. As for myself, none of the computers I have access to exhibit
the problem you are discussing, so it's not very far up the list of
itches I want to scratch. But if your system is crashing, you are in
an ideal position to have a look at it.

It's certainly possible that a bit of funding might lubricate the
maintenance process, but in the mean time there is nothing stopping
you or me or anybody else from working on any of these issues.
If you wait for "somebody else" to work on it, you may be waiting
a long time.

Folks, this is open source. Including the VM.

Dave

Hi Tim,

I understand Slang is designed for generation into C, my point is that
trapping tagged
arithmetic should be easy and fast to generate in C (if they have a tag of
0), so why not generate it?   And there is a big advantage because modulo a
trap happening, which signals design failure, you get exactly the same
semantics, so debugging the VM would be easier.  Why wouldn't that be a
better way?  You wouldn't get bopped on the nose so much.

Yes, but using 0 for the SmallInt tag bit is much better because then the
integer
arithmetic is very fast, and having an immediate constant offset
on object dereferencing shouldn't cost anything anyway, so there
is no "fiddling".

> I have to doubt that 64bit  integers would be fast on all new
> machines. A substantial subset of 'all new machines' use ARM
> processors and they're still 32bit...... :-)

That's a good point.  But as I said, on those platforms you still would
have up to 512MB object space even with 32bits.  And won't the ARM'ers be
jealous if they find out they are missing out on 64bit tagged
double-precision
floating point ;-)

Cheers,
Dave

Am 26.09.2006 um 00:28 schrieb Alan Grimes:

> 1. Linux on my system currently has a major major bug that causes my
> machine to be unusable for all but the most trivial tasks. linux blows
> donkies.

We know by now, and we could not care less. Please stop bitching  
about operating systems on Squeak lists. There are better places for  
that.

> 2. The current Slang > C compiler is very hackish and behaves in very
> counterintuitive ways, especially with regards to the VM.

Actually, the translation is quite straight-forward, in particular  
compared to similar translation projects in other languages. If there  
is something specific you want to know, just ask.

> 3. It is very difficult to understand how the Slang compiler interacts
> with the system compiler so it would be very very difficult for anyone
> except the most elite Squeak programmer to work on it. -- One gripe I
> have with the current (recient?) system compiler is that it doesn't
> provide information about the scope of a variable (method context  
> versus
> block context)

What do you mean by "system compiler"? The (old) Squeak compiler? It  
indeed does handle block-local variables as method temps. Why is this  
a problem for Slang?

> 4. Because of how the current compiler works, it is impossible to  
> get a
> working VM that isn't inlined and therefore doing normal code
> coverage/profiling becomes many times more difficult.

That would be a bug, I think, but it would not be high on the  
priority list. Would be a nice little project to clean that up I guess.

> 5. Mere mortals don't have access to the current version. Any hacker
> trying to get the "development" version can't (or at least couldn't  
> last
> time I checked). You COULD develop based off of what's on the  
> website or
> try to get the SVN source working (VERY HARD/IMPOSSIBLE -- used to be
> very reliable), but then you'd have to completely re-do your changes
> when you finally get a chance to sync with the mainline version. =(

Not true anymore, at least for the Unix VM. SVN head works perfectly  
for me.

- Bert -