An issue with Slang, the interpreter & the VM, and a period in biasToGrow.

I was chasing a problem with Sophie where the VM would suddenly grow  
by 800MB. In looking I discovered a problem with how Slang interprets
the following code.

Background: When an incremental garbage collection is done and the  
free space is less than the grow head room and the bias to grow is  
turned on,
then we call biasToGrow.  I'll note normally the gcBiasToGrow would  
be false unless you turn it on so few users have seen this problem.

The problem is that Slang converts the "growSize :=  growHeadroom*3/2  
- (self sizeOfFree: freeBlock)" to nothing. In the past I know this
worked because I had worked on choosing the magic 3/2 number.

ObjectMemory>>incrementalGC

...

                        (((self sizeOfFree: freeBlock) < growHeadroom) and:
                                [gcBiasToGrow > 0])
                                ifTrue: [self biasToGrow.
                                                weDidGrow := true].
                        youngStart := freeBlock].

...


ObjectMemory>>biasToGrow
        | growSize |
        growSize :=  growHeadroom*3/2 - (self sizeOfFree: freeBlock)
        self growObjectMemory: growSize



ObjectMemory>>growObjectMemory: delta
        "Attempt to grow the object memory by the given delta
        amount "
        | limit |
        statGrowMemory := statGrowMemory + 1.
        limit := self sqGrowMemory: memoryLimit By: delta.
        limit = memoryLimit
                ifFalse: [memoryLimit := limit - 24.
                        "remove a tad for safety"
                        self initializeMemoryFirstFree: freeBlock]


---------------------------------------------------

But the C code that Slang generates looks like so:


                if ((((longAt(foo->freeBlock)) & AllButTypeMask) < foo-
 >growHeadroom) && (foo->gcBiasToGrow > 0)) {
                        /* begin biasToGrow */
                        /* begin growObjectMemory: */
                        foo->statGrowMemory += 1;
                        limit = sqGrowMemoryBy(foo->memoryLimit, growSize);
                        if (!(limit == foo->memoryLimit)) {
                                foo->memoryLimit = limit - 24;
                                initializeMemoryFirstFree(foo->freeBlock);
                        }
                        weDidGrow = 1;
                }

The C compiler is happy (mostly) and depending on what it sticks in  
the growSize memory location, perhaps it's perhaps zero, perhaps not,  
we might as I did some odd behaviour.

How clever, and if you've come this far, you've like seen the missing  
period. at the end of
        "growSize :=  growHeadroom*3/2 - (self sizeOfFree: freeBlock)"

Helpfully Slang doesn't complaint about the missing period and forges  
onward creating in-valid C code, without the proper meaning.
Hopefully people writing Slang code are muttering, gee I wonder if I  
have some code that isn't quite what I think it is?


With the period of course the code becomes

                if ((((longAt(foo->freeBlock)) & AllButTypeMask) < foo-
 >growHeadroom) && (foo->gcBiasToGrow > 0)) {
                        /* begin biasToGrow */
                        growSize = (((sqInt) (foo->growHeadroom * 3) >> 1)) - ((longAt(foo-
 >freeBlock)) & AllButTypeMask);
                        /* begin growObjectMemory: */
                        foo->statGrowMemory += 1;
                        limit = sqGrowMemoryBy(foo->memoryLimit, growSize);
                        if (!(limit == foo->memoryLimit)) {
                                foo->memoryLimit = limit - 24;
                                initializeMemoryFirstFree(foo->freeBlock);
                        }
                        weDidGrow = 1;
                }

Mostly likely in working with Tim I managed to copy/paste some code  
and missed the '.'

Lastly I'll note the biasToGrow logic is there because there is a  
problem when Squeak reaches the boundary point where it must decide  
to grow, before doing that it does an incremental GC, the issue is  
that if the incremental GC can recover just enough few bytes the grow  
will not happen, but we will do another GC really really soon, thus  
you see Squeak at 100% CPU in the GC logic, yet not quite growing, at  
some point you will go over the boundary and grow, solving the problem.

The bias to grow logic then ensures the VM will grow in increments up  
to a certain threshold before forcing a full GC, thus avoids the  
particular problem.

--
========================================================================
===
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

Am Jan 23, 2007 um 6:15  schrieb John M McIntosh:

Ugh. Evil slang.

So the fix would be that the receiver of a keyword message must be a  
variable in Slang, and if it is not, we should complain? Or, make it  
translate to the actual equivalent C code, calling growObjectMemory  
with two arguments, the first of which would be

        self(growHeadroom*3/2-sizeOfFree(freeBlock))

so this becomes

        growSize = growObjectMemory(self(growHeadroom*3/2-sizeOfFree
(freeBlock)), growSize)

Hrmm. For C code that looks even sensible. I guess comnplaining is  
way safer.

> Hopefully people writing Slang code are muttering, gee I wonder if  
> I have some code that isn't quite what I think it is?

Nah, couldn't ever happen to me. Ever. Err. (... goes looking ...)

- Bert -

On Tue, Jan 23, 2007 at 11:36:16AM +0100, Bert Freudenberg wrote:
Um, wait a minute. The Smalltalk compiler does not complain about it,
so why would we expect the Slang translator to complain about it? It's
perfectly valid Smalltalk, and it does not blow up until you run it
in Squeak.

Wasn't being able to write and test the code in Squeak sort of the
whole point of Slang in the first place?

Another thing that might help is to have some simple way to show
generated C code for a method, perhaps right in a browser.

Dave

On Tue, 23 Jan 2007 05:59:48 -0500, "David T. Lewis" <[hidden email]>
wrote:

> Another thing that might help is to have some simple way to show
> generated C code for a method, perhaps right in a browser.

With Pic/Smalltalk, which is sorta-like slang (but for PIC assembler), I
added this capability - you can generate the PIC assembler for any method
from the browser, and it pops a workspace with the assembler code. You
can't do anything useful with that code, but it is good to look at to
ensure it is doing what you think it should be doing.

Later,
Jon

--------------------------------------------------------------
   Jon Hylands      [hidden email]      http://www.huv.com/jon

  Project: Micro Raptor (Small Biped Velociraptor Robot)
           http://www.huv.com/blog

David T. Lewis wrote:
> Um, wait a minute. The Smalltalk compiler does not complain about it,
> so why would we expect the Slang translator to complain about it? It's
> perfectly valid Smalltalk, and it does not blow up until you run it
> in Squeak.

Exactly - my guess is that if this code had ever been executed it would
have failed with a "does not understand #self" error. So I don't see
this system as broken (perhaps very fragile, though).

-- Jecel

Am Jan 23, 2007 um 14:12  schrieb Jecel Assumpcao Jr:

It is broken because Slang does *not* produce equivalent C code. So  
either we need to fix Slang to actually generate the right code, or  
make it throw up an error. *Silently* dropping an expression is  
unacceptable.

As for "ever been executed" ... For one reason or the other, most  
Slang hackers I know rarely use the simulation. Well, actually it  
depends. Like, core VM stuff, nobody simulates, it's just too slow to  
be interesting. Well perhaps except for Craig with his super-tiny  
image. With Slang-only plugins I have indeed used it (e.g., I wrote a  
TweakPlugin that speeds up a basic operation). But as soon as you are  
interfacing to an external library, simulation is next to useless.

Your point is valid in so far that *had* it ever been executed it  
would have been caught easily. But this requires a correct translation.

- Bert -

Hi Bert--

> As for "ever been executed" ... For one reason or the other, most
> Slang hackers I know rarely use the simulation.

     Wow, how many Slang hackers do you know? ;)

> Like, core VM stuff, nobody simulates, it's just too slow to be
> interesting.

     Sometimes simulation is still the fastest way to isolate a bug.
It's got to take about a day before I'll give up on it. :)

> With Slang-only plugins I have indeed used it (e.g., I wrote a
> TweakPlugin that speeds up a basic operation). But as soon as you are
> interfacing to an external library, simulation is next to useless.

     Well, if a certain core of your plugin's primitives are known to
work in their translated form, you can extend the simulation to invoke
them, and go on to use the simulator to very useful effect (isolating
bugs which may not be directly related to the plugin, but which require
the plugin's functionality to duplicate).

     For example, I sometimes need to isolate subtle ordering bugs in
remote message-sending between multiple systems. Simulation seems to be
the only way to catch them, and that requires the ability to do
networking during simulation. The Flow networking plugin's C code is
entirely handwritten (for now), with system calls all over the place,
but I also wrote a Smalltalk simulation harness for it so everything
works. BitBltSimulator is another example.


-C

--
Craig Latta
http://netjam.org/resume

On Tue, Jan 23, 2007 at 06:54:43AM -0500, Jon Hylands wrote: The attached change set adds #asCString and #asCString: for object memory,
interpreter, and plugin classes.

For example, in a browser source window for ObjectMemory>>growObjectMemory:,
you can inspect the generated C source code by evaluating:
  "self asCString: #growObjectMemory:"

To look at the C code for a plugin method:
  "OSProcessPlugin concreteClass asCString: #forkSqueak:"

I did not try to incorporate this into any of the browsers.

Dave