[squeak-dev] Getting back to push/pop remappable oop

Hi list,
>From previous discussion, where Eliot mentioned about removal push/pop
remappable oop oddity and modifying GC to make it work w/o it, i'm
just thinking
- there was no need in having these methods from the very start.
Any oop at any time can be temporary pushed on stack and then popped
back again using #pop:/#push: methods.
Of course, primitives need to be careful to not override the
arguments, which can be reused in method when primitive fails - so
they should push values past the arguments.
But in most cases, a primitive creates new object(s) at the point,
where it needs to form a returned value - at this point most things is
done: arguments is read & already popped out of stack, values
calculated etc etc. So it is safe to put temporary values on stack,
making GC happy and without need in use push/popRemmappableOop.

Going in this way, there is no need in adding safety schemes, like
reserving extra free space and delaying GC before primitive finishes
working.

--
Best regards,
Igor Stasenko AKA sig.

Igor Stasenko wrote:
> - there was no need in having these methods from the very start.
> Any oop at any time can be temporary pushed on stack and then popped
> back again using #pop:/#push: methods.
> Of course, primitives need to be careful to not override the
> arguments, which can be reused in method when primitive fails - so
> they should push values past the arguments.

This won't work. Stack frames are limited in size and sized in the image
based on however much space is required by the byte codes without any
regards of potential additional space needed by primitives (which is
variable). And, since primitives are executed before activation (meaning
that the "primitive stack" would have to share the frame with the caller
method) every frame would have to reserve the maximum additional stack
space any single primitive might require.

Cheers,
   - Andreas

2008/11/11 Andreas Raab <[hidden email]>:
Okay, but what about reusing the arguments stack slots?
In most cases, when primitive generating response its:
- already checked everything (can't fail)
- already evaluated results
- arguments no more in use

And , since primitive should return only single object, its easy to
prove, that it requires MAXIMUM SINGLE temp value on stack, which
needs to be remapped.

(in primitiveSoundGetVolume)
       ....
        interpreterProxy pushRemappableOop: (right asOop: Float).
        interpreterProxy pushRemappableOop: (left asOop: Float).
        interpreterProxy pushRemappableOop: (interpreterProxy
instantiateClass: (interpreterProxy classArray) indexableSize: 2).
        results := interpreterProxy popRemappableOop.
        interpreterProxy storePointer: 0 ofObject: results withValue:
interpreterProxy popRemappableOop.
        interpreterProxy storePointer: 1 ofObject: results withValue:
interpreterProxy popRemappableOop.
do:

| oop |
...
interpreterProxy pop: 1. "pop the receiver from stack"
interpreterProxy push: (interpreterProxy instantiateClass:
(interpreterProxy classArray) indexableSize: 2).
oop :=  right asOop: Float. "may cause GC"
interpreterProxy storePointer: 0 ofObject: (interpreterProxy stackTop)
withValue: oop.
oop :=  left asOop: Float. "may cause GC"
interpreterProxy storePointer: 1 ofObject: (interpreterProxy stackTop)
withValue: oop.
"result already on stack.. nothing else to do"


same in BalloonEngineBase>>primitiveGetClipRect
 ...
        interpreterProxy pushRemappableOop: rectOop.
        pointOop := interpreterProxy makePointwithxValue: self clipMinXGet
yValue: self clipMinYGet.
        rectOop := interpreterProxy popRemappableOop.
        interpreterProxy storePointer: 0 ofObject: rectOop withValue: pointOop.
        interpreterProxy pushRemappableOop: rectOop.
        pointOop := interpreterProxy makePointwithxValue: self clipMaxXGet
yValue: self clipMaxYGet.
        rectOop := interpreterProxy popRemappableOop.
        interpreterProxy storePointer: 1 ofObject: rectOop withValue: pointOop.

        interpreterProxy pop: 2.
        interpreterProxy push: rectOop.

rewrite:
...

        interpreterProxy pop: 2.
        interpreterProxy push: rectOop.

        pointOop := interpreterProxy makePointwithxValue: self clipMinXGet
yValue: self clipMinYGet.
        interpreterProxy storePointer: 0 ofObject: (interpreterProxy
stackTop) withValue: pointOop.
        pointOop := interpreterProxy makePointwithxValue: self clipMaxXGet
yValue: self clipMaxYGet.
        interpreterProxy storePointer: 1 ofObject: (interpreterProxy
stackTop) withValue: pointOop.

see? its even less lines of code :)

> Cheers,
>  - Andreas

--
Best regards,
Igor Stasenko AKA sig.

Igor Stasenko wrote:
> Okay, but what about reusing the arguments stack slots?

And what if there are none? There are tons of primitives returning
complex objects (arrays, rectangles) that take no arguments.

> In most cases, when primitive generating response its:
> - already checked everything (can't fail)

Primitives can fail for other reasons than argument mismatches.

> - already evaluated results
> - arguments no more in use

> And , since primitive should return only single object, its easy to
> prove, that it requires MAXIMUM SINGLE temp value on stack, which
> needs to be remapped.

Not true either. ObjectMemory>>allocate: remaps the classOop so a depth
of AT LEAST 2 is required even for your example. There are almost
certainly other places that have a similar effect.

> see? its even less lines of code :)

Except that it doesn't work.

Cheers,
   - Andreas

2008/11/11 Igor Stasenko <[hidden email]>:
>
> And , since primitive should return only single object, its easy to
> prove, that it requires MAXIMUM SINGLE temp value on stack, which
> needs to be remapped.
>

btw, from this follows as well, that there is no need in supporting a
bunch of remappable oops.
It really should be only a single remappable oop all the times.
So, instead of push/pop, it can be set/get. And this single value
should be set to nil, or smallint when primitive returns.
But i'm really doubt that it needed at all.
At the stage when primitive generating new object(s), which can be
subject of remapping , a primitive failure is quite unlikely, so
arguments on stack can be overidden.


--
Best regards,
Igor Stasenko AKA sig.

2008/11/11 Andreas Raab <[hidden email]>:
> Igor Stasenko wrote:
>>
>> Okay, but what about reusing the arguments stack slots?
>
> And what if there are none? There are tons of primitives returning complex
> objects (arrays, rectangles) that take no arguments.

there is always at least one argument - receiver.

>
>> In most cases, when primitive generating response its:
>> - already checked everything (can't fail)
>
> Primitives can fail for other reasons than argument mismatches.
>
yes, but what will happen if primitive fails on allocating new oop?
if you see the examples i shown, primitives in their epilogue never
check success flag after allocation, they blindly filling the complex
oop with values, and then returning it, inevitably damaging the stack
by pops/pushing result oop. This makes impossible to restore stack
into 'untouched' state for passing correct arguments into method if
primitive fails.
So, a proposed scheme having same flaws as previous in this case.

Interpreter + ObjectMemory having 7 senders of pushRemappableOop:, 3
of them in primitives.
It can use own , single remappable oop for its needs, but not stacked buffer.

>> see? its even less lines of code :)
>
> Except that it doesn't work.
>
err.. where i made a mistake? I think the rewrites will work 1:1 as
old code, except that they not using remappable buffer.

> Cheers,
>  - Andreas
>

--
Best regards,
Igor Stasenko AKA sig.

btw, concerning failures to allocate memory (and therefore primitive
failure when building up the complex oop).
An allocation followed by  #storePointer:ofObject:withValue: .
If you take a look, there is no checks if object is valid or having
enough slots for storing value at given offset.
So, in case if allocation fails, and return invalid object (or
non-object), there is 100% guarantee that it will corrupt object
memory.
So, what the difference between corrupting object memory and
corrupting the stack?
Btw the latter case can be detected much faster than first one.


--
Best regards,
Igor Stasenko AKA sig.

2008/11/11 Igor Stasenko <[hidden email]>: oh, wait, i just checked: allocation never fails.
If it fails, VM is bailing out immediately. :)
In this case, i insist that, if carefully written, primitives could
use the argument/receiver slots in stack for building up thier complex
oops as response.

--
Best regards,
Igor Stasenko AKA sig.

Hi Igor -

I think it's worthwhile to step back a bit and think about what you're
actually trying to achieve here. While it may be possible to replace
push/popRemappableOop with just push/pop it seems to me that you are
just replacing one set of obscure rules with an even more obscure set of
rules. Considering that such a change would also require rewriting every
single plugin it strikes me as being of little practical relevance.

If you are trying to simplify the programming model and improve the
robustness of plugins the better approach is to make
push/popRemappableOop be effective no-ops by disabling GCs in
primitives. This way you don't need to remap oops (except in places
where you may want to run an explicit GC such as primitiveNew) and can
keep all existing plugins with no change. The only requirement for this
is a bit of extra headroom which we have already (it is reserved for
forwarding blocks in full GCs) and it seems reasonable to take a small
bite out of that reserved space to trigger a GC after the primitive
completes.

Cheers,
   - Andreas

Igor Stasenko wrote:

On Nov 11, 2008, at 8:25 PM, Andreas Raab wrote:

> . The only requirement for this is a bit of extra headroom which we  
> have already (it is reserved for forwarding blocks in full GCs) and  
> it seems reasonable to take a small bite out of that reserved space  
> to trigger a GC after the primitive completes.
>
> Cheers,
>  - Andreas


In case anyone wonders I used this headroom as a *fix* for the low  
memory issue a couple of years back,  somewhere I've some code that
would creep young space into the forward block area as we ran out of  
memory.   However as Tim and I discovered
it's really hard to fix the low memory issue, this *fix* which didn't  
go into product would only delay the out of memory failure
by yet another few seconds, so it was quite worthless.

However as a solution to Oops memory allocation for primitives, versus  
a GC it's quite worthwhile.  I'd guess to
prevent changing every plugin to use some different allocator we could  
have a global switch that we turn on/off
when we enter/exit a plugin call, and if we actually do run out of  
memory based on young space/forwarding block area
then exit(42) if the flag is set?

Someone might want to check to see if  plugin memory allocation  
patterns are for small pieces. or does someone
allocate 100MB and that is enabled by doing the GC cycle?


--
=
=
=
========================================================================
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
=
=
=
========================================================================