[squeak-dev] x86 linux VM FFI FPU stack overflow

Apologies for the cryptic subject...

I have some news about http://bugs.squeak.org/view.php?id=3929
I detected with gdb that the example I provided triggers a FPU stack
overflow. When this occurs, a qNan is placed on FPU stack top.

Thus, though ffiFloatReturnValue == 0.0, testing in Smalltalk (self
myFFICall = 0.0) will result in 0.0==Nan due to stack overflow.
primitive which execute special send #= will incorrectly return false...

Don't tell me to not use = on Floats, this is not the problem, you can
replace the test with (self myFFICall < 1.0), you will get (0.0<qNan) ->
false all the same.

At this point, I see two possibilities:
- either I totally messed up with calling conventions
- or the FFI glue ffiCallAddressOf is broken
($VM/Squeak-3.10-1/platforms/unix/plugins/SqueakFFIPrims/x86-sysv-asm)

I have no knowledge about IA-32 and don't wish to learn further, using
gdb at instruction level was already painfull enough for me.
Can somebody aware of x86 and calling convention check my guess?

Nicolas

nicolas cellier a écrit :
Any one that can provide light on this subject?
Or just get a contact with Ian Piumarta?

I feel a bit alone

Nicolas

nicolas cellier writes:
 > nicolas cellier a écrit :
 > >
 > > Apologies for the cryptic subject...
 > >
 > > I have some news about http://bugs.squeak.org/view.php?id=3929
 > > I detected with gdb that the example I provided triggers a FPU stack
 > > overflow. When this occurs, a qNan is placed on FPU stack top.
 > >
 > > Thus, though ffiFloatReturnValue == 0.0, testing in Smalltalk (self
 > > myFFICall = 0.0) will result in 0.0==Nan due to stack overflow.
 > > primitive which execute special send #= will incorrectly return false...
 > >
 > > Don't tell me to not use = on Floats, this is not the problem, you can
 > > replace the test with (self myFFICall < 1.0), you will get (0.0<qNan) ->
 > > false all the same.
 > >
 > > At this point, I see two possibilities:
 > > - either I totally messed up with calling conventions
 > > - or the FFI glue ffiCallAddressOf is broken
 > > ($VM/Squeak-3.10-1/platforms/unix/plugins/SqueakFFIPrims/x86-sysv-asm)
 > >
 > > I have no knowledge about IA-32 and don't wish to learn further, using
 > > gdb at instruction level was already painfull enough for me.
 > > Can somebody aware of x86 and calling convention check my guess?
 > >
 > > Nicolas
 > >
 > >
 >
 > Any one that can provide light on this subject?
 > Or just get a contact with Ian Piumarta?
 >
 > I feel a bit alone

Try creating your own mock call with the same prototype that just
logs it's arguments. Or place a gdb call inside the called function
or mock.

I'd guess that floats work for FFI as it's used for OpenGL in Croquet.

I haven't looked at floating point calling conventions so can't help
more.

Bryce

Hi Nicholas,

Eliot Miranda <eliot.miranda <at> gmail.com> writes:

>
>
> Hi Nicholas,
>
>     the x87 FPU (embedded in all modern x86's and used by default to pass
floating point arguments in ABI calls) is a stack of only 8 registers.
>
> Imagine what happens if you attempt an FFI call that parse, say, 4 of its
floating-point arguments before hitting an invalid parameter.  Likely the FFI
marshalling machinery will leave 4 items on the x87's stack and forget to clean
them up.
>
> Imagine what happens if you call some broken code that doesn't leave the x87
register stack empty.
>
> In either of these cases the next time you try and make a call and load items
onto the x87's stack it'll quite possibly overflow.
>

Yes, that's exactly what happens in my test case (gdb output provided at
http://bugs.squeak.org/view.php?id=3929).
After the seventh call, there are 7 slots occupied on stack.
If I then attempt a comparison of the result with another float, I need two
slots, the stack overflows, pushes a qNan and the answer of comparison is always
false.

But this is not because of the Callee: it seems weel behaved, the only double
being on FPU stack on function return is the returned double value.
It is the FFI glue Caller itself which fails to unstack the result.

Changing line 51 of x86-sysv-asm.S solved this problem:

    fstpl ffiFloatReturnValue

But that's only a guess, I'm not sure if there are other implications.


> So.... the FFI marshalling machinery really needs to reset the FPU's stack
pointer to zero both before and after making a call.  I don't know if the FFI
does this but chances are it doesn't.
>
> In my time VisualWorks suffered problems like this, including its machine code
floating point primitives not zeroing the x87 stack pointer on primitive fail.
 Now it does an fninit (the relevant instruction to intialize the x87 FPU) in
relevant places.
>
> Cheers
> Eliot
>

As suggested by my above tests, the fninit does not seem to be called in FFI.
Would you know how to force such an instruction in a portable way via VMMaker?

There must be some fninit called somewhere in the VM, because I see the stack is
reset after a few UI. That would be interesting to know if they are in the right
place indeed. Only you (and a few others) could check I'm afraid.

Thank you

Nicolas

Nicolas Cellier a écrit :
> Eliot Miranda <eliot.miranda <at> gmail.com> writes:

Ian just mailed that all parameters are passed in memory, so the right
place for fninit is before foreign call, and before ffi return.

However, I see this does reset the whole FPU state.
Bad if in the future we want to control hardware trap on FPU...

nicolas cellier a écrit :
>
> Ian just mailed that all parameters are passed in memory, so the right
> place for fninit is before foreign call, and before ffi return.
>
> However, I see this does reset the whole FPU state.
> Bad if in the future we want to control hardware trap on FPU...
>

fninit is the right thing to do for current Squeak VM.
The eventual future VM would better call ffree for less side effetcs.