Smalltalk › Squeak › Squeak VM

Re: [Pharo-dev] FFICallback crashes windows spur i386 vm

‹ Previous Topic Next Topic ›
 
classic Classic list List threaded Threaded
♦
♦
 6 messages Options
Nicolas Cellier
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [Pharo-dev] FFICallback crashes windows spur i386 vm

Nicolas Cellier
 
I've reproduced the crash in squeak.stack.spur Win32 with latest vm sources (SHA aa11221d9ebb5d60b0b88a2f1f8e27f19909d718)
This crashing VM is compiled with gcc via ./mvm -f

The Win64 version passes correctly. But it is compiled with clang by default.
The Win32 compiled with clang also passes correctly for me.

The test is translated in pure Alien for purpose of testing on Squeak 5.0 (see instructions below).

--------------------------------------------------------------

from cygwin (or cygwin64) shell:

$ i686-w64-mingw32-gcc --version
i686-w64-mingw32-gcc (GCC) 5.4.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ i686-w64-mingw32-clang --version
clang version 3.8.1 (tags/RELEASE_381/final)
Target: i686-w64-windows-gnu
Thread model: posix
InstalledDir: /usr/bin

$ x86_64-w64-mingw32-clang --version
clang version 3.8.1 (tags/RELEASE_381/final)
Target: x86_64-w64-windows-gnu
Thread model: posix
InstalledDir: /usr/bin

--------------------------------------------------------------

$ cd build.win32x86/squeak.stack.spur
$ ./mvm -f -- CC=i686-w64-mingw32-clang

or for 64 bits:

$ cd build.win64x64/squeak.stack.spur
$ ./mvm -f

--------------------------------------------------------------

$ cat >test.c << END
#include "stdint.h"
int8_t test(int8_t(*function)(int8_t), int8_t value) {
       return function(value);
    }
END

$ i686-w64-mingw32-clang -c test.c
$ i686-w64-mingw32-clang -shared -o build/vm/test.dll test.o
$ build/vm/Squeak.exe

--------------------------------------------------------------

then from Smalltalk image with Alien properly loaded (Alien-eem.39):

Callback>>int8RetInt8: callbackContext regs: regsAlien
    <signature: #(int8 (*)(int8))>
    ^callbackContext wordResult:
        (block value: (regsAlien signedCharAt: 1) asInteger)

(for now, above method must be compiled in each Callback subclass since we do not ask Pragma to scan superclass for signatures)

TestCase subclass: #AlienCallbackSunit
    instanceVariableNames: ''
    classVariableNames: ''
    poolDictionaries: ''
    category: 'Alien-CoreTest'

AlienCallbackSunit>>createCallback
       ^Callback
          signature: #(int8 (*)(int8))
          block: [ :value | value ]

AlienCallbackSunit>>primCall: aCallback int8: aNumber
       | r |
    (Alien lookup: 'test'  inLibrary: 'test.dll')
        primFFICallResult: (r := Alien new: 1)
        with: aCallback thunk
        with: aNumber.
    ^(r signedCharAt: 1) asInteger

AlienCallbackSunit>>test
      self
        assert: (self primCall: self createCallback int8: 0)
        equals: 0

AlienCallbackSunit new test.


2017-01-14 2:53 GMT+01:00 Eliot Miranda <[hidden email]>:
Hi Aliaksei,

On Jan 13, 2017, at 10:27 AM, Aliaksei Syrel <[hidden email]> wrote:

Hi

The following code crashes windows spur32 VM without crash.dmp:
(crashes not only in case of int8, but ulong, int16/32 and so on...)

Callback instantiation: 

createCallback
   ^ FFICallback 
  signature: #(int8 (int8))
  block: [ :value | value ]

C function:

int8_t test(int8_t(*function)(int8_t), int8_t value) {
   return function(value);
}

FFI call:

primCall: aCallback int8: aNumber
   ^ self ffiCall: #(int8 test(FFICallback aCallback, int8 aNumber))

Test case:

test
  self 
    assert: (self primCall: self createCallback int8: 0)
    equals: 0


Windows 10.
VM from today:
Image 60343:

P.S. what is the best place to report FFI / Alien issues?

vm-dev

P.P.S Iceberg works

Cheers,
Alex
Eliot Miranda-2
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [Pharo-dev] FFICallback crashes windows spur i386 vm

Eliot Miranda-2
 
Hi Nicolas, Hi Esteban,

    so is it feasible to insist on clang?  The mingw compiler is ancient, so my preference is to move to clang.  If so, could someone update the HowToBuild to describe the problem with mingw and how to install the clang-based toolchain?

On Tue, Jan 17, 2017 at 6:47 AM, Nicolas Cellier <[hidden email]> wrote:
 
I've reproduced the crash in squeak.stack.spur Win32 with latest vm sources (SHA aa11221d9ebb5d60b0b88a2f1f8e27f19909d718)
This crashing VM is compiled with gcc via ./mvm -f

The Win64 version passes correctly. But it is compiled with clang by default.
The Win32 compiled with clang also passes correctly for me.

The test is translated in pure Alien for purpose of testing on Squeak 5.0 (see instructions below).

--------------------------------------------------------------

from cygwin (or cygwin64) shell:

$ i686-w64-mingw32-gcc --version
i686-w64-mingw32-gcc (GCC) 5.4.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ i686-w64-mingw32-clang --version
clang version 3.8.1 (tags/RELEASE_381/final)
Target: i686-w64-windows-gnu
Thread model: posix
InstalledDir: /usr/bin

$ x86_64-w64-mingw32-clang --version
clang version 3.8.1 (tags/RELEASE_381/final)
Target: x86_64-w64-windows-gnu
Thread model: posix
InstalledDir: /usr/bin

--------------------------------------------------------------

$ cd build.win32x86/squeak.stack.spur
$ ./mvm -f -- CC=i686-w64-mingw32-clang

or for 64 bits:

$ cd build.win64x64/squeak.stack.spur
$ ./mvm -f

--------------------------------------------------------------

$ cat >test.c << END
#include "stdint.h"
int8_t test(int8_t(*function)(int8_t), int8_t value) {
       return function(value);
    }
END

$ i686-w64-mingw32-clang -c test.c
$ i686-w64-mingw32-clang -shared -o build/vm/test.dll test.o
$ build/vm/Squeak.exe

--------------------------------------------------------------

then from Smalltalk image with Alien properly loaded (Alien-eem.39):

Callback>>int8RetInt8: callbackContext regs: regsAlien
    <signature: #(int8 (*)(int8))>
    ^callbackContext wordResult:
        (block value: (regsAlien signedCharAt: 1) asInteger)

(for now, above method must be compiled in each Callback subclass since we do not ask Pragma to scan superclass for signatures)

TestCase subclass: #AlienCallbackSunit
    instanceVariableNames: ''
    classVariableNames: ''
    poolDictionaries: ''
    category: 'Alien-CoreTest'

AlienCallbackSunit>>createCallback
       ^Callback
          signature: #(int8 (*)(int8))
          block: [ :value | value ]

AlienCallbackSunit>>primCall: aCallback int8: aNumber
       | r |
    (Alien lookup: 'test'  inLibrary: 'test.dll')
        primFFICallResult: (r := Alien new: 1)
        with: aCallback thunk
        with: aNumber.
    ^(r signedCharAt: 1) asInteger

AlienCallbackSunit>>test
      self
        assert: (self primCall: self createCallback int8: 0)
        equals: 0

AlienCallbackSunit new test.


2017-01-14 2:53 GMT+01:00 Eliot Miranda <[hidden email]>:
Hi Aliaksei,

On Jan 13, 2017, at 10:27 AM, Aliaksei Syrel <[hidden email]> wrote:

Hi

The following code crashes windows spur32 VM without crash.dmp:
(crashes not only in case of int8, but ulong, int16/32 and so on...)

Callback instantiation: 

createCallback
   ^ FFICallback 
  signature: #(int8 (int8))
  block: [ :value | value ]

C function:

int8_t test(int8_t(*function)(int8_t), int8_t value) {
   return function(value);
}

FFI call:

primCall: aCallback int8: aNumber
   ^ self ffiCall: #(int8 test(FFICallback aCallback, int8 aNumber))

Test case:

test
  self 
    assert: (self primCall: self createCallback int8: 0)
    equals: 0


Windows 10.
VM from today:
Image 60343:

P.S. what is the best place to report FFI / Alien issues?

vm-dev

P.P.S Iceberg works

Cheers,
Alex





--
_,,,^..^,,,_
best, Eliot
Holger Freyther
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [Pharo-dev] FFICallback crashes windows spur i386 vm

Holger Freyther
 

> On 17 Jan 2017, at 19:43, Eliot Miranda <[hidden email]> wrote:
>
> Hi Nicolas, Hi Esteban,

Hi all,



>
>     so is it feasible to insist on clang?  The mingw compiler is ancient, so my preference is to move to clang.  If so, could someone update the HowToBuild to describe the problem with mingw and how to install the clang-based toolchain?


> from cygwin (or cygwin64) shell:
>
> $ i686-w64-mingw32-gcc --version
> i686-w64-mingw32-gcc (GCC) 5.4.0
> Copyright (C) 2015 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions.  There is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


maybe I am missing something but GCC 5.4 was released in Summer 2016 and is
still supported. Couldn't it be that the better optimizers in GCC expose an
issue that clang is not capable of exposing (yet?)?

just my two cents

        holger
Nicolas Cellier
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [Pharo-dev] FFICallback crashes windows spur i386 vm

Nicolas Cellier
In reply to this post by Eliot Miranda-2
 


2017-01-17 19:43 GMT+01:00 Eliot Miranda <[hidden email]>:
 
Hi Nicolas, Hi Esteban,

    so is it feasible to insist on clang?  The mingw compiler is ancient, so my preference is to move to clang.  If so, could someone update the HowToBuild to describe the problem with mingw and how to install the clang-based toolchain?


Hi Eliot,
from a recent cygwin, it's simple, detect the clang packages among:

https://cygwin.com/cgi-bin2/package-grep.cgi?grep=clang&

then load:
mingw64-i686-clang-3.8.1-1 for 32 bits
mingw64-x86_64-clang-3.8.1-1 for 64 bits

https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/.appveyor.yml contains an example for installing such package from a cygwin setup command line.

Note: it does not matter whether a 32bits or 64bits cygwin is installed, those packages enable cross-compilation.
However, it matters for gdb if ever we have to debug at such low level...


AFAIU, Pharo depends on extra libs that have no builtin cygwin compilation support, so Esteban is using mingw/msys.
I don't know how to install clang on such brand (except recompiling from sources which is not super for a bot like appveyor)
Maybe Esteban has a solution?

 
On Tue, Jan 17, 2017 at 6:47 AM, Nicolas Cellier <[hidden email]> wrote:
 
I've reproduced the crash in squeak.stack.spur Win32 with latest vm sources (SHA aa11221d9ebb5d60b0b88a2f1f8e27f19909d718)
This crashing VM is compiled with gcc via ./mvm -f

The Win64 version passes correctly. But it is compiled with clang by default.
The Win32 compiled with clang also passes correctly for me.

The test is translated in pure Alien for purpose of testing on Squeak 5.0 (see instructions below).

--------------------------------------------------------------

from cygwin (or cygwin64) shell:

$ i686-w64-mingw32-gcc --version
i686-w64-mingw32-gcc (GCC) 5.4.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ i686-w64-mingw32-clang --version
clang version 3.8.1 (tags/RELEASE_381/final)
Target: i686-w64-windows-gnu
Thread model: posix
InstalledDir: /usr/bin

$ x86_64-w64-mingw32-clang --version
clang version 3.8.1 (tags/RELEASE_381/final)
Target: x86_64-w64-windows-gnu
Thread model: posix
InstalledDir: /usr/bin

--------------------------------------------------------------

$ cd build.win32x86/squeak.stack.spur
$ ./mvm -f -- CC=i686-w64-mingw32-clang

or for 64 bits:

$ cd build.win64x64/squeak.stack.spur
$ ./mvm -f

--------------------------------------------------------------

$ cat >test.c << END
#include "stdint.h"
int8_t test(int8_t(*function)(int8_t), int8_t value) {
       return function(value);
    }
END

$ i686-w64-mingw32-clang -c test.c
$ i686-w64-mingw32-clang -shared -o build/vm/test.dll test.o
$ build/vm/Squeak.exe

--------------------------------------------------------------

then from Smalltalk image with Alien properly loaded (Alien-eem.39):

Callback>>int8RetInt8: callbackContext regs: regsAlien
    <signature: #(int8 (*)(int8))>
    ^callbackContext wordResult:
        (block value: (regsAlien signedCharAt: 1) asInteger)

(for now, above method must be compiled in each Callback subclass since we do not ask Pragma to scan superclass for signatures)

TestCase subclass: #AlienCallbackSunit
    instanceVariableNames: ''
    classVariableNames: ''
    poolDictionaries: ''
    category: 'Alien-CoreTest'

AlienCallbackSunit>>createCallback
       ^Callback
          signature: #(int8 (*)(int8))
          block: [ :value | value ]

AlienCallbackSunit>>primCall: aCallback int8: aNumber
       | r |
    (Alien lookup: 'test'  inLibrary: 'test.dll')
        primFFICallResult: (r := Alien new: 1)
        with: aCallback thunk
        with: aNumber.
    ^(r signedCharAt: 1) asInteger

AlienCallbackSunit>>test
      self
        assert: (self primCall: self createCallback int8: 0)
        equals: 0

AlienCallbackSunit new test.


2017-01-14 2:53 GMT+01:00 Eliot Miranda <[hidden email]>:
Hi Aliaksei,

On Jan 13, 2017, at 10:27 AM, Aliaksei Syrel <[hidden email]> wrote:

Hi

The following code crashes windows spur32 VM without crash.dmp:
(crashes not only in case of int8, but ulong, int16/32 and so on...)

Callback instantiation: 

createCallback
   ^ FFICallback 
  signature: #(int8 (int8))
  block: [ :value | value ]

C function:

int8_t test(int8_t(*function)(int8_t), int8_t value) {
   return function(value);
}

FFI call:

primCall: aCallback int8: aNumber
   ^ self ffiCall: #(int8 test(FFICallback aCallback, int8 aNumber))

Test case:

test
  self 
    assert: (self primCall: self createCallback int8: 0)
    equals: 0


Windows 10.
VM from today:
Image 60343:

P.S. what is the best place to report FFI / Alien issues?

vm-dev

P.P.S Iceberg works

Cheers,
Alex





--
_,,,^..^,,,_
best, Eliot
Nicolas Cellier
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [Pharo-dev] FFICallback crashes windows spur i386 vm

Nicolas Cellier
In reply to this post by Holger Freyther
 


2017-01-17 19:49 GMT+01:00 Holger Freyther <[hidden email]>:


> On 17 Jan 2017, at 19:43, Eliot Miranda <[hidden email]> wrote:
>
> Hi Nicolas, Hi Esteban,

Hi all,



>
>     so is it feasible to insist on clang?  The mingw compiler is ancient, so my preference is to move to clang.  If so, could someone update the HowToBuild to describe the problem with mingw and how to install the clang-based toolchain?


> from cygwin (or cygwin64) shell:
>
> $ i686-w64-mingw32-gcc --version
> i686-w64-mingw32-gcc (GCC) 5.4.0
> Copyright (C) 2015 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions.  There is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


maybe I am missing something but GCC 5.4 was released in Summer 2016 and is
still supported. Couldn't it be that the better optimizers in GCC expose an
issue that clang is not capable of exposing (yet?)?

just my two cents

        holger

Hi Holger,
maybe... One should compile with -O0 to check if it's really about optimization and retry if it solves anything.
If yes, then play the game of selecting each individual optimization option.

What I know is that the mingw version of gcc have strange stack management with bytes reserved below alloca which is causing some complications in our implemenation of FFI.
Personnally I don't have a clue of the reason behind the difference with linux gcc, so without further information, I tend to perceive such differences as a useless and never-ending-source of complications. I'm speaking for myself, but maybe Eliot has the same feeling.

Not sure that it's a great choice to abandon gcc support, but it's quite tempting in those circumstances ;)
Nicolas Cellier
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [Pharo-dev] FFICallback crashes windows spur i386 vm

Nicolas Cellier
 


2017-01-17 22:24 GMT+01:00 Nicolas Cellier <[hidden email]>:


2017-01-17 19:49 GMT+01:00 Holger Freyther <[hidden email]>:


> On 17 Jan 2017, at 19:43, Eliot Miranda <[hidden email]> wrote:
>
> Hi Nicolas, Hi Esteban,

Hi all,



>
>     so is it feasible to insist on clang?  The mingw compiler is ancient, so my preference is to move to clang.  If so, could someone update the HowToBuild to describe the problem with mingw and how to install the clang-based toolchain?


> from cygwin (or cygwin64) shell:
>
> $ i686-w64-mingw32-gcc --version
> i686-w64-mingw32-gcc (GCC) 5.4.0
> Copyright (C) 2015 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions.  There is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


maybe I am missing something but GCC 5.4 was released in Summer 2016 and is
still supported. Couldn't it be that the better optimizers in GCC expose an
issue that clang is not capable of exposing (yet?)?

just my two cents

        holger

Hi Holger,
maybe... One should compile with -O0 to check if it's really about optimization and retry if it solves anything.
If yes, then play the game of selecting each individual optimization option.


I confirm that the test pass when we compile with gcc thru mvm -d
When such thing happens, most often, this is a sign that we still rely on some undefined behavior...
I would recommend playing with clang runtime checks to anyone having some time to invest or finding it fun (ok, it's not everyone).

What I know is that the mingw version of gcc have strange stack management with bytes reserved below alloca which is causing some complications in our implemenation of FFI.
Personnally I don't have a clue of the reason behind the difference with linux gcc, so without further information, I tend to perceive such differences as a useless and never-ending-source of complications. I'm speaking for myself, but maybe Eliot has the same feeling.

Not sure that it's a great choice to abandon gcc support, but it's quite tempting in those circumstances ;)
Free forum by Nabble Edit this page