Maintainer of FFI-Package

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Maintainer of FFI-Package

Nicolai Hess
Who is the maintainer of the FFI-Package for
Squeak and/or Pharo ?
I especially interested in the FFI-Unix examples
 and the state on working FFI on linux for current squeak release
(AFAIK it does not work with "squeak all in one" 4.5)

regards
Nicolai
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Maintainer of FFI-Package

David T. Lewis
On Sun, Sep 28, 2014 at 11:19:30AM +0200, Nicolai Hess wrote:
> Who is the maintainer of the FFI-Package for
> Squeak and/or Pharo ?
> I especially interested in the FFI-Unix examples
>  and the state on working FFI on linux for current squeak release
> (AFAIK it does not work with "squeak all in one" 4.5)
>

The FFI package, including the FFI-Unix examples that you mention, is at
http://source.squeak.org/FFI.

The primary author was Andreas Raab. Eliot Miranda is the most recent
contributor, and he is probably the most knowledgeable in this area.
Currently, the Squeak core developers (those with commit rights to Squeak
trunk) have commit rights to the FFI project, and can make updates as needed.

The FFI examples for Unix work with the old FFI plugin, which is still
present in the interpreter VM. The active development of Squeak FFI is
taking place in Cog, and I'm not sure if the old FFI examples work with
the newer plugins (which would be present in the Squeak all-in-one).
The FFI plugin in the interpreter VM (i.e. trunk VMMaker) needs to be
updated, and presumably the FFI examples may need some updates as well.

Dave


Reply | Threaded
Open this post in threaded view
|

Re: Maintainer of FFI-Package

Eliot Miranda-2
In reply to this post by Nicolai Hess
Hi Nicolai,

On Sep 28, 2014, at 2:19 AM, Nicolai Hess <[hidden email]> wrote:

> Who is the maintainer of the FFI-Package for
> Squeak and/or Pharo ?

I guess I'm the de facto maintainer because I'm the one who has changed things most recently, reimplementing the FFI plugin via the ThreadedFFIPlugin.


> I especially interested in the FFI-Unix examples
>  and the state on working FFI on linux for current squeak release
> (AFAIK it does not work with "squeak all in one" 4.5)

Can you give me a reproducible case?  I'd this with a Cog,Stack or Interpreter VM?  Which specific things fail and how?
>
> regards
> Nicolai

Eliot (phone)
Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] Re: [squeak-dev] Maintainer of FFI-Package

David T. Lewis
In reply to this post by David T. Lewis
On Sun, Sep 28, 2014 at 09:04:59AM -0400, David T. Lewis wrote:

>  
> On Sun, Sep 28, 2014 at 11:19:30AM +0200, Nicolai Hess wrote:
> > Who is the maintainer of the FFI-Package for
> > Squeak and/or Pharo ?
> > I especially interested in the FFI-Unix examples
> >  and the state on working FFI on linux for current squeak release
> > (AFAIK it does not work with "squeak all in one" 4.5)
> >
>
> The FFI package, including the FFI-Unix examples that you mention, is at
> http://source.squeak.org/FFI.
>
> The primary author was Andreas Raab. Eliot Miranda is the most recent
> contributor, and he is probably the most knowledgeable in this area.
> Currently, the Squeak core developers (those with commit rights to Squeak
> trunk) have commit rights to the FFI project, and can make updates as needed.
>
> The FFI examples for Unix work with the old FFI plugin, which is still
> present in the interpreter VM. The active development of Squeak FFI is
> taking place in Cog, and I'm not sure if the old FFI examples work with
> the newer plugins (which would be present in the Squeak all-in-one).
> The FFI plugin in the interpreter VM (i.e. trunk VMMaker) needs to be
> updated, and presumably the FFI examples may need some updates as well.

A small but important correction: For the old plugin (currently in the
interpreter VM), FFI works only for a 32-bit VM. It also works for a 64-bit
VM, but only if the patches in http://bugs.squeak.org/view.php?id=7237 have
been applied (in other words, if you compiled it yourself and included
those patches).

This is mainly of historical interest, although it is possible that some
of the 64-bit patches will prove helpful for the newer FFI plugin. But
the emphasis now should be on Eliot's more recent FFI work.

Dave


Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] Re: [squeak-dev] Maintainer of FFI-Package

Eliot Miranda-2
Hi David,

On Sep 28, 2014, at 12:44 PM, "David T. Lewis" <[hidden email]> wrote:

> On Sun, Sep 28, 2014 at 09:04:59AM -0400, David T. Lewis wrote:
>>
>> On Sun, Sep 28, 2014 at 11:19:30AM +0200, Nicolai Hess wrote:
>>> Who is the maintainer of the FFI-Package for
>>> Squeak and/or Pharo ?
>>> I especially interested in the FFI-Unix examples
>>> and the state on working FFI on linux for current squeak release
>>> (AFAIK it does not work with "squeak all in one" 4.5)
>>
>> The FFI package, including the FFI-Unix examples that you mention, is at
>> http://source.squeak.org/FFI.
>>
>> The primary author was Andreas Raab. Eliot Miranda is the most recent
>> contributor, and he is probably the most knowledgeable in this area.
>> Currently, the Squeak core developers (those with commit rights to Squeak
>> trunk) have commit rights to the FFI project, and can make updates as needed.
>>
>> The FFI examples for Unix work with the old FFI plugin, which is still
>> present in the interpreter VM. The active development of Squeak FFI is
>> taking place in Cog, and I'm not sure if the old FFI examples work with
>> the newer plugins (which would be present in the Squeak all-in-one).
>> The FFI plugin in the interpreter VM (i.e. trunk VMMaker) needs to be
>> updated, and presumably the FFI examples may need some updates as well.
>
> A small but important correction: For the old plugin (currently in the
> interpreter VM), FFI works only for a 32-bit VM. It also works for a 64-bit
> VM, but only if the patches in http://bugs.squeak.org/view.php?id=7237 have
> been applied (in other words, if you compiled it yourself and included
> those patches).


Good point.  It should be easier to adapt the ThreadedFFIPlugin to 64 bits because there's no assembler involved.

But this gets to the crux of why we need to rearchitect, because the x86-64/IA64 ABI is extremely complex to use interpretively and much easier to approach with a special-purpose compiler.  Some steps are being taken in this direction in the Pharo community.

In fact there's more possibility of getting  x86-64/IA64 working with the Squeak FFI than with eg VisualWorks because Andreas' type signature scheme (ExternalType et al) is richer than VW's.  but it's work; much of the marshaling code in the plugin would need rewriting.

We're just starting the Spur 64-but VM and do will tackle this soon enough (some time mid next year?).  Whether by that time it'll make more sense to extend the existing ThreadedFFIPlugin or go with the new architecture is too early to tell.

HTH

> This is mainly of historical interest, although it is possible that some
> of the 64-bit patches will prove helpful for the newer FFI plugin. But
> the emphasis now should be on Eliot's more recent FFI work.

+1


>
> Dave

Cheers,
Eliot
Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] Re: Maintainer of FFI-Package

Nicolai Hess
In reply to this post by Eliot Miranda-2
Thank you David,
Thank you Eliot

2014-09-28 21:24 GMT+02:00 Eliot Miranda <[hidden email]>:

Hi Nicolai,

On Sep 28, 2014, at 2:19 AM, Nicolai Hess <[hidden email]> wrote:

> Who is the maintainer of the FFI-Package for
> Squeak and/or Pharo ?

I guess I'm the de facto maintainer because I'm the one who has changed things most recently, reimplementing the FFI plugin via the ThreadedFFIPlugin.


> I especially interested in the FFI-Unix examples
>  and the state on working FFI on linux for current squeak release
> (AFAIK it does not work with "squeak all in one" 4.5)

Can you give me a reproducible case?  I'd this with a Cog,Stack or Interpreter VM?  Which specific things fail and how?

Image
-----
/home/nicolai/Downloads/Squeak-4.5-All-in-One.app/Contents/Resources/Squeak4.5-13680.image
Squeak4.5
latest update: #13680
Current Change Set: Unnamed1
Image format 6505 (32 bit)

Virtual Machine
---------------
/home/nicolai/Downloads/Squeak-4.5-All-in-One.app/Contents/Linux-i686/lib/squeak/4.0-2776/squeak
Croquet Closure Cog VM [CoInterpreter VMMaker.oscog-eem.331]
Unix built on Aug 22 2013 10:35:37 Compiler: 4.1.2 20080704 (Red Hat 4.1.2-48)
platform sources revision VM: r2776 http://www.squeakvm.org/svn/squeak/branches/Cog Plugins: r2545 http://squeakvm.org/svn/squeak/trunk/platforms/Cross/plugins
CoInterpreter VMMaker.oscog-eem.331 uuid: 37d2e4b0-2f37-4e2d-8313-c63637785e59 Aug 22 2013
StackToRegisterMappingCogit VMMaker.oscog-eem.333 uuid: 84da9cb8-7f30-4cb7-b4fb-239a11f63b54 Aug 22 2013

Load
Name: FFI-Pools-eem.3
Name: FFI-Kernel-eem.26
Name: FFI-Tests-djm.7

Run FFI-Test -> 32 run, 24 passes, 0 expected failures, 0 failures, 8 errors, 0 unexpected passes

Load
Name: FFI-Unix-mtf.4
 
run X11Display class>>#coloredRectangles
-> Error: Could not coerse arguments

Load newer cogvm
Virtual Machine
---------------
/media/Programs/squeak/Squeak-4.5-All-in-One.app/coglinux/lib/squeak/4.0-2987/squeak
Croquet Closure Cog VM [CoInterpreter VMMaker.oscog-eem.746]
Unix built on Jun 11 2014 14:59:18 Compiler: 4.1.2 20080704 (Red Hat 4.1.2-48)
platform sources revision VM: r2987 http://www.squeakvm.org/svn/squeak/branches/Cog Date: 2014-06-11 14:26:16 -0700 Plugins: r2954 http://squeakvm.org/svn/squeak/trunk/platforms/Cross/plugins
CoInterpreter VMMaker.oscog-eem.746 uuid: cc4961d3-e629-4e28-b308-88eab314a8c9 Jun 11 2014
StackToRegisterMappingCogit VMMaker.oscog-eem.766 uuid: 3045c341-cfbf-494c-a7b1-fb29c26a7340 Jun 11 2014

Run FFI-Test -> 32 run, 32 passes, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes


The error in FFI-Unix is strange. The last package comment from
Name: FFI-Unix-mtf.4
Author: mtf
is
"made the XLib examples actually work (Window is an unsigned int handle, not a struct)"

That the window is an int handle and not a struct explains why this version replaces the parent class
of XDrawable from ExternalStructur to Object. But all calls with an XDrawable still have this
object as parameter ( and not its instance var "xid"). I don't understand how this ever
worked (or was there some kind of automatic argument conversion like:
transform an object with an instvar -> a int, if the function expects an int handle?)

Sure we can replace all XDrawable arguments in X11 calls from "aXDrawable" to "aXDrawable xid".
But I only wonder, did this ever work and if Author "mtf" did the last change, can I contact
him or is he still maintaining the FF-Unix examples?



>
> regards
> Nicolai

Eliot (phone)

Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] [squeak-dev] Maintainer of FFI-Package

Eliot Miranda-2
In reply to this post by David T. Lewis
Hi Annick,

On Mon, Sep 29, 2014 at 5:30 AM, Annick Fron <[hidden email]> wrote:
Hi,


I have recently made some experiments with pharo and FFI, and I have ran into problems with callbacks.
I am using linux 32 bits.
First question : when calling a dynamic  library is it necessary that it exports all symbols ? What if symbols are noted « U » (undefined) ?

If you have undefined symbols in an external library, they need to be resoled by the VM or other libraries loaded in the system.  If any are not resolved the library will fail to load.
 
2) On one of my callbacks got the message « callback failed to own the VM » . Have you any idea of where it comes from ?

Yes.  This means you're trying to callback from a different thread than the VM thread.  hat isn't supported yet.
 
3) If the callback is called  in a different thread, is it a problem ?

Yes.

4) On Pharo I have an error on the following : is it incorrect ? It is a 64 bits problem ?
Using FFI Linux examples.

testGetOurWindowLocation
"self new testGetOurWindowLocation"

| display ourWindow aParent anX anY aWidth aHeight aBorderWidth aDepth |
display := X11Display XOpenDisplay: nil.
ourWindow := display ourWindow.
(Alien lookup: 'XGetGeometry' inLibrary: 'X11')
    primFFICallResult: nil
    withArguments:
        {display.
        ourWindow.
        (aParent := Alien new: 4).
        (anX := Alien new: 4).
        (anY := Alien new: 4).
        (aWidth := Alien new: 4).
        (aHeight := Alien new: 4).
        (aBorderWidth := Alien new: 4).
        (aDepth := Alien new: 4)}.
(anX unsignedLongAt: 1) inspect

What's the error you get?

Annick
Le 28 sept. 2014 à 21:44, David T. Lewis <[hidden email]> a écrit :

> On Sun, Sep 28, 2014 at 09:04:59AM -0400, David T. Lewis wrote:
>>
>> On Sun, Sep 28, 2014 at 11:19:30AM +0200, Nicolai Hess wrote:
>>> Who is the maintainer of the FFI-Package for
>>> Squeak and/or Pharo ?
>>> I especially interested in the FFI-Unix examples
>>> and the state on working FFI on linux for current squeak release
>>> (AFAIK it does not work with "squeak all in one" 4.5)
>>>
>>
>> The FFI package, including the FFI-Unix examples that you mention, is at
>> http://source.squeak.org/FFI.
>>
>> The primary author was Andreas Raab. Eliot Miranda is the most recent
>> contributor, and he is probably the most knowledgeable in this area.
>> Currently, the Squeak core developers (those with commit rights to Squeak
>> trunk) have commit rights to the FFI project, and can make updates as needed.
>>
>> The FFI examples for Unix work with the old FFI plugin, which is still
>> present in the interpreter VM. The active development of Squeak FFI is
>> taking place in Cog, and I'm not sure if the old FFI examples work with
>> the newer plugins (which would be present in the Squeak all-in-one).
>> The FFI plugin in the interpreter VM (i.e. trunk VMMaker) needs to be
>> updated, and presumably the FFI examples may need some updates as well.
>
> A small but important correction: For the old plugin (currently in the
> interpreter VM), FFI works only for a 32-bit VM. It also works for a 64-bit
> VM, but only if the patches in http://bugs.squeak.org/view.php?id=7237 have
> been applied (in other words, if you compiled it yourself and included
> those patches).
>
> This is mainly of historical interest, although it is possible that some
> of the 64-bit patches will prove helpful for the newer FFI plugin. But
> the emphasis now should be on Eliot's more recent FFI work.
>
> Dave
>
>





--
best,
Eliot
Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] Re: Maintainer of FFI-Package

Nicolai Hess
In reply to this post by Nicolai Hess


2014-09-28 22:04 GMT+02:00 Nicolai Hess <[hidden email]>:
Thank you David,
Thank you Eliot

2014-09-28 21:24 GMT+02:00 Eliot Miranda <[hidden email]>:

Hi Nicolai,

On Sep 28, 2014, at 2:19 AM, Nicolai Hess <[hidden email]> wrote:

> Who is the maintainer of the FFI-Package for
> Squeak and/or Pharo ?

I guess I'm the de facto maintainer because I'm the one who has changed things most recently, reimplementing the FFI plugin via the ThreadedFFIPlugin.


> I especially interested in the FFI-Unix examples
>  and the state on working FFI on linux for current squeak release
> (AFAIK it does not work with "squeak all in one" 4.5)

Can you give me a reproducible case?  I'd this with a Cog,Stack or Interpreter VM?  Which specific things fail and how?

Image
-----
/home/nicolai/Downloads/Squeak-4.5-All-in-One.app/Contents/Resources/Squeak4.5-13680.image
Squeak4.5
latest update: #13680
Current Change Set: Unnamed1
Image format 6505 (32 bit)

Virtual Machine
---------------
/home/nicolai/Downloads/Squeak-4.5-All-in-One.app/Contents/Linux-i686/lib/squeak/4.0-2776/squeak
Croquet Closure Cog VM [CoInterpreter VMMaker.oscog-eem.331]
Unix built on Aug 22 2013 10:35:37 Compiler: 4.1.2 20080704 (Red Hat 4.1.2-48)
platform sources revision VM: r2776 http://www.squeakvm.org/svn/squeak/branches/Cog Plugins: r2545 http://squeakvm.org/svn/squeak/trunk/platforms/Cross/plugins
CoInterpreter VMMaker.oscog-eem.331 uuid: 37d2e4b0-2f37-4e2d-8313-c63637785e59 Aug 22 2013
StackToRegisterMappingCogit VMMaker.oscog-eem.333 uuid: 84da9cb8-7f30-4cb7-b4fb-239a11f63b54 Aug 22 2013

Load
Name: FFI-Pools-eem.3
Name: FFI-Kernel-eem.26
Name: FFI-Tests-djm.7

Run FFI-Test -> 32 run, 24 passes, 0 expected failures, 0 failures, 8 errors, 0 unexpected passes

Load
Name: FFI-Unix-mtf.4
 
run X11Display class>>#coloredRectangles
-> Error: Could not coerse arguments

Load newer cogvm
Virtual Machine
---------------
/media/Programs/squeak/Squeak-4.5-All-in-One.app/coglinux/lib/squeak/4.0-2987/squeak
Croquet Closure Cog VM [CoInterpreter VMMaker.oscog-eem.746]
Unix built on Jun 11 2014 14:59:18 Compiler: 4.1.2 20080704 (Red Hat 4.1.2-48)
platform sources revision VM: r2987 http://www.squeakvm.org/svn/squeak/branches/Cog Date: 2014-06-11 14:26:16 -0700 Plugins: r2954 http://squeakvm.org/svn/squeak/trunk/platforms/Cross/plugins
CoInterpreter VMMaker.oscog-eem.746 uuid: cc4961d3-e629-4e28-b308-88eab314a8c9 Jun 11 2014
StackToRegisterMappingCogit VMMaker.oscog-eem.766 uuid: 3045c341-cfbf-494c-a7b1-fb29c26a7340 Jun 11 2014

Run FFI-Test -> 32 run, 32 passes, 0 expected failures, 0 failures, 0 errors, 0 unexpected passes


The error in FFI-Unix is strange. The last package comment from
Name: FFI-Unix-mtf.4
Author: mtf
is
"made the XLib examples actually work (Window is an unsigned int handle, not a struct)"

That the window is an int handle and not a struct explains why this version replaces the parent class
of XDrawable from ExternalStructur to Object. But all calls with an XDrawable still have this
object as parameter ( and not its instance var "xid"). I don't understand how this ever
worked (or was there some kind of automatic argument conversion like:
transform an object with an instvar -> a int, if the function expects an int handle?)

Sure we can replace all XDrawable arguments in X11 calls from "aXDrawable" to "aXDrawable xid".
But I only wonder, did this ever work and if Author "mtf" did the last change, can I contact
him or is he still maintaining the FF-Unix examples?


I would like to raise this topic again.

Nothing has changed, the linux example still don't work and I
can not get information about who maintains the unix-examples.

From my analysis (see above), the fix may be simple (in another discussion
around that time some one else asked how to make that working, and confirmed it
works with that fix). But I am confused because the latest update for the examples had
the comment "made the XLib examlpes actually work". So maybe my usage or version
of FFI is wrong.

The last time I asked, someone said this works for him with the current stable vm, but it was
actually the latest cog vm he used.

But now, a year later, with the current 4.6 cog and 5.0 spur vm, some FFI unit tests still failing
so maybe I am using the wrong FFI package (FFI-Kernel-EstebanLorenzano.27) ?

(BTW, there is a typo in
ExternalAddress class >> loadSymbol: moduleSymbol module: module
    <primitive: 'primitiveLoadSymbolFromModule' module: 'SqueakFFIPrims'>
    ^ self primitiveFail
it should call
    ^ self primitiveFailed



 


>
> regards
> Nicolai

Eliot (phone)