Re: [squeak-dev] Usability of Squeak FFI for anything serious...

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Usability of Squeak FFI for anything serious...

Eliot Miranda-2
 


On Fri, Apr 13, 2018 at 10:59 AM, Nicolas Cellier <[hidden email]> wrote:


2018-04-13 17:08 GMT+02:00 Eliot Miranda <[hidden email]>:
Hi Nicolas,


On Apr 13, 2018, at 7:01 AM, Nicolas Cellier <[hidden email]> wrote:

So I started to improve the Squeak FFI:
-1) I corrected pointer arithmetic on 64 bits http://source.squeak.org/FFI/FFI-Kernel-nice.48.diff
-2) I prepared the possibility of having automated layout spec updates http://source.squeak.org/FFI/FFI-Kernel-nice.49.diff

Note that the FFI plugin by itself did not change.
The next steps should be to provide support for native platform alignment of fields and unions.
I'll focus on these when i can.

I shall join you as soon as I am able.  Right now I have a pressing task, but within a few weeks I should be able to help.


Hi Eliot,
thanks.
I think that I can manage the easiest image side issues, but as soon as a feature will require an evolution of the plugin, your review is mandatory, as well as coordination with Pharo team, Backward compatibility counts and Esteban spent enough time on UFFI, so this will mean either unifying our forces or forking a plugin!

Sounds good.  A fork is to be avoided at all costs.  I don't see any pressure for a fork though because the plugin is essentially defined by the platform's ABI and Andreas' design for the type specification language is sufficiently general.  Am I missing something?  (Esteban are you reading?)

2018-04-12 23:27 GMT+02:00 Denis Kudriashov <[hidden email]>:
Hi

There is project TalkFFI from Ciprian Teodorov.

2018-04-12 22:50 GMT+02:00 Bert Freudenberg <[hidden email]>:
On 12 April 2018 at 22:34, Nicolas Cellier <[hidden email]> wrote:

There are at least two problems:
- documentation (lack of)
- limitations (unions, pointer on pointer on ...)
- automation (I said at least two, not at most)

For the first problem, I opened the SO issues. Not ideal, but it's a start.
For the second, I'll see if I can do something...
For the third, I did import some .h header with VW DLLCC, and I'm in the process of translating to other dialects.
It's easier because Smalltalk syntax is order of magnitudes smaller than C syntax.

​I always thought it should be fairly straightforward to make a FFI bindings generator using SWIG (http://www.swig.org/).

SWIG parses C headers and outputs XML. Starting from that XML should be way easier than trying to parse C ourselves.

​- Bert -​

 




















--
_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Usability of Squeak FFI for anything serious...

Nicolas Cellier
 


2018-04-13 21:28 GMT+02:00 Eliot Miranda <[hidden email]>:
 


On Fri, Apr 13, 2018 at 10:59 AM, Nicolas Cellier <[hidden email]> wrote:


2018-04-13 17:08 GMT+02:00 Eliot Miranda <[hidden email]>:
Hi Nicolas,


On Apr 13, 2018, at 7:01 AM, Nicolas Cellier <[hidden email]> wrote:

So I started to improve the Squeak FFI:
-1) I corrected pointer arithmetic on 64 bits http://source.squeak.org/FFI/FFI-Kernel-nice.48.diff
-2) I prepared the possibility of having automated layout spec updates http://source.squeak.org/FFI/FFI-Kernel-nice.49.diff

Note that the FFI plugin by itself did not change.
The next steps should be to provide support for native platform alignment of fields and unions.
I'll focus on these when i can.

I shall join you as soon as I am able.  Right now I have a pressing task, but within a few weeks I should be able to help.


Hi Eliot,
thanks.
I think that I can manage the easiest image side issues, but as soon as a feature will require an evolution of the plugin, your review is mandatory, as well as coordination with Pharo team, Backward compatibility counts and Esteban spent enough time on UFFI, so this will mean either unifying our forces or forking a plugin!

Sounds good.  A fork is to be avoided at all costs.  I don't see any pressure for a fork though because the plugin is essentially defined by the platform's ABI and Andreas' design for the type specification language is sufficiently general.  Am I missing something?  (Esteban are you reading?)


With my current understanding, two essential features are missing in the type compiledSpec:
- multiple indirection pointers (I've got an example of these in HDF5)
- fixed size arrays in struct members (I've got an example in HDF5 too!)

For now, we can workaround with
- void * declaration and human crafted pointer handling
- unrolled array members (1 member per array slot!)

A modification of FFI would be necessary only for enforcing type checking.
I don't plan anything right now, and I agree that doing so with backward compatibility is by far preferred.

2018-04-12 23:27 GMT+02:00 Denis Kudriashov <[hidden email]>:
Hi

There is project TalkFFI from Ciprian Teodorov.

2018-04-12 22:50 GMT+02:00 Bert Freudenberg <[hidden email]>:
On 12 April 2018 at 22:34, Nicolas Cellier <[hidden email]> wrote:

There are at least two problems:
- documentation (lack of)
- limitations (unions, pointer on pointer on ...)
- automation (I said at least two, not at most)

For the first problem, I opened the SO issues. Not ideal, but it's a start.
For the second, I'll see if I can do something...
For the third, I did import some .h header with VW DLLCC, and I'm in the process of translating to other dialects.
It's easier because Smalltalk syntax is order of magnitudes smaller than C syntax.

​I always thought it should be fairly straightforward to make a FFI bindings generator using SWIG (http://www.swig.org/).

SWIG parses C headers and outputs XML. Starting from that XML should be way easier than trying to parse C ourselves.

​- Bert -​

 




















--
_,,,^..^,,,_
best, Eliot


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Usability of Squeak FFI for anything serious...

Eliot Miranda-2
 
Hi Nicolas,

On Fri, Apr 13, 2018 at 12:40 PM, Nicolas Cellier <[hidden email]> wrote:

2018-04-13 21:28 GMT+02:00 Eliot Miranda <[hidden email]>:
 
On Fri, Apr 13, 2018 at 10:59 AM, Nicolas Cellier <[hidden email]> wrote:

2018-04-13 17:08 GMT+02:00 Eliot Miranda <[hidden email]>:
Hi Nicolas,

On Apr 13, 2018, at 7:01 AM, Nicolas Cellier <[hidden email]> wrote:
So I started to improve the Squeak FFI:
-1) I corrected pointer arithmetic on 64 bits http://source.squeak.org/FFI/FFI-Kernel-nice.48.diff
-2) I prepared the possibility of having automated layout spec updates http://source.squeak.org/FFI/FFI-Kernel-nice.49.diff

Note that the FFI plugin by itself did not change.
The next steps should be to provide support for native platform alignment of fields and unions.
I'll focus on these when i can.
I shall join you as soon as I am able.  Right now I have a pressing task, but within a few weeks I should be able to help.

Hi Eliot,
thanks.
I think that I can manage the easiest image side issues, but as soon as a feature will require an evolution of the plugin, your review is mandatory, as well as coordination with Pharo team, Backward compatibility counts and Esteban spent enough time on UFFI, so this will mean either unifying our forces or forking a plugin!

Sounds good.  A fork is to be avoided at all costs.  I don't see any pressure for a fork though because the plugin is essentially defined by the platform's ABI and Andreas' design for the type specification language is sufficiently general.  Am I missing something?  (Esteban are you reading?)

With my current understanding, two essential features are missing in the type compiledSpec:
- multiple indirection pointers (I've got an example of these in HDF5)

What do you mean, 64 vs 32 bit pointers or?
 
- fixed size arrays in struct members (I've got an example in HDF5 too!)

For now, we can workaround with
- void * declaration and human crafted pointer handling
- unrolled array members (1 member per array slot!)

I think unrolling might be a good solution.  There are relatively few types in a typical FFI interface and even fewer that have fixed size arrays in structs, so adding this would indeed create a few large specs, but in practice not many, whereas introducing support for arrays of types in the specs would add recursion into the marshaling in the plugin, and that could be error prone and slow.  Unrolling in the type to spec compiler (and re-rolling in printTypedefOn:) is probably simpler and I suspect would be far more robust.

A modification of FFI would be necessary only for enforcing type checking.

I'm not sure where this comes in if we unroll.  At least on callout the plugin merely checks that a structure is of the right size, and if specified the right subclass of ExternalStructure.  That's still sufficient if we realign instances of ExternalStructure on startup right?
 
I don't plan anything right now, and I agree that doing so with backward compatibility is by far preferred.

+1000.
2018-04-12 23:27 GMT+02:00 Denis Kudriashov <[hidden email]>:
Hi

There is project TalkFFI from Ciprian Teodorov.

2018-04-12 22:50 GMT+02:00 Bert Freudenberg <[hidden email]>:
On 12 April 2018 at 22:34, Nicolas Cellier <[hidden email]> wrote:

There are at least two problems:
- documentation (lack of)
- limitations (unions, pointer on pointer on ...)
- automation (I said at least two, not at most)

For the first problem, I opened the SO issues. Not ideal, but it's a start.
For the second, I'll see if I can do something...
For the third, I did import some .h header with VW DLLCC, and I'm in the process of translating to other dialects.
It's easier because Smalltalk syntax is order of magnitudes smaller than C syntax.

​I always thought it should be fairly straightforward to make a FFI bindings generator using SWIG (http://www.swig.org/).

SWIG parses C headers and outputs XML. Starting from that XML should be way easier than trying to parse C ourselves.

​- Bert -​
--
_,,,^..^,,,_
best, Eliot

_,,,^..^,,,_
best, Eliot