Usability of Squeak FFI for anything serious...

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Usability of Squeak FFI for anything serious...

Nicolas Cellier
I'm trying to see how hard it is to port my HDF5 work from VW to Squeak.
It was not a pleasure in VW, because that already raised some questions despite that there is a somehow detailed DLLCC manual:

https://stackoverflow.com/questions/49544642/why-cant-i-pass-an-uninterpretedbytes-to-a-void-thru-dll-c-connect
https://stackoverflow.com/questions/49564024/isnt-pointer-type-checking-disabled-in-dll-c-connect-and-is-that-ok

But in Squeak, it sounds like hard++:

https://stackoverflow.com/questions/49782651/how-one-aligns-structure-fields-in-squeak-ffi
https://stackoverflow.com/questions/49783126/how-one-defines-a-union-type-in-squeak-ffi
https://stackoverflow.com/questions/49783443/how-one-defines-a-fixed-size-array-member-in-a-struct-in-squeak-ffi
https://stackoverflow.com/questions/49783882/how-one-deals-with-typedefs-in-squeak-ffi
https://stackoverflow.com/questions/49784253/how-one-deals-with-multiple-pointer-level-like-char-in-squeak-ffi
https://stackoverflow.com/questions/49784522/how-one-supports-both-32-and-64-bits-target-in-squeak-ffi

Either I'm blind, or the very light documentation that I found lacks those details
http://wiki.squeak.org/squeak/2426

I think that I could throw a few more questions (dealing with enums...) but I don't want to excede some unknown quota and start irritating a touchy SO moderator ;)

I'm afraid that there is no answer but "you're on your own"...
I'm open to studying Alien or UnifiedFFI of Pharo.
Or I can try and improve FFI.
At this stage, my trajectory has already been quite perturbed, and despite my knowledge of rocket science, I can see that I'm not going to put HDF5 on the intended orbit :(

Thoughts?




Reply | Threaded
Open this post in threaded view
|

Re: Usability of Squeak FFI for anything serious...

Eliot Miranda-2
Hi Nicolas,

On Wed, Apr 11, 2018 at 2:23 PM, Nicolas Cellier <[hidden email]> wrote:
I'm trying to see how hard it is to port my HDF5 work from VW to Squeak.
It was not a pleasure in VW, because that already raised some questions despite that there is a somehow detailed DLLCC manual:

https://stackoverflow.com/questions/49544642/why-cant-i-pass-an-uninterpretedbytes-to-a-void-thru-dll-c-connect
https://stackoverflow.com/questions/49564024/isnt-pointer-type-checking-disabled-in-dll-c-connect-and-is-that-ok

What are you actually trying to do?  Can you post the C definitions?  I'm very familiar with both the SqueakFFI and DLLCC (as of 7.4.1) and the basic differences are to do with DLLCC's better cross-platform support, in that it auto-redefines typedefs on load.  But other than that I think the SqueakFFI is either just as good, or in some cases better (e.g. DLLCC has a bug in that it is the actual parameter that carries detailed type information for a struct, whereas in Andreas' SqueakFFI it is, correctly, the formal parameter).

 
I think that I could throw a few more questions (dealing with enums...) but I don't want to excede some unknown quota and start irritating a touchy SO moderator ;)

I'm afraid that there is no answer but "you're on your own"...
I'm open to studying Alien or UnifiedFFI of Pharo.
Or I can try and improve FFI.
At this stage, my trajectory has already been quite perturbed, and despite my knowledge of rocket science, I can see that I'm not going to put HDF5 on the intended orbit :(

Thoughts?








--
_,,,^..^,,,_
best, Eliot


Reply | Threaded
Open this post in threaded view
|

Re: Usability of Squeak FFI for anything serious...

Nicolas Cellier


2018-04-12 3:10 GMT+02:00 Eliot Miranda <[hidden email]>:
Hi Nicolas,

On Wed, Apr 11, 2018 at 2:23 PM, Nicolas Cellier <[hidden email]> wrote:
I'm trying to see how hard it is to port my HDF5 work from VW to Squeak.
It was not a pleasure in VW, because that already raised some questions despite that there is a somehow detailed DLLCC manual:

https://stackoverflow.com/questions/49544642/why-cant-i-pass-an-uninterpretedbytes-to-a-void-thru-dll-c-connect
https://stackoverflow.com/questions/49564024/isnt-pointer-type-checking-disabled-in-dll-c-connect-and-is-that-ok

What are you actually trying to do?  Can you post the C definitions?  I'm very familiar with both the SqueakFFI and DLLCC (as of 7.4.1) and the basic differences are to do with DLLCC's better cross-platform support, in that it auto-redefines typedefs on load.  But other than that I think the SqueakFFI is either just as good, or in some cases better (e.g. DLLCC has a bug in that it is the actual parameter that carries detailed type information for a struct, whereas in Andreas' SqueakFFI it is, correctly, the formal parameter).

Hi Eliot,
the library is huge (HDF5).
All the interfacing code is available at cincom public store (HDF5 bundle)
To get an idea, you can inspect a clone of HDF5 library at https://github.com/live-clones/hdf5/tree/develop/src
The main structures/enum/typedefs/prototypes are in the H5*public.h files

The problems are all exposed in SO questions, but I can try and priotitize.

The essential barrier is that those headers use almost exclusively typedef'ed types.
Library is too huge for manual subsitution; this must be mechanized.
I would prefer some kind of support in FFI rather than having to hack own fragile ad-hoc pre-processor.

Then I have a problem because of alignment of struct fields which is compact in Squeak FFI rather than conforming to platform conventions.
Same here, manually hacking with field size is a no go, this must be mechanized.

Then some unions are used too and there is no support at all in FFI.

Then definitions like size_t are dependent on machine word size (32 or 64 bits), and there is no support for that in FFI.
I'll have to duplicate some work for supporting both 32 and 64 bits Squeak, or explore new hacks...
But there is no support and no documentation (how to).

Then while digging in source coden I did not see any double pointer support in struct fields.
The example I provide on SO shows what I would qualify as a bug.
I don't know yet whether I'll need those, but it's not good.

Maybe I can shoot each problem with more or less ugly workarounds.
At the end i can declare void * and deal with offset directly, but that's both:
- a lot of work
- a good recipe for generating very hard to find bugs - crashes if lucky, silent corruption if not
 (dealing with offsets is like writing in assembler instead of C)

Given the size of the library, the probability of not creating such fault in manual translation is 0%
Plus, there are a few evolutions from one version of the library to the other...

That's my definition of hard++

 
I think that I could throw a few more questions (dealing with enums...) but I don't want to excede some unknown quota and start irritating a touchy SO moderator ;)

I'm afraid that there is no answer but "you're on your own"...
I'm open to studying Alien or UnifiedFFI of Pharo.
Or I can try and improve FFI.
At this stage, my trajectory has already been quite perturbed, and despite my knowledge of rocket science, I can see that I'm not going to put HDF5 on the intended orbit :(

Thoughts?








--
_,,,^..^,,,_
best, Eliot






Reply | Threaded
Open this post in threaded view
|

Re: Usability of Squeak FFI for anything serious...

Ben Coman


On 12 April 2018 at 16:00, Nicolas Cellier <[hidden email]> wrote:


2018-04-12 3:10 GMT+02:00 Eliot Miranda <[hidden email]>:
Hi Nicolas,

On Wed, Apr 11, 2018 at 2:23 PM, Nicolas Cellier <[hidden email]> wrote:
I'm trying to see how hard it is to port my HDF5 work from VW to Squeak.
It was not a pleasure in VW, because that already raised some questions despite that there is a somehow detailed DLLCC manual:

https://stackoverflow.com/questions/49544642/why-cant-i-pass-an-uninterpretedbytes-to-a-void-thru-dll-c-connect
https://stackoverflow.com/questions/49564024/isnt-pointer-type-checking-disabled-in-dll-c-connect-and-is-that-ok

What are you actually trying to do?  Can you post the C definitions?  I'm very familiar with both the SqueakFFI and DLLCC (as of 7.4.1) and the basic differences are to do with DLLCC's better cross-platform support, in that it auto-redefines typedefs on load.  But other than that I think the SqueakFFI is either just as good, or in some cases better (e.g. DLLCC has a bug in that it is the actual parameter that carries detailed type information for a struct, whereas in Andreas' SqueakFFI it is, correctly, the formal parameter).

Hi Eliot,
the library is huge (HDF5).
All the interfacing code is available at cincom public store (HDF5 bundle)
To get an idea, you can inspect a clone of HDF5 library at https://github.com/live-clones/hdf5/tree/develop/src
The main structures/enum/typedefs/prototypes are in the H5*public.h files

The problems are all exposed in SO questions, but I can try and priotitize.

The essential barrier is that those headers use almost exclusively typedef'ed types.
Library is too huge for manual subsitution; this must be mechanized.
I would prefer some kind of support in FFI rather than having to hack own fragile ad-hoc pre-processor.

Maybe more of a distraction than a help, but browsing around because its an interesting topic, 
some discoveries...

cheers -ben

 

Then I have a problem because of alignment of struct fields which is compact in Squeak FFI rather than conforming to platform conventions.
Same here, manually hacking with field size is a no go, this must be mechanized.

Then some unions are used too and there is no support at all in FFI.

Then definitions like size_t are dependent on machine word size (32 or 64 bits), and there is no support for that in FFI.
I'll have to duplicate some work for supporting both 32 and 64 bits Squeak, or explore new hacks...
But there is no support and no documentation (how to).

Then while digging in source coden I did not see any double pointer support in struct fields.
The example I provide on SO shows what I would qualify as a bug.
I don't know yet whether I'll need those, but it's not good.

Maybe I can shoot each problem with more or less ugly workarounds.
At the end i can declare void * and deal with offset directly, but that's both:
- a lot of work
- a good recipe for generating very hard to find bugs - crashes if lucky, silent corruption if not
 (dealing with offsets is like writing in assembler instead of C)

Given the size of the library, the probability of not creating such fault in manual translation is 0%
Plus, there are a few evolutions from one version of the library to the other...

That's my definition of hard++

 
I think that I could throw a few more questions (dealing with enums...) but I don't want to excede some unknown quota and start irritating a touchy SO moderator ;)

I'm afraid that there is no answer but "you're on your own"...
I'm open to studying Alien or UnifiedFFI of Pharo.
Or I can try and improve FFI.
At this stage, my trajectory has already been quite perturbed, and despite my knowledge of rocket science, I can see that I'm not going to put HDF5 on the intended orbit :(

Thoughts?








--
_,,,^..^,,,_
best, Eliot










Reply | Threaded
Open this post in threaded view
|

Re: Usability of Squeak FFI for anything serious...

Nicolas Cellier


2018-04-12 15:24 GMT+02:00 Ben Coman <[hidden email]>:


On 12 April 2018 at 16:00, Nicolas Cellier <[hidden email]> wrote:


2018-04-12 3:10 GMT+02:00 Eliot Miranda <[hidden email]>:
Hi Nicolas,

On Wed, Apr 11, 2018 at 2:23 PM, Nicolas Cellier <[hidden email]> wrote:
I'm trying to see how hard it is to port my HDF5 work from VW to Squeak.
It was not a pleasure in VW, because that already raised some questions despite that there is a somehow detailed DLLCC manual:

https://stackoverflow.com/questions/49544642/why-cant-i-pass-an-uninterpretedbytes-to-a-void-thru-dll-c-connect
https://stackoverflow.com/questions/49564024/isnt-pointer-type-checking-disabled-in-dll-c-connect-and-is-that-ok

What are you actually trying to do?  Can you post the C definitions?  I'm very familiar with both the SqueakFFI and DLLCC (as of 7.4.1) and the basic differences are to do with DLLCC's better cross-platform support, in that it auto-redefines typedefs on load.  But other than that I think the SqueakFFI is either just as good, or in some cases better (e.g. DLLCC has a bug in that it is the actual parameter that carries detailed type information for a struct, whereas in Andreas' SqueakFFI it is, correctly, the formal parameter).

Hi Eliot,
the library is huge (HDF5).
All the interfacing code is available at cincom public store (HDF5 bundle)
To get an idea, you can inspect a clone of HDF5 library at https://github.com/live-clones/hdf5/tree/develop/src
The main structures/enum/typedefs/prototypes are in the H5*public.h files

The problems are all exposed in SO questions, but I can try and priotitize.

The essential barrier is that those headers use almost exclusively typedef'ed types.
Library is too huge for manual subsitution; this must be mechanized.
I would prefer some kind of support in FFI rather than having to hack own fragile ad-hoc pre-processor.

Maybe more of a distraction than a help, but browsing around because its an interesting topic, 
some discoveries...

cheers -ben

 
Hi Ben,
no time to read now, but I keep your pointers preciously :)
In the interim, I found how to deal with aliases (typedef) and answerd in SO
https://stackoverflow.com/questions/49783882/how-one-deals-with-typedefs-in-squeak-ffi
There is then a problem of initialization order which is not yet solved though...

But maybe not the problem of LLP64 vs ILP64, some native unsigned long being used...
(I'll have to redefine some ExternalStructure when detecting a change of platform...)

 

Then I have a problem because of alignment of struct fields which is compact in Squeak FFI rather than conforming to platform conventions.
Same here, manually hacking with field size is a no go, this must be mechanized.

Then some unions are used too and there is no support at all in FFI.

Then definitions like size_t are dependent on machine word size (32 or 64 bits), and there is no support for that in FFI.
I'll have to duplicate some work for supporting both 32 and 64 bits Squeak, or explore new hacks...
But there is no support and no documentation (how to).

Then while digging in source coden I did not see any double pointer support in struct fields.
The example I provide on SO shows what I would qualify as a bug.
I don't know yet whether I'll need those, but it's not good.

Maybe I can shoot each problem with more or less ugly workarounds.
At the end i can declare void * and deal with offset directly, but that's both:
- a lot of work
- a good recipe for generating very hard to find bugs - crashes if lucky, silent corruption if not
 (dealing with offsets is like writing in assembler instead of C)

Given the size of the library, the probability of not creating such fault in manual translation is 0%
Plus, there are a few evolutions from one version of the library to the other...

That's my definition of hard++

 
I think that I could throw a few more questions (dealing with enums...) but I don't want to excede some unknown quota and start irritating a touchy SO moderator ;)

I'm afraid that there is no answer but "you're on your own"...
I'm open to studying Alien or UnifiedFFI of Pharo.
Or I can try and improve FFI.
At this stage, my trajectory has already been quite perturbed, and despite my knowledge of rocket science, I can see that I'm not going to put HDF5 on the intended orbit :(

Thoughts?








--
_,,,^..^,,,_
best, Eliot














Reply | Threaded
Open this post in threaded view
|

Re: Usability of Squeak FFI for anything serious...

Phil B
In reply to this post by Nicolas Cellier
Nicolas,

I've had to deal with a variety of libraries (OpenGL, databases, etc) and have yet to run into a situation where I can't make things work with vanilla FFI (with one caveat: I haven't needed callback support which has been discussed several times here and on vmdev and solutions exists, I just haven't needed them to date.)  But this has required writing a variety of preprocessors (for static APIs) or bridges (for dynamic) as I'm not aware of any tooling to universally deal with headers/bridging (FFI doesn't help you with header files etc. so you will have to deal with things like unions in your preprocessor). 32/64-bit is pretty easily dealt with provided you check Smalltalk>>wordSize and adjust as needed (i.e. it can be dealt with statically or dynamically depending on your needs.)  The way to think of FFI is as just providing the lowest level plumbing materials that allows the image to communicate with the outside world.  (i.e. pipes, elbows, etc.. but you have to assemble it and supply the liquid)  I'm not saying that this is ideal... it would be great if one could point to an arbitrary API and say 'FFI, figure it out' but thats not what exists.  What we have currently does the job but does require some work.

Just my $0.02,
Phil

On Thu, Apr 12, 2018, 4:00 AM Nicolas Cellier <[hidden email]> wrote:


2018-04-12 3:10 GMT+02:00 Eliot Miranda <[hidden email]>:
Hi Nicolas,

On Wed, Apr 11, 2018 at 2:23 PM, Nicolas Cellier <[hidden email]> wrote:
I'm trying to see how hard it is to port my HDF5 work from VW to Squeak.
It was not a pleasure in VW, because that already raised some questions despite that there is a somehow detailed DLLCC manual:

https://stackoverflow.com/questions/49544642/why-cant-i-pass-an-uninterpretedbytes-to-a-void-thru-dll-c-connect
https://stackoverflow.com/questions/49564024/isnt-pointer-type-checking-disabled-in-dll-c-connect-and-is-that-ok

What are you actually trying to do?  Can you post the C definitions?  I'm very familiar with both the SqueakFFI and DLLCC (as of 7.4.1) and the basic differences are to do with DLLCC's better cross-platform support, in that it auto-redefines typedefs on load.  But other than that I think the SqueakFFI is either just as good, or in some cases better (e.g. DLLCC has a bug in that it is the actual parameter that carries detailed type information for a struct, whereas in Andreas' SqueakFFI it is, correctly, the formal parameter).

Hi Eliot,
the library is huge (HDF5).
All the interfacing code is available at cincom public store (HDF5 bundle)
To get an idea, you can inspect a clone of HDF5 library at https://github.com/live-clones/hdf5/tree/develop/src
The main structures/enum/typedefs/prototypes are in the H5*public.h files

The problems are all exposed in SO questions, but I can try and priotitize.

The essential barrier is that those headers use almost exclusively typedef'ed types.
Library is too huge for manual subsitution; this must be mechanized.
I would prefer some kind of support in FFI rather than having to hack own fragile ad-hoc pre-processor.

Then I have a problem because of alignment of struct fields which is compact in Squeak FFI rather than conforming to platform conventions.
Same here, manually hacking with field size is a no go, this must be mechanized.

Then some unions are used too and there is no support at all in FFI.

Then definitions like size_t are dependent on machine word size (32 or 64 bits), and there is no support for that in FFI.
I'll have to duplicate some work for supporting both 32 and 64 bits Squeak, or explore new hacks...
But there is no support and no documentation (how to).

Then while digging in source coden I did not see any double pointer support in struct fields.
The example I provide on SO shows what I would qualify as a bug.
I don't know yet whether I'll need those, but it's not good.

Maybe I can shoot each problem with more or less ugly workarounds.
At the end i can declare void * and deal with offset directly, but that's both:
- a lot of work
- a good recipe for generating very hard to find bugs - crashes if lucky, silent corruption if not
 (dealing with offsets is like writing in assembler instead of C)

Given the size of the library, the probability of not creating such fault in manual translation is 0%
Plus, there are a few evolutions from one version of the library to the other...

That's my definition of hard++

 
I think that I could throw a few more questions (dealing with enums...) but I don't want to excede some unknown quota and start irritating a touchy SO moderator ;)

I'm afraid that there is no answer but "you're on your own"...
I'm open to studying Alien or UnifiedFFI of Pharo.
Or I can try and improve FFI.
At this stage, my trajectory has already been quite perturbed, and despite my knowledge of rocket science, I can see that I'm not going to put HDF5 on the intended orbit :(

Thoughts?








--
_,,,^..^,,,_
best, Eliot







Reply | Threaded
Open this post in threaded view
|

Re: Usability of Squeak FFI for anything serious...

Nicolas Cellier


2018-04-12 21:09 GMT+02:00 Phil B <[hidden email]>:
Nicolas,

I've had to deal with a variety of libraries (OpenGL, databases, etc) and have yet to run into a situation where I can't make things work with vanilla FFI (with one caveat: I haven't needed callback support which has been discussed several times here and on vmdev and solutions exists, I just haven't needed them to date.)  But this has required writing a variety of preprocessors (for static APIs) or bridges (for dynamic) as I'm not aware of any tooling to universally deal with headers/bridging (FFI doesn't help you with header files etc. so you will have to deal with things like unions in your preprocessor). 32/64-bit is pretty easily dealt with provided you check Smalltalk>>wordSize and adjust as needed (i.e. it can be dealt with statically or dynamically depending on your needs.)  The way to think of FFI is as just providing the lowest level plumbing materials that allows the image to communicate with the outside world.  (i.e. pipes, elbows, etc.. but you have to assemble it and supply the liquid)  I'm not saying that this is ideal... it would be great if one could point to an arbitrary API and say 'FFI, figure it out' but thats not what exists.  What we have currently does the job but does require some work.

Just my $0.02,
Phil


$0.02 taken :)
There are at least two problems:
- documentation (lack of)
- limitations (unions, pointer on pointer on ...)
- automation (I said at least two, not at most)

For the first problem, I opened the SO issues. Not ideal, but it's a start.
For the second, I'll see if I can do something...
For the third, I did import some .h header with VW DLLCC, and I'm in the process of translating to other dialects.
It's easier because Smalltalk syntax is order of magnitudes smaller than C syntax.

cheers
 

On Thu, Apr 12, 2018, 4:00 AM Nicolas Cellier <[hidden email]> wrote:


2018-04-12 3:10 GMT+02:00 Eliot Miranda <[hidden email]>:
Hi Nicolas,

On Wed, Apr 11, 2018 at 2:23 PM, Nicolas Cellier <[hidden email]> wrote:
I'm trying to see how hard it is to port my HDF5 work from VW to Squeak.
It was not a pleasure in VW, because that already raised some questions despite that there is a somehow detailed DLLCC manual:

https://stackoverflow.com/questions/49544642/why-cant-i-pass-an-uninterpretedbytes-to-a-void-thru-dll-c-connect
https://stackoverflow.com/questions/49564024/isnt-pointer-type-checking-disabled-in-dll-c-connect-and-is-that-ok

What are you actually trying to do?  Can you post the C definitions?  I'm very familiar with both the SqueakFFI and DLLCC (as of 7.4.1) and the basic differences are to do with DLLCC's better cross-platform support, in that it auto-redefines typedefs on load.  But other than that I think the SqueakFFI is either just as good, or in some cases better (e.g. DLLCC has a bug in that it is the actual parameter that carries detailed type information for a struct, whereas in Andreas' SqueakFFI it is, correctly, the formal parameter).

Hi Eliot,
the library is huge (HDF5).
All the interfacing code is available at cincom public store (HDF5 bundle)
To get an idea, you can inspect a clone of HDF5 library at https://github.com/live-clones/hdf5/tree/develop/src
The main structures/enum/typedefs/prototypes are in the H5*public.h files

The problems are all exposed in SO questions, but I can try and priotitize.

The essential barrier is that those headers use almost exclusively typedef'ed types.
Library is too huge for manual subsitution; this must be mechanized.
I would prefer some kind of support in FFI rather than having to hack own fragile ad-hoc pre-processor.

Then I have a problem because of alignment of struct fields which is compact in Squeak FFI rather than conforming to platform conventions.
Same here, manually hacking with field size is a no go, this must be mechanized.

Then some unions are used too and there is no support at all in FFI.

Then definitions like size_t are dependent on machine word size (32 or 64 bits), and there is no support for that in FFI.
I'll have to duplicate some work for supporting both 32 and 64 bits Squeak, or explore new hacks...
But there is no support and no documentation (how to).

Then while digging in source coden I did not see any double pointer support in struct fields.
The example I provide on SO shows what I would qualify as a bug.
I don't know yet whether I'll need those, but it's not good.

Maybe I can shoot each problem with more or less ugly workarounds.
At the end i can declare void * and deal with offset directly, but that's both:
- a lot of work
- a good recipe for generating very hard to find bugs - crashes if lucky, silent corruption if not
 (dealing with offsets is like writing in assembler instead of C)

Given the size of the library, the probability of not creating such fault in manual translation is 0%
Plus, there are a few evolutions from one version of the library to the other...

That's my definition of hard++

 
I think that I could throw a few more questions (dealing with enums...) but I don't want to excede some unknown quota and start irritating a touchy SO moderator ;)

I'm afraid that there is no answer but "you're on your own"...
I'm open to studying Alien or UnifiedFFI of Pharo.
Or I can try and improve FFI.
At this stage, my trajectory has already been quite perturbed, and despite my knowledge of rocket science, I can see that I'm not going to put HDF5 on the intended orbit :(

Thoughts?








--
_,,,^..^,,,_
best, Eliot











Reply | Threaded
Open this post in threaded view
|

Re: Usability of Squeak FFI for anything serious...

Bert Freudenberg
On 12 April 2018 at 22:34, Nicolas Cellier <[hidden email]> wrote:

There are at least two problems:
- documentation (lack of)
- limitations (unions, pointer on pointer on ...)
- automation (I said at least two, not at most)

For the first problem, I opened the SO issues. Not ideal, but it's a start.
For the second, I'll see if I can do something...
For the third, I did import some .h header with VW DLLCC, and I'm in the process of translating to other dialects.
It's easier because Smalltalk syntax is order of magnitudes smaller than C syntax.

​I always thought it should be fairly straightforward to make a FFI bindings generator using SWIG (http://www.swig.org/).

SWIG parses C headers and outputs XML. Starting from that XML should be way easier than trying to parse C ourselves.

​- Bert -​

 



Reply | Threaded
Open this post in threaded view
|

Re: Usability of Squeak FFI for anything serious...

Denis Kudriashov
Hi

There is project TalkFFI from Ciprian Teodorov.

2018-04-12 22:50 GMT+02:00 Bert Freudenberg <[hidden email]>:
On 12 April 2018 at 22:34, Nicolas Cellier <[hidden email]> wrote:

There are at least two problems:
- documentation (lack of)
- limitations (unions, pointer on pointer on ...)
- automation (I said at least two, not at most)

For the first problem, I opened the SO issues. Not ideal, but it's a start.
For the second, I'll see if I can do something...
For the third, I did import some .h header with VW DLLCC, and I'm in the process of translating to other dialects.
It's easier because Smalltalk syntax is order of magnitudes smaller than C syntax.

​I always thought it should be fairly straightforward to make a FFI bindings generator using SWIG (http://www.swig.org/).

SWIG parses C headers and outputs XML. Starting from that XML should be way easier than trying to parse C ourselves.

​- Bert -​

 







Reply | Threaded
Open this post in threaded view
|

Re: Usability of Squeak FFI for anything serious...

Nicolas Cellier
So I started to improve the Squeak FFI:
-1) I corrected pointer arithmetic on 64 bits http://source.squeak.org/FFI/FFI-Kernel-nice.48.diff
-2) I prepared the possibility of having automated layout spec updates http://source.squeak.org/FFI/FFI-Kernel-nice.49.diff

Note that the FFI plugin by itself did not change.
The next steps should be to provide support for native platform alignment of fields and unions.
I'll focus on these when i can.


2018-04-12 23:27 GMT+02:00 Denis Kudriashov <[hidden email]>:
Hi

There is project TalkFFI from Ciprian Teodorov.

2018-04-12 22:50 GMT+02:00 Bert Freudenberg <[hidden email]>:
On 12 April 2018 at 22:34, Nicolas Cellier <[hidden email]> wrote:

There are at least two problems:
- documentation (lack of)
- limitations (unions, pointer on pointer on ...)
- automation (I said at least two, not at most)

For the first problem, I opened the SO issues. Not ideal, but it's a start.
For the second, I'll see if I can do something...
For the third, I did import some .h header with VW DLLCC, and I'm in the process of translating to other dialects.
It's easier because Smalltalk syntax is order of magnitudes smaller than C syntax.

​I always thought it should be fairly straightforward to make a FFI bindings generator using SWIG (http://www.swig.org/).

SWIG parses C headers and outputs XML. Starting from that XML should be way easier than trying to parse C ourselves.

​- Bert -​

 











Reply | Threaded
Open this post in threaded view
|

Re: Usability of Squeak FFI for anything serious...

Eliot Miranda-2
Hi Nicolas,


On Apr 13, 2018, at 7:01 AM, Nicolas Cellier <[hidden email]> wrote:

So I started to improve the Squeak FFI:
-1) I corrected pointer arithmetic on 64 bits http://source.squeak.org/FFI/FFI-Kernel-nice.48.diff
-2) I prepared the possibility of having automated layout spec updates http://source.squeak.org/FFI/FFI-Kernel-nice.49.diff

Note that the FFI plugin by itself did not change.
The next steps should be to provide support for native platform alignment of fields and unions.
I'll focus on these when i can.

I shall join you as soon as I am able.  Right now I have a pressing task, but within a few weeks I should be able to help.



2018-04-12 23:27 GMT+02:00 Denis Kudriashov <[hidden email]>:
Hi

There is project TalkFFI from Ciprian Teodorov.

2018-04-12 22:50 GMT+02:00 Bert Freudenberg <[hidden email]>:
On 12 April 2018 at 22:34, Nicolas Cellier <[hidden email]> wrote:

There are at least two problems:
- documentation (lack of)
- limitations (unions, pointer on pointer on ...)
- automation (I said at least two, not at most)

For the first problem, I opened the SO issues. Not ideal, but it's a start.
For the second, I'll see if I can do something...
For the third, I did import some .h header with VW DLLCC, and I'm in the process of translating to other dialects.
It's easier because Smalltalk syntax is order of magnitudes smaller than C syntax.

​I always thought it should be fairly straightforward to make a FFI bindings generator using SWIG (http://www.swig.org/).

SWIG parses C headers and outputs XML. Starting from that XML should be way easier than trying to parse C ourselves.

​- Bert -​

 












Reply | Threaded
Open this post in threaded view
|

Re: Usability of Squeak FFI for anything serious...

Nicolas Cellier


2018-04-13 17:08 GMT+02:00 Eliot Miranda <[hidden email]>:
Hi Nicolas,


On Apr 13, 2018, at 7:01 AM, Nicolas Cellier <[hidden email]> wrote:

So I started to improve the Squeak FFI:
-1) I corrected pointer arithmetic on 64 bits http://source.squeak.org/FFI/FFI-Kernel-nice.48.diff
-2) I prepared the possibility of having automated layout spec updates http://source.squeak.org/FFI/FFI-Kernel-nice.49.diff

Note that the FFI plugin by itself did not change.
The next steps should be to provide support for native platform alignment of fields and unions.
I'll focus on these when i can.

I shall join you as soon as I am able.  Right now I have a pressing task, but within a few weeks I should be able to help.


Hi Eliot,
thanks.
I think that I can manage the easiest image side issues, but as soon as a feature will require an evolution of the plugin, your review is mandatory, as well as coordination with Pharo team, Backward compatibility counts and Esteban spent enough time on UFFI, so this will mean either unifying our forces or forking a plugin!


2018-04-12 23:27 GMT+02:00 Denis Kudriashov <[hidden email]>:
Hi

There is project TalkFFI from Ciprian Teodorov.

2018-04-12 22:50 GMT+02:00 Bert Freudenberg <[hidden email]>:
On 12 April 2018 at 22:34, Nicolas Cellier <[hidden email]> wrote:

There are at least two problems:
- documentation (lack of)
- limitations (unions, pointer on pointer on ...)
- automation (I said at least two, not at most)

For the first problem, I opened the SO issues. Not ideal, but it's a start.
For the second, I'll see if I can do something...
For the third, I did import some .h header with VW DLLCC, and I'm in the process of translating to other dialects.
It's easier because Smalltalk syntax is order of magnitudes smaller than C syntax.

​I always thought it should be fairly straightforward to make a FFI bindings generator using SWIG (http://www.swig.org/).

SWIG parses C headers and outputs XML. Starting from that XML should be way easier than trying to parse C ourselves.

​- Bert -​