I think that I could throw a few more questions (dealing with enums...) but I don't want to excede some unknown quota and start irritating a touchy SO moderator ;)
I'm trying to see how hard it is to port my HDF5 work from VW to Squeak.
It was not a pleasure in VW, because that already raised some questions despite that there is a somehow detailed DLLCC manual:
But in Squeak, it sounds like hard++:
Either I'm blind, or the very light documentation that I found lacks those details
I'm afraid that there is no answer but "you're on your own"...
I'm open to studying Alien or UnifiedFFI of Pharo.
Or I can try and improve FFI.
At this stage, my trajectory has already been quite perturbed, and despite my knowledge of rocket science, I can see that I'm not going to put HDF5 on the intended orbit :(
On Wed, Apr 11, 2018 at 2:23 PM, Nicolas Cellier <[hidden email]> wrote:
What are you actually trying to do? Can you post the C definitions? I'm very familiar with both the SqueakFFI and DLLCC (as of 7.4.1) and the basic differences are to do with DLLCC's better cross-platform support, in that it auto-redefines typedefs on load. But other than that I think the SqueakFFI is either just as good, or in some cases better (e.g. DLLCC has a bug in that it is the actual parameter that carries detailed type information for a struct, whereas in Andreas' SqueakFFI it is, correctly, the formal parameter).
2018-04-12 3:10 GMT+02:00 Eliot Miranda <[hidden email]>:
the library is huge (HDF5).
All the interfacing code is available at cincom public store (HDF5 bundle)
To get an idea, you can inspect a clone of HDF5 library at https://github.com/live-clones/hdf5/tree/develop/src
The main structures/enum/typedefs/prototypes are in the H5*public.h files
The problems are all exposed in SO questions, but I can try and priotitize.
The essential barrier is that those headers use almost exclusively typedef'ed types.
Library is too huge for manual subsitution; this must be mechanized.
I would prefer some kind of support in FFI rather than having to hack own fragile ad-hoc pre-processor.
Then I have a problem because of alignment of struct fields which is compact in Squeak FFI rather than conforming to platform conventions.
Same here, manually hacking with field size is a no go, this must be mechanized.
Then some unions are used too and there is no support at all in FFI.
Then definitions like size_t are dependent on machine word size (32 or 64 bits), and there is no support for that in FFI.
I'll have to duplicate some work for supporting both 32 and 64 bits Squeak, or explore new hacks...
But there is no support and no documentation (how to).
Then while digging in source coden I did not see any double pointer support in struct fields.
The example I provide on SO shows what I would qualify as a bug.
I don't know yet whether I'll need those, but it's not good.
Maybe I can shoot each problem with more or less ugly workarounds.
At the end i can declare void * and deal with offset directly, but that's both:
- a lot of work
- a good recipe for generating very hard to find bugs - crashes if lucky, silent corruption if not
(dealing with offsets is like writing in assembler instead of C)
Given the size of the library, the probability of not creating such fault in manual translation is 0%
Plus, there are a few evolutions from one version of the library to the other...
That's my definition of hard++
On 12 April 2018 at 16:00, Nicolas Cellier <[hidden email]> wrote:
Maybe more of a distraction than a help, but browsing around because its an interesting topic,
2018-04-12 15:24 GMT+02:00 Ben Coman <[hidden email]>:
no time to read now, but I keep your pointers preciously :)
In the interim, I found how to deal with aliases (typedef) and answerd in SO
There is then a problem of initialization order which is not yet solved though...
That will partly solve my problem of portability 32/64 bits.
But maybe not the problem of LLP64 vs ILP64, some native unsigned long being used...
(I'll have to redefine some ExternalStructure when detecting a change of platform...)
In reply to this post by Nicolas Cellier
I've had to deal with a variety of libraries (OpenGL, databases, etc) and have yet to run into a situation where I can't make things work with vanilla FFI (with one caveat: I haven't needed callback support which has been discussed several times here and on vmdev and solutions exists, I just haven't needed them to date.) But this has required writing a variety of preprocessors (for static APIs) or bridges (for dynamic) as I'm not aware of any tooling to universally deal with headers/bridging (FFI doesn't help you with header files etc. so you will have to deal with things like unions in your preprocessor). 32/64-bit is pretty easily dealt with provided you check Smalltalk>>wordSize and adjust as needed (i.e. it can be dealt with statically or dynamically depending on your needs.) The way to think of FFI is as just providing the lowest level plumbing materials that allows the image to communicate with the outside world. (i.e. pipes, elbows, etc.. but you have to assemble it and supply the liquid) I'm not saying that this is ideal... it would be great if one could point to an arbitrary API and say 'FFI, figure it out' but thats not what exists. What we have currently does the job but does require some work.
Just my $0.02,
On Thu, Apr 12, 2018, 4:00 AM Nicolas Cellier <[hidden email]> wrote:
2018-04-12 21:09 GMT+02:00 Phil B <[hidden email]>:
$0.02 taken :)
There are at least two problems:
- documentation (lack of)
- limitations (unions, pointer on pointer on ...)
- automation (I said at least two, not at most)
For the first problem, I opened the SO issues. Not ideal, but it's a start.
For the second, I'll see if I can do something...
For the third, I did import some .h header with VW DLLCC, and I'm in the process of translating to other dialects.
It's easier because Smalltalk syntax is order of magnitudes smaller than C syntax.
I always thought it should be fairly straightforward to make a FFI bindings generator using SWIG (http://www.swig.org/).
SWIG parses C headers and outputs XML. Starting from that XML should be way easier than trying to parse C ourselves.
- Bert -
2018-04-12 22:50 GMT+02:00 Bert Freudenberg <[hidden email]>:
I'll focus on these when i can.
The next steps should be to provide support for native platform alignment of fields and unions.
Note that the FFI plugin by itself did not change.
-2) I prepared the possibility of having automated layout spec updates http://source.squeak.org/FFI/FFI-Kernel-nice.49.diff
So I started to improve the Squeak FFI:-1) I corrected pointer arithmetic on 64 bits http://source.squeak.org/FFI/FFI-Kernel-nice.48.diff
2018-04-12 23:27 GMT+02:00 Denis Kudriashov <[hidden email]>:
On Apr 13, 2018, at 7:01 AM, Nicolas Cellier <[hidden email]> wrote:
I shall join you as soon as I am able. Right now I have a pressing task, but within a few weeks I should be able to help.
2018-04-13 17:08 GMT+02:00 Eliot Miranda <[hidden email]>:
I think that I can manage the easiest image side issues, but as soon as a feature will require an evolution of the plugin, your review is mandatory, as well as coordination with Pharo team, Backward compatibility counts and Esteban spent enough time on UFFI, so this will mean either unifying our forces or forking a plugin!
|Free forum by Nabble||Edit this page|