UFFI and opaque(ish) types

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

UFFI and opaque(ish) types

alistairgrant
How are opaque(ish) types normally handled in UFFI (or FFI)?

E.g. if I want to use libstuff which defines the following structure:

struct stuffstruct {
    typeA a;
    typeB b;
    } stufftype;


I might know they're integers, but not the size, signed vs. unsigned,
etc.

The definitions of typeA and typeB are buried in multiple layers of
header files, dependent on #if tests, can be different sizes on
different platforms (e.g. Linux vs. MacOS), etc.?

Is this documented somewhere?

Thanks,
Alistair

Reply | Threaded
Open this post in threaded view
|

Re: UFFI and opaque(ish) types

Pharo Smalltalk Users mailing list

On Fri, Nov 30, 2018 at 2:39 PM Alistair Grant <[hidden email]> wrote:
How are opaque(ish) types normally handled in UFFI (or FFI)?

E.g. if I want to use libstuff which defines the following structure:

struct stuffstruct {
    typeA a;
    typeB b;
    } stufftype;


I might know they're integers, but not the size, signed vs. unsigned,
etc.

The definitions of typeA and typeB are buried in multiple layers of
header files, dependent on #if tests, can be different sizes on
different platforms (e.g. Linux vs. MacOS), etc.?

Is this documented somewhere?

Thanks,
Alistair


Reply | Threaded
Open this post in threaded view
|

Re: UFFI and opaque(ish) types

EstebanLM


On 30 Nov 2018, at 19:50, phil--- via Pharo-users <[hidden email]> wrote:


Subject: Re: [Pharo-users] UFFI and opaque(ish) types
Date: 30 November 2018 at 19:50:03 CET
To: Any question about pharo is welcome <[hidden email]>



Yes, as he says/shows :)

Esteban




On Fri, Nov 30, 2018 at 2:39 PM Alistair Grant <[hidden email]> wrote:
How are opaque(ish) types normally handled in UFFI (or FFI)?

E.g. if I want to use libstuff which defines the following structure:

struct stuffstruct {
    typeA a;
    typeB b;
    } stufftype;


I might know they're integers, but not the size, signed vs. unsigned,
etc.

The definitions of typeA and typeB are buried in multiple layers of
header files, dependent on #if tests, can be different sizes on
different platforms (e.g. Linux vs. MacOS), etc.?

Is this documented somewhere?

Thanks,
Alistair





Reply | Threaded
Open this post in threaded view
|

Re: UFFI and opaque(ish) types

alistairgrant
Hi Phil and Esteban,

Thanks for your replies.

I've realised that I badly worded my question.  By "opaque" I meant that
it is difficult to determine the underlying type of the
variables, but I still want to be access the values of typeA and typeB
in my example structure.

Phil, I had a look through your XMPP code, if it does show an example of
actually retrieving a structure member such as "typeA" in my example, my
apologies and please let me know where it is.

Thanks again,
Alistair


On Fri, 30 Nov 2018 at 23:46, Esteban Lorenzano <[hidden email]> wrote:

>
> On 30 Nov 2018, at 19:50, phil--- via Pharo-users <[hidden email]> wrote:
>
> From: "[hidden email]" <[hidden email]>
> Subject: Re: [Pharo-users] UFFI and opaque(ish) types
> Date: 30 November 2018 at 19:50:03 CET
> To: Any question about pharo is welcome <[hidden email]>
>
>
> Inherit from FFIOpaqueObject
>
> + initialize TypeMap
>
> I have a few here:
>
> https://github.com/Pharophile/XmppTalk
>
> https://github.com/Pharophile/XmppTalk/tree/master/packages/XmppTalk-ExternalStructures.package
> https://github.com/Pharophile/XmppTalk/blob/master/packages/XmppTalk-LibStrophe.package/StropheLib.class/class/initializeTypeMap.st
>
> HTH
> Phil
>
>
> Yes, as he says/shows :)
>
> Esteban

> On Fri, Nov 30, 2018 at 2:39 PM Alistair Grant <[hidden email]> wrote:
>>
>> How are opaque(ish) types normally handled in UFFI (or FFI)?
>>
>> E.g. if I want to use libstuff which defines the following structure:
>>
>> struct stuffstruct {
>>     typeA a;
>>     typeB b;
>>     } stufftype;
>>
>>
>> I might know they're integers, but not the size, signed vs. unsigned,
>> etc.
>>
>> The definitions of typeA and typeB are buried in multiple layers of
>> header files, dependent on #if tests, can be different sizes on
>> different platforms (e.g. Linux vs. MacOS), etc.?
>>
>> Is this documented somewhere?
>>
>> Thanks,
>> Alistair

Reply | Threaded
Open this post in threaded view
|

Re: UFFI and opaque(ish) types

Ben Coman
One option could be to create a small shared library that you compile for each platform that just exported the member offsets... 
https://www.embedded.com/design/prototyping-and-development/4024941/Learn-a-new-trick-with-the-offsetof--macro

You first FFI to that read struct member offsets and dynamically create the definitions for your target library.
I haven't tried it in practice.

cheers -ben

P.S. Occasionally I dream about compilers being able to embed a pre-compiled header declaration block in the library for FFI users to iterate... 
https://clang.llvm.org/docs/PCHInternals.html#declarations-block


On Sun, 2 Dec 2018 at 17:04, Alistair Grant <[hidden email]> wrote:
Hi Phil and Esteban,

Thanks for your replies.

I've realised that I badly worded my question.  By "opaque" I meant that
it is difficult to determine the underlying type of the
variables, but I still want to be access the values of typeA and typeB
in my example structure.

Phil, I had a look through your XMPP code, if it does show an example of
actually retrieving a structure member such as "typeA" in my example, my
apologies and please let me know where it is.

Thanks again,
Alistair


On Fri, 30 Nov 2018 at 23:46, Esteban Lorenzano <[hidden email]> wrote:
>
> On 30 Nov 2018, at 19:50, phil--- via Pharo-users <[hidden email]> wrote:
>
> From: "[hidden email]" <[hidden email]>
> Subject: Re: [Pharo-users] UFFI and opaque(ish) types
> Date: 30 November 2018 at 19:50:03 CET
> To: Any question about pharo is welcome <[hidden email]>
>
>
> Inherit from FFIOpaqueObject
>
> + initialize TypeMap
>
> I have a few here:
>
> https://github.com/Pharophile/XmppTalk
>
> https://github.com/Pharophile/XmppTalk/tree/master/packages/XmppTalk-ExternalStructures.package
> https://github.com/Pharophile/XmppTalk/blob/master/packages/XmppTalk-LibStrophe.package/StropheLib.class/class/initializeTypeMap.st
>
> HTH
> Phil
>
>
> Yes, as he says/shows :)
>
> Esteban

> On Fri, Nov 30, 2018 at 2:39 PM Alistair Grant <[hidden email]> wrote:
>>
>> How are opaque(ish) types normally handled in UFFI (or FFI)?
>>
>> E.g. if I want to use libstuff which defines the following structure:
>>
>> struct stuffstruct {
>>     typeA a;
>>     typeB b;
>>     } stufftype;
>>
>>
>> I might know they're integers, but not the size, signed vs. unsigned,
>> etc.
>>
>> The definitions of typeA and typeB are buried in multiple layers of
>> header files, dependent on #if tests, can be different sizes on
>> different platforms (e.g. Linux vs. MacOS), etc.?
>>
>> Is this documented somewhere?
>>
>> Thanks,
>> Alistair

Reply | Threaded
Open this post in threaded view
|

Re: UFFI and opaque(ish) types

philippeback
In reply to this post by alistairgrant
No, but what about FFIExternalPackedStructureTest  and FFITestStructure ?

Never tried with a typedef in the fieldsDesc but may work.

Phil


On Sun, Dec 2, 2018 at 10:03 AM Alistair Grant <[hidden email]> wrote:
Hi Phil and Esteban,

Thanks for your replies.

I've realised that I badly worded my question.  By "opaque" I meant that
it is difficult to determine the underlying type of the
variables, but I still want to be access the values of typeA and typeB
in my example structure.

Phil, I had a look through your XMPP code, if it does show an example of
actually retrieving a structure member such as "typeA" in my example, my
apologies and please let me know where it is.

Thanks again,
Alistair


On Fri, 30 Nov 2018 at 23:46, Esteban Lorenzano <[hidden email]> wrote:
>
> On 30 Nov 2018, at 19:50, phil--- via Pharo-users <[hidden email]> wrote:
>
> From: "[hidden email]" <[hidden email]>
> Subject: Re: [Pharo-users] UFFI and opaque(ish) types
> Date: 30 November 2018 at 19:50:03 CET
> To: Any question about pharo is welcome <[hidden email]>
>
>
> Inherit from FFIOpaqueObject
>
> + initialize TypeMap
>
> I have a few here:
>
> https://github.com/Pharophile/XmppTalk
>
> https://github.com/Pharophile/XmppTalk/tree/master/packages/XmppTalk-ExternalStructures.package
> https://github.com/Pharophile/XmppTalk/blob/master/packages/XmppTalk-LibStrophe.package/StropheLib.class/class/initializeTypeMap.st
>
> HTH
> Phil
>
>
> Yes, as he says/shows :)
>
> Esteban

> On Fri, Nov 30, 2018 at 2:39 PM Alistair Grant <[hidden email]> wrote:
>>
>> How are opaque(ish) types normally handled in UFFI (or FFI)?
>>
>> E.g. if I want to use libstuff which defines the following structure:
>>
>> struct stuffstruct {
>>     typeA a;
>>     typeB b;
>>     } stufftype;
>>
>>
>> I might know they're integers, but not the size, signed vs. unsigned,
>> etc.
>>
>> The definitions of typeA and typeB are buried in multiple layers of
>> header files, dependent on #if tests, can be different sizes on
>> different platforms (e.g. Linux vs. MacOS), etc.?
>>
>> Is this documented somewhere?
>>
>> Thanks,
>> Alistair


Reply | Threaded
Open this post in threaded view
|

Re: UFFI and opaque(ish) types

alistairgrant
Hi Ben and Phil,

Thanks for your replies.

On Sun, Dec 02, 2018 at 10:53:11PM +0800, Ben Coman wrote:

> One option could be to create a small shared library that you compile for each
> platform that just exported the member offsets...
> https://www.embedded.com/design/prototyping-and-development/4024941/
> Learn-a-new-trick-with-the-offsetof--macro
>
> You first FFI to that read struct member offsets and dynamically create the
> definitions for your target library.
> I haven't tried it in practice.
>
> cheers -ben
>
> P.S. Occasionally I dream about compilers being able to embed a pre-compiled
> header declaration block in the library for FFI users to iterate...
> https://clang.llvm.org/docs/PCHInternals.html#declarations-block



On Sun, Dec 02, 2018 at 04:41:38PM +0100, [hidden email] wrote:
> No, but what about FFIExternalPackedStructureTest  and FFITestStructure ?
>
> Never tried with a typedef in the fieldsDesc but may work.
>
> Phil


OK, it sounds like this still hasn't been adressed.

The other idea that we've discussed in the past is to supply human
readable text files that describe the structures for each platform, e.g.
a STON file.  The file would be generated by automatically creating a
program that dumps the required information to the STON text file.  This
would allow the definitions to be used on platforms that don't have a
compiler or the dev versions of the libraries installed.  I think we
could even use the current #fieldDesc methods to generate these programs
(maybe with some additional information stored in the class).

A couple of links:

- http://forum.world.st/PATCH-added-FilePlugin-error-code-lookup-function-tp5071376p5071620.html
- http://forum.world.st/PATCH-added-FilePlugin-error-code-lookup-function-tp5071376p5075016.html


Thanks again,
Alistair


Reply | Threaded
Open this post in threaded view
|

Re: UFFI and opaque(ish) types

philippeback
Alistair,

I also would like to see how this works.




(actually this is a fork from Ronie's work) that does some Pharo magic for bindings.


And https://github.com/pavel-krivanek/CCodeGenerator may help to generate some quick external things to reload (it is Slang externalized).

We need a solid push on all of this front since it takes wayyy too much time to produce a binding. It is error prone, boring, and frankly UFFI deserves better given its power.

Best,
Phil


On Mon, Dec 3, 2018 at 10:27 AM Alistair Grant <[hidden email]> wrote:
Hi Ben and Phil,

Thanks for your replies.

On Sun, Dec 02, 2018 at 10:53:11PM +0800, Ben Coman wrote:
> One option could be to create a small shared library that you compile for each
> platform that just exported the member offsets...
> https://www.embedded.com/design/prototyping-and-development/4024941/
> Learn-a-new-trick-with-the-offsetof--macro
>
> You first FFI to that read struct member offsets and dynamically create the
> definitions for your target library.
> I haven't tried it in practice.
>
> cheers -ben
>
> P.S. Occasionally I dream about compilers being able to embed a pre-compiled
> header declaration block in the library for FFI users to iterate...
> https://clang.llvm.org/docs/PCHInternals.html#declarations-block



On Sun, Dec 02, 2018 at 04:41:38PM +0100, [hidden email] wrote:
> No, but what about FFIExternalPackedStructureTest  and FFITestStructure ?
>
> Never tried with a typedef in the fieldsDesc but may work.
>
> Phil


OK, it sounds like this still hasn't been adressed.

The other idea that we've discussed in the past is to supply human
readable text files that describe the structures for each platform, e.g.
a STON file.  The file would be generated by automatically creating a
program that dumps the required information to the STON text file.  This
would allow the definitions to be used on platforms that don't have a
compiler or the dev versions of the libraries installed.  I think we
could even use the current #fieldDesc methods to generate these programs
(maybe with some additional information stored in the class).

A couple of links:

- http://forum.world.st/PATCH-added-FilePlugin-error-code-lookup-function-tp5071376p5071620.html
- http://forum.world.st/PATCH-added-FilePlugin-error-code-lookup-function-tp5071376p5075016.html


Thanks again,
Alistair