uFFI: What is the difference between FFIExternalObject and FFIOpaqueObject?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

uFFI: What is the difference between FFIExternalObject and FFIOpaqueObject?

Sean P. DeNigris
Administrator
I'm working on improving the documentation. I'd like examples of situations
when you'd use one or the other and why...

Also, the FFIOpaqueObject class comment could be fleshed out a bit. Check
out this excerpt:
    "this means we always access them through a reference.
    Now, we could declare the pointer to the structures, but
    then our declarations wouldn't be as close to C as we want."

What would be an example of "declar[ing] the pointer to the structures" and
the resulting declarations that "wouldn't be as close to C as we want"?



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: uFFI: What is the difference between FFIExternalObject and FFIOpaqueObject?

Esteban Lorenzano
Hi, 

The class comments are not clear?

External object = a pointer to something. 

On 21 May 2020, at 21:53, Sean P. DeNigris <[hidden email]> wrote:

I'm working on improving the documentation. I'd like examples of situations
when you'd use one or the other and why...

Also, the FFIOpaqueObject class comment could be fleshed out a bit. Check
out this excerpt: 
   "this means we always access them through a reference. 
   Now, we could declare the pointer to the structures, but 
   then our declarations wouldn't be as close to C as we want." 

What would be an example of "declar[ing] the pointer to the structures" and
the resulting declarations that "wouldn't be as close to C as we want"?


There are plenty of cases in C where you do: 

struct MyStruct *s;

Buy tou cannot do (or you cannot use): 

struct MyStruct s;

This is like that because most modern frameworks use the opaque type as a place to keep state, but to interact with them you use functions. e.g: 

MyOpaqueType *ot = newOpaqueType();
opaque_type_someFunction(ot, “a parameter”);


Then an opaque object would be: 

FFIOpaqueObject subclass: #MyOpaqueType.

And you can do an ffi call like this: 

self ffiCall: #(void opaque_type_someFunction(MyOpaqueType *ot, char *aString)).

Note the declaration of "MyOpaqueType *ot

This is how C declaration looks. 

We can say that an opaque type is a pointer type with arity 0.

Now, an FFIExternalObject (which should be called FFIExternalPointer), has already an arity of 1.

Which means, if you declare: 

FFIExternalObject subclass: #MyOpaqueTypePtr.

The same ffiCall you did before would be: 

self ffiCall: #(void opaque_type_someFunction(MyOpaqueTypePtr ot, char *aString)).

Note the declaration without the *… this is because FFIExternalObject is already a pointer. 

Of course, you can write the function call both ways and it works identically… but the first one is closer to what you do in C regularly. 

Esteban

Reply | Threaded
Open this post in threaded view
|

Re: uFFI: What is the difference between FFIExternalObject and FFIOpaqueObject?

Sean P. DeNigris
Administrator
Esteban Lorenzano wrote
> The class comments are not clear?

Maybe it's me ;-)


Esteban Lorenzano wrote
> Opaque object = https://en.wikipedia.org/wiki/Opaque_data_type
> &lt;https://en.wikipedia.org/wiki/Opaque_data_type&gt;
> External object = a pointer to something.

For example, why is ObjCClass a subclass of FFIExternalObject instead of
FFIOpaqueObject when the docs say that a "Class [is] An opaque type that
represents an Objective-C class"? Also, is the distinction just conceptual?
I don't see what either class "buys" you in behavior since they are siblings
that both pretty much have no methods.



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: uFFI: What is the difference between FFIExternalObject and FFIOpaqueObject?

Esteban Lorenzano
On 22 May 2020, at 04:24, Sean P. DeNigris <[hidden email]> wrote:

Esteban Lorenzano wrote
The class comments are not clear?

Maybe it's me ;-)


Esteban Lorenzano wrote
Opaque object = https://en.wikipedia.org/wiki/Opaque_data_type
&lt;https://en.wikipedia.org/wiki/Opaque_data_type&gt;
External object = a pointer to something. 

For example, why is ObjCClass a subclass of FFIExternalObject instead of
FFIOpaqueObject when the docs say that a "Class [is] An opaque type that
represents an Objective-C class"? Also, is the distinction just conceptual?
I don't see what either class "buys" you in behavior since they are siblings
that both pretty much have no methods.

This is because ObjCClass = SomeOpaqueStructurePointer (is already a pointer), same as ObjCSelector and ObjCMethod.
Difference is subtle, because is just a different way of representing the same. 

The easiest way I have to explain is: 

You use an OpaqueObject when you will refer to the opaque type in calls as:  someFunction(opaqueobject *someArg).
You use an ExternalObject when you will refer to the opaque type in calls as: someFunction(externalobject someArg).

Esteban




-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Reply | Threaded
Open this post in threaded view
|

Re: uFFI: What is the difference between FFIExternalObject and FFIOpaqueObject?

Ben Coman


On Fri, 22 May 2020 at 15:25, Esteban Lorenzano <[hidden email]> wrote:
On 22 May 2020, at 04:24, Sean P. DeNigris <[hidden email]> wrote:

Esteban Lorenzano wrote
The class comments are not clear?

Maybe it's me ;-)

No. I struggled with this also, and still don't have it fully grasped.  I could follow Eteban's advice in a particular case I asked about, but still feel hard to reason about next time. 
 


Esteban Lorenzano wrote
Opaque object = https://en.wikipedia.org/wiki/Opaque_data_type
&lt;https://en.wikipedia.org/wiki/Opaque_data_type&gt;
External object = a pointer to something. 

For example, why is ObjCClass a subclass of FFIExternalObject instead of
FFIOpaqueObject when the docs say that a "Class [is] An opaque type that
represents an Objective-C class"? Also, is the distinction just conceptual?
I don't see what either class "buys" you in behavior since they are siblings
that both pretty much have no methods. 

This is because ObjCClass = SomeOpaqueStructurePointer (is already a pointer), same as ObjCSelector and ObjCMethod.
Difference is subtle, because is just a different way of representing the same. 

The easiest way I have to explain is: 

As a prelude to my next comment, how does "External" distinguish between the two case?
Aren't both cases dealing with external C data? 

You use an OpaqueObject when you will refer to "the opaque type" in calls as:  someFunction(opaqueobject *someArg).
You use an ExternalObject when you will refer to "the opaque type" in calls as: someFunction(externalobject someArg).

That makes the distinction really clear.  So if now IIUC, both cases deal with "opaque types" 
with the *only* difference between the two being the arity?
Then I propose the following naming would be more intention revealing...

OpaqueObject when you will refer to "the opaque type" in calls as:  someFunction(opaqueobject *someArg).
OpaquePointer when you will refer to "the opaque type" in calls as: someFunction(opaquepointer someArg).  


And to test my new understanding, the matching typdefs and FFI calls would be...

OPAQUE OBJECT
-  typedef MyOpaqueType opaqueobject.
-  int someFunction(opaqueobject *someArg) 

-  FFIOpaqueObject subclass: #MyOpaqueType.
-  self ffiCall: #(int someFunction( MyOpaqueType *ot ) ).
- opaque type has arity 0, so FFI needs to take a pointer to it  
 
OPAQUE POINTER
-  typedef MyOpaqueObject *opaquepointer.
-  int someFunction(opaquepointer someArg) 

-  FFIOpaquePointer subclass: # MyOpaqueType.
-  self ffiCall: #(int someFunction( MyOpaqueType ot ) ).
- opaque type has arity of 1,  so FFI uses it directly     

The latter FFI call is without the *… because FFIOpaquePointer is already a pointer. 
In practice, both FFI calls are identical, and you use the one that matches the C header definitions.


...or maybe I still don't get it.

 cheers -ben
Reply | Threaded
Open this post in threaded view
|

Re: uFFI: What is the difference between FFIExternalObject and FFIOpaqueObject?

Esteban Lorenzano

On 22 May 2020, at 15:07, Ben Coman <[hidden email]> wrote:



On Fri, 22 May 2020 at 15:25, Esteban Lorenzano <[hidden email]> wrote:
On 22 May 2020, at 04:24, Sean P. DeNigris <[hidden email]> wrote:

Esteban Lorenzano wrote
The class comments are not clear?

Maybe it's me ;-)

No. I struggled with this also, and still don't have it fully grasped.  I could follow Eteban's advice in a particular case I asked about, but still feel hard to reason about next time. 
 


Esteban Lorenzano wrote
Opaque object = https://en.wikipedia.org/wiki/Opaque_data_type
&lt;https://en.wikipedia.org/wiki/Opaque_data_type&gt;
External object = a pointer to something. 

For example, why is ObjCClass a subclass of FFIExternalObject instead of
FFIOpaqueObject when the docs say that a "Class [is] An opaque type that
represents an Objective-C class"? Also, is the distinction just conceptual?
I don't see what either class "buys" you in behavior since they are siblings
that both pretty much have no methods. 

This is because ObjCClass = SomeOpaqueStructurePointer (is already a pointer), same as ObjCSelector and ObjCMethod.
Difference is subtle, because is just a different way of representing the same. 

The easiest way I have to explain is: 

As a prelude to my next comment, how does "External" distinguish between the two case?
Aren't both cases dealing with external C data? 

You use an OpaqueObject when you will refer to "the opaque type" in calls as:  someFunction(opaqueobject *someArg).
You use an ExternalObject when you will refer to "the opaque type" in calls as: someFunction(externalobject someArg).

That makes the distinction really clear.  So if now IIUC, both cases deal with "opaque types" 
with the *only* difference between the two being the arity?
Then I propose the following naming would be more intention revealing...

OpaqueObject when you will refer to "the opaque type" in calls as:  someFunction(opaqueobject *someArg).
OpaquePointer when you will refer to "the opaque type" in calls as: someFunction(opaquepointer someArg).  


And to test my new understanding, the matching typdefs and FFI calls would be...

OPAQUE OBJECT
-  typedef MyOpaqueType opaqueobject.
-  int someFunction(opaqueobject *someArg) 

-  FFIOpaqueObject subclass: #MyOpaqueType.
-  self ffiCall: #(int someFunction( MyOpaqueType *ot ) ).
- opaque type has arity 0, so FFI needs to take a pointer to it  
 
OPAQUE POINTER
-  typedef MyOpaqueObject *opaquepointer.
-  int someFunction(opaquepointer someArg) 

-  FFIOpaquePointer subclass: # MyOpaqueType.
-  self ffiCall: #(int someFunction( MyOpaqueType ot ) ).
- opaque type has arity of 1,  so FFI uses it directly     

The latter FFI call is without the *… because FFIOpaquePointer is already a pointer. 
In practice, both FFI calls are identical, and you use the one that matches the C header definitions.


...or maybe I still don't get it.

No, you got it :)
Part of the confusion is the horribly bad name that FFIExternalObject has :/
Other part is my inability to explain it ;)

But yes, this is all as you say.
If someone wants to do a PR changing the names and maybe enhancing my comments… it would be nice :D

Esteban


 cheers -ben

Reply | Threaded
Open this post in threaded view
|

Re: uFFI: What is the difference between FFIExternalObject and FFIOpaqueObject?

Esteban Lorenzano


On 22 May 2020, at 15:25, Esteban Lorenzano <[hidden email]> wrote:


On 22 May 2020, at 15:07, Ben Coman <[hidden email]> wrote:



On Fri, 22 May 2020 at 15:25, Esteban Lorenzano <[hidden email]> wrote:
On 22 May 2020, at 04:24, Sean P. DeNigris <[hidden email]> wrote:

Esteban Lorenzano wrote
The class comments are not clear?

Maybe it's me ;-)

No. I struggled with this also, and still don't have it fully grasped.  I could follow Eteban's advice in a particular case I asked about, but still feel hard to reason about next time. 
 


Esteban Lorenzano wrote
Opaque object = https://en.wikipedia.org/wiki/Opaque_data_type
&lt;https://en.wikipedia.org/wiki/Opaque_data_type&gt;
External object = a pointer to something. 

For example, why is ObjCClass a subclass of FFIExternalObject instead of
FFIOpaqueObject when the docs say that a "Class [is] An opaque type that
represents an Objective-C class"? Also, is the distinction just conceptual?
I don't see what either class "buys" you in behavior since they are siblings
that both pretty much have no methods. 

This is because ObjCClass = SomeOpaqueStructurePointer (is already a pointer), same as ObjCSelector and ObjCMethod.
Difference is subtle, because is just a different way of representing the same. 

The easiest way I have to explain is: 

As a prelude to my next comment, how does "External" distinguish between the two case?
Aren't both cases dealing with external C data? 

You use an OpaqueObject when you will refer to "the opaque type" in calls as:  someFunction(opaqueobject *someArg).
You use an ExternalObject when you will refer to "the opaque type" in calls as: someFunction(externalobject someArg).

That makes the distinction really clear.  So if now IIUC, both cases deal with "opaque types" 
with the *only* difference between the two being the arity?
Then I propose the following naming would be more intention revealing...

OpaqueObject when you will refer to "the opaque type" in calls as:  someFunction(opaqueobject *someArg).
OpaquePointer when you will refer to "the opaque type" in calls as: someFunction(opaquepointer someArg).  


And to test my new understanding, the matching typdefs and FFI calls would be...

OPAQUE OBJECT
-  typedef MyOpaqueType opaqueobject.
-  int someFunction(opaqueobject *someArg) 

-  FFIOpaqueObject subclass: #MyOpaqueType.
-  self ffiCall: #(int someFunction( MyOpaqueType *ot ) ).
- opaque type has arity 0, so FFI needs to take a pointer to it  
 
OPAQUE POINTER
-  typedef MyOpaqueObject *opaquepointer.
-  int someFunction(opaquepointer someArg) 

-  FFIOpaquePointer subclass: # MyOpaqueType.
-  self ffiCall: #(int someFunction( MyOpaqueType ot ) ).
- opaque type has arity of 1,  so FFI uses it directly     

The latter FFI call is without the *… because FFIOpaquePointer is already a pointer. 
In practice, both FFI calls are identical, and you use the one that matches the C header definitions.


...or maybe I still don't get it.

No, you got it :)
Part of the confusion is the horribly bad name that FFIExternalObject has :/
Other part is my inability to explain it ;)

But yes, this is all as you say.
If someone wants to do a PR changing the names and maybe enhancing my comments… it would be nice :D
Of course keeping the old names as deprecated! ;)


Esteban


 cheers -ben


Reply | Threaded
Open this post in threaded view
|

Re: uFFI: What is the difference between FFIExternalObject and FFIOpaqueObject?

Nicolas Cellier
It's very important to name the types explicitely in order to avoid confusion


Le ven. 22 mai 2020 à 15:30, Esteban Lorenzano <[hidden email]> a écrit :


On 22 May 2020, at 15:25, Esteban Lorenzano <[hidden email]> wrote:


On 22 May 2020, at 15:07, Ben Coman <[hidden email]> wrote:



On Fri, 22 May 2020 at 15:25, Esteban Lorenzano <[hidden email]> wrote:
On 22 May 2020, at 04:24, Sean P. DeNigris <[hidden email]> wrote:

Esteban Lorenzano wrote
The class comments are not clear?

Maybe it's me ;-)

No. I struggled with this also, and still don't have it fully grasped.  I could follow Eteban's advice in a particular case I asked about, but still feel hard to reason about next time. 
 


Esteban Lorenzano wrote
Opaque object = https://en.wikipedia.org/wiki/Opaque_data_type
&lt;https://en.wikipedia.org/wiki/Opaque_data_type&gt;
External object = a pointer to something. 

For example, why is ObjCClass a subclass of FFIExternalObject instead of
FFIOpaqueObject when the docs say that a "Class [is] An opaque type that
represents an Objective-C class"? Also, is the distinction just conceptual?
I don't see what either class "buys" you in behavior since they are siblings
that both pretty much have no methods. 

This is because ObjCClass = SomeOpaqueStructurePointer (is already a pointer), same as ObjCSelector and ObjCMethod.
Difference is subtle, because is just a different way of representing the same. 

The easiest way I have to explain is: 

As a prelude to my next comment, how does "External" distinguish between the two case?
Aren't both cases dealing with external C data? 

You use an OpaqueObject when you will refer to "the opaque type" in calls as:  someFunction(opaqueobject *someArg).
You use an ExternalObject when you will refer to "the opaque type" in calls as: someFunction(externalobject someArg).

That makes the distinction really clear.  So if now IIUC, both cases deal with "opaque types" 
with the *only* difference between the two being the arity?
Then I propose the following naming would be more intention revealing...

OpaqueObject when you will refer to "the opaque type" in calls as:  someFunction(opaqueobject *someArg).
OpaquePointer when you will refer to "the opaque type" in calls as: someFunction(opaquepointer someArg).  


And to test my new understanding, the matching typdefs and FFI calls would be...

OPAQUE OBJECT
-  typedef MyOpaqueType opaqueobject.
-  int someFunction(opaqueobject *someArg) 

-  FFIOpaqueObject subclass: #MyOpaqueType.
-  self ffiCall: #(int someFunction( MyOpaqueType *ot ) ).
- opaque type has arity 0, so FFI needs to take a pointer to it  
 
OPAQUE POINTER
-  typedef MyOpaqueObject *opaquepointer.
-  int someFunction(opaquepointer someArg) 

-  FFIOpaquePointer subclass: # MyOpaqueType.
-  self ffiCall: #(int someFunction( MyOpaqueType ot ) ).
- opaque type has arity of 1,  so FFI uses it directly     

The latter FFI call is without the *… because FFIOpaquePointer is already a pointer. 
In practice, both FFI calls are identical, and you use the one that matches the C header definitions.


...or maybe I still don't get it.

No, you got it :)
Part of the confusion is the horribly bad name that FFIExternalObject has :/
Other part is my inability to explain it ;)

But yes, this is all as you say.
If someone wants to do a PR changing the names and maybe enhancing my comments… it would be nice :D
Of course keeping the old names as deprecated! ;)


Esteban


 cheers -ben


Reply | Threaded
Open this post in threaded view
|

Re: uFFI: What is the difference between FFIExternalObject and FFIOpaqueObject?

Sean P. DeNigris
Administrator
In reply to this post by Esteban Lorenzano
Esteban Lorenzano wrote
> The easiest way I have to explain is:
>
> You use an OpaqueObject when you will refer to the opaque type in calls
> as:  someFunction(opaqueobject *someArg).
> You use an ExternalObject when you will refer to the opaque type in calls
> as: someFunction(externalobject someArg).

Very clear! We're getting somewhere :) But...

What if a type is used both ways?

For example:
    ^ self ffiCall: #(Method * class_copyMethodList(Class self, uint
*outCount))
and
    ^ self ffiCall: #(struct objc_method_description *
method_getDescription(Method self))



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: uFFI: What is the difference between FFIExternalObject and FFIOpaqueObject?

Ben Coman


On Sun, 24 May 2020 at 02:01, Sean P. DeNigris <[hidden email]> wrote:
Esteban Lorenzano wrote
> The easiest way I have to explain is:
>
> You use an OpaqueObject when you will refer to the opaque type in calls
> as:  someFunction(opaqueobject *someArg).
> You use an ExternalObject when you will refer to the opaque type in calls
> as: someFunction(externalobject someArg).

Very clear! We're getting somewhere :) But...

What if a type is used both ways?

For example:
    ^ self ffiCall: #(Method * class_copyMethodList(Class self, uint
*outCount))
and
    ^ self ffiCall: #(struct objc_method_description *
method_getDescription(Method self))

In advance of Esteban's answer I'd like to understand better your situation above.
    Passing a method to a function seems like setting up a call-back? 
    Do you have a concrete C example that can be viewed in its full context?

cheers -ben
Reply | Threaded
Open this post in threaded view
|

Re: uFFI: What is the difference between FFIExternalObject and FFIOpaqueObject?

Esteban Lorenzano
In reply to this post by Sean P. DeNigris
Hi,

That’s not really possible :)
In your case, Method is already a pointer (with arity 1).
Which means

Method *

Is in fact a pointer to a pointer (arity 2), something like "void**"

This is a typical case, but you are not “having the two cases”, you still have one :)

UFFI will try to box/unbox that for you, btw… but sometimes this is not what you want (because pointers to pointers is a typical way to answer lists, for example). And UFFI does not know what you want there… in this cases (when you want the list), the recommended way is to get the pointer and create the list yourself. Instead:

Method *methodList()

You call

void *methodList()

To avoid the unbox attempt, and then you process your answer “in raw” (for example using the #readArrayOf:until: method).

If it happens that you use both SomeStructure, and SomeStructure*, then is not an opaque type, and you need to map it by subclasses FFIExternalStructure and implementing fieldsDesc, etc.

Esteban

> On 23 May 2020, at 20:00, Sean P. DeNigris <[hidden email]> wrote:
>
> Esteban Lorenzano wrote
>> The easiest way I have to explain is:
>>
>> You use an OpaqueObject when you will refer to the opaque type in calls
>> as:  someFunction(opaqueobject *someArg).
>> You use an ExternalObject when you will refer to the opaque type in calls
>> as: someFunction(externalobject someArg).
>
> Very clear! We're getting somewhere :) But...
>
> What if a type is used both ways?
>
> For example:
>   ^ self ffiCall: #(Method * class_copyMethodList(Class self, uint
> *outCount))
> and
>   ^ self ffiCall: #(struct objc_method_description *
> method_getDescription(Method self))
>
>
>
> -----
> Cheers,
> Sean
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
>