Smalltalk › Pharo › Pharo Smalltalk Users

FFI return by reference

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

17 messages Options

Thibault Raffaillac

FFI return by reference

Hi again,

Does anyone know what is the state of return by reference in FFI?
Example:
glGetAttribute: attr into: value
<primitive: #primitiveNativeCall module: #NativeBoostPlugin error: errorCode>
^ self nbCall: #( int SDL_GL_GetAttribute(SDL_GLattr attr, int *value) )

On my image (50666) it does nothing on value yet (if nil, remains nil, if SmallInteger 0, remains 0).
(SDL2 glGetAttribute: 6 into: value) should return 16.
I've seen NBOpenGL put "out" before value in method header, should it matter? (does not make it work though)
Last but not least, should we attach '*' to the type or variable for FFI to understand it is a pointer?

Cheers,
Thibault Raffaillac

Damien Pollet-2

Re: FFI return by reference

I think in that case you need to pass an instance of FFIExternalValueHolder.

On 7 April 2016 at 17:42, Thibault Raffaillac <[hidden email]> wrote:

Hi again,

Does anyone know what is the state of return by reference in FFI?
Example:
glGetAttribute: attr into: value
<primitive: #primitiveNativeCall module: #NativeBoostPlugin error: errorCode>
^ self nbCall: #( int SDL_GL_GetAttribute(SDL_GLattr attr, int *value) )

On my image (50666) it does nothing on value yet (if nil, remains nil, if SmallInteger 0, remains 0).
(SDL2 glGetAttribute: 6 into: value) should return 16.
I've seen NBOpenGL put "out" before value in method header, should it matter? (does not make it work though)
Last but not least, should we attach '*' to the type or variable for FFI to understand it is a pointer?

Cheers,
Thibault Raffaillac

Thibault Raffaillac

Re: FFI return by reference

In reply to this post by Thibault Raffaillac

Is there something like a generic int value holder then?

>
> I think in that case you need to pass an instance of FFIExternalValueHolder.
>
> > Hi again,
> >
> > Does anyone know what is the state of return by reference in FFI?
> > Example:
> > glGetAttribute: attr into: value
> > <primitive: #primitiveNativeCall module: #NativeBoostPlugin error:
> > errorCode>
> > ^ self nbCall: #( int SDL_GL_GetAttribute(SDL_GLattr attr, int
> > *value) )
> >
> > On my image (50666) it does nothing on value yet (if nil, remains nil, if
> > SmallInteger 0, remains 0).
> > (SDL2 glGetAttribute: 6 into: value) should return 16.
> > I've seen NBOpenGL put "out" before value in method header, should it
> > matter? (does not make it work though)
> > Last but not least, should we attach '*' to the type or variable for FFI
> > to understand it is a pointer?
> >
> > Cheers,
> > Thibault Raffaillac

EstebanLM

Re: FFI return by reference

In reply to this post by Damien Pollet-2

On 07 Apr 2016, at 14:15, Damien Pollet <[hidden email]> wrote:

I think in that case you need to pass an instance of FFIExternalValueHolder.

yes.

you can take the test FFIExternalValueHolderTests>>testCall as an example.

But since it uses anonymous classes it might require a better explanation, there is the class comment, but basically, you do this:

1) Declare a class variable

MyClass class>>initialize

INT_PTR := FFIExternalValueHolder ofType: ‘int’.

2) Replace "int *” in your function calls with your new type:

glGetAttribute: attr into: value

^ self ffiCall: #( int SDL_GL_GetAttribute(SDL_GLattr attr, INT_PTR value) )

3) pass an instance of INT_PTR to get the value:

| ptr value |

ptr := INT_PTR new.

self glGetAttribute: anAttributeIDontKnow into: ptr.

value := ptr value.

Esteban

On 7 April 2016 at 17:42, Thibault Raffaillac <[hidden email]> wrote:
Hi again,

Does anyone know what is the state of return by reference in FFI?
Example:
glGetAttribute: attr into: value
<primitive: #primitiveNativeCall module: #NativeBoostPlugin error: errorCode>
^ self nbCall: #( int SDL_GL_GetAttribute(SDL_GLattr attr, int *value) )

On my image (50666) it does nothing on value yet (if nil, remains nil, if SmallInteger 0, remains 0).
(SDL2 glGetAttribute: 6 into: value) should return 16.
I've seen NBOpenGL put "out" before value in method header, should it matter? (does not make it work though)
Last but not least, should we attach '*' to the type or variable for FFI to understand it is a pointer?

Cheers,
Thibault Raffaillac

EstebanLM

Re: FFI return by reference

On 08 Apr 2016, at 11:34, Esteban Lorenzano <[hidden email]> wrote:

On 07 Apr 2016, at 14:15, Damien Pollet <[hidden email]> wrote:

I think in that case you need to pass an instance of FFIExternalValueHolder.

yes.
you can take the test FFIExternalValueHolderTests>>testCall as an example.

But since it uses anonymous classes it might require a better explanation, there is the class comment, but basically, you do this:

1) Declare a class variable

MyClass class>>initialize
INT_PTR := FFIExternalValueHolder ofType: ‘int’.

as a larger explanation, this will create an anonymous subclass to keep “int” values, than later you can use as a type in your function call. Some people would call this obscure, but is a good way of solving this problem (I adopted it from old NB).

If you want to have a named class instead an anonymous class, you can just subclass FFIExternalValueHolder:

FFIExternalValueHolder subclass: #MyIntPtr

and then implement typeDecl class method

MyIntPtr class>>typeDecl

^ ‘int'

and this will work as the anonymous class

2) Replace "int *” in your function calls with your new type:

glGetAttribute: attr into: value
^ self ffiCall: #( int SDL_GL_GetAttribute(SDL_GLattr attr, INT_PTR value) )

3) pass an instance of INT_PTR to get the value:

| ptr value |

ptr := INT_PTR new.
self glGetAttribute: anAttributeIDontKnow into: ptr.

value := ptr value.

Esteban

On 7 April 2016 at 17:42, Thibault Raffaillac <[hidden email]> wrote:
Hi again,

Does anyone know what is the state of return by reference in FFI?
Example:
glGetAttribute: attr into: value
<primitive: #primitiveNativeCall module: #NativeBoostPlugin error: errorCode>
^ self nbCall: #( int SDL_GL_GetAttribute(SDL_GLattr attr, int *value) )

On my image (50666) it does nothing on value yet (if nil, remains nil, if SmallInteger 0, remains 0).
(SDL2 glGetAttribute: 6 into: value) should return 16.
I've seen NBOpenGL put "out" before value in method header, should it matter? (does not make it work though)
Last but not least, should we attach '*' to the type or variable for FFI to understand it is a pointer?

Cheers,
Thibault Raffaillac

Thibault Raffaillac

Re: FFI return by reference

In reply to this post by Thibault Raffaillac

> > 1) Declare a class variable
> >
> > MyClass class>>initialize
> > INT_PTR := FFIExternalValueHolder ofType: ?int?.
>
> as a larger explanation, this will create an anonymous subclass to keep ?int?
> values, than later you can use as a type in your function call. Some people
> would call this obscure, but is a good way of solving this problem (I
> adopted it from old NB).
> If you want to have a named class instead an anonymous class, you can just
> subclass FFIExternalValueHolder:
>
> FFIExternalValueHolder subclass: #MyIntPtr
>
> and then implement typeDecl class method
>
> MyIntPtr class>>typeDecl
> ^ ?int'
>
> and this will work as the anonymous class

The second option is definitely what I want. I am trying to expose a function returning several ints, as in SDL2>>mouseStateX: y: (excluding the possibility to return a Point).
However I would rather not define my own int holder, can't we have generic holders to rely on?

Thibault

EstebanLM

Re: FFI return by reference

> On 11 Apr 2016, at 06:24, Thibault Raffaillac <[hidden email]> wrote:
>
>>> 1) Declare a class variable
>>>
>>> MyClass class>>initialize
>>> INT_PTR := FFIExternalValueHolder ofType: ?int?.
>>
>> as a larger explanation, this will create an anonymous subclass to keep ?int?
>> values, than later you can use as a type in your function call. Some people
>> would call this obscure, but is a good way of solving this problem (I
>> adopted it from old NB).
>> If you want to have a named class instead an anonymous class, you can just
>> subclass FFIExternalValueHolder:
>>
>> FFIExternalValueHolder subclass: #MyIntPtr
>>
>> and then implement typeDecl class method
>>
>> MyIntPtr class>>typeDecl
>> ^ ?int'
>>
>> and this will work as the anonymous class
>
> The second option is definitely what I want.

Personally my option would be to put my types in a class pool and use them, not to extend…
but option is there so you can choose what you want :)

> I am trying to expose a function returning several ints, as in SDL2>>mouseStateX: y: (excluding the possibility to return a Point).
> However I would rather not define my own int holder, can't we have generic holders to rely on?

I will not add for the moment that predefined types value holders… We would need to add one for each type we support to be consistent, and document it, and add tests.
And btw… you do have a generic holder… is just that you need to refine it to what you want :)
You could always use a ByteArray and took values later (some functions in Athens do it like that). The advantage of this approach (using plain ByteArrays) is that you do not need to change function signature at the cost of needing to do manually what the value holder does for you.

My advise is that you go with what you need, we can consider add predefined value holders later.

Esteban

>
> Thibault
>

Thibault Raffaillac

Re: FFI return by reference

In reply to this post by Thibault Raffaillac

> I will not add for the moment that predefined types value holders? We would
> need to add one for each type we support to be consistent, and document it,
> and add tests.
> And btw? you do have a generic holder? is just that you need to refine it to
> what you want :)

Couldn't we use FFIInt32? To me it would make sense to use the class to refer to the type, and an instance of it for a value holder in memory (this is not asking for a change, only reporting it looks strange).

> My advise is that you go with what you need, we can consider add predefined
> value holders later.

Yep, I am sticking to the basic features as much as possible anyway.

Cheers,
Thibault

EstebanLM

Re: FFI return by reference

> On 12 Apr 2016, at 09:44, Thibault Raffaillac <[hidden email]> wrote:
>
>> I will not add for the moment that predefined types value holders? We would
>> need to add one for each type we support to be consistent, and document it,
>> and add tests.
>> And btw? you do have a generic holder? is just that you need to refine it to
>> what you want :)
>
> Couldn't we use FFIInt32? To me it would make sense to use the class to refer to the type, and an instance of it for a value holder in memory (this is not asking for a change, only reporting it looks strange).

nope.
we try to hide the border between “in” and “out” to make the FFI experience simpler… but it is there.
immutable types (like integers in Pharo) cannot be swap (become) and because of that we cannot make the pack/unpack algorithm to work in any efficient way. The benefit to have this functionality does not pay the performance penalty for doing it (it introduces an extra check for any parameter marshalled).
So, for now… no, we cannot… you need to stick with the value holder :(

Esteban

>
>> My advise is that you go with what you need, we can consider add predefined
>> value holders later.
>
> Yep, I am sticking to the basic features as much as possible anyway.
>
> Cheers,
> Thibault
>

Nicolai Hess-3-2

Re: FFI return by reference

In reply to this post by EstebanLM

2016-04-11 15:10 GMT+02:00 Esteban Lorenzano <[hidden email]>:

> On 11 Apr 2016, at 06:24, Thibault Raffaillac <[hidden email]> wrote:
>
>>> 1) Declare a class variable
>>>
>>> MyClass class>>initialize
>>> INT_PTR := FFIExternalValueHolder ofType: ?int?.
>>
>> as a larger explanation, this will create an anonymous subclass to keep ?int?
>> values, than later you can use as a type in your function call. Some people
>> would call this obscure, but is a good way of solving this problem (I
>> adopted it from old NB).
>> If you want to have a named class instead an anonymous class, you can just
>> subclass FFIExternalValueHolder:
>>
>> FFIExternalValueHolder subclass: #MyIntPtr
>>
>> and then implement typeDecl class method
>>
>> MyIntPtr class>>typeDecl
>> ^ ?int'
>>
>> and this will work as the anonymous class
>
> The second option is definitely what I want.

Personally my option would be to put my types in a class pool and use them, not to extend…
but option is there so you can choose what you want :)

> I am trying to expose a function returning several ints, as in SDL2>>mouseStateX: y: (excluding the possibility to return a Point).
> However I would rather not define my own int holder, can't we have generic holders to rely on?

I will not add for the moment that predefined types value holders… We would need to add one for each type we support to be consistent, and document it, and add tests.
And btw… you do have a generic holder… is just that you need to refine it to what you want :)
You could always use a ByteArray and took values later (some functions in Athens do it like that). The advantage of this approach (using plain ByteArrays) is that you do not need to change function signature at the cost of needing to do manually what the value holder does for you.

Hi Esteban,

is this the way it should be done for Strings ?

For example, old NBOpenGL had this Windows method to get the Window Title:

getWindowText: hWnd buffer: lpString bufferSize: nMaxCount
        <primitive: #primitiveNativeCall module: #NativeBoostPlugin>

        ^self nbCall: #(int GetWindowTextA(HWND hWnd, char* lpString, int nMaxCount))
            module: #user32

called as:

getWindowText
    | len str |
    str := ByteString new: 1000.
    len := self getWindowText: self buffer: str bufferSize: 1000.
    ^ str first: len.

When I just modify the ffiCall to

getWindowText: hWnd buffer: lpString bufferSize: nMaxCount

^self ffiCall: #(int GetWindowTextA(HWND hWnd, char* lpString, int nMaxCount)) module: #user32

I get just an empty String.

But If I call it with:

| len str |
    str := ByteArray new: 1000.
    len := self getWindowText: self buffer: str bufferSize: 1000.
    ^ String fromByteArray: (str first: len).

I get the real string text. Is this the way to go?

My advise is that you go with what you need, we can consider add predefined value holders later.

Esteban

>
> Thibault
>

EstebanLM

Re: FFI return by reference

Hi,

Just checked… contrary to what I believed, strings are passed by copy (not by reference, as ByteArray)… logical for safety but with penalty on performance... so yes, using a ByteArray is the correct way to do it. (Or an external value holder, which is a glorified way for passing a ByteArray, but this probably do not work fine for “char*" because I didn’t preview it :P)

Esteban

On 13 Apr 2016, at 05:24, Nicolai Hess <[hidden email]> wrote:

2016-04-11 15:10 GMT+02:00 Esteban Lorenzano <[hidden email]>:

> On 11 Apr 2016, at 06:24, Thibault Raffaillac <[hidden email]> wrote:
>
>>> 1) Declare a class variable
>>>
>>> MyClass class>>initialize
>>> INT_PTR := FFIExternalValueHolder ofType: ?int?.
>>
>> as a larger explanation, this will create an anonymous subclass to keep ?int?
>> values, than later you can use as a type in your function call. Some people
>> would call this obscure, but is a good way of solving this problem (I
>> adopted it from old NB).
>> If you want to have a named class instead an anonymous class, you can just
>> subclass FFIExternalValueHolder:
>>
>> FFIExternalValueHolder subclass: #MyIntPtr
>>
>> and then implement typeDecl class method
>>
>> MyIntPtr class>>typeDecl
>> ^ ?int'
>>
>> and this will work as the anonymous class
>
> The second option is definitely what I want.

Personally my option would be to put my types in a class pool and use them, not to extend…
but option is there so you can choose what you want :)

> I am trying to expose a function returning several ints, as in SDL2>>mouseStateX: y: (excluding the possibility to return a Point).
> However I would rather not define my own int holder, can't we have generic holders to rely on?

I will not add for the moment that predefined types value holders… We would need to add one for each type we support to be consistent, and document it, and add tests.
And btw… you do have a generic holder… is just that you need to refine it to what you want :)
You could always use a ByteArray and took values later (some functions in Athens do it like that). The advantage of this approach (using plain ByteArrays) is that you do not need to change function signature at the cost of needing to do manually what the value holder does for you.

Hi Esteban,

is this the way it should be done for Strings ?
For example, old NBOpenGL had this Windows method to get the Window Title:

getWindowText: hWnd buffer: lpString bufferSize: nMaxCount
        <primitive: #primitiveNativeCall module: #NativeBoostPlugin>

        ^self nbCall: #(int GetWindowTextA(HWND hWnd, char* lpString, int nMaxCount))
            module: #user32

called as:

getWindowText
    | len str |
    str := ByteString new: 1000.
    len := self getWindowText: self buffer: str bufferSize: 1000.
    ^ str first: len.

When I just modify the ffiCall to

getWindowText: hWnd buffer: lpString bufferSize: nMaxCount

    ^self ffiCall: #(int GetWindowTextA(HWND hWnd, char* lpString, int nMaxCount)) module: #user32

I get just an empty String.

But If I call it with:

| len str |
    str := ByteArray new: 1000.
    len := self getWindowText: self buffer: str bufferSize: 1000.
    ^ String fromByteArray: (str first: len).

I get the real string text. Is this the way to go?

My advise is that you go with what you need, we can consider add predefined value holders later.

Esteban

>
> Thibault
>

Rob Rothwell

Re: FFI return by reference

On Wed, Apr 13, 2016 at 8:52 AM, Esteban Lorenzano <[hidden email]> wrote:

Hi,

Just checked… contrary to what I believed, strings are passed by copy (not by reference, as ByteArray)… logical for safety but with penalty on performance... so yes, using a ByteArray is the correct way to do it. (Or an external value holder, which is a glorified way for passing a ByteArray, but this probably do not work fine for “char*" because I didn’t preview it :P)

Hello,

I've never used UFFI before, but would like to call setxattr and getxattr in LibC on OS X (which also pass strings). I don't have a lot of Mac or Unix background, but I have past experience fiddling with parameters to call external Windows libraries from VB.

Is there any general information about UFFI usage anywhere, or is reviewing this thread my best place to start?

Thank you,

Rob

Rob Rothwell

Re: FFI return by reference

Is there any general information about UFFI usage anywhere, or is reviewing this thread my best place to start?

It looks like Torsten's new WebBrowser package might get me started!

Rob

CyrilFerlicot

Re: FFI return by reference

In reply to this post by Rob Rothwell

Le 13/04/2016 15:23, Rob Rothwell a écrit :

> On Wed, Apr 13, 2016 at 8:52 AM, Esteban Lorenzano <[hidden email]
> <mailto:[hidden email]>> wrote:
>
> Hi,
>
> Just checked… contrary to what I believed, strings are passed by
> copy (not by reference, as ByteArray)… logical for safety but with
> penalty on performance... so yes, using a ByteArray is the correct
> way to do it. (Or an external value holder, which is a glorified way
> for passing a ByteArray, but this probably do not work fine for
> “char*" because I didn’t preview it :P)
>
> Hello,
>
> I've never used UFFI before, but would like to call setxattr and
> getxattr in LibC on OS X (which also pass strings). I don't have a lot
> of Mac or Unix background, but I have past experience fiddling with
> parameters to call external Windows libraries from VB.
>
> Is there any general information about UFFI usage anywhere, or is
> reviewing this thread my best place to start?
>

This is pretty new but there is some thread in the mailing list about it.

Esteban also begun a documentation:
https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgress/lastSuccessfulBuild/artifact/book-result/UnifiedFFI/UnifiedFFI.pdf

> Thank you,
>
> Rob

--
Cyril Ferlicot

http://www.synectique.eu

165 Avenue Bretagne
Lille 59000 France

signature.asc (836 bytes) Download Attachment

Rob Rothwell

Re: FFI return by reference

This is pretty new but there is some thread in the mailing list about it.

Esteban also begun a documentation:
https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgress/lastSuccessfulBuild/artifact/book-result/UnifiedFFI/UnifiedFFI.pdf

Thanks Cyril, this is a very nice start and helps a lot!

Take care,

Rob

CyrilFerlicot

Re: FFI return by reference

Le 13/04/2016 20:41, Rob Rothwell a écrit :

> This is pretty new but there is some thread in the mailing list
> about it.
>
> Esteban also begun a documentation:
> https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgress/lastSuccessfulBuild/artifact/book-result/UnifiedFFI/UnifiedFFI.pdf
>
>
> Thanks Cyril, this is a very nice start and helps a lot!
>
> Take care,
>
> Rob

There was also a presentation at Pharo Days but I think it is not on the
web for now. Latter it will be available on slideshare. Or maybe Esteban
will send it before :)

--
Cyril Ferlicot

http://www.synectique.eu

165 Avenue Bretagne
Lille 59000 France

signature.asc (836 bytes) Download Attachment

Thibault Raffaillac

Re: FFI return by reference

In reply to this post by Thibault Raffaillac

Hi,

I tried lots of ways to pass an array, based on the few examples found here and there.
My example is:
GLES2Buffer class>>gen: n into: buffers
self ffiCall: #(void glGenBuffers(int n, int* buffers))

With buffers := FFIExternalArray newType: 'int' size: n (or FFIExternalArray gcExternalNewType: 'int' size: n), the above signature executes without error, but buffers is not updated, so I guess it is copied beforehand.
If I replace int* with FFIExternalArray, the values are written correctly in my array.

If I replace int* with ByteArray instead I can use any one of ByteArray/IntegerArray/FloatArray ! Only ShortArray, LongArray and DoubleArray are missing then, but they are not crucial right now.
So for example:
GLES2Buffer class>>create
buffer := IntegerArray new: 1.
GLES2Buffer gen: 1 into: buffer.
^self new setHandle: (buffer at: 1)

:)

Same goes for passing coordinates to OpenGL, just with #(-1 -1 -1 1 1 1 1 -1) asFloatArray. Perfect.

Cheers,
Thibault

> Hi,
>
> Just checked? contrary to what I believed, strings are passed by copy (not by
> reference, as ByteArray)? logical for safety but with penalty on
> performance... so yes, using a ByteArray is the correct way to do it. (Or an
> external value holder, which is a glorified way for passing a ByteArray, but
> this probably do not work fine for ?char*" because I didn?t preview it :P)
>
> Esteban