Smalltalk › Pharo › Pharo Smalltalk Users

FFI structs

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

12 messages Options

Annick

FFI structs

I I have a C struct with a char*

struct result {
char* time }

I define an ExternalStructure in FFI , with one field of type char* (note that the syntax char [5] is not accepted ! ).

defineFields generates the accessors time and time:

If I use
time: ’05:45’

I get an error.
How do I set a string value in an ExternalStructure ?

Annick

stepharo

Re: FFI structs

did you read the NativeBoost tutorial on the PharoForTheEntreprise book?
If you do please report potential mistakes so that we can improve.

On 29/10/14 05:59, Annick Fron wrote:

> I I have a C struct with a char*
>
> struct result {
> char* time }
>
> I define an ExternalStructure in FFI , with one field of type char* (note that the syntax char [5] is not accepted ! ).
>
> defineFields generates the accessors time and time:
>
> If I use
> time: ’05:45’
>
> I get an error.
> How do I set a string value in an ExternalStructure ?
>
> Annick
>
>
>
>
>

Thomas Bany

Re: FFI structs

Hi Annick !

I'll explain first why you got the error.

The assignment time: ’05:45’ failed because you tried to put an array of char (of size 5*sizeof(char)) inside an address.

At some point, you need to allocate the memory that will hold the data. In your case, you can do it quite easily with the class-side method #fromString: of class NBExternalAdress. This will return a pointer, which you can assign to the field of your struct.

Regarding what you really want to do, having a char[5] field in your struct, I'm not totaly sure how you can do it with NativeBoost. A C struc is just a chunk of memory with a well defined size, so if you want to specify a field as a char[5], you must have a type that match that size. It does not look like the field parser of NBExternalStruct can do it dynamicaly. You could maybe hard code it by subclassing NBExternalType into NBExternalArrayOf5Char, with a valueSize of 5.

As a final word, be wary of where you have allocated your objects, whether it is in object-space (#new), or in C heap (#externalNew). The first is garbage collected so the address are volatile, but you bare the duty of freeing the memory in the second.

2014-10-29 12:18 GMT+01:00 stepharo <[hidden email]>:

did you read the NativeBoost tutorial on the PharoForTheEntreprise book?
If you do please report potential mistakes so that we can improve.

On 29/10/14 05:59, Annick Fron wrote:

I I have a C struct with a char*

struct result {
char* time }

I define an ExternalStructure in FFI , with one field of type char* (note that the syntax char [5] is not accepted ! ).

defineFields generates the accessors time and time:

If I use
time: ’05:45’

I get an error.
How do I set a string value in an ExternalStructure ?

Annick

Annick

Re: FFI structs

In reply to this post by stepharo

I can’t use NativeBoost on the Raspberry, I would be glad if I could !!!

Le 29 oct. 2014 à 12:18, stepharo <[hidden email]> a écrit :

> did you read the NativeBoost tutorial on the PharoForTheEntreprise book?
> If you do please report potential mistakes so that we can improve.
>
>
> On 29/10/14 05:59, Annick Fron wrote:
>> I I have a C struct with a char*
>>
>> struct result {
>> char* time }
>>
>> I define an ExternalStructure in FFI , with one field of type char* (note that the syntax char [5] is not accepted ! ).
>>
>> defineFields generates the accessors time and time:
>>
>> If I use
>> time: ’05:45’
>>
>> I get an error.
>> How do I set a string value in an ExternalStructure ?
>>
>> Annick
>>
>>
>>
>>
>>
>
>

Thomas Bany

Re: FFI structs

Oops, sorry, I was misslead by Steph reply.

Some of the comments might still hold. For the time: ’05:45’ code to work, you would need FFI to handle all the hassle of memory management and actually allocate the memory for 5 characters and copying it. I never used FFI but I doubt it does this.

Looking at FFI library, you both have ExternalAddress (for the struct with char*) and ExternalType (for the struct with char[5]) that you can use. For the later solution, it still looks like you should define a specific 5 bytes ArrayOf5Char type.

2014-10-29 18:34 GMT+01:00 Annick Fron <[hidden email]>:

I can’t use NativeBoost on the Raspberry, I would be glad if I could !!!

Le 29 oct. 2014 à 12:18, stepharo <[hidden email]> a écrit :

> did you read the NativeBoost tutorial on the PharoForTheEntreprise book?
> If you do please report potential mistakes so that we can improve.
>
>
> On 29/10/14 05:59, Annick Fron wrote:
>> I I have a C struct with a char*
>>
>> struct result {
>> char* time }
>>
>> I define an ExternalStructure in FFI , with one field of type char* (note that the syntax char [5] is not accepted ! ).
>>
>> defineFields generates the accessors time and time:
>>
>> If I use
>> time: ’05:45’
>>
>> I get an error.
>> How do I set a string value in an ExternalStructure ?
>>
>> Annick
>>
>>
>>
>>
>>
>
>

stepharo

Re: FFI structs

Ronie is working on unifying all the api and proposing one syntax mechanism for
    - nativeBoost back-end
    - FFI
    and may be Alien

The FFI situation is not good and we know we should improve it.

On 29/10/14 20:18, Thomas Bany wrote:

Oops, sorry, I was misslead by Steph reply.

Some of the comments might still hold. For the time: ’05:45’ code to work, you would need FFI to handle all the hassle of memory management and actually allocate the memory for 5 characters and copying it. I never used FFI but I doubt it does this.

Looking at FFI library, you both have ExternalAddress (for the struct with char*) and ExternalType (for the struct with char[5]) that you can use. For the later solution, it still looks like you should define a specific 5 bytes ArrayOf5Char type.

2014-10-29 18:34 GMT+01:00 Annick Fron <[hidden email]>:

I can’t use NativeBoost on the Raspberry, I would be glad if I could !!!

Le 29 oct. 2014 à 12:18, stepharo <[hidden email]> a écrit :

> did you read the NativeBoost tutorial on the PharoForTheEntreprise book?
> If you do please report potential mistakes so that we can improve.
>
>
> On 29/10/14 05:59, Annick Fron wrote:
>> I I have a C struct with a char*
>>
>> struct result {
>> char* time }
>>
>> I define an ExternalStructure in FFI , with one field of type char* (note that the syntax char [5] is not accepted ! ).
>>
>> defineFields generates the accessors time and time:
>>
>> If I use
>> time: ’05:45’
>>
>> I get an error.
>> How do I set a string value in an ExternalStructure ?
>>
>> Annick
>>
>>
>>
>>
>>
>
>

Tudor Girba-2

Re: FFI structs

That is cool.

Doru

On Thu, Oct 30, 2014 at 11:37 PM, stepharo <[hidden email]> wrote:

Ronie is working on unifying all the api and proposing one syntax mechanism for
    - nativeBoost back-end
    - FFI
    and may be Alien

The FFI situation is not good and we know we should improve it.

On 29/10/14 20:18, Thomas Bany wrote:

Oops, sorry, I was misslead by Steph reply.

Some of the comments might still hold. For the time: ’05:45’ code to work, you would need FFI to handle all the hassle of memory management and actually allocate the memory for 5 characters and copying it. I never used FFI but I doubt it does this.

Looking at FFI library, you both have ExternalAddress (for the struct with char*) and ExternalType (for the struct with char[5]) that you can use. For the later solution, it still looks like you should define a specific 5 bytes ArrayOf5Char type.

2014-10-29 18:34 GMT+01:00 Annick Fron <[hidden email]>:

I can’t use NativeBoost on the Raspberry, I would be glad if I could !!!

Le 29 oct. 2014 à 12:18, stepharo <[hidden email]> a écrit :

> did you read the NativeBoost tutorial on the PharoForTheEntreprise book?
> If you do please report potential mistakes so that we can improve.
>
>
> On 29/10/14 05:59, Annick Fron wrote:
>> I I have a C struct with a char*
>>
>> struct result {
>> char* time }
>>
>> I define an ExternalStructure in FFI , with one field of type char* (note that the syntax char [5] is not accepted ! ).
>>
>> defineFields generates the accessors time and time:
>>
>> If I use
>> time: ’05:45’
>>
>> I get an error.
>> How do I set a string value in an ExternalStructure ?
>>
>> Annick
>>
>>
>>
>>
>>
>
>

www.tudorgirba.com

"Every thing has its own flow"

philippeback

Re: FFI structs

On Fri, Oct 31, 2014 at 6:51 AM, Tudor Girba <[hidden email]> wrote:

That is cool.

As long as we keep what is working, ahem, working. I now depend on quite a couple of FFI related libs.

What I need is an easy FFI wrapping tool, like https://github.com/ronsaldo/swig. Is UFFI going that way?

Phil

Doru

On Thu, Oct 30, 2014 at 11:37 PM, stepharo <[hidden email]> wrote:

Ronie is working on unifying all the api and proposing one syntax mechanism for
    - nativeBoost back-end
    - FFI
    and may be Alien

The FFI situation is not good and we know we should improve it.

On 29/10/14 20:18, Thomas Bany wrote:

Oops, sorry, I was misslead by Steph reply.

Some of the comments might still hold. For the time: ’05:45’ code to work, you would need FFI to handle all the hassle of memory management and actually allocate the memory for 5 characters and copying it. I never used FFI but I doubt it does this.

Looking at FFI library, you both have ExternalAddress (for the struct with char*) and ExternalType (for the struct with char[5]) that you can use. For the later solution, it still looks like you should define a specific 5 bytes ArrayOf5Char type.

2014-10-29 18:34 GMT+01:00 Annick Fron <[hidden email]>:

I can’t use NativeBoost on the Raspberry, I would be glad if I could !!!

Le 29 oct. 2014 à 12:18, stepharo <[hidden email]> a écrit :

> did you read the NativeBoost tutorial on the PharoForTheEntreprise book?
> If you do please report potential mistakes so that we can improve.
>
>
> On 29/10/14 05:59, Annick Fron wrote:
>> I I have a C struct with a char*
>>
>> struct result {
>> char* time }
>>
>> I define an ExternalStructure in FFI , with one field of type char* (note that the syntax char [5] is not accepted ! ).
>>
>> defineFields generates the accessors time and time:
>>
>> If I use
>> time: ’05:45’
>>
>> I get an error.
>> How do I set a string value in an ExternalStructure ?
>>
>> Annick
>>
>>
>>
>>
>>
>
>

--
www.tudorgirba.com

"Every thing has its own flow"

Alain Rastoul-2

Re: FFI structs

In reply to this post by Annick

Hi Annick,

you can't pass smalltalk strings or objects to ffi as parameters,
you have to allocate the buffer member of your structure
on the external heap ( plus to avoid being gc'd before ffi call)
with ExternalAddress allocate: <your buffer/string size +1 for string
null termination>
fill your buffer with byteAt:put: method,
and release this memory with free when you have finished.

not sure, but you could try something like:

s := '05:45' copyWith: (Character value: 0).
a := ExternalAddress allocate: s size.
s asByteArray withIndexDo: [ :b :i | a byteAt: i put: b ].
myStruct time: a.
... call your ffi ...
a free

Regards,

Alain

Le 29/10/2014 11:59, Annick Fron a écrit :

Ronie Salgado

Re: FFI structs

Ronie is working on unifying all the api and proposing one syntax mechanism for
    - nativeBoost back-end
    - FFI
    and may be Alien

I had a skype session with Eliot the last. He taught me about the callback mechanism and I am actually going to use Alien for the callbacks, by adapting it to perform the marshalling into the UFFI/NativeBoost types.

In the future, I am planning in using Lowcode to perform the whole callback process, but now for simplicity I am just going to piggy back in Alien.

I still have to finish the callout by fixing a complicated bug. With Eliot we discussed a workaround for this bug. I haven't implemented those workaround yet, because I have been a bit busy with my university studies.

As long as we keep what is working, ahem, working. I now depend on quite a couple of FFI related libs.

What I need is an easy FFI wrapping tool, like https://github.com/ronsaldo/swig. Is UFFI going that way?

Well, I made the swig adaptation for NativeBoost. Making it for the UFFI is actually something trivial. So, we are going to have a powerful FFI and a binding generator for it.

The only problem with Swig, is that the bindings generated by it are not perfect, because it exposes the exact same C API. A C library that uses some kind of object model, cannot be exposed in a clean object oriented way cleanly by a generic automatic tool.

The FFI situation is not good and we know we should improve it.

For the record, the current FFI situation can be traced to the historical origins of Smalltalk and even Squeak. The original Smalltalk was an actual operating system, and Squeak was also made as an operating system. Because of this, they were not designed to interact with other languages.

Getting a perfect FFI is going to take a lot of time, because of other problems, such as Multi-threading and the mandatory use of non-blocking IO. Fortunately, Eliot made a way to have a threaded FFI, in which a single thread owns the VM at a single time. With this threaded FFI we can have things such as a callback from a different thread, which is required by some libraries such as the sound API of SDL2.

Of course the UFFI is going to support this threaded FFI in some point of time. To support the threaded FFI I only have to pin/unpin some object, and call two VM functions, disownVM() and ownVM().

As for getting some actual parallelism in Pharo, this is something that I think is going to take a big amount of time. For now the best thing we can do is to use OpenCL. This week I have been doing some experimenting in making an API for linear algebra, that uses OpenCL to perform computations in parallel. Later I will publish a video.

Greetings,
Ronie

2014-10-31 19:23 GMT-03:00 Alain Rastoul <[hidden email]>:

Hi Annick,

you can't pass smalltalk strings or objects to ffi as parameters,
you have to allocate the buffer member of your structure
on the external heap ( plus to avoid being gc'd before ffi call)
with ExternalAddress allocate: <your buffer/string size +1 for string null termination>
fill your buffer with byteAt:put: method,
and release this memory with free when you have finished.

not sure, but you could try something like:

s := '05:45' copyWith: (Character value: 0).
a := ExternalAddress allocate: s size.
s asByteArray withIndexDo: [ :b :i | a byteAt: i put: b ].
myStruct time: a.
... call your ffi ...
a free

Regards,

Alain

Le 29/10/2014 11:59, Annick Fron a écrit :

I I have a C struct with a char*

struct result {
char* time }

I define an ExternalStructure in FFI , with one field of type char* (note that the syntax char [5] is not accepted ! ).

defineFields generates the accessors time and time:

If I use
time: ’05:45’

I get an error.
How do I set a string value in an ExternalStructure ?

Annick

Annick

Re: FFI structs

In reply to this post by stepharo

This is good news !

Annick

Le 30 oct. 2014 à 23:37, stepharo <[hidden email]> a écrit :

Ronie is working on unifying all the api and proposing one syntax mechanism for
    - nativeBoost back-end
    - FFI
    and may be Alien

The FFI situation is not good and we know we should improve it.

On 29/10/14 20:18, Thomas Bany wrote:

Oops, sorry, I was misslead by Steph reply.

Some of the comments might still hold. For the time: ’05:45’ code to work, you would need FFI to handle all the hassle of memory management and actually allocate the memory for 5 characters and copying it. I never used FFI but I doubt it does this.

Looking at FFI library, you both have ExternalAddress (for the struct with char*) and ExternalType (for the struct with char[5]) that you can use. For the later solution, it still looks like you should define a specific 5 bytes ArrayOf5Char type.

2014-10-29 18:34 GMT+01:00 Annick Fron <[hidden email]>:

I can’t use NativeBoost on the Raspberry, I would be glad if I could !!!

Le 29 oct. 2014 à 12:18, stepharo <[hidden email]> a écrit :

> did you read the NativeBoost tutorial on the PharoForTheEntreprise book?
> If you do please report potential mistakes so that we can improve.
>
>
> On 29/10/14 05:59, Annick Fron wrote:
>> I I have a C struct with a char*
>>
>> struct result {
>> char* time }
>>
>> I define an ExternalStructure in FFI , with one field of type char* (note that the syntax char [5] is not accepted ! ).
>>
>> defineFields generates the accessors time and time:
>>
>> If I use
>> time: ’05:45’
>>
>> I get an error.
>> How do I set a string value in an ExternalStructure ?
>>
>> Annick
>>
>>
>>
>>
>>
>
>

Alain Rastoul-2

Re: FFI structs

In reply to this post by Ronie Salgado

Hi Ronie,

Great that you work on a threaded FFI because it is a pain for some
libraries. FFI ODBC for example, works very well but is not usable in
real applications because of sql locks that locks the pharo vm.
I don't like FFI too much, NativeBoost is much better, unfortunately I don't
know about other libraries you are talking about (Alien, LowCode and UFFI),
but if you achieve doing something better than NativeBoost, it will be a
great thing.
I wish you succeed.

Regards,

Alain

Le 01/11/2014 00:54, Ronie Salgado a écrit :

> Ronie is working on unifying all the api and proposing one syntax
> mechanism for
> - nativeBoost back-end
> - FFI
> and may be Alien
>
> I had a skype session with Eliot the last. He taught me about the
> callback mechanism and I am actually going to use Alien for the
> callbacks, by adapting it to perform the marshalling into the
> UFFI/NativeBoost types.
>
> In the future, I am planning in using Lowcode to perform the whole
> callback process, but now for simplicity I am just going to piggy back
> in Alien.
>
> I still have to finish the callout by fixing a complicated bug. With
> Eliot we discussed a workaround for this bug. I haven't implemented
> those workaround yet, because I have been a bit busy with my university
> studies.
>
> As long as we keep what is working, ahem, working. I now depend on
> quite a couple of FFI related libs.
>
> What I need is an easy FFI wrapping tool, like
> https://github.com/ronsaldo/swig. Is UFFI going that way?
>
> Well, I made the swig adaptation for NativeBoost. Making it for the UFFI
> is actually something trivial. So, we are going to have a powerful FFI
> and a binding generator for it.
>
> The only problem with Swig, is that the bindings generated by it are not
> perfect, because it exposes the exact same C API. A C library that uses
> some kind of object model, cannot be exposed in a clean object oriented
> way cleanly by a generic automatic tool.
>
> The FFI situation is not good and we know we should improve it.
>
> For the record, the current FFI situation can be traced to the
> historical origins of Smalltalk and even Squeak. The original Smalltalk
> was an actual operating system, and Squeak was also made as an operating
> system. Because of this, they were not designed to interact with other
> languages.
>
> Getting a perfect FFI is going to take a lot of time, because of other
> problems, such as Multi-threading and the mandatory use of non-blocking
> IO. Fortunately, Eliot made a way to have a threaded FFI, in which a
> single thread owns the VM at a single time. With this threaded FFI we
> can have things such as a callback from a different thread, which is
> required by some libraries such as the sound API of SDL2.
>
> Of course the UFFI is going to support this threaded FFI in some point
> of time. To support the threaded FFI I only have to pin/unpin some
> object, and call two VM functions, disownVM() and ownVM().
>
> As for getting some actual parallelism in Pharo, this is something that
> I think is going to take a big amount of time. For now the best thing we
> can do is to use OpenCL. This week I have been doing some experimenting
> in making an API for linear algebra, that uses OpenCL to perform
> computations in parallel. Later I will publish a video.
>
> Greetings,
> Ronie
>
> 2014-10-31 19:23 GMT-03:00 Alain Rastoul
> <[hidden email]
> <mailto:[hidden email]>>:
>
> Hi Annick,
>
> you can't pass smalltalk strings or objects to ffi as parameters,
> you have to allocate the buffer member of your structure
> on the external heap ( plus to avoid being gc'd before ffi call)
> with ExternalAddress allocate: <your buffer/string size +1 for
> string null termination>
> fill your buffer with byteAt:put: method,
> and release this memory with free when you have finished.
>
> not sure, but you could try something like:
>
> s := '05:45' copyWith: (Character value: 0).
> a := ExternalAddress allocate: s size.
> s asByteArray withIndexDo: [ :b :i | a byteAt: i put: b ].
> myStruct time: a.
> ... call your ffi ...
> a free
>
>
> Regards,
>
> Alain
>
> Le 29/10/2014 11:59, Annick Fron a écrit :
>
> I I have a C struct with a char*
>
> struct result {
> char* time }
>
> I define an ExternalStructure in FFI , with one field of type
> char* (note that the syntax char [5] is not accepted ! ).
>
> defineFields generates the accessors time and time:
>
> If I use
> time: ’05:45’
>
> I get an error.
> How do I set a string value in an ExternalStructure ?
>
> Annick
>
>
>
>
>
>
>
>
>