UFFI and ByteArray

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

UFFI and ByteArray

raffaello.giulietti
Hi,

I'm having a hard time finding documentation on how to copy a portion of
a ByteArray to/from another portion of a heap allocated C byte array
with UFFI.

My current reference is
https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgress/lastSuccessfulBuild/artifact/book-result/UnifiedFFI/UnifiedFFI.pdf

Thanks for directing me at relevant docs or examples and sorry in
advance if I missed some important point in the docs.

RG


Reply | Threaded
Open this post in threaded view
|

Re: UFFI and ByteArray

EstebanLM
usually, something more or less like this:

sourceByteArrayOrExternalAddress := … some ...
destEnternalAddressOrByteArray
        replaceFrom: 1
        to: sourceByteArrayOrExternalAddress size
  with: sourceByteArrayOrExternalAddress
        startingAt: 1

and or course you can play with the starting points and sizes.

cheers,
Esteban


> On 21 Feb 2017, at 17:13, Raffaello Giulietti <[hidden email]> wrote:
>
> Hi,
>
> I'm having a hard time finding documentation on how to copy a portion of a ByteArray to/from another portion of a heap allocated C byte array with UFFI.
>
> My current reference is https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgress/lastSuccessfulBuild/artifact/book-result/UnifiedFFI/UnifiedFFI.pdf
>
> Thanks for directing me at relevant docs or examples and sorry in advance if I missed some important point in the docs.
>
> RG
>
>


Reply | Threaded
Open this post in threaded view
|

Re: UFFI and ByteArray

raffaello.giulietti
Hi Esteban,

not sure I'm understanding your snippet correctly.

Below is my try. It throws a SubscriptOutOfBounds exception in the last
line. The reason is that the implementation of ByteArray
replaceFrom:to:with:startingAt: does not consider that ea is meant to be
a pointer to a C byte array, not an object whose content is to be copied
directly.


| ea ba |
ea := ExternalAddress gcallocate: 200.
ba := ByteArray new: 100.
ba replaceFrom: 1 to: 50 with: ea startingAt: 51.


Am I missing some point?




On 2017-02-21 17:20, Esteban Lorenzano wrote:

> usually, something more or less like this:
>
> sourceByteArrayOrExternalAddress := … some ...
> destEnternalAddressOrByteArray
> replaceFrom: 1
> to: sourceByteArrayOrExternalAddress size
>   with: sourceByteArrayOrExternalAddress
> startingAt: 1
>
> and or course you can play with the starting points and sizes.
>
> cheers,
> Esteban
>
>
>> On 21 Feb 2017, at 17:13, Raffaello Giulietti <[hidden email]> wrote:
>>
>> Hi,
>>
>> I'm having a hard time finding documentation on how to copy a portion of a ByteArray to/from another portion of a heap allocated C byte array with UFFI.
>>
>> My current reference is https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgress/lastSuccessfulBuild/artifact/book-result/UnifiedFFI/UnifiedFFI.pdf
>>
>> Thanks for directing me at relevant docs or examples and sorry in advance if I missed some important point in the docs.
>>
>> RG
>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: UFFI and ByteArray

raffaello.giulietti
Hi,

browsing through ByteArray's and ExternalAddress's code, I'm getting the
impression that copying *portions* of ByteArrays from/to the C heap can
be quite costly.

Copying *to* the C heap sometimes requires first copying the portion of
the ByteArray into a newly instantiated temporary ByteArray, then
copying the latter to the C heap.

Copying *from* the C heap might require explicitly looping over the
single bytes. I say "might" because I still cannot find more direct way,
as using ByteArray>>replaceFrom:to:with:startingAt: doesn't seem to work.

Any suggestion on more effective copy operations?

Thanks
Raffaello






On 22/02/17 15:08, Raffaello Giulietti wrote:

> Hi Esteban,
>
> not sure I'm understanding your snippet correctly.
>
> Below is my try. It throws a SubscriptOutOfBounds exception in the last
> line. The reason is that the implementation of ByteArray
> replaceFrom:to:with:startingAt: does not consider that ea is meant to be
> a pointer to a C byte array, not an object whose content is to be copied
> directly.
>
>
> | ea ba |
> ea := ExternalAddress gcallocate: 200.
> ba := ByteArray new: 100.
> ba replaceFrom: 1 to: 50 with: ea startingAt: 51.
>
>
> Am I missing some point?
>
>
>
>
> On 2017-02-21 17:20, Esteban Lorenzano wrote:
>> usually, something more or less like this:
>>
>> sourceByteArrayOrExternalAddress := … some ...
>> destEnternalAddressOrByteArray
>>     replaceFrom: 1
>>     to: sourceByteArrayOrExternalAddress size
>>       with: sourceByteArrayOrExternalAddress
>>     startingAt: 1
>>
>> and or course you can play with the starting points and sizes.
>>
>> cheers,
>> Esteban
>>
>>
>>> On 21 Feb 2017, at 17:13, Raffaello Giulietti
>>> <[hidden email]> wrote:
>>>
>>> Hi,
>>>
>>> I'm having a hard time finding documentation on how to copy a portion
>>> of a ByteArray to/from another portion of a heap allocated C byte
>>> array with UFFI.
>>>
>>> My current reference is
>>> https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgress/lastSuccessfulBuild/artifact/book-result/UnifiedFFI/UnifiedFFI.pdf
>>>
>>>
>>> Thanks for directing me at relevant docs or examples and sorry in
>>> advance if I missed some important point in the docs.
>>>
>>> RG
>>>
>>>
>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: UFFI and ByteArray

EstebanLM
Sorry, I forgot to answer and now I'm in movement.
Look in LibC memcpy method (is not called exactly like that) and his uses. That's the faster you can move things inside the image.

And remember: an EA is a pointer and a ByteArray is an array who will be passed as pointer too.

Esteban

> On 23 Feb 2017, at 17:21, [hidden email] wrote:
>
> Hi,
>
> browsing through ByteArray's and ExternalAddress's code, I'm getting the
> impression that copying *portions* of ByteArrays from/to the C heap can
> be quite costly.
>
> Copying *to* the C heap sometimes requires first copying the portion of
> the ByteArray into a newly instantiated temporary ByteArray, then
> copying the latter to the C heap.
>
> Copying *from* the C heap might require explicitly looping over the
> single bytes. I say "might" because I still cannot find more direct way,
> as using ByteArray>>replaceFrom:to:with:startingAt: doesn't seem to work.
>
> Any suggestion on more effective copy operations?
>
> Thanks
> Raffaello
>
>
>
>
>
>
>> On 22/02/17 15:08, Raffaello Giulietti wrote:
>> Hi Esteban,
>>
>> not sure I'm understanding your snippet correctly.
>>
>> Below is my try. It throws a SubscriptOutOfBounds exception in the last
>> line. The reason is that the implementation of ByteArray
>> replaceFrom:to:with:startingAt: does not consider that ea is meant to be
>> a pointer to a C byte array, not an object whose content is to be copied
>> directly.
>>
>>
>> | ea ba |
>> ea := ExternalAddress gcallocate: 200.
>> ba := ByteArray new: 100.
>> ba replaceFrom: 1 to: 50 with: ea startingAt: 51.
>>
>>
>> Am I missing some point?
>>
>>
>>
>>
>>> On 2017-02-21 17:20, Esteban Lorenzano wrote:
>>> usually, something more or less like this:
>>>
>>> sourceByteArrayOrExternalAddress := … some ...
>>> destEnternalAddressOrByteArray
>>>    replaceFrom: 1
>>>    to: sourceByteArrayOrExternalAddress size
>>>      with: sourceByteArrayOrExternalAddress
>>>    startingAt: 1
>>>
>>> and or course you can play with the starting points and sizes.
>>>
>>> cheers,
>>> Esteban
>>>
>>>
>>>> On 21 Feb 2017, at 17:13, Raffaello Giulietti
>>>> <[hidden email]> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I'm having a hard time finding documentation on how to copy a portion
>>>> of a ByteArray to/from another portion of a heap allocated C byte
>>>> array with UFFI.
>>>>
>>>> My current reference is
>>>> https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgress/lastSuccessfulBuild/artifact/book-result/UnifiedFFI/UnifiedFFI.pdf
>>>>
>>>>
>>>> Thanks for directing me at relevant docs or examples and sorry in
>>>> advance if I missed some important point in the docs.
>>>>
>>>> RG
>>>>
>>>>
>>>
>>>
>>
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: UFFI and ByteArray

stepharong
In reply to this post by raffaello.giulietti
Hi rafaello

once you find the solution can you do a pull request on the UFFI doc?
Your question is important we should improve the documentation.
tx

Stf

> Hi,
>
> browsing through ByteArray's and ExternalAddress's code, I'm getting the
> impression that copying *portions* of ByteArrays from/to the C heap can
> be quite costly.
>
> Copying *to* the C heap sometimes requires first copying the portion of
> the ByteArray into a newly instantiated temporary ByteArray, then
> copying the latter to the C heap.
>
> Copying *from* the C heap might require explicitly looping over the
> single bytes. I say "might" because I still cannot find more direct way,
> as using ByteArray>>replaceFrom:to:with:startingAt: doesn't seem to work.
>
> Any suggestion on more effective copy operations?
>
> Thanks
> Raffaello
>
>
>
>
>
>
> On 22/02/17 15:08, Raffaello Giulietti wrote:
>> Hi Esteban,
>>
>> not sure I'm understanding your snippet correctly.
>>
>> Below is my try. It throws a SubscriptOutOfBounds exception in the last
>> line. The reason is that the implementation of ByteArray
>> replaceFrom:to:with:startingAt: does not consider that ea is meant to be
>> a pointer to a C byte array, not an object whose content is to be copied
>> directly.
>>
>>
>> | ea ba |
>> ea := ExternalAddress gcallocate: 200.
>> ba := ByteArray new: 100.
>> ba replaceFrom: 1 to: 50 with: ea startingAt: 51.
>>
>>
>> Am I missing some point?
>>
>>
>>
>>
>> On 2017-02-21 17:20, Esteban Lorenzano wrote:
>>> usually, something more or less like this:
>>>
>>> sourceByteArrayOrExternalAddress := … some ...
>>> destEnternalAddressOrByteArray
>>>     replaceFrom: 1
>>>     to: sourceByteArrayOrExternalAddress size
>>>       with: sourceByteArrayOrExternalAddress
>>>     startingAt: 1
>>>
>>> and or course you can play with the starting points and sizes.
>>>
>>> cheers,
>>> Esteban
>>>
>>>
>>>> On 21 Feb 2017, at 17:13, Raffaello Giulietti
>>>> <[hidden email]> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I'm having a hard time finding documentation on how to copy a portion
>>>> of a ByteArray to/from another portion of a heap allocated C byte
>>>> array with UFFI.
>>>>
>>>> My current reference is
>>>> https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgress/lastSuccessfulBuild/artifact/book-result/UnifiedFFI/UnifiedFFI.pdf
>>>>
>>>>
>>>> Thanks for directing me at relevant docs or examples and sorry in
>>>> advance if I missed some important point in the docs.
>>>>
>>>> RG
>>>>
>>>>
>>>
>>>
>>
>>
>
>


--
Using Opera's mail client: http://www.opera.com/mail/

Reply | Threaded
Open this post in threaded view
|

Re: UFFI and ByteArray

raffaello.giulietti
In reply to this post by EstebanLM
Hi Esteban,

yes, I already looked at LibC's memcpy, but that doesn't help in case
you need to copy a *portion* of the ByteArray that does not start at
index 1. While one can do pointer arithmetic on ExternalAddress, this
does not, of course, apply to ByteArray.

To sum up, I think Pharo 5 currently lacks an efficient way to copy
parts of ByteArrays to/from the C heap. One has to instantiate
additional temporary ByteArrays and perform copies between ByteArrays in
addition to LibC>>memcpy.

But I see there's an Alien class which seems to better support such
copying. Thus, I wonder if it is somehow compatible with the rest of UFFI.






On 23/02/17 17:35, Esteban Lorenzano wrote:

> Sorry, I forgot to answer and now I'm in movement.
> Look in LibC memcpy method (is not called exactly like that) and his uses. That's the faster you can move things inside the image.
>
> And remember: an EA is a pointer and a ByteArray is an array who will be passed as pointer too.
>
> Esteban
>
>> On 23 Feb 2017, at 17:21, [hidden email] wrote:
>>
>> Hi,
>>
>> browsing through ByteArray's and ExternalAddress's code, I'm getting the
>> impression that copying *portions* of ByteArrays from/to the C heap can
>> be quite costly.
>>
>> Copying *to* the C heap sometimes requires first copying the portion of
>> the ByteArray into a newly instantiated temporary ByteArray, then
>> copying the latter to the C heap.
>>
>> Copying *from* the C heap might require explicitly looping over the
>> single bytes. I say "might" because I still cannot find more direct way,
>> as using ByteArray>>replaceFrom:to:with:startingAt: doesn't seem to work.
>>
>> Any suggestion on more effective copy operations?
>>
>> Thanks
>> Raffaello
>>
>>
>>
>>
>>
>>
>>> On 22/02/17 15:08, Raffaello Giulietti wrote:
>>> Hi Esteban,
>>>
>>> not sure I'm understanding your snippet correctly.
>>>
>>> Below is my try. It throws a SubscriptOutOfBounds exception in the last
>>> line. The reason is that the implementation of ByteArray
>>> replaceFrom:to:with:startingAt: does not consider that ea is meant to be
>>> a pointer to a C byte array, not an object whose content is to be copied
>>> directly.
>>>
>>>
>>> | ea ba |
>>> ea := ExternalAddress gcallocate: 200.
>>> ba := ByteArray new: 100.
>>> ba replaceFrom: 1 to: 50 with: ea startingAt: 51.
>>>
>>>
>>> Am I missing some point?
>>>
>>>
>>>
>>>
>>>> On 2017-02-21 17:20, Esteban Lorenzano wrote:
>>>> usually, something more or less like this:
>>>>
>>>> sourceByteArrayOrExternalAddress := … some ...
>>>> destEnternalAddressOrByteArray
>>>>    replaceFrom: 1
>>>>    to: sourceByteArrayOrExternalAddress size
>>>>      with: sourceByteArrayOrExternalAddress
>>>>    startingAt: 1
>>>>
>>>> and or course you can play with the starting points and sizes.
>>>>
>>>> cheers,
>>>> Esteban
>>>>
>>>>
>>>>> On 21 Feb 2017, at 17:13, Raffaello Giulietti
>>>>> <[hidden email]> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I'm having a hard time finding documentation on how to copy a portion
>>>>> of a ByteArray to/from another portion of a heap allocated C byte
>>>>> array with UFFI.
>>>>>
>>>>> My current reference is
>>>>> https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgress/lastSuccessfulBuild/artifact/book-result/UnifiedFFI/UnifiedFFI.pdf
>>>>>
>>>>>
>>>>> Thanks for directing me at relevant docs or examples and sorry in
>>>>> advance if I missed some important point in the docs.
>>>>>
>>>>> RG
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>


Reply | Threaded
Open this post in threaded view
|

Re: UFFI and ByteArray

EstebanLM

> On 23 Feb 2017, at 19:11, [hidden email] wrote:
>
> Hi Esteban,
>
> yes, I already looked at LibC's memcpy, but that doesn't help in case
> you need to copy a *portion* of the ByteArray that does not start at
> index 1. While one can do pointer arithmetic on ExternalAddress, this
> does not, of course, apply to ByteArray.
>
> To sum up, I think Pharo 5 currently lacks an efficient way to copy
> parts of ByteArrays to/from the C heap. One has to instantiate
> additional temporary ByteArrays and perform copies between ByteArrays in
> addition to LibC>>memcpy.
>
> But I see there's an Alien class which seems to better support such
> copying. Thus, I wonder if it is somehow compatible with the rest of UFFI.

Which Alien class?

Esteban

>
>
>
>
>
>
> On 23/02/17 17:35, Esteban Lorenzano wrote:
>> Sorry, I forgot to answer and now I'm in movement.
>> Look in LibC memcpy method (is not called exactly like that) and his uses. That's the faster you can move things inside the image.
>>
>> And remember: an EA is a pointer and a ByteArray is an array who will be passed as pointer too.
>>
>> Esteban
>>
>>> On 23 Feb 2017, at 17:21, [hidden email] wrote:
>>>
>>> Hi,
>>>
>>> browsing through ByteArray's and ExternalAddress's code, I'm getting the
>>> impression that copying *portions* of ByteArrays from/to the C heap can
>>> be quite costly.
>>>
>>> Copying *to* the C heap sometimes requires first copying the portion of
>>> the ByteArray into a newly instantiated temporary ByteArray, then
>>> copying the latter to the C heap.
>>>
>>> Copying *from* the C heap might require explicitly looping over the
>>> single bytes. I say "might" because I still cannot find more direct way,
>>> as using ByteArray>>replaceFrom:to:with:startingAt: doesn't seem to work.
>>>
>>> Any suggestion on more effective copy operations?
>>>
>>> Thanks
>>> Raffaello
>>>
>>>
>>>
>>>
>>>
>>>
>>>> On 22/02/17 15:08, Raffaello Giulietti wrote:
>>>> Hi Esteban,
>>>>
>>>> not sure I'm understanding your snippet correctly.
>>>>
>>>> Below is my try. It throws a SubscriptOutOfBounds exception in the last
>>>> line. The reason is that the implementation of ByteArray
>>>> replaceFrom:to:with:startingAt: does not consider that ea is meant to be
>>>> a pointer to a C byte array, not an object whose content is to be copied
>>>> directly.
>>>>
>>>>
>>>> | ea ba |
>>>> ea := ExternalAddress gcallocate: 200.
>>>> ba := ByteArray new: 100.
>>>> ba replaceFrom: 1 to: 50 with: ea startingAt: 51.
>>>>
>>>>
>>>> Am I missing some point?
>>>>
>>>>
>>>>
>>>>
>>>>> On 2017-02-21 17:20, Esteban Lorenzano wrote:
>>>>> usually, something more or less like this:
>>>>>
>>>>> sourceByteArrayOrExternalAddress := … some ...
>>>>> destEnternalAddressOrByteArray
>>>>>   replaceFrom: 1
>>>>>   to: sourceByteArrayOrExternalAddress size
>>>>>     with: sourceByteArrayOrExternalAddress
>>>>>   startingAt: 1
>>>>>
>>>>> and or course you can play with the starting points and sizes.
>>>>>
>>>>> cheers,
>>>>> Esteban
>>>>>
>>>>>
>>>>>> On 21 Feb 2017, at 17:13, Raffaello Giulietti
>>>>>> <[hidden email]> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I'm having a hard time finding documentation on how to copy a portion
>>>>>> of a ByteArray to/from another portion of a heap allocated C byte
>>>>>> array with UFFI.
>>>>>>
>>>>>> My current reference is
>>>>>> https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgress/lastSuccessfulBuild/artifact/book-result/UnifiedFFI/UnifiedFFI.pdf
>>>>>>
>>>>>>
>>>>>> Thanks for directing me at relevant docs or examples and sorry in
>>>>>> advance if I missed some important point in the docs.
>>>>>>
>>>>>> RG
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: UFFI and ByteArray

raffaello.giulietti
On 23/02/17 19:17, Esteban Lorenzano wrote:

>
>> On 23 Feb 2017, at 19:11, [hidden email] wrote:
>>
>> Hi Esteban,
>>
>> yes, I already looked at LibC's memcpy, but that doesn't help in case
>> you need to copy a *portion* of the ByteArray that does not start at
>> index 1. While one can do pointer arithmetic on ExternalAddress, this
>> does not, of course, apply to ByteArray.
>>
>> To sum up, I think Pharo 5 currently lacks an efficient way to copy
>> parts of ByteArrays to/from the C heap. One has to instantiate
>> additional temporary ByteArrays and perform copies between ByteArrays in
>> addition to LibC>>memcpy.
>>
>> But I see there's an Alien class which seems to better support such
>> copying. Thus, I wonder if it is somehow compatible with the rest of UFFI.
>
> Which Alien class?
>
> Esteban

What do you mean?
I see it in package Alien-Core.




>
>>
>>
>>
>>
>>
>>
>> On 23/02/17 17:35, Esteban Lorenzano wrote:
>>> Sorry, I forgot to answer and now I'm in movement.
>>> Look in LibC memcpy method (is not called exactly like that) and his uses. That's the faster you can move things inside the image.
>>>
>>> And remember: an EA is a pointer and a ByteArray is an array who will be passed as pointer too.
>>>
>>> Esteban
>>>
>>>> On 23 Feb 2017, at 17:21, [hidden email] wrote:
>>>>
>>>> Hi,
>>>>
>>>> browsing through ByteArray's and ExternalAddress's code, I'm getting the
>>>> impression that copying *portions* of ByteArrays from/to the C heap can
>>>> be quite costly.
>>>>
>>>> Copying *to* the C heap sometimes requires first copying the portion of
>>>> the ByteArray into a newly instantiated temporary ByteArray, then
>>>> copying the latter to the C heap.
>>>>
>>>> Copying *from* the C heap might require explicitly looping over the
>>>> single bytes. I say "might" because I still cannot find more direct way,
>>>> as using ByteArray>>replaceFrom:to:with:startingAt: doesn't seem to work.
>>>>
>>>> Any suggestion on more effective copy operations?
>>>>
>>>> Thanks
>>>> Raffaello
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> On 22/02/17 15:08, Raffaello Giulietti wrote:
>>>>> Hi Esteban,
>>>>>
>>>>> not sure I'm understanding your snippet correctly.
>>>>>
>>>>> Below is my try. It throws a SubscriptOutOfBounds exception in the last
>>>>> line. The reason is that the implementation of ByteArray
>>>>> replaceFrom:to:with:startingAt: does not consider that ea is meant to be
>>>>> a pointer to a C byte array, not an object whose content is to be copied
>>>>> directly.
>>>>>
>>>>>
>>>>> | ea ba |
>>>>> ea := ExternalAddress gcallocate: 200.
>>>>> ba := ByteArray new: 100.
>>>>> ba replaceFrom: 1 to: 50 with: ea startingAt: 51.
>>>>>
>>>>>
>>>>> Am I missing some point?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> On 2017-02-21 17:20, Esteban Lorenzano wrote:
>>>>>> usually, something more or less like this:
>>>>>>
>>>>>> sourceByteArrayOrExternalAddress := … some ...
>>>>>> destEnternalAddressOrByteArray
>>>>>>   replaceFrom: 1
>>>>>>   to: sourceByteArrayOrExternalAddress size
>>>>>>     with: sourceByteArrayOrExternalAddress
>>>>>>   startingAt: 1
>>>>>>
>>>>>> and or course you can play with the starting points and sizes.
>>>>>>
>>>>>> cheers,
>>>>>> Esteban
>>>>>>
>>>>>>
>>>>>>> On 21 Feb 2017, at 17:13, Raffaello Giulietti
>>>>>>> <[hidden email]> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I'm having a hard time finding documentation on how to copy a portion
>>>>>>> of a ByteArray to/from another portion of a heap allocated C byte
>>>>>>> array with UFFI.
>>>>>>>
>>>>>>> My current reference is
>>>>>>> https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgress/lastSuccessfulBuild/artifact/book-result/UnifiedFFI/UnifiedFFI.pdf
>>>>>>>
>>>>>>>
>>>>>>> Thanks for directing me at relevant docs or examples and sorry in
>>>>>>> advance if I missed some important point in the docs.
>>>>>>>
>>>>>>> RG
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: UFFI and ByteArray

EstebanLM

On 23 Feb 2017, at 19:21, [hidden email] wrote:

On 23/02/17 19:17, Esteban Lorenzano wrote:

On 23 Feb 2017, at 19:11, [hidden email] wrote:

Hi Esteban,

yes, I already looked at LibC's memcpy, but that doesn't help in case
you need to copy a *portion* of the ByteArray that does not start at
index 1. While one can do pointer arithmetic on ExternalAddress, this
does not, of course, apply to ByteArray.

To sum up, I think Pharo 5 currently lacks an efficient way to copy
parts of ByteArrays to/from the C heap. One has to instantiate
additional temporary ByteArrays and perform copies between ByteArrays in
addition to LibC>>memcpy.

But I see there's an Alien class which seems to better support such
copying. Thus, I wonder if it is somehow compatible with the rest of UFFI.

Which Alien class?

Esteban

What do you mean?
I see it in package Alien-Core.

ah, problem is that #copyInto:from:to:in:startingAt: requires an Alien instead a ExternalAddress (and they are not the same). But I guess you can use it if you have two ByteArrays.

in the times of old NativeBoost, Igor implemented a memcpy as you want it, because yes, is a problem we have… if is seriously needed, you can always compile your own library with such symbol… a memcpy with start positions. 
function in C is more or less trivial because you will receive two pointers and you can perform pointer arithmetic on both.

Esteban










On 23/02/17 17:35, Esteban Lorenzano wrote:
Sorry, I forgot to answer and now I'm in movement. 
Look in LibC memcpy method (is not called exactly like that) and his uses. That's the faster you can move things inside the image.

And remember: an EA is a pointer and a ByteArray is an array who will be passed as pointer too.

Esteban 

On 23 Feb 2017, at 17:21, [hidden email] wrote:

Hi,

browsing through ByteArray's and ExternalAddress's code, I'm getting the
impression that copying *portions* of ByteArrays from/to the C heap can
be quite costly.

Copying *to* the C heap sometimes requires first copying the portion of
the ByteArray into a newly instantiated temporary ByteArray, then
copying the latter to the C heap.

Copying *from* the C heap might require explicitly looping over the
single bytes. I say "might" because I still cannot find more direct way,
as using ByteArray>>replaceFrom:to:with:startingAt: doesn't seem to work.

Any suggestion on more effective copy operations?

Thanks
Raffaello






On 22/02/17 15:08, Raffaello Giulietti wrote:
Hi Esteban,

not sure I'm understanding your snippet correctly.

Below is my try. It throws a SubscriptOutOfBounds exception in the last
line. The reason is that the implementation of ByteArray
replaceFrom:to:with:startingAt: does not consider that ea is meant to be
a pointer to a C byte array, not an object whose content is to be copied
directly.


| ea ba |
ea := ExternalAddress gcallocate: 200.
ba := ByteArray new: 100.
ba replaceFrom: 1 to: 50 with: ea startingAt: 51.


Am I missing some point?




On 2017-02-21 17:20, Esteban Lorenzano wrote:
usually, something more or less like this:

sourceByteArrayOrExternalAddress := … some ...
destEnternalAddressOrByteArray
 replaceFrom: 1
 to: sourceByteArrayOrExternalAddress size
   with: sourceByteArrayOrExternalAddress
 startingAt: 1

and or course you can play with the starting points and sizes.

cheers,
Esteban


On 21 Feb 2017, at 17:13, Raffaello Giulietti
<[hidden email]> wrote:

Hi,

I'm having a hard time finding documentation on how to copy a portion
of a ByteArray to/from another portion of a heap allocated C byte
array with UFFI.

My current reference is
https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgress/lastSuccessfulBuild/artifact/book-result/UnifiedFFI/UnifiedFFI.pdf


Thanks for directing me at relevant docs or examples and sorry in
advance if I missed some important point in the docs.

RG

Reply | Threaded
Open this post in threaded view
|

Re: UFFI and ByteArray

raffaello.giulietti
On 23/02/17 19:26, Esteban Lorenzano wrote:

>
>> On 23 Feb 2017, at 19:21, [hidden email]
>> <mailto:[hidden email]> wrote:
>>
>> On 23/02/17 19:17, Esteban Lorenzano wrote:
>>>
>>>> On 23 Feb 2017, at 19:11, [hidden email]
>>>> <mailto:[hidden email]> wrote:
>>>>
>>>> Hi Esteban,
>>>>
>>>> yes, I already looked at LibC's memcpy, but that doesn't help in case
>>>> you need to copy a *portion* of the ByteArray that does not start at
>>>> index 1. While one can do pointer arithmetic on ExternalAddress, this
>>>> does not, of course, apply to ByteArray.
>>>>
>>>> To sum up, I think Pharo 5 currently lacks an efficient way to copy
>>>> parts of ByteArrays to/from the C heap. One has to instantiate
>>>> additional temporary ByteArrays and perform copies between ByteArrays in
>>>> addition to LibC>>memcpy.
>>>>
>>>> But I see there's an Alien class which seems to better support such
>>>> copying. Thus, I wonder if it is somehow compatible with the rest of
>>>> UFFI.
>>>
>>> Which Alien class?
>>>
>>> Esteban
>>
>> What do you mean?
>> I see it in package Alien-Core.
>
> ah, problem is that #copyInto:from:to:in:startingAt: requires an Alien
> instead a ExternalAddress (and they are not the same). But I guess you
> can use it if you have two ByteArrays.
>
> in the times of old NativeBoost, Igor implemented a memcpy as you want
> it, because yes, is a problem we have… if is seriously needed, you can
> always compile your own library with such symbol… a memcpy with start
> positions.
> function in C is more or less trivial because you will receive two
> pointers and you can perform pointer arithmetic on both.
>
> Esteban
>

What about an existing library that I don't want or cannot modify?

Really, I agree that this is a problem that Pharo has to solve. I think
the UFFI deserves a more general and cheaper way of copying between
ByteArrays and the C heap.

Can I expect it to appear in the short term?
Can I help somehow?

Raffaello



Reply | Threaded
Open this post in threaded view
|

Re: UFFI and ByteArray

stepharong

>>
>
> What about an existing library that I don't want or cannot modify?
>
> Really, I agree that this is a problem that Pharo has to solve.

Yes we all agree on that.
For the record, esteban rewrote from Scratch an FFI and it was not in our  
plans due to the change in Spur.
We would have preferred that somebody else does it because we burnt around  
8 months of work
to get throw.

> I think
> the UFFI deserves a more general and cheaper way of copying between
> ByteArrays and the C heap.

Yes anything that makes interaction with C libraries easier, faster, nicer  
is needed.
Now having 64 bits is also key and we cannot clone esteban so far.

> Can I expect it to appear in the short term?
> Can I help somehow?

I'm sure.

Stef