Alien signature: first attempt

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Alien signature: first attempt

Schwab,Wilhelm K
Esteban,

Will this work?  Even if so, is there a better way to define it?

longVoidStarLongRetInt: callbackContext sp: spAlien
        <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
        ^callbackContext wordResult:
                (block
                        value: (Alien newC:4)
                        value: (Alien forPointer: (spAlien unsignedLongAt: 5))
                        value: (Alien newC:4)
                )

Bill

________________________________________
From: [hidden email] [[hidden email]] on behalf of Schwab,Wilhelm K [[hidden email]]
Sent: Thursday, March 01, 2012 12:15 AM
To: [hidden email]
Subject: Re: [Pharo-project] FFI on 1.3: compile fields

thanks!!!!!




________________________________________
From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
Sent: Wednesday, February 29, 2012 10:18 PM
To: [hidden email]
Subject: Re: [Pharo-project] FFI on 1.3: compile fields

you need to define a signature... not at home now, but wait 'til tomorrow and I'll send an example

best,
Esteban

El 29/02/2012, a las 11:21p.m., Schwab,Wilhelm K escribió:

> What does "cannot find callback signature" mean from #signature:block:?  I'm stuck.
>
>
>
>
> ________________________________________
> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
> Sent: Wednesday, February 29, 2012 6:54 PM
> To: [hidden email]
> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>
> yeah... sorry about that
> is a bug on FFI... I managed to worked around it for HPDF, for a customer's project... but of course is not a good and definitive solution.
>
> best,
> Esteban
>
> El 29/02/2012, a las 8:37p.m., Schwab,Wilhelm K escribió:
>
>> This is gonna take a while...   I had structs flying around as void* and was moderately happy.  Nonetheless, your suggestion worked, provided I add a lot getHandle asInteger and change the void* to long :(
>>
>>
>>
>>
>> ________________________________________
>> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
>> Sent: Wednesday, February 29, 2012 6:23 PM
>> To: [hidden email]
>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>
>> I also found some problems using void* in linux... maybe you want to use long (which has same size)... I "fixed" my problems that way.
>>
>> yes... maybe we need to look at FFI to see why void* has problems some times, but well, that can help you atm (sorry for not having a better answer)
>>
>> Esteban
>>
>> El 29/02/2012, a las 8:11p.m., Igor Stasenko escribió:
>>
>>> On 1 March 2012 00:58, Schwab,Wilhelm K <[hidden email]> wrote:
>>>> Another glitch: is there any problem passing things as void*?  I'm getting failure to coerce errors that did not arise before.
>>>>
>>> No idea. As you may suspect, i stopped using FFI/Alien once i got
>>> NativeBoost toy to play with.
>>>
>>> Please file the issue, describing the problem. so we can look over it
>>> and fix it.
>>>
>>>>
>>>>
>>>> ________________________________________
>>>> From: [hidden email] [[hidden email]] on behalf of Igor Stasenko [[hidden email]]
>>>> Sent: Wednesday, February 29, 2012 5:51 PM
>>>> To: [hidden email]
>>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>>>
>>>> On 1 March 2012 00:37, Schwab,Wilhelm K <[hidden email]> wrote:
>>>>> Does 1.3 by default not create field accessors?  Why is that?  I thought
>>>>> nothing was happening.
>>>>>
>>>>
>>>> Good question, i'd like to know the answer too.
>>>> It is related to MC final 'installation' phase,
>>>> where it initializing all classes.
>>>>
>>>> In NativeBoost i was also using
>>>>
>>>> noteCompilationOf: aSelector meta: isMeta
>>>>      "A hook allowing some classes to react to recompilation of certain selectors"
>>>>
>>>> but there was a big question, at which point this hook is triggered,
>>>> and looks like some changes in MC stop triggering it/triggering at
>>>> wrong time (not all methods get into a class/ class not initialized
>>>> etc),
>>>> which makes it not very useful.
>>>>
>>>> So i ended up creating a DNU handler on instance side, then on DNU i
>>>> check if i have field accessors and if not,
>>>> compile them on the fly.
>>>>
>>>>
>>>>> Bill
>>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Igor Stasenko.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko.
>>>
>>
>>
>>
>
>
>




Reply | Threaded
Open this post in threaded view
|

Re: Alien signature: first attempt

EstebanLM
Hi,

sorry for not sending before: work is hard :P
yes, that should work, and no, I don't know a better way to define it. I think Eliot's idea is to form "libraries" of generic callbacks (for mars, for instance, I have CocoaCallback, child of Callback, who defines all my needed signatures, looks a lot at first instance, but if you see in detail, finally there are not so much variants)

best,
Esteban

El 01/03/2012, a las 3:57p.m., Schwab,Wilhelm K escribió:

> Esteban,
>
> Will this work?  Even if so, is there a better way to define it?
>
> longVoidStarLongRetInt: callbackContext sp: spAlien
> <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
> ^callbackContext wordResult:
> (block
> value: (Alien newC:4)
> value: (Alien forPointer: (spAlien unsignedLongAt: 5))
> value: (Alien newC:4)
> )
>
> Bill
>
> ________________________________________
> From: [hidden email] [[hidden email]] on behalf of Schwab,Wilhelm K [[hidden email]]
> Sent: Thursday, March 01, 2012 12:15 AM
> To: [hidden email]
> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>
> thanks!!!!!
>
>
>
>
> ________________________________________
> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
> Sent: Wednesday, February 29, 2012 10:18 PM
> To: [hidden email]
> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>
> you need to define a signature... not at home now, but wait 'til tomorrow and I'll send an example
>
> best,
> Esteban
>
> El 29/02/2012, a las 11:21p.m., Schwab,Wilhelm K escribió:
>
>> What does "cannot find callback signature" mean from #signature:block:?  I'm stuck.
>>
>>
>>
>>
>> ________________________________________
>> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
>> Sent: Wednesday, February 29, 2012 6:54 PM
>> To: [hidden email]
>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>
>> yeah... sorry about that
>> is a bug on FFI... I managed to worked around it for HPDF, for a customer's project... but of course is not a good and definitive solution.
>>
>> best,
>> Esteban
>>
>> El 29/02/2012, a las 8:37p.m., Schwab,Wilhelm K escribió:
>>
>>> This is gonna take a while...   I had structs flying around as void* and was moderately happy.  Nonetheless, your suggestion worked, provided I add a lot getHandle asInteger and change the void* to long :(
>>>
>>>
>>>
>>>
>>> ________________________________________
>>> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
>>> Sent: Wednesday, February 29, 2012 6:23 PM
>>> To: [hidden email]
>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>>
>>> I also found some problems using void* in linux... maybe you want to use long (which has same size)... I "fixed" my problems that way.
>>>
>>> yes... maybe we need to look at FFI to see why void* has problems some times, but well, that can help you atm (sorry for not having a better answer)
>>>
>>> Esteban
>>>
>>> El 29/02/2012, a las 8:11p.m., Igor Stasenko escribió:
>>>
>>>> On 1 March 2012 00:58, Schwab,Wilhelm K <[hidden email]> wrote:
>>>>> Another glitch: is there any problem passing things as void*?  I'm getting failure to coerce errors that did not arise before.
>>>>>
>>>> No idea. As you may suspect, i stopped using FFI/Alien once i got
>>>> NativeBoost toy to play with.
>>>>
>>>> Please file the issue, describing the problem. so we can look over it
>>>> and fix it.
>>>>
>>>>>
>>>>>
>>>>> ________________________________________
>>>>> From: [hidden email] [[hidden email]] on behalf of Igor Stasenko [[hidden email]]
>>>>> Sent: Wednesday, February 29, 2012 5:51 PM
>>>>> To: [hidden email]
>>>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>>>>
>>>>> On 1 March 2012 00:37, Schwab,Wilhelm K <[hidden email]> wrote:
>>>>>> Does 1.3 by default not create field accessors?  Why is that?  I thought
>>>>>> nothing was happening.
>>>>>>
>>>>>
>>>>> Good question, i'd like to know the answer too.
>>>>> It is related to MC final 'installation' phase,
>>>>> where it initializing all classes.
>>>>>
>>>>> In NativeBoost i was also using
>>>>>
>>>>> noteCompilationOf: aSelector meta: isMeta
>>>>>     "A hook allowing some classes to react to recompilation of certain selectors"
>>>>>
>>>>> but there was a big question, at which point this hook is triggered,
>>>>> and looks like some changes in MC stop triggering it/triggering at
>>>>> wrong time (not all methods get into a class/ class not initialized
>>>>> etc),
>>>>> which makes it not very useful.
>>>>>
>>>>> So i ended up creating a DNU handler on instance side, then on DNU i
>>>>> check if i have field accessors and if not,
>>>>> compile them on the fly.
>>>>>
>>>>>
>>>>>> Bill
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Igor Stasenko.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Igor Stasenko.
>>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Alien signature: first attempt

Schwab,Wilhelm K
Esteban,

Would you be willing to post your subclass?  It might be useful both in defining signatures and in naming them.

Bill



________________________________________
From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
Sent: Thursday, March 01, 2012 2:08 PM
To: [hidden email]
Subject: Re: [Pharo-project] Alien signature: first attempt

Hi,

sorry for not sending before: work is hard :P
yes, that should work, and no, I don't know a better way to define it. I think Eliot's idea is to form "libraries" of generic callbacks (for mars, for instance, I have CocoaCallback, child of Callback, who defines all my needed signatures, looks a lot at first instance, but if you see in detail, finally there are not so much variants)

best,
Esteban

El 01/03/2012, a las 3:57p.m., Schwab,Wilhelm K escribió:

> Esteban,
>
> Will this work?  Even if so, is there a better way to define it?
>
> longVoidStarLongRetInt: callbackContext sp: spAlien
>       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
>       ^callbackContext wordResult:
>               (block
>                       value: (Alien newC:4)
>                       value: (Alien forPointer: (spAlien unsignedLongAt: 5))
>                       value: (Alien newC:4)
>               )
>
> Bill
>
> ________________________________________
> From: [hidden email] [[hidden email]] on behalf of Schwab,Wilhelm K [[hidden email]]
> Sent: Thursday, March 01, 2012 12:15 AM
> To: [hidden email]
> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>
> thanks!!!!!
>
>
>
>
> ________________________________________
> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
> Sent: Wednesday, February 29, 2012 10:18 PM
> To: [hidden email]
> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>
> you need to define a signature... not at home now, but wait 'til tomorrow and I'll send an example
>
> best,
> Esteban
>
> El 29/02/2012, a las 11:21p.m., Schwab,Wilhelm K escribió:
>
>> What does "cannot find callback signature" mean from #signature:block:?  I'm stuck.
>>
>>
>>
>>
>> ________________________________________
>> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
>> Sent: Wednesday, February 29, 2012 6:54 PM
>> To: [hidden email]
>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>
>> yeah... sorry about that
>> is a bug on FFI... I managed to worked around it for HPDF, for a customer's project... but of course is not a good and definitive solution.
>>
>> best,
>> Esteban
>>
>> El 29/02/2012, a las 8:37p.m., Schwab,Wilhelm K escribió:
>>
>>> This is gonna take a while...   I had structs flying around as void* and was moderately happy.  Nonetheless, your suggestion worked, provided I add a lot getHandle asInteger and change the void* to long :(
>>>
>>>
>>>
>>>
>>> ________________________________________
>>> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
>>> Sent: Wednesday, February 29, 2012 6:23 PM
>>> To: [hidden email]
>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>>
>>> I also found some problems using void* in linux... maybe you want to use long (which has same size)... I "fixed" my problems that way.
>>>
>>> yes... maybe we need to look at FFI to see why void* has problems some times, but well, that can help you atm (sorry for not having a better answer)
>>>
>>> Esteban
>>>
>>> El 29/02/2012, a las 8:11p.m., Igor Stasenko escribió:
>>>
>>>> On 1 March 2012 00:58, Schwab,Wilhelm K <[hidden email]> wrote:
>>>>> Another glitch: is there any problem passing things as void*?  I'm getting failure to coerce errors that did not arise before.
>>>>>
>>>> No idea. As you may suspect, i stopped using FFI/Alien once i got
>>>> NativeBoost toy to play with.
>>>>
>>>> Please file the issue, describing the problem. so we can look over it
>>>> and fix it.
>>>>
>>>>>
>>>>>
>>>>> ________________________________________
>>>>> From: [hidden email] [[hidden email]] on behalf of Igor Stasenko [[hidden email]]
>>>>> Sent: Wednesday, February 29, 2012 5:51 PM
>>>>> To: [hidden email]
>>>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>>>>
>>>>> On 1 March 2012 00:37, Schwab,Wilhelm K <[hidden email]> wrote:
>>>>>> Does 1.3 by default not create field accessors?  Why is that?  I thought
>>>>>> nothing was happening.
>>>>>>
>>>>>
>>>>> Good question, i'd like to know the answer too.
>>>>> It is related to MC final 'installation' phase,
>>>>> where it initializing all classes.
>>>>>
>>>>> In NativeBoost i was also using
>>>>>
>>>>> noteCompilationOf: aSelector meta: isMeta
>>>>>     "A hook allowing some classes to react to recompilation of certain selectors"
>>>>>
>>>>> but there was a big question, at which point this hook is triggered,
>>>>> and looks like some changes in MC stop triggering it/triggering at
>>>>> wrong time (not all methods get into a class/ class not initialized
>>>>> etc),
>>>>> which makes it not very useful.
>>>>>
>>>>> So i ended up creating a DNU handler on instance side, then on DNU i
>>>>> check if i have field accessors and if not,
>>>>> compile them on the fly.
>>>>>
>>>>>
>>>>>> Bill
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Igor Stasenko.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Igor Stasenko.
>>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>
>



Reply | Threaded
Open this post in threaded view
|

Re: Alien signature: first attempt

EstebanLM
Hi,

It is published on http://ss3.gemstone.com/ss/Mars, package Mars-Cocoa.
You can also see the Alien ObjectiveC-Bridge as an example, on http://squeaksource.com/Alien, packages Alien-MaxOSX-* (those are hard to follow, but they work :)

but... my names are more or less equal to yours (I just followed Eliot "de facto" convention)

best,
Esteban

El 01/03/2012, a las 4:23p.m., Schwab,Wilhelm K escribió:

> Esteban,
>
> Would you be willing to post your subclass?  It might be useful both in defining signatures and in naming them.
>
> Bill
>
>
>
> ________________________________________
> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
> Sent: Thursday, March 01, 2012 2:08 PM
> To: [hidden email]
> Subject: Re: [Pharo-project] Alien signature: first attempt
>
> Hi,
>
> sorry for not sending before: work is hard :P
> yes, that should work, and no, I don't know a better way to define it. I think Eliot's idea is to form "libraries" of generic callbacks (for mars, for instance, I have CocoaCallback, child of Callback, who defines all my needed signatures, looks a lot at first instance, but if you see in detail, finally there are not so much variants)
>
> best,
> Esteban
>
> El 01/03/2012, a las 3:57p.m., Schwab,Wilhelm K escribió:
>
>> Esteban,
>>
>> Will this work?  Even if so, is there a better way to define it?
>>
>> longVoidStarLongRetInt: callbackContext sp: spAlien
>>      <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
>>      ^callbackContext wordResult:
>>              (block
>>                      value: (Alien newC:4)
>>                      value: (Alien forPointer: (spAlien unsignedLongAt: 5))
>>                      value: (Alien newC:4)
>>              )
>>
>> Bill
>>
>> ________________________________________
>> From: [hidden email] [[hidden email]] on behalf of Schwab,Wilhelm K [[hidden email]]
>> Sent: Thursday, March 01, 2012 12:15 AM
>> To: [hidden email]
>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>
>> thanks!!!!!
>>
>>
>>
>>
>> ________________________________________
>> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
>> Sent: Wednesday, February 29, 2012 10:18 PM
>> To: [hidden email]
>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>
>> you need to define a signature... not at home now, but wait 'til tomorrow and I'll send an example
>>
>> best,
>> Esteban
>>
>> El 29/02/2012, a las 11:21p.m., Schwab,Wilhelm K escribió:
>>
>>> What does "cannot find callback signature" mean from #signature:block:?  I'm stuck.
>>>
>>>
>>>
>>>
>>> ________________________________________
>>> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
>>> Sent: Wednesday, February 29, 2012 6:54 PM
>>> To: [hidden email]
>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>>
>>> yeah... sorry about that
>>> is a bug on FFI... I managed to worked around it for HPDF, for a customer's project... but of course is not a good and definitive solution.
>>>
>>> best,
>>> Esteban
>>>
>>> El 29/02/2012, a las 8:37p.m., Schwab,Wilhelm K escribió:
>>>
>>>> This is gonna take a while...   I had structs flying around as void* and was moderately happy.  Nonetheless, your suggestion worked, provided I add a lot getHandle asInteger and change the void* to long :(
>>>>
>>>>
>>>>
>>>>
>>>> ________________________________________
>>>> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
>>>> Sent: Wednesday, February 29, 2012 6:23 PM
>>>> To: [hidden email]
>>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>>>
>>>> I also found some problems using void* in linux... maybe you want to use long (which has same size)... I "fixed" my problems that way.
>>>>
>>>> yes... maybe we need to look at FFI to see why void* has problems some times, but well, that can help you atm (sorry for not having a better answer)
>>>>
>>>> Esteban
>>>>
>>>> El 29/02/2012, a las 8:11p.m., Igor Stasenko escribió:
>>>>
>>>>> On 1 March 2012 00:58, Schwab,Wilhelm K <[hidden email]> wrote:
>>>>>> Another glitch: is there any problem passing things as void*?  I'm getting failure to coerce errors that did not arise before.
>>>>>>
>>>>> No idea. As you may suspect, i stopped using FFI/Alien once i got
>>>>> NativeBoost toy to play with.
>>>>>
>>>>> Please file the issue, describing the problem. so we can look over it
>>>>> and fix it.
>>>>>
>>>>>>
>>>>>>
>>>>>> ________________________________________
>>>>>> From: [hidden email] [[hidden email]] on behalf of Igor Stasenko [[hidden email]]
>>>>>> Sent: Wednesday, February 29, 2012 5:51 PM
>>>>>> To: [hidden email]
>>>>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>>>>>
>>>>>> On 1 March 2012 00:37, Schwab,Wilhelm K <[hidden email]> wrote:
>>>>>>> Does 1.3 by default not create field accessors?  Why is that?  I thought
>>>>>>> nothing was happening.
>>>>>>>
>>>>>>
>>>>>> Good question, i'd like to know the answer too.
>>>>>> It is related to MC final 'installation' phase,
>>>>>> where it initializing all classes.
>>>>>>
>>>>>> In NativeBoost i was also using
>>>>>>
>>>>>> noteCompilationOf: aSelector meta: isMeta
>>>>>>    "A hook allowing some classes to react to recompilation of certain selectors"
>>>>>>
>>>>>> but there was a big question, at which point this hook is triggered,
>>>>>> and looks like some changes in MC stop triggering it/triggering at
>>>>>> wrong time (not all methods get into a class/ class not initialized
>>>>>> etc),
>>>>>> which makes it not very useful.
>>>>>>
>>>>>> So i ended up creating a DNU handler on instance side, then on DNU i
>>>>>> check if i have field accessors and if not,
>>>>>> compile them on the fly.
>>>>>>
>>>>>>
>>>>>>> Bill
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best regards,
>>>>>> Igor Stasenko.
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Igor Stasenko.
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>>
>>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Alien signature: first attempt

Eliot Miranda-2
In reply to this post by Schwab,Wilhelm K


On Thu, Mar 1, 2012 at 10:57 AM, Schwab,Wilhelm K <[hidden email]> wrote:
Esteban,

Will this work?  Even if so, is there a better way to define it?

No it won't work, because you don't understand what it does.
 

longVoidStarLongRetInt: callbackContext sp: spAlien
       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
       ^callbackContext wordResult:
               (block
                       value: (Alien newC:4)
                       value: (Alien forPointer: (spAlien unsignedLongAt: 5))
                       value: (Alien newC:4)
               )

The above takes only one parameter's value from the spAlien (the void * value) and passes garbage for the other parameters. The spAlien is a pointer to the incoming C callback stack frame, i.e. a pointer to the callback's parameters.  So spAlien unsignedLongAt: 5 gets the four-byte pointer for the void * parameter, wraps it up in a convenient pointer Alien (so the callback block can e.g. indirect through it to get at the data the pointer is pointing to), and passes it as the second argument of the callback block.  But the two Alien newC: 4's are entirely bogus, passing effectively uninitialized data for the first and third parameters of the callback block.  They're just empty Aliens, when in fact you want to pass two longs to the block for the first and third parameters.

So to pass values in you need something like:

longVoidStarLongRetInt: callbackContext sp: spAlien
       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
       ^callbackContext wordResult:
               (block
                       value: (
spAlien signedLongAt: 1)
                       value: (Alien forPointer: (spAlien unsignedLongAt: 5))
                       value: (
spAlien signedLongAt: 9)
               )

You might also need to consider passing the result of the block back out.  If you only want to use it to answer integers then the above is fine.  But take a look at the intcharstarRetint:sp: signature:

intcharstarRetint: callbackContext sp: spAlien
<signature: 'int (*)(int, char *)' abi: 'IA32'>
^callbackContext wordResult:
((block
value: (spAlien signedLongAt: 1) ~= 0
value: (Alien forPointer: (spAlien unsignedLongAt: 5)) strcpyUTF8)
ifNil: [0]
ifNotNil:
[:result|
result isInteger
ifTrue: [result]
ifFalse: [result == true ifTrue: [1] ifFalse: [0]]])

Analogously it passes a signed integer as the first argument and a UTF8 string derived from the second char * parameter. (The method should probably be called intcharstarasUTF8Retint:sp: or some such; forgive me, we're getting there).

But on return it takes the result of the callback block and passes it to the ifNil:ifNotNil: which
- maps a nil result form the block to 0 (ifNil: [0])
- maps integers to integers (which will only deal correctly with integers in the 32-bit range), and
- maps true to 1, mapping all other results to 0.

Does what's going on now make more sense?

HTH,
Eliot


Bill

________________________________________
From: [hidden email] [[hidden email]] on behalf of Schwab,Wilhelm K [[hidden email]]
Sent: Thursday, March 01, 2012 12:15 AM
To: [hidden email]
Subject: Re: [Pharo-project] FFI on 1.3: compile fields

thanks!!!!!




________________________________________
From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
Sent: Wednesday, February 29, 2012 10:18 PM
To: [hidden email]
Subject: Re: [Pharo-project] FFI on 1.3: compile fields

you need to define a signature... not at home now, but wait 'til tomorrow and I'll send an example

best,
Esteban

El 29/02/2012, a las 11:21p.m., Schwab,Wilhelm K escribió:

> What does "cannot find callback signature" mean from #signature:block:?  I'm stuck.
>
>
>
>
> ________________________________________
> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
> Sent: Wednesday, February 29, 2012 6:54 PM
> To: [hidden email]
> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>
> yeah... sorry about that
> is a bug on FFI... I managed to worked around it for HPDF, for a customer's project... but of course is not a good and definitive solution.
>
> best,
> Esteban
>
> El 29/02/2012, a las 8:37p.m., Schwab,Wilhelm K escribió:
>
>> This is gonna take a while...   I had structs flying around as void* and was moderately happy.  Nonetheless, your suggestion worked, provided I add a lot getHandle asInteger and change the void* to long :(
>>
>>
>>
>>
>> ________________________________________
>> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
>> Sent: Wednesday, February 29, 2012 6:23 PM
>> To: [hidden email]
>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>
>> I also found some problems using void* in linux... maybe you want to use long (which has same size)... I "fixed" my problems that way.
>>
>> yes... maybe we need to look at FFI to see why void* has problems some times, but well, that can help you atm (sorry for not having a better answer)
>>
>> Esteban
>>
>> El 29/02/2012, a las 8:11p.m., Igor Stasenko escribió:
>>
>>> On 1 March 2012 00:58, Schwab,Wilhelm K <[hidden email]> wrote:
>>>> Another glitch: is there any problem passing things as void*?  I'm getting failure to coerce errors that did not arise before.
>>>>
>>> No idea. As you may suspect, i stopped using FFI/Alien once i got
>>> NativeBoost toy to play with.
>>>
>>> Please file the issue, describing the problem. so we can look over it
>>> and fix it.
>>>
>>>>
>>>>
>>>> ________________________________________
>>>> From: [hidden email] [[hidden email]] on behalf of Igor Stasenko [[hidden email]]
>>>> Sent: Wednesday, February 29, 2012 5:51 PM
>>>> To: [hidden email]
>>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>>>
>>>> On 1 March 2012 00:37, Schwab,Wilhelm K <[hidden email]> wrote:
>>>>> Does 1.3 by default not create field accessors?  Why is that?  I thought
>>>>> nothing was happening.
>>>>>
>>>>
>>>> Good question, i'd like to know the answer too.
>>>> It is related to MC final 'installation' phase,
>>>> where it initializing all classes.
>>>>
>>>> In NativeBoost i was also using
>>>>
>>>> noteCompilationOf: aSelector meta: isMeta
>>>>      "A hook allowing some classes to react to recompilation of certain selectors"
>>>>
>>>> but there was a big question, at which point this hook is triggered,
>>>> and looks like some changes in MC stop triggering it/triggering at
>>>> wrong time (not all methods get into a class/ class not initialized
>>>> etc),
>>>> which makes it not very useful.
>>>>
>>>> So i ended up creating a DNU handler on instance side, then on DNU i
>>>> check if i have field accessors and if not,
>>>> compile them on the fly.
>>>>
>>>>
>>>>> Bill
>>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Igor Stasenko.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko.
>>>
>>
>>
>>
>
>
>







--
best,
Eliot

Reply | Threaded
Open this post in threaded view
|

Re: Alien signature: first attempt

Eliot Miranda-2
In reply to this post by EstebanLM


On Thu, Mar 1, 2012 at 11:08 AM, Esteban Lorenzano <[hidden email]> wrote:
Hi,

sorry for not sending before: work is hard :P
yes, that should work, and no, I don't know a better way to define it.

No it won't work.  See my message.
 
I think Eliot's idea is to form "libraries" of generic callbacks (for mars, for instance, I have CocoaCallback, child of Callback, who defines all my needed signatures, looks a lot at first instance, but if you see in detail, finally there are not so much variants)

Yes, the idea is to have a library of signature methods that marshal specific C callback signatures to/from Smalltalk blocks.  The pragma specifies what ABI (application binary interface, or stack layout) the signature is for so that the scheme is cross-platform.  But I expect that these signature methods could be the output of an ABI compiler, so that one wouldn't have to write them by hand.  The pragma doesn't need to include a word-size since IA64 is a different ABI to IA32, and so the ABI subsumes word size.

HTH
 

best,
Esteban

El 01/03/2012, a las 3:57p.m., Schwab,Wilhelm K escribió:

> Esteban,
>
> Will this work?  Even if so, is there a better way to define it?
>
> longVoidStarLongRetInt: callbackContext sp: spAlien
>       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
>       ^callbackContext wordResult:
>               (block
>                       value: (Alien newC:4)
>                       value: (Alien forPointer: (spAlien unsignedLongAt: 5))
>                       value: (Alien newC:4)
>               )
>
> Bill
>
> ________________________________________
> From: [hidden email] [[hidden email]] on behalf of Schwab,Wilhelm K [[hidden email]]
> Sent: Thursday, March 01, 2012 12:15 AM
> To: [hidden email]
> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>
> thanks!!!!!
>
>
>
>
> ________________________________________
> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
> Sent: Wednesday, February 29, 2012 10:18 PM
> To: [hidden email]
> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>
> you need to define a signature... not at home now, but wait 'til tomorrow and I'll send an example
>
> best,
> Esteban
>
> El 29/02/2012, a las 11:21p.m., Schwab,Wilhelm K escribió:
>
>> What does "cannot find callback signature" mean from #signature:block:?  I'm stuck.
>>
>>
>>
>>
>> ________________________________________
>> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
>> Sent: Wednesday, February 29, 2012 6:54 PM
>> To: [hidden email]
>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>
>> yeah... sorry about that
>> is a bug on FFI... I managed to worked around it for HPDF, for a customer's project... but of course is not a good and definitive solution.
>>
>> best,
>> Esteban
>>
>> El 29/02/2012, a las 8:37p.m., Schwab,Wilhelm K escribió:
>>
>>> This is gonna take a while...   I had structs flying around as void* and was moderately happy.  Nonetheless, your suggestion worked, provided I add a lot getHandle asInteger and change the void* to long :(
>>>
>>>
>>>
>>>
>>> ________________________________________
>>> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
>>> Sent: Wednesday, February 29, 2012 6:23 PM
>>> To: [hidden email]
>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>>
>>> I also found some problems using void* in linux... maybe you want to use long (which has same size)... I "fixed" my problems that way.
>>>
>>> yes... maybe we need to look at FFI to see why void* has problems some times, but well, that can help you atm (sorry for not having a better answer)
>>>
>>> Esteban
>>>
>>> El 29/02/2012, a las 8:11p.m., Igor Stasenko escribió:
>>>
>>>> On 1 March 2012 00:58, Schwab,Wilhelm K <[hidden email]> wrote:
>>>>> Another glitch: is there any problem passing things as void*?  I'm getting failure to coerce errors that did not arise before.
>>>>>
>>>> No idea. As you may suspect, i stopped using FFI/Alien once i got
>>>> NativeBoost toy to play with.
>>>>
>>>> Please file the issue, describing the problem. so we can look over it
>>>> and fix it.
>>>>
>>>>>
>>>>>
>>>>> ________________________________________
>>>>> From: [hidden email] [[hidden email]] on behalf of Igor Stasenko [[hidden email]]
>>>>> Sent: Wednesday, February 29, 2012 5:51 PM
>>>>> To: [hidden email]
>>>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>>>>
>>>>> On 1 March 2012 00:37, Schwab,Wilhelm K <[hidden email]> wrote:
>>>>>> Does 1.3 by default not create field accessors?  Why is that?  I thought
>>>>>> nothing was happening.
>>>>>>
>>>>>
>>>>> Good question, i'd like to know the answer too.
>>>>> It is related to MC final 'installation' phase,
>>>>> where it initializing all classes.
>>>>>
>>>>> In NativeBoost i was also using
>>>>>
>>>>> noteCompilationOf: aSelector meta: isMeta
>>>>>     "A hook allowing some classes to react to recompilation of certain selectors"
>>>>>
>>>>> but there was a big question, at which point this hook is triggered,
>>>>> and looks like some changes in MC stop triggering it/triggering at
>>>>> wrong time (not all methods get into a class/ class not initialized
>>>>> etc),
>>>>> which makes it not very useful.
>>>>>
>>>>> So i ended up creating a DNU handler on instance side, then on DNU i
>>>>> check if i have field accessors and if not,
>>>>> compile them on the fly.
>>>>>
>>>>>
>>>>>> Bill
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Igor Stasenko.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Igor Stasenko.
>>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>
>





--
best,
Eliot

Reply | Threaded
Open this post in threaded view
|

Re: Alien signature: first attempt

EstebanLM
yeah, I see... I was thinking on method structure (pragma signature, etc.), not the contents evaluated inside... but of course, you are right :)

best,
Esteban

El 01/03/2012, a las 5:30p.m., Eliot Miranda escribió:



On Thu, Mar 1, 2012 at 11:08 AM, Esteban Lorenzano <[hidden email]> wrote:
Hi,

sorry for not sending before: work is hard :P
yes, that should work, and no, I don't know a better way to define it.

No it won't work.  See my message.
 
I think Eliot's idea is to form "libraries" of generic callbacks (for mars, for instance, I have CocoaCallback, child of Callback, who defines all my needed signatures, looks a lot at first instance, but if you see in detail, finally there are not so much variants)

Yes, the idea is to have a library of signature methods that marshal specific C callback signatures to/from Smalltalk blocks.  The pragma specifies what ABI (application binary interface, or stack layout) the signature is for so that the scheme is cross-platform.  But I expect that these signature methods could be the output of an ABI compiler, so that one wouldn't have to write them by hand.  The pragma doesn't need to include a word-size since IA64 is a different ABI to IA32, and so the ABI subsumes word size.

HTH
 

best,
Esteban

El 01/03/2012, a las 3:57p.m., Schwab,Wilhelm K escribió:

> Esteban,
>
> Will this work?  Even if so, is there a better way to define it?
>
> longVoidStarLongRetInt: callbackContext sp: spAlien
>       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
>       ^callbackContext wordResult:
>               (block
>                       value: (Alien newC:4)
>                       value: (Alien forPointer: (spAlien unsignedLongAt: 5))
>                       value: (Alien newC:4)
>               )
>
> Bill
>
> ________________________________________
> From: [hidden email] [[hidden email]] on behalf of Schwab,Wilhelm K [[hidden email]]
> Sent: Thursday, March 01, 2012 12:15 AM
> To: [hidden email]
> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>
> thanks!!!!!
>
>
>
>
> ________________________________________
> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
> Sent: Wednesday, February 29, 2012 10:18 PM
> To: [hidden email]
> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>
> you need to define a signature... not at home now, but wait 'til tomorrow and I'll send an example
>
> best,
> Esteban
>
> El 29/02/2012, a las 11:21p.m., Schwab,Wilhelm K escribió:
>
>> What does "cannot find callback signature" mean from #signature:block:?  I'm stuck.
>>
>>
>>
>>
>> ________________________________________
>> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
>> Sent: Wednesday, February 29, 2012 6:54 PM
>> To: [hidden email]
>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>
>> yeah... sorry about that
>> is a bug on FFI... I managed to worked around it for HPDF, for a customer's project... but of course is not a good and definitive solution.
>>
>> best,
>> Esteban
>>
>> El 29/02/2012, a las 8:37p.m., Schwab,Wilhelm K escribió:
>>
>>> This is gonna take a while...   I had structs flying around as void* and was moderately happy.  Nonetheless, your suggestion worked, provided I add a lot getHandle asInteger and change the void* to long :(
>>>
>>>
>>>
>>>
>>> ________________________________________
>>> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
>>> Sent: Wednesday, February 29, 2012 6:23 PM
>>> To: [hidden email]
>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>>
>>> I also found some problems using void* in linux... maybe you want to use long (which has same size)... I "fixed" my problems that way.
>>>
>>> yes... maybe we need to look at FFI to see why void* has problems some times, but well, that can help you atm (sorry for not having a better answer)
>>>
>>> Esteban
>>>
>>> El 29/02/2012, a las 8:11p.m., Igor Stasenko escribió:
>>>
>>>> On 1 March 2012 00:58, Schwab,Wilhelm K <[hidden email]> wrote:
>>>>> Another glitch: is there any problem passing things as void*?  I'm getting failure to coerce errors that did not arise before.
>>>>>
>>>> No idea. As you may suspect, i stopped using FFI/Alien once i got
>>>> NativeBoost toy to play with.
>>>>
>>>> Please file the issue, describing the problem. so we can look over it
>>>> and fix it.
>>>>
>>>>>
>>>>>
>>>>> ________________________________________
>>>>> From: [hidden email] [[hidden email]] on behalf of Igor Stasenko [[hidden email]]
>>>>> Sent: Wednesday, February 29, 2012 5:51 PM
>>>>> To: [hidden email]
>>>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>>>>
>>>>> On 1 March 2012 00:37, Schwab,Wilhelm K <[hidden email]> wrote:
>>>>>> Does 1.3 by default not create field accessors?  Why is that?  I thought
>>>>>> nothing was happening.
>>>>>>
>>>>>
>>>>> Good question, i'd like to know the answer too.
>>>>> It is related to MC final 'installation' phase,
>>>>> where it initializing all classes.
>>>>>
>>>>> In NativeBoost i was also using
>>>>>
>>>>> noteCompilationOf: aSelector meta: isMeta
>>>>>     "A hook allowing some classes to react to recompilation of certain selectors"
>>>>>
>>>>> but there was a big question, at which point this hook is triggered,
>>>>> and looks like some changes in MC stop triggering it/triggering at
>>>>> wrong time (not all methods get into a class/ class not initialized
>>>>> etc),
>>>>> which makes it not very useful.
>>>>>
>>>>> So i ended up creating a DNU handler on instance side, then on DNU i
>>>>> check if i have field accessors and if not,
>>>>> compile them on the fly.
>>>>>
>>>>>
>>>>>> Bill
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Igor Stasenko.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Igor Stasenko.
>>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>
>





--
best,
Eliot


Reply | Threaded
Open this post in threaded view
|

Re: Alien signature: first attempt

Nicolas Cellier
In reply to this post by Eliot Miranda-2
Le 1 mars 2012 21:30, Eliot Miranda <[hidden email]> a écrit :

>
>
> On Thu, Mar 1, 2012 at 11:08 AM, Esteban Lorenzano <[hidden email]>
> wrote:
>>
>> Hi,
>>
>> sorry for not sending before: work is hard :P
>> yes, that should work, and no, I don't know a better way to define it.
>
>
> No it won't work.  See my message.
>
>>
>> I think Eliot's idea is to form "libraries" of generic callbacks (for
>> mars, for instance, I have CocoaCallback, child of Callback, who defines all
>> my needed signatures, looks a lot at first instance, but if you see in
>> detail, finally there are not so much variants)
>
>
> Yes, the idea is to have a library of signature methods that marshal
> specific C callback signatures to/from Smalltalk blocks.  The pragma
> specifies what ABI (application binary interface, or stack layout) the
> signature is for so that the scheme is cross-platform.  But I expect that
> these signature methods could be the output of an ABI compiler, so that one
> wouldn't have to write them by hand.  The pragma doesn't need to include a
> word-size since IA64 is a different ABI to IA32, and so the ABI subsumes
> word size.
>
> HTH
>


Let me understand...
If I have a signature (char x,double y, short z) IA-32, where is the
spalien pointing to?
- the original call stack (with proper ABI alignment, that is offsets
0,4,12, or 1-based indices 1,5,13)
- a serialized copy without alignment padding (offsets 0,1,9, or
1-based indices 1,2,10)

In the former case, that means that the indices must be computed
according to ABI (might be different in PPC for example).

And even if such code is generated with an ABI-aware interface
compiler, is it really platform independent?
I'm quite sure the ABI have subtle variations, which are not even
platform but compiler specific...
For example of such niceties in win32, did you ever try returning a
structure by value from a cdecl mingw-gcc compiled function to a cdecl
visual studio compiled caller, not even speaking of different internal
structure alignment padding (controlled with #pragma packed(4) or
other compiler options...)

This apple page helps a lot, for example it tells how gcc returns a
structure by value...
https://developer.apple.com/library/mac/#documentation/developertools/Conceptual/LowLevelABI/130-IA-32_Function_Calling_Conventions/IA32.html

Nicolas

>>
>>
>> best,
>> Esteban
>>
>> El 01/03/2012, a las 3:57p.m., Schwab,Wilhelm K escribió:
>>
>> > Esteban,
>> >
>> > Will this work?  Even if so, is there a better way to define it?
>> >
>> > longVoidStarLongRetInt: callbackContext sp: spAlien
>> >       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
>> >       ^callbackContext wordResult:
>> >               (block
>> >                       value: (Alien newC:4)
>> >                       value: (Alien forPointer: (spAlien unsignedLongAt:
>> > 5))
>> >                       value: (Alien newC:4)
>> >               )
>> >
>> > Bill
>> >
>> > ________________________________________
>> > From: [hidden email]
>> > [[hidden email]] on behalf of Schwab,Wilhelm K
>> > [[hidden email]]
>> > Sent: Thursday, March 01, 2012 12:15 AM
>> > To: [hidden email]
>> > Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>> >
>> > thanks!!!!!
>> >
>> >
>> >
>> >
>> > ________________________________________
>> > From: [hidden email]
>> > [[hidden email]] on behalf of Esteban Lorenzano
>> > [[hidden email]]
>> > Sent: Wednesday, February 29, 2012 10:18 PM
>> > To: [hidden email]
>> > Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>> >
>> > you need to define a signature... not at home now, but wait 'til
>> > tomorrow and I'll send an example
>> >
>> > best,
>> > Esteban
>> >
>> > El 29/02/2012, a las 11:21p.m., Schwab,Wilhelm K escribió:
>> >
>> >> What does "cannot find callback signature" mean from #signature:block:?
>> >>  I'm stuck.
>> >>
>> >>
>> >>
>> >>
>> >> ________________________________________
>> >> From: [hidden email]
>> >> [[hidden email]] on behalf of Esteban Lorenzano
>> >> [[hidden email]]
>> >> Sent: Wednesday, February 29, 2012 6:54 PM
>> >> To: [hidden email]
>> >> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>> >>
>> >> yeah... sorry about that
>> >> is a bug on FFI... I managed to worked around it for HPDF, for a
>> >> customer's project... but of course is not a good and definitive solution.
>> >>
>> >> best,
>> >> Esteban
>> >>
>> >> El 29/02/2012, a las 8:37p.m., Schwab,Wilhelm K escribió:
>> >>
>> >>> This is gonna take a while...   I had structs flying around as void*
>> >>> and was moderately happy.  Nonetheless, your suggestion worked, provided I
>> >>> add a lot getHandle asInteger and change the void* to long :(
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> ________________________________________
>> >>> From: [hidden email]
>> >>> [[hidden email]] on behalf of Esteban Lorenzano
>> >>> [[hidden email]]
>> >>> Sent: Wednesday, February 29, 2012 6:23 PM
>> >>> To: [hidden email]
>> >>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>> >>>
>> >>> I also found some problems using void* in linux... maybe you want to
>> >>> use long (which has same size)... I "fixed" my problems that way.
>> >>>
>> >>> yes... maybe we need to look at FFI to see why void* has problems some
>> >>> times, but well, that can help you atm (sorry for not having a better
>> >>> answer)
>> >>>
>> >>> Esteban
>> >>>
>> >>> El 29/02/2012, a las 8:11p.m., Igor Stasenko escribió:
>> >>>
>> >>>> On 1 March 2012 00:58, Schwab,Wilhelm K <[hidden email]>
>> >>>> wrote:
>> >>>>> Another glitch: is there any problem passing things as void*?  I'm
>> >>>>> getting failure to coerce errors that did not arise before.
>> >>>>>
>> >>>> No idea. As you may suspect, i stopped using FFI/Alien once i got
>> >>>> NativeBoost toy to play with.
>> >>>>
>> >>>> Please file the issue, describing the problem. so we can look over it
>> >>>> and fix it.
>> >>>>
>> >>>>>
>> >>>>>
>> >>>>> ________________________________________
>> >>>>> From: [hidden email]
>> >>>>> [[hidden email]] on behalf of Igor Stasenko
>> >>>>> [[hidden email]]
>> >>>>> Sent: Wednesday, February 29, 2012 5:51 PM
>> >>>>> To: [hidden email]
>> >>>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>> >>>>>
>> >>>>> On 1 March 2012 00:37, Schwab,Wilhelm K <[hidden email]>
>> >>>>> wrote:
>> >>>>>> Does 1.3 by default not create field accessors?  Why is that?  I
>> >>>>>> thought
>> >>>>>> nothing was happening.
>> >>>>>>
>> >>>>>
>> >>>>> Good question, i'd like to know the answer too.
>> >>>>> It is related to MC final 'installation' phase,
>> >>>>> where it initializing all classes.
>> >>>>>
>> >>>>> In NativeBoost i was also using
>> >>>>>
>> >>>>> noteCompilationOf: aSelector meta: isMeta
>> >>>>>     "A hook allowing some classes to react to recompilation of
>> >>>>> certain selectors"
>> >>>>>
>> >>>>> but there was a big question, at which point this hook is triggered,
>> >>>>> and looks like some changes in MC stop triggering it/triggering at
>> >>>>> wrong time (not all methods get into a class/ class not initialized
>> >>>>> etc),
>> >>>>> which makes it not very useful.
>> >>>>>
>> >>>>> So i ended up creating a DNU handler on instance side, then on DNU i
>> >>>>> check if i have field accessors and if not,
>> >>>>> compile them on the fly.
>> >>>>>
>> >>>>>
>> >>>>>> Bill
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Best regards,
>> >>>>> Igor Stasenko.
>> >>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Best regards,
>> >>>> Igor Stasenko.
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>
>> >>
>> >>
>> >
>> >
>> >
>> >
>>
>>
>
>
>
> --
> best,
> Eliot
>

Reply | Threaded
Open this post in threaded view
|

Re: Alien signature: first attempt

Schwab,Wilhelm K
In reply to this post by Eliot Miranda-2
Eliot,

That helps a bit; it's still unfamiliar territory.  I am a little bothered by #wordResult: and #floatResult.  There are times when one is expected to return 16 bit values (rare) and either a float or a double (common).  Does that flexibility exist?

Bill





From: [hidden email] [[hidden email]] on behalf of Eliot Miranda [[hidden email]]
Sent: Thursday, March 01, 2012 3:26 PM
To: [hidden email]
Subject: Re: [Pharo-project] Alien signature: first attempt



On Thu, Mar 1, 2012 at 10:57 AM, Schwab,Wilhelm K <[hidden email]> wrote:
Esteban,

Will this work?  Even if so, is there a better way to define it?

No it won't work, because you don't understand what it does.
 

longVoidStarLongRetInt: callbackContext sp: spAlien
       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
       ^callbackContext wordResult:
               (block
                       value: (Alien newC:4)
                       value: (Alien forPointer: (spAlien unsignedLongAt: 5))
                       value: (Alien newC:4)
               )

The above takes only one parameter's value from the spAlien (the void * value) and passes garbage for the other parameters. The spAlien is a pointer to the incoming C callback stack frame, i.e. a pointer to the callback's parameters.  So spAlien unsignedLongAt: 5 gets the four-byte pointer for the void * parameter, wraps it up in a convenient pointer Alien (so the callback block can e.g. indirect through it to get at the data the pointer is pointing to), and passes it as the second argument of the callback block.  But the two Alien newC: 4's are entirely bogus, passing effectively uninitialized data for the first and third parameters of the callback block.  They're just empty Aliens, when in fact you want to pass two longs to the block for the first and third parameters.

So to pass values in you need something like:

longVoidStarLongRetInt: callbackContext sp: spAlien
       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
       ^callbackContext wordResult:
               (block
                       value: (
spAlien signedLongAt: 1)
                       value: (Alien forPointer: (spAlien unsignedLongAt: 5))
                       value: (
spAlien signedLongAt: 9)
               )

You might also need to consider passing the result of the block back out.  If you only want to use it to answer integers then the above is fine.  But take a look at the intcharstarRetint:sp: signature:

intcharstarRetint: callbackContext sp: spAlien
<signature: 'int (*)(int, char *)' abi: 'IA32'>
^callbackContext wordResult:
((block
value: (spAlien signedLongAt: 1) ~= 0
value: (Alien forPointer: (spAlien unsignedLongAt: 5)) strcpyUTF8)
ifNil: [0]
ifNotNil:
[:result|
result isInteger
ifTrue: [result]
ifFalse: [result == true ifTrue: [1] ifFalse: [0]]])

Analogously it passes a signed integer as the first argument and a UTF8 string derived from the second char * parameter. (The method should probably be called intcharstarasUTF8Retint:sp: or some such; forgive me, we're getting there).

But on return it takes the result of the callback block and passes it to the ifNil:ifNotNil: which
- maps a nil result form the block to 0 (ifNil: [0])
- maps integers to integers (which will only deal correctly with integers in the 32-bit range), and
- maps true to 1, mapping all other results to 0.

Does what's going on now make more sense?

HTH,
Eliot


Bill

________________________________________
From: [hidden email] [[hidden email]] on behalf of Schwab,Wilhelm K [[hidden email]]
Sent: Thursday, March 01, 2012 12:15 AM
To: [hidden email]
Subject: Re: [Pharo-project] FFI on 1.3: compile fields

thanks!!!!!




________________________________________
From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
Sent: Wednesday, February 29, 2012 10:18 PM
To: [hidden email]
Subject: Re: [Pharo-project] FFI on 1.3: compile fields

you need to define a signature... not at home now, but wait 'til tomorrow and I'll send an example

best,
Esteban

El 29/02/2012, a las 11:21p.m., Schwab,Wilhelm K escribió:

> What does "cannot find callback signature" mean from #signature:block:?  I'm stuck.
>
>
>
>
> ________________________________________
> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
> Sent: Wednesday, February 29, 2012 6:54 PM
> To: [hidden email]
> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>
> yeah... sorry about that
> is a bug on FFI... I managed to worked around it for HPDF, for a customer's project... but of course is not a good and definitive solution.
>
> best,
> Esteban
>
> El 29/02/2012, a las 8:37p.m., Schwab,Wilhelm K escribió:
>
>> This is gonna take a while...   I had structs flying around as void* and was moderately happy.  Nonetheless, your suggestion worked, provided I add a lot getHandle asInteger and change the void* to long :(
>>
>>
>>
>>
>> ________________________________________
>> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
>> Sent: Wednesday, February 29, 2012 6:23 PM
>> To: [hidden email]
>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>
>> I also found some problems using void* in linux... maybe you want to use long (which has same size)... I "fixed" my problems that way.
>>
>> yes... maybe we need to look at FFI to see why void* has problems some times, but well, that can help you atm (sorry for not having a better answer)
>>
>> Esteban
>>
>> El 29/02/2012, a las 8:11p.m., Igor Stasenko escribió:
>>
>>> On 1 March 2012 00:58, Schwab,Wilhelm K <[hidden email]> wrote:
>>>> Another glitch: is there any problem passing things as void*?  I'm getting failure to coerce errors that did not arise before.
>>>>
>>> No idea. As you may suspect, i stopped using FFI/Alien once i got
>>> NativeBoost toy to play with.
>>>
>>> Please file the issue, describing the problem. so we can look over it
>>> and fix it.
>>>
>>>>
>>>>
>>>> ________________________________________
>>>> From: [hidden email] [[hidden email]] on behalf of Igor Stasenko [[hidden email]]
>>>> Sent: Wednesday, February 29, 2012 5:51 PM
>>>> To: [hidden email]
>>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>>>
>>>> On 1 March 2012 00:37, Schwab,Wilhelm K <[hidden email]> wrote:
>>>>> Does 1.3 by default not create field accessors?  Why is that?  I thought
>>>>> nothing was happening.
>>>>>
>>>>
>>>> Good question, i'd like to know the answer too.
>>>> It is related to MC final 'installation' phase,
>>>> where it initializing all classes.
>>>>
>>>> In NativeBoost i was also using
>>>>
>>>> noteCompilationOf: aSelector meta: isMeta
>>>>      "A hook allowing some classes to react to recompilation of certain selectors"
>>>>
>>>> but there was a big question, at which point this hook is triggered,
>>>> and looks like some changes in MC stop triggering it/triggering at
>>>> wrong time (not all methods get into a class/ class not initialized
>>>> etc),
>>>> which makes it not very useful.
>>>>
>>>> So i ended up creating a DNU handler on instance side, then on DNU i
>>>> check if i have field accessors and if not,
>>>> compile them on the fly.
>>>>
>>>>
>>>>> Bill
>>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Igor Stasenko.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko.
>>>
>>
>>
>>
>
>
>







--
best,
Eliot

Reply | Threaded
Open this post in threaded view
|

Re: Alien signature: first attempt

Eliot Miranda-2
In reply to this post by Nicolas Cellier


On Thu, Mar 1, 2012 at 1:15 PM, Nicolas Cellier <[hidden email]> wrote:
Le 1 mars 2012 21:30, Eliot Miranda <[hidden email]> a écrit :
>
>
> On Thu, Mar 1, 2012 at 11:08 AM, Esteban Lorenzano <[hidden email]>
> wrote:
>>
>> Hi,
>>
>> sorry for not sending before: work is hard :P
>> yes, that should work, and no, I don't know a better way to define it.
>
>
> No it won't work.  See my message.
>
>>
>> I think Eliot's idea is to form "libraries" of generic callbacks (for
>> mars, for instance, I have CocoaCallback, child of Callback, who defines all
>> my needed signatures, looks a lot at first instance, but if you see in
>> detail, finally there are not so much variants)
>
>
> Yes, the idea is to have a library of signature methods that marshal
> specific C callback signatures to/from Smalltalk blocks.  The pragma
> specifies what ABI (application binary interface, or stack layout) the
> signature is for so that the scheme is cross-platform.  But I expect that
> these signature methods could be the output of an ABI compiler, so that one
> wouldn't have to write them by hand.  The pragma doesn't need to include a
> word-size since IA64 is a different ABI to IA32, and so the ABI subsumes
> word size.
>
> HTH
>


Let me understand...
If I have a signature (char x,double y, short z) IA-32, where is the
spalien pointing to?

At the first parameter immediately below the return address.
 
- the original call stack (with proper ABI alignment, that is offsets
0,4,12, or 1-based indices 1,5,13)

Right.
 
- a serialized copy without alignment padding (offsets 0,1,9, or
1-based indices 1,2,10)

No.
 
In the former case, that means that the indices must be computed
according to ABI (might be different in PPC for example).

Right.  Hence the pragma containing the name of the ABI to allow the system to select the appropriate version.  Note that the implementation allows for passing a pointer to a struct containing marshalled register arguments for RISC platforms, although this isn't used yet since it's not needed on x86.


And even if such code is generated with an ABI-aware interface
compiler, is it really platform independent?

The scheme is platform-independent, allowing platform-specific versions of marshalling code to exist side-by-side.
 
I'm quite sure the ABI have subtle variations, which are not even
platform but compiler specific...

No.  That's the definition of an ABI.  The ABI specifies the calling convention for a particular platform, and compilers must conform to the ABI for all external and system calls.
 
For example of such niceties in win32, did you ever try returning a
structure by value from a cdecl mingw-gcc compiled function to a cdecl
visual studio compiled caller, not even speaking of different internal
structure alignment padding (controlled with #pragma packed(4) or
other compiler options...)

That's different.  Internal interfaces used by particular compilers are no covered by the ABI but then they aren't used for inter-component interfaces such as callbacks.  If you look, for example, at the Intel x86 compiler you'll see it'll use a register-based calling convention, multiple entry-points for functions (an internal and an external one), etc.  But these non-standard interfaces are only used internally, typically between functions within a single compilation unit.
 
This apple page helps a lot, for example it tells how gcc returns a
structure by value...
https://developer.apple.com/library/mac/#documentation/developertools/Conceptual/LowLevelABI/130-IA-32_Function_Calling_Conventions/IA32.html

I know.  But the bibles in these cases are documents such as "SYSTEM V APPLICATION BINARY INTERFACE, Intel386? Architecture Processor Supplement, Fourth Edition".  You'll find a concise specification of the calling convention of the ABI, and structure layout etc, in chapter 3.

cheers,
Eliot


Nicolas

>>
>>
>> best,
>> Esteban
>>
>> El 01/03/2012, a las 3:57p.m., Schwab,Wilhelm K escribió:
>>
>> > Esteban,
>> >
>> > Will this work?  Even if so, is there a better way to define it?
>> >
>> > longVoidStarLongRetInt: callbackContext sp: spAlien
>> >       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
>> >       ^callbackContext wordResult:
>> >               (block
>> >                       value: (Alien newC:4)
>> >                       value: (Alien forPointer: (spAlien unsignedLongAt:
>> > 5))
>> >                       value: (Alien newC:4)
>> >               )
>> >
>> > Bill
>> >
>> > ________________________________________
>> > From: [hidden email]
>> > [[hidden email]] on behalf of Schwab,Wilhelm K
>> > [[hidden email]]
>> > Sent: Thursday, March 01, 2012 12:15 AM
>> > To: [hidden email]
>> > Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>> >
>> > thanks!!!!!
>> >
>> >
>> >
>> >
>> > ________________________________________
>> > From: [hidden email]
>> > [[hidden email]] on behalf of Esteban Lorenzano
>> > [[hidden email]]
>> > Sent: Wednesday, February 29, 2012 10:18 PM
>> > To: [hidden email]
>> > Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>> >
>> > you need to define a signature... not at home now, but wait 'til
>> > tomorrow and I'll send an example
>> >
>> > best,
>> > Esteban
>> >
>> > El 29/02/2012, a las 11:21p.m., Schwab,Wilhelm K escribió:
>> >
>> >> What does "cannot find callback signature" mean from #signature:block:?
>> >>  I'm stuck.
>> >>
>> >>
>> >>
>> >>
>> >> ________________________________________
>> >> From: [hidden email]
>> >> [[hidden email]] on behalf of Esteban Lorenzano
>> >> [[hidden email]]
>> >> Sent: Wednesday, February 29, 2012 6:54 PM
>> >> To: [hidden email]
>> >> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>> >>
>> >> yeah... sorry about that
>> >> is a bug on FFI... I managed to worked around it for HPDF, for a
>> >> customer's project... but of course is not a good and definitive solution.
>> >>
>> >> best,
>> >> Esteban
>> >>
>> >> El 29/02/2012, a las 8:37p.m., Schwab,Wilhelm K escribió:
>> >>
>> >>> This is gonna take a while...   I had structs flying around as void*
>> >>> and was moderately happy.  Nonetheless, your suggestion worked, provided I
>> >>> add a lot getHandle asInteger and change the void* to long :(
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> ________________________________________
>> >>> From: [hidden email]
>> >>> [[hidden email]] on behalf of Esteban Lorenzano
>> >>> [[hidden email]]
>> >>> Sent: Wednesday, February 29, 2012 6:23 PM
>> >>> To: [hidden email]
>> >>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>> >>>
>> >>> I also found some problems using void* in linux... maybe you want to
>> >>> use long (which has same size)... I "fixed" my problems that way.
>> >>>
>> >>> yes... maybe we need to look at FFI to see why void* has problems some
>> >>> times, but well, that can help you atm (sorry for not having a better
>> >>> answer)
>> >>>
>> >>> Esteban
>> >>>
>> >>> El 29/02/2012, a las 8:11p.m., Igor Stasenko escribió:
>> >>>
>> >>>> On 1 March 2012 00:58, Schwab,Wilhelm K <[hidden email]>
>> >>>> wrote:
>> >>>>> Another glitch: is there any problem passing things as void*?  I'm
>> >>>>> getting failure to coerce errors that did not arise before.
>> >>>>>
>> >>>> No idea. As you may suspect, i stopped using FFI/Alien once i got
>> >>>> NativeBoost toy to play with.
>> >>>>
>> >>>> Please file the issue, describing the problem. so we can look over it
>> >>>> and fix it.
>> >>>>
>> >>>>>
>> >>>>>
>> >>>>> ________________________________________
>> >>>>> From: [hidden email]
>> >>>>> [[hidden email]] on behalf of Igor Stasenko
>> >>>>> [[hidden email]]
>> >>>>> Sent: Wednesday, February 29, 2012 5:51 PM
>> >>>>> To: [hidden email]
>> >>>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>> >>>>>
>> >>>>> On 1 March 2012 00:37, Schwab,Wilhelm K <[hidden email]>
>> >>>>> wrote:
>> >>>>>> Does 1.3 by default not create field accessors?  Why is that?  I
>> >>>>>> thought
>> >>>>>> nothing was happening.
>> >>>>>>
>> >>>>>
>> >>>>> Good question, i'd like to know the answer too.
>> >>>>> It is related to MC final 'installation' phase,
>> >>>>> where it initializing all classes.
>> >>>>>
>> >>>>> In NativeBoost i was also using
>> >>>>>
>> >>>>> noteCompilationOf: aSelector meta: isMeta
>> >>>>>     "A hook allowing some classes to react to recompilation of
>> >>>>> certain selectors"
>> >>>>>
>> >>>>> but there was a big question, at which point this hook is triggered,
>> >>>>> and looks like some changes in MC stop triggering it/triggering at
>> >>>>> wrong time (not all methods get into a class/ class not initialized
>> >>>>> etc),
>> >>>>> which makes it not very useful.
>> >>>>>
>> >>>>> So i ended up creating a DNU handler on instance side, then on DNU i
>> >>>>> check if i have field accessors and if not,
>> >>>>> compile them on the fly.
>> >>>>>
>> >>>>>
>> >>>>>> Bill
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Best regards,
>> >>>>> Igor Stasenko.
>> >>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Best regards,
>> >>>> Igor Stasenko.
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>
>> >>
>> >>
>> >
>> >
>> >
>> >
>>
>>
>
>
>
> --
> best,
> Eliot
>




--
best,
Eliot

Reply | Threaded
Open this post in threaded view
|

Re: Alien signature: first attempt

Eliot Miranda-2
In reply to this post by Schwab,Wilhelm K


On Thu, Mar 1, 2012 at 1:52 PM, Schwab,Wilhelm K <[hidden email]> wrote:
Eliot,

That helps a bit; it's still unfamiliar territory.  I am a little bothered by #wordResult: and #floatResult.  There are times when one is expected to return 16 bit values (rare) and either a float or a double (common).  Does that flexibility exist?

Yes, the flexibility exists.  But the mechanism needed to provide the flexibility is less than one might expect.  For example, on x86 there are three fundamental return conventions, one for integral values, one for floating-point values, and one for structure values.  See e.g. chapter 3 of the Sys V ABI for 386 (in my reply to Nicolas).  Read it; it's good for the soul; makes one value Smalltalk's simplicity.

Integral values are returned in %eax, %edx.  edx is not used if the value fits in 32-bits.  So the same sequence can be used to answer all integral values from 1 byte to 8 bytes, irrespective of sign.  That said, I use two sequences, one for 32-bits or less and one for 33 to 64 bits.  In your example of answering 16 bits one simply answers either the sign-extension of 16-bits (if signed) to 32-bits or the zero-extension of 16-bits to 32-bits (if unsigned).  In any case, the result is answered in %eax, and it is up to the caller to access the result correctly.

Float values are returned on the top of the floating point register stack (%st(0)).   The same machine-code sequence can be used to answer single, double and extended precision floats, since on the fp stack, they are all represented in the same extended format. 
 
So on x86 the callback implementation supports 4 return mechanisms, retword, retword64, retdouble and retstruct.  So far only two of these are fleshed out in Smalltalk code.  But look at http://www.squeakvm.org/svn/squeak/branches/Cog/src/vm/vmCallback.h &  http://squeakvm.org/svn/squeak/trunk/platforms/Cross/plugins/IA32ABI/ia32abicc.c and you'll see the other two.

cheers,
Eliot
 

Bill





From: [hidden email] [[hidden email]] on behalf of Eliot Miranda [[hidden email]]
Sent: Thursday, March 01, 2012 3:26 PM
To: [hidden email]
Subject: Re: [Pharo-project] Alien signature: first attempt



On Thu, Mar 1, 2012 at 10:57 AM, Schwab,Wilhelm K <[hidden email]> wrote:
Esteban,

Will this work?  Even if so, is there a better way to define it?

No it won't work, because you don't understand what it does.
 

longVoidStarLongRetInt: callbackContext sp: spAlien
       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
       ^callbackContext wordResult:
               (block
                       value: (Alien newC:4)
                       value: (Alien forPointer: (spAlien unsignedLongAt: 5))
                       value: (Alien newC:4)
               )

The above takes only one parameter's value from the spAlien (the void * value) and passes garbage for the other parameters. The spAlien is a pointer to the incoming C callback stack frame, i.e. a pointer to the callback's parameters.  So spAlien unsignedLongAt: 5 gets the four-byte pointer for the void * parameter, wraps it up in a convenient pointer Alien (so the callback block can e.g. indirect through it to get at the data the pointer is pointing to), and passes it as the second argument of the callback block.  But the two Alien newC: 4's are entirely bogus, passing effectively uninitialized data for the first and third parameters of the callback block.  They're just empty Aliens, when in fact you want to pass two longs to the block for the first and third parameters.

So to pass values in you need something like:

longVoidStarLongRetInt: callbackContext sp: spAlien
       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
       ^callbackContext wordResult:
               (block
                       value: (
spAlien signedLongAt: 1)
                       value: (Alien forPointer: (spAlien unsignedLongAt: 5))
                       value: (
spAlien signedLongAt: 9)
               )

You might also need to consider passing the result of the block back out.  If you only want to use it to answer integers then the above is fine.  But take a look at the intcharstarRetint:sp: signature:

intcharstarRetint: callbackContext sp: spAlien
<signature: 'int (*)(int, char *)' abi: 'IA32'>
^callbackContext wordResult:
((block
value: (spAlien signedLongAt: 1) ~= 0
value: (Alien forPointer: (spAlien unsignedLongAt: 5)) strcpyUTF8)
ifNil: [0]
ifNotNil:
[:result|
result isInteger
ifTrue: [result]
ifFalse: [result == true ifTrue: [1] ifFalse: [0]]])

Analogously it passes a signed integer as the first argument and a UTF8 string derived from the second char * parameter. (The method should probably be called intcharstarasUTF8Retint:sp: or some such; forgive me, we're getting there).

But on return it takes the result of the callback block and passes it to the ifNil:ifNotNil: which
- maps a nil result form the block to 0 (ifNil: [0])
- maps integers to integers (which will only deal correctly with integers in the 32-bit range), and
- maps true to 1, mapping all other results to 0.

Does what's going on now make more sense?

HTH,
Eliot


Bill

________________________________________
From: [hidden email] [[hidden email]] on behalf of Schwab,Wilhelm K [[hidden email]]
Sent: Thursday, March 01, 2012 12:15 AM
To: [hidden email]
Subject: Re: [Pharo-project] FFI on 1.3: compile fields

thanks!!!!!




________________________________________
From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
Sent: Wednesday, February 29, 2012 10:18 PM
To: [hidden email]
Subject: Re: [Pharo-project] FFI on 1.3: compile fields

you need to define a signature... not at home now, but wait 'til tomorrow and I'll send an example

best,
Esteban

El 29/02/2012, a las 11:21p.m., Schwab,Wilhelm K escribió:

> What does "cannot find callback signature" mean from #signature:block:?  I'm stuck.
>
>
>
>
> ________________________________________
> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
> Sent: Wednesday, February 29, 2012 6:54 PM
> To: [hidden email]
> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>
> yeah... sorry about that
> is a bug on FFI... I managed to worked around it for HPDF, for a customer's project... but of course is not a good and definitive solution.
>
> best,
> Esteban
>
> El 29/02/2012, a las 8:37p.m., Schwab,Wilhelm K escribió:
>
>> This is gonna take a while...   I had structs flying around as void* and was moderately happy.  Nonetheless, your suggestion worked, provided I add a lot getHandle asInteger and change the void* to long :(
>>
>>
>>
>>
>> ________________________________________
>> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
>> Sent: Wednesday, February 29, 2012 6:23 PM
>> To: [hidden email]
>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>
>> I also found some problems using void* in linux... maybe you want to use long (which has same size)... I "fixed" my problems that way.
>>
>> yes... maybe we need to look at FFI to see why void* has problems some times, but well, that can help you atm (sorry for not having a better answer)
>>
>> Esteban
>>
>> El 29/02/2012, a las 8:11p.m., Igor Stasenko escribió:
>>
>>> On 1 March 2012 00:58, Schwab,Wilhelm K <[hidden email]> wrote:
>>>> Another glitch: is there any problem passing things as void*?  I'm getting failure to coerce errors that did not arise before.
>>>>
>>> No idea. As you may suspect, i stopped using FFI/Alien once i got
>>> NativeBoost toy to play with.
>>>
>>> Please file the issue, describing the problem. so we can look over it
>>> and fix it.
>>>
>>>>
>>>>
>>>> ________________________________________
>>>> From: [hidden email] [[hidden email]] on behalf of Igor Stasenko [[hidden email]]
>>>> Sent: Wednesday, February 29, 2012 5:51 PM
>>>> To: [hidden email]
>>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>>>
>>>> On 1 March 2012 00:37, Schwab,Wilhelm K <[hidden email]> wrote:
>>>>> Does 1.3 by default not create field accessors?  Why is that?  I thought
>>>>> nothing was happening.
>>>>>
>>>>
>>>> Good question, i'd like to know the answer too.
>>>> It is related to MC final 'installation' phase,
>>>> where it initializing all classes.
>>>>
>>>> In NativeBoost i was also using
>>>>
>>>> noteCompilationOf: aSelector meta: isMeta
>>>>      "A hook allowing some classes to react to recompilation of certain selectors"
>>>>
>>>> but there was a big question, at which point this hook is triggered,
>>>> and looks like some changes in MC stop triggering it/triggering at
>>>> wrong time (not all methods get into a class/ class not initialized
>>>> etc),
>>>> which makes it not very useful.
>>>>
>>>> So i ended up creating a DNU handler on instance side, then on DNU i
>>>> check if i have field accessors and if not,
>>>> compile them on the fly.
>>>>
>>>>
>>>>> Bill
>>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Igor Stasenko.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko.
>>>
>>
>>
>>
>
>
>







--
best,
Eliot




--
best,
Eliot

Reply | Threaded
Open this post in threaded view
|

Re: Alien signature: first attempt

Nicolas Cellier
In reply to this post by Eliot Miranda-2
Le 1 mars 2012 23:10, Eliot Miranda <[hidden email]> a écrit :

>
>
> On Thu, Mar 1, 2012 at 1:15 PM, Nicolas Cellier
> <[hidden email]> wrote:
>>
>> Le 1 mars 2012 21:30, Eliot Miranda <[hidden email]> a écrit :
>> >
>> >
>> > On Thu, Mar 1, 2012 at 11:08 AM, Esteban Lorenzano <[hidden email]>
>> > wrote:
>> >>
>> >> Hi,
>> >>
>> >> sorry for not sending before: work is hard :P
>> >> yes, that should work, and no, I don't know a better way to define it.
>> >
>> >
>> > No it won't work.  See my message.
>> >
>> >>
>> >> I think Eliot's idea is to form "libraries" of generic callbacks (for
>> >> mars, for instance, I have CocoaCallback, child of Callback, who
>> >> defines all
>> >> my needed signatures, looks a lot at first instance, but if you see in
>> >> detail, finally there are not so much variants)
>> >
>> >
>> > Yes, the idea is to have a library of signature methods that marshal
>> > specific C callback signatures to/from Smalltalk blocks.  The pragma
>> > specifies what ABI (application binary interface, or stack layout) the
>> > signature is for so that the scheme is cross-platform.  But I expect
>> > that
>> > these signature methods could be the output of an ABI compiler, so that
>> > one
>> > wouldn't have to write them by hand.  The pragma doesn't need to include
>> > a
>> > word-size since IA64 is a different ABI to IA32, and so the ABI subsumes
>> > word size.
>> >
>> > HTH
>> >
>>
>>
>> Let me understand...
>> If I have a signature (char x,double y, short z) IA-32, where is the
>> spalien pointing to?
>
>
> At the first parameter immediately below the return address.
>
>>
>> - the original call stack (with proper ABI alignment, that is offsets
>> 0,4,12, or 1-based indices 1,5,13)
>
>
> Right.
>
>>
>> - a serialized copy without alignment padding (offsets 0,1,9, or
>> 1-based indices 1,2,10)
>
>
> No.
>
>>
>> In the former case, that means that the indices must be computed
>> according to ABI (might be different in PPC for example).
>
>
> Right.  Hence the pragma containing the name of the ABI to allow the system
> to select the appropriate version.  Note that the implementation allows for
> passing a pointer to a struct containing marshalled register arguments for
> RISC platforms, although this isn't used yet since it's not needed on x86.
>
>>
>> And even if such code is generated with an ABI-aware interface
>> compiler, is it really platform independent?
>
>
> The scheme is platform-independent, allowing platform-specific versions of
> marshalling code to exist side-by-side.
>

OK, thanks that's clear.

>>
>> I'm quite sure the ABI have subtle variations, which are not even
>> platform but compiler specific...
>
>
> No.  That's the definition of an ABI.  The ABI specifies the calling
> convention for a particular platform, and compilers must conform to the ABI
> for all external and system calls.
>
>>
>> For example of such niceties in win32, did you ever try returning a
>> structure by value from a cdecl mingw-gcc compiled function to a cdecl
>> visual studio compiled caller, not even speaking of different internal
>> structure alignment padding (controlled with #pragma packed(4) or
>> other compiler options...)
>
>
> That's different.  Internal interfaces used by particular compilers are no
> covered by the ABI but then they aren't used for inter-component interfaces
> such as callbacks.  If you look, for example, at the Intel x86 compiler
> you'll see it'll use a register-based calling convention, multiple
> entry-points for functions (an internal and an external one), etc.  But
> these non-standard interfaces are only used internally, typically between
> functions within a single compilation unit.
>

OK, but in the structure by value case, I created functions exported
as public DLL entry points, and that did not make them
compiler-agnostic.
Unless I did something wrong, VisualStudio and gcc seem to have both a
different idea of cdecl (and stdcall too).

>>
>> This apple page helps a lot, for example it tells how gcc returns a
>> structure by value...
>>
>> https://developer.apple.com/library/mac/#documentation/developertools/Conceptual/LowLevelABI/130-IA-32_Function_Calling_Conventions/IA32.html
>
>
> I know.  But the bibles in these cases are documents such as "SYSTEM V
> APPLICATION BINARY INTERFACE, Intel386? Architecture Processor
> Supplement, Fourth Edition".  You'll find a concise specification of the
> calling convention of the ABI, and structure layout etc, in chapter 3.
>
> cheers,
> Eliot

Yes, I'm pretty sure you know much more than me on the subject, I'm
very reluctant in dealing with such low level unless I'm forced to ;)
And a link to the bible can be found at bottom of apple page for the
other guys having to put some grease on their hands ;)

But from what I read, we should then have a MacOSX-IA-32 ABI and a
SystemV-i386 ABI which are mostly the same with a few exceptions noted
on top of apple's page, and structure returned by value are one such
exception.
My 2¢ advice would be to avoid such exception (for example write a
wrapper that pass a pointer to the struct instead).
Otherwise, one should be prepared to read technical specs and
understand how to write his own marshaller.

Nicolas

>>
>>
>>
>> Nicolas
>>
>> >>
>> >>
>> >> best,
>> >> Esteban
>> >>
>> >> El 01/03/2012, a las 3:57p.m., Schwab,Wilhelm K escribió:
>> >>
>> >> > Esteban,
>> >> >
>> >> > Will this work?  Even if so, is there a better way to define it?
>> >> >
>> >> > longVoidStarLongRetInt: callbackContext sp: spAlien
>> >> >       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
>> >> >       ^callbackContext wordResult:
>> >> >               (block
>> >> >                       value: (Alien newC:4)
>> >> >                       value: (Alien forPointer: (spAlien
>> >> > unsignedLongAt:
>> >> > 5))
>> >> >                       value: (Alien newC:4)
>> >> >               )
>> >> >
>> >> > Bill
>> >> >
>> >> > ________________________________________
>> >> > From: [hidden email]
>> >> > [[hidden email]] on behalf of
>> >> > Schwab,Wilhelm K
>> >> > [[hidden email]]
>> >> > Sent: Thursday, March 01, 2012 12:15 AM
>> >> > To: [hidden email]
>> >> > Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>> >> >
>> >> > thanks!!!!!
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > ________________________________________
>> >> > From: [hidden email]
>> >> > [[hidden email]] on behalf of Esteban
>> >> > Lorenzano
>> >> > [[hidden email]]
>> >> > Sent: Wednesday, February 29, 2012 10:18 PM
>> >> > To: [hidden email]
>> >> > Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>> >> >
>> >> > you need to define a signature... not at home now, but wait 'til
>> >> > tomorrow and I'll send an example
>> >> >
>> >> > best,
>> >> > Esteban
>> >> >
>> >> > El 29/02/2012, a las 11:21p.m., Schwab,Wilhelm K escribió:
>> >> >
>> >> >> What does "cannot find callback signature" mean from
>> >> >> #signature:block:?
>> >> >>  I'm stuck.
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> ________________________________________
>> >> >> From: [hidden email]
>> >> >> [[hidden email]] on behalf of Esteban
>> >> >> Lorenzano
>> >> >> [[hidden email]]
>> >> >> Sent: Wednesday, February 29, 2012 6:54 PM
>> >> >> To: [hidden email]
>> >> >> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>> >> >>
>> >> >> yeah... sorry about that
>> >> >> is a bug on FFI... I managed to worked around it for HPDF, for a
>> >> >> customer's project... but of course is not a good and definitive
>> >> >> solution.
>> >> >>
>> >> >> best,
>> >> >> Esteban
>> >> >>
>> >> >> El 29/02/2012, a las 8:37p.m., Schwab,Wilhelm K escribió:
>> >> >>
>> >> >>> This is gonna take a while...   I had structs flying around as
>> >> >>> void*
>> >> >>> and was moderately happy.  Nonetheless, your suggestion worked,
>> >> >>> provided I
>> >> >>> add a lot getHandle asInteger and change the void* to long :(
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> ________________________________________
>> >> >>> From: [hidden email]
>> >> >>> [[hidden email]] on behalf of Esteban
>> >> >>> Lorenzano
>> >> >>> [[hidden email]]
>> >> >>> Sent: Wednesday, February 29, 2012 6:23 PM
>> >> >>> To: [hidden email]
>> >> >>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>> >> >>>
>> >> >>> I also found some problems using void* in linux... maybe you want
>> >> >>> to
>> >> >>> use long (which has same size)... I "fixed" my problems that way.
>> >> >>>
>> >> >>> yes... maybe we need to look at FFI to see why void* has problems
>> >> >>> some
>> >> >>> times, but well, that can help you atm (sorry for not having a
>> >> >>> better
>> >> >>> answer)
>> >> >>>
>> >> >>> Esteban
>> >> >>>
>> >> >>> El 29/02/2012, a las 8:11p.m., Igor Stasenko escribió:
>> >> >>>
>> >> >>>> On 1 March 2012 00:58, Schwab,Wilhelm K <[hidden email]>
>> >> >>>> wrote:
>> >> >>>>> Another glitch: is there any problem passing things as void*?
>> >> >>>>>  I'm
>> >> >>>>> getting failure to coerce errors that did not arise before.
>> >> >>>>>
>> >> >>>> No idea. As you may suspect, i stopped using FFI/Alien once i got
>> >> >>>> NativeBoost toy to play with.
>> >> >>>>
>> >> >>>> Please file the issue, describing the problem. so we can look over
>> >> >>>> it
>> >> >>>> and fix it.
>> >> >>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>> ________________________________________
>> >> >>>>> From: [hidden email]
>> >> >>>>> [[hidden email]] on behalf of Igor
>> >> >>>>> Stasenko
>> >> >>>>> [[hidden email]]
>> >> >>>>> Sent: Wednesday, February 29, 2012 5:51 PM
>> >> >>>>> To: [hidden email]
>> >> >>>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>> >> >>>>>
>> >> >>>>> On 1 March 2012 00:37, Schwab,Wilhelm K <[hidden email]>
>> >> >>>>> wrote:
>> >> >>>>>> Does 1.3 by default not create field accessors?  Why is that?  I
>> >> >>>>>> thought
>> >> >>>>>> nothing was happening.
>> >> >>>>>>
>> >> >>>>>
>> >> >>>>> Good question, i'd like to know the answer too.
>> >> >>>>> It is related to MC final 'installation' phase,
>> >> >>>>> where it initializing all classes.
>> >> >>>>>
>> >> >>>>> In NativeBoost i was also using
>> >> >>>>>
>> >> >>>>> noteCompilationOf: aSelector meta: isMeta
>> >> >>>>>     "A hook allowing some classes to react to recompilation of
>> >> >>>>> certain selectors"
>> >> >>>>>
>> >> >>>>> but there was a big question, at which point this hook is
>> >> >>>>> triggered,
>> >> >>>>> and looks like some changes in MC stop triggering it/triggering
>> >> >>>>> at
>> >> >>>>> wrong time (not all methods get into a class/ class not
>> >> >>>>> initialized
>> >> >>>>> etc),
>> >> >>>>> which makes it not very useful.
>> >> >>>>>
>> >> >>>>> So i ended up creating a DNU handler on instance side, then on
>> >> >>>>> DNU i
>> >> >>>>> check if i have field accessors and if not,
>> >> >>>>> compile them on the fly.
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>> Bill
>> >> >>>>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>> --
>> >> >>>>> Best regards,
>> >> >>>>> Igor Stasenko.
>> >> >>>>>
>> >> >>>>>
>> >> >>>>
>> >> >>>>
>> >> >>>>
>> >> >>>> --
>> >> >>>> Best regards,
>> >> >>>> Igor Stasenko.
>> >> >>>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>
>> >> >>
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> >
>> >>
>> >>
>> >
>> >
>> >
>> > --
>> > best,
>> > Eliot
>> >
>>
>
>
>
> --
> best,
> Eliot
>

Reply | Threaded
Open this post in threaded view
|

Re: Alien signature: first attempt

Eliot Miranda-2
In reply to this post by Eliot Miranda-2


On Thu, Mar 1, 2012 at 2:10 PM, Eliot Miranda <[hidden email]> wrote:


On Thu, Mar 1, 2012 at 1:15 PM, Nicolas Cellier <[hidden email]> wrote:
Le 1 mars 2012 21:30, Eliot Miranda <[hidden email]> a écrit :
>
>
> On Thu, Mar 1, 2012 at 11:08 AM, Esteban Lorenzano <[hidden email]>
> wrote:
>>
>> Hi,
>>
>> sorry for not sending before: work is hard :P
>> yes, that should work, and no, I don't know a better way to define it.
>
>
> No it won't work.  See my message.
>
>>
>> I think Eliot's idea is to form "libraries" of generic callbacks (for
>> mars, for instance, I have CocoaCallback, child of Callback, who defines all
>> my needed signatures, looks a lot at first instance, but if you see in
>> detail, finally there are not so much variants)
>
>
> Yes, the idea is to have a library of signature methods that marshal
> specific C callback signatures to/from Smalltalk blocks.  The pragma
> specifies what ABI (application binary interface, or stack layout) the
> signature is for so that the scheme is cross-platform.  But I expect that
> these signature methods could be the output of an ABI compiler, so that one
> wouldn't have to write them by hand.  The pragma doesn't need to include a
> word-size since IA64 is a different ABI to IA32, and so the ABI subsumes
> word size.
>
> HTH
>


Let me understand...
If I have a signature (char x,double y, short z) IA-32, where is the
spalien pointing to?

At the first parameter immediately below the return address.
 
- the original call stack (with proper ABI alignment, that is offsets
0,4,12, or 1-based indices 1,5,13)

Right.
 
- a serialized copy without alignment padding (offsets 0,1,9, or
1-based indices 1,2,10)

No.
 
In the former case, that means that the indices must be computed
according to ABI (might be different in PPC for example).

Right.  Hence the pragma containing the name of the ABI to allow the system to select the appropriate version.  Note that the implementation allows for passing a pointer to a struct containing marshalled register arguments for RISC platforms, although this isn't used yet since it's not needed on x86.


And even if such code is generated with an ABI-aware interface
compiler, is it really platform independent?

The scheme is platform-independent, allowing platform-specific versions of marshalling code to exist side-by-side.
 
I'm quite sure the ABI have subtle variations, which are not even
platform but compiler specific...

No.  That's the definition of an ABI.  The ABI specifies the calling convention for a particular platform, and compilers must conform to the ABI for all external and system calls.
 
For example of such niceties in win32, did you ever try returning a
structure by value from a cdecl mingw-gcc compiled function to a cdecl
visual studio compiled caller, not even speaking of different internal
structure alignment padding (controlled with #pragma packed(4) or
other compiler options...)

That's different.  Internal interfaces used by particular compilers are no covered by the ABI but then they aren't used for inter-component interfaces such as callbacks.  If you look, for example, at the Intel x86 compiler you'll see it'll use a register-based calling convention, multiple entry-points for functions (an internal and an external one), etc.  But these non-standard interfaces are only used internally, typically between functions within a single compilation unit.
 
This apple page helps a lot, for example it tells how gcc returns a
structure by value...
https://developer.apple.com/library/mac/#documentation/developertools/Conceptual/LowLevelABI/130-IA-32_Function_Calling_Conventions/IA32.html

I know.  But the bibles in these cases are documents such as "SYSTEM V APPLICATION BINARY INTERFACE, Intel386? Architecture Processor Supplement, Fourth Edition".  You'll find a concise specification of the calling convention of the ABI, and structure layout etc, in chapter 3.

Actually Nicolas, you're quite right.  One needs to use the Apple document on those platforms that use SSE2.  These mandate a 16-byte stack alignment to allow SSE instructions to function properly (many of them require 8-byte alignment).  But there's a small difference between the two and the one set of VM code copes with linux, Mac and win32, including the pascal calling convention.
 

cheers,
Eliot


Nicolas

>>
>>
>> best,
>> Esteban
>>
>> El 01/03/2012, a las 3:57p.m., Schwab,Wilhelm K escribió:
>>
>> > Esteban,
>> >
>> > Will this work?  Even if so, is there a better way to define it?
>> >
>> > longVoidStarLongRetInt: callbackContext sp: spAlien
>> >       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
>> >       ^callbackContext wordResult:
>> >               (block
>> >                       value: (Alien newC:4)
>> >                       value: (Alien forPointer: (spAlien unsignedLongAt:
>> > 5))
>> >                       value: (Alien newC:4)
>> >               )
>> >
>> > Bill
>> >
>> > ________________________________________
>> > From: [hidden email]
>> > [[hidden email]] on behalf of Schwab,Wilhelm K
>> > [[hidden email]]
>> > Sent: Thursday, March 01, 2012 12:15 AM
>> > To: [hidden email]
>> > Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>> >
>> > thanks!!!!!
>> >
>> >
>> >
>> >
>> > ________________________________________
>> > From: [hidden email]
>> > [[hidden email]] on behalf of Esteban Lorenzano
>> > [[hidden email]]
>> > Sent: Wednesday, February 29, 2012 10:18 PM
>> > To: [hidden email]
>> > Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>> >
>> > you need to define a signature... not at home now, but wait 'til
>> > tomorrow and I'll send an example
>> >
>> > best,
>> > Esteban
>> >
>> > El 29/02/2012, a las 11:21p.m., Schwab,Wilhelm K escribió:
>> >
>> >> What does "cannot find callback signature" mean from #signature:block:?
>> >>  I'm stuck.
>> >>
>> >>
>> >>
>> >>
>> >> ________________________________________
>> >> From: [hidden email]
>> >> [[hidden email]] on behalf of Esteban Lorenzano
>> >> [[hidden email]]
>> >> Sent: Wednesday, February 29, 2012 6:54 PM
>> >> To: [hidden email]
>> >> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>> >>
>> >> yeah... sorry about that
>> >> is a bug on FFI... I managed to worked around it for HPDF, for a
>> >> customer's project... but of course is not a good and definitive solution.
>> >>
>> >> best,
>> >> Esteban
>> >>
>> >> El 29/02/2012, a las 8:37p.m., Schwab,Wilhelm K escribió:
>> >>
>> >>> This is gonna take a while...   I had structs flying around as void*
>> >>> and was moderately happy.  Nonetheless, your suggestion worked, provided I
>> >>> add a lot getHandle asInteger and change the void* to long :(
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> ________________________________________
>> >>> From: [hidden email]
>> >>> [[hidden email]] on behalf of Esteban Lorenzano
>> >>> [[hidden email]]
>> >>> Sent: Wednesday, February 29, 2012 6:23 PM
>> >>> To: [hidden email]
>> >>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>> >>>
>> >>> I also found some problems using void* in linux... maybe you want to
>> >>> use long (which has same size)... I "fixed" my problems that way.
>> >>>
>> >>> yes... maybe we need to look at FFI to see why void* has problems some
>> >>> times, but well, that can help you atm (sorry for not having a better
>> >>> answer)
>> >>>
>> >>> Esteban
>> >>>
>> >>> El 29/02/2012, a las 8:11p.m., Igor Stasenko escribió:
>> >>>
>> >>>> On 1 March 2012 00:58, Schwab,Wilhelm K <[hidden email]>
>> >>>> wrote:
>> >>>>> Another glitch: is there any problem passing things as void*?  I'm
>> >>>>> getting failure to coerce errors that did not arise before.
>> >>>>>
>> >>>> No idea. As you may suspect, i stopped using FFI/Alien once i got
>> >>>> NativeBoost toy to play with.
>> >>>>
>> >>>> Please file the issue, describing the problem. so we can look over it
>> >>>> and fix it.
>> >>>>
>> >>>>>
>> >>>>>
>> >>>>> ________________________________________
>> >>>>> From: [hidden email]
>> >>>>> [[hidden email]] on behalf of Igor Stasenko
>> >>>>> [[hidden email]]
>> >>>>> Sent: Wednesday, February 29, 2012 5:51 PM
>> >>>>> To: [hidden email]
>> >>>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>> >>>>>
>> >>>>> On 1 March 2012 00:37, Schwab,Wilhelm K <[hidden email]>
>> >>>>> wrote:
>> >>>>>> Does 1.3 by default not create field accessors?  Why is that?  I
>> >>>>>> thought
>> >>>>>> nothing was happening.
>> >>>>>>
>> >>>>>
>> >>>>> Good question, i'd like to know the answer too.
>> >>>>> It is related to MC final 'installation' phase,
>> >>>>> where it initializing all classes.
>> >>>>>
>> >>>>> In NativeBoost i was also using
>> >>>>>
>> >>>>> noteCompilationOf: aSelector meta: isMeta
>> >>>>>     "A hook allowing some classes to react to recompilation of
>> >>>>> certain selectors"
>> >>>>>
>> >>>>> but there was a big question, at which point this hook is triggered,
>> >>>>> and looks like some changes in MC stop triggering it/triggering at
>> >>>>> wrong time (not all methods get into a class/ class not initialized
>> >>>>> etc),
>> >>>>> which makes it not very useful.
>> >>>>>
>> >>>>> So i ended up creating a DNU handler on instance side, then on DNU i
>> >>>>> check if i have field accessors and if not,
>> >>>>> compile them on the fly.
>> >>>>>
>> >>>>>
>> >>>>>> Bill
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Best regards,
>> >>>>> Igor Stasenko.
>> >>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Best regards,
>> >>>> Igor Stasenko.
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>
>> >>
>> >>
>> >
>> >
>> >
>> >
>>
>>
>
>
>
> --
> best,
> Eliot
>




--
best,
Eliot




--
best,
Eliot

Reply | Threaded
Open this post in threaded view
|

Re: Alien signature: first attempt

Schwab,Wilhelm K
In reply to this post by Eliot Miranda-2
Eliot,

I've unearthed some code from Dolphin, and might want to use something like this:

vectorStarVoidStarMatrixStarRetInt: callbackContext sp: spAlien
     <signature: 'int (*) (Gsl_vector *, void *, Gsl_matrix *)' abi: 'IA32'>
    ^callbackContext wordResult:(
        block
            value: (Alien forPointer:( spAlien unsignedLongAt: 1))
            value: (Alien forPointer: (spAlien unsignedLongAt: 5))
            value: (Alien forPointer:( spAlien unsignedLongAt: 9))
    ).

Am I at all on the right track?   The vector and matrix are structs.

Bill





From: [hidden email] [[hidden email]] on behalf of Eliot Miranda [[hidden email]]
Sent: Thursday, March 01, 2012 5:38 PM
To: [hidden email]
Subject: Re: [Pharo-project] Alien signature: first attempt



On Thu, Mar 1, 2012 at 1:52 PM, Schwab,Wilhelm K <[hidden email]> wrote:
Eliot,

That helps a bit; it's still unfamiliar territory.  I am a little bothered by #wordResult: and #floatResult.  There are times when one is expected to return 16 bit values (rare) and either a float or a double (common).  Does that flexibility exist?

Yes, the flexibility exists.  But the mechanism needed to provide the flexibility is less than one might expect.  For example, on x86 there are three fundamental return conventions, one for integral values, one for floating-point values, and one for structure values.  See e.g. chapter 3 of the Sys V ABI for 386 (in my reply to Nicolas).  Read it; it's good for the soul; makes one value Smalltalk's simplicity.

Integral values are returned in %eax, %edx.  edx is not used if the value fits in 32-bits.  So the same sequence can be used to answer all integral values from 1 byte to 8 bytes, irrespective of sign.  That said, I use two sequences, one for 32-bits or less and one for 33 to 64 bits.  In your example of answering 16 bits one simply answers either the sign-extension of 16-bits (if signed) to 32-bits or the zero-extension of 16-bits to 32-bits (if unsigned).  In any case, the result is answered in %eax, and it is up to the caller to access the result correctly.

Float values are returned on the top of the floating point register stack (%st(0)).   The same machine-code sequence can be used to answer single, double and extended precision floats, since on the fp stack, they are all represented in the same extended format. 
 
So on x86 the callback implementation supports 4 return mechanisms, retword, retword64, retdouble and retstruct.  So far only two of these are fleshed out in Smalltalk code.  But look at http://www.squeakvm.org/svn/squeak/branches/Cog/src/vm/vmCallback.h &  http://squeakvm.org/svn/squeak/trunk/platforms/Cross/plugins/IA32ABI/ia32abicc.c and you'll see the other two.

cheers,
Eliot
 

Bill





From: [hidden email] [[hidden email]] on behalf of Eliot Miranda [[hidden email]]
Sent: Thursday, March 01, 2012 3:26 PM
To: [hidden email]
Subject: Re: [Pharo-project] Alien signature: first attempt



On Thu, Mar 1, 2012 at 10:57 AM, Schwab,Wilhelm K <[hidden email]> wrote:
Esteban,

Will this work?  Even if so, is there a better way to define it?

No it won't work, because you don't understand what it does.
 

longVoidStarLongRetInt: callbackContext sp: spAlien
       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
       ^callbackContext wordResult:
               (block
                       value: (Alien newC:4)
                       value: (Alien forPointer: (spAlien unsignedLongAt: 5))
                       value: (Alien newC:4)
               )

The above takes only one parameter's value from the spAlien (the void * value) and passes garbage for the other parameters. The spAlien is a pointer to the incoming C callback stack frame, i.e. a pointer to the callback's parameters.  So spAlien unsignedLongAt: 5 gets the four-byte pointer for the void * parameter, wraps it up in a convenient pointer Alien (so the callback block can e.g. indirect through it to get at the data the pointer is pointing to), and passes it as the second argument of the callback block.  But the two Alien newC: 4's are entirely bogus, passing effectively uninitialized data for the first and third parameters of the callback block.  They're just empty Aliens, when in fact you want to pass two longs to the block for the first and third parameters.

So to pass values in you need something like:

longVoidStarLongRetInt: callbackContext sp: spAlien
       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
       ^callbackContext wordResult:
               (block
                       value: (
spAlien signedLongAt: 1)
                       value: (Alien forPointer: (spAlien unsignedLongAt: 5))
                       value: (
spAlien signedLongAt: 9)
               )

You might also need to consider passing the result of the block back out.  If you only want to use it to answer integers then the above is fine.  But take a look at the intcharstarRetint:sp: signature:

intcharstarRetint: callbackContext sp: spAlien
<signature: 'int (*)(int, char *)' abi: 'IA32'>
^callbackContext wordResult:
((block
value: (spAlien signedLongAt: 1) ~= 0
value: (Alien forPointer: (spAlien unsignedLongAt: 5)) strcpyUTF8)
ifNil: [0]
ifNotNil:
[:result|
result isInteger
ifTrue: [result]
ifFalse: [result == true ifTrue: [1] ifFalse: [0]]])

Analogously it passes a signed integer as the first argument and a UTF8 string derived from the second char * parameter. (The method should probably be called intcharstarasUTF8Retint:sp: or some such; forgive me, we're getting there).

But on return it takes the result of the callback block and passes it to the ifNil:ifNotNil: which
- maps a nil result form the block to 0 (ifNil: [0])
- maps integers to integers (which will only deal correctly with integers in the 32-bit range), and
- maps true to 1, mapping all other results to 0.

Does what's going on now make more sense?

HTH,
Eliot


Bill

________________________________________
From: [hidden email] [[hidden email]] on behalf of Schwab,Wilhelm K [[hidden email]]
Sent: Thursday, March 01, 2012 12:15 AM
To: [hidden email]
Subject: Re: [Pharo-project] FFI on 1.3: compile fields

thanks!!!!!




________________________________________
From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
Sent: Wednesday, February 29, 2012 10:18 PM
To: [hidden email]
Subject: Re: [Pharo-project] FFI on 1.3: compile fields

you need to define a signature... not at home now, but wait 'til tomorrow and I'll send an example

best,
Esteban

El 29/02/2012, a las 11:21p.m., Schwab,Wilhelm K escribió:

> What does "cannot find callback signature" mean from #signature:block:?  I'm stuck.
>
>
>
>
> ________________________________________
> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
> Sent: Wednesday, February 29, 2012 6:54 PM
> To: [hidden email]
> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>
> yeah... sorry about that
> is a bug on FFI... I managed to worked around it for HPDF, for a customer's project... but of course is not a good and definitive solution.
>
> best,
> Esteban
>
> El 29/02/2012, a las 8:37p.m., Schwab,Wilhelm K escribió:
>
>> This is gonna take a while...   I had structs flying around as void* and was moderately happy.  Nonetheless, your suggestion worked, provided I add a lot getHandle asInteger and change the void* to long :(
>>
>>
>>
>>
>> ________________________________________
>> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
>> Sent: Wednesday, February 29, 2012 6:23 PM
>> To: [hidden email]
>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>
>> I also found some problems using void* in linux... maybe you want to use long (which has same size)... I "fixed" my problems that way.
>>
>> yes... maybe we need to look at FFI to see why void* has problems some times, but well, that can help you atm (sorry for not having a better answer)
>>
>> Esteban
>>
>> El 29/02/2012, a las 8:11p.m., Igor Stasenko escribió:
>>
>>> On 1 March 2012 00:58, Schwab,Wilhelm K <[hidden email]> wrote:
>>>> Another glitch: is there any problem passing things as void*?  I'm getting failure to coerce errors that did not arise before.
>>>>
>>> No idea. As you may suspect, i stopped using FFI/Alien once i got
>>> NativeBoost toy to play with.
>>>
>>> Please file the issue, describing the problem. so we can look over it
>>> and fix it.
>>>
>>>>
>>>>
>>>> ________________________________________
>>>> From: [hidden email] [[hidden email]] on behalf of Igor Stasenko [[hidden email]]
>>>> Sent: Wednesday, February 29, 2012 5:51 PM
>>>> To: [hidden email]
>>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>>>
>>>> On 1 March 2012 00:37, Schwab,Wilhelm K <[hidden email]> wrote:
>>>>> Does 1.3 by default not create field accessors?  Why is that?  I thought
>>>>> nothing was happening.
>>>>>
>>>>
>>>> Good question, i'd like to know the answer too.
>>>> It is related to MC final 'installation' phase,
>>>> where it initializing all classes.
>>>>
>>>> In NativeBoost i was also using
>>>>
>>>> noteCompilationOf: aSelector meta: isMeta
>>>>      "A hook allowing some classes to react to recompilation of certain selectors"
>>>>
>>>> but there was a big question, at which point this hook is triggered,
>>>> and looks like some changes in MC stop triggering it/triggering at
>>>> wrong time (not all methods get into a class/ class not initialized
>>>> etc),
>>>> which makes it not very useful.
>>>>
>>>> So i ended up creating a DNU handler on instance side, then on DNU i
>>>> check if i have field accessors and if not,
>>>> compile them on the fly.
>>>>
>>>>
>>>>> Bill
>>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Igor Stasenko.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko.
>>>
>>
>>
>>
>
>
>







--
best,
Eliot




--
best,
Eliot

Reply | Threaded
Open this post in threaded view
|

Re: Alien signature: first attempt

Eliot Miranda-2


On Thu, Mar 1, 2012 at 6:45 PM, Schwab,Wilhelm K <[hidden email]> wrote:
Eliot,

I've unearthed some code from Dolphin, and might want to use something like this:

vectorStarVoidStarMatrixStarRetInt: callbackContext sp: spAlien
     <signature: 'int (*) (Gsl_vector *, void *, Gsl_matrix *)' abi: 'IA32'>
    ^callbackContext wordResult:(
        block
            value: (Alien forPointer:( spAlien unsignedLongAt: 1))
            value: (Alien forPointer: (spAlien unsignedLongAt: 5))
            value: (Alien forPointer:( spAlien unsignedLongAt: 9))
    ).

Am I at all on the right track?   The vector and matrix are structs.

Yes, exactly.  The next level is to create Alien subclasses for Gsl_vector and Gsl_matrix and use these in place of the simple Aliens.  These e.g. GslMatrixIA32Alien and GslVectorIA32Alien classes would define higher-level accessors so that your callback can access their members by name, not by offset.  Of course these classes should be auto-generated by parsing e.g. a C header file using the rules of the platform's ABI.

You've got it.



Bill





From: [hidden email] [[hidden email]] on behalf of Eliot Miranda [[hidden email]]
Sent: Thursday, March 01, 2012 5:38 PM

To: [hidden email]
Subject: Re: [Pharo-project] Alien signature: first attempt



On Thu, Mar 1, 2012 at 1:52 PM, Schwab,Wilhelm K <[hidden email]> wrote:
Eliot,

That helps a bit; it's still unfamiliar territory.  I am a little bothered by #wordResult: and #floatResult.  There are times when one is expected to return 16 bit values (rare) and either a float or a double (common).  Does that flexibility exist?

Yes, the flexibility exists.  But the mechanism needed to provide the flexibility is less than one might expect.  For example, on x86 there are three fundamental return conventions, one for integral values, one for floating-point values, and one for structure values.  See e.g. chapter 3 of the Sys V ABI for 386 (in my reply to Nicolas).  Read it; it's good for the soul; makes one value Smalltalk's simplicity.

Integral values are returned in %eax, %edx.  edx is not used if the value fits in 32-bits.  So the same sequence can be used to answer all integral values from 1 byte to 8 bytes, irrespective of sign.  That said, I use two sequences, one for 32-bits or less and one for 33 to 64 bits.  In your example of answering 16 bits one simply answers either the sign-extension of 16-bits (if signed) to 32-bits or the zero-extension of 16-bits to 32-bits (if unsigned).  In any case, the result is answered in %eax, and it is up to the caller to access the result correctly.

Float values are returned on the top of the floating point register stack (%st(0)).   The same machine-code sequence can be used to answer single, double and extended precision floats, since on the fp stack, they are all represented in the same extended format. 
 
So on x86 the callback implementation supports 4 return mechanisms, retword, retword64, retdouble and retstruct.  So far only two of these are fleshed out in Smalltalk code.  But look at http://www.squeakvm.org/svn/squeak/branches/Cog/src/vm/vmCallback.h &  http://squeakvm.org/svn/squeak/trunk/platforms/Cross/plugins/IA32ABI/ia32abicc.c and you'll see the other two.

cheers,
Eliot
 

Bill





From: [hidden email] [[hidden email]] on behalf of Eliot Miranda [[hidden email]]
Sent: Thursday, March 01, 2012 3:26 PM
To: [hidden email]
Subject: Re: [Pharo-project] Alien signature: first attempt



On Thu, Mar 1, 2012 at 10:57 AM, Schwab,Wilhelm K <[hidden email]> wrote:
Esteban,

Will this work?  Even if so, is there a better way to define it?

No it won't work, because you don't understand what it does.
 

longVoidStarLongRetInt: callbackContext sp: spAlien
       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
       ^callbackContext wordResult:
               (block
                       value: (Alien newC:4)
                       value: (Alien forPointer: (spAlien unsignedLongAt: 5))
                       value: (Alien newC:4)
               )

The above takes only one parameter's value from the spAlien (the void * value) and passes garbage for the other parameters. The spAlien is a pointer to the incoming C callback stack frame, i.e. a pointer to the callback's parameters.  So spAlien unsignedLongAt: 5 gets the four-byte pointer for the void * parameter, wraps it up in a convenient pointer Alien (so the callback block can e.g. indirect through it to get at the data the pointer is pointing to), and passes it as the second argument of the callback block.  But the two Alien newC: 4's are entirely bogus, passing effectively uninitialized data for the first and third parameters of the callback block.  They're just empty Aliens, when in fact you want to pass two longs to the block for the first and third parameters.

So to pass values in you need something like:

longVoidStarLongRetInt: callbackContext sp: spAlien
       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
       ^callbackContext wordResult:
               (block
                       value: (
spAlien signedLongAt: 1)
                       value: (Alien forPointer: (spAlien unsignedLongAt: 5))
                       value: (
spAlien signedLongAt: 9)
               )

You might also need to consider passing the result of the block back out.  If you only want to use it to answer integers then the above is fine.  But take a look at the intcharstarRetint:sp: signature:

intcharstarRetint: callbackContext sp: spAlien
<signature: 'int (*)(int, char *)' abi: 'IA32'>
^callbackContext wordResult:
((block
value: (spAlien signedLongAt: 1) ~= 0
value: (Alien forPointer: (spAlien unsignedLongAt: 5)) strcpyUTF8)
ifNil: [0]
ifNotNil:
[:result|
result isInteger
ifTrue: [result]
ifFalse: [result == true ifTrue: [1] ifFalse: [0]]])

Analogously it passes a signed integer as the first argument and a UTF8 string derived from the second char * parameter. (The method should probably be called intcharstarasUTF8Retint:sp: or some such; forgive me, we're getting there).

But on return it takes the result of the callback block and passes it to the ifNil:ifNotNil: which
- maps a nil result form the block to 0 (ifNil: [0])
- maps integers to integers (which will only deal correctly with integers in the 32-bit range), and
- maps true to 1, mapping all other results to 0.

Does what's going on now make more sense?

HTH,
Eliot


Bill

________________________________________
From: [hidden email] [[hidden email]] on behalf of Schwab,Wilhelm K [[hidden email]]
Sent: Thursday, March 01, 2012 12:15 AM
To: [hidden email]
Subject: Re: [Pharo-project] FFI on 1.3: compile fields

thanks!!!!!




________________________________________
From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
Sent: Wednesday, February 29, 2012 10:18 PM
To: [hidden email]
Subject: Re: [Pharo-project] FFI on 1.3: compile fields

you need to define a signature... not at home now, but wait 'til tomorrow and I'll send an example

best,
Esteban

El 29/02/2012, a las 11:21p.m., Schwab,Wilhelm K escribió:

> What does "cannot find callback signature" mean from #signature:block:?  I'm stuck.
>
>
>
>
> ________________________________________
> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
> Sent: Wednesday, February 29, 2012 6:54 PM
> To: [hidden email]
> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>
> yeah... sorry about that
> is a bug on FFI... I managed to worked around it for HPDF, for a customer's project... but of course is not a good and definitive solution.
>
> best,
> Esteban
>
> El 29/02/2012, a las 8:37p.m., Schwab,Wilhelm K escribió:
>
>> This is gonna take a while...   I had structs flying around as void* and was moderately happy.  Nonetheless, your suggestion worked, provided I add a lot getHandle asInteger and change the void* to long :(
>>
>>
>>
>>
>> ________________________________________
>> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
>> Sent: Wednesday, February 29, 2012 6:23 PM
>> To: [hidden email]
>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>
>> I also found some problems using void* in linux... maybe you want to use long (which has same size)... I "fixed" my problems that way.
>>
>> yes... maybe we need to look at FFI to see why void* has problems some times, but well, that can help you atm (sorry for not having a better answer)
>>
>> Esteban
>>
>> El 29/02/2012, a las 8:11p.m., Igor Stasenko escribió:
>>
>>> On 1 March 2012 00:58, Schwab,Wilhelm K <[hidden email]> wrote:
>>>> Another glitch: is there any problem passing things as void*?  I'm getting failure to coerce errors that did not arise before.
>>>>
>>> No idea. As you may suspect, i stopped using FFI/Alien once i got
>>> NativeBoost toy to play with.
>>>
>>> Please file the issue, describing the problem. so we can look over it
>>> and fix it.
>>>
>>>>
>>>>
>>>> ________________________________________
>>>> From: [hidden email] [[hidden email]] on behalf of Igor Stasenko [[hidden email]]
>>>> Sent: Wednesday, February 29, 2012 5:51 PM
>>>> To: [hidden email]
>>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>>>
>>>> On 1 March 2012 00:37, Schwab,Wilhelm K <[hidden email]> wrote:
>>>>> Does 1.3 by default not create field accessors?  Why is that?  I thought
>>>>> nothing was happening.
>>>>>
>>>>
>>>> Good question, i'd like to know the answer too.
>>>> It is related to MC final 'installation' phase,
>>>> where it initializing all classes.
>>>>
>>>> In NativeBoost i was also using
>>>>
>>>> noteCompilationOf: aSelector meta: isMeta
>>>>      "A hook allowing some classes to react to recompilation of certain selectors"
>>>>
>>>> but there was a big question, at which point this hook is triggered,
>>>> and looks like some changes in MC stop triggering it/triggering at
>>>> wrong time (not all methods get into a class/ class not initialized
>>>> etc),
>>>> which makes it not very useful.
>>>>
>>>> So i ended up creating a DNU handler on instance side, then on DNU i
>>>> check if i have field accessors and if not,
>>>> compile them on the fly.
>>>>
>>>>
>>>>> Bill
>>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Igor Stasenko.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko.
>>>
>>
>>
>>
>
>
>







--
best,
Eliot




--
best,
Eliot




--
best,
Eliot

Reply | Threaded
Open this post in threaded view
|

Re: Alien signature: first attempt

Schwab,Wilhelm K
Eliot,

Thanks for the help.  I'm not as convinced as you that I have it<g>, but I'm getting there.
Alien subclasses per struct - interesting!

Bill





From: [hidden email] [[hidden email]] on behalf of Eliot Miranda [[hidden email]]
Sent: Thursday, March 01, 2012 9:56 PM
To: [hidden email]
Subject: Re: [Pharo-project] Alien signature: first attempt



On Thu, Mar 1, 2012 at 6:45 PM, Schwab,Wilhelm K <[hidden email]> wrote:
Eliot,

I've unearthed some code from Dolphin, and might want to use something like this:

vectorStarVoidStarMatrixStarRetInt: callbackContext sp: spAlien
     <signature: 'int (*) (Gsl_vector *, void *, Gsl_matrix *)' abi: 'IA32'>
    ^callbackContext wordResult:(
        block
            value: (Alien forPointer:( spAlien unsignedLongAt: 1))
            value: (Alien forPointer: (spAlien unsignedLongAt: 5))
            value: (Alien forPointer:( spAlien unsignedLongAt: 9))
    ).

Am I at all on the right track?   The vector and matrix are structs.

Yes, exactly.  The next level is to create Alien subclasses for Gsl_vector and Gsl_matrix and use these in place of the simple Aliens.  These e.g. GslMatrixIA32Alien and GslVectorIA32Alien classes would define higher-level accessors so that your callback can access their members by name, not by offset.  Of course these classes should be auto-generated by parsing e.g. a C header file using the rules of the platform's ABI.

You've got it.



Bill





From: [hidden email] [[hidden email]] on behalf of Eliot Miranda [[hidden email]]
Sent: Thursday, March 01, 2012 5:38 PM

To: [hidden email]
Subject: Re: [Pharo-project] Alien signature: first attempt



On Thu, Mar 1, 2012 at 1:52 PM, Schwab,Wilhelm K <[hidden email]> wrote:
Eliot,

That helps a bit; it's still unfamiliar territory.  I am a little bothered by #wordResult: and #floatResult.  There are times when one is expected to return 16 bit values (rare) and either a float or a double (common).  Does that flexibility exist?

Yes, the flexibility exists.  But the mechanism needed to provide the flexibility is less than one might expect.  For example, on x86 there are three fundamental return conventions, one for integral values, one for floating-point values, and one for structure values.  See e.g. chapter 3 of the Sys V ABI for 386 (in my reply to Nicolas).  Read it; it's good for the soul; makes one value Smalltalk's simplicity.

Integral values are returned in %eax, %edx.  edx is not used if the value fits in 32-bits.  So the same sequence can be used to answer all integral values from 1 byte to 8 bytes, irrespective of sign.  That said, I use two sequences, one for 32-bits or less and one for 33 to 64 bits.  In your example of answering 16 bits one simply answers either the sign-extension of 16-bits (if signed) to 32-bits or the zero-extension of 16-bits to 32-bits (if unsigned).  In any case, the result is answered in %eax, and it is up to the caller to access the result correctly.

Float values are returned on the top of the floating point register stack (%st(0)).   The same machine-code sequence can be used to answer single, double and extended precision floats, since on the fp stack, they are all represented in the same extended format. 
 
So on x86 the callback implementation supports 4 return mechanisms, retword, retword64, retdouble and retstruct.  So far only two of these are fleshed out in Smalltalk code.  But look at http://www.squeakvm.org/svn/squeak/branches/Cog/src/vm/vmCallback.h &  http://squeakvm.org/svn/squeak/trunk/platforms/Cross/plugins/IA32ABI/ia32abicc.c and you'll see the other two.

cheers,
Eliot
 

Bill





From: [hidden email] [[hidden email]] on behalf of Eliot Miranda [[hidden email]]
Sent: Thursday, March 01, 2012 3:26 PM
To: [hidden email]
Subject: Re: [Pharo-project] Alien signature: first attempt



On Thu, Mar 1, 2012 at 10:57 AM, Schwab,Wilhelm K <[hidden email]> wrote:
Esteban,

Will this work?  Even if so, is there a better way to define it?

No it won't work, because you don't understand what it does.
 

longVoidStarLongRetInt: callbackContext sp: spAlien
       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
       ^callbackContext wordResult:
               (block
                       value: (Alien newC:4)
                       value: (Alien forPointer: (spAlien unsignedLongAt: 5))
                       value: (Alien newC:4)
               )

The above takes only one parameter's value from the spAlien (the void * value) and passes garbage for the other parameters. The spAlien is a pointer to the incoming C callback stack frame, i.e. a pointer to the callback's parameters.  So spAlien unsignedLongAt: 5 gets the four-byte pointer for the void * parameter, wraps it up in a convenient pointer Alien (so the callback block can e.g. indirect through it to get at the data the pointer is pointing to), and passes it as the second argument of the callback block.  But the two Alien newC: 4's are entirely bogus, passing effectively uninitialized data for the first and third parameters of the callback block.  They're just empty Aliens, when in fact you want to pass two longs to the block for the first and third parameters.

So to pass values in you need something like:

longVoidStarLongRetInt: callbackContext sp: spAlien
       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
       ^callbackContext wordResult:
               (block
                       value: (
spAlien signedLongAt: 1)
                       value: (Alien forPointer: (spAlien unsignedLongAt: 5))
                       value: (
spAlien signedLongAt: 9)
               )

You might also need to consider passing the result of the block back out.  If you only want to use it to answer integers then the above is fine.  But take a look at the intcharstarRetint:sp: signature:

intcharstarRetint: callbackContext sp: spAlien
<signature: 'int (*)(int, char *)' abi: 'IA32'>
^callbackContext wordResult:
((block
value: (spAlien signedLongAt: 1) ~= 0
value: (Alien forPointer: (spAlien unsignedLongAt: 5)) strcpyUTF8)
ifNil: [0]
ifNotNil:
[:result|
result isInteger
ifTrue: [result]
ifFalse: [result == true ifTrue: [1] ifFalse: [0]]])

Analogously it passes a signed integer as the first argument and a UTF8 string derived from the second char * parameter. (The method should probably be called intcharstarasUTF8Retint:sp: or some such; forgive me, we're getting there).

But on return it takes the result of the callback block and passes it to the ifNil:ifNotNil: which
- maps a nil result form the block to 0 (ifNil: [0])
- maps integers to integers (which will only deal correctly with integers in the 32-bit range), and
- maps true to 1, mapping all other results to 0.

Does what's going on now make more sense?

HTH,
Eliot


Bill

________________________________________
From: [hidden email] [[hidden email]] on behalf of Schwab,Wilhelm K [[hidden email]]
Sent: Thursday, March 01, 2012 12:15 AM
To: [hidden email]
Subject: Re: [Pharo-project] FFI on 1.3: compile fields

thanks!!!!!




________________________________________
From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
Sent: Wednesday, February 29, 2012 10:18 PM
To: [hidden email]
Subject: Re: [Pharo-project] FFI on 1.3: compile fields

you need to define a signature... not at home now, but wait 'til tomorrow and I'll send an example

best,
Esteban

El 29/02/2012, a las 11:21p.m., Schwab,Wilhelm K escribió:

> What does "cannot find callback signature" mean from #signature:block:?  I'm stuck.
>
>
>
>
> ________________________________________
> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
> Sent: Wednesday, February 29, 2012 6:54 PM
> To: [hidden email]
> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>
> yeah... sorry about that
> is a bug on FFI... I managed to worked around it for HPDF, for a customer's project... but of course is not a good and definitive solution.
>
> best,
> Esteban
>
> El 29/02/2012, a las 8:37p.m., Schwab,Wilhelm K escribió:
>
>> This is gonna take a while...   I had structs flying around as void* and was moderately happy.  Nonetheless, your suggestion worked, provided I add a lot getHandle asInteger and change the void* to long :(
>>
>>
>>
>>
>> ________________________________________
>> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
>> Sent: Wednesday, February 29, 2012 6:23 PM
>> To: [hidden email]
>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>
>> I also found some problems using void* in linux... maybe you want to use long (which has same size)... I "fixed" my problems that way.
>>
>> yes... maybe we need to look at FFI to see why void* has problems some times, but well, that can help you atm (sorry for not having a better answer)
>>
>> Esteban
>>
>> El 29/02/2012, a las 8:11p.m., Igor Stasenko escribió:
>>
>>> On 1 March 2012 00:58, Schwab,Wilhelm K <[hidden email]> wrote:
>>>> Another glitch: is there any problem passing things as void*?  I'm getting failure to coerce errors that did not arise before.
>>>>
>>> No idea. As you may suspect, i stopped using FFI/Alien once i got
>>> NativeBoost toy to play with.
>>>
>>> Please file the issue, describing the problem. so we can look over it
>>> and fix it.
>>>
>>>>
>>>>
>>>> ________________________________________
>>>> From: [hidden email] [[hidden email]] on behalf of Igor Stasenko [[hidden email]]
>>>> Sent: Wednesday, February 29, 2012 5:51 PM
>>>> To: [hidden email]
>>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>>>
>>>> On 1 March 2012 00:37, Schwab,Wilhelm K <[hidden email]> wrote:
>>>>> Does 1.3 by default not create field accessors?  Why is that?  I thought
>>>>> nothing was happening.
>>>>>
>>>>
>>>> Good question, i'd like to know the answer too.
>>>> It is related to MC final 'installation' phase,
>>>> where it initializing all classes.
>>>>
>>>> In NativeBoost i was also using
>>>>
>>>> noteCompilationOf: aSelector meta: isMeta
>>>>      "A hook allowing some classes to react to recompilation of certain selectors"
>>>>
>>>> but there was a big question, at which point this hook is triggered,
>>>> and looks like some changes in MC stop triggering it/triggering at
>>>> wrong time (not all methods get into a class/ class not initialized
>>>> etc),
>>>> which makes it not very useful.
>>>>
>>>> So i ended up creating a DNU handler on instance side, then on DNU i
>>>> check if i have field accessors and if not,
>>>> compile them on the fly.
>>>>
>>>>
>>>>> Bill
>>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Igor Stasenko.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko.
>>>
>>
>>
>>
>
>
>







--
best,
Eliot




--
best,
Eliot




--
best,
Eliot

Reply | Threaded
Open this post in threaded view
|

Re: Alien signature: first attempt

Eliot Miranda-2


On Thu, Mar 1, 2012 at 7:01 PM, Schwab,Wilhelm K <[hidden email]> wrote:
Eliot,

Thanks for the help.  I'm not as convinced as you that I have it<g>, but I'm getting there.
Alien subclasses per struct - interesting!

There are some Win32 examples in the Alien-Win32 package, LOGFONTA, ENUMLOGFONTEXA and OPENFILENAME.  But my favourite is the BochsIA32Alien the the Cog VMMaker.  It is an interface to Bochs' x86 processor simulator, written in C++, upon which the Cog JIT simulates its machine code when developing the Cog JIT in Smalltalk.  It has methods such as

eax
^self unsignedLongAt: 469

eax: anUnsignedInteger
^self unsignedLongAt: 469 put: anUnsignedInteger

and the accessing methods get generated via C code [ :) ]:

#define stoffsetof(type,field) (offsetof(type,field)+1)
#define print(r,n) \
printf("!BochsIA32Alien methodsFor: 'accessing' stamp: 'eem %d/%d/%d %d:%02d'!\r"\  
    "%s\r\t^self unsignedLongAt: %ld! !\r", m,d,y,h,i, lower(#r), \
    stoffsetof(BX_CPU_C,gen_reg[n].dword.erx));\
printf("!BochsIA32Alien methodsFor: 'accessing' stamp: 'eem %d/%d/%d %d:%02d'!\r"\  
    "%s: anUnsignedInteger\r\t^self unsignedLongAt: %ld put: anUnsignedInteger! !\r", m,d,y,h,i, lower(#r), \
    stoffsetof(BX_CPU_C,gen_reg[n].dword.erx))

    print(EAX,BX_32BIT_REG_EAX);
    print(EBX,BX_32BIT_REG_EBX);
    print(ECX,BX_32BIT_REG_ECX);
    print(EDX,BX_32BIT_REG_EDX);
    print(ESP,BX_32BIT_REG_ESP);
    print(EBP,BX_32BIT_REG_EBP);
    print(ESI,BX_32BIT_REG_ESI);
    print(EDI,BX_32BIT_REG_EDI);
    print(EIP,BX_32BIT_REG_EIP);

see <a href="http://www.squeakvm.org/svn/squeak/branches/Cog/processors/IA32/bochs/exploration/{printcpu.c,printcpucr.c,printcpuxmm.c}">http://www.squeakvm.org/svn/squeak/branches/Cog/processors/IA32/bochs/exploration/{printcpu.c,printcpucr.c,printcpuxmm.c}



 

Bill





From: [hidden email] [[hidden email]] on behalf of Eliot Miranda [[hidden email]]
Sent: Thursday, March 01, 2012 9:56 PM

To: [hidden email]
Subject: Re: [Pharo-project] Alien signature: first attempt



On Thu, Mar 1, 2012 at 6:45 PM, Schwab,Wilhelm K <[hidden email]> wrote:
Eliot,

I've unearthed some code from Dolphin, and might want to use something like this:

vectorStarVoidStarMatrixStarRetInt: callbackContext sp: spAlien
     <signature: 'int (*) (Gsl_vector *, void *, Gsl_matrix *)' abi: 'IA32'>
    ^callbackContext wordResult:(
        block
            value: (Alien forPointer:( spAlien unsignedLongAt: 1))
            value: (Alien forPointer: (spAlien unsignedLongAt: 5))
            value: (Alien forPointer:( spAlien unsignedLongAt: 9))
    ).

Am I at all on the right track?   The vector and matrix are structs.

Yes, exactly.  The next level is to create Alien subclasses for Gsl_vector and Gsl_matrix and use these in place of the simple Aliens.  These e.g. GslMatrixIA32Alien and GslVectorIA32Alien classes would define higher-level accessors so that your callback can access their members by name, not by offset.  Of course these classes should be auto-generated by parsing e.g. a C header file using the rules of the platform's ABI.

You've got it.



Bill





From: [hidden email] [[hidden email]] on behalf of Eliot Miranda [[hidden email]]
Sent: Thursday, March 01, 2012 5:38 PM

To: [hidden email]
Subject: Re: [Pharo-project] Alien signature: first attempt



On Thu, Mar 1, 2012 at 1:52 PM, Schwab,Wilhelm K <[hidden email]> wrote:
Eliot,

That helps a bit; it's still unfamiliar territory.  I am a little bothered by #wordResult: and #floatResult.  There are times when one is expected to return 16 bit values (rare) and either a float or a double (common).  Does that flexibility exist?

Yes, the flexibility exists.  But the mechanism needed to provide the flexibility is less than one might expect.  For example, on x86 there are three fundamental return conventions, one for integral values, one for floating-point values, and one for structure values.  See e.g. chapter 3 of the Sys V ABI for 386 (in my reply to Nicolas).  Read it; it's good for the soul; makes one value Smalltalk's simplicity.

Integral values are returned in %eax, %edx.  edx is not used if the value fits in 32-bits.  So the same sequence can be used to answer all integral values from 1 byte to 8 bytes, irrespective of sign.  That said, I use two sequences, one for 32-bits or less and one for 33 to 64 bits.  In your example of answering 16 bits one simply answers either the sign-extension of 16-bits (if signed) to 32-bits or the zero-extension of 16-bits to 32-bits (if unsigned).  In any case, the result is answered in %eax, and it is up to the caller to access the result correctly.

Float values are returned on the top of the floating point register stack (%st(0)).   The same machine-code sequence can be used to answer single, double and extended precision floats, since on the fp stack, they are all represented in the same extended format. 
 
So on x86 the callback implementation supports 4 return mechanisms, retword, retword64, retdouble and retstruct.  So far only two of these are fleshed out in Smalltalk code.  But look at http://www.squeakvm.org/svn/squeak/branches/Cog/src/vm/vmCallback.h &  http://squeakvm.org/svn/squeak/trunk/platforms/Cross/plugins/IA32ABI/ia32abicc.c and you'll see the other two.

cheers,
Eliot
 

Bill





From: [hidden email] [[hidden email]] on behalf of Eliot Miranda [[hidden email]]
Sent: Thursday, March 01, 2012 3:26 PM
To: [hidden email]
Subject: Re: [Pharo-project] Alien signature: first attempt



On Thu, Mar 1, 2012 at 10:57 AM, Schwab,Wilhelm K <[hidden email]> wrote:
Esteban,

Will this work?  Even if so, is there a better way to define it?

No it won't work, because you don't understand what it does.
 

longVoidStarLongRetInt: callbackContext sp: spAlien
       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
       ^callbackContext wordResult:
               (block
                       value: (Alien newC:4)
                       value: (Alien forPointer: (spAlien unsignedLongAt: 5))
                       value: (Alien newC:4)
               )

The above takes only one parameter's value from the spAlien (the void * value) and passes garbage for the other parameters. The spAlien is a pointer to the incoming C callback stack frame, i.e. a pointer to the callback's parameters.  So spAlien unsignedLongAt: 5 gets the four-byte pointer for the void * parameter, wraps it up in a convenient pointer Alien (so the callback block can e.g. indirect through it to get at the data the pointer is pointing to), and passes it as the second argument of the callback block.  But the two Alien newC: 4's are entirely bogus, passing effectively uninitialized data for the first and third parameters of the callback block.  They're just empty Aliens, when in fact you want to pass two longs to the block for the first and third parameters.

So to pass values in you need something like:

longVoidStarLongRetInt: callbackContext sp: spAlien
       <signature: 'int (*) (long, void *, long)' abi: 'IA32'>
       ^callbackContext wordResult:
               (block
                       value: (
spAlien signedLongAt: 1)
                       value: (Alien forPointer: (spAlien unsignedLongAt: 5))
                       value: (
spAlien signedLongAt: 9)
               )

You might also need to consider passing the result of the block back out.  If you only want to use it to answer integers then the above is fine.  But take a look at the intcharstarRetint:sp: signature:

intcharstarRetint: callbackContext sp: spAlien
<signature: 'int (*)(int, char *)' abi: 'IA32'>
^callbackContext wordResult:
((block
value: (spAlien signedLongAt: 1) ~= 0
value: (Alien forPointer: (spAlien unsignedLongAt: 5)) strcpyUTF8)
ifNil: [0]
ifNotNil:
[:result|
result isInteger
ifTrue: [result]
ifFalse: [result == true ifTrue: [1] ifFalse: [0]]])

Analogously it passes a signed integer as the first argument and a UTF8 string derived from the second char * parameter. (The method should probably be called intcharstarasUTF8Retint:sp: or some such; forgive me, we're getting there).

But on return it takes the result of the callback block and passes it to the ifNil:ifNotNil: which
- maps a nil result form the block to 0 (ifNil: [0])
- maps integers to integers (which will only deal correctly with integers in the 32-bit range), and
- maps true to 1, mapping all other results to 0.

Does what's going on now make more sense?

HTH,
Eliot


Bill

________________________________________
From: [hidden email] [[hidden email]] on behalf of Schwab,Wilhelm K [[hidden email]]
Sent: Thursday, March 01, 2012 12:15 AM
To: [hidden email]
Subject: Re: [Pharo-project] FFI on 1.3: compile fields

thanks!!!!!




________________________________________
From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
Sent: Wednesday, February 29, 2012 10:18 PM
To: [hidden email]
Subject: Re: [Pharo-project] FFI on 1.3: compile fields

you need to define a signature... not at home now, but wait 'til tomorrow and I'll send an example

best,
Esteban

El 29/02/2012, a las 11:21p.m., Schwab,Wilhelm K escribió:

> What does "cannot find callback signature" mean from #signature:block:?  I'm stuck.
>
>
>
>
> ________________________________________
> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
> Sent: Wednesday, February 29, 2012 6:54 PM
> To: [hidden email]
> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>
> yeah... sorry about that
> is a bug on FFI... I managed to worked around it for HPDF, for a customer's project... but of course is not a good and definitive solution.
>
> best,
> Esteban
>
> El 29/02/2012, a las 8:37p.m., Schwab,Wilhelm K escribió:
>
>> This is gonna take a while...   I had structs flying around as void* and was moderately happy.  Nonetheless, your suggestion worked, provided I add a lot getHandle asInteger and change the void* to long :(
>>
>>
>>
>>
>> ________________________________________
>> From: [hidden email] [[hidden email]] on behalf of Esteban Lorenzano [[hidden email]]
>> Sent: Wednesday, February 29, 2012 6:23 PM
>> To: [hidden email]
>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>
>> I also found some problems using void* in linux... maybe you want to use long (which has same size)... I "fixed" my problems that way.
>>
>> yes... maybe we need to look at FFI to see why void* has problems some times, but well, that can help you atm (sorry for not having a better answer)
>>
>> Esteban
>>
>> El 29/02/2012, a las 8:11p.m., Igor Stasenko escribió:
>>
>>> On 1 March 2012 00:58, Schwab,Wilhelm K <[hidden email]> wrote:
>>>> Another glitch: is there any problem passing things as void*?  I'm getting failure to coerce errors that did not arise before.
>>>>
>>> No idea. As you may suspect, i stopped using FFI/Alien once i got
>>> NativeBoost toy to play with.
>>>
>>> Please file the issue, describing the problem. so we can look over it
>>> and fix it.
>>>
>>>>
>>>>
>>>> ________________________________________
>>>> From: [hidden email] [[hidden email]] on behalf of Igor Stasenko [[hidden email]]
>>>> Sent: Wednesday, February 29, 2012 5:51 PM
>>>> To: [hidden email]
>>>> Subject: Re: [Pharo-project] FFI on 1.3: compile fields
>>>>
>>>> On 1 March 2012 00:37, Schwab,Wilhelm K <[hidden email]> wrote:
>>>>> Does 1.3 by default not create field accessors?  Why is that?  I thought
>>>>> nothing was happening.
>>>>>
>>>>
>>>> Good question, i'd like to know the answer too.
>>>> It is related to MC final 'installation' phase,
>>>> where it initializing all classes.
>>>>
>>>> In NativeBoost i was also using
>>>>
>>>> noteCompilationOf: aSelector meta: isMeta
>>>>      "A hook allowing some classes to react to recompilation of certain selectors"
>>>>
>>>> but there was a big question, at which point this hook is triggered,
>>>> and looks like some changes in MC stop triggering it/triggering at
>>>> wrong time (not all methods get into a class/ class not initialized
>>>> etc),
>>>> which makes it not very useful.
>>>>
>>>> So i ended up creating a DNU handler on instance side, then on DNU i
>>>> check if i have field accessors and if not,
>>>> compile them on the fly.
>>>>
>>>>
>>>>> Bill
>>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Igor Stasenko.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko.
>>>
>>
>>
>>
>
>
>







--
best,
Eliot




--
best,
Eliot




--
best,
Eliot




--
best,
Eliot