NativeBoost and variadic functions

Hello,

Is it possible with NativeBoost to create a binding for a variadic function ?

I've seen the printf example in NBCPrinter but this implementation is kind of cheating since it always pass just a %s as format and one already formatted string to the C function.

I've written a simple variadic function which adds every integer it receives as argument (first argument is for the number of following arguments) :

int add(int number,...);

In Pharo I've tried something like this :

add: number arg1: first arg2: second
<primitive: #primitiveNativeCall module: #NativeBoostPlugin>

^ self nbCall: #( int add (int number, int first, int second))
  module: 'libMyLib.so'

and it works fine with two arguments.

Basically, doing so, I would need one method per number of arguments so it's not very cool.

I thought that maybe I could pass an array as argument to my Pharo method but I didn't really find a way to figure out how to define the nbCall without having a "Generic failure".

add: number args: anArray
<primitive: #primitiveNativeCall module: #NativeBoostPlugin>

^ self nbCall: #( int add (int number, ??? anArray))
  module: 'libMyLib.so'

Do you have an idea ?

Thanks,

Matthieu

On Wed, Jul 15, 2015 at 5:16 PM, Matthieu Lacaton
<[hidden email]> wrote:
> Hello Igor,
>
> Thanks for your answer.
>
> I implemented something like that for the printf function:
> Basically, it generates a method with matching arguments and executes it.
>
>> printf: stringFormat args: tab
>>
>>     | argNumber functionArgs functionPrototype methodCorpse methodSelector
>> argsArray |
>>
>>     ((tab size % 2) = 0) ifFalse: [
>>         Transcript show: 'error'.
>>         ^self.
>>     ].
>>
>>     argNumber := 0.
>>     functionPrototype := 'printf: stringFormat'.
>>     functionArgs := ''.
>>     methodCorpse := ''.
>>     argsArray := (Array new: (tab size / 2) + 1).
>>     argsArray at: 1 put: stringFormat.
>>
>>     1 to: tab size by: 2 do: [ :i |
>>         functionPrototype := functionPrototype, ' arg', argNumber
>> asString, ': ', (tab at: i) asString, argNumber asString.
>>         functionArgs := functionArgs, ' ', (tab at: i) asString, ' ', (tab
>> at: i) asString, argNumber asString, ','.
>>         argsArray at: argNumber + 2 put: (tab at: i + 1).
>>         argNumber := argNumber + 1.
>>     ].
>>     functionArgs := functionArgs allButLast.
>>
>>     methodCorpse := functionPrototype, Character cr asString, '
>> <primitive: #primitiveNativeCall module: #NativeBoostPlugin>', Character cr
>> asString, Character cr asString, '    ^self nbCall: #( void printf ( String
>> stringFormat,', functionArgs asString, ' ) )', Character cr asString, '
>> module: NativeBoost CLibrary'.
>>
>>     methodSelector := self class compile: methodCorpse.
>>
>>     self perform: methodSelector withArguments: argsArray.
>
>
>
> Then you can call it like that :
>
> MyClass printf: 'Test of printf. String: %s, Int : %d, Long: %ld, Char: %c,
> Double: %lf' args: { 'String'. 'This is a string'. 'int'. 100. 'long'.
> 10000000. 'char'. 89. 'double'. 3.14159 }.
>
>
> I also tried it for some other variadic functions and, on my computer (I am
> running archlinux), it seemed to work for every type of argument except
> "float".

Maybe NativeBoost does not do implicit type conversions that a C
compiler would do?

[1] "Because C will promote floats to doubles for functions that take
variable arguments"

More info search [2] for:
*  "implicit type conversion"
*  "guess wrongly when the program uses floating-point formats in
scanf() or printf()"

[1] http://stackoverflow.com/questions/210590/why-does-scanf-need-lf-for-doubles-when-printf-is-okay-with-just-f

[2] http://www.electroons.com/8051/ebooks/expert%20C%20programming.pdf

cheers -ben


> It works fine for "double" though.
> For "char" you need to pass the integer ASCII value directly for it to work.
> I tried with "Character value: xxx" but it didn't work.
>
> I know that this is very hackish and very bad, and I am aware it has some
> drawbacks. Moreover I am not even sure it will work everytime.
> But for now it seems to work ...
>
> 2015-07-13 19:24 GMT+02:00 Igor Stasenko <[hidden email]>:
>>
>>
>>
>> On 10 July 2015 at 10:18, Matthieu Lacaton <[hidden email]>

>> wrote:
>>>
>>> Hello,
>>>
>>> Is it possible with NativeBoost to create a binding for a variadic
>>> function ?
>>>
>>> I've seen the printf example in NBCPrinter but this implementation is
>>> kind of cheating since it always pass just a %s as format and one already
>>> formatted string to the C function.
>>>
>>> I've written a simple variadic function which adds every integer it
>>> receives as argument (first argument is for the number of following
>>> arguments) :
>>>
>>> int add(int number,...);
>>>
>>> In Pharo I've tried something like this :
>>>
>>>> add: number arg1: first arg2: second
>>>> <primitive: #primitiveNativeCall module: #NativeBoostPlugin>
>>>>
>>>> ^ self nbCall: #( int add (int number, int first, int second))
>>>>   module: 'libMyLib.so'
>>>
>>>
>>> and it works fine with two arguments.
>>>
>>> Basically, doing so, I would need one method per number of arguments so
>>> it's not very cool.
>>>
>>> I thought that maybe I could pass an array as argument to my Pharo method
>>> but I didn't really find a way to figure out how to define the nbCall
>>> without having a "Generic failure".
>>>
>>>> add: number args: anArray
>>>> <primitive: #primitiveNativeCall module: #NativeBoostPlugin>
>>>>
>>>> ^ self nbCall: #( int add (int number, ??? anArray))
>>>>   module: 'libMyLib.so'
>>>
>>>
>>> Do you have an idea ?
>>>
>>
>> In short, there's no marshaller for converting an array of items, to same
>> number of things on stack. That could solve the problem with your example:
>> passing array of objects of *same* type. But in general, it is not what C
>> variadic function(s) standing for. Because they stand for any number of
>> arguments, of any type.
>> In C, since all program is compiled statically, compiler knows the number
>> of passed arguments and their types through compiling each particular call
>> site(s) to variadic function. Which means that in fact, you are still
>> supplying all information needed by compiler *before* run time.
>>
>> In variadic functions, you can pass any arguments of any type,
>> but for converting each of them, you must tell NB what kind of  marshaller
>> should be used for it , which means, that it is impossible to know before
>> run time, since you cannot know how many arguments you may pass, not
>> speaking about their types.
>>
>>
>>
>>>
>>> Thanks,
>>>
>>> Matthieu
>>>
>>>
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko.
>
>

Hello Igor,

Thanks for your answer.

I implemented something like that for the printf function:

Basically, it generates a method with matching arguments and executes it.

printf: stringFormat args: tab

    | argNumber functionArgs functionPrototype methodCorpse methodSelector argsArray |
   
    ((tab size % 2) = 0) ifFalse: [
        Transcript show: 'error'.
        ^self.
    ].

    argNumber := 0.
    functionPrototype := 'printf: stringFormat'.
    functionArgs := ''.
    methodCorpse := ''.
    argsArray := (Array new: (tab size / 2) + 1).
    argsArray at: 1 put: stringFormat.
   
    1 to: tab size by: 2 do: [ :i |
        functionPrototype := functionPrototype, ' arg', argNumber asString, ': ', (tab at: i) asString, argNumber asString.
        functionArgs := functionArgs, ' ', (tab at: i) asString, ' ', (tab at: i) asString, argNumber asString, ','.
        argsArray at: argNumber + 2 put: (tab at: i + 1).
        argNumber := argNumber + 1.
    ].
    functionArgs := functionArgs allButLast.
   
    methodCorpse := functionPrototype, Character cr asString, '    <primitive: #primitiveNativeCall module: #NativeBoostPlugin>', Character cr asString, Character cr asString, '    ^self nbCall: #( void printf ( String stringFormat,', functionArgs asString, ' ) )', Character cr asString, '        module: NativeBoost CLibrary'.

    methodSelector := self class compile: methodCorpse.
   
    self perform: methodSelector withArguments: argsArray.

Then you can call it like that :

MyClass printf: 'Test of printf. String: %s, Int : %d, Long: %ld, Char: %c, Double: %lf' args: { 'String'. 'This is a string'. 'int'. 100. 'long'. 10000000. 'char'. 89. 'double'. 3.14159 }.

I also tried it for some other variadic functions and, on my computer (I am running archlinux), it seemed to work for every type of argument except "float". It works fine for "double" though.

For "char" you need to pass the integer ASCII value directly for it to work. I tried with "Character value: xxx" but it didn't work.

I know that this is very hackish and very bad, and I am aware it has some drawbacks. Moreover I am not even sure it will work everytime.

But for now it seems to work ...

2015-07-13 19:24 GMT+02:00 Igor Stasenko <[hidden email]>:

On 10 July 2015 at 10:18, Matthieu Lacaton <[hidden email]> wrote:

Hello,

Is it possible with NativeBoost to create a binding for a variadic function ?

I've seen the printf example in NBCPrinter but this implementation is kind of cheating since it always pass just a %s as format and one already formatted string to the C function.

I've written a simple variadic function which adds every integer it receives as argument (first argument is for the number of following arguments) :

int add(int number,...);

In Pharo I've tried something like this :

add: number arg1: first arg2: second
<primitive: #primitiveNativeCall module: #NativeBoostPlugin>

^ self nbCall: #( int add (int number, int first, int second))
  module: 'libMyLib.so'

and it works fine with two arguments.

Basically, doing so, I would need one method per number of arguments so it's not very cool.

I thought that maybe I could pass an array as argument to my Pharo method but I didn't really find a way to figure out how to define the nbCall without having a "Generic failure".

add: number args: anArray
<primitive: #primitiveNativeCall module: #NativeBoostPlugin>

^ self nbCall: #( int add (int number, ??? anArray))
  module: 'libMyLib.so'

Do you have an idea ?

In short, there's no marshaller for converting an array of items, to same number of things on stack. That could solve the problem with your example: passing array of objects of *same* type. But in general, it is not what C variadic function(s) standing for. Because they stand for any number of arguments, of any type.

In C, since all program is compiled statically, compiler knows the number of passed arguments and their types through compiling each particular call site(s) to variadic function. Which means that in fact, you are still supplying all information needed by compiler *before* run time.

In variadic functions, you can pass any arguments of any type,

but for converting each of them, you must tell NB what kind of  marshaller should be used for it , which means, that it is impossible to know before run time, since you cannot know how many arguments you may pass, not speaking about their types.

 

Thanks,

Matthieu

--

Best regards,
Igor Stasenko.

>
> And let me remind you that despite that NB implements FFI to speak with C,
> it is not obliged to implement features of C language itself. It lets you
> speak with C programs, but not lets you write programs like in C (see the
> difference? :)

I wasn't implying that implicit type conversion was a good thing that
needed implementing -- but just a bump if it hadn't needed attention
to it before when C did it automagically.
cheers -ben