Smalltalk › Pharo › Pharo Smalltalk Developers

Pointer types

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

10 messages Options

Schwab,Wilhelm K

Pointer types

Sig,

You mentioned something that might be useful, depending on where it lives. Our ability to use underscores now leaves me thinking I really should rework some things that suffered mightily: there are some external things that are named assuming underscores, and in mixed case, they turn into an unreadable mess. Along with this, I have a large number of FFI calls that take double pointers, and I have for some time been demoralized into (or stupid enough to<g>) expressing them as void pointers to allow me to pass byte arrays into them.

Is there a way that I can get FFI to recognize DOUBLEArray (which uses a ByteArray to hold its data) as something that should be acceptable for a double* call? If not FFI, how does NB handle it?

Bill

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Igor Stasenko

Re: Pointer types

On 4 October 2010 06:15, Schwab,Wilhelm K <[hidden email]> wrote:
> Sig,
>
> You mentioned something that might be useful, depending on where it lives. Our ability to use underscores now leaves me thinking I really should rework some things that suffered mightily: there are some external things that are named assuming underscores, and in mixed case, they turn into an unreadable mess. Along with this, I have a large number of FFI calls that take double pointers, and I have for some time been demoralized into (or stupid enough to<g>) expressing them as void pointers to allow me to pass byte arrays into them.
>
> Is there a way that I can get FFI to recognize DOUBLEArray (which uses a ByteArray to hold its data) as something that should be acceptable for a double* call? If not FFI, how does NB handle it?
>

In NB, there is a type system, which implements C<->Smalltalk coercions.
The responsibility of type coercions taken by a subclasses of NBExternalType,
so NB callout code generator actually knows nothing about types and
how to coerce them.

To create own custom type, you need to subclass from NBExternalType
and override methods which responsible for coercions.
There you can define any kind of operations , what needs to be done in
order to push value on stack, or
convert it to some smalltalk object.

Then, in your DoubleArray class, simply implement a message on a class side:

asNBExternalType: gen

^ MyDoubleArrayExternalType new

And then, in FFI callouts you can use a class name directly:

DoubleArray foo (DoubleArray arr, DoubleArray * arrPtr)

etc.

Or, if you want make double* type to use it as a DoubleArray, then you
can add an alias:

MyClass>>externalTypeAlias: aTypeName
aTypeName = 'double*' ifTrue: [ ^ #DoubleArray ].
^ nil

Then, any method which contains an FFI callout in MyClass,
will use that alias. So, you can write callout as:

double* foo (double *arr, double ** arrPtrPtr)

but it will be understood as:

DoubleArray foo (DoubleArray arr, DoubleArray * arrPtr)

About handling pointers.. well for smalltalk there is no pointers, just objects.
So, in NB NBExternalType keeps two things:
- a value type
- a pointer arity

So, for example, when you specifying type, like:
char ***

a value type will be char
and pointer arity will be 3.

It is up to NBExternalType subclass how to push values whose pointer arity > 0.
By default, if pointerArity > 0, then it is treated as a pointer,
which can be either: - nil, ByteArray, or NBExternalAddress

> Bill
>
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Schwab,Wilhelm K

Re: Pointer types

Sig,

Am I correct in taking this as a description of how to solve it with NB, not Squeak/Pharo FFI? That's ok, just want to know if there is an applicable lesson on FFI to be extracted.

You say the NB callout generator knows nothing about types. Is that how it should be? The scenario I have in mind is a bogus type and whether that can/should be flagged at compile time vs. runtime. Maybe all it takes is a scan for a suitable handler before or after the method is compiled. I ask because I hit a related snag with FFI runtime errors being reported as coercion errors when in fact I had used errant types in the callout definitions. IIRC, Andreas patched the compiler to report the bogus types.

"Or, if you want make double* type to use it as a DoubleArray, then you can add an alias:

MyClass>>externalTypeAlias: aTypeName
aTypeName = 'double*' ifTrue: [ ^ #DoubleArray ].
^ nil

Then, any method which contains an FFI callout in MyClass, will use that alias."

What is MyClass in the above? Is it the class holding the callout method, or it elsewhere in the chain? I ask because it could be a sore spot for maintenance/pluggability. I'm wondering how you and I would independently add such instructions. If it goes in the external type subclass, then it is probably fine; if it goes in a class that makes the call, we might stomp on each other. Make sense? My hunch is that you have it handled.

Bill

________________________________________
From: [hidden email] [[hidden email]] On Behalf Of Igor Stasenko [[hidden email]]
Sent: Monday, October 04, 2010 7:36 AM
To: [hidden email]
Subject: Re: [Pharo-project] Pointer types

On 4 October 2010 06:15, Schwab,Wilhelm K <[hidden email]> wrote:
> Sig,
>
> You mentioned something that might be useful, depending on where it lives. Our ability to use underscores now leaves me thinking I really should rework some things that suffered mightily: there are some external things that are named assuming underscores, and in mixed case, they turn into an unreadable mess. Along with this, I have a large number of FFI calls that take double pointers, and I have for some time been demoralized into (or stupid enough to<g>) expressing them as void pointers to allow me to pass byte arrays into them.
>
> Is there a way that I can get FFI to recognize DOUBLEArray (which uses a ByteArray to hold its data) as something that should be acceptable for a double* call? If not FFI, how does NB handle it?
>

In NB, there is a type system, which implements C<->Smalltalk coercions.
The responsibility of type coercions taken by a subclasses of NBExternalType,
so NB callout code generator actually knows nothing about types and
how to coerce them.

To create own custom type, you need to subclass from NBExternalType
and override methods which responsible for coercions.
There you can define any kind of operations , what needs to be done in
order to push value on stack, or
convert it to some smalltalk object.

Then, in your DoubleArray class, simply implement a message on a class side:

asNBExternalType: gen

^ MyDoubleArrayExternalType new

And then, in FFI callouts you can use a class name directly:

DoubleArray foo (DoubleArray arr, DoubleArray * arrPtr)

etc.

Or, if you want make double* type to use it as a DoubleArray, then you
can add an alias:

MyClass>>externalTypeAlias: aTypeName
aTypeName = 'double*' ifTrue: [ ^ #DoubleArray ].
^ nil

Then, any method which contains an FFI callout in MyClass,
will use that alias. So, you can write callout as:

double* foo (double *arr, double ** arrPtrPtr)

but it will be understood as:

DoubleArray foo (DoubleArray arr, DoubleArray * arrPtr)

About handling pointers.. well for smalltalk there is no pointers, just objects.
So, in NB NBExternalType keeps two things:
- a value type
- a pointer arity

So, for example, when you specifying type, like:
char ***

a value type will be char
and pointer arity will be 3.

It is up to NBExternalType subclass how to push values whose pointer arity > 0.
By default, if pointerArity > 0, then it is treated as a pointer,
which can be either: - nil, ByteArray, or NBExternalAddress

> Bill
>
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Igor Stasenko

Re: Pointer types

On 7 October 2010 01:27, Schwab,Wilhelm K <[hidden email]> wrote:

> Sig,
>
> Am I correct in taking this as a description of how to solve it with NB, not Squeak/Pharo FFI? That's ok, just want to know if there is an applicable lesson on FFI to be extracted.
>
> You say the NB callout generator knows nothing about types. Is that how it should be? The scenario I have in mind is a bogus type and whether that can/should be flagged at compile time vs. runtime. Maybe all it takes is a scan for a suitable handler before or after the method is compiled. I ask because I hit a related snag with FFI runtime errors being reported as coercion errors when in fact I had used errant types in the callout definitions. IIRC, Andreas patched the compiler to report the bogus types.
>
>
> "Or, if you want make double* type to use it as a DoubleArray, then you can add an alias:
>
> MyClass>>externalTypeAlias: aTypeName
> aTypeName = 'double*' ifTrue: [ ^ #DoubleArray ].
> ^ nil
>
> Then, any method which contains an FFI callout in MyClass, will use that alias."
>
> What is MyClass in the above? Is it the class holding the callout method, or it elsewhere in the chain? I ask because it could be a sore spot for maintenance/pluggability. I'm wondering how you and I would independently add such instructions. If it goes in the external type subclass, then it is probably fine; if it goes in a class that makes the call, we might stomp on each other. Make sense? My hunch is that you have it handled.
>

Its in an any class, which using FFI callouts. We can stomp on each
other if you define aliases in Object class :)
But this won't be a good idea, isnt?
NBExternalType subclasses defining coercions between ST oops and
concrete C type, they are not for defining aliases.

>
> Bill
>
>
> ________________________________________
> From: [hidden email] [[hidden email]] On Behalf Of Igor Stasenko [[hidden email]]
> Sent: Monday, October 04, 2010 7:36 AM
> To: [hidden email]
> Subject: Re: [Pharo-project] Pointer types
>
> On 4 October 2010 06:15, Schwab,Wilhelm K <[hidden email]> wrote:
>> Sig,
>>
>> You mentioned something that might be useful, depending on where it lives. Our ability to use underscores now leaves me thinking I really should rework some things that suffered mightily: there are some external things that are named assuming underscores, and in mixed case, they turn into an unreadable mess. Along with this, I have a large number of FFI calls that take double pointers, and I have for some time been demoralized into (or stupid enough to<g>) expressing them as void pointers to allow me to pass byte arrays into them.
>>
>> Is there a way that I can get FFI to recognize DOUBLEArray (which uses a ByteArray to hold its data) as something that should be acceptable for a double* call? If not FFI, how does NB handle it?
>>
>
> In NB, there is a type system, which implements C<->Smalltalk coercions.
> The responsibility of type coercions taken by a subclasses of NBExternalType,
> so NB callout code generator actually knows nothing about types and
> how to coerce them.
>
> To create own custom type, you need to subclass from NBExternalType
> and override methods which responsible for coercions.
> There you can define any kind of operations , what needs to be done in
> order to push value on stack, or
> convert it to some smalltalk object.
>
> Then, in your DoubleArray class, simply implement a message on a class side:
>
> asNBExternalType: gen
>
> ^ MyDoubleArrayExternalType new
>
>
> And then, in FFI callouts you can use a class name directly:
>
> DoubleArray foo (DoubleArray arr, DoubleArray * arrPtr)
>
> etc.
>
> Or, if you want make double* type to use it as a DoubleArray, then you
> can add an alias:
>
> MyClass>>externalTypeAlias: aTypeName
> aTypeName = 'double*' ifTrue: [ ^ #DoubleArray ].
> ^ nil
>
> Then, any method which contains an FFI callout in MyClass,
> will use that alias. So, you can write callout as:
>
> double* foo (double *arr, double ** arrPtrPtr)
>
> but it will be understood as:
>
> DoubleArray foo (DoubleArray arr, DoubleArray * arrPtr)
>
>
> About handling pointers.. well for smalltalk there is no pointers, just objects.
> So, in NB NBExternalType keeps two things:
> - a value type
> - a pointer arity
>
>
> So, for example, when you specifying type, like:
> char ***
>
> a value type will be char
> and pointer arity will be 3.
>
> It is up to NBExternalType subclass how to push values whose pointer arity > 0.
> By default, if pointerArity > 0, then it is treated as a pointer,
> which can be either: - nil, ByteArray, or NBExternalAddress
>
>
>> Bill
>>
>>
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Schwab,Wilhelm K

Re: Pointer types

Sig,

Ultimately, I will have to simply try this, but it reminds me of something that I can't place. Have you considered using a dictionary or other type of collection to store the types? A new handler would then be created and finally hooked into the system by sending a message to connect it; opposite on removal. It might turn out to be very much one-to-one so that what I am wondering about does not apply.

Anything that can be double* can also be used with void*. How does that work?

Bill

________________________________________
From: [hidden email] [[hidden email]] On Behalf Of Igor Stasenko [[hidden email]]
Sent: Thursday, October 07, 2010 12:41 AM
To: [hidden email]
Subject: Re: [Pharo-project] Pointer types

On 7 October 2010 01:27, Schwab,Wilhelm K <[hidden email]> wrote:

--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Igor Stasenko

Re: Pointer types

On 7 October 2010 13:35, Schwab,Wilhelm K <[hidden email]> wrote:
> Sig,
>
> Ultimately, I will have to simply try this, but it reminds me of something that I can't place. Have you considered using a dictionary or other type of collection to store the types? A new handler would then be created and finally hooked into the system by sending a message to connect it; opposite on removal. It might turn out to be very much one-to-one so that what I am wondering about does not apply.
>

Sure, you can use dictionaries in alias lookup implementation:

MyClass>>externalTypeAlias: aTypeName
^ myAliasesDict at: aTypeName ifAbsent: [ nil ]

> Anything that can be double* can also be used with void*. How does that work?
>

Pointer is a pointer. If function expects a pointer argument, you can
pass any pointer to it.
Code generator just checks that an argument you passes can be
converted to C pointer (bytearray, wordarray, external pointer),
but sure thing it can't check if you passed char* or double* or void*.
Such kind of type checking belongs to C compiler :)

So, a pointer types in FFI callout declarations is more like a hint,
what function expects, rather than something ,which
influence the generated callout code.

> Bill
>
>
>
>
>
> ________________________________________
> From: [hidden email] [[hidden email]] On Behalf Of Igor Stasenko [[hidden email]]
> Sent: Thursday, October 07, 2010 12:41 AM
> To: [hidden email]
> Subject: Re: [Pharo-project] Pointer types
>
> On 7 October 2010 01:27, Schwab,Wilhelm K <[hidden email]> wrote:
>> Sig,
>>
>> Am I correct in taking this as a description of how to solve it with NB, not Squeak/Pharo FFI? That's ok, just want to know if there is an applicable lesson on FFI to be extracted.
>>
>> You say the NB callout generator knows nothing about types. Is that how it should be? The scenario I have in mind is a bogus type and whether that can/should be flagged at compile time vs. runtime. Maybe all it takes is a scan for a suitable handler before or after the method is compiled. I ask because I hit a related snag with FFI runtime errors being reported as coercion errors when in fact I had used errant types in the callout definitions. IIRC, Andreas patched the compiler to report the bogus types.
>>
>>
>> "Or, if you want make double* type to use it as a DoubleArray, then you can add an alias:
>>
>> MyClass>>externalTypeAlias: aTypeName
>> aTypeName = 'double*' ifTrue: [ ^ #DoubleArray ].
>> ^ nil
>>
>> Then, any method which contains an FFI callout in MyClass, will use that alias."
>>
>> What is MyClass in the above? Is it the class holding the callout method, or it elsewhere in the chain? I ask because it could be a sore spot for maintenance/pluggability. I'm wondering how you and I would independently add such instructions. If it goes in the external type subclass, then it is probably fine; if it goes in a class that makes the call, we might stomp on each other. Make sense? My hunch is that you have it handled.
>>
>
> Its in an any class, which using FFI callouts. We can stomp on each
> other if you define aliases in Object class :)
> But this won't be a good idea, isnt?
> NBExternalType subclasses defining coercions between ST oops and
> concrete C type, they are not for defining aliases.
>
>>
>> Bill
>>
>>
>> ________________________________________
>> From: [hidden email] [[hidden email]] On Behalf Of Igor Stasenko [[hidden email]]
>> Sent: Monday, October 04, 2010 7:36 AM
>> To: [hidden email]
>> Subject: Re: [Pharo-project] Pointer types
>>
>> On 4 October 2010 06:15, Schwab,Wilhelm K <[hidden email]> wrote:
>>> Sig,
>>>
>>> You mentioned something that might be useful, depending on where it lives. Our ability to use underscores now leaves me thinking I really should rework some things that suffered mightily: there are some external things that are named assuming underscores, and in mixed case, they turn into an unreadable mess. Along with this, I have a large number of FFI calls that take double pointers, and I have for some time been demoralized into (or stupid enough to<g>) expressing them as void pointers to allow me to pass byte arrays into them.
>>>
>>> Is there a way that I can get FFI to recognize DOUBLEArray (which uses a ByteArray to hold its data) as something that should be acceptable for a double* call? If not FFI, how does NB handle it?
>>>
>>
>> In NB, there is a type system, which implements C<->Smalltalk coercions.
>> The responsibility of type coercions taken by a subclasses of NBExternalType,
>> so NB callout code generator actually knows nothing about types and
>> how to coerce them.
>>
>> To create own custom type, you need to subclass from NBExternalType
>> and override methods which responsible for coercions.
>> There you can define any kind of operations , what needs to be done in
>> order to push value on stack, or
>> convert it to some smalltalk object.
>>
>> Then, in your DoubleArray class, simply implement a message on a class side:
>>
>> asNBExternalType: gen
>>
>> ^ MyDoubleArrayExternalType new
>>
>>
>> And then, in FFI callouts you can use a class name directly:
>>
>> DoubleArray foo (DoubleArray arr, DoubleArray * arrPtr)
>>
>> etc.
>>
>> Or, if you want make double* type to use it as a DoubleArray, then you
>> can add an alias:
>>
>> MyClass>>externalTypeAlias: aTypeName
>> aTypeName = 'double*' ifTrue: [ ^ #DoubleArray ].
>> ^ nil
>>
>> Then, any method which contains an FFI callout in MyClass,
>> will use that alias. So, you can write callout as:
>>
>> double* foo (double *arr, double ** arrPtrPtr)
>>
>> but it will be understood as:
>>
>> DoubleArray foo (DoubleArray arr, DoubleArray * arrPtr)
>>
>>
>> About handling pointers.. well for smalltalk there is no pointers, just objects.
>> So, in NB NBExternalType keeps two things:
>> - a value type
>> - a pointer arity
>>
>>
>> So, for example, when you specifying type, like:
>> char ***
>>
>> a value type will be char
>> and pointer arity will be 3.
>>
>> It is up to NBExternalType subclass how to push values whose pointer arity > 0.
>> By default, if pointerArity > 0, then it is treated as a pointer,
>> which can be either: - nil, ByteArray, or NBExternalAddress
>>
>>
>>> Bill
>>>
>>>
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Schwab,Wilhelm K

Re: Pointer types

Sig,

I have to start watching the clock here, so I will just throw this out for your consideration. Yes a pointer is a pointer, but there are layers of specificity. If a function expects a double*, ideally passing a FloatArray would "take some work," perhaps in the form of wrapping the argument in a "cast"/adpater of some type and passing that to the function. If I describe an argument to the function as void*, then I do not expect much help. In fact, this is a growing source of trouble for me, because my DOUBLEArray leads me to describe parameters as void* when they should/could be double*. The current FFI *appears* to be helping in other situations by complaining when type A does not fit in slot B.

Clearly if the function is mapped as the wrong type in the code I give you, then the seg fault is on me. But it would be nice if when given a correct (double*, long) argument sequence, you could raise an error if those are reversed. Am I missing something? I gotta go =:0

Bill

________________________________________
From: [hidden email] [[hidden email]] On Behalf Of Igor Stasenko [[hidden email]]
Sent: Thursday, October 07, 2010 6:49 AM
To: [hidden email]
Subject: Re: [Pharo-project] Pointer types

On 7 October 2010 13:35, Schwab,Wilhelm K <[hidden email]> wrote:
> Sig,
>
> Ultimately, I will have to simply try this, but it reminds me of something that I can't place. Have you considered using a dictionary or other type of collection to store the types? A new handler would then be created and finally hooked into the system by sending a message to connect it; opposite on removal. It might turn out to be very much one-to-one so that what I am wondering about does not apply.
>

Sure, you can use dictionaries in alias lookup implementation:

MyClass>>externalTypeAlias: aTypeName
^ myAliasesDict at: aTypeName ifAbsent: [ nil ]

> Anything that can be double* can also be used with void*. How does that work?
>

Pointer is a pointer. If function expects a pointer argument, you can
pass any pointer to it.
Code generator just checks that an argument you passes can be
converted to C pointer (bytearray, wordarray, external pointer),
but sure thing it can't check if you passed char* or double* or void*.
Such kind of type checking belongs to C compiler :)

So, a pointer types in FFI callout declarations is more like a hint,
what function expects, rather than something ,which
influence the generated callout code.

Igor Stasenko

Re: Pointer types

On 7 October 2010 14:09, Schwab,Wilhelm K <[hidden email]> wrote:
> Sig,
>
> I have to start watching the clock here, so I will just throw this out for your consideration. Yes a pointer is a pointer, but there are layers of specificity. If a function expects a double*, ideally passing a FloatArray would "take some work," perhaps in the form of wrapping the argument in a "cast"/adpater of some type and passing that to the function. If I describe an argument to the function as void*, then I do not expect much help. In fact, this is a growing source of trouble for me, because my DOUBLEArray leads me to describe parameters as void* when they should/could be double*. The current FFI *appears* to be helping in other situations by complaining when type A does not fit in slot B.
>
> Clearly if the function is mapped as the wrong type in the code I give you, then the seg fault is on me. But it would be nice if when given a correct (double*, long) argument sequence, you could raise an error if those are reversed. Am I missing something? I gotta go =:0

You could even specify that a double* argument expects an argument to
be an instance of DoubleArray, which then will be coerced to pointer.
There's many other things possible, but before you start using it,
you'll never know :)

>
> Bill
>
>
>
> ________________________________________
> From: [hidden email] [[hidden email]] On Behalf Of Igor Stasenko [[hidden email]]
> Sent: Thursday, October 07, 2010 6:49 AM
> To: [hidden email]
> Subject: Re: [Pharo-project] Pointer types
>
> On 7 October 2010 13:35, Schwab,Wilhelm K <[hidden email]> wrote:
>> Sig,
>>
>> Ultimately, I will have to simply try this, but it reminds me of something that I can't place. Have you considered using a dictionary or other type of collection to store the types? A new handler would then be created and finally hooked into the system by sending a message to connect it; opposite on removal. It might turn out to be very much one-to-one so that what I am wondering about does not apply.
>>
>
> Sure, you can use dictionaries in alias lookup implementation:
>
> MyClass>>externalTypeAlias: aTypeName
> ^ myAliasesDict at: aTypeName ifAbsent: [ nil ]
>
>> Anything that can be double* can also be used with void*. How does that work?
>>
>
> Pointer is a pointer. If function expects a pointer argument, you can
> pass any pointer to it.
> Code generator just checks that an argument you passes can be
> converted to C pointer (bytearray, wordarray, external pointer),
> but sure thing it can't check if you passed char* or double* or void*.
> Such kind of type checking belongs to C compiler :)
>
> So, a pointer types in FFI callout declarations is more like a hint,
> what function expects, rather than something ,which
> influence the generated callout code.
>
>
>> Bill
>>
>>
>>
>>
>>
>> ________________________________________
>> From: [hidden email] [[hidden email]] On Behalf Of Igor Stasenko [[hidden email]]
>> Sent: Thursday, October 07, 2010 12:41 AM
>> To: [hidden email]
>> Subject: Re: [Pharo-project] Pointer types
>>
>> On 7 October 2010 01:27, Schwab,Wilhelm K <[hidden email]> wrote:
>>> Sig,
>>>
>>> Am I correct in taking this as a description of how to solve it with NB, not Squeak/Pharo FFI? That's ok, just want to know if there is an applicable lesson on FFI to be extracted.
>>>
>>> You say the NB callout generator knows nothing about types. Is that how it should be? The scenario I have in mind is a bogus type and whether that can/should be flagged at compile time vs. runtime. Maybe all it takes is a scan for a suitable handler before or after the method is compiled. I ask because I hit a related snag with FFI runtime errors being reported as coercion errors when in fact I had used errant types in the callout definitions. IIRC, Andreas patched the compiler to report the bogus types.
>>>
>>>
>>> "Or, if you want make double* type to use it as a DoubleArray, then you can add an alias:
>>>
>>> MyClass>>externalTypeAlias: aTypeName
>>> aTypeName = 'double*' ifTrue: [ ^ #DoubleArray ].
>>> ^ nil
>>>
>>> Then, any method which contains an FFI callout in MyClass, will use that alias."
>>>
>>> What is MyClass in the above? Is it the class holding the callout method, or it elsewhere in the chain? I ask because it could be a sore spot for maintenance/pluggability. I'm wondering how you and I would independently add such instructions. If it goes in the external type subclass, then it is probably fine; if it goes in a class that makes the call, we might stomp on each other. Make sense? My hunch is that you have it handled.
>>>
>>
>> Its in an any class, which using FFI callouts. We can stomp on each
>> other if you define aliases in Object class :)
>> But this won't be a good idea, isnt?
>> NBExternalType subclasses defining coercions between ST oops and
>> concrete C type, they are not for defining aliases.
>>
>>>
>>> Bill
>>>
>>>
>>> ________________________________________
>>> From: [hidden email] [[hidden email]] On Behalf Of Igor Stasenko [[hidden email]]
>>> Sent: Monday, October 04, 2010 7:36 AM
>>> To: [hidden email]
>>> Subject: Re: [Pharo-project] Pointer types
>>>
>>> On 4 October 2010 06:15, Schwab,Wilhelm K <[hidden email]> wrote:
>>>> Sig,
>>>>
>>>> You mentioned something that might be useful, depending on where it lives. Our ability to use underscores now leaves me thinking I really should rework some things that suffered mightily: there are some external things that are named assuming underscores, and in mixed case, they turn into an unreadable mess. Along with this, I have a large number of FFI calls that take double pointers, and I have for some time been demoralized into (or stupid enough to<g>) expressing them as void pointers to allow me to pass byte arrays into them.
>>>>
>>>> Is there a way that I can get FFI to recognize DOUBLEArray (which uses a ByteArray to hold its data) as something that should be acceptable for a double* call? If not FFI, how does NB handle it?
>>>>
>>>
>>> In NB, there is a type system, which implements C<->Smalltalk coercions.
>>> The responsibility of type coercions taken by a subclasses of NBExternalType,
>>> so NB callout code generator actually knows nothing about types and
>>> how to coerce them.
>>>
>>> To create own custom type, you need to subclass from NBExternalType
>>> and override methods which responsible for coercions.
>>> There you can define any kind of operations , what needs to be done in
>>> order to push value on stack, or
>>> convert it to some smalltalk object.
>>>
>>> Then, in your DoubleArray class, simply implement a message on a class side:
>>>
>>> asNBExternalType: gen
>>>
>>> ^ MyDoubleArrayExternalType new
>>>
>>>
>>> And then, in FFI callouts you can use a class name directly:
>>>
>>> DoubleArray foo (DoubleArray arr, DoubleArray * arrPtr)
>>>
>>> etc.
>>>
>>> Or, if you want make double* type to use it as a DoubleArray, then you
>>> can add an alias:
>>>
>>> MyClass>>externalTypeAlias: aTypeName
>>> aTypeName = 'double*' ifTrue: [ ^ #DoubleArray ].
>>> ^ nil
>>>
>>> Then, any method which contains an FFI callout in MyClass,
>>> will use that alias. So, you can write callout as:
>>>
>>> double* foo (double *arr, double ** arrPtrPtr)
>>>
>>> but it will be understood as:
>>>
>>> DoubleArray foo (DoubleArray arr, DoubleArray * arrPtr)
>>>
>>>
>>> About handling pointers.. well for smalltalk there is no pointers, just objects.
>>> So, in NB NBExternalType keeps two things:
>>> - a value type
>>> - a pointer arity
>>>
>>>
>>> So, for example, when you specifying type, like:
>>> char ***
>>>
>>> a value type will be char
>>> and pointer arity will be 3.
>>>
>>> It is up to NBExternalType subclass how to push values whose pointer arity > 0.
>>> By default, if pointerArity > 0, then it is treated as a pointer,
>>> which can be either: - nil, ByteArray, or NBExternalAddress
>>>
>>>
>>>> Bill
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Pharo-project mailing list
>>>> [hidden email]
>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko AKA sig.
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Igor Stasenko

Re: Pointer types

On 7 October 2010 14:24, Igor Stasenko <[hidden email]> wrote:

> On 7 October 2010 14:09, Schwab,Wilhelm K <[hidden email]> wrote:
>> Sig,
>>
>> I have to start watching the clock here, so I will just throw this out for your consideration. Yes a pointer is a pointer, but there are layers of specificity. If a function expects a double*, ideally passing a FloatArray would "take some work," perhaps in the form of wrapping the argument in a "cast"/adpater of some type and passing that to the function. If I describe an argument to the function as void*, then I do not expect much help. In fact, this is a growing source of trouble for me, because my DOUBLEArray leads me to describe parameters as void* when they should/could be double*. The current FFI *appears* to be helping in other situations by complaining when type A does not fit in slot B.
>>
>> Clearly if the function is mapped as the wrong type in the code I give you, then the seg fault is on me. But it would be nice if when given a correct (double*, long) argument sequence, you could raise an error if those are reversed. Am I missing something? I gotta go =:0
>
> You could even specify that a double* argument expects an argument to
> be an instance of DoubleArray, which then will be coerced to pointer.
> There's many other things possible, but before you start using it,
> you'll never know :)
>

or , as i said before, just write a function specification as following:

foo (DoubleArray array, long dummy)

then, in
DoubleArray class>>asNBExternalType:

answer an instance of your own
DoubleArrayType

which will be responsible for argument type-checking its coercion to
double * and pushing to stack.

As an example of implementing custom external type, you can take a look
at SqS/NBOpenGL repository
a package NBOpelGL-Core

For example:

NBOpenGL>>color3f_red: "in" red green: "in" green blue: "in" blue
"This method was automatically generated from OpenGL specs"
"See http://squeaksource.com/OpenGLSpecs for details"
<primitive: #primitiveNativeCall module: #NativeBoostPlugin>

^ self
glApiCall: #( void glColor3f ( GLfloat red , GLfloat green , GLfloat blue ) )
index: 141
attributes: #(
#category #'VERSION_1_0_DEPRECATED'
#deprecated #'3.1'
#version #'1.0'
)

GLfloat is defined in NBOpenGLTypes pool as:

GLfloat := #NBGLFloat.

And NBGLFloat is a external type subclass:

NBFloat32 subclass: #NBGLFloat
instanceVariableNames: ''
classVariableNames: ''
poolDictionaries: ''
category: 'NBOpenGL-Core'

which overrides #pushAsValue:

pushAsValue: gen
| proxy asm oop EAX |

proxy := gen proxy.
asm := gen asm.
EAX := asm EAX.
oop := loader emitLoad: gen.
asm mov: oop to: EAX.

proxy oop: EAX
ifSmallInt: [
asm sar: EAX with: 1;
push: EAX;
fild: asm ESP ptr32;
fstp: asm ESP ptr32 ]
ifNotSmallInt: [
proxy pushFloatOopAsFloat32: EAX.
].

So, in contrast to its superclas (NBFloat32),
it accepting a smallints as valid argument, not only instances of Float.

>>
>> Bill
>>
>>
>>
>> ________________________________________
>> From: [hidden email] [[hidden email]] On Behalf Of Igor Stasenko [[hidden email]]
>> Sent: Thursday, October 07, 2010 6:49 AM
>> To: [hidden email]
>> Subject: Re: [Pharo-project] Pointer types
>>
>> On 7 October 2010 13:35, Schwab,Wilhelm K <[hidden email]> wrote:
>>> Sig,
>>>
>>> Ultimately, I will have to simply try this, but it reminds me of something that I can't place. Have you considered using a dictionary or other type of collection to store the types? A new handler would then be created and finally hooked into the system by sending a message to connect it; opposite on removal. It might turn out to be very much one-to-one so that what I am wondering about does not apply.
>>>
>>
>> Sure, you can use dictionaries in alias lookup implementation:
>>
>> MyClass>>externalTypeAlias: aTypeName
>> ^ myAliasesDict at: aTypeName ifAbsent: [ nil ]
>>
>>> Anything that can be double* can also be used with void*. How does that work?
>>>
>>
>> Pointer is a pointer. If function expects a pointer argument, you can
>> pass any pointer to it.
>> Code generator just checks that an argument you passes can be
>> converted to C pointer (bytearray, wordarray, external pointer),
>> but sure thing it can't check if you passed char* or double* or void*.
>> Such kind of type checking belongs to C compiler :)
>>
>> So, a pointer types in FFI callout declarations is more like a hint,
>> what function expects, rather than something ,which
>> influence the generated callout code.
>>
>>
>>> Bill
>>>
>>>
>>>
>>>
>>>
>>> ________________________________________
>>> From: [hidden email] [[hidden email]] On Behalf Of Igor Stasenko [[hidden email]]
>>> Sent: Thursday, October 07, 2010 12:41 AM
>>> To: [hidden email]
>>> Subject: Re: [Pharo-project] Pointer types
>>>
>>> On 7 October 2010 01:27, Schwab,Wilhelm K <[hidden email]> wrote:
>>>> Sig,
>>>>
>>>> Am I correct in taking this as a description of how to solve it with NB, not Squeak/Pharo FFI? That's ok, just want to know if there is an applicable lesson on FFI to be extracted.
>>>>
>>>> You say the NB callout generator knows nothing about types. Is that how it should be? The scenario I have in mind is a bogus type and whether that can/should be flagged at compile time vs. runtime. Maybe all it takes is a scan for a suitable handler before or after the method is compiled. I ask because I hit a related snag with FFI runtime errors being reported as coercion errors when in fact I had used errant types in the callout definitions. IIRC, Andreas patched the compiler to report the bogus types.
>>>>
>>>>
>>>> "Or, if you want make double* type to use it as a DoubleArray, then you can add an alias:
>>>>
>>>> MyClass>>externalTypeAlias: aTypeName
>>>> aTypeName = 'double*' ifTrue: [ ^ #DoubleArray ].
>>>> ^ nil
>>>>
>>>> Then, any method which contains an FFI callout in MyClass, will use that alias."
>>>>
>>>> What is MyClass in the above? Is it the class holding the callout method, or it elsewhere in the chain? I ask because it could be a sore spot for maintenance/pluggability. I'm wondering how you and I would independently add such instructions. If it goes in the external type subclass, then it is probably fine; if it goes in a class that makes the call, we might stomp on each other. Make sense? My hunch is that you have it handled.
>>>>
>>>
>>> Its in an any class, which using FFI callouts. We can stomp on each
>>> other if you define aliases in Object class :)
>>> But this won't be a good idea, isnt?
>>> NBExternalType subclasses defining coercions between ST oops and
>>> concrete C type, they are not for defining aliases.
>>>
>>>>
>>>> Bill
>>>>
>>>>
>>>> ________________________________________
>>>> From: [hidden email] [[hidden email]] On Behalf Of Igor Stasenko [[hidden email]]
>>>> Sent: Monday, October 04, 2010 7:36 AM
>>>> To: [hidden email]
>>>> Subject: Re: [Pharo-project] Pointer types
>>>>
>>>> On 4 October 2010 06:15, Schwab,Wilhelm K <[hidden email]> wrote:
>>>>> Sig,
>>>>>
>>>>> You mentioned something that might be useful, depending on where it lives. Our ability to use underscores now leaves me thinking I really should rework some things that suffered mightily: there are some external things that are named assuming underscores, and in mixed case, they turn into an unreadable mess. Along with this, I have a large number of FFI calls that take double pointers, and I have for some time been demoralized into (or stupid enough to<g>) expressing them as void pointers to allow me to pass byte arrays into them.
>>>>>
>>>>> Is there a way that I can get FFI to recognize DOUBLEArray (which uses a ByteArray to hold its data) as something that should be acceptable for a double* call? If not FFI, how does NB handle it?
>>>>>
>>>>
>>>> In NB, there is a type system, which implements C<->Smalltalk coercions.
>>>> The responsibility of type coercions taken by a subclasses of NBExternalType,
>>>> so NB callout code generator actually knows nothing about types and
>>>> how to coerce them.
>>>>
>>>> To create own custom type, you need to subclass from NBExternalType
>>>> and override methods which responsible for coercions.
>>>> There you can define any kind of operations , what needs to be done in
>>>> order to push value on stack, or
>>>> convert it to some smalltalk object.
>>>>
>>>> Then, in your DoubleArray class, simply implement a message on a class side:
>>>>
>>>> asNBExternalType: gen
>>>>
>>>> ^ MyDoubleArrayExternalType new
>>>>
>>>>
>>>> And then, in FFI callouts you can use a class name directly:
>>>>
>>>> DoubleArray foo (DoubleArray arr, DoubleArray * arrPtr)
>>>>
>>>> etc.
>>>>
>>>> Or, if you want make double* type to use it as a DoubleArray, then you
>>>> can add an alias:
>>>>
>>>> MyClass>>externalTypeAlias: aTypeName
>>>> aTypeName = 'double*' ifTrue: [ ^ #DoubleArray ].
>>>> ^ nil
>>>>
>>>> Then, any method which contains an FFI callout in MyClass,
>>>> will use that alias. So, you can write callout as:
>>>>
>>>> double* foo (double *arr, double ** arrPtrPtr)
>>>>
>>>> but it will be understood as:
>>>>
>>>> DoubleArray foo (DoubleArray arr, DoubleArray * arrPtr)
>>>>
>>>>
>>>> About handling pointers.. well for smalltalk there is no pointers, just objects.
>>>> So, in NB NBExternalType keeps two things:
>>>> - a value type
>>>> - a pointer arity
>>>>
>>>>
>>>> So, for example, when you specifying type, like:
>>>> char ***
>>>>
>>>> a value type will be char
>>>> and pointer arity will be 3.
>>>>
>>>> It is up to NBExternalType subclass how to push values whose pointer arity > 0.
>>>> By default, if pointerArity > 0, then it is treated as a pointer,
>>>> which can be either: - nil, ByteArray, or NBExternalAddress
>>>>
>>>>
>>>>> Bill
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Pharo-project mailing list
>>>>> [hidden email]
>>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Igor Stasenko AKA sig.
>>>>
>>>> _______________________________________________
>>>> Pharo-project mailing list
>>>> [hidden email]
>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>
>>>> _______________________________________________
>>>> Pharo-project mailing list
>>>> [hidden email]
>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Igor Stasenko AKA sig.
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>>
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>

--
Best regards,
Igor Stasenko AKA sig.

_______________________________________________
Pharo-project mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Schwab,Wilhelm K

Re: Pointer types

Sig,

Thanks for the insights. Your use of the Smalltalk types as parameters should not surprise me (I've been using StructClass* for a long time), but it does a little. Part of what I am trying to sort out is how all of this will look in the things I have going from meta data to calls. You might have just made that easier.

The ease with which you drop into assembly language is impressive.

Bill

________________________________________
From: [hidden email] [[hidden email]] On Behalf Of Igor Stasenko [[hidden email]]
Sent: Thursday, October 07, 2010 7:58 AM
To: [hidden email]
Subject: Re: [Pharo-project] Pointer types

On 7 October 2010 14:24, Igor Stasenko <[hidden email]> wrote: