FFI: FFI-Kernel-nice.118.mcz

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

FFI: FFI-Kernel-nice.118.mcz

commits-2
Nicolas Cellier uploaded a new version of FFI-Kernel to project FFI:
http://source.squeak.org/FFI/FFI-Kernel-nice.118.mcz

==================== Summary ====================

Name: FFI-Kernel-nice.118
Author: nice
Time: 21 June 2020, 1:20:34.369348 pm
UUID: a8c48997-991d-4e32-9b67-6d93fc0dabe0
Ancestors: FFI-Kernel-mt.117

Add the ability to allocate: an ExternalType or an ExternalStructure/Alias.

It's equivalent to new: but we want a specific idiom, C types are not exactly classes.

Add the ability to compare aliases (at least for equality)

=============== Diff against FFI-Kernel-mt.117 ===============

Item was added:
+ ----- Method: ExternalStructure class>>allocate: (in category 'instance creation') -----
+ allocate: anInteger
+ "Create an ExternalData with enough room for storing an array of size anInteger of such structure"
+ ^self externalType allocate: anInteger!

Item was added:
+ ----- Method: ExternalType>>allocate: (in category 'external data') -----
+ allocate: anInteger
+ "Allocate space for containing an array of size anInteger of this dataType"
+
+ | handle |
+ handle := ByteArray new: self byteSize * anInteger.
+ ^(ExternalData fromHandle: handle type: self) size: anInteger!

Item was added:
+ ----- Method: ExternalTypeAlias>>= (in category 'comparing') -----
+ = anExternalTypeAlias
+ ^self class = anExternalTypeAlias class and: [self value = anExternalTypeAlias value]!

Item was added:
+ ----- Method: ExternalTypeAlias>>hash (in category 'comparing') -----
+ hash
+ ^self class hash hashMultiply bitXor: self value hash!


Reply | Threaded
Open this post in threaded view
|

Re: FFI: FFI-Kernel-nice.118.mcz

marcel.taeumel
Hi Nicolas.

I like the idea of having #allocate: ... but maybe not in ExternalType:

- The term "allocate:" is somewhat reserved for heap memory from the SqueakFFI perspective; See ExternalAddress class >> #allocate:.
- The counterpart #free is missing ... which means that, maybe, both #allocate: and #free should be implemented in ExternalData, not ExternalType

What about such a this programming interface:

data := ExternalData
   fromHandle: nil
   type: MyStruct externalType asPointerType.
data allocate.
myStruct := data asExternalStructure.

You could also copy it back to object memory:

myStruct := data getExternalStructure. "get/pull/fetch ...."
data free.

You are proposing a way to start off in object memory. Maybe like this:

data := ExternalData
   fromHandle: nil "maybe optional?"
   type: MyStruct externalType asPointerType.
data allocateLocal.
myStruct := data asExternalStructure.

Then we don't need to call #free. See how #allocate and #allocateLocal in ExternalData could mirror #new and #externalNew of ExternalStructure?

There is really an interesting releationship between ExternalStructure (incl. ExternalUnion/ExternalTypeAlias/ExternalPackedStructure) and ExternalData. Can you see it? ^__^

Arrays could be similar:

data := ExternalData type: Externaltype int32_t asPointerType.
data size: 20.
data allocate. "or #allocateLocal"

Now you would have an array of 20 integers. :-) Note that #allocate: with argument could inline that #size: call.

Best,
Marcel

Am 21.06.2020 13:20:44 schrieb [hidden email] <[hidden email]>:

Nicolas Cellier uploaded a new version of FFI-Kernel to project FFI:
http://source.squeak.org/FFI/FFI-Kernel-nice.118.mcz

==================== Summary ====================

Name: FFI-Kernel-nice.118
Author: nice
Time: 21 June 2020, 1:20:34.369348 pm
UUID: a8c48997-991d-4e32-9b67-6d93fc0dabe0
Ancestors: FFI-Kernel-mt.117

Add the ability to allocate: an ExternalType or an ExternalStructure/Alias.

It's equivalent to new: but we want a specific idiom, C types are not exactly classes.

Add the ability to compare aliases (at least for equality)

=============== Diff against FFI-Kernel-mt.117 ===============

Item was added:
+ ----- Method: ExternalStructure class>>allocate: (in category 'instance creation') -----
+ allocate: anInteger
+ "Create an ExternalData with enough room for storing an array of size anInteger of such structure"
+ ^self externalType allocate: anInteger!

Item was added:
+ ----- Method: ExternalType>>allocate: (in category 'external data') -----
+ allocate: anInteger
+ "Allocate space for containing an array of size anInteger of this dataType"
+
+ | handle |
+ handle := ByteArray new: self byteSize * anInteger.
+ ^(ExternalData fromHandle: handle type: self) size: anInteger!

Item was added:
+ ----- Method: ExternalTypeAlias>>= (in category 'comparing') -----
+ = anExternalTypeAlias
+ ^self class = anExternalTypeAlias class and: [self value = anExternalTypeAlias value]!

Item was added:
+ ----- Method: ExternalTypeAlias>>hash (in category 'comparing') -----
+ hash
+ ^self class hash hashMultiply bitXor: self value hash!




Reply | Threaded
Open this post in threaded view
|

Re: FFI: FFI-Kernel-nice.118.mcz

Nicolas Cellier
Hi Marcel,
The idea is to manipulate the alias or structures types as if they were classes.

I already can write MyStruct new, and pass that to an external function call.
I'd like to simply write (MyStruct new: 25) to get an array of 25 MyStruct.
I DO NOT want to see the ExternalData, it's an implementation detail blurring the intentions.
I want to see/handle my domain specific "objects" DIRECTLY.
That's what I manipulate in function signatures, etc...

Since external types are not exactly classes, i prefer to have another vocabulary in place of new:.
In VW DLLCC, the selectors malloc malloc: calloc calloc: gcMalloc gcMalloc: are sent to the CType.

Unlike us, DLLCC does not allocate in objectMemory, always on the heap.
So having the C vocabulary is logical for them.
Allocating in ObjectMemory is one essential feature of Squeak FFI.
That's why I prefer not use C vocabulary.
#allocate: is something in between.
I know that it is already used on the low level for ExternalAddress, which is unfortunate.



Le lun. 22 juin 2020 à 09:31, Marcel Taeumel <[hidden email]> a écrit :
Hi Nicolas.

I like the idea of having #allocate: ... but maybe not in ExternalType:

- The term "allocate:" is somewhat reserved for heap memory from the SqueakFFI perspective; See ExternalAddress class >> #allocate:.
- The counterpart #free is missing ... which means that, maybe, both #allocate: and #free should be implemented in ExternalData, not ExternalType

What about such a this programming interface:

data := ExternalData
   fromHandle: nil
   type: MyStruct externalType asPointerType.
data allocate.
myStruct := data asExternalStructure.

You could also copy it back to object memory:

myStruct := data getExternalStructure. "get/pull/fetch ...."
data free.

You are proposing a way to start off in object memory. Maybe like this:

data := ExternalData
   fromHandle: nil "maybe optional?"
   type: MyStruct externalType asPointerType.
data allocateLocal.
myStruct := data asExternalStructure.

Then we don't need to call #free. See how #allocate and #allocateLocal in ExternalData could mirror #new and #externalNew of ExternalStructure?

There is really an interesting releationship between ExternalStructure (incl. ExternalUnion/ExternalTypeAlias/ExternalPackedStructure) and ExternalData. Can you see it? ^__^

Arrays could be similar:

data := ExternalData type: Externaltype int32_t asPointerType.
data size: 20.
data allocate. "or #allocateLocal"

Now you would have an array of 20 integers. :-) Note that #allocate: with argument could inline that #size: call.

Best,
Marcel

Am 21.06.2020 13:20:44 schrieb [hidden email] <[hidden email]>:

Nicolas Cellier uploaded a new version of FFI-Kernel to project FFI:
http://source.squeak.org/FFI/FFI-Kernel-nice.118.mcz

==================== Summary ====================

Name: FFI-Kernel-nice.118
Author: nice
Time: 21 June 2020, 1:20:34.369348 pm
UUID: a8c48997-991d-4e32-9b67-6d93fc0dabe0
Ancestors: FFI-Kernel-mt.117

Add the ability to allocate: an ExternalType or an ExternalStructure/Alias.

It's equivalent to new: but we want a specific idiom, C types are not exactly classes.

Add the ability to compare aliases (at least for equality)

=============== Diff against FFI-Kernel-mt.117 ===============

Item was added:
+ ----- Method: ExternalStructure class>>allocate: (in category 'instance creation') -----
+ allocate: anInteger
+ "Create an ExternalData with enough room for storing an array of size anInteger of such structure"
+ ^self externalType allocate: anInteger!

Item was added:
+ ----- Method: ExternalType>>allocate: (in category 'external data') -----
+ allocate: anInteger
+ "Allocate space for containing an array of size anInteger of this dataType"
+
+ | handle |
+ handle := ByteArray new: self byteSize * anInteger.
+ ^(ExternalData fromHandle: handle type: self) size: anInteger!

Item was added:
+ ----- Method: ExternalTypeAlias>>= (in category 'comparing') -----
+ = anExternalTypeAlias
+ ^self class = anExternalTypeAlias class and: [self value = anExternalTypeAlias value]!

Item was added:
+ ----- Method: ExternalTypeAlias>>hash (in category 'comparing') -----
+ hash
+ ^self class hash hashMultiply bitXor: self value hash!





Reply | Threaded
Open this post in threaded view
|

Re: FFI: FFI-Kernel-nice.118.mcz

marcel.taeumel
Hi Nicolas,

> I'd like to simply write (MyStruct new: 25) to get an array of 25 MyStruct.
> I DO NOT want to see the ExternalData, it's an implementation detail blurring the intentions.

Then you would just hide what I just proposed behind #new: in your struct. :-) I just argued to not put 
#allocate: in ExternalType.

However, you want to treat MyStruct to be able to answer an array of itself. This is not okay. Well, it might be C-style, but not Smalltalk-style. We should try to find a connection between C-world and Smalltalk-world. Would you do that with "Morph new: 25"? Expect an array of 25 morphs?

If you want to have an array, ExternalData is the closes we have at the moment. It can directly work like Collection.

So, #new: (or similar) is a thing you want to implement in ExternalData, not MyStruct.

Well ... if you insist on creating an array of MyStruct through MyStruct .... I would at least not use #new: but rather #newArray: ... 

> I want to see/handle my domain specific "objects" DIRECTLY.

Sure. But having an array of your domain specific objects is handled via ExternalData. ExternalData is like Collection (or Array).

Best,
Marcel

Am 22.06.2020 10:21:23 schrieb Nicolas Cellier <[hidden email]>:

Hi Marcel,
The idea is to manipulate the alias or structures types as if they were classes.

I already can write MyStruct new, and pass that to an external function call.
I'd like to simply write (MyStruct new: 25) to get an array of 25 MyStruct.
I DO NOT want to see the ExternalData, it's an implementation detail blurring the intentions.
I want to see/handle my domain specific "objects" DIRECTLY.
That's what I manipulate in function signatures, etc...

Since external types are not exactly classes, i prefer to have another vocabulary in place of new:.
In VW DLLCC, the selectors malloc malloc: calloc calloc: gcMalloc gcMalloc: are sent to the CType.

Unlike us, DLLCC does not allocate in objectMemory, always on the heap.
So having the C vocabulary is logical for them.
Allocating in ObjectMemory is one essential feature of Squeak FFI.
That's why I prefer not use C vocabulary.
#allocate: is something in between.
I know that it is already used on the low level for ExternalAddress, which is unfortunate.



Le lun. 22 juin 2020 à 09:31, Marcel Taeumel <[hidden email]> a écrit :
Hi Nicolas.

I like the idea of having #allocate: ... but maybe not in ExternalType:

- The term "allocate:" is somewhat reserved for heap memory from the SqueakFFI perspective; See ExternalAddress class >> #allocate:.
- The counterpart #free is missing ... which means that, maybe, both #allocate: and #free should be implemented in ExternalData, not ExternalType

What about such a this programming interface:

data := ExternalData
   fromHandle: nil
   type: MyStruct externalType asPointerType.
data allocate.
myStruct := data asExternalStructure.

You could also copy it back to object memory:

myStruct := data getExternalStructure. "get/pull/fetch ...."
data free.

You are proposing a way to start off in object memory. Maybe like this:

data := ExternalData
   fromHandle: nil "maybe optional?"
   type: MyStruct externalType asPointerType.
data allocateLocal.
myStruct := data asExternalStructure.

Then we don't need to call #free. See how #allocate and #allocateLocal in ExternalData could mirror #new and #externalNew of ExternalStructure?

There is really an interesting releationship between ExternalStructure (incl. ExternalUnion/ExternalTypeAlias/ExternalPackedStructure) and ExternalData. Can you see it? ^__^

Arrays could be similar:

data := ExternalData type: Externaltype int32_t asPointerType.
data size: 20.
data allocate. "or #allocateLocal"

Now you would have an array of 20 integers. :-) Note that #allocate: with argument could inline that #size: call.

Best,
Marcel

Am 21.06.2020 13:20:44 schrieb [hidden email] <[hidden email]>:

Nicolas Cellier uploaded a new version of FFI-Kernel to project FFI:
http://source.squeak.org/FFI/FFI-Kernel-nice.118.mcz

==================== Summary ====================

Name: FFI-Kernel-nice.118
Author: nice
Time: 21 June 2020, 1:20:34.369348 pm
UUID: a8c48997-991d-4e32-9b67-6d93fc0dabe0
Ancestors: FFI-Kernel-mt.117

Add the ability to allocate: an ExternalType or an ExternalStructure/Alias.

It's equivalent to new: but we want a specific idiom, C types are not exactly classes.

Add the ability to compare aliases (at least for equality)

=============== Diff against FFI-Kernel-mt.117 ===============

Item was added:
+ ----- Method: ExternalStructure class>>allocate: (in category 'instance creation') -----
+ allocate: anInteger
+ "Create an ExternalData with enough room for storing an array of size anInteger of such structure"
+ ^self externalType allocate: anInteger!

Item was added:
+ ----- Method: ExternalType>>allocate: (in category 'external data') -----
+ allocate: anInteger
+ "Allocate space for containing an array of size anInteger of this dataType"
+
+ | handle |
+ handle := ByteArray new: self byteSize * anInteger.
+ ^(ExternalData fromHandle: handle type: self) size: anInteger!

Item was added:
+ ----- Method: ExternalTypeAlias>>= (in category 'comparing') -----
+ = anExternalTypeAlias
+ ^self class = anExternalTypeAlias class and: [self value = anExternalTypeAlias value]!

Item was added:
+ ----- Method: ExternalTypeAlias>>hash (in category 'comparing') -----
+ hash
+ ^self class hash hashMultiply bitXor: self value hash!





Reply | Threaded
Open this post in threaded view
|

Re: FFI: FFI-Kernel-nice.118.mcz

Nicolas Cellier


Le lun. 22 juin 2020 à 10:30, Marcel Taeumel <[hidden email]> a écrit :
Hi Nicolas,

> I'd like to simply write (MyStruct new: 25) to get an array of 25 MyStruct.
> I DO NOT want to see the ExternalData, it's an implementation detail blurring the intentions.

Then you would just hide what I just proposed behind #new: in your struct. :-) I just argued to not put 
#allocate: in ExternalType.

However, you want to treat MyStruct to be able to answer an array of itself. This is not okay. Well, it might be C-style, but not Smalltalk-style. We should try to find a connection between C-world and Smalltalk-world. Would you do that with "Morph new: 25"? Expect an array of 25 morphs?

If you want to have an array, ExternalData is the closes we have at the moment. It can directly work like Collection.

So, #new: (or similar) is a thing you want to implement in ExternalData, not MyStruct.

Well ... if you insist on creating an array of MyStruct through MyStruct .... I would at least not use #new: but rather #newArray: ... 


In your mind, ExternalData <=> Array.
(Array of: MyStruct) new: 10 <=> (ExternalData type: MyStruct) new: 10.
That's what you tend toward: add some of Collection protocol to ExternalData.

If you have a look at Smallapack ExternalArray, that's somehow what I tried to achieve years ago...
have Collection protocol for arrays external objects.

My POV is different.
We're interfacing C.
In C, I can simply have MyStruct foo; or MyStruct foo[10];
So direct manipulation of MyStruct without any intermediary is much more appealing/convenient.

ExternalData? The name does not speak as Collection.
In fact it's a hurdle. It's verbiage. It's too low level, like all the external*
I do not see the ExternalFunction, can live without EternalAddress, etc...
I do not want to manipulate ExternalData by myself. I do not want to see it.
I do not want to have to tell an ExternalData to create an array of the type I need for myself.
I do want to handle my domain objects DIRECTLY, without intermediary (or at least have the illusion of).
See it?

> I want to see/handle my domain specific "objects" DIRECTLY.

Sure. But having an array of your domain specific objects is handled via ExternalData. ExternalData is like Collection (or Array).

Best,
Marcel

Am 22.06.2020 10:21:23 schrieb Nicolas Cellier <[hidden email]>:

Hi Marcel,
The idea is to manipulate the alias or structures types as if they were classes.

I already can write MyStruct new, and pass that to an external function call.
I'd like to simply write (MyStruct new: 25) to get an array of 25 MyStruct.
I DO NOT want to see the ExternalData, it's an implementation detail blurring the intentions.
I want to see/handle my domain specific "objects" DIRECTLY.
That's what I manipulate in function signatures, etc...

Since external types are not exactly classes, i prefer to have another vocabulary in place of new:.
In VW DLLCC, the selectors malloc malloc: calloc calloc: gcMalloc gcMalloc: are sent to the CType.

Unlike us, DLLCC does not allocate in objectMemory, always on the heap.
So having the C vocabulary is logical for them.
Allocating in ObjectMemory is one essential feature of Squeak FFI.
That's why I prefer not use C vocabulary.
#allocate: is something in between.
I know that it is already used on the low level for ExternalAddress, which is unfortunate.



Le lun. 22 juin 2020 à 09:31, Marcel Taeumel <[hidden email]> a écrit :
Hi Nicolas.

I like the idea of having #allocate: ... but maybe not in ExternalType:

- The term "allocate:" is somewhat reserved for heap memory from the SqueakFFI perspective; See ExternalAddress class >> #allocate:.
- The counterpart #free is missing ... which means that, maybe, both #allocate: and #free should be implemented in ExternalData, not ExternalType

What about such a this programming interface:

data := ExternalData
   fromHandle: nil
   type: MyStruct externalType asPointerType.
data allocate.
myStruct := data asExternalStructure.

You could also copy it back to object memory:

myStruct := data getExternalStructure. "get/pull/fetch ...."
data free.

You are proposing a way to start off in object memory. Maybe like this:

data := ExternalData
   fromHandle: nil "maybe optional?"
   type: MyStruct externalType asPointerType.
data allocateLocal.
myStruct := data asExternalStructure.

Then we don't need to call #free. See how #allocate and #allocateLocal in ExternalData could mirror #new and #externalNew of ExternalStructure?

There is really an interesting releationship between ExternalStructure (incl. ExternalUnion/ExternalTypeAlias/ExternalPackedStructure) and ExternalData. Can you see it? ^__^

Arrays could be similar:

data := ExternalData type: Externaltype int32_t asPointerType.
data size: 20.
data allocate. "or #allocateLocal"

Now you would have an array of 20 integers. :-) Note that #allocate: with argument could inline that #size: call.

Best,
Marcel

Am 21.06.2020 13:20:44 schrieb [hidden email] <[hidden email]>:

Nicolas Cellier uploaded a new version of FFI-Kernel to project FFI:
http://source.squeak.org/FFI/FFI-Kernel-nice.118.mcz

==================== Summary ====================

Name: FFI-Kernel-nice.118
Author: nice
Time: 21 June 2020, 1:20:34.369348 pm
UUID: a8c48997-991d-4e32-9b67-6d93fc0dabe0
Ancestors: FFI-Kernel-mt.117

Add the ability to allocate: an ExternalType or an ExternalStructure/Alias.

It's equivalent to new: but we want a specific idiom, C types are not exactly classes.

Add the ability to compare aliases (at least for equality)

=============== Diff against FFI-Kernel-mt.117 ===============

Item was added:
+ ----- Method: ExternalStructure class>>allocate: (in category 'instance creation') -----
+ allocate: anInteger
+ "Create an ExternalData with enough room for storing an array of size anInteger of such structure"
+ ^self externalType allocate: anInteger!

Item was added:
+ ----- Method: ExternalType>>allocate: (in category 'external data') -----
+ allocate: anInteger
+ "Allocate space for containing an array of size anInteger of this dataType"
+
+ | handle |
+ handle := ByteArray new: self byteSize * anInteger.
+ ^(ExternalData fromHandle: handle type: self) size: anInteger!

Item was added:
+ ----- Method: ExternalTypeAlias>>= (in category 'comparing') -----
+ = anExternalTypeAlias
+ ^self class = anExternalTypeAlias class and: [self value = anExternalTypeAlias value]!

Item was added:
+ ----- Method: ExternalTypeAlias>>hash (in category 'comparing') -----
+ hash
+ ^self class hash hashMultiply bitXor: self value hash!






Reply | Threaded
Open this post in threaded view
|

Re: FFI: FFI-Kernel-nice.118.mcz

marcel.taeumel
We're interfacing C.

From within Smalltalk. Let's not forget that. We have to find trade-offs.

In C, I can simply have MyStruct foo; or MyStruct foo[10];

Then add #newArray: to ExternalStructure, not #new:. To not confuse C and Smalltalk terminology more than we must. Yet, I would not want to write an FFI bridge such a way. But both ways should be possible. :-)

ExternalData? The name does not speak as Collection.

Could be renamed, but maybe not. Because it could remain a transperent helper.

I do not want to manipulate ExternalData by myself. I do not want to see it.

Then hide the use of ExternalData  through ExternalStructure. But favor #newArray: over #new:. As argued before.

I do want to handle my domain objects DIRECTLY, without intermediary (or at least have the illusion of).

That's where we agree on. :-) 

So direct manipulation of MyStruct without any intermediary is much more appealing/convenient.

Ehm ... sure. But MyStruct and array of MyStruct should not both be handled via the same MyStruct class in Squeak FFI. Your ExternalArray could be a simple subclass of ExternalData. If you like such a name better. Implementation wouldn't change at all. :-)

Best,
Marcel

Am 22.06.2020 10:55:14 schrieb Nicolas Cellier <[hidden email]>:



Le lun. 22 juin 2020 à 10:30, Marcel Taeumel <[hidden email]> a écrit :
Hi Nicolas,

> I'd like to simply write (MyStruct new: 25) to get an array of 25 MyStruct.
> I DO NOT want to see the ExternalData, it's an implementation detail blurring the intentions.

Then you would just hide what I just proposed behind #new: in your struct. :-) I just argued to not put 
#allocate: in ExternalType.

However, you want to treat MyStruct to be able to answer an array of itself. This is not okay. Well, it might be C-style, but not Smalltalk-style. We should try to find a connection between C-world and Smalltalk-world. Would you do that with "Morph new: 25"? Expect an array of 25 morphs?

If you want to have an array, ExternalData is the closes we have at the moment. It can directly work like Collection.

So, #new: (or similar) is a thing you want to implement in ExternalData, not MyStruct.

Well ... if you insist on creating an array of MyStruct through MyStruct .... I would at least not use #new: but rather #newArray: ... 


In your mind, ExternalData <=> Array.
(Array of: MyStruct) new: 10 <=> (ExternalData type: MyStruct) new: 10.
That's what you tend toward: add some of Collection protocol to ExternalData.

If you have a look at Smallapack ExternalArray, that's somehow what I tried to achieve years ago...
have Collection protocol for arrays external objects.

My POV is different.
We're interfacing C.
In C, I can simply have MyStruct foo; or MyStruct foo[10];
So direct manipulation of MyStruct without any intermediary is much more appealing/convenient.

ExternalData? The name does not speak as Collection.
In fact it's a hurdle. It's verbiage. It's too low level, like all the external*
I do not see the ExternalFunction, can live without EternalAddress, etc...
I do not want to manipulate ExternalData by myself. I do not want to see it.
I do not want to have to tell an ExternalData to create an array of the type I need for myself.
I do want to handle my domain objects DIRECTLY, without intermediary (or at least have the illusion of).
See it?

> I want to see/handle my domain specific "objects" DIRECTLY.

Sure. But having an array of your domain specific objects is handled via ExternalData. ExternalData is like Collection (or Array).

Best,
Marcel

Am 22.06.2020 10:21:23 schrieb Nicolas Cellier <[hidden email]>:

Hi Marcel,
The idea is to manipulate the alias or structures types as if they were classes.

I already can write MyStruct new, and pass that to an external function call.
I'd like to simply write (MyStruct new: 25) to get an array of 25 MyStruct.
I DO NOT want to see the ExternalData, it's an implementation detail blurring the intentions.
I want to see/handle my domain specific "objects" DIRECTLY.
That's what I manipulate in function signatures, etc...

Since external types are not exactly classes, i prefer to have another vocabulary in place of new:.
In VW DLLCC, the selectors malloc malloc: calloc calloc: gcMalloc gcMalloc: are sent to the CType.

Unlike us, DLLCC does not allocate in objectMemory, always on the heap.
So having the C vocabulary is logical for them.
Allocating in ObjectMemory is one essential feature of Squeak FFI.
That's why I prefer not use C vocabulary.
#allocate: is something in between.
I know that it is already used on the low level for ExternalAddress, which is unfortunate.



Le lun. 22 juin 2020 à 09:31, Marcel Taeumel <[hidden email]> a écrit :
Hi Nicolas.

I like the idea of having #allocate: ... but maybe not in ExternalType:

- The term "allocate:" is somewhat reserved for heap memory from the SqueakFFI perspective; See ExternalAddress class >> #allocate:.
- The counterpart #free is missing ... which means that, maybe, both #allocate: and #free should be implemented in ExternalData, not ExternalType

What about such a this programming interface:

data := ExternalData
   fromHandle: nil
   type: MyStruct externalType asPointerType.
data allocate.
myStruct := data asExternalStructure.

You could also copy it back to object memory:

myStruct := data getExternalStructure. "get/pull/fetch ...."
data free.

You are proposing a way to start off in object memory. Maybe like this:

data := ExternalData
   fromHandle: nil "maybe optional?"
   type: MyStruct externalType asPointerType.
data allocateLocal.
myStruct := data asExternalStructure.

Then we don't need to call #free. See how #allocate and #allocateLocal in ExternalData could mirror #new and #externalNew of ExternalStructure?

There is really an interesting releationship between ExternalStructure (incl. ExternalUnion/ExternalTypeAlias/ExternalPackedStructure) and ExternalData. Can you see it? ^__^

Arrays could be similar:

data := ExternalData type: Externaltype int32_t asPointerType.
data size: 20.
data allocate. "or #allocateLocal"

Now you would have an array of 20 integers. :-) Note that #allocate: with argument could inline that #size: call.

Best,
Marcel

Am 21.06.2020 13:20:44 schrieb [hidden email] <[hidden email]>:

Nicolas Cellier uploaded a new version of FFI-Kernel to project FFI:
http://source.squeak.org/FFI/FFI-Kernel-nice.118.mcz

==================== Summary ====================

Name: FFI-Kernel-nice.118
Author: nice
Time: 21 June 2020, 1:20:34.369348 pm
UUID: a8c48997-991d-4e32-9b67-6d93fc0dabe0
Ancestors: FFI-Kernel-mt.117

Add the ability to allocate: an ExternalType or an ExternalStructure/Alias.

It's equivalent to new: but we want a specific idiom, C types are not exactly classes.

Add the ability to compare aliases (at least for equality)

=============== Diff against FFI-Kernel-mt.117 ===============

Item was added:
+ ----- Method: ExternalStructure class>>allocate: (in category 'instance creation') -----
+ allocate: anInteger
+ "Create an ExternalData with enough room for storing an array of size anInteger of such structure"
+ ^self externalType allocate: anInteger!

Item was added:
+ ----- Method: ExternalType>>allocate: (in category 'external data') -----
+ allocate: anInteger
+ "Allocate space for containing an array of size anInteger of this dataType"
+
+ | handle |
+ handle := ByteArray new: self byteSize * anInteger.
+ ^(ExternalData fromHandle: handle type: self) size: anInteger!

Item was added:
+ ----- Method: ExternalTypeAlias>>= (in category 'comparing') -----
+ = anExternalTypeAlias
+ ^self class = anExternalTypeAlias class and: [self value = anExternalTypeAlias value]!

Item was added:
+ ----- Method: ExternalTypeAlias>>hash (in category 'comparing') -----
+ hash
+ ^self class hash hashMultiply bitXor: self value hash!






Reply | Threaded
Open this post in threaded view
|

Re: FFI: FFI-Kernel-nice.118.mcz

Nicolas Cellier


Le lun. 22 juin 2020 à 11:05, Marcel Taeumel <[hidden email]> a écrit :
We're interfacing C.

From within Smalltalk. Let's not forget that. We have to find trade-offs.

In C, I can simply have MyStruct foo; or MyStruct foo[10];

Then add #newArray: to ExternalStructure, not #new:. To not confuse C and Smalltalk terminology more than we must. Yet, I would not want to write an FFI bridge such a way. But both ways should be possible. :-)

ExternalData? The name does not speak as Collection.

Could be renamed, but maybe not. Because it could remain a transperent helper.

I do not want to manipulate ExternalData by myself. I do not want to see it.

Then hide the use of ExternalData  through ExternalStructure. But favor #newArray: over #new:. As argued before.

exactly, that's why I have used #allocate:, not new:, those types are not exactly classes, though MyStruct is a class...

I do want to handle my domain objects DIRECTLY, without intermediary (or at least have the illusion of).

That's where we agree on. :-) 

So direct manipulation of MyStruct without any intermediary is much more appealing/convenient.

Ehm ... sure. But MyStruct and array of MyStruct should not both be handled via the same MyStruct class in Squeak FFI. Your ExternalArray could be a simple subclass of ExternalData. If you like such a name better. Implementation wouldn't change at all. :-)


Absolutely!
I want to create those ExternalData. It's just that I want to ask my object to create those, I don't want to ask ExternalData to create my objects.

For the generalization to other atomic types, we indeed need to go through the ExternalType and implement allocate: there too.
That's somehow how it is done in DLLCC, and the API is convenient IMO.

Best,
Marcel

Am 22.06.2020 10:55:14 schrieb Nicolas Cellier <[hidden email]>:



Le lun. 22 juin 2020 à 10:30, Marcel Taeumel <[hidden email]> a écrit :
Hi Nicolas,

> I'd like to simply write (MyStruct new: 25) to get an array of 25 MyStruct.
> I DO NOT want to see the ExternalData, it's an implementation detail blurring the intentions.

Then you would just hide what I just proposed behind #new: in your struct. :-) I just argued to not put 
#allocate: in ExternalType.

However, you want to treat MyStruct to be able to answer an array of itself. This is not okay. Well, it might be C-style, but not Smalltalk-style. We should try to find a connection between C-world and Smalltalk-world. Would you do that with "Morph new: 25"? Expect an array of 25 morphs?

If you want to have an array, ExternalData is the closes we have at the moment. It can directly work like Collection.

So, #new: (or similar) is a thing you want to implement in ExternalData, not MyStruct.

Well ... if you insist on creating an array of MyStruct through MyStruct .... I would at least not use #new: but rather #newArray: ... 


In your mind, ExternalData <=> Array.
(Array of: MyStruct) new: 10 <=> (ExternalData type: MyStruct) new: 10.
That's what you tend toward: add some of Collection protocol to ExternalData.

If you have a look at Smallapack ExternalArray, that's somehow what I tried to achieve years ago...
have Collection protocol for arrays external objects.

My POV is different.
We're interfacing C.
In C, I can simply have MyStruct foo; or MyStruct foo[10];
So direct manipulation of MyStruct without any intermediary is much more appealing/convenient.

ExternalData? The name does not speak as Collection.
In fact it's a hurdle. It's verbiage. It's too low level, like all the external*
I do not see the ExternalFunction, can live without EternalAddress, etc...
I do not want to manipulate ExternalData by myself. I do not want to see it.
I do not want to have to tell an ExternalData to create an array of the type I need for myself.
I do want to handle my domain objects DIRECTLY, without intermediary (or at least have the illusion of).
See it?

> I want to see/handle my domain specific "objects" DIRECTLY.

Sure. But having an array of your domain specific objects is handled via ExternalData. ExternalData is like Collection (or Array).

Best,
Marcel

Am 22.06.2020 10:21:23 schrieb Nicolas Cellier <[hidden email]>:

Hi Marcel,
The idea is to manipulate the alias or structures types as if they were classes.

I already can write MyStruct new, and pass that to an external function call.
I'd like to simply write (MyStruct new: 25) to get an array of 25 MyStruct.
I DO NOT want to see the ExternalData, it's an implementation detail blurring the intentions.
I want to see/handle my domain specific "objects" DIRECTLY.
That's what I manipulate in function signatures, etc...

Since external types are not exactly classes, i prefer to have another vocabulary in place of new:.
In VW DLLCC, the selectors malloc malloc: calloc calloc: gcMalloc gcMalloc: are sent to the CType.

Unlike us, DLLCC does not allocate in objectMemory, always on the heap.
So having the C vocabulary is logical for them.
Allocating in ObjectMemory is one essential feature of Squeak FFI.
That's why I prefer not use C vocabulary.
#allocate: is something in between.
I know that it is already used on the low level for ExternalAddress, which is unfortunate.



Le lun. 22 juin 2020 à 09:31, Marcel Taeumel <[hidden email]> a écrit :
Hi Nicolas.

I like the idea of having #allocate: ... but maybe not in ExternalType:

- The term "allocate:" is somewhat reserved for heap memory from the SqueakFFI perspective; See ExternalAddress class >> #allocate:.
- The counterpart #free is missing ... which means that, maybe, both #allocate: and #free should be implemented in ExternalData, not ExternalType

What about such a this programming interface:

data := ExternalData
   fromHandle: nil
   type: MyStruct externalType asPointerType.
data allocate.
myStruct := data asExternalStructure.

You could also copy it back to object memory:

myStruct := data getExternalStructure. "get/pull/fetch ...."
data free.

You are proposing a way to start off in object memory. Maybe like this:

data := ExternalData
   fromHandle: nil "maybe optional?"
   type: MyStruct externalType asPointerType.
data allocateLocal.
myStruct := data asExternalStructure.

Then we don't need to call #free. See how #allocate and #allocateLocal in ExternalData could mirror #new and #externalNew of ExternalStructure?

There is really an interesting releationship between ExternalStructure (incl. ExternalUnion/ExternalTypeAlias/ExternalPackedStructure) and ExternalData. Can you see it? ^__^

Arrays could be similar:

data := ExternalData type: Externaltype int32_t asPointerType.
data size: 20.
data allocate. "or #allocateLocal"

Now you would have an array of 20 integers. :-) Note that #allocate: with argument could inline that #size: call.

Best,
Marcel

Am 21.06.2020 13:20:44 schrieb [hidden email] <[hidden email]>:

Nicolas Cellier uploaded a new version of FFI-Kernel to project FFI:
http://source.squeak.org/FFI/FFI-Kernel-nice.118.mcz

==================== Summary ====================

Name: FFI-Kernel-nice.118
Author: nice
Time: 21 June 2020, 1:20:34.369348 pm
UUID: a8c48997-991d-4e32-9b67-6d93fc0dabe0
Ancestors: FFI-Kernel-mt.117

Add the ability to allocate: an ExternalType or an ExternalStructure/Alias.

It's equivalent to new: but we want a specific idiom, C types are not exactly classes.

Add the ability to compare aliases (at least for equality)

=============== Diff against FFI-Kernel-mt.117 ===============

Item was added:
+ ----- Method: ExternalStructure class>>allocate: (in category 'instance creation') -----
+ allocate: anInteger
+ "Create an ExternalData with enough room for storing an array of size anInteger of such structure"
+ ^self externalType allocate: anInteger!

Item was added:
+ ----- Method: ExternalType>>allocate: (in category 'external data') -----
+ allocate: anInteger
+ "Allocate space for containing an array of size anInteger of this dataType"
+
+ | handle |
+ handle := ByteArray new: self byteSize * anInteger.
+ ^(ExternalData fromHandle: handle type: self) size: anInteger!

Item was added:
+ ----- Method: ExternalTypeAlias>>= (in category 'comparing') -----
+ = anExternalTypeAlias
+ ^self class = anExternalTypeAlias class and: [self value = anExternalTypeAlias value]!

Item was added:
+ ----- Method: ExternalTypeAlias>>hash (in category 'comparing') -----
+ hash
+ ^self class hash hashMultiply bitXor: self value hash!







Reply | Threaded
Open this post in threaded view
|

Re: FFI: FFI-Kernel-nice.118.mcz

marcel.taeumel
For the generalization to other atomic types, we indeed need to go through the ExternalType and implement allocate: there too.

Not needed. You can access ExternalType >> #byteSize from within ExternalData. It's public.

Best,
Marcel

Am 22.06.2020 12:08:47 schrieb Nicolas Cellier <[hidden email]>:



Le lun. 22 juin 2020 à 11:05, Marcel Taeumel <[hidden email]> a écrit :
We're interfacing C.

From within Smalltalk. Let's not forget that. We have to find trade-offs.

In C, I can simply have MyStruct foo; or MyStruct foo[10];

Then add #newArray: to ExternalStructure, not #new:. To not confuse C and Smalltalk terminology more than we must. Yet, I would not want to write an FFI bridge such a way. But both ways should be possible. :-)

ExternalData? The name does not speak as Collection.

Could be renamed, but maybe not. Because it could remain a transperent helper.

I do not want to manipulate ExternalData by myself. I do not want to see it.

Then hide the use of ExternalData  through ExternalStructure. But favor #newArray: over #new:. As argued before.

exactly, that's why I have used #allocate:, not new:, those types are not exactly classes, though MyStruct is a class...

I do want to handle my domain objects DIRECTLY, without intermediary (or at least have the illusion of).

That's where we agree on. :-) 

So direct manipulation of MyStruct without any intermediary is much more appealing/convenient.

Ehm ... sure. But MyStruct and array of MyStruct should not both be handled via the same MyStruct class in Squeak FFI. Your ExternalArray could be a simple subclass of ExternalData. If you like such a name better. Implementation wouldn't change at all. :-)


Absolutely!
I want to create those ExternalData. It's just that I want to ask my object to create those, I don't want to ask ExternalData to create my objects.

For the generalization to other atomic types, we indeed need to go through the ExternalType and implement allocate: there too.
That's somehow how it is done in DLLCC, and the API is convenient IMO.

Best,
Marcel

Am 22.06.2020 10:55:14 schrieb Nicolas Cellier <[hidden email]>:



Le lun. 22 juin 2020 à 10:30, Marcel Taeumel <[hidden email]> a écrit :
Hi Nicolas,

> I'd like to simply write (MyStruct new: 25) to get an array of 25 MyStruct.
> I DO NOT want to see the ExternalData, it's an implementation detail blurring the intentions.

Then you would just hide what I just proposed behind #new: in your struct. :-) I just argued to not put 
#allocate: in ExternalType.

However, you want to treat MyStruct to be able to answer an array of itself. This is not okay. Well, it might be C-style, but not Smalltalk-style. We should try to find a connection between C-world and Smalltalk-world. Would you do that with "Morph new: 25"? Expect an array of 25 morphs?

If you want to have an array, ExternalData is the closes we have at the moment. It can directly work like Collection.

So, #new: (or similar) is a thing you want to implement in ExternalData, not MyStruct.

Well ... if you insist on creating an array of MyStruct through MyStruct .... I would at least not use #new: but rather #newArray: ... 


In your mind, ExternalData <=> Array.
(Array of: MyStruct) new: 10 <=> (ExternalData type: MyStruct) new: 10.
That's what you tend toward: add some of Collection protocol to ExternalData.

If you have a look at Smallapack ExternalArray, that's somehow what I tried to achieve years ago...
have Collection protocol for arrays external objects.

My POV is different.
We're interfacing C.
In C, I can simply have MyStruct foo; or MyStruct foo[10];
So direct manipulation of MyStruct without any intermediary is much more appealing/convenient.

ExternalData? The name does not speak as Collection.
In fact it's a hurdle. It's verbiage. It's too low level, like all the external*
I do not see the ExternalFunction, can live without EternalAddress, etc...
I do not want to manipulate ExternalData by myself. I do not want to see it.
I do not want to have to tell an ExternalData to create an array of the type I need for myself.
I do want to handle my domain objects DIRECTLY, without intermediary (or at least have the illusion of).
See it?

> I want to see/handle my domain specific "objects" DIRECTLY.

Sure. But having an array of your domain specific objects is handled via ExternalData. ExternalData is like Collection (or Array).

Best,
Marcel

Am 22.06.2020 10:21:23 schrieb Nicolas Cellier <[hidden email]>:

Hi Marcel,
The idea is to manipulate the alias or structures types as if they were classes.

I already can write MyStruct new, and pass that to an external function call.
I'd like to simply write (MyStruct new: 25) to get an array of 25 MyStruct.
I DO NOT want to see the ExternalData, it's an implementation detail blurring the intentions.
I want to see/handle my domain specific "objects" DIRECTLY.
That's what I manipulate in function signatures, etc...

Since external types are not exactly classes, i prefer to have another vocabulary in place of new:.
In VW DLLCC, the selectors malloc malloc: calloc calloc: gcMalloc gcMalloc: are sent to the CType.

Unlike us, DLLCC does not allocate in objectMemory, always on the heap.
So having the C vocabulary is logical for them.
Allocating in ObjectMemory is one essential feature of Squeak FFI.
That's why I prefer not use C vocabulary.
#allocate: is something in between.
I know that it is already used on the low level for ExternalAddress, which is unfortunate.



Le lun. 22 juin 2020 à 09:31, Marcel Taeumel <[hidden email]> a écrit :
Hi Nicolas.

I like the idea of having #allocate: ... but maybe not in ExternalType:

- The term "allocate:" is somewhat reserved for heap memory from the SqueakFFI perspective; See ExternalAddress class >> #allocate:.
- The counterpart #free is missing ... which means that, maybe, both #allocate: and #free should be implemented in ExternalData, not ExternalType

What about such a this programming interface:

data := ExternalData
   fromHandle: nil
   type: MyStruct externalType asPointerType.
data allocate.
myStruct := data asExternalStructure.

You could also copy it back to object memory:

myStruct := data getExternalStructure. "get/pull/fetch ...."
data free.

You are proposing a way to start off in object memory. Maybe like this:

data := ExternalData
   fromHandle: nil "maybe optional?"
   type: MyStruct externalType asPointerType.
data allocateLocal.
myStruct := data asExternalStructure.

Then we don't need to call #free. See how #allocate and #allocateLocal in ExternalData could mirror #new and #externalNew of ExternalStructure?

There is really an interesting releationship between ExternalStructure (incl. ExternalUnion/ExternalTypeAlias/ExternalPackedStructure) and ExternalData. Can you see it? ^__^

Arrays could be similar:

data := ExternalData type: Externaltype int32_t asPointerType.
data size: 20.
data allocate. "or #allocateLocal"

Now you would have an array of 20 integers. :-) Note that #allocate: with argument could inline that #size: call.

Best,
Marcel

Am 21.06.2020 13:20:44 schrieb [hidden email] <[hidden email]>:

Nicolas Cellier uploaded a new version of FFI-Kernel to project FFI:
http://source.squeak.org/FFI/FFI-Kernel-nice.118.mcz

==================== Summary ====================

Name: FFI-Kernel-nice.118
Author: nice
Time: 21 June 2020, 1:20:34.369348 pm
UUID: a8c48997-991d-4e32-9b67-6d93fc0dabe0
Ancestors: FFI-Kernel-mt.117

Add the ability to allocate: an ExternalType or an ExternalStructure/Alias.

It's equivalent to new: but we want a specific idiom, C types are not exactly classes.

Add the ability to compare aliases (at least for equality)

=============== Diff against FFI-Kernel-mt.117 ===============

Item was added:
+ ----- Method: ExternalStructure class>>allocate: (in category 'instance creation') -----
+ allocate: anInteger
+ "Create an ExternalData with enough room for storing an array of size anInteger of such structure"
+ ^self externalType allocate: anInteger!

Item was added:
+ ----- Method: ExternalType>>allocate: (in category 'external data') -----
+ allocate: anInteger
+ "Allocate space for containing an array of size anInteger of this dataType"
+
+ | handle |
+ handle := ByteArray new: self byteSize * anInteger.
+ ^(ExternalData fromHandle: handle type: self) size: anInteger!

Item was added:
+ ----- Method: ExternalTypeAlias>>= (in category 'comparing') -----
+ = anExternalTypeAlias
+ ^self class = anExternalTypeAlias class and: [self value = anExternalTypeAlias value]!

Item was added:
+ ----- Method: ExternalTypeAlias>>hash (in category 'comparing') -----
+ hash
+ ^self class hash hashMultiply bitXor: self value hash!







Reply | Threaded
Open this post in threaded view
|

Re: FFI: FFI-Kernel-nice.118.mcz

Nicolas Cellier


Le lun. 22 juin 2020 à 12:11, Marcel Taeumel <[hidden email]> a écrit :
For the generalization to other atomic types, we indeed need to go through the ExternalType and implement allocate: there too.

Not needed. You can access ExternalType >> #byteSize from within ExternalData. It's public.

Best,
Marcel

Hi Marcel,
you constantly bring me to a lower level API ...
I know that byteSize is public, but I don't want to bother with that.
I want to handle an array of N things, not an ExternalData withHandle: (ByteArray new: thing byteSize * N) type: thing.
I want (thing allocate: N), that's all I need...
I've used similar constructs provided by VW DLLCC, and believe me, it's convenient!

Beside, think of it, an ExternalData has to provide both memory AND type right?
    ExternalData fromHandle: (ByteArray new: type byteSize * anInteger) type: type.
Who knows better about its own size than the type itself?
Why scatter the responsibilities that could be gathered in the type?
Where would this snippet go?

Am 22.06.2020 12:08:47 schrieb Nicolas Cellier <[hidden email]>:



Le lun. 22 juin 2020 à 11:05, Marcel Taeumel <[hidden email]> a écrit :
We're interfacing C.

From within Smalltalk. Let's not forget that. We have to find trade-offs.

In C, I can simply have MyStruct foo; or MyStruct foo[10];

Then add #newArray: to ExternalStructure, not #new:. To not confuse C and Smalltalk terminology more than we must. Yet, I would not want to write an FFI bridge such a way. But both ways should be possible. :-)

ExternalData? The name does not speak as Collection.

Could be renamed, but maybe not. Because it could remain a transperent helper.

I do not want to manipulate ExternalData by myself. I do not want to see it.

Then hide the use of ExternalData  through ExternalStructure. But favor #newArray: over #new:. As argued before.

exactly, that's why I have used #allocate:, not new:, those types are not exactly classes, though MyStruct is a class...

I do want to handle my domain objects DIRECTLY, without intermediary (or at least have the illusion of).

That's where we agree on. :-) 

So direct manipulation of MyStruct without any intermediary is much more appealing/convenient.

Ehm ... sure. But MyStruct and array of MyStruct should not both be handled via the same MyStruct class in Squeak FFI. Your ExternalArray could be a simple subclass of ExternalData. If you like such a name better. Implementation wouldn't change at all. :-)


Absolutely!
I want to create those ExternalData. It's just that I want to ask my object to create those, I don't want to ask ExternalData to create my objects.

For the generalization to other atomic types, we indeed need to go through the ExternalType and implement allocate: there too.
That's somehow how it is done in DLLCC, and the API is convenient IMO.

Best,
Marcel

Am 22.06.2020 10:55:14 schrieb Nicolas Cellier <[hidden email]>:



Le lun. 22 juin 2020 à 10:30, Marcel Taeumel <[hidden email]> a écrit :
Hi Nicolas,

> I'd like to simply write (MyStruct new: 25) to get an array of 25 MyStruct.
> I DO NOT want to see the ExternalData, it's an implementation detail blurring the intentions.

Then you would just hide what I just proposed behind #new: in your struct. :-) I just argued to not put 
#allocate: in ExternalType.

However, you want to treat MyStruct to be able to answer an array of itself. This is not okay. Well, it might be C-style, but not Smalltalk-style. We should try to find a connection between C-world and Smalltalk-world. Would you do that with "Morph new: 25"? Expect an array of 25 morphs?

If you want to have an array, ExternalData is the closes we have at the moment. It can directly work like Collection.

So, #new: (or similar) is a thing you want to implement in ExternalData, not MyStruct.

Well ... if you insist on creating an array of MyStruct through MyStruct .... I would at least not use #new: but rather #newArray: ... 


In your mind, ExternalData <=> Array.
(Array of: MyStruct) new: 10 <=> (ExternalData type: MyStruct) new: 10.
That's what you tend toward: add some of Collection protocol to ExternalData.

If you have a look at Smallapack ExternalArray, that's somehow what I tried to achieve years ago...
have Collection protocol for arrays external objects.

My POV is different.
We're interfacing C.
In C, I can simply have MyStruct foo; or MyStruct foo[10];
So direct manipulation of MyStruct without any intermediary is much more appealing/convenient.

ExternalData? The name does not speak as Collection.
In fact it's a hurdle. It's verbiage. It's too low level, like all the external*
I do not see the ExternalFunction, can live without EternalAddress, etc...
I do not want to manipulate ExternalData by myself. I do not want to see it.
I do not want to have to tell an ExternalData to create an array of the type I need for myself.
I do want to handle my domain objects DIRECTLY, without intermediary (or at least have the illusion of).
See it?

> I want to see/handle my domain specific "objects" DIRECTLY.

Sure. But having an array of your domain specific objects is handled via ExternalData. ExternalData is like Collection (or Array).

Best,
Marcel

Am 22.06.2020 10:21:23 schrieb Nicolas Cellier <[hidden email]>:

Hi Marcel,
The idea is to manipulate the alias or structures types as if they were classes.

I already can write MyStruct new, and pass that to an external function call.
I'd like to simply write (MyStruct new: 25) to get an array of 25 MyStruct.
I DO NOT want to see the ExternalData, it's an implementation detail blurring the intentions.
I want to see/handle my domain specific "objects" DIRECTLY.
That's what I manipulate in function signatures, etc...

Since external types are not exactly classes, i prefer to have another vocabulary in place of new:.
In VW DLLCC, the selectors malloc malloc: calloc calloc: gcMalloc gcMalloc: are sent to the CType.

Unlike us, DLLCC does not allocate in objectMemory, always on the heap.
So having the C vocabulary is logical for them.
Allocating in ObjectMemory is one essential feature of Squeak FFI.
That's why I prefer not use C vocabulary.
#allocate: is something in between.
I know that it is already used on the low level for ExternalAddress, which is unfortunate.



Le lun. 22 juin 2020 à 09:31, Marcel Taeumel <[hidden email]> a écrit :
Hi Nicolas.

I like the idea of having #allocate: ... but maybe not in ExternalType:

- The term "allocate:" is somewhat reserved for heap memory from the SqueakFFI perspective; See ExternalAddress class >> #allocate:.
- The counterpart #free is missing ... which means that, maybe, both #allocate: and #free should be implemented in ExternalData, not ExternalType

What about such a this programming interface:

data := ExternalData
   fromHandle: nil
   type: MyStruct externalType asPointerType.
data allocate.
myStruct := data asExternalStructure.

You could also copy it back to object memory:

myStruct := data getExternalStructure. "get/pull/fetch ...."
data free.

You are proposing a way to start off in object memory. Maybe like this:

data := ExternalData
   fromHandle: nil "maybe optional?"
   type: MyStruct externalType asPointerType.
data allocateLocal.
myStruct := data asExternalStructure.

Then we don't need to call #free. See how #allocate and #allocateLocal in ExternalData could mirror #new and #externalNew of ExternalStructure?

There is really an interesting releationship between ExternalStructure (incl. ExternalUnion/ExternalTypeAlias/ExternalPackedStructure) and ExternalData. Can you see it? ^__^

Arrays could be similar:

data := ExternalData type: Externaltype int32_t asPointerType.
data size: 20.
data allocate. "or #allocateLocal"

Now you would have an array of 20 integers. :-) Note that #allocate: with argument could inline that #size: call.

Best,
Marcel

Am 21.06.2020 13:20:44 schrieb [hidden email] <[hidden email]>:

Nicolas Cellier uploaded a new version of FFI-Kernel to project FFI:
http://source.squeak.org/FFI/FFI-Kernel-nice.118.mcz

==================== Summary ====================

Name: FFI-Kernel-nice.118
Author: nice
Time: 21 June 2020, 1:20:34.369348 pm
UUID: a8c48997-991d-4e32-9b67-6d93fc0dabe0
Ancestors: FFI-Kernel-mt.117

Add the ability to allocate: an ExternalType or an ExternalStructure/Alias.

It's equivalent to new: but we want a specific idiom, C types are not exactly classes.

Add the ability to compare aliases (at least for equality)

=============== Diff against FFI-Kernel-mt.117 ===============

Item was added:
+ ----- Method: ExternalStructure class>>allocate: (in category 'instance creation') -----
+ allocate: anInteger
+ "Create an ExternalData with enough room for storing an array of size anInteger of such structure"
+ ^self externalType allocate: anInteger!

Item was added:
+ ----- Method: ExternalType>>allocate: (in category 'external data') -----
+ allocate: anInteger
+ "Allocate space for containing an array of size anInteger of this dataType"
+
+ | handle |
+ handle := ByteArray new: self byteSize * anInteger.
+ ^(ExternalData fromHandle: handle type: self) size: anInteger!

Item was added:
+ ----- Method: ExternalTypeAlias>>= (in category 'comparing') -----
+ = anExternalTypeAlias
+ ^self class = anExternalTypeAlias class and: [self value = anExternalTypeAlias value]!

Item was added:
+ ----- Method: ExternalTypeAlias>>hash (in category 'comparing') -----
+ hash
+ ^self class hash hashMultiply bitXor: self value hash!








Reply | Threaded
Open this post in threaded view
|

SWIG for Squeak/Cuis?

Stephen Pope
In reply to this post by marcel.taeumel

Hi all,

Does anyone else want (or miss) the Smalltalk SWIG interface?  

SWIG is a software development tool that connects libraries written in C and C++ with high-level programming languages; see http://swig.org. It is supported by most of the cerrent “scripting” languages.

There’s code in the VisualWorks repo that’s really stale and relies on a set of compiler extensions.  It worked great a few releases ago.

It would be useful for a range of apps (like mine) to be able to call out to C/C++ libraries via auto-generated OO APIs.

stp


--

                    Stephen Travis Pope   Santa Barbara, California, USA    
          http://HeavenEverywhere.com        http://FASTLabInc.com





Reply | Threaded
Open this post in threaded view
|

Re: SWIG for Squeak/Cuis?

Jakob Reschke-2
I would use it if it existed. Interfacing with C libraries should be easier. Marcel is currently improving the FFI, but a binding generator I have not seen to far.


[hidden email] <[hidden email]> schrieb am Mo., 22. Juni 2020, 19:13:

Hi all,

Does anyone else want (or miss) the Smalltalk SWIG interface?  

SWIG is a software development tool that connects libraries written in C and C++ with high-level programming languages; see http://swig.org. It is supported by most of the cerrent “scripting” languages.

There’s code in the VisualWorks repo that’s really stale and relies on a set of compiler extensions.  It worked great a few releases ago.

It would be useful for a range of apps (like mine) to be able to call out to C/C++ libraries via auto-generated OO APIs.

stp


--

                    Stephen Travis Pope   Santa Barbara, California, USA    
          http://HeavenEverywhere.com        http://FASTLabInc.com







pastedGraphic.tiff (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: SWIG for Squeak/Cuis?

David T. Lewis
I remember discussing this a few times in earlier years. I'm not entirely
sure why none of us ever got around to implementing it for Squeak/Cuis,
but it still sounds like a good idea to me.

Dave

On Mon, Jun 22, 2020 at 08:24:15PM +0200, Jakob Reschke wrote:

> I would use it if it existed. Interfacing with C libraries should be
> easier. Marcel is currently improving the FFI, but a binding generator I
> have not seen to far.
>
>
> [hidden email] <[hidden email]> schrieb am Mo.,
> 22. Juni 2020, 19:13:
>
> >
> > Hi all,
> >
> > Does anyone else want (or miss) the Smalltalk SWIG interface?
> >
> > SWIG is a software development tool that connects libraries written in C
> > and C++ with high-level programming languages; see http://swig.org. It is
> > supported by most of the cerrent ???scripting??? languages.
> >
> > There???s code in the VisualWorks repo that???s really stale and relies on a
> > set of compiler extensions.  It worked great a few releases ago.
> >
> > It would be useful for a range of apps (like mine) to be able to call out
> > to C/C++ libraries via auto-generated OO APIs.
> >
> > stp
> >
> >
> > --
> >
> >                     Stephen Travis Pope   Santa Barbara, California, USA
> >
> >           http://HeavenEverywhere.com        http://FASTLabInc.com
> >                        https://vimeo.com/user19434036/videos      *http://heaveneverywhere.com/Reflections
> > <http://heaveneverywhere.com/Reflections>*
> >
> >
> >
> >
> >


>


Reply | Threaded
Open this post in threaded view
|

Re: FFI: FFI-Kernel-nice.118.mcz

Nicolas Cellier
In reply to this post by Nicolas Cellier


Le lun. 22 juin 2020 à 15:20, Nicolas Cellier <[hidden email]> a écrit :


Le lun. 22 juin 2020 à 12:11, Marcel Taeumel <[hidden email]> a écrit :
For the generalization to other atomic types, we indeed need to go through the ExternalType and implement allocate: there too.

Not needed. You can access ExternalType >> #byteSize from within ExternalData. It's public.

Best,
Marcel

Hi Marcel,
you constantly bring me to a lower level API ...
I know that byteSize is public, but I don't want to bother with that.
I want to handle an array of N things, not an ExternalData withHandle: (ByteArray new: thing byteSize * N) type: thing.
I want (thing allocate: N), that's all I need...
I've used similar constructs provided by VW DLLCC, and believe me, it's convenient!

Beside, think of it, an ExternalData has to provide both memory AND type right?
    ExternalData fromHandle: (ByteArray new: type byteSize * anInteger) type: type.
Who knows better about its own size than the type itself?
Why scatter the responsibilities that could be gathered in the type?
Where would this snippet go?

The funny thing is that I refactored some dusty parts today, so we can flesh out this discussion:

Name: HDF5-Interface-nice.10
Time: 22 June 2020, 6:13:15.137449 pm

The old version using byteSize was:

dimensions
     "Answer an array of integers representing the dimensions of this dataspace.
     Note that a dimension -1 means unlimited."
     | ndims effective hdims maxDims bsize method |
     ndims := self ndims.
     bsize := Hsize_t byteSize.
    method := #(unsignedByteAt: unsignedShortAt: unsignedLongAt: unsignedLongLongAt:) at: bsize highBit.
    hdims := ByteArray new: ndims * bsize.
    maxDims := ByteArray new: ndims * bsize.
    effective := API H5Sget_simple_extent_dims: handle with: hdims with: maxDims.
    effective = ndims ifFalse: [self error: 'unexpected effective dimensionality'].
    ^(0 to: ndims - 1) collect: [:i | hdims perform: method with: i * bsize + 1]

I don't remember if ByteArray did work without using ExternalData wrapper around it, this could only be worse!
The new one is using allocate: which create an ExternalData and deal with all the low level ByteArray container, type byteSize etc...
It also uses the collection-like protocol on the external data (hdims at:)

dimensions
     "Answer an array of integers representing the dimensions of this dataspace.
     Note that a dimension -1 means unlimited."
     | ndims effective hdims maxDims  |
     ndims := self ndims.
     hdims := Hsize_t allocate: ndims.
     maxDims := Hsize_t allocate: ndims.
     effective := API H5Sget_simple_extent_dims: handle with: hdims with: maxDims.
     effective = ndims ifFalse: [self error: 'unexpected effective dimensionality'].
     ^(1 to: ndims) collect: [:i | (hdims at: i) value]

We can all agree that the later is better.
I have the ExternalData playing its role in the best of the manners:
Tansparently! Exactly like ExternalFunction.
A great economy of verbiage - unlike this thread ;)
So thank you Marcel for enhancing those API, let me just add my grain of salt!

The only annoying thing is the value which unwraps the Hsize_t ExternalTypeAlias value.
If I would replace Hsize_t with a simple ExternalType hsize_t, then
1) I would omit the value
2) I would like to write
     hdims := ExternalType hsize_t allocate: ndims.
Because it's conceptually the same, allocate an array (ExternalData) of given type.
That's why I implement allocate: in ExternalType if ever you are still not convinced.