FFI ExternalTypeAlias

Hi Nicolas, hi all! :-)

> Type alias are subclasses of ExternalStructure.

And with it, they get their own instance of ExternalType, which is managed in the classVar StructTypes. Let's call them struct type.

> Now we have another mean by adding a class side method in ExternalType.

That kind of aliasing is for compile-time type referencing only -- such as in FFI calls (i.e. <apicall:...> pragmas) or struct-field definitions (i.e. #fields). (And soon <callback: ...> if I have the first version of FFI-Callback to show you :)

Because those aliases become fully transparent after compiling the method for the FFI call into an instance of ExternalLibraryFunction and after compiling the field specs of external structures into compiledSpec for struct types. 

For example, on my machine in 32-bit Squeak, a 'size_t' becomes a 'long', which is Squeak FFI term for 'int'. :-) See #ffiLongVsInt. And 'int32_t' also becomes a 'long'. So, it just translates C vocabulary into Squeak FFI vocabulary. Nothing more. Nothing less.

Having that, please do not use class-side methods in ExternalType to alias a (complex) struct type. Use them only for renaming atomic types. Please. We may want to add checks to enforce that.

I suppose that this discussion focuses on type aliases that are represented as subclasses of ExternalStructure (or ExternalTypeAlias) and thus get their own struct type. So, let's ignore this other mechanism for now.

> This has one advantage: type safety. You cannot pass a Foo to a function expecting a Bar, even if they both alias int...

Yes! So, we are now talking about type aliases to atomic types. The struct type you get when aliasing a 'char' as Foo, for example, looks like this:

(...forgive my "creative" representation of WordArray here...)

compiledSpec: 0A 04 00 01

referentClass: Foo

How does the atomic type for 'char' look like?

compiledSpec: 0A 04 00 01

referentClass: nil

Consequently, argument coercing will fail if you pass a 'char' when a 'Foo' is expected. Because the referentClass does not match --- even if the compiledSpec does match.

> But this has one major drawback: with current FFI, you cannot pass a Smalltalk Integer, to a method expecting a Bar, even if Bar is an alias for an integer type...

That's correct. If the origin of such an argument is from within the image, you have to wrap it. But when returned from another call ... well ... that wrapping should have happened in the plugin ... But read on! :-)

> FFI plugin is clever enough to recognize an atomic type alias and simply return a Smalltalk Integer...

> So it's not Bar all the way down, I have to manually wrap Bar...

> This is not consistent.

Uhh! That's a bug. The FFI Plugin must take a look at referentClass because it does so for argument coercing.

Now I understand why the code generation of struct-field accessors was so apparently broken for me. I changed that so that having an alias type as part of a larger struct will automatically wrap that for you into the alias structure. :-) Even if you are just aliasing a plain 'int' ... or 'long' ... Ha ha. ;-) #ffiLongVsInt.

It should be "Bar all the way down". Definitely.

> If we return an int, we must accept an int, otherwise, we cannot chain FFI calls...

Well, not for type aliases. I would like keep the path of type safety here, you mentioned above.

Just fix the FFI plugin to respect referentClass when packaging the return value.

> I could use the lightweight alias instead, [...]

Please don't. See above. Those "lightweight alias" are for renaming atomic types only. Let's keep it simple and try to address you concern here with struct types and ExternalTypeAlias. :-)

> there was another usage where it was convenient to use ExternalTypeAlias: enum.

> Indeed, I simply defined all enum symbols as class side method. 

I like that! It reads like a next step for ExternalPool. Once you have extracted the constant values from header files, and once you have those values for your platform in classVars in your external pool, use that pool to define enums through ExternalStructure.

Like this:

ExternalTypeAlias subclass: #MyEnumBar

instanceVariableNames: ''

classVariableNames: ''

poolDictionaries: 'HDF5Pool'

category: 'HDF5'

MyEnumBar class >> #poolFields

   "

   self defineFields.

   "

   ^ #(

      (beg HDF5_BEG)

      (mid HDF5_MID)

      (end HDF5_END)

   )

Note that HDF5_BEG etc. are from the HDF5Pool, which is managed via external-pool definitions. See class comment in ExternalPool to get started.

Note that #poolFields would translate to class-side methods as you suggested.

> If I use a simpler alias, where are those ids going to?

Please don't. See above. :-)

> IMO we cannot keep status quo in the plugin, we gotta to make the inputs/outputs behave consistently.

Absolutely!

> - should a function expecting an ExternalTypeAlias of atomic type accept an immediate value of compatible type?

Considering type safety, I think not.

Well ... since this is a convenience issue and thus not about saving run-time cost ... What about adding a callback into the image to react on FFIErrorCoercionFailed? Maybe the selector for such a callback could be stored in ExternalLibraryFunction since this is accessible to the plugin anyway?

<apicall: void 'foo' (MyInt YourInt in) ifFailCoerceVia: #wrapInt: >

... would it be possible? Like that #doesNotUnderstand: callback?

I would love to do ad-hoc packaging of arguments. (Also thinking about FFI-Callback. But ignore me here :)

> - should a function returning an ExternalTypeAlias of atomic type instantiate the Alias?

Yes, please! See above.

If you want to trade type safety in for a performance gain, just use a "lightweight alias" as explained above. And package that return value manually.

Best,

Marcel

Am 18.06.2020 21:03:53 schrieb Nicolas Cellier <[hidden email]>:

Hi all,

following the question of Marcel about usage of ExternalTypeAlias:

Type alias are subclasses of ExternalStructure.

I used them in HDF5 because there was no other means to define an alias at the time.

Now we have another mean by adding a class side method in ExternalType (I would have rather imagined the usage of some annotation to get a declarative style)

This has one advantage: type safety. You cannot pass a Foo to a function expecting a Bar, even if they both alias int...

But this has one major drawback: with current FFI, you cannot pass a Smalltalk Integer, to a method expecting a Bar, even if Bar is an alias for an integer type...

Thus, you have to wrap every Bar argument into a Bar instance...

Gasp, this is going too far...

What about functions returning such alias?

Function returning struct by value allocate a struct, but it's not the case here, FFI plugin is clever enough to recognize an atomic type alias and simply return a Smalltalk Integer...

So it's not Bar all the way down, I have to manually wrap Bar...

This is not consistent.

If we return an int, we must accept an int, otherwise, we cannot chain FFI calls...

I could use the lightweight alias instead, but there was another usage where it was convenient to use ExternalTypeAlias: enum.

Indeed, I simply defined all enum symbols as class side method. For example for enum bar { beg=0, mid=1, end=2 }; I simply create a type Bar aliasing int and having class side methods beg ^0, mid ^1, end ^2.

This way, I normally can pass a Bar mid to an external function.

(currently, we must defne mid ^Bar new value: 1; yourself)

If I use a simpler alias, where are those ids going to?

IMO we cannot keep statu quo in the plugin, we gotta to make the inputs/outputs behave consistently.

 The choice is this one:

- should a function expecting an ExternalTypeAlias of atomic type accept an immediate value of compatible type? (the permissive implementation)

- should a function returning an ExternalTypeAlias of atomic type instantiate the Alias? (the typesafe implementation, with higher runtime cost due to pressure on GC)