FFI: FFI-Kernel-mt.101.mcz

Marcel Taeumel uploaded a new version of FFI-Kernel to project FFI:
http://source.squeak.org/FFI/FFI-Kernel-mt.101.mcz

==================== Summary ====================

Name: FFI-Kernel-mt.101
Author: mt
Time: 10 June 2020, 6:10:37.422765 pm
UUID: a5283cbb-1343-6f45-85c9-d434c38a88ac
Ancestors: FFI-Kernel-mt.100

Adds a mechanism for lightweight type aliases for atomic types by looking at the messages of ExternalType class for a referenced selector. This has entails no overhead for FFI calls, only a tiny one when compiling struct fields and methods that do FFI calls. Because that's when type names are looked up to find actual instances of ExternalType.

Adds such lightweight aliases for standard typedefs from stddef.h and stdint.h. This includes size_t (and similar), which makes a dynamic dispatch on the platform's #wordSize. Consequently, we have to re-compile all FFI calls when a platform change is detected, which now happens.

Adds more lightweight aliases to address the #ffiLongVsInt problem in Squeak FFI. You can now use 'int' and 'uint' etc. and it will correctly map to 'long' and 'ulong'. But you may be better off using 'int32_t' and 'uint32_t'; see above.

(Note that there is still no convenient alias for the platform-specific C long type yet.)

=============== Diff against FFI-Kernel-mt.100 ===============

Item was changed:
  ----- Method: ExternalStructure class>>platformChangedFrom:to: (in category 'system startup') -----
  platformChangedFrom: lastPlatform to: currentPlatform
+ "The system is coming up on a new platform. Compile all field specs and install them in the respective struct types."
+
- "The system is coming up on a new platform. Clear out the existing handles."
  self compileAllFields.!

Item was changed:
+ ----- Method: ExternalStructure>>externalType (in category 'accessing') -----
- ----- Method: ExternalStructure>>externalType (in category 'converting') -----
  externalType
 
+ | type |
+ self flag: #ffiDesignSmell. "mt: Note that type aliases to pointer types store pointers via handle as ByteArray, not ExternalAddress."
+ type := self class externalType.
+ ^ handle class == ExternalAddress
+ ifTrue: [type asPointerType]
+ ifFalse: [type]!
- ^ self class externalType!

Item was added:
+ ----- Method: ExternalType class>>int (in category 'type constants') -----
+ int
+
+ self flag: #ffiLongVsInt.
+ ^ self long!

Item was added:
+ ----- Method: ExternalType class>>int16_t (in category 'type constants - stdint.h') -----
+ int16_t
+
+ ^ self signedShort!

Item was added:
+ ----- Method: ExternalType class>>int32_t (in category 'type constants - stdint.h') -----
+ int32_t
+
+ ^ self signedInt!

Item was added:
+ ----- Method: ExternalType class>>int64_t (in category 'type constants - stdint.h') -----
+ int64_t
+
+ ^ self signedLongLong!

Item was added:
+ ----- Method: ExternalType class>>int8_t (in category 'type constants - stdint.h') -----
+ int8_t
+
+ ^ self signedByte!

Item was added:
+ ----- Method: ExternalType class>>intptr_t (in category 'type constants - stddef.h') -----
+ intptr_t
+ "Answer a signed integer type that can hold a data pointer on the current platform, that is, 4-byte pointers on 32-bit und 8-byte pointers on 64-bit. Unlike actual C pointers (e.g., void*), you can do (pointer) arithmetic (e.g., #+) and bitwise manipulation (e.g., #bitOr:) on the resulting Smalltalk Integer objects.
+
+ Note that the equivalent type for a void* yields an instance if ExternalData, whose handle is an ExternalAddress, which actually impelements #+ as of June 2020. So, you can also do pointer arithmetic using that type.
+
+ Note that all FFI calls and struct types will be updated automatically on startup if a platform change is detected."
+
+ ^ FFIPlatformDescription current wordSize = 4
+ ifTrue: [self signedInt]
+ ifFalse: [self signedLongLong]!

Item was changed:
  ----- Method: ExternalType class>>long (in category 'type constants') -----
  long
+
+ self flag: #ffiLongVsInt.
+ ^ self signedLong!
- ^self signedLong!

Item was added:
+ ----- Method: ExternalType class>>longlong (in category 'type constants') -----
+ longlong
+ ^ self signedLongLong!

Item was changed:
  ----- Method: ExternalType class>>platformChangedFrom:to: (in category 'system startup') -----
  platformChangedFrom: lastPlatform to: currentPlatform
  "Byte size or byte alignment for atomic types might be different on the new platform."
 
+ "1) Update all atomic types for platform-specifc byte size and alignment."
  self initializeAtomicTypes.
+
+ "2) Discard all compiled specs for all structure types."
+ self initializeStructureTypes.
+
+ "3) Update all type-name mappings for all FFI calls."
+ lastPlatform wordSize ~= currentPlatform wordSize
+ ifTrue: [self recompileAllLibraryFunctions].!
- self initializeStructureTypes.!

Item was added:
+ ----- Method: ExternalType class>>ptrdiff_t (in category 'type constants - stddef.h') -----
+ ptrdiff_t
+
+ ^ self intptr_t!

Item was added:
+ ----- Method: ExternalType class>>recompileAllLibraryFunctions (in category 'housekeeping') -----
+ recompileAllLibraryFunctions
+ "Recompile all methods that do FFI calls (e.g. <apicall ... > or <cdecl ... >) to update all mappings from type name to atomic type and struct type. Note that unknown struct types will be created on-the-fly and can later be completed by defining fields in the particular structure class via #defineFields or #compileFields. Note that such a recompilation is especially useful if 'type constants' for atomic types to additional dispatch such as according to the current platform's #wordSize."
+
+ SystemNavigation default allSelectorsAndMethodsDo: [:behavior :selector :method |
+ method externalLibraryFunction ifNotNil: [behavior recompile: selector]].!

Item was changed:
  ----- Method: ExternalType class>>resetAllStructureTypes (in category 'housekeeping') -----
  resetAllStructureTypes
  "Warning: This call is only required if you change the container for StructTypes!! Note that (2) and (3) can be swapped but that puts unnecessary pressure on the GC."
 
  StructTypes := nil.
 
  "1) Initialize the container for structure types."
  self initializeStructureTypes.
 
  "2) Recompile all FFI calls to create and persist structure types."
+ self recompileAllLibraryFunctions.
- SystemNavigation default allSelectorsAndMethodsDo: [:behavior :selector :method |
- method externalLibraryFunction ifNotNil: [behavior recompile: selector]].
 
  "3) Update all structure types' spec and alignment."
  ExternalStructure compileAllFields.
  !

Item was added:
+ ----- Method: ExternalType class>>signedInt (in category 'type constants') -----
+ signedInt
+
+ self flag: #ffiLongVsInt.
+ ^ self signedLong!

Item was changed:
  ----- Method: ExternalType class>>signedLong (in category 'type constants') -----
  signedLong
+
+ self flag: #ffiLongVsInt.
  ^AtomicTypes at: 'long'!

Item was added:
+ ----- Method: ExternalType class>>size_t (in category 'type constants - stddef.h') -----
+ size_t
+
+ ^ self uintptr_t!

Item was changed:
  ----- Method: ExternalType class>>string (in category 'type constants') -----
  string
+ ^ self char asPointerType!
- ^(AtomicTypes at: 'char') asPointerType!

Item was added:
+ ----- Method: ExternalType class>>uint (in category 'type constants') -----
+ uint
+
+ self flag: #ffiLongVsInt.
+ ^ self ulong!

Item was added:
+ ----- Method: ExternalType class>>uint16_t (in category 'type constants - stdint.h') -----
+ uint16_t
+
+ ^ self unsignedShort!

Item was added:
+ ----- Method: ExternalType class>>uint32_t (in category 'type constants - stdint.h') -----
+ uint32_t
+
+ ^ self unsignedInt!

Item was added:
+ ----- Method: ExternalType class>>uint64_t (in category 'type constants - stdint.h') -----
+ uint64_t
+
+ ^ self unsignedLongLong!

Item was added:
+ ----- Method: ExternalType class>>uint8_t (in category 'type constants - stdint.h') -----
+ uint8_t
+
+ ^ self unsignedByte!

Item was added:
+ ----- Method: ExternalType class>>uintptr_t (in category 'type constants - stddef.h') -----
+ uintptr_t
+ "Answer an unsigned integer type that can hold a data pointer on the current platform, that is, 4-byte pointers on 32-bit und 8-byte pointers on 64-bit. Unlike actual C pointers (e.g., void*), you can do (pointer) arithmetic (e.g., #+) and bitwise manipulation (e.g., #bitOr:) on the resulting Smalltalk Integer objects.
+
+ Note that the equivalent type for a void* yields an instance if ExternalData, whose handle is an ExternalAddress, which actually impelements #+ as of June 2020. So, you can also do pointer arithmetic using that type.
+
+ Note that all FFI calls and struct types will be updated automatically on startup if a platform change is detected."
+
+ ^ FFIPlatformDescription current wordSize = 4
+ ifTrue: [self unsignedInt]
+ ifFalse: [self unsignedLongLong]!

Item was changed:
  ----- Method: ExternalType class>>ulong (in category 'type constants') -----
  ulong
+
+ self flag: #ffiLongVsInt.
+ ^ self unsignedLong!
- ^self unsignedLong!

Item was added:
+ ----- Method: ExternalType class>>ulonglong (in category 'type constants') -----
+ ulonglong
+ ^ self unsignedLongLong!

Item was added:
+ ----- Method: ExternalType class>>unsignedInt (in category 'type constants') -----
+ unsignedInt
+
+ self flag: #ffiLongVsInt.
+ ^ self unsignedLong!

Item was changed:
  ----- Method: ExternalType class>>unsignedLong (in category 'type constants') -----
  unsignedLong
+
+ self flag: #ffiLongVsInt.
+ ^ AtomicTypes at: 'ulong'!
- ^AtomicTypes at: 'ulong'!