Re: [squeak-dev] 64 bit FFI (was: porting Croquet to Squeak6.0 alpha...)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] 64 bit FFI (was: porting Croquet to Squeak6.0 alpha...)

Nicolas Cellier
 
Hi all,
beware of FFI 'long' and 'ulong'.
They are currently meaning (32 bits) int and unsigned int!
This dates back from the 80s when there were still odd OS with segmented memory and 16-bits int.
long was thought as a safe int32_t at that time... and still means just that in current FFI.

I'd like to replace them with 'int' and 'uint', but we need to ensure backward compatibility etc...

FFI has ExternalType signedLongLong or just 'longlong' and ExternalType unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.
See implementors of #initializeAtomicTypes #initializeFFIConstants

If you want a type that can hold a pointer on both 32 and 64bits arch, then it's doable with a FFI type alias
For example I have things like this in HDF5 module
Notice the single #(fieldName fieldType) which defines an alias, while ordinary FFI struct gets an array of #(fieldName fieldType)
(and replace Size_t with Intptr_t if the purpose is hodling a pointer, otherwise the ayatollahs of C standard will lash you)

ExternalStructure subclass: #'Size_t'
    instanceVariableNames: ''
    classVariableNames: ''
    poolDictionaries: ''
    category: 'HDF5-External-atomic'

Size_t class >>fields
    "Size_t defineFields"
    ^Smalltalk wordSize = 4
        ifTrue: [#( value 'ulong')]
        ifFalse: [#( value 'ulonglong')]

However the "fields" must be re-generated when platform changes...
I tried to automate that sometimes ago, see:

ExternalStructure class>>install
     "Resuming the image on another architecture may require a re-compilation of structure layout."
     | newPlatform |
     newPlatform := Smalltalk platformName.
     PreviousPlatform = newPlatform
         ifFalse:
             [self recompileStructures.
            PreviousPlatform := newPlatform]

It works if you switch from Win64 to MacOS64 for example...
However, this mechanism is not used yet at time of code loading, so importing code generated on a different platform won't automatically update the structures (or alias) - the FFI package has no mechanism for that (maybe we could add a hook in MC?).

Le mer. 11 mars 2020 à 23:58, Bert Freudenberg <[hidden email]> a écrit :
I'd suggest to get OpenGL working outside of Croquet first:


Step 1: Verify this works in 32 bits. (assuming you are doing this on Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)
Step 2: Make it work in 64 bits.

The second step requires that you understand how FFI works, and how it handles e.g. pointers and integer sizes.
I am assuming we do have a working 64 bit FFI, at least for x86_64 machines.

E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a 32 bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit system, a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I don't know how many bits 'ulong' has in our 64 bit FFI, but that declaration may have to change. Etc. pp.

If you have questions about FFI then those are best directed at the vm-dev list since it is dealing with VM-level interfaces. CC'ing, please follow up there.

- Bert -

On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <[hidden email]> wrote:
Okey dokey,

Poking along, there is a stray glyph in OGLUnix openGLLibraryName after
openGLLibraryName
^Smalltalk osVersion = 'linux'
ifTrue: ['libGL.so.1']
ifFalse: ['GL']
I removed it in my install and got past that error.

Working exclusively with Croquet(Master)...



My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode: 
glMatrixMode: mode
"This method was automatically generated."
"void glMatrixMode(GLenum mode);"
<apicall: void 'glMatrixMode' (ulong) module: '#openGLLibraryName'>
^self externalCallFailed
The <apicall:...> fails

How to think about this?

Is Croquet behind OpenGL latest?
Would teaching myself OpenGL programming be of use to the Croquet project?

cheers,

tty



Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] 64 bit FFI (was: porting Croquet to Squeak6.0 alpha...)

marcel.taeumel
 
Hi Nicolas,

what is the role of ExternalStructure class >> #pointerSize then?

Best,
Marcel


Nicolas Cellier wrote

> Hi all,
> beware of FFI 'long' and 'ulong'.
> They are currently meaning (32 bits) int and unsigned int!
> This dates back from the 80s when there were still odd OS with segmented
> memory and 16-bits int.
> long was thought as a safe int32_t at that time... and still means just
> that in current FFI.
>
> I'd like to replace them with 'int' and 'uint', but we need to ensure
> backward compatibility etc...
>
> FFI has ExternalType signedLongLong or just 'longlong' and ExternalType
> unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.
> See implementors of #initializeAtomicTypes #initializeFFIConstants
>
> If you want a type that can hold a pointer on both 32 and 64bits arch,
> then
> it's doable with a FFI type alias
> For example I have things like this in HDF5 module
> Notice the single #(fieldName fieldType) which defines an alias, while
> ordinary FFI struct gets an array of #(fieldName fieldType)
> (and replace Size_t with Intptr_t if the purpose is hodling a pointer,
> otherwise the ayatollahs of C standard will lash you)
>
> ExternalStructure subclass: #'Size_t'
>     instanceVariableNames: ''
>     classVariableNames: ''
>     poolDictionaries: ''
>     category: 'HDF5-External-atomic'
>
> Size_t class >>fields
>     "Size_t defineFields"
>     ^Smalltalk wordSize = 4
>         ifTrue: [#( value 'ulong')]
>         ifFalse: [#( value 'ulonglong')]
>
> However the "fields" must be re-generated when platform changes...
> I tried to automate that sometimes ago, see:
>
> ExternalStructure class>>install
>      "Resuming the image on another architecture may require a
> re-compilation of structure layout."
>      | newPlatform |
>      newPlatform := Smalltalk platformName.
>      PreviousPlatform = newPlatform
>          ifFalse:
>              [self recompileStructures.
>             PreviousPlatform := newPlatform]
>
> It works if you switch from Win64 to MacOS64 for example...
> However, this mechanism is not used yet at time of code loading, so
> importing code generated on a different platform won't automatically
> update
> the structures (or alias) - the FFI package has no mechanism for that
> (maybe we could add a hook in MC?).
>
> Le mer. 11 mars 2020 à 23:58, Bert Freudenberg &lt;

> bert@

> &gt; a
> écrit :
>
>> I'd suggest to get OpenGL working outside of Croquet first:
>>
>> http://www.squeaksource.com/CroquetGL.html
>>
>> Step 1: Verify this works in 32 bits. (assuming you are doing this on
>> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)
>> Step 2: Make it work in 64 bits.
>>
>> The second step requires that you understand how FFI works, and how it
>> handles e.g. pointers and integer sizes.
>> I am assuming we do have a working 64 bit FFI, at least for x86_64
>> machines.
>>
>> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a 32
>> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit
>> system,
>> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I
>> don't know how many bits 'ulong' has in our 64 bit FFI, but that
>> declaration may have to change. Etc. pp.
>>
>> If you have questions about FFI then those are best directed at the
>> vm-dev
>> list since it is dealing with VM-level interfaces. CC'ing, please follow
>> up
>> there.
>>
>> - Bert -
>>
>> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <
>>

> squeak-dev@.squeakfoundation

>> wrote:
>>
>>> Okey dokey,
>>>
>>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName after
>>>
>>> openGLLibraryName
>>> ^Smalltalk osVersion = 'linux'
>>> ifTrue: ['libGL.so.1']
>>> ifFalse: ['GL']
>>>
>>> I removed it in my install and got past that error.
>>>
>>> Working exclusively with Croquet(Master)...
>>>
>>>
>>>
>>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:
>>>
>>> glMatrixMode: mode
>>> "This method was automatically generated."
>>> "void glMatrixMode(GLenum mode);"
>>>
> <apicall: void 'glMatrixMode' (ulong) module: '#openGLLibraryName'>
>>> ^self externalCallFailed
>>>
>>> The
> <apicall:...>
>  fails
>>>
>>> How to think about this?
>>>
>>> Is Croquet behind OpenGL latest?
>>> Would teaching myself OpenGL programming be of use to the Croquet
>>> project?
>>>
>>> cheers,
>>>
>>> tty
>>>
>>>
>>>
>>





--
Sent from: http://forum.world.st/Squeak-VM-f104410.html
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] 64 bit FFI (was: porting Croquet to Squeak6.0 alpha...)

Nicolas Cellier
 
Hi Marcel,
no idea...
Look at timestamp:
eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set FFI-Kernel-EstebanLorenzano.45  ·
So presumably Eliot added that.

Le jeu. 28 mai 2020 à 17:00, marcel.taeumel <[hidden email]> a écrit :
 
Hi Nicolas,

what is the role of ExternalStructure class >> #pointerSize then?

Best,
Marcel


Nicolas Cellier wrote
> Hi all,
> beware of FFI 'long' and 'ulong'.
> They are currently meaning (32 bits) int and unsigned int!
> This dates back from the 80s when there were still odd OS with segmented
> memory and 16-bits int.
> long was thought as a safe int32_t at that time... and still means just
> that in current FFI.
>
> I'd like to replace them with 'int' and 'uint', but we need to ensure
> backward compatibility etc...
>
> FFI has ExternalType signedLongLong or just 'longlong' and ExternalType
> unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.
> See implementors of #initializeAtomicTypes #initializeFFIConstants
>
> If you want a type that can hold a pointer on both 32 and 64bits arch,
> then
> it's doable with a FFI type alias
> For example I have things like this in HDF5 module
> Notice the single #(fieldName fieldType) which defines an alias, while
> ordinary FFI struct gets an array of #(fieldName fieldType)
> (and replace Size_t with Intptr_t if the purpose is hodling a pointer,
> otherwise the ayatollahs of C standard will lash you)
>
> ExternalStructure subclass: #'Size_t'
>     instanceVariableNames: ''
>     classVariableNames: ''
>     poolDictionaries: ''
>     category: 'HDF5-External-atomic'
>
> Size_t class >>fields
>     "Size_t defineFields"
>     ^Smalltalk wordSize = 4
>         ifTrue: [#( value 'ulong')]
>         ifFalse: [#( value 'ulonglong')]
>
> However the "fields" must be re-generated when platform changes...
> I tried to automate that sometimes ago, see:
>
> ExternalStructure class>>install
>      "Resuming the image on another architecture may require a
> re-compilation of structure layout."
>      | newPlatform |
>      newPlatform := Smalltalk platformName.
>      PreviousPlatform = newPlatform
>          ifFalse:
>              [self recompileStructures.
>             PreviousPlatform := newPlatform]
>
> It works if you switch from Win64 to MacOS64 for example...
> However, this mechanism is not used yet at time of code loading, so
> importing code generated on a different platform won't automatically
> update
> the structures (or alias) - the FFI package has no mechanism for that
> (maybe we could add a hook in MC?).
>
> Le mer. 11 mars 2020 à 23:58, Bert Freudenberg &lt;

> bert@

> &gt; a
> écrit :
>
>> I'd suggest to get OpenGL working outside of Croquet first:
>>
>> http://www.squeaksource.com/CroquetGL.html
>>
>> Step 1: Verify this works in 32 bits. (assuming you are doing this on
>> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)
>> Step 2: Make it work in 64 bits.
>>
>> The second step requires that you understand how FFI works, and how it
>> handles e.g. pointers and integer sizes.
>> I am assuming we do have a working 64 bit FFI, at least for x86_64
>> machines.
>>
>> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a 32
>> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit
>> system,
>> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I
>> don't know how many bits 'ulong' has in our 64 bit FFI, but that
>> declaration may have to change. Etc. pp.
>>
>> If you have questions about FFI then those are best directed at the
>> vm-dev
>> list since it is dealing with VM-level interfaces. CC'ing, please follow
>> up
>> there.
>>
>> - Bert -
>>
>> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <
>>

> squeak-dev@.squeakfoundation

>> wrote:
>>
>>> Okey dokey,
>>>
>>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName after
>>>
>>> openGLLibraryName
>>> ^Smalltalk osVersion = 'linux'
>>> ifTrue: ['libGL.so.1']
>>> ifFalse: ['GL']
>>>
>>> I removed it in my install and got past that error.
>>>
>>> Working exclusively with Croquet(Master)...
>>>
>>>
>>>
>>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:
>>>
>>> glMatrixMode: mode
>>> "This method was automatically generated."
>>> "void glMatrixMode(GLenum mode);"
>>>
> <apicall: void 'glMatrixMode' (ulong) module: '#openGLLibraryName'>
>>> ^self externalCallFailed
>>>
>>> The
> <apicall:...>
>  fails
>>>
>>> How to think about this?
>>>
>>> Is Croquet behind OpenGL latest?
>>> Would teaching myself OpenGL programming be of use to the Croquet
>>> project?
>>>
>>> cheers,
>>>
>>> tty
>>>
>>>
>>>
>>





--
Sent from: http://forum.world.st/Squeak-VM-f104410.html
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] 64 bit FFI (was: porting Croquet to Squeak6.0 alpha...)

Eliot Miranda-2
 


On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <[hidden email]> wrote:
 
Hi Marcel,
no idea...
Look at timestamp:
eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set FFI-Kernel-EstebanLorenzano.45  ·
So presumably Eliot added that.

Perhaps.  I don't remember.  But if it is obsolete now, nuke it.
 

Le jeu. 28 mai 2020 à 17:00, marcel.taeumel <[hidden email]> a écrit :
 
Hi Nicolas,

what is the role of ExternalStructure class >> #pointerSize then?

Best,
Marcel


Nicolas Cellier wrote
> Hi all,
> beware of FFI 'long' and 'ulong'.
> They are currently meaning (32 bits) int and unsigned int!
> This dates back from the 80s when there were still odd OS with segmented
> memory and 16-bits int.
> long was thought as a safe int32_t at that time... and still means just
> that in current FFI.
>
> I'd like to replace them with 'int' and 'uint', but we need to ensure
> backward compatibility etc...
>
> FFI has ExternalType signedLongLong or just 'longlong' and ExternalType
> unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.
> See implementors of #initializeAtomicTypes #initializeFFIConstants
>
> If you want a type that can hold a pointer on both 32 and 64bits arch,
> then
> it's doable with a FFI type alias
> For example I have things like this in HDF5 module
> Notice the single #(fieldName fieldType) which defines an alias, while
> ordinary FFI struct gets an array of #(fieldName fieldType)
> (and replace Size_t with Intptr_t if the purpose is hodling a pointer,
> otherwise the ayatollahs of C standard will lash you)
>
> ExternalStructure subclass: #'Size_t'
>     instanceVariableNames: ''
>     classVariableNames: ''
>     poolDictionaries: ''
>     category: 'HDF5-External-atomic'
>
> Size_t class >>fields
>     "Size_t defineFields"
>     ^Smalltalk wordSize = 4
>         ifTrue: [#( value 'ulong')]
>         ifFalse: [#( value 'ulonglong')]
>
> However the "fields" must be re-generated when platform changes...
> I tried to automate that sometimes ago, see:
>
> ExternalStructure class>>install
>      "Resuming the image on another architecture may require a
> re-compilation of structure layout."
>      | newPlatform |
>      newPlatform := Smalltalk platformName.
>      PreviousPlatform = newPlatform
>          ifFalse:
>              [self recompileStructures.
>             PreviousPlatform := newPlatform]
>
> It works if you switch from Win64 to MacOS64 for example...
> However, this mechanism is not used yet at time of code loading, so
> importing code generated on a different platform won't automatically
> update
> the structures (or alias) - the FFI package has no mechanism for that
> (maybe we could add a hook in MC?).
>
> Le mer. 11 mars 2020 à 23:58, Bert Freudenberg &lt;

> bert@

> &gt; a
> écrit :
>
>> I'd suggest to get OpenGL working outside of Croquet first:
>>
>> http://www.squeaksource.com/CroquetGL.html
>>
>> Step 1: Verify this works in 32 bits. (assuming you are doing this on
>> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)
>> Step 2: Make it work in 64 bits.
>>
>> The second step requires that you understand how FFI works, and how it
>> handles e.g. pointers and integer sizes.
>> I am assuming we do have a working 64 bit FFI, at least for x86_64
>> machines.
>>
>> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a 32
>> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit
>> system,
>> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I
>> don't know how many bits 'ulong' has in our 64 bit FFI, but that
>> declaration may have to change. Etc. pp.
>>
>> If you have questions about FFI then those are best directed at the
>> vm-dev
>> list since it is dealing with VM-level interfaces. CC'ing, please follow
>> up
>> there.
>>
>> - Bert -
>>
>> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <
>>

> squeak-dev@.squeakfoundation

>> wrote:
>>
>>> Okey dokey,
>>>
>>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName after
>>>
>>> openGLLibraryName
>>> ^Smalltalk osVersion = 'linux'
>>> ifTrue: ['libGL.so.1']
>>> ifFalse: ['GL']
>>>
>>> I removed it in my install and got past that error.
>>>
>>> Working exclusively with Croquet(Master)...
>>>
>>>
>>>
>>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:
>>>
>>> glMatrixMode: mode
>>> "This method was automatically generated."
>>> "void glMatrixMode(GLenum mode);"
>>>
> <apicall: void 'glMatrixMode' (ulong) module: '#openGLLibraryName'>
>>> ^self externalCallFailed
>>>
>>> The
> <apicall:...>
>  fails
>>>
>>> How to think about this?
>>>
>>> Is Croquet behind OpenGL latest?
>>> Would teaching myself OpenGL programming be of use to the Croquet
>>> project?
>>>
>>> cheers,
>>>
>>> tty
>>>
>>>
>>>
>>





--
Sent from: http://forum.world.st/Squeak-VM-f104410.html


--
_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] 64 bit FFI (was: porting Croquet to Squeak6.0 alpha...)

marcel.taeumel
 
But if it is obsolete now, nuke it.

I cannot tell. That information is stored in the external type's compiledSpec to be used from within the FFI plugin I suppose.

Best,
Marcel

Am 29.05.2020 00:15:25 schrieb Eliot Miranda <[hidden email]>:

On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <>
[hidden email]> wrote:

>
> Hi Marcel,
> no idea...
> Look at timestamp:
> eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set
> FFI-Kernel-EstebanLorenzano.45 ·
> So presumably Eliot added that.
>

Perhaps. I don't remember. But if it is obsolete now, nuke it.


>
> Le jeu. 28 mai 2020 à 17:00, marcel.taeumel a
> écrit :
>
>>
>> Hi Nicolas,
>>
>> what is the role of ExternalStructure class >> #pointerSize then?
>>
>> Best,
>> Marcel
>>
>>
>> Nicolas Cellier wrote
>> > Hi all,
>> > beware of FFI 'long' and 'ulong'.
>> > They are currently meaning (32 bits) int and unsigned int!
>> > This dates back from the 80s when there were still odd OS with segmented
>> > memory and 16-bits int.
>> > long was thought as a safe int32_t at that time... and still means just
>> > that in current FFI.
>> >
>> > I'd like to replace them with 'int' and 'uint', but we need to ensure
>> > backward compatibility etc...
>> >
>> > FFI has ExternalType signedLongLong or just 'longlong' and ExternalType
>> > unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.
>> > See implementors of #initializeAtomicTypes #initializeFFIConstants
>> >
>> > If you want a type that can hold a pointer on both 32 and 64bits arch,
>> > then
>> > it's doable with a FFI type alias
>> > For example I have things like this in HDF5 module
>> > Notice the single #(fieldName fieldType) which defines an alias, while
>> > ordinary FFI struct gets an array of #(fieldName fieldType)
>> > (and replace Size_t with Intptr_t if the purpose is hodling a pointer,
>> > otherwise the ayatollahs of C standard will lash you)
>> >
>> > ExternalStructure subclass: #'Size_t'
>> > instanceVariableNames: ''
>> > classVariableNames: ''
>> > poolDictionaries: ''
>> > category: 'HDF5-External-atomic'
>> >
>> > Size_t class >>fields
>> > "Size_t defineFields"
>> > ^Smalltalk wordSize = 4
>> > ifTrue: [#( value 'ulong')]
>> > ifFalse: [#( value 'ulonglong')]
>> >
>> > However the "fields" must be re-generated when platform changes...
>> > I tried to automate that sometimes ago, see:
>> >
>> > ExternalStructure class>>install
>> > "Resuming the image on another architecture may require a
>> > re-compilation of structure layout."
>> > | newPlatform |
>> > newPlatform := Smalltalk platformName.
>> > PreviousPlatform = newPlatform
>> > ifFalse:
>> > [self recompileStructures.
>> > PreviousPlatform := newPlatform]
>> >
>> > It works if you switch from Win64 to MacOS64 for example...
>> > However, this mechanism is not used yet at time of code loading, so
>> > importing code generated on a different platform won't automatically
>> > update
>> > the structures (or alias) - the FFI package has no mechanism for that
>> > (maybe we could add a hook in MC?).
>> >
>> > Le mer. 11 mars 2020 à 23:58, Bert Freudenberg <
>>
>> > bert@
>>
>> > > a
>> > écrit :
>> >
>> >> I'd suggest to get OpenGL working outside of Croquet first:
>> >>
>> >> http://www.squeaksource.com/CroquetGL.html
>> >>
>> >> Step 1: Verify this works in 32 bits. (assuming you are doing this on
>> >> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)
>> >> Step 2: Make it work in 64 bits.
>> >>
>> >> The second step requires that you understand how FFI works, and how it
>> >> handles e.g. pointers and integer sizes.
>> >> I am assuming we do have a working 64 bit FFI, at least for x86_64
>> >> machines.
>> >>
>> >> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a
>> 32
>> >> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit
>> >> system,
>> >> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I
>> >> don't know how many bits 'ulong' has in our 64 bit FFI, but that
>> >> declaration may have to change. Etc. pp.
>> >>
>> >> If you have questions about FFI then those are best directed at the
>> >> vm-dev
>> >> list since it is dealing with VM-level interfaces. CC'ing, please
>> follow
>> >> up
>> >> there.
>> >>
>> >> - Bert -
>> >>
>> >> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <>
>> >>
>>
>> > squeak-dev@.squeakfoundation
>>
>> >> wrote:
>> >>
>> >>> Okey dokey,
>> >>>
>> >>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName
>> after
>> >>>
>> >>> openGLLibraryName
>> >>> ^Smalltalk osVersion = 'linux'
>> >>> ifTrue: ['libGL.so.1']
>> >>> ifFalse: ['GL']
>> >>>
>> >>> I removed it in my install and got past that error.
>> >>>
>> >>> Working exclusively with Croquet(Master)...
>> >>>
>> >>>
>> >>>
>> >>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:
>> >>>
>> >>> glMatrixMode: mode
>> >>> "This method was automatically generated."
>> >>> "void glMatrixMode(GLenum mode);"
>> >>>
>> >
>> >>> ^self externalCallFailed
>> >>>
>> >>> The
>> >
>> > fails
>> >>>
>> >>> How to think about this?
>> >>>
>> >>> Is Croquet behind OpenGL latest?
>> >>> Would teaching myself OpenGL programming be of use to the Croquet
>> >>> project?
>> >>>
>> >>> cheers,
>> >>>
>> >>> tty
>> >>>
>> >>>
>> >>>
>> >>
>>
>>
>>
>>
>>
>> --
>> Sent from: http://forum.world.st/Squeak-VM-f104410.html
>>
>

--
_,,,^..^,,,_
best, Eliot


On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <[hidden email]> wrote:
 
Hi Marcel,
no idea...
Look at timestamp:
eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set FFI-Kernel-EstebanLorenzano.45  ·
So presumably Eliot added that.

Perhaps.  I don't remember.  But if it is obsolete now, nuke it.
 

Le jeu. 28 mai 2020 à 17:00, marcel.taeumel <[hidden email]> a écrit :
 

Hi Nicolas,



what is the role of ExternalStructure class >> #pointerSize then?



Best,

Marcel





Nicolas Cellier wrote

> Hi all,

> beware of FFI 'long' and 'ulong'.

> They are currently meaning (32 bits) int and unsigned int!

> This dates back from the 80s when there were still odd OS with segmented

> memory and 16-bits int.

> long was thought as a safe int32_t at that time... and still means just

> that in current FFI.

>

> I'd like to replace them with 'int' and 'uint', but we need to ensure

> backward compatibility etc...

>

> FFI has ExternalType signedLongLong or just 'longlong' and ExternalType

> unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.

> See implementors of #initializeAtomicTypes #initializeFFIConstants

>

> If you want a type that can hold a pointer on both 32 and 64bits arch,

> then

> it's doable with a FFI type alias

> For example I have things like this in HDF5 module

> Notice the single #(fieldName fieldType) which defines an alias, while

> ordinary FFI struct gets an array of #(fieldName fieldType)

> (and replace Size_t with Intptr_t if the purpose is hodling a pointer,

> otherwise the ayatollahs of C standard will lash you)

>

> ExternalStructure subclass: #'Size_t'

>     instanceVariableNames: ''

>     classVariableNames: ''

>     poolDictionaries: ''

>     category: 'HDF5-External-atomic'

>

> Size_t class >>fields

>     "Size_t defineFields"

>     ^Smalltalk wordSize = 4

>         ifTrue: [#( value 'ulong')]

>         ifFalse: [#( value 'ulonglong')]

>

> However the "fields" must be re-generated when platform changes...

> I tried to automate that sometimes ago, see:

>

> ExternalStructure class>>install

>      "Resuming the image on another architecture may require a

> re-compilation of structure layout."

>      | newPlatform |

>      newPlatform := Smalltalk platformName.

>      PreviousPlatform = newPlatform

>          ifFalse:

>              [self recompileStructures.

>             PreviousPlatform := newPlatform]

>

> It works if you switch from Win64 to MacOS64 for example...

> However, this mechanism is not used yet at time of code loading, so

> importing code generated on a different platform won't automatically

> update

> the structures (or alias) - the FFI package has no mechanism for that

> (maybe we could add a hook in MC?).

>

> Le mer. 11 mars 2020 à 23:58, Bert Freudenberg &lt;



> bert@



> &gt; a

> écrit :

>

>> I'd suggest to get OpenGL working outside of Croquet first:

>>

>> http://www.squeaksource.com/CroquetGL.html

>>

>> Step 1: Verify this works in 32 bits. (assuming you are doing this on

>> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)

>> Step 2: Make it work in 64 bits.

>>

>> The second step requires that you understand how FFI works, and how it

>> handles e.g. pointers and integer sizes.

>> I am assuming we do have a working 64 bit FFI, at least for x86_64

>> machines.

>>

>> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a 32

>> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit

>> system,

>> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I

>> don't know how many bits 'ulong' has in our 64 bit FFI, but that

>> declaration may have to change. Etc. pp.

>>

>> If you have questions about FFI then those are best directed at the

>> vm-dev

>> list since it is dealing with VM-level interfaces. CC'ing, please follow

>> up

>> there.

>>

>> - Bert -

>>

>> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <

>>



> squeak-dev@.squeakfoundation



>> wrote:

>>

>>> Okey dokey,

>>>

>>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName after

>>>

>>> openGLLibraryName

>>> ^Smalltalk osVersion = 'linux'

>>> ifTrue: ['libGL.so.1']

>>> ifFalse: ['GL']

>>>

>>> I removed it in my install and got past that error.

>>>

>>> Working exclusively with Croquet(Master)...

>>>

>>>

>>>

>>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:

>>>

>>> glMatrixMode: mode

>>> "This method was automatically generated."

>>> "void glMatrixMode(GLenum mode);"

>>>

> <apicall: void 'glMatrixMode' (ulong) module: '#openGLLibraryName'>

>>> ^self externalCallFailed

>>>

>>> The

> <apicall:...>

>  fails

>>>

>>> How to think about this?

>>>

>>> Is Croquet behind OpenGL latest?

>>> Would teaching myself OpenGL programming be of use to the Croquet

>>> project?

>>>

>>> cheers,

>>>

>>> tty

>>>

>>>

>>>

>>











--

Sent from: http://forum.world.st/Squeak-VM-f104410.html




--
_,,,^..^,,,_
best, Eliot

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] 64 bit FFI (was: porting Croquet to Squeak6.0 alpha...)

marcel.taeumel
 
So, if the atomic types and structure types are initialized, they make use of ExternalType class >> #pointerSpec to store the size of the pointer in the #headerWord, which is the first element of an external type's 'compiledSpec'. For atomic types, this happens when the FFI package is loaded.

For structure types, this happens when a an external structure is mentioned for the first time in an FFI call method or in during compilation of another external structure's fields. See ExternalType class >> #newTypeNamed:force:. Squeak's Smalltalk parser or the compile logic in ExternalStructure invokes that type creation respectively..

Currently, if a platform change at system startup is detected, all atomic types and structure types are updated --- which implies a call to #pointerSpec in ExternalType class >> #initializeStructureTypes.

Now, what could be the situation where a struct-pointer type had the wrong pointer size? During development, your platform does not magically change the #wordSize. When loading code, methods in the external struct might be wrong (bc. Monticello does not trigger that field re-definition) BUT any external struct-pointer type that is already in the image must have the right size. And for new external struct-pointer types, it should work as explained above.

So, I think we should make the following changes:
- Remove ExternalType >> #asPointerType: (! note the colon !)
- Remove ExternalType >> #pointerSize:
- Change ExternalType >> #pointerSize to just read the compiledSpec
- Update all sends to #asPointerType: to #asPointerType and assume that the current pointer-type instances have the right size, as described above.

Best,
Marcel

Am 30.05.2020 15:59:31 schrieb Marcel Taeumel <[hidden email]>:

But if it is obsolete now, nuke it.

I cannot tell. That information is stored in the external type's compiledSpec to be used from within the FFI plugin I suppose.

Best,
Marcel

Am 29.05.2020 00:15:25 schrieb Eliot Miranda <[hidden email]>:

On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <>
[hidden email]> wrote:

>
> Hi Marcel,
> no idea...
> Look at timestamp:
> eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set
> FFI-Kernel-EstebanLorenzano.45 ·
> So presumably Eliot added that.
>

Perhaps. I don't remember. But if it is obsolete now, nuke it.


>
> Le jeu. 28 mai 2020 à 17:00, marcel.taeumel a
> écrit :
>
>>
>> Hi Nicolas,
>>
>> what is the role of ExternalStructure class >> #pointerSize then?
>>
>> Best,
>> Marcel
>>
>>
>> Nicolas Cellier wrote
>> > Hi all,
>> > beware of FFI 'long' and 'ulong'.
>> > They are currently meaning (32 bits) int and unsigned int!
>> > This dates back from the 80s when there were still odd OS with segmented
>> > memory and 16-bits int.
>> > long was thought as a safe int32_t at that time... and still means just
>> > that in current FFI.
>> >
>> > I'd like to replace them with 'int' and 'uint', but we need to ensure
>> > backward compatibility etc...
>> >
>> > FFI has ExternalType signedLongLong or just 'longlong' and ExternalType
>> > unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.
>> > See implementors of #initializeAtomicTypes #initializeFFIConstants
>> >
>> > If you want a type that can hold a pointer on both 32 and 64bits arch,
>> > then
>> > it's doable with a FFI type alias
>> > For example I have things like this in HDF5 module
>> > Notice the single #(fieldName fieldType) which defines an alias, while
>> > ordinary FFI struct gets an array of #(fieldName fieldType)
>> > (and replace Size_t with Intptr_t if the purpose is hodling a pointer,
>> > otherwise the ayatollahs of C standard will lash you)
>> >
>> > ExternalStructure subclass: #'Size_t'
>> > instanceVariableNames: ''
>> > classVariableNames: ''
>> > poolDictionaries: ''
>> > category: 'HDF5-External-atomic'
>> >
>> > Size_t class >>fields
>> > "Size_t defineFields"
>> > ^Smalltalk wordSize = 4
>> > ifTrue: [#( value 'ulong')]
>> > ifFalse: [#( value 'ulonglong')]
>> >
>> > However the "fields" must be re-generated when platform changes...
>> > I tried to automate that sometimes ago, see:
>> >
>> > ExternalStructure class>>install
>> > "Resuming the image on another architecture may require a
>> > re-compilation of structure layout."
>> > | newPlatform |
>> > newPlatform := Smalltalk platformName.
>> > PreviousPlatform = newPlatform
>> > ifFalse:
>> > [self recompileStructures.
>> > PreviousPlatform := newPlatform]
>> >
>> > It works if you switch from Win64 to MacOS64 for example...
>> > However, this mechanism is not used yet at time of code loading, so
>> > importing code generated on a different platform won't automatically
>> > update
>> > the structures (or alias) - the FFI package has no mechanism for that
>> > (maybe we could add a hook in MC?).
>> >
>> > Le mer. 11 mars 2020 à 23:58, Bert Freudenberg <
>>
>> > bert@
>>
>> > > a
>> > écrit :
>> >
>> >> I'd suggest to get OpenGL working outside of Croquet first:
>> >>
>> >> http://www.squeaksource.com/CroquetGL.html
>> >>
>> >> Step 1: Verify this works in 32 bits. (assuming you are doing this on
>> >> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)
>> >> Step 2: Make it work in 64 bits.
>> >>
>> >> The second step requires that you understand how FFI works, and how it
>> >> handles e.g. pointers and integer sizes.
>> >> I am assuming we do have a working 64 bit FFI, at least for x86_64
>> >> machines.
>> >>
>> >> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a
>> 32
>> >> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit
>> >> system,
>> >> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I
>> >> don't know how many bits 'ulong' has in our 64 bit FFI, but that
>> >> declaration may have to change. Etc. pp.
>> >>
>> >> If you have questions about FFI then those are best directed at the
>> >> vm-dev
>> >> list since it is dealing with VM-level interfaces. CC'ing, please
>> follow
>> >> up
>> >> there.
>> >>
>> >> - Bert -
>> >>
>> >> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <>
>> >>
>>
>> > squeak-dev@.squeakfoundation
>>
>> >> wrote:
>> >>
>> >>> Okey dokey,
>> >>>
>> >>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName
>> after
>> >>>
>> >>> openGLLibraryName
>> >>> ^Smalltalk osVersion = 'linux'
>> >>> ifTrue: ['libGL.so.1']
>> >>> ifFalse: ['GL']
>> >>>
>> >>> I removed it in my install and got past that error.
>> >>>
>> >>> Working exclusively with Croquet(Master)...
>> >>>
>> >>>
>> >>>
>> >>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:
>> >>>
>> >>> glMatrixMode: mode
>> >>> "This method was automatically generated."
>> >>> "void glMatrixMode(GLenum mode);"
>> >>>
>> >
>> >>> ^self externalCallFailed
>> >>>
>> >>> The
>> >
>> > fails
>> >>>
>> >>> How to think about this?
>> >>>
>> >>> Is Croquet behind OpenGL latest?
>> >>> Would teaching myself OpenGL programming be of use to the Croquet
>> >>> project?
>> >>>
>> >>> cheers,
>> >>>
>> >>> tty
>> >>>
>> >>>
>> >>>
>> >>
>>
>>
>>
>>
>>
>> --
>> Sent from: http://forum.world.st/Squeak-VM-f104410.html
>>
>

--
_,,,^..^,,,_
best, Eliot


On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <[hidden email]> wrote:
 
Hi Marcel,
no idea...
Look at timestamp:
eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set FFI-Kernel-EstebanLorenzano.45  ·
So presumably Eliot added that.

Perhaps.  I don't remember.  But if it is obsolete now, nuke it.
 

Le jeu. 28 mai 2020 à 17:00, marcel.taeumel <[hidden email]> a écrit :
 

Hi Nicolas,



what is the role of ExternalStructure class >> #pointerSize then?



Best,

Marcel





Nicolas Cellier wrote

> Hi all,

> beware of FFI 'long' and 'ulong'.

> They are currently meaning (32 bits) int and unsigned int!

> This dates back from the 80s when there were still odd OS with segmented

> memory and 16-bits int.

> long was thought as a safe int32_t at that time... and still means just

> that in current FFI.

>

> I'd like to replace them with 'int' and 'uint', but we need to ensure

> backward compatibility etc...

>

> FFI has ExternalType signedLongLong or just 'longlong' and ExternalType

> unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.

> See implementors of #initializeAtomicTypes #initializeFFIConstants

>

> If you want a type that can hold a pointer on both 32 and 64bits arch,

> then

> it's doable with a FFI type alias

> For example I have things like this in HDF5 module

> Notice the single #(fieldName fieldType) which defines an alias, while

> ordinary FFI struct gets an array of #(fieldName fieldType)

> (and replace Size_t with Intptr_t if the purpose is hodling a pointer,

> otherwise the ayatollahs of C standard will lash you)

>

> ExternalStructure subclass: #'Size_t'

>     instanceVariableNames: ''

>     classVariableNames: ''

>     poolDictionaries: ''

>     category: 'HDF5-External-atomic'

>

> Size_t class >>fields

>     "Size_t defineFields"

>     ^Smalltalk wordSize = 4

>         ifTrue: [#( value 'ulong')]

>         ifFalse: [#( value 'ulonglong')]

>

> However the "fields" must be re-generated when platform changes...

> I tried to automate that sometimes ago, see:

>

> ExternalStructure class>>install

>      "Resuming the image on another architecture may require a

> re-compilation of structure layout."

>      | newPlatform |

>      newPlatform := Smalltalk platformName.

>      PreviousPlatform = newPlatform

>          ifFalse:

>              [self recompileStructures.

>             PreviousPlatform := newPlatform]

>

> It works if you switch from Win64 to MacOS64 for example...

> However, this mechanism is not used yet at time of code loading, so

> importing code generated on a different platform won't automatically

> update

> the structures (or alias) - the FFI package has no mechanism for that

> (maybe we could add a hook in MC?).

>

> Le mer. 11 mars 2020 à 23:58, Bert Freudenberg &lt;



> bert@



> &gt; a

> écrit :

>

>> I'd suggest to get OpenGL working outside of Croquet first:

>>

>> http://www.squeaksource.com/CroquetGL.html

>>

>> Step 1: Verify this works in 32 bits. (assuming you are doing this on

>> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)

>> Step 2: Make it work in 64 bits.

>>

>> The second step requires that you understand how FFI works, and how it

>> handles e.g. pointers and integer sizes.

>> I am assuming we do have a working 64 bit FFI, at least for x86_64

>> machines.

>>

>> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a 32

>> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit

>> system,

>> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I

>> don't know how many bits 'ulong' has in our 64 bit FFI, but that

>> declaration may have to change. Etc. pp.

>>

>> If you have questions about FFI then those are best directed at the

>> vm-dev

>> list since it is dealing with VM-level interfaces. CC'ing, please follow

>> up

>> there.

>>

>> - Bert -

>>

>> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <

>>



> squeak-dev@.squeakfoundation



>> wrote:

>>

>>> Okey dokey,

>>>

>>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName after

>>>

>>> openGLLibraryName

>>> ^Smalltalk osVersion = 'linux'

>>> ifTrue: ['libGL.so.1']

>>> ifFalse: ['GL']

>>>

>>> I removed it in my install and got past that error.

>>>

>>> Working exclusively with Croquet(Master)...

>>>

>>>

>>>

>>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:

>>>

>>> glMatrixMode: mode

>>> "This method was automatically generated."

>>> "void glMatrixMode(GLenum mode);"

>>>

> <apicall: void 'glMatrixMode' (ulong) module: '#openGLLibraryName'>

>>> ^self externalCallFailed

>>>

>>> The

> <apicall:...>

>  fails

>>>

>>> How to think about this?

>>>

>>> Is Croquet behind OpenGL latest?

>>> Would teaching myself OpenGL programming be of use to the Croquet

>>> project?

>>>

>>> cheers,

>>>

>>> tty

>>>

>>>

>>>

>>











--

Sent from: http://forum.world.st/Squeak-VM-f104410.html




--
_,,,^..^,,,_
best, Eliot

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] 64 bit FFI (was: porting Croquet to Squeak6.0 alpha...)

marcel.taeumel
 
Hi Nicolas, hi Eliot.

If you want a type that can hold a pointer on both 32 and 64bits arch, then it's doable with a FFI type alias

You can just make an alias to a void*. It will have the right size. The only drawback would be that you might get an issue with argument coercion.

#(nil 'void*')

Your "type alias struct" will wrap around ExternalData<void*>. Why bother with ulong/ulonglong. Only if you want to work with integers in the image, you might want to use ulong or ulonglong depending on the #wordSize.

I think I will add 'word' as a type name in ExternalType's class-side "type constants" in Squeak FFI. Then you can use it in FFI call defs and struct-field defs:

ExternalType class >> #word
"Use this atomic-type alias if the external library works with opaque handles that are different on 32bit and 64bit. On a platform change, all FFI calls will be re-compiled and this method is called again by the parser."
^ FFIPlatformDescription current wordSize = 4
ifTrue: [self unsignedLong]
ifFalse: [self unsignedLongLong]

... maybe parts in the FFI plugin are faster with ulong/ulonglong compared to void* compared to type alias to void*... I don't know.

Best,
Marcel

Am 30.05.2020 18:00:33 schrieb Marcel Taeumel <[hidden email]>:

So, if the atomic types and structure types are initialized, they make use of ExternalType class >> #pointerSpec to store the size of the pointer in the #headerWord, which is the first element of an external type's 'compiledSpec'. For atomic types, this happens when the FFI package is loaded.

For structure types, this happens when a an external structure is mentioned for the first time in an FFI call method or in during compilation of another external structure's fields. See ExternalType class >> #newTypeNamed:force:. Squeak's Smalltalk parser or the compile logic in ExternalStructure invokes that type creation respectively..

Currently, if a platform change at system startup is detected, all atomic types and structure types are updated --- which implies a call to #pointerSpec in ExternalType class >> #initializeStructureTypes.

Now, what could be the situation where a struct-pointer type had the wrong pointer size? During development, your platform does not magically change the #wordSize. When loading code, methods in the external struct might be wrong (bc. Monticello does not trigger that field re-definition) BUT any external struct-pointer type that is already in the image must have the right size. And for new external struct-pointer types, it should work as explained above.

So, I think we should make the following changes:
- Remove ExternalType >> #asPointerType: (! note the colon !)
- Remove ExternalType >> #pointerSize:
- Change ExternalType >> #pointerSize to just read the compiledSpec
- Update all sends to #asPointerType: to #asPointerType and assume that the current pointer-type instances have the right size, as described above.

Best,
Marcel

Am 30.05.2020 15:59:31 schrieb Marcel Taeumel <[hidden email]>:

But if it is obsolete now, nuke it.

I cannot tell. That information is stored in the external type's compiledSpec to be used from within the FFI plugin I suppose.

Best,
Marcel

Am 29.05.2020 00:15:25 schrieb Eliot Miranda <[hidden email]>:

On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <>
[hidden email]> wrote:

>
> Hi Marcel,
> no idea...
> Look at timestamp:
> eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set
> FFI-Kernel-EstebanLorenzano.45 ·
> So presumably Eliot added that.
>

Perhaps. I don't remember. But if it is obsolete now, nuke it.


>
> Le jeu. 28 mai 2020 à 17:00, marcel.taeumel a
> écrit :
>
>>
>> Hi Nicolas,
>>
>> what is the role of ExternalStructure class >> #pointerSize then?
>>
>> Best,
>> Marcel
>>
>>
>> Nicolas Cellier wrote
>> > Hi all,
>> > beware of FFI 'long' and 'ulong'.
>> > They are currently meaning (32 bits) int and unsigned int!
>> > This dates back from the 80s when there were still odd OS with segmented
>> > memory and 16-bits int.
>> > long was thought as a safe int32_t at that time... and still means just
>> > that in current FFI.
>> >
>> > I'd like to replace them with 'int' and 'uint', but we need to ensure
>> > backward compatibility etc...
>> >
>> > FFI has ExternalType signedLongLong or just 'longlong' and ExternalType
>> > unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.
>> > See implementors of #initializeAtomicTypes #initializeFFIConstants
>> >
>> > If you want a type that can hold a pointer on both 32 and 64bits arch,
>> > then
>> > it's doable with a FFI type alias
>> > For example I have things like this in HDF5 module
>> > Notice the single #(fieldName fieldType) which defines an alias, while
>> > ordinary FFI struct gets an array of #(fieldName fieldType)
>> > (and replace Size_t with Intptr_t if the purpose is hodling a pointer,
>> > otherwise the ayatollahs of C standard will lash you)
>> >
>> > ExternalStructure subclass: #'Size_t'
>> > instanceVariableNames: ''
>> > classVariableNames: ''
>> > poolDictionaries: ''
>> > category: 'HDF5-External-atomic'
>> >
>> > Size_t class >>fields
>> > "Size_t defineFields"
>> > ^Smalltalk wordSize = 4
>> > ifTrue: [#( value 'ulong')]
>> > ifFalse: [#( value 'ulonglong')]
>> >
>> > However the "fields" must be re-generated when platform changes...
>> > I tried to automate that sometimes ago, see:
>> >
>> > ExternalStructure class>>install
>> > "Resuming the image on another architecture may require a
>> > re-compilation of structure layout."
>> > | newPlatform |
>> > newPlatform := Smalltalk platformName.
>> > PreviousPlatform = newPlatform
>> > ifFalse:
>> > [self recompileStructures.
>> > PreviousPlatform := newPlatform]
>> >
>> > It works if you switch from Win64 to MacOS64 for example...
>> > However, this mechanism is not used yet at time of code loading, so
>> > importing code generated on a different platform won't automatically
>> > update
>> > the structures (or alias) - the FFI package has no mechanism for that
>> > (maybe we could add a hook in MC?).
>> >
>> > Le mer. 11 mars 2020 à 23:58, Bert Freudenberg <
>>
>> > bert@
>>
>> > > a
>> > écrit :
>> >
>> >> I'd suggest to get OpenGL working outside of Croquet first:
>> >>
>> >> http://www.squeaksource.com/CroquetGL.html
>> >>
>> >> Step 1: Verify this works in 32 bits. (assuming you are doing this on
>> >> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)
>> >> Step 2: Make it work in 64 bits.
>> >>
>> >> The second step requires that you understand how FFI works, and how it
>> >> handles e.g. pointers and integer sizes.
>> >> I am assuming we do have a working 64 bit FFI, at least for x86_64
>> >> machines.
>> >>
>> >> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a
>> 32
>> >> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit
>> >> system,
>> >> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I
>> >> don't know how many bits 'ulong' has in our 64 bit FFI, but that
>> >> declaration may have to change. Etc. pp.
>> >>
>> >> If you have questions about FFI then those are best directed at the
>> >> vm-dev
>> >> list since it is dealing with VM-level interfaces. CC'ing, please
>> follow
>> >> up
>> >> there.
>> >>
>> >> - Bert -
>> >>
>> >> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <>
>> >>
>>
>> > squeak-dev@.squeakfoundation
>>
>> >> wrote:
>> >>
>> >>> Okey dokey,
>> >>>
>> >>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName
>> after
>> >>>
>> >>> openGLLibraryName
>> >>> ^Smalltalk osVersion = 'linux'
>> >>> ifTrue: ['libGL.so.1']
>> >>> ifFalse: ['GL']
>> >>>
>> >>> I removed it in my install and got past that error.
>> >>>
>> >>> Working exclusively with Croquet(Master)...
>> >>>
>> >>>
>> >>>
>> >>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:
>> >>>
>> >>> glMatrixMode: mode
>> >>> "This method was automatically generated."
>> >>> "void glMatrixMode(GLenum mode);"
>> >>>
>> >
>> >>> ^self externalCallFailed
>> >>>
>> >>> The
>> >
>> > fails
>> >>>
>> >>> How to think about this?
>> >>>
>> >>> Is Croquet behind OpenGL latest?
>> >>> Would teaching myself OpenGL programming be of use to the Croquet
>> >>> project?
>> >>>
>> >>> cheers,
>> >>>
>> >>> tty
>> >>>
>> >>>
>> >>>
>> >>
>>
>>
>>
>>
>>
>> --
>> Sent from: http://forum.world.st/Squeak-VM-f104410.html
>>
>

--
_,,,^..^,,,_
best, Eliot


On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <[hidden email]> wrote:
 
Hi Marcel,
no idea...
Look at timestamp:
eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set FFI-Kernel-EstebanLorenzano.45  ·
So presumably Eliot added that.

Perhaps.  I don't remember.  But if it is obsolete now, nuke it.
 

Le jeu. 28 mai 2020 à 17:00, marcel.taeumel <[hidden email]> a écrit :
 

Hi Nicolas,



what is the role of ExternalStructure class >> #pointerSize then?



Best,

Marcel





Nicolas Cellier wrote

> Hi all,

> beware of FFI 'long' and 'ulong'.

> They are currently meaning (32 bits) int and unsigned int!

> This dates back from the 80s when there were still odd OS with segmented

> memory and 16-bits int.

> long was thought as a safe int32_t at that time... and still means just

> that in current FFI.

>

> I'd like to replace them with 'int' and 'uint', but we need to ensure

> backward compatibility etc...

>

> FFI has ExternalType signedLongLong or just 'longlong' and ExternalType

> unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.

> See implementors of #initializeAtomicTypes #initializeFFIConstants

>

> If you want a type that can hold a pointer on both 32 and 64bits arch,

> then

> it's doable with a FFI type alias

> For example I have things like this in HDF5 module

> Notice the single #(fieldName fieldType) which defines an alias, while

> ordinary FFI struct gets an array of #(fieldName fieldType)

> (and replace Size_t with Intptr_t if the purpose is hodling a pointer,

> otherwise the ayatollahs of C standard will lash you)

>

> ExternalStructure subclass: #'Size_t'

>     instanceVariableNames: ''

>     classVariableNames: ''

>     poolDictionaries: ''

>     category: 'HDF5-External-atomic'

>

> Size_t class >>fields

>     "Size_t defineFields"

>     ^Smalltalk wordSize = 4

>         ifTrue: [#( value 'ulong')]

>         ifFalse: [#( value 'ulonglong')]

>

> However the "fields" must be re-generated when platform changes...

> I tried to automate that sometimes ago, see:

>

> ExternalStructure class>>install

>      "Resuming the image on another architecture may require a

> re-compilation of structure layout."

>      | newPlatform |

>      newPlatform := Smalltalk platformName.

>      PreviousPlatform = newPlatform

>          ifFalse:

>              [self recompileStructures.

>             PreviousPlatform := newPlatform]

>

> It works if you switch from Win64 to MacOS64 for example...

> However, this mechanism is not used yet at time of code loading, so

> importing code generated on a different platform won't automatically

> update

> the structures (or alias) - the FFI package has no mechanism for that

> (maybe we could add a hook in MC?).

>

> Le mer. 11 mars 2020 à 23:58, Bert Freudenberg &lt;



> bert@



> &gt; a

> écrit :

>

>> I'd suggest to get OpenGL working outside of Croquet first:

>>

>> http://www.squeaksource.com/CroquetGL.html

>>

>> Step 1: Verify this works in 32 bits. (assuming you are doing this on

>> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)

>> Step 2: Make it work in 64 bits.

>>

>> The second step requires that you understand how FFI works, and how it

>> handles e.g. pointers and integer sizes.

>> I am assuming we do have a working 64 bit FFI, at least for x86_64

>> machines.

>>

>> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a 32

>> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit

>> system,

>> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I

>> don't know how many bits 'ulong' has in our 64 bit FFI, but that

>> declaration may have to change. Etc. pp.

>>

>> If you have questions about FFI then those are best directed at the

>> vm-dev

>> list since it is dealing with VM-level interfaces. CC'ing, please follow

>> up

>> there.

>>

>> - Bert -

>>

>> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <

>>



> squeak-dev@.squeakfoundation



>> wrote:

>>

>>> Okey dokey,

>>>

>>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName after

>>>

>>> openGLLibraryName

>>> ^Smalltalk osVersion = 'linux'

>>> ifTrue: ['libGL.so.1']

>>> ifFalse: ['GL']

>>>

>>> I removed it in my install and got past that error.

>>>

>>> Working exclusively with Croquet(Master)...

>>>

>>>

>>>

>>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:

>>>

>>> glMatrixMode: mode

>>> "This method was automatically generated."

>>> "void glMatrixMode(GLenum mode);"

>>>

> <apicall: void 'glMatrixMode' (ulong) module: '#openGLLibraryName'>

>>> ^self externalCallFailed

>>>

>>> The

> <apicall:...>

>  fails

>>>

>>> How to think about this?

>>>

>>> Is Croquet behind OpenGL latest?

>>> Would teaching myself OpenGL programming be of use to the Croquet

>>> project?

>>>

>>> cheers,

>>>

>>> tty

>>>

>>>

>>>

>>











--

Sent from: http://forum.world.st/Squeak-VM-f104410.html




--
_,,,^..^,,,_
best, Eliot

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] 64 bit FFI (was: porting Croquet to Squeak6.0 alpha...)

Eliot Miranda-2
 

On Jun 9, 2020, at 5:10 AM, Marcel Taeumel <[hidden email]> wrote:


Hi Nicolas, hi Eliot.

If you want a type that can hold a pointer on both 32 and 64bits arch, then it's doable with a FFI type alias

You can just make an alias to a void*. It will have the right size. The only drawback would be that you might get an issue with argument coercion.

#(nil 'void*')

Your "type alias struct" will wrap around ExternalData<void*>. Why bother with ulong/ulonglong. Only if you want to work with integers in the image, you might want to use ulong or ulonglong depending on the #wordSize.

I think I will add 'word' as a type name in ExternalType's class-side "type constants" in Squeak FFI. Then you can use it in FFI call defs and struct-field defs:

ExternalType class >> #word
"Use this atomic-type alias if the external library works with opaque handles that are different on 32bit and 64bit. On a platform change, all FFI calls will be re-compiled and this method is called again by the parser."
^ FFIPlatformDescription current wordSize = 4
ifTrue: [self unsignedLong]
ifFalse: [self unsignedLongLong]

... maybe parts in the FFI plugin are faster with ulong/ulonglong compared to void* compared to type alias to void*... I don't know.

It might be better to use a C name such as intptr_t or uintptr_t.


Best,
Marcel

Am 30.05.2020 18:00:33 schrieb Marcel Taeumel <[hidden email]>:

So, if the atomic types and structure types are initialized, they make use of ExternalType class >> #pointerSpec to store the size of the pointer in the #headerWord, which is the first element of an external type's 'compiledSpec'. For atomic types, this happens when the FFI package is loaded.

For structure types, this happens when a an external structure is mentioned for the first time in an FFI call method or in during compilation of another external structure's fields. See ExternalType class >> #newTypeNamed:force:. Squeak's Smalltalk parser or the compile logic in ExternalStructure invokes that type creation respectively..

Currently, if a platform change at system startup is detected, all atomic types and structure types are updated --- which implies a call to #pointerSpec in ExternalType class >> #initializeStructureTypes.

Now, what could be the situation where a struct-pointer type had the wrong pointer size? During development, your platform does not magically change the #wordSize. When loading code, methods in the external struct might be wrong (bc. Monticello does not trigger that field re-definition) BUT any external struct-pointer type that is already in the image must have the right size. And for new external struct-pointer types, it should work as explained above.

So, I think we should make the following changes:
- Remove ExternalType >> #asPointerType: (! note the colon !)
- Remove ExternalType >> #pointerSize:
- Change ExternalType >> #pointerSize to just read the compiledSpec
- Update all sends to #asPointerType: to #asPointerType and assume that the current pointer-type instances have the right size, as described above.

Best,
Marcel

Am 30.05.2020 15:59:31 schrieb Marcel Taeumel <[hidden email]>:

But if it is obsolete now, nuke it.

I cannot tell. That information is stored in the external type's compiledSpec to be used from within the FFI plugin I suppose.

Best,
Marcel

Am 29.05.2020 00:15:25 schrieb Eliot Miranda <[hidden email]>:

On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <>
[hidden email]> wrote:

>
> Hi Marcel,
> no idea...
> Look at timestamp:
> eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set
> FFI-Kernel-EstebanLorenzano.45 ·
> So presumably Eliot added that.
>

Perhaps. I don't remember. But if it is obsolete now, nuke it.


>
> Le jeu. 28 mai 2020 à 17:00, marcel.taeumel a
> écrit :
>
>>
>> Hi Nicolas,
>>
>> what is the role of ExternalStructure class >> #pointerSize then?
>>
>> Best,
>> Marcel
>>
>>
>> Nicolas Cellier wrote
>> > Hi all,
>> > beware of FFI 'long' and 'ulong'.
>> > They are currently meaning (32 bits) int and unsigned int!
>> > This dates back from the 80s when there were still odd OS with segmented
>> > memory and 16-bits int.
>> > long was thought as a safe int32_t at that time... and still means just
>> > that in current FFI.
>> >
>> > I'd like to replace them with 'int' and 'uint', but we need to ensure
>> > backward compatibility etc...
>> >
>> > FFI has ExternalType signedLongLong or just 'longlong' and ExternalType
>> > unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.
>> > See implementors of #initializeAtomicTypes #initializeFFIConstants
>> >
>> > If you want a type that can hold a pointer on both 32 and 64bits arch,
>> > then
>> > it's doable with a FFI type alias
>> > For example I have things like this in HDF5 module
>> > Notice the single #(fieldName fieldType) which defines an alias, while
>> > ordinary FFI struct gets an array of #(fieldName fieldType)
>> > (and replace Size_t with Intptr_t if the purpose is hodling a pointer,
>> > otherwise the ayatollahs of C standard will lash you)
>> >
>> > ExternalStructure subclass: #'Size_t'
>> > instanceVariableNames: ''
>> > classVariableNames: ''
>> > poolDictionaries: ''
>> > category: 'HDF5-External-atomic'
>> >
>> > Size_t class >>fields
>> > "Size_t defineFields"
>> > ^Smalltalk wordSize = 4
>> > ifTrue: [#( value 'ulong')]
>> > ifFalse: [#( value 'ulonglong')]
>> >
>> > However the "fields" must be re-generated when platform changes...
>> > I tried to automate that sometimes ago, see:
>> >
>> > ExternalStructure class>>install
>> > "Resuming the image on another architecture may require a
>> > re-compilation of structure layout."
>> > | newPlatform |
>> > newPlatform := Smalltalk platformName.
>> > PreviousPlatform = newPlatform
>> > ifFalse:
>> > [self recompileStructures.
>> > PreviousPlatform := newPlatform]
>> >
>> > It works if you switch from Win64 to MacOS64 for example...
>> > However, this mechanism is not used yet at time of code loading, so
>> > importing code generated on a different platform won't automatically
>> > update
>> > the structures (or alias) - the FFI package has no mechanism for that
>> > (maybe we could add a hook in MC?).
>> >
>> > Le mer. 11 mars 2020 à 23:58, Bert Freudenberg <
>>
>> > bert@
>>
>> > > a
>> > écrit :
>> >
>> >> I'd suggest to get OpenGL working outside of Croquet first:
>> >>
>> >> http://www.squeaksource.com/CroquetGL.html
>> >>
>> >> Step 1: Verify this works in 32 bits. (assuming you are doing this on
>> >> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)
>> >> Step 2: Make it work in 64 bits.
>> >>
>> >> The second step requires that you understand how FFI works, and how it
>> >> handles e.g. pointers and integer sizes.
>> >> I am assuming we do have a working 64 bit FFI, at least for x86_64
>> >> machines.
>> >>
>> >> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a
>> 32
>> >> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit
>> >> system,
>> >> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I
>> >> don't know how many bits 'ulong' has in our 64 bit FFI, but that
>> >> declaration may have to change. Etc. pp.
>> >>
>> >> If you have questions about FFI then those are best directed at the
>> >> vm-dev
>> >> list since it is dealing with VM-level interfaces. CC'ing, please
>> follow
>> >> up
>> >> there.
>> >>
>> >> - Bert -
>> >>
>> >> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <>
>> >>
>>
>> > squeak-dev@.squeakfoundation
>>
>> >> wrote:
>> >>
>> >>> Okey dokey,
>> >>>
>> >>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName
>> after
>> >>>
>> >>> openGLLibraryName
>> >>> ^Smalltalk osVersion = 'linux'
>> >>> ifTrue: ['libGL.so.1']
>> >>> ifFalse: ['GL']
>> >>>
>> >>> I removed it in my install and got past that error.
>> >>>
>> >>> Working exclusively with Croquet(Master)...
>> >>>
>> >>>
>> >>>
>> >>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:
>> >>>
>> >>> glMatrixMode: mode
>> >>> "This method was automatically generated."
>> >>> "void glMatrixMode(GLenum mode);"
>> >>>
>> >
>> >>> ^self externalCallFailed
>> >>>
>> >>> The
>> >
>> > fails
>> >>>
>> >>> How to think about this?
>> >>>
>> >>> Is Croquet behind OpenGL latest?
>> >>> Would teaching myself OpenGL programming be of use to the Croquet
>> >>> project?
>> >>>
>> >>> cheers,
>> >>>
>> >>> tty
>> >>>
>> >>>
>> >>>
>> >>
>>
>>
>>
>>
>>
>> --
>> Sent from: http://forum.world.st/Squeak-VM-f104410.html
>>
>

--
_,,,^..^,,,_
best, Eliot


On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <[hidden email]> wrote:
 
Hi Marcel,
no idea...
Look at timestamp:
eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set FFI-Kernel-EstebanLorenzano.45  ·
So presumably Eliot added that.

Perhaps.  I don't remember.  But if it is obsolete now, nuke it.
 

Le jeu. 28 mai 2020 à 17:00, marcel.taeumel <[hidden email]> a écrit :
 

Hi Nicolas,



what is the role of ExternalStructure class >> #pointerSize then?



Best,

Marcel





Nicolas Cellier wrote

> Hi all,

> beware of FFI 'long' and 'ulong'.

> They are currently meaning (32 bits) int and unsigned int!

> This dates back from the 80s when there were still odd OS with segmented

> memory and 16-bits int.

> long was thought as a safe int32_t at that time... and still means just

> that in current FFI.

>

> I'd like to replace them with 'int' and 'uint', but we need to ensure

> backward compatibility etc...

>

> FFI has ExternalType signedLongLong or just 'longlong' and ExternalType

> unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.

> See implementors of #initializeAtomicTypes #initializeFFIConstants

>

> If you want a type that can hold a pointer on both 32 and 64bits arch,

> then

> it's doable with a FFI type alias

> For example I have things like this in HDF5 module

> Notice the single #(fieldName fieldType) which defines an alias, while

> ordinary FFI struct gets an array of #(fieldName fieldType)

> (and replace Size_t with Intptr_t if the purpose is hodling a pointer,

> otherwise the ayatollahs of C standard will lash you)

>

> ExternalStructure subclass: #'Size_t'

>     instanceVariableNames: ''

>     classVariableNames: ''

>     poolDictionaries: ''

>     category: 'HDF5-External-atomic'

>

> Size_t class >>fields

>     "Size_t defineFields"

>     ^Smalltalk wordSize = 4

>         ifTrue: [#( value 'ulong')]

>         ifFalse: [#( value 'ulonglong')]

>

> However the "fields" must be re-generated when platform changes...

> I tried to automate that sometimes ago, see:

>

> ExternalStructure class>>install

>      "Resuming the image on another architecture may require a

> re-compilation of structure layout."

>      | newPlatform |

>      newPlatform := Smalltalk platformName.

>      PreviousPlatform = newPlatform

>          ifFalse:

>              [self recompileStructures.

>             PreviousPlatform := newPlatform]

>

> It works if you switch from Win64 to MacOS64 for example...

> However, this mechanism is not used yet at time of code loading, so

> importing code generated on a different platform won't automatically

> update

> the structures (or alias) - the FFI package has no mechanism for that

> (maybe we could add a hook in MC?).

>

> Le mer. 11 mars 2020 à 23:58, Bert Freudenberg &lt;



> bert@



> &gt; a

> écrit :

>

>> I'd suggest to get OpenGL working outside of Croquet first:

>>

>> http://www.squeaksource.com/CroquetGL.html

>>

>> Step 1: Verify this works in 32 bits. (assuming you are doing this on

>> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)

>> Step 2: Make it work in 64 bits.

>>

>> The second step requires that you understand how FFI works, and how it

>> handles e.g. pointers and integer sizes.

>> I am assuming we do have a working 64 bit FFI, at least for x86_64

>> machines.

>>

>> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a 32

>> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit

>> system,

>> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I

>> don't know how many bits 'ulong' has in our 64 bit FFI, but that

>> declaration may have to change. Etc. pp.

>>

>> If you have questions about FFI then those are best directed at the

>> vm-dev

>> list since it is dealing with VM-level interfaces. CC'ing, please follow

>> up

>> there.

>>

>> - Bert -

>>

>> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <

>>



> squeak-dev@.squeakfoundation



>> wrote:

>>

>>> Okey dokey,

>>>

>>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName after

>>>

>>> openGLLibraryName

>>> ^Smalltalk osVersion = 'linux'

>>> ifTrue: ['libGL.so.1']

>>> ifFalse: ['GL']

>>>

>>> I removed it in my install and got past that error.

>>>

>>> Working exclusively with Croquet(Master)...

>>>

>>>

>>>

>>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:

>>>

>>> glMatrixMode: mode

>>> "This method was automatically generated."

>>> "void glMatrixMode(GLenum mode);"

>>>

> <apicall: void 'glMatrixMode' (ulong) module: '#openGLLibraryName'>

>>> ^self externalCallFailed

>>>

>>> The

> <apicall:...>

>  fails

>>>

>>> How to think about this?

>>>

>>> Is Croquet behind OpenGL latest?

>>> Would teaching myself OpenGL programming be of use to the Croquet

>>> project?

>>>

>>> cheers,

>>>

>>> tty

>>>

>>>

>>>

>>











--

Sent from: http://forum.world.st/Squeak-VM-f104410.html




--
_,,,^..^,,,_
best, Eliot

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] 64 bit FFI (was: porting Croquet to Squeak6.0 alpha...)

marcel.taeumel
 
Hi Eliot.

It might be better to use a C name such as intptr_t or uintptr_t.

Sounds good. :-)

Best,
Marcel

Am 09.06.2020 18:33:36 schrieb Eliot Miranda <[hidden email]>:


On Jun 9, 2020, at 5:10 AM, Marcel Taeumel <[hidden email]> wrote:








Hi Nicolas, hi Eliot.

If you want a type that can hold a pointer on both 32 and 64bits arch, then it's doable with a FFI type alias

You can just make an alias to a void*. It will have the right size. The only drawback would be that you might get an issue with argument coercion.

#(nil 'void*')

Your "type alias struct" will wrap around ExternalData<void*>. Why bother with ulong/ulonglong. Only if you want to work with integers in the image, you might want to use ulong or ulonglong depending on the #wordSize.

I think I will add 'word' as a type name in ExternalType's class-side "type constants" in Squeak FFI. Then you can use it in FFI call defs and struct-field defs:

ExternalType class >> #word
"Use this atomic-type alias if the external library works with opaque handles that are different on 32bit and 64bit. On a platform change, all FFI calls will be re-compiled and this method is called again by the parser."
^ FFIPlatformDescription current wordSize = 4
ifTrue: [self unsignedLong]
ifFalse: [self unsignedLongLong]

... maybe parts in the FFI plugin are faster with ulong/ulonglong compared to void* compared to type alias to void*... I don't know.

It might be better to use a C name such as intptr_t or uintptr_t.


Best,
Marcel



Am 30.05.2020 18:00:33 schrieb Marcel Taeumel <[hidden email]>:







So, if the atomic types and structure types are initialized, they make use of ExternalType class >> #pointerSpec to store the size of the pointer in the #headerWord, which is the first element of an external type's 'compiledSpec'. For atomic types, this happens when the FFI package is loaded.

For structure types, this happens when a an external structure is mentioned for the first time in an FFI call method or in during compilation of another external structure's fields. See ExternalType class >> #newTypeNamed:force:. Squeak's Smalltalk parser or the compile logic in ExternalStructure invokes that type creation respectively..

Currently, if a platform change at system startup is detected, all atomic types and structure types are updated --- which implies a call to #pointerSpec in ExternalType class >> #initializeStructureTypes.

Now, what could be the situation where a struct-pointer type had the wrong pointer size? During development, your platform does not magically change the #wordSize. When loading code, methods in the external struct might be wrong (bc. Monticello does not trigger that field re-definition) BUT any external struct-pointer type that is already in the image must have the right size. And for new external struct-pointer types, it should work as explained above.

So, I think we should make the following changes:
- Remove ExternalType >> #asPointerType: (! note the colon !)
- Remove ExternalType >> #pointerSize:
- Change ExternalType >> #pointerSize to just read the compiledSpec
- Update all sends to #asPointerType: to #asPointerType and assume that the current pointer-type instances have the right size, as described above.

Best,
Marcel




Am 30.05.2020 15:59:31 schrieb Marcel Taeumel <[hidden email]>:


But if it is obsolete now, nuke it.

I cannot tell. That information is stored in the external type's compiledSpec to be used from within the FFI plugin I suppose.

Best,
Marcel

Am 29.05.2020 00:15:25 schrieb Eliot Miranda <[hidden email]>:

On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <>
[hidden email]> wrote:



>

> Hi Marcel,

> no idea...

> Look at timestamp:

> eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set

> FFI-Kernel-EstebanLorenzano.45 ·

> So presumably Eliot added that.

>



Perhaps. I don't remember. But if it is obsolete now, nuke it.





>

> Le jeu. 28 mai 2020 à 17:00, marcel.taeumel a

> écrit :

>

>>

>> Hi Nicolas,

>>

>> what is the role of ExternalStructure class >> #pointerSize then?

>>

>> Best,

>> Marcel

>>

>>

>> Nicolas Cellier wrote

>> > Hi all,

>> > beware of FFI 'long' and 'ulong'.

>> > They are currently meaning (32 bits) int and unsigned int!

>> > This dates back from the 80s when there were still odd OS with segmented

>> > memory and 16-bits int.

>> > long was thought as a safe int32_t at that time... and still means just

>> > that in current FFI.

>> >

>> > I'd like to replace them with 'int' and 'uint', but we need to ensure

>> > backward compatibility etc...

>> >

>> > FFI has ExternalType signedLongLong or just 'longlong' and ExternalType

>> > unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.

>> > See implementors of #initializeAtomicTypes #initializeFFIConstants

>> >

>> > If you want a type that can hold a pointer on both 32 and 64bits arch,

>> > then

>> > it's doable with a FFI type alias

>> > For example I have things like this in HDF5 module

>> > Notice the single #(fieldName fieldType) which defines an alias, while

>> > ordinary FFI struct gets an array of #(fieldName fieldType)

>> > (and replace Size_t with Intptr_t if the purpose is hodling a pointer,

>> > otherwise the ayatollahs of C standard will lash you)

>> >

>> > ExternalStructure subclass: #'Size_t'

>> > instanceVariableNames: ''

>> > classVariableNames: ''

>> > poolDictionaries: ''

>> > category: 'HDF5-External-atomic'

>> >

>> > Size_t class >>fields

>> > "Size_t defineFields"

>> > ^Smalltalk wordSize = 4

>> > ifTrue: [#( value 'ulong')]

>> > ifFalse: [#( value 'ulonglong')]

>> >

>> > However the "fields" must be re-generated when platform changes...

>> > I tried to automate that sometimes ago, see:

>> >

>> > ExternalStructure class>>install

>> > "Resuming the image on another architecture may require a

>> > re-compilation of structure layout."

>> > | newPlatform |

>> > newPlatform := Smalltalk platformName.

>> > PreviousPlatform = newPlatform

>> > ifFalse:

>> > [self recompileStructures.

>> > PreviousPlatform := newPlatform]

>> >

>> > It works if you switch from Win64 to MacOS64 for example...

>> > However, this mechanism is not used yet at time of code loading, so

>> > importing code generated on a different platform won't automatically

>> > update

>> > the structures (or alias) - the FFI package has no mechanism for that

>> > (maybe we could add a hook in MC?).

>> >

>> > Le mer. 11 mars 2020 à 23:58, Bert Freudenberg <

>>

>> > bert@

>>

>> > > a

>> > écrit :

>> >

>> >> I'd suggest to get OpenGL working outside of Croquet first:

>> >>

>> >> http://www.squeaksource.com/CroquetGL.html

>> >>

>> >> Step 1: Verify this works in 32 bits. (assuming you are doing this on

>> >> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)

>> >> Step 2: Make it work in 64 bits.

>> >>

>> >> The second step requires that you understand how FFI works, and how it

>> >> handles e.g. pointers and integer sizes.

>> >> I am assuming we do have a working 64 bit FFI, at least for x86_64

>> >> machines.

>> >>

>> >> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a

>> 32

>> >> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit

>> >> system,

>> >> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I

>> >> don't know how many bits 'ulong' has in our 64 bit FFI, but that

>> >> declaration may have to change. Etc. pp.

>> >>

>> >> If you have questions about FFI then those are best directed at the

>> >> vm-dev

>> >> list since it is dealing with VM-level interfaces. CC'ing, please

>> follow

>> >> up

>> >> there.

>> >>

>> >> - Bert -

>> >>

>> >> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <>
>> >>

>>

>> > squeak-dev@.squeakfoundation

>>

>> >> wrote:

>> >>

>> >>> Okey dokey,

>> >>>

>> >>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName

>> after

>> >>>

>> >>> openGLLibraryName

>> >>> ^Smalltalk osVersion = 'linux'

>> >>> ifTrue: ['libGL.so.1']

>> >>> ifFalse: ['GL']

>> >>>

>> >>> I removed it in my install and got past that error.

>> >>>

>> >>> Working exclusively with Croquet(Master)...

>> >>>

>> >>>

>> >>>

>> >>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:

>> >>>

>> >>> glMatrixMode: mode

>> >>> "This method was automatically generated."

>> >>> "void glMatrixMode(GLenum mode);"

>> >>>

>> >

>> >>> ^self externalCallFailed

>> >>>

>> >>> The

>> >

>> > fails

>> >>>

>> >>> How to think about this?

>> >>>

>> >>> Is Croquet behind OpenGL latest?

>> >>> Would teaching myself OpenGL programming be of use to the Croquet

>> >>> project?

>> >>>

>> >>> cheers,

>> >>>

>> >>> tty

>> >>>

>> >>>

>> >>>

>> >>

>>

>>

>>

>>

>>

>> --

>> Sent from: http://forum.world.st/Squeak-VM-f104410.html

>>

>



--

_,,,^..^,,,_

best, Eliot



On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <[hidden email]> wrote:
 
Hi Marcel,
no idea...
Look at timestamp:
eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set FFI-Kernel-EstebanLorenzano.45  ·
So presumably Eliot added that.

Perhaps.  I don't remember.  But if it is obsolete now, nuke it.
 

Le jeu. 28 mai 2020 à 17:00, marcel.taeumel <[hidden email]> a écrit :
 


Hi Nicolas,





what is the role of ExternalStructure class >> #pointerSize then?





Best,


Marcel








Nicolas Cellier wrote


> Hi all,


> beware of FFI 'long' and 'ulong'.


> They are currently meaning (32 bits) int and unsigned int!


> This dates back from the 80s when there were still odd OS with segmented


> memory and 16-bits int.


> long was thought as a safe int32_t at that time... and still means just


> that in current FFI.


>


> I'd like to replace them with 'int' and 'uint', but we need to ensure


> backward compatibility etc...


>


> FFI has ExternalType signedLongLong or just 'longlong' and ExternalType


> unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.


> See implementors of #initializeAtomicTypes #initializeFFIConstants


>


> If you want a type that can hold a pointer on both 32 and 64bits arch,


> then


> it's doable with a FFI type alias


> For example I have things like this in HDF5 module


> Notice the single #(fieldName fieldType) which defines an alias, while


> ordinary FFI struct gets an array of #(fieldName fieldType)


> (and replace Size_t with Intptr_t if the purpose is hodling a pointer,


> otherwise the ayatollahs of C standard will lash you)


>


> ExternalStructure subclass: #'Size_t'


>     instanceVariableNames: ''


>     classVariableNames: ''


>     poolDictionaries: ''


>     category: 'HDF5-External-atomic'


>


> Size_t class >>fields


>     "Size_t defineFields"


>     ^Smalltalk wordSize = 4


>         ifTrue: [#( value 'ulong')]


>         ifFalse: [#( value 'ulonglong')]


>


> However the "fields" must be re-generated when platform changes...


> I tried to automate that sometimes ago, see:


>


> ExternalStructure class>>install


>      "Resuming the image on another architecture may require a


> re-compilation of structure layout."


>      | newPlatform |


>      newPlatform := Smalltalk platformName.


>      PreviousPlatform = newPlatform


>          ifFalse:


>              [self recompileStructures.


>             PreviousPlatform := newPlatform]


>


> It works if you switch from Win64 to MacOS64 for example...


> However, this mechanism is not used yet at time of code loading, so


> importing code generated on a different platform won't automatically


> update


> the structures (or alias) - the FFI package has no mechanism for that


> (maybe we could add a hook in MC?).


>


> Le mer. 11 mars 2020 à 23:58, Bert Freudenberg &lt;





> bert@





> &gt; a


> écrit :


>


>> I'd suggest to get OpenGL working outside of Croquet first:


>>


>> http://www.squeaksource.com/CroquetGL.html


>>


>> Step 1: Verify this works in 32 bits. (assuming you are doing this on


>> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)


>> Step 2: Make it work in 64 bits.


>>


>> The second step requires that you understand how FFI works, and how it


>> handles e.g. pointers and integer sizes.


>> I am assuming we do have a working 64 bit FFI, at least for x86_64


>> machines.


>>


>> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a 32


>> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit


>> system,


>> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I


>> don't know how many bits 'ulong' has in our 64 bit FFI, but that


>> declaration may have to change. Etc. pp.


>>


>> If you have questions about FFI then those are best directed at the


>> vm-dev


>> list since it is dealing with VM-level interfaces. CC'ing, please follow


>> up


>> there.


>>


>> - Bert -


>>


>> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <


>>





> squeak-dev@.squeakfoundation





>> wrote:


>>


>>> Okey dokey,


>>>


>>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName after


>>>


>>> openGLLibraryName


>>> ^Smalltalk osVersion = 'linux'


>>> ifTrue: ['libGL.so.1']


>>> ifFalse: ['GL']


>>>


>>> I removed it in my install and got past that error.


>>>


>>> Working exclusively with Croquet(Master)...


>>>


>>>


>>>


>>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:


>>>


>>> glMatrixMode: mode


>>> "This method was automatically generated."


>>> "void glMatrixMode(GLenum mode);"


>>>


> <apicall: void 'glMatrixMode' (ulong) module: '#openGLLibraryName'>


>>> ^self externalCallFailed


>>>


>>> The


> <apicall:...>


>  fails


>>>


>>> How to think about this?


>>>


>>> Is Croquet behind OpenGL latest?


>>> Would teaching myself OpenGL programming be of use to the Croquet


>>> project?


>>>


>>> cheers,


>>>


>>> tty


>>>


>>>


>>>


>>

















--


Sent from: http://forum.world.st/Squeak-VM-f104410.html






--
_,,,^..^,,,_
best, Eliot