Re: [squeak-dev] 64 bit FFI (was: porting Croquet to Squeak6.0 alpha...)

Am 29.05.2020 00:15:25 schrieb Eliot Miranda <[hidden email]>:

On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <>
[hidden email]> wrote:

>
> Hi Marcel,
> no idea...
> Look at timestamp:
> eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set
> FFI-Kernel-EstebanLorenzano.45 ·
> So presumably Eliot added that.
>

Perhaps. I don't remember. But if it is obsolete now, nuke it.


>
> Le jeu. 28 mai 2020 à 17:00, marcel.taeumel a
> écrit :
>
>>
>> Hi Nicolas,
>>
>> what is the role of ExternalStructure class >> #pointerSize then?
>>
>> Best,
>> Marcel
>>
>>
>> Nicolas Cellier wrote
>> > Hi all,
>> > beware of FFI 'long' and 'ulong'.
>> > They are currently meaning (32 bits) int and unsigned int!
>> > This dates back from the 80s when there were still odd OS with segmented
>> > memory and 16-bits int.
>> > long was thought as a safe int32_t at that time... and still means just
>> > that in current FFI.
>> >
>> > I'd like to replace them with 'int' and 'uint', but we need to ensure
>> > backward compatibility etc...
>> >
>> > FFI has ExternalType signedLongLong or just 'longlong' and ExternalType
>> > unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.
>> > See implementors of #initializeAtomicTypes #initializeFFIConstants
>> >
>> > If you want a type that can hold a pointer on both 32 and 64bits arch,
>> > then
>> > it's doable with a FFI type alias
>> > For example I have things like this in HDF5 module
>> > Notice the single #(fieldName fieldType) which defines an alias, while
>> > ordinary FFI struct gets an array of #(fieldName fieldType)
>> > (and replace Size_t with Intptr_t if the purpose is hodling a pointer,
>> > otherwise the ayatollahs of C standard will lash you)
>> >
>> > ExternalStructure subclass: #'Size_t'
>> > instanceVariableNames: ''
>> > classVariableNames: ''
>> > poolDictionaries: ''
>> > category: 'HDF5-External-atomic'
>> >
>> > Size_t class >>fields
>> > "Size_t defineFields"
>> > ^Smalltalk wordSize = 4
>> > ifTrue: [#( value 'ulong')]
>> > ifFalse: [#( value 'ulonglong')]
>> >
>> > However the "fields" must be re-generated when platform changes...
>> > I tried to automate that sometimes ago, see:
>> >
>> > ExternalStructure class>>install
>> > "Resuming the image on another architecture may require a
>> > re-compilation of structure layout."
>> > | newPlatform |
>> > newPlatform := Smalltalk platformName.
>> > PreviousPlatform = newPlatform
>> > ifFalse:
>> > [self recompileStructures.
>> > PreviousPlatform := newPlatform]
>> >
>> > It works if you switch from Win64 to MacOS64 for example...
>> > However, this mechanism is not used yet at time of code loading, so
>> > importing code generated on a different platform won't automatically
>> > update
>> > the structures (or alias) - the FFI package has no mechanism for that
>> > (maybe we could add a hook in MC?).
>> >
>> > Le mer. 11 mars 2020 à 23:58, Bert Freudenberg <
>>
>> > bert@
>>
>> > > a
>> > écrit :
>> >
>> >> I'd suggest to get OpenGL working outside of Croquet first:
>> >>
>> >> http://www.squeaksource.com/CroquetGL.html
>> >>
>> >> Step 1: Verify this works in 32 bits. (assuming you are doing this on
>> >> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)
>> >> Step 2: Make it work in 64 bits.
>> >>
>> >> The second step requires that you understand how FFI works, and how it
>> >> handles e.g. pointers and integer sizes.
>> >> I am assuming we do have a working 64 bit FFI, at least for x86_64
>> >> machines.
>> >>
>> >> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a
>> 32
>> >> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit
>> >> system,
>> >> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I
>> >> don't know how many bits 'ulong' has in our 64 bit FFI, but that
>> >> declaration may have to change. Etc. pp.
>> >>
>> >> If you have questions about FFI then those are best directed at the
>> >> vm-dev
>> >> list since it is dealing with VM-level interfaces. CC'ing, please
>> follow
>> >> up
>> >> there.
>> >>
>> >> - Bert -
>> >>
>> >> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <>
>> >>
>>
>> > squeak-dev@.squeakfoundation
>>
>> >> wrote:
>> >>
>> >>> Okey dokey,
>> >>>
>> >>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName
>> after
>> >>>
>> >>> openGLLibraryName
>> >>> ^Smalltalk osVersion = 'linux'
>> >>> ifTrue: ['libGL.so.1']
>> >>> ifFalse: ['GL']
>> >>>
>> >>> I removed it in my install and got past that error.
>> >>>
>> >>> Working exclusively with Croquet(Master)...
>> >>>
>> >>>
>> >>>
>> >>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:
>> >>>
>> >>> glMatrixMode: mode
>> >>> "This method was automatically generated."
>> >>> "void glMatrixMode(GLenum mode);"
>> >>>
>> >
>> >>> ^self externalCallFailed
>> >>>
>> >>> The
>> >
>> > fails
>> >>>
>> >>> How to think about this?
>> >>>
>> >>> Is Croquet behind OpenGL latest?
>> >>> Would teaching myself OpenGL programming be of use to the Croquet
>> >>> project?
>> >>>
>> >>> cheers,
>> >>>
>> >>> tty
>> >>>
>> >>>
>> >>>
>> >>
>>
>>
>>
>>
>>
>> --
>> Sent from: http://forum.world.st/Squeak-VM-f104410.html
>>
>

--
_,,,^..^,,,_
best, Eliot

On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <[hidden email]> wrote:

 

Hi Marcel,

no idea...

Look at timestamp:

eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set FFI-Kernel-EstebanLorenzano.45  ·

So presumably Eliot added that.

Perhaps.  I don't remember.  But if it is obsolete now, nuke it.

 

Le jeu. 28 mai 2020 à 17:00, marcel.taeumel <[hidden email]> a écrit :

 

Hi Nicolas,



what is the role of ExternalStructure class >> #pointerSize then?



Best,

Marcel





Nicolas Cellier wrote

> Hi all,

> beware of FFI 'long' and 'ulong'.

> They are currently meaning (32 bits) int and unsigned int!

> This dates back from the 80s when there were still odd OS with segmented

> memory and 16-bits int.

> long was thought as a safe int32_t at that time... and still means just

> that in current FFI.

>

> I'd like to replace them with 'int' and 'uint', but we need to ensure

> backward compatibility etc...

>

> FFI has ExternalType signedLongLong or just 'longlong' and ExternalType

> unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.

> See implementors of #initializeAtomicTypes #initializeFFIConstants

>

> If you want a type that can hold a pointer on both 32 and 64bits arch,

> then

> it's doable with a FFI type alias

> For example I have things like this in HDF5 module

> Notice the single #(fieldName fieldType) which defines an alias, while

> ordinary FFI struct gets an array of #(fieldName fieldType)

> (and replace Size_t with Intptr_t if the purpose is hodling a pointer,

> otherwise the ayatollahs of C standard will lash you)

>

> ExternalStructure subclass: #'Size_t'

>     instanceVariableNames: ''

>     classVariableNames: ''

>     poolDictionaries: ''

>     category: 'HDF5-External-atomic'

>

> Size_t class >>fields

>     "Size_t defineFields"

>     ^Smalltalk wordSize = 4

>         ifTrue: [#( value 'ulong')]

>         ifFalse: [#( value 'ulonglong')]

>

> However the "fields" must be re-generated when platform changes...

> I tried to automate that sometimes ago, see:

>

> ExternalStructure class>>install

>      "Resuming the image on another architecture may require a

> re-compilation of structure layout."

>      | newPlatform |

>      newPlatform := Smalltalk platformName.

>      PreviousPlatform = newPlatform

>          ifFalse:

>              [self recompileStructures.

>             PreviousPlatform := newPlatform]

>

> It works if you switch from Win64 to MacOS64 for example...

> However, this mechanism is not used yet at time of code loading, so

> importing code generated on a different platform won't automatically

> update

> the structures (or alias) - the FFI package has no mechanism for that

> (maybe we could add a hook in MC?).

>

> Le mer. 11 mars 2020 à 23:58, Bert Freudenberg &lt;



> bert@



> &gt; a

> écrit :

>

>> I'd suggest to get OpenGL working outside of Croquet first:

>>

>> http://www.squeaksource.com/CroquetGL.html

>>

>> Step 1: Verify this works in 32 bits. (assuming you are doing this on

>> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)

>> Step 2: Make it work in 64 bits.

>>

>> The second step requires that you understand how FFI works, and how it

>> handles e.g. pointers and integer sizes.

>> I am assuming we do have a working 64 bit FFI, at least for x86_64

>> machines.

>>

>> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a 32

>> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit

>> system,

>> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I

>> don't know how many bits 'ulong' has in our 64 bit FFI, but that

>> declaration may have to change. Etc. pp.

>>

>> If you have questions about FFI then those are best directed at the

>> vm-dev

>> list since it is dealing with VM-level interfaces. CC'ing, please follow

>> up

>> there.

>>

>> - Bert -

>>

>> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <

>>



> squeak-dev@.squeakfoundation



>> wrote:

>>

>>> Okey dokey,

>>>

>>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName after

>>>

>>> openGLLibraryName

>>> ^Smalltalk osVersion = 'linux'

>>> ifTrue: ['libGL.so.1']

>>> ifFalse: ['GL']

>>>

>>> I removed it in my install and got past that error.

>>>

>>> Working exclusively with Croquet(Master)...

>>>

>>>

>>>

>>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:

>>>

>>> glMatrixMode: mode

>>> "This method was automatically generated."

>>> "void glMatrixMode(GLenum mode);"

>>>

> <apicall: void 'glMatrixMode' (ulong) module: '#openGLLibraryName'>

>>> ^self externalCallFailed

>>>

>>> The

> <apicall:...>

>  fails

>>>

>>> How to think about this?

>>>

>>> Is Croquet behind OpenGL latest?

>>> Would teaching myself OpenGL programming be of use to the Croquet

>>> project?

>>>

>>> cheers,

>>>

>>> tty

>>>

>>>

>>>

>>











--

Sent from: http://forum.world.st/Squeak-VM-f104410.html

--

_,,,^..^,,,_

best, Eliot

Am 30.05.2020 15:59:31 schrieb Marcel Taeumel <[hidden email]>:

> But if it is obsolete now, nuke it.

I cannot tell. That information is stored in the external type's compiledSpec to be used from within the FFI plugin I suppose.

Best,

Marcel

Am 29.05.2020 00:15:25 schrieb Eliot Miranda <[hidden email]>:

On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <>
[hidden email]> wrote:

>
> Hi Marcel,
> no idea...
> Look at timestamp:
> eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set
> FFI-Kernel-EstebanLorenzano.45 ·
> So presumably Eliot added that.
>

Perhaps. I don't remember. But if it is obsolete now, nuke it.

>
> Le jeu. 28 mai 2020 à 17:00, marcel.taeumel a
> écrit :
>
>>
>> Hi Nicolas,
>>
>> what is the role of ExternalStructure class >> #pointerSize then?
>>
>> Best,
>> Marcel
>>
>>
>> Nicolas Cellier wrote
>> > Hi all,
>> > beware of FFI 'long' and 'ulong'.
>> > They are currently meaning (32 bits) int and unsigned int!
>> > This dates back from the 80s when there were still odd OS with segmented
>> > memory and 16-bits int.
>> > long was thought as a safe int32_t at that time... and still means just
>> > that in current FFI.
>> >
>> > I'd like to replace them with 'int' and 'uint', but we need to ensure
>> > backward compatibility etc...
>> >
>> > FFI has ExternalType signedLongLong or just 'longlong' and ExternalType
>> > unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.
>> > See implementors of #initializeAtomicTypes #initializeFFIConstants
>> >
>> > If you want a type that can hold a pointer on both 32 and 64bits arch,
>> > then
>> > it's doable with a FFI type alias
>> > For example I have things like this in HDF5 module
>> > Notice the single #(fieldName fieldType) which defines an alias, while
>> > ordinary FFI struct gets an array of #(fieldName fieldType)
>> > (and replace Size_t with Intptr_t if the purpose is hodling a pointer,
>> > otherwise the ayatollahs of C standard will lash you)
>> >
>> > ExternalStructure subclass: #'Size_t'
>> > instanceVariableNames: ''
>> > classVariableNames: ''
>> > poolDictionaries: ''
>> > category: 'HDF5-External-atomic'
>> >
>> > Size_t class >>fields
>> > "Size_t defineFields"
>> > ^Smalltalk wordSize = 4
>> > ifTrue: [#( value 'ulong')]
>> > ifFalse: [#( value 'ulonglong')]
>> >
>> > However the "fields" must be re-generated when platform changes...
>> > I tried to automate that sometimes ago, see:
>> >
>> > ExternalStructure class>>install
>> > "Resuming the image on another architecture may require a
>> > re-compilation of structure layout."
>> > | newPlatform |
>> > newPlatform := Smalltalk platformName.
>> > PreviousPlatform = newPlatform
>> > ifFalse:
>> > [self recompileStructures.
>> > PreviousPlatform := newPlatform]
>> >
>> > It works if you switch from Win64 to MacOS64 for example...
>> > However, this mechanism is not used yet at time of code loading, so
>> > importing code generated on a different platform won't automatically
>> > update
>> > the structures (or alias) - the FFI package has no mechanism for that
>> > (maybe we could add a hook in MC?).
>> >
>> > Le mer. 11 mars 2020 à 23:58, Bert Freudenberg <
>>
>> > bert@
>>
>> > > a
>> > écrit :
>> >
>> >> I'd suggest to get OpenGL working outside of Croquet first:
>> >>
>> >> http://www.squeaksource.com/CroquetGL.html
>> >>
>> >> Step 1: Verify this works in 32 bits. (assuming you are doing this on
>> >> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)
>> >> Step 2: Make it work in 64 bits.
>> >>
>> >> The second step requires that you understand how FFI works, and how it
>> >> handles e.g. pointers and integer sizes.
>> >> I am assuming we do have a working 64 bit FFI, at least for x86_64
>> >> machines.
>> >>
>> >> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a
>> 32
>> >> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit
>> >> system,
>> >> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I
>> >> don't know how many bits 'ulong' has in our 64 bit FFI, but that
>> >> declaration may have to change. Etc. pp.
>> >>
>> >> If you have questions about FFI then those are best directed at the
>> >> vm-dev
>> >> list since it is dealing with VM-level interfaces. CC'ing, please
>> follow
>> >> up
>> >> there.
>> >>
>> >> - Bert -
>> >>
>> >> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <>
>> >>
>>
>> > squeak-dev@.squeakfoundation
>>
>> >> wrote:
>> >>
>> >>> Okey dokey,
>> >>>
>> >>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName
>> after
>> >>>
>> >>> openGLLibraryName
>> >>> ^Smalltalk osVersion = 'linux'
>> >>> ifTrue: ['libGL.so.1']
>> >>> ifFalse: ['GL']
>> >>>
>> >>> I removed it in my install and got past that error.
>> >>>
>> >>> Working exclusively with Croquet(Master)...
>> >>>
>> >>>
>> >>>
>> >>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:
>> >>>
>> >>> glMatrixMode: mode
>> >>> "This method was automatically generated."
>> >>> "void glMatrixMode(GLenum mode);"
>> >>>
>> >
>> >>> ^self externalCallFailed
>> >>>
>> >>> The
>> >
>> > fails
>> >>>
>> >>> How to think about this?
>> >>>
>> >>> Is Croquet behind OpenGL latest?
>> >>> Would teaching myself OpenGL programming be of use to the Croquet
>> >>> project?
>> >>>
>> >>> cheers,
>> >>>
>> >>> tty
>> >>>
>> >>>
>> >>>
>> >>
>>
>>
>>
>>
>>
>> --
>> Sent from: http://forum.world.st/Squeak-VM-f104410.html
>>
>

--
_,,,^..^,,,_
best, Eliot

On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <[hidden email]> wrote:

 

Hi Marcel,

no idea...

Look at timestamp:

eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set FFI-Kernel-EstebanLorenzano.45  ·

So presumably Eliot added that.

Perhaps.  I don't remember.  But if it is obsolete now, nuke it.

 

Le jeu. 28 mai 2020 à 17:00, marcel.taeumel <[hidden email]> a écrit :

 

Hi Nicolas,



what is the role of ExternalStructure class >> #pointerSize then?



Best,

Marcel





Nicolas Cellier wrote

> Hi all,

> beware of FFI 'long' and 'ulong'.

> They are currently meaning (32 bits) int and unsigned int!

> This dates back from the 80s when there were still odd OS with segmented

> memory and 16-bits int.

> long was thought as a safe int32_t at that time... and still means just

> that in current FFI.

>

> I'd like to replace them with 'int' and 'uint', but we need to ensure

> backward compatibility etc...

>

> FFI has ExternalType signedLongLong or just 'longlong' and ExternalType

> unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.

> See implementors of #initializeAtomicTypes #initializeFFIConstants

>

> If you want a type that can hold a pointer on both 32 and 64bits arch,

> then

> it's doable with a FFI type alias

> For example I have things like this in HDF5 module

> Notice the single #(fieldName fieldType) which defines an alias, while

> ordinary FFI struct gets an array of #(fieldName fieldType)

> (and replace Size_t with Intptr_t if the purpose is hodling a pointer,

> otherwise the ayatollahs of C standard will lash you)

>

> ExternalStructure subclass: #'Size_t'

>     instanceVariableNames: ''

>     classVariableNames: ''

>     poolDictionaries: ''

>     category: 'HDF5-External-atomic'

>

> Size_t class >>fields

>     "Size_t defineFields"

>     ^Smalltalk wordSize = 4

>         ifTrue: [#( value 'ulong')]

>         ifFalse: [#( value 'ulonglong')]

>

> However the "fields" must be re-generated when platform changes...

> I tried to automate that sometimes ago, see:

>

> ExternalStructure class>>install

>      "Resuming the image on another architecture may require a

> re-compilation of structure layout."

>      | newPlatform |

>      newPlatform := Smalltalk platformName.

>      PreviousPlatform = newPlatform

>          ifFalse:

>              [self recompileStructures.

>             PreviousPlatform := newPlatform]

>

> It works if you switch from Win64 to MacOS64 for example...

> However, this mechanism is not used yet at time of code loading, so

> importing code generated on a different platform won't automatically

> update

> the structures (or alias) - the FFI package has no mechanism for that

> (maybe we could add a hook in MC?).

>

> Le mer. 11 mars 2020 à 23:58, Bert Freudenberg &lt;



> bert@



> &gt; a

> écrit :

>

>> I'd suggest to get OpenGL working outside of Croquet first:

>>

>> http://www.squeaksource.com/CroquetGL.html

>>

>> Step 1: Verify this works in 32 bits. (assuming you are doing this on

>> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)

>> Step 2: Make it work in 64 bits.

>>

>> The second step requires that you understand how FFI works, and how it

>> handles e.g. pointers and integer sizes.

>> I am assuming we do have a working 64 bit FFI, at least for x86_64

>> machines.

>>

>> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a 32

>> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit

>> system,

>> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I

>> don't know how many bits 'ulong' has in our 64 bit FFI, but that

>> declaration may have to change. Etc. pp.

>>

>> If you have questions about FFI then those are best directed at the

>> vm-dev

>> list since it is dealing with VM-level interfaces. CC'ing, please follow

>> up

>> there.

>>

>> - Bert -

>>

>> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <

>>



> squeak-dev@.squeakfoundation



>> wrote:

>>

>>> Okey dokey,

>>>

>>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName after

>>>

>>> openGLLibraryName

>>> ^Smalltalk osVersion = 'linux'

>>> ifTrue: ['libGL.so.1']

>>> ifFalse: ['GL']

>>>

>>> I removed it in my install and got past that error.

>>>

>>> Working exclusively with Croquet(Master)...

>>>

>>>

>>>

>>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:

>>>

>>> glMatrixMode: mode

>>> "This method was automatically generated."

>>> "void glMatrixMode(GLenum mode);"

>>>

> <apicall: void 'glMatrixMode' (ulong) module: '#openGLLibraryName'>

>>> ^self externalCallFailed

>>>

>>> The

> <apicall:...>

>  fails

>>>

>>> How to think about this?

>>>

>>> Is Croquet behind OpenGL latest?

>>> Would teaching myself OpenGL programming be of use to the Croquet

>>> project?

>>>

>>> cheers,

>>>

>>> tty

>>>

>>>

>>>

>>











--

Sent from: http://forum.world.st/Squeak-VM-f104410.html

--

_,,,^..^,,,_

best, Eliot

Am 30.05.2020 18:00:33 schrieb Marcel Taeumel <[hidden email]>:

So, if the atomic types and structure types are initialized, they make use of ExternalType class >> #pointerSpec to store the size of the pointer in the #headerWord, which is the first element of an external type's 'compiledSpec'. For atomic types, this happens when the FFI package is loaded.

For structure types, this happens when a an external structure is mentioned for the first time in an FFI call method or in during compilation of another external structure's fields. See ExternalType class >> #newTypeNamed:force:. Squeak's Smalltalk parser or the compile logic in ExternalStructure invokes that type creation respectively..

Currently, if a platform change at system startup is detected, all atomic types and structure types are updated --- which implies a call to #pointerSpec in ExternalType class >> #initializeStructureTypes.

Now, what could be the situation where a struct-pointer type had the wrong pointer size? During development, your platform does not magically change the #wordSize. When loading code, methods in the external struct might be wrong (bc. Monticello does not trigger that field re-definition) BUT any external struct-pointer type that is already in the image must have the right size. And for new external struct-pointer types, it should work as explained above.

So, I think we should make the following changes:

- Remove ExternalType >> #asPointerType: (! note the colon !)

- Remove ExternalType >> #pointerSize:

- Change ExternalType >> #pointerSize to just read the compiledSpec

- Update all sends to #asPointerType: to #asPointerType and assume that the current pointer-type instances have the right size, as described above.

Best,

Marcel

Am 30.05.2020 15:59:31 schrieb Marcel Taeumel <[hidden email]>:

> But if it is obsolete now, nuke it.

I cannot tell. That information is stored in the external type's compiledSpec to be used from within the FFI plugin I suppose.

Best,

Marcel

Am 29.05.2020 00:15:25 schrieb Eliot Miranda <[hidden email]>:

On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <>
[hidden email]> wrote:

>
> Hi Marcel,
> no idea...
> Look at timestamp:
> eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set
> FFI-Kernel-EstebanLorenzano.45 ·
> So presumably Eliot added that.
>

Perhaps. I don't remember. But if it is obsolete now, nuke it.

>
> Le jeu. 28 mai 2020 à 17:00, marcel.taeumel a
> écrit :
>
>>
>> Hi Nicolas,
>>
>> what is the role of ExternalStructure class >> #pointerSize then?
>>
>> Best,
>> Marcel
>>
>>
>> Nicolas Cellier wrote
>> > Hi all,
>> > beware of FFI 'long' and 'ulong'.
>> > They are currently meaning (32 bits) int and unsigned int!
>> > This dates back from the 80s when there were still odd OS with segmented
>> > memory and 16-bits int.
>> > long was thought as a safe int32_t at that time... and still means just
>> > that in current FFI.
>> >
>> > I'd like to replace them with 'int' and 'uint', but we need to ensure
>> > backward compatibility etc...
>> >
>> > FFI has ExternalType signedLongLong or just 'longlong' and ExternalType
>> > unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.
>> > See implementors of #initializeAtomicTypes #initializeFFIConstants
>> >
>> > If you want a type that can hold a pointer on both 32 and 64bits arch,
>> > then
>> > it's doable with a FFI type alias
>> > For example I have things like this in HDF5 module
>> > Notice the single #(fieldName fieldType) which defines an alias, while
>> > ordinary FFI struct gets an array of #(fieldName fieldType)
>> > (and replace Size_t with Intptr_t if the purpose is hodling a pointer,
>> > otherwise the ayatollahs of C standard will lash you)
>> >
>> > ExternalStructure subclass: #'Size_t'
>> > instanceVariableNames: ''
>> > classVariableNames: ''
>> > poolDictionaries: ''
>> > category: 'HDF5-External-atomic'
>> >
>> > Size_t class >>fields
>> > "Size_t defineFields"
>> > ^Smalltalk wordSize = 4
>> > ifTrue: [#( value 'ulong')]
>> > ifFalse: [#( value 'ulonglong')]
>> >
>> > However the "fields" must be re-generated when platform changes...
>> > I tried to automate that sometimes ago, see:
>> >
>> > ExternalStructure class>>install
>> > "Resuming the image on another architecture may require a
>> > re-compilation of structure layout."
>> > | newPlatform |
>> > newPlatform := Smalltalk platformName.
>> > PreviousPlatform = newPlatform
>> > ifFalse:
>> > [self recompileStructures.
>> > PreviousPlatform := newPlatform]
>> >
>> > It works if you switch from Win64 to MacOS64 for example...
>> > However, this mechanism is not used yet at time of code loading, so
>> > importing code generated on a different platform won't automatically
>> > update
>> > the structures (or alias) - the FFI package has no mechanism for that
>> > (maybe we could add a hook in MC?).
>> >
>> > Le mer. 11 mars 2020 à 23:58, Bert Freudenberg <
>>
>> > bert@
>>
>> > > a
>> > écrit :
>> >
>> >> I'd suggest to get OpenGL working outside of Croquet first:
>> >>
>> >> http://www.squeaksource.com/CroquetGL.html
>> >>
>> >> Step 1: Verify this works in 32 bits. (assuming you are doing this on
>> >> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)
>> >> Step 2: Make it work in 64 bits.
>> >>
>> >> The second step requires that you understand how FFI works, and how it
>> >> handles e.g. pointers and integer sizes.
>> >> I am assuming we do have a working 64 bit FFI, at least for x86_64
>> >> machines.
>> >>
>> >> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a
>> 32
>> >> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit
>> >> system,
>> >> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I
>> >> don't know how many bits 'ulong' has in our 64 bit FFI, but that
>> >> declaration may have to change. Etc. pp.
>> >>
>> >> If you have questions about FFI then those are best directed at the
>> >> vm-dev
>> >> list since it is dealing with VM-level interfaces. CC'ing, please
>> follow
>> >> up
>> >> there.
>> >>
>> >> - Bert -
>> >>
>> >> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <>
>> >>
>>
>> > squeak-dev@.squeakfoundation
>>
>> >> wrote:
>> >>
>> >>> Okey dokey,
>> >>>
>> >>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName
>> after
>> >>>
>> >>> openGLLibraryName
>> >>> ^Smalltalk osVersion = 'linux'
>> >>> ifTrue: ['libGL.so.1']
>> >>> ifFalse: ['GL']
>> >>>
>> >>> I removed it in my install and got past that error.
>> >>>
>> >>> Working exclusively with Croquet(Master)...
>> >>>
>> >>>
>> >>>
>> >>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:
>> >>>
>> >>> glMatrixMode: mode
>> >>> "This method was automatically generated."
>> >>> "void glMatrixMode(GLenum mode);"
>> >>>
>> >
>> >>> ^self externalCallFailed
>> >>>
>> >>> The
>> >
>> > fails
>> >>>
>> >>> How to think about this?
>> >>>
>> >>> Is Croquet behind OpenGL latest?
>> >>> Would teaching myself OpenGL programming be of use to the Croquet
>> >>> project?
>> >>>
>> >>> cheers,
>> >>>
>> >>> tty
>> >>>
>> >>>
>> >>>
>> >>
>>
>>
>>
>>
>>
>> --
>> Sent from: http://forum.world.st/Squeak-VM-f104410.html
>>
>

--
_,,,^..^,,,_
best, Eliot

On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <[hidden email]> wrote:

 

Hi Marcel,

no idea...

Look at timestamp:

eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set FFI-Kernel-EstebanLorenzano.45  ·

So presumably Eliot added that.

Perhaps.  I don't remember.  But if it is obsolete now, nuke it.

 

Le jeu. 28 mai 2020 à 17:00, marcel.taeumel <[hidden email]> a écrit :

 

Hi Nicolas,



what is the role of ExternalStructure class >> #pointerSize then?



Best,

Marcel





Nicolas Cellier wrote

> Hi all,

> beware of FFI 'long' and 'ulong'.

> They are currently meaning (32 bits) int and unsigned int!

> This dates back from the 80s when there were still odd OS with segmented

> memory and 16-bits int.

> long was thought as a safe int32_t at that time... and still means just

> that in current FFI.

>

> I'd like to replace them with 'int' and 'uint', but we need to ensure

> backward compatibility etc...

>

> FFI has ExternalType signedLongLong or just 'longlong' and ExternalType

> unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.

> See implementors of #initializeAtomicTypes #initializeFFIConstants

>

> If you want a type that can hold a pointer on both 32 and 64bits arch,

> then

> it's doable with a FFI type alias

> For example I have things like this in HDF5 module

> Notice the single #(fieldName fieldType) which defines an alias, while

> ordinary FFI struct gets an array of #(fieldName fieldType)

> (and replace Size_t with Intptr_t if the purpose is hodling a pointer,

> otherwise the ayatollahs of C standard will lash you)

>

> ExternalStructure subclass: #'Size_t'

>     instanceVariableNames: ''

>     classVariableNames: ''

>     poolDictionaries: ''

>     category: 'HDF5-External-atomic'

>

> Size_t class >>fields

>     "Size_t defineFields"

>     ^Smalltalk wordSize = 4

>         ifTrue: [#( value 'ulong')]

>         ifFalse: [#( value 'ulonglong')]

>

> However the "fields" must be re-generated when platform changes...

> I tried to automate that sometimes ago, see:

>

> ExternalStructure class>>install

>      "Resuming the image on another architecture may require a

> re-compilation of structure layout."

>      | newPlatform |

>      newPlatform := Smalltalk platformName.

>      PreviousPlatform = newPlatform

>          ifFalse:

>              [self recompileStructures.

>             PreviousPlatform := newPlatform]

>

> It works if you switch from Win64 to MacOS64 for example...

> However, this mechanism is not used yet at time of code loading, so

> importing code generated on a different platform won't automatically

> update

> the structures (or alias) - the FFI package has no mechanism for that

> (maybe we could add a hook in MC?).

>

> Le mer. 11 mars 2020 à 23:58, Bert Freudenberg &lt;



> bert@



> &gt; a

> écrit :

>

>> I'd suggest to get OpenGL working outside of Croquet first:

>>

>> http://www.squeaksource.com/CroquetGL.html

>>

>> Step 1: Verify this works in 32 bits. (assuming you are doing this on

>> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)

>> Step 2: Make it work in 64 bits.

>>

>> The second step requires that you understand how FFI works, and how it

>> handles e.g. pointers and integer sizes.

>> I am assuming we do have a working 64 bit FFI, at least for x86_64

>> machines.

>>

>> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a 32

>> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit

>> system,

>> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I

>> don't know how many bits 'ulong' has in our 64 bit FFI, but that

>> declaration may have to change. Etc. pp.

>>

>> If you have questions about FFI then those are best directed at the

>> vm-dev

>> list since it is dealing with VM-level interfaces. CC'ing, please follow

>> up

>> there.

>>

>> - Bert -

>>

>> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <

>>



> squeak-dev@.squeakfoundation



>> wrote:

>>

>>> Okey dokey,

>>>

>>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName after

>>>

>>> openGLLibraryName

>>> ^Smalltalk osVersion = 'linux'

>>> ifTrue: ['libGL.so.1']

>>> ifFalse: ['GL']

>>>

>>> I removed it in my install and got past that error.

>>>

>>> Working exclusively with Croquet(Master)...

>>>

>>>

>>>

>>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:

>>>

>>> glMatrixMode: mode

>>> "This method was automatically generated."

>>> "void glMatrixMode(GLenum mode);"

>>>

> <apicall: void 'glMatrixMode' (ulong) module: '#openGLLibraryName'>

>>> ^self externalCallFailed

>>>

>>> The

> <apicall:...>

>  fails

>>>

>>> How to think about this?

>>>

>>> Is Croquet behind OpenGL latest?

>>> Would teaching myself OpenGL programming be of use to the Croquet

>>> project?

>>>

>>> cheers,

>>>

>>> tty

>>>

>>>

>>>

>>











--

Sent from: http://forum.world.st/Squeak-VM-f104410.html

--

_,,,^..^,,,_

best, Eliot

On Jun 9, 2020, at 5:10 AM, Marcel Taeumel <[hidden email]> wrote:

Best,

Marcel

Am 30.05.2020 18:00:33 schrieb Marcel Taeumel <[hidden email]>:

So, if the atomic types and structure types are initialized, they make use of ExternalType class >> #pointerSpec to store the size of the pointer in the #headerWord, which is the first element of an external type's 'compiledSpec'. For atomic types, this happens when the FFI package is loaded.

For structure types, this happens when a an external structure is mentioned for the first time in an FFI call method or in during compilation of another external structure's fields. See ExternalType class >> #newTypeNamed:force:. Squeak's Smalltalk parser or the compile logic in ExternalStructure invokes that type creation respectively..

Currently, if a platform change at system startup is detected, all atomic types and structure types are updated --- which implies a call to #pointerSpec in ExternalType class >> #initializeStructureTypes.

Now, what could be the situation where a struct-pointer type had the wrong pointer size? During development, your platform does not magically change the #wordSize. When loading code, methods in the external struct might be wrong (bc. Monticello does not trigger that field re-definition) BUT any external struct-pointer type that is already in the image must have the right size. And for new external struct-pointer types, it should work as explained above.

So, I think we should make the following changes:

- Remove ExternalType >> #asPointerType: (! note the colon !)

- Remove ExternalType >> #pointerSize:

- Change ExternalType >> #pointerSize to just read the compiledSpec

- Update all sends to #asPointerType: to #asPointerType and assume that the current pointer-type instances have the right size, as described above.

Best,

Marcel

Am 30.05.2020 15:59:31 schrieb Marcel Taeumel <[hidden email]>:

> But if it is obsolete now, nuke it.

I cannot tell. That information is stored in the external type's compiledSpec to be used from within the FFI plugin I suppose.

Best,

Marcel

Am 29.05.2020 00:15:25 schrieb Eliot Miranda <[hidden email]>:

On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <>
[hidden email]> wrote:

>
> Hi Marcel,
> no idea...
> Look at timestamp:
> eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set
> FFI-Kernel-EstebanLorenzano.45 ·
> So presumably Eliot added that.
>

Perhaps. I don't remember. But if it is obsolete now, nuke it.

>
> Le jeu. 28 mai 2020 à 17:00, marcel.taeumel a
> écrit :
>
>>
>> Hi Nicolas,
>>
>> what is the role of ExternalStructure class >> #pointerSize then?
>>
>> Best,
>> Marcel
>>
>>
>> Nicolas Cellier wrote
>> > Hi all,
>> > beware of FFI 'long' and 'ulong'.
>> > They are currently meaning (32 bits) int and unsigned int!
>> > This dates back from the 80s when there were still odd OS with segmented
>> > memory and 16-bits int.
>> > long was thought as a safe int32_t at that time... and still means just
>> > that in current FFI.
>> >
>> > I'd like to replace them with 'int' and 'uint', but we need to ensure
>> > backward compatibility etc...
>> >
>> > FFI has ExternalType signedLongLong or just 'longlong' and ExternalType
>> > unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.
>> > See implementors of #initializeAtomicTypes #initializeFFIConstants
>> >
>> > If you want a type that can hold a pointer on both 32 and 64bits arch,
>> > then
>> > it's doable with a FFI type alias
>> > For example I have things like this in HDF5 module
>> > Notice the single #(fieldName fieldType) which defines an alias, while
>> > ordinary FFI struct gets an array of #(fieldName fieldType)
>> > (and replace Size_t with Intptr_t if the purpose is hodling a pointer,
>> > otherwise the ayatollahs of C standard will lash you)
>> >
>> > ExternalStructure subclass: #'Size_t'
>> > instanceVariableNames: ''
>> > classVariableNames: ''
>> > poolDictionaries: ''
>> > category: 'HDF5-External-atomic'
>> >
>> > Size_t class >>fields
>> > "Size_t defineFields"
>> > ^Smalltalk wordSize = 4
>> > ifTrue: [#( value 'ulong')]
>> > ifFalse: [#( value 'ulonglong')]
>> >
>> > However the "fields" must be re-generated when platform changes...
>> > I tried to automate that sometimes ago, see:
>> >
>> > ExternalStructure class>>install
>> > "Resuming the image on another architecture may require a
>> > re-compilation of structure layout."
>> > | newPlatform |
>> > newPlatform := Smalltalk platformName.
>> > PreviousPlatform = newPlatform
>> > ifFalse:
>> > [self recompileStructures.
>> > PreviousPlatform := newPlatform]
>> >
>> > It works if you switch from Win64 to MacOS64 for example...
>> > However, this mechanism is not used yet at time of code loading, so
>> > importing code generated on a different platform won't automatically
>> > update
>> > the structures (or alias) - the FFI package has no mechanism for that
>> > (maybe we could add a hook in MC?).
>> >
>> > Le mer. 11 mars 2020 à 23:58, Bert Freudenberg <
>>
>> > bert@
>>
>> > > a
>> > écrit :
>> >
>> >> I'd suggest to get OpenGL working outside of Croquet first:
>> >>
>> >> http://www.squeaksource.com/CroquetGL.html
>> >>
>> >> Step 1: Verify this works in 32 bits. (assuming you are doing this on
>> >> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)
>> >> Step 2: Make it work in 64 bits.
>> >>
>> >> The second step requires that you understand how FFI works, and how it
>> >> handles e.g. pointers and integer sizes.
>> >> I am assuming we do have a working 64 bit FFI, at least for x86_64
>> >> machines.
>> >>
>> >> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a
>> 32
>> >> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit
>> >> system,
>> >> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I
>> >> don't know how many bits 'ulong' has in our 64 bit FFI, but that
>> >> declaration may have to change. Etc. pp.
>> >>
>> >> If you have questions about FFI then those are best directed at the
>> >> vm-dev
>> >> list since it is dealing with VM-level interfaces. CC'ing, please
>> follow
>> >> up
>> >> there.
>> >>
>> >> - Bert -
>> >>
>> >> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <>
>> >>
>>
>> > squeak-dev@.squeakfoundation
>>
>> >> wrote:
>> >>
>> >>> Okey dokey,
>> >>>
>> >>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName
>> after
>> >>>
>> >>> openGLLibraryName
>> >>> ^Smalltalk osVersion = 'linux'
>> >>> ifTrue: ['libGL.so.1']
>> >>> ifFalse: ['GL']
>> >>>
>> >>> I removed it in my install and got past that error.
>> >>>
>> >>> Working exclusively with Croquet(Master)...
>> >>>
>> >>>
>> >>>
>> >>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:
>> >>>
>> >>> glMatrixMode: mode
>> >>> "This method was automatically generated."
>> >>> "void glMatrixMode(GLenum mode);"
>> >>>
>> >
>> >>> ^self externalCallFailed
>> >>>
>> >>> The
>> >
>> > fails
>> >>>
>> >>> How to think about this?
>> >>>
>> >>> Is Croquet behind OpenGL latest?
>> >>> Would teaching myself OpenGL programming be of use to the Croquet
>> >>> project?
>> >>>
>> >>> cheers,
>> >>>
>> >>> tty
>> >>>
>> >>>
>> >>>
>> >>
>>
>>
>>
>>
>>
>> --
>> Sent from: http://forum.world.st/Squeak-VM-f104410.html
>>
>

--
_,,,^..^,,,_
best, Eliot

On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <[hidden email]> wrote:

 

Hi Marcel,

no idea...

Look at timestamp:

eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set FFI-Kernel-EstebanLorenzano.45  ·

So presumably Eliot added that.

Perhaps.  I don't remember.  But if it is obsolete now, nuke it.

 

Le jeu. 28 mai 2020 à 17:00, marcel.taeumel <[hidden email]> a écrit :

 

Hi Nicolas,



what is the role of ExternalStructure class >> #pointerSize then?



Best,

Marcel





Nicolas Cellier wrote

> Hi all,

> beware of FFI 'long' and 'ulong'.

> They are currently meaning (32 bits) int and unsigned int!

> This dates back from the 80s when there were still odd OS with segmented

> memory and 16-bits int.

> long was thought as a safe int32_t at that time... and still means just

> that in current FFI.

>

> I'd like to replace them with 'int' and 'uint', but we need to ensure

> backward compatibility etc...

>

> FFI has ExternalType signedLongLong or just 'longlong' and ExternalType

> unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.

> See implementors of #initializeAtomicTypes #initializeFFIConstants

>

> If you want a type that can hold a pointer on both 32 and 64bits arch,

> then

> it's doable with a FFI type alias

> For example I have things like this in HDF5 module

> Notice the single #(fieldName fieldType) which defines an alias, while

> ordinary FFI struct gets an array of #(fieldName fieldType)

> (and replace Size_t with Intptr_t if the purpose is hodling a pointer,

> otherwise the ayatollahs of C standard will lash you)

>

> ExternalStructure subclass: #'Size_t'

>     instanceVariableNames: ''

>     classVariableNames: ''

>     poolDictionaries: ''

>     category: 'HDF5-External-atomic'

>

> Size_t class >>fields

>     "Size_t defineFields"

>     ^Smalltalk wordSize = 4

>         ifTrue: [#( value 'ulong')]

>         ifFalse: [#( value 'ulonglong')]

>

> However the "fields" must be re-generated when platform changes...

> I tried to automate that sometimes ago, see:

>

> ExternalStructure class>>install

>      "Resuming the image on another architecture may require a

> re-compilation of structure layout."

>      | newPlatform |

>      newPlatform := Smalltalk platformName.

>      PreviousPlatform = newPlatform

>          ifFalse:

>              [self recompileStructures.

>             PreviousPlatform := newPlatform]

>

> It works if you switch from Win64 to MacOS64 for example...

> However, this mechanism is not used yet at time of code loading, so

> importing code generated on a different platform won't automatically

> update

> the structures (or alias) - the FFI package has no mechanism for that

> (maybe we could add a hook in MC?).

>

> Le mer. 11 mars 2020 à 23:58, Bert Freudenberg &lt;



> bert@



> &gt; a

> écrit :

>

>> I'd suggest to get OpenGL working outside of Croquet first:

>>

>> http://www.squeaksource.com/CroquetGL.html

>>

>> Step 1: Verify this works in 32 bits. (assuming you are doing this on

>> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)

>> Step 2: Make it work in 64 bits.

>>

>> The second step requires that you understand how FFI works, and how it

>> handles e.g. pointers and integer sizes.

>> I am assuming we do have a working 64 bit FFI, at least for x86_64

>> machines.

>>

>> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a 32

>> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit

>> system,

>> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I

>> don't know how many bits 'ulong' has in our 64 bit FFI, but that

>> declaration may have to change. Etc. pp.

>>

>> If you have questions about FFI then those are best directed at the

>> vm-dev

>> list since it is dealing with VM-level interfaces. CC'ing, please follow

>> up

>> there.

>>

>> - Bert -

>>

>> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <

>>



> squeak-dev@.squeakfoundation



>> wrote:

>>

>>> Okey dokey,

>>>

>>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName after

>>>

>>> openGLLibraryName

>>> ^Smalltalk osVersion = 'linux'

>>> ifTrue: ['libGL.so.1']

>>> ifFalse: ['GL']

>>>

>>> I removed it in my install and got past that error.

>>>

>>> Working exclusively with Croquet(Master)...

>>>

>>>

>>>

>>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:

>>>

>>> glMatrixMode: mode

>>> "This method was automatically generated."

>>> "void glMatrixMode(GLenum mode);"

>>>

> <apicall: void 'glMatrixMode' (ulong) module: '#openGLLibraryName'>

>>> ^self externalCallFailed

>>>

>>> The

> <apicall:...>

>  fails

>>>

>>> How to think about this?

>>>

>>> Is Croquet behind OpenGL latest?

>>> Would teaching myself OpenGL programming be of use to the Croquet

>>> project?

>>>

>>> cheers,

>>>

>>> tty

>>>

>>>

>>>

>>











--

Sent from: http://forum.world.st/Squeak-VM-f104410.html

--

_,,,^..^,,,_

best, Eliot

Am 09.06.2020 18:33:36 schrieb Eliot Miranda <[hidden email]>:

On Jun 9, 2020, at 5:10 AM, Marcel Taeumel <[hidden email]> wrote:



Hi Nicolas, hi Eliot.

> If you want a type that can hold a pointer on both 32 and 64bits arch, then it's doable with a FFI type alias

You can just make an alias to a void*. It will have the right size. The only drawback would be that you might get an issue with argument coercion.

#(nil 'void*')

Your "type alias struct" will wrap around ExternalData<void*>. Why bother with ulong/ulonglong. Only if you want to work with integers in the image, you might want to use ulong or ulonglong depending on the #wordSize.

I think I will add 'word' as a type name in ExternalType's class-side "type constants" in Squeak FFI. Then you can use it in FFI call defs and struct-field defs:

ExternalType class >> #word

"Use this atomic-type alias if the external library works with opaque handles that are different on 32bit and 64bit. On a platform change, all FFI calls will be re-compiled and this method is called again by the parser."

^ FFIPlatformDescription current wordSize = 4

ifTrue: [self unsignedLong]

ifFalse: [self unsignedLongLong]

... maybe parts in the FFI plugin are faster with ulong/ulonglong compared to void* compared to type alias to void*... I don't know.

It might be better to use a C name such as intptr_t or uintptr_t.

Best,

Marcel

Am 30.05.2020 18:00:33 schrieb Marcel Taeumel <[hidden email]>:

So, if the atomic types and structure types are initialized, they make use of ExternalType class >> #pointerSpec to store the size of the pointer in the #headerWord, which is the first element of an external type's 'compiledSpec'. For atomic types, this happens when the FFI package is loaded.

For structure types, this happens when a an external structure is mentioned for the first time in an FFI call method or in during compilation of another external structure's fields. See ExternalType class >> #newTypeNamed:force:. Squeak's Smalltalk parser or the compile logic in ExternalStructure invokes that type creation respectively..

Currently, if a platform change at system startup is detected, all atomic types and structure types are updated --- which implies a call to #pointerSpec in ExternalType class >> #initializeStructureTypes.

Now, what could be the situation where a struct-pointer type had the wrong pointer size? During development, your platform does not magically change the #wordSize. When loading code, methods in the external struct might be wrong (bc. Monticello does not trigger that field re-definition) BUT any external struct-pointer type that is already in the image must have the right size. And for new external struct-pointer types, it should work as explained above.

So, I think we should make the following changes:

- Remove ExternalType >> #asPointerType: (! note the colon !)

- Remove ExternalType >> #pointerSize:

- Change ExternalType >> #pointerSize to just read the compiledSpec

- Update all sends to #asPointerType: to #asPointerType and assume that the current pointer-type instances have the right size, as described above.

Best,

Marcel

Am 30.05.2020 15:59:31 schrieb Marcel Taeumel <[hidden email]>:

> But if it is obsolete now, nuke it.

I cannot tell. That information is stored in the external type's compiledSpec to be used from within the FFI plugin I suppose.

Best,

Marcel

Am 29.05.2020 00:15:25 schrieb Eliot Miranda <[hidden email]>:

On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <>
[hidden email]> wrote:



>

> Hi Marcel,

> no idea...

> Look at timestamp:

> eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set

> FFI-Kernel-EstebanLorenzano.45 ·

> So presumably Eliot added that.

>



Perhaps. I don't remember. But if it is obsolete now, nuke it.





>

> Le jeu. 28 mai 2020 à 17:00, marcel.taeumel a

> écrit :

>

>>

>> Hi Nicolas,

>>

>> what is the role of ExternalStructure class >> #pointerSize then?

>>

>> Best,

>> Marcel

>>

>>

>> Nicolas Cellier wrote

>> > Hi all,

>> > beware of FFI 'long' and 'ulong'.

>> > They are currently meaning (32 bits) int and unsigned int!

>> > This dates back from the 80s when there were still odd OS with segmented

>> > memory and 16-bits int.

>> > long was thought as a safe int32_t at that time... and still means just

>> > that in current FFI.

>> >

>> > I'd like to replace them with 'int' and 'uint', but we need to ensure

>> > backward compatibility etc...

>> >

>> > FFI has ExternalType signedLongLong or just 'longlong' and ExternalType

>> > unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.

>> > See implementors of #initializeAtomicTypes #initializeFFIConstants

>> >

>> > If you want a type that can hold a pointer on both 32 and 64bits arch,

>> > then

>> > it's doable with a FFI type alias

>> > For example I have things like this in HDF5 module

>> > Notice the single #(fieldName fieldType) which defines an alias, while

>> > ordinary FFI struct gets an array of #(fieldName fieldType)

>> > (and replace Size_t with Intptr_t if the purpose is hodling a pointer,

>> > otherwise the ayatollahs of C standard will lash you)

>> >

>> > ExternalStructure subclass: #'Size_t'

>> > instanceVariableNames: ''

>> > classVariableNames: ''

>> > poolDictionaries: ''

>> > category: 'HDF5-External-atomic'

>> >

>> > Size_t class >>fields

>> > "Size_t defineFields"

>> > ^Smalltalk wordSize = 4

>> > ifTrue: [#( value 'ulong')]

>> > ifFalse: [#( value 'ulonglong')]

>> >

>> > However the "fields" must be re-generated when platform changes...

>> > I tried to automate that sometimes ago, see:

>> >

>> > ExternalStructure class>>install

>> > "Resuming the image on another architecture may require a

>> > re-compilation of structure layout."

>> > | newPlatform |

>> > newPlatform := Smalltalk platformName.

>> > PreviousPlatform = newPlatform

>> > ifFalse:

>> > [self recompileStructures.

>> > PreviousPlatform := newPlatform]

>> >

>> > It works if you switch from Win64 to MacOS64 for example...

>> > However, this mechanism is not used yet at time of code loading, so

>> > importing code generated on a different platform won't automatically

>> > update

>> > the structures (or alias) - the FFI package has no mechanism for that

>> > (maybe we could add a hook in MC?).

>> >

>> > Le mer. 11 mars 2020 à 23:58, Bert Freudenberg <

>>

>> > bert@

>>

>> > > a

>> > écrit :

>> >

>> >> I'd suggest to get OpenGL working outside of Croquet first:

>> >>

>> >> http://www.squeaksource.com/CroquetGL.html

>> >>

>> >> Step 1: Verify this works in 32 bits. (assuming you are doing this on

>> >> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)

>> >> Step 2: Make it work in 64 bits.

>> >>

>> >> The second step requires that you understand how FFI works, and how it

>> >> handles e.g. pointers and integer sizes.

>> >> I am assuming we do have a working 64 bit FFI, at least for x86_64

>> >> machines.

>> >>

>> >> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a

>> 32

>> >> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit

>> >> system,

>> >> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I

>> >> don't know how many bits 'ulong' has in our 64 bit FFI, but that

>> >> declaration may have to change. Etc. pp.

>> >>

>> >> If you have questions about FFI then those are best directed at the

>> >> vm-dev

>> >> list since it is dealing with VM-level interfaces. CC'ing, please

>> follow

>> >> up

>> >> there.

>> >>

>> >> - Bert -

>> >>

>> >> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <>
>> >>

>>

>> > squeak-dev@.squeakfoundation

>>

>> >> wrote:

>> >>

>> >>> Okey dokey,

>> >>>

>> >>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName

>> after

>> >>>

>> >>> openGLLibraryName

>> >>> ^Smalltalk osVersion = 'linux'

>> >>> ifTrue: ['libGL.so.1']

>> >>> ifFalse: ['GL']

>> >>>

>> >>> I removed it in my install and got past that error.

>> >>>

>> >>> Working exclusively with Croquet(Master)...

>> >>>

>> >>>

>> >>>

>> >>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:

>> >>>

>> >>> glMatrixMode: mode

>> >>> "This method was automatically generated."

>> >>> "void glMatrixMode(GLenum mode);"

>> >>>

>> >

>> >>> ^self externalCallFailed

>> >>>

>> >>> The

>> >

>> > fails

>> >>>

>> >>> How to think about this?

>> >>>

>> >>> Is Croquet behind OpenGL latest?

>> >>> Would teaching myself OpenGL programming be of use to the Croquet

>> >>> project?

>> >>>

>> >>> cheers,

>> >>>

>> >>> tty

>> >>>

>> >>>

>> >>>

>> >>

>>

>>

>>

>>

>>

>> --

>> Sent from: http://forum.world.st/Squeak-VM-f104410.html

>>

>



--

_,,,^..^,,,_

best, Eliot

On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <[hidden email]> wrote:

 

Hi Marcel,

no idea...

Look at timestamp:

eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set FFI-Kernel-EstebanLorenzano.45  ·

So presumably Eliot added that.

Perhaps.  I don't remember.  But if it is obsolete now, nuke it.

 

Le jeu. 28 mai 2020 à 17:00, marcel.taeumel <[hidden email]> a écrit :

 


Hi Nicolas,





what is the role of ExternalStructure class >> #pointerSize then?





Best,


Marcel








Nicolas Cellier wrote


> Hi all,


> beware of FFI 'long' and 'ulong'.


> They are currently meaning (32 bits) int and unsigned int!


> This dates back from the 80s when there were still odd OS with segmented


> memory and 16-bits int.


> long was thought as a safe int32_t at that time... and still means just


> that in current FFI.


>


> I'd like to replace them with 'int' and 'uint', but we need to ensure


> backward compatibility etc...


>


> FFI has ExternalType signedLongLong or just 'longlong' and ExternalType


> unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.


> See implementors of #initializeAtomicTypes #initializeFFIConstants


>


> If you want a type that can hold a pointer on both 32 and 64bits arch,


> then


> it's doable with a FFI type alias


> For example I have things like this in HDF5 module


> Notice the single #(fieldName fieldType) which defines an alias, while


> ordinary FFI struct gets an array of #(fieldName fieldType)


> (and replace Size_t with Intptr_t if the purpose is hodling a pointer,


> otherwise the ayatollahs of C standard will lash you)


>


> ExternalStructure subclass: #'Size_t'


>     instanceVariableNames: ''


>     classVariableNames: ''


>     poolDictionaries: ''


>     category: 'HDF5-External-atomic'


>


> Size_t class >>fields


>     "Size_t defineFields"


>     ^Smalltalk wordSize = 4


>         ifTrue: [#( value 'ulong')]


>         ifFalse: [#( value 'ulonglong')]


>


> However the "fields" must be re-generated when platform changes...


> I tried to automate that sometimes ago, see:


>


> ExternalStructure class>>install


>      "Resuming the image on another architecture may require a


> re-compilation of structure layout."


>      | newPlatform |


>      newPlatform := Smalltalk platformName.


>      PreviousPlatform = newPlatform


>          ifFalse:


>              [self recompileStructures.


>             PreviousPlatform := newPlatform]


>


> It works if you switch from Win64 to MacOS64 for example...


> However, this mechanism is not used yet at time of code loading, so


> importing code generated on a different platform won't automatically


> update


> the structures (or alias) - the FFI package has no mechanism for that


> (maybe we could add a hook in MC?).


>


> Le mer. 11 mars 2020 à 23:58, Bert Freudenberg &lt;





> bert@





> &gt; a


> écrit :


>


>> I'd suggest to get OpenGL working outside of Croquet first:


>>


>> http://www.squeaksource.com/CroquetGL.html


>>


>> Step 1: Verify this works in 32 bits. (assuming you are doing this on


>> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)


>> Step 2: Make it work in 64 bits.


>>


>> The second step requires that you understand how FFI works, and how it


>> handles e.g. pointers and integer sizes.


>> I am assuming we do have a working 64 bit FFI, at least for x86_64


>> machines.


>>


>> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a 32


>> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit


>> system,


>> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I


>> don't know how many bits 'ulong' has in our 64 bit FFI, but that


>> declaration may have to change. Etc. pp.


>>


>> If you have questions about FFI then those are best directed at the


>> vm-dev


>> list since it is dealing with VM-level interfaces. CC'ing, please follow


>> up


>> there.


>>


>> - Bert -


>>


>> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <


>>





> squeak-dev@.squeakfoundation





>> wrote:


>>


>>> Okey dokey,


>>>


>>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName after


>>>


>>> openGLLibraryName


>>> ^Smalltalk osVersion = 'linux'


>>> ifTrue: ['libGL.so.1']


>>> ifFalse: ['GL']


>>>


>>> I removed it in my install and got past that error.


>>>


>>> Working exclusively with Croquet(Master)...


>>>


>>>


>>>


>>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:


>>>


>>> glMatrixMode: mode


>>> "This method was automatically generated."


>>> "void glMatrixMode(GLenum mode);"


>>>


> <apicall: void 'glMatrixMode' (ulong) module: '#openGLLibraryName'>


>>> ^self externalCallFailed


>>>


>>> The


> <apicall:...>


>  fails


>>>


>>> How to think about this?


>>>


>>> Is Croquet behind OpenGL latest?


>>> Would teaching myself OpenGL programming be of use to the Croquet


>>> project?


>>>


>>> cheers,


>>>


>>> tty


>>>


>>>


>>>


>>

















--


Sent from: http://forum.world.st/Squeak-VM-f104410.html

--

_,,,^..^,,,_

best, Eliot