FFI | Byte alignment

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

FFI | Byte alignment

marcel.taeumel
Hi, there.

In ExternalType class >> #initializeAtomicTypes, there is an interesting claim and a piece of dead code:

"On 32 bits Windows and MacOS, double and long have an alignment of 8. But on Linux, their alignment is 4"
(Smalltalk wordSize = 4 and: [Smalltalk platformName = 'unix']) ifTrue: [
(#('double longlong ulonglong') includes: typeName) ifTrue: [
byteAlignment := 4
]
].

As you can see, there are single quotes missing and so will the path "byteAlignment  := 4" never be reached.

I tried to figure out whether one should either fix the conditional or remove the entire passage. Maybe this got long fixed inside the FFI plugin?

Best,
Marcel


Reply | Threaded
Open this post in threaded view
|

Re: FFI | Byte alignment

Nicolas Cellier
Good catch!
Wikipedia page does not even agree on longlong... But it's not the ultimate normative reference, better check by ourselves!

Le ven. 29 mai 2020 à 15:53, Marcel Taeumel <[hidden email]> a écrit :
Hi, there.

In ExternalType class >> #initializeAtomicTypes, there is an interesting claim and a piece of dead code:

"On 32 bits Windows and MacOS, double and long have an alignment of 8. But on Linux, their alignment is 4"
(Smalltalk wordSize = 4 and: [Smalltalk platformName = 'unix']) ifTrue: [
(#('double longlong ulonglong') includes: typeName) ifTrue: [
byteAlignment := 4
]
].

As you can see, there are single quotes missing and so will the path "byteAlignment  := 4" never be reached.

I tried to figure out whether one should either fix the conditional or remove the entire passage. Maybe this got long fixed inside the FFI plugin?

Best,
Marcel



Reply | Threaded
Open this post in threaded view
|

Re: FFI | Byte alignment

Nicolas Cellier
Try with this:

cat > test_align.c <<END
#include <stdio.h>

struct foo1 {int a; long long b; int d;};
struct foo2 {int a; double c; int d;};

int main() {
struct foo1 x;
struct foo2 y;
printf("size of x = %d\n",sizeof(x));
printf("size of y = %d\n",sizeof(y));
return 0;
}
END
i686-w64-mingw32-gcc test_align.c
 ./a.exe

size of x = 24
size of y = 24

so at leat in windows (mingw) the alignment is 8 bytes for both long long and double.
I have no 32bits linux handy, but it should be easy enough to pass the same test...

Le ven. 29 mai 2020 à 16:45, Nicolas Cellier <[hidden email]> a écrit :
Good catch!
Wikipedia page does not even agree on longlong... But it's not the ultimate normative reference, better check by ourselves!

Le ven. 29 mai 2020 à 15:53, Marcel Taeumel <[hidden email]> a écrit :
Hi, there.

In ExternalType class >> #initializeAtomicTypes, there is an interesting claim and a piece of dead code:

"On 32 bits Windows and MacOS, double and long have an alignment of 8. But on Linux, their alignment is 4"
(Smalltalk wordSize = 4 and: [Smalltalk platformName = 'unix']) ifTrue: [
(#('double longlong ulonglong') includes: typeName) ifTrue: [
byteAlignment := 4
]
].

As you can see, there are single quotes missing and so will the path "byteAlignment  := 4" never be reached.

I tried to figure out whether one should either fix the conditional or remove the entire passage. Maybe this got long fixed inside the FFI plugin?

Best,
Marcel



Reply | Threaded
Open this post in threaded view
|

Re: FFI | Byte alignment

Nicolas Cellier
Found a 32bits debian VM:

st@debi9:~$ gcc test_align.c
st@debi9:~$ ./a.out
size of x = 16
size of y = 16

So keep the code, just correct it!

Le ven. 29 mai 2020 à 17:01, Nicolas Cellier <[hidden email]> a écrit :
Try with this:

cat > test_align.c <<END
#include <stdio.h>

struct foo1 {int a; long long b; int d;};
struct foo2 {int a; double c; int d;};

int main() {
struct foo1 x;
struct foo2 y;
printf("size of x = %d\n",sizeof(x));
printf("size of y = %d\n",sizeof(y));
return 0;
}
END
i686-w64-mingw32-gcc test_align.c
 ./a.exe

size of x = 24
size of y = 24

so at leat in windows (mingw) the alignment is 8 bytes for both long long and double.
I have no 32bits linux handy, but it should be easy enough to pass the same test...

Le ven. 29 mai 2020 à 16:45, Nicolas Cellier <[hidden email]> a écrit :
Good catch!
Wikipedia page does not even agree on longlong... But it's not the ultimate normative reference, better check by ourselves!

Le ven. 29 mai 2020 à 15:53, Marcel Taeumel <[hidden email]> a écrit :
Hi, there.

In ExternalType class >> #initializeAtomicTypes, there is an interesting claim and a piece of dead code:

"On 32 bits Windows and MacOS, double and long have an alignment of 8. But on Linux, their alignment is 4"
(Smalltalk wordSize = 4 and: [Smalltalk platformName = 'unix']) ifTrue: [
(#('double longlong ulonglong') includes: typeName) ifTrue: [
byteAlignment := 4
]
].

As you can see, there are single quotes missing and so will the path "byteAlignment  := 4" never be reached.

I tried to figure out whether one should either fix the conditional or remove the entire passage. Maybe this got long fixed inside the FFI plugin?

Best,
Marcel



Reply | Threaded
Open this post in threaded view
|

Re: FFI | Byte alignment

Nicolas Cellier
And for your other question concerning the plugin:

1. it's been fixed only recently
2. it's not completely fixed with this regard!

See ThreadedFFIPlugin>>checkAlignmentOfStructSpec:OfLength:StartingAt:
We are always using the filedSize in case of atomic type as fieldAlignment...
So it ain't gonna work for double/longlong/ulonglong on 32 bits linux
(unless library is compiled with -malign-double or other exotic flags, but how could FFI guess that...)
Ideally, the alignment would be occupying some bits in the type spec.
Or we hardcode the well known exception in a second place.
Anyway, the plugin also had to duplicate the alignment algorithm as you can see...

We live in a world full of such accidental complexity...
At least, while mankind is taking care of such detail, this leaves less time for killing each other, so let's declare that it's a good thing ;)

Le ven. 29 mai 2020 à 17:37, Nicolas Cellier <[hidden email]> a écrit :
Found a 32bits debian VM:

st@debi9:~$ gcc test_align.c
st@debi9:~$ ./a.out
size of x = 16
size of y = 16

So keep the code, just correct it!

Le ven. 29 mai 2020 à 17:01, Nicolas Cellier <[hidden email]> a écrit :
Try with this:

cat > test_align.c <<END
#include <stdio.h>

struct foo1 {int a; long long b; int d;};
struct foo2 {int a; double c; int d;};

int main() {
struct foo1 x;
struct foo2 y;
printf("size of x = %d\n",sizeof(x));
printf("size of y = %d\n",sizeof(y));
return 0;
}
END
i686-w64-mingw32-gcc test_align.c
 ./a.exe

size of x = 24
size of y = 24

so at leat in windows (mingw) the alignment is 8 bytes for both long long and double.
I have no 32bits linux handy, but it should be easy enough to pass the same test...

Le ven. 29 mai 2020 à 16:45, Nicolas Cellier <[hidden email]> a écrit :
Good catch!
Wikipedia page does not even agree on longlong... But it's not the ultimate normative reference, better check by ourselves!

Le ven. 29 mai 2020 à 15:53, Marcel Taeumel <[hidden email]> a écrit :
Hi, there.

In ExternalType class >> #initializeAtomicTypes, there is an interesting claim and a piece of dead code:

"On 32 bits Windows and MacOS, double and long have an alignment of 8. But on Linux, their alignment is 4"
(Smalltalk wordSize = 4 and: [Smalltalk platformName = 'unix']) ifTrue: [
(#('double longlong ulonglong') includes: typeName) ifTrue: [
byteAlignment := 4
]
].

As you can see, there are single quotes missing and so will the path "byteAlignment  := 4" never be reached.

I tried to figure out whether one should either fix the conditional or remove the entire passage. Maybe this got long fixed inside the FFI plugin?

Best,
Marcel



Reply | Threaded
Open this post in threaded view
|

Re: FFI complexity (was FFI | Byte alignment)

Nicolas Cellier


Le ven. 29 mai 2020 à 18:00, Nicolas Cellier <[hidden email]> a écrit :
And for your other question concerning the plugin:

1. it's been fixed only recently
2. it's not completely fixed with this regard!

See ThreadedFFIPlugin>>checkAlignmentOfStructSpec:OfLength:StartingAt:
We are always using the filedSize in case of atomic type as fieldAlignment...
So it ain't gonna work for double/longlong/ulonglong on 32 bits linux
(unless library is compiled with -malign-double or other exotic flags, but how could FFI guess that...)
Ideally, the alignment would be occupying some bits in the type spec.
Or we hardcode the well known exception in a second place.
Anyway, the plugin also had to duplicate the alignment algorithm as you can see...

We live in a world full of such accidental complexity...
At least, while mankind is taking care of such detail, this leaves less time for killing each other, so let's declare that it's a good thing ;)


And that's the negative side of FFI: import the useless complexity inside the image instead of burrying it under the VM carpet.
Exactly the same feeling as when I introduced the lineEnding zoo inside the image instead of the simple CR.
It helps to play well in a complex world, but it's also importing absolutely pointless complexity inside the image.
It's funny that we still pay such high a tribute to CR-LF for the decision of a single person writing QDOS.
No matter that it was solved for long time by printcap/termcap unix databases...
We are all adopting quick and dirty solutions from time to time, then forget to think twice before distributing!

I still prefer having an opened platform to being lock in own autistic world, but sometimes i wonder until which limit it is really worth...


Le ven. 29 mai 2020 à 17:37, Nicolas Cellier <[hidden email]> a écrit :
Found a 32bits debian VM:

st@debi9:~$ gcc test_align.c
st@debi9:~$ ./a.out
size of x = 16
size of y = 16

So keep the code, just correct it!

Le ven. 29 mai 2020 à 17:01, Nicolas Cellier <[hidden email]> a écrit :
Try with this:

cat > test_align.c <<END
#include <stdio.h>

struct foo1 {int a; long long b; int d;};
struct foo2 {int a; double c; int d;};

int main() {
struct foo1 x;
struct foo2 y;
printf("size of x = %d\n",sizeof(x));
printf("size of y = %d\n",sizeof(y));
return 0;
}
END
i686-w64-mingw32-gcc test_align.c
 ./a.exe

size of x = 24
size of y = 24

so at leat in windows (mingw) the alignment is 8 bytes for both long long and double.
I have no 32bits linux handy, but it should be easy enough to pass the same test...

Le ven. 29 mai 2020 à 16:45, Nicolas Cellier <[hidden email]> a écrit :
Good catch!
Wikipedia page does not even agree on longlong... But it's not the ultimate normative reference, better check by ourselves!

Le ven. 29 mai 2020 à 15:53, Marcel Taeumel <[hidden email]> a écrit :
Hi, there.

In ExternalType class >> #initializeAtomicTypes, there is an interesting claim and a piece of dead code:

"On 32 bits Windows and MacOS, double and long have an alignment of 8. But on Linux, their alignment is 4"
(Smalltalk wordSize = 4 and: [Smalltalk platformName = 'unix']) ifTrue: [
(#('double longlong ulonglong') includes: typeName) ifTrue: [
byteAlignment := 4
]
].

As you can see, there are single quotes missing and so will the path "byteAlignment  := 4" never be reached.

I tried to figure out whether one should either fix the conditional or remove the entire passage. Maybe this got long fixed inside the FFI plugin?

Best,
Marcel



Reply | Threaded
Open this post in threaded view
|

Re: FFI | Byte alignment

timrowledge
In reply to this post by Nicolas Cellier


> On 2020-05-29, at 8:37 AM, Nicolas Cellier <[hidden email]> wrote:
>
> Found a 32bits debian VM:
>
> st@debi9:~$ gcc test_align.c
> st@debi9:~$ ./a.out
> size of x = 16
> size of y = 16
>
> So keep the code, just correct it!

On a 32 bit Raspberry Pi OS machine (note the OS name has officially changed as of this week!)
size of x = 24
size of y = 24


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Fractured Idiom:- J'Y SUIS, J'Y PESTES - I can stay for the weekend



Reply | Threaded
Open this post in threaded view
|

Re: FFI | Byte alignment

marcel.taeumel
Ha! So #isUnix but not #isARM? I can work with that. :-)

Best,
Marcel

Am 29.05.2020 19:16:22 schrieb tim Rowledge <[hidden email]>:



> On 2020-05-29, at 8:37 AM, Nicolas Cellier wrote:
>
> Found a 32bits debian VM:
>
> st@debi9:~$ gcc test_align.c
> st@debi9:~$ ./a.out
> size of x = 16
> size of y = 16
>
> So keep the code, just correct it!

On a 32 bit Raspberry Pi OS machine (note the OS name has officially changed as of this week!)
size of x = 24
size of y = 24


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Fractured Idiom:- J'Y SUIS, J'Y PESTES - I can stay for the weekend





Reply | Threaded
Open this post in threaded view
|

Re: FFI | Byte alignment

timrowledge
Could always do this dynamically; write out the file, compile and run, read results, set values.

Obviously there might be a small problem with needing to have ffi working to call the OS routines to write the file etc in order to derive the type parameter sizes... :-) A bit more seriously one could do this as part of some install/run scripting that we have already touched on doing to derive the right VM for the chosen image file and OS etc.

> On 2020-05-29, at 10:18 AM, Marcel Taeumel <[hidden email]> wrote:
>
> Ha! So #isUnix but not #isARM? I can work with that. :-)
>
> Best,
> Marcel
>> Am 29.05.2020 19:16:22 schrieb tim Rowledge <[hidden email]>:
>>
>>
>>
>> > On 2020-05-29, at 8:37 AM, Nicolas Cellier wrote:
>> >
>> > Found a 32bits debian VM:
>> >
>> > st@debi9:~$ gcc test_align.c
>> > st@debi9:~$ ./a.out
>> > size of x = 16
>> > size of y = 16
>> >
>> > So keep the code, just correct it!
>>
>> On a 32 bit Raspberry Pi OS machine (note the OS name has officially changed as of this week!)
>> size of x = 24
>> size of y = 24
>>
>>
>> tim
>> --
>> tim Rowledge; [hidden email]; http://www.rowledge.org/tim
>> Fractured Idiom:- J'Y SUIS, J'Y PESTES - I can stay for the weekend
>>
>>
>>
>


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: DTF: Dump Tape to Floor



Reply | Threaded
Open this post in threaded view
|

Re: FFI | Byte alignment

Jakob Reschke
tim Rowledge <[hidden email]> schrieb am Fr., 29. Mai 2020, 20:58:
Could always do this dynamically; write out the file, compile and run, read results, set values.

Obviously there might be a small problem with needing to have ffi working to call the OS routines to write the file etc in order to derive the type parameter sizes... :-) A bit more seriously one could do this as part of some install/run scripting that we have already touched on doing to derive the right VM for the chosen image file and OS etc.

You do not want to require a compiler being set up generally accessible on a Windows machine because, out of the box, the requirement is never met. ;-)

Don't want to install a C compiler to run Squeak on a fresh box.


> On 2020-05-29, at 10:18 AM, Marcel Taeumel <[hidden email]> wrote:
>
> Ha! So #isUnix but not #isARM? I can work with that. :-)
>
> Best,
> Marcel

Does this depend on the OS at all? I thought it would depend on the processor architecture alone.


Reply | Threaded
Open this post in threaded view
|

Re: FFI | Byte alignment

timrowledge


> On 2020-05-29, at 12:05 PM, Jakob Reschke <[hidden email]> wrote:
>
> tim Rowledge <[hidden email]> schrieb am Fr., 29. Mai 2020, 20:58:
> Could always do this dynamically; write out the file, compile and run, read results, set values.
>
> Obviously there might be a small problem with needing to have ffi working to call the OS routines to write the file etc in order to derive the type parameter sizes... :-) A bit more seriously one could do this as part of some install/run scripting that we have already touched on doing to derive the right VM for the chosen image file and OS etc.
>
> You do not want to require a compiler being set up generally accessible on a Windows machine because, out of the box, the requirement is never met. ;-)
>
> Don't want to install a C compiler to run Squeak on a fresh box.

But it's the Year of Linux on the Desktop! Nobody need ever use Windows again!

Also moderately seriously, I suspect that the values aren't so variable for Windows/Mac and thus running a suitable program once (maybe for 32 and 64 bit variants?) would be enough. 'nix runs across a lot of hardware and thus a test-on-site might be suitable.

tim

Give a man a compliment, he’ll feel good for a day.
Teach a man to fish for compliments and he’ll never feel good enough for the rest of his life.


Reply | Threaded
Open this post in threaded view
|

Re: FFI | Byte alignment

marcel.taeumel
Also moderately seriously, I suspect that the values aren't so variable for Windows/Mac and thus running a suitable program once (maybe for 32 and 64 bit variants?) would be enough. 'nix runs across a lot of hardware and thus a test-on-site might be suitable.

That's what the machinery behind Monty's FFIExternalSharedPool is for. I am currently working on it and will present the next iteration for it in a few days. :-) 

Best,
Marcel

Am 29.05.2020 21:28:27 schrieb tim Rowledge <[hidden email]>:



> On 2020-05-29, at 12:05 PM, Jakob Reschke wrote:
>
> tim Rowledge schrieb am Fr., 29. Mai 2020, 20:58:
> Could always do this dynamically; write out the file, compile and run, read results, set values.
>
> Obviously there might be a small problem with needing to have ffi working to call the OS routines to write the file etc in order to derive the type parameter sizes... :-) A bit more seriously one could do this as part of some install/run scripting that we have already touched on doing to derive the right VM for the chosen image file and OS etc.
>
> You do not want to require a compiler being set up generally accessible on a Windows machine because, out of the box, the requirement is never met. ;-)
>
> Don't want to install a C compiler to run Squeak on a fresh box.

But it's the Year of Linux on the Desktop! Nobody need ever use Windows again!

Also moderately seriously, I suspect that the values aren't so variable for Windows/Mac and thus running a suitable program once (maybe for 32 and 64 bit variants?) would be enough. 'nix runs across a lot of hardware and thus a test-on-site might be suitable.

tim

Give a man a compliment, he’ll feel good for a day.
Teach a man to fish for compliments and he’ll never feel good enough for the rest of his life.




Reply | Threaded
Open this post in threaded view
|

Re: FFI complexity (was FFI | Byte alignment)

Stéphane Rollandin
In reply to this post by Nicolas Cellier
> I still prefer having an opened platform to being lock in own autistic
> world, but sometimes i wonder until which limit it is really worth...

This remark has a wide and deep reach and applies pretty much to any
life situation, individual or collective. In short, where is the limit
between 'self' and 'the rest of the world'?

Since there is no definite answer to this question then maybe, following
the traditional Smalltalk way, should we reify it and construct the
proper objects and protocols able to deal with its different specific
aspects- have classes reflecting the 'external world' so to say.

In other words, the implementation details of the external systems do
not have to enter the image, but there could be a (rather wide) layer of
abstraction within the image that tells the rest of the system what is
outside and how we are interfaced to it, with switches to allow, for a
given situation, either to retract in our autistic world (where backward
compatibility and resilience are ensured) or open up to a specific API,
(where the risk of obsolescence and deprecation exist, but are then made
explicit and dealable with).

Just hand-waving here...

Stef

Reply | Threaded
Open this post in threaded view
|

Re: FFI complexity (was FFI | Byte alignment)

Karl Ramberg
A little off topic but nice reflections on Smalltalk vs. outside world:

Best,
Karl

On Sat, May 30, 2020 at 10:30 AM Stéphane Rollandin <[hidden email]> wrote:
> I still prefer having an opened platform to being lock in own autistic
> world, but sometimes i wonder until which limit it is really worth...

This remark has a wide and deep reach and applies pretty much to any
life situation, individual or collective. In short, where is the limit
between 'self' and 'the rest of the world'?

Since there is no definite answer to this question then maybe, following
the traditional Smalltalk way, should we reify it and construct the
proper objects and protocols able to deal with its different specific
aspects- have classes reflecting the 'external world' so to say.

In other words, the implementation details of the external systems do
not have to enter the image, but there could be a (rather wide) layer of
abstraction within the image that tells the rest of the system what is
outside and how we are interfaced to it, with switches to allow, for a
given situation, either to retract in our autistic world (where backward
compatibility and resilience are ensured) or open up to a specific API,
(where the risk of obsolescence and deprecation exist, but are then made
explicit and dealable with).

Just hand-waving here...

Stef



Reply | Threaded
Open this post in threaded view
|

Re: FFI complexity (was FFI | Byte alignment)

marcel.taeumel
In reply to this post by Stéphane Rollandin
Hi Stéphane.

I enjoyed reading your thoughts in this matter. ^__^ Let's see how we can provide and maintain those layers and switches. Maybe they don't have to be "rather wide" at all.

Best,
Marcel

Am 30.05.2020 10:30:16 schrieb Stéphane Rollandin <[hidden email]>:

> I still prefer having an opened platform to being lock in own autistic
> world, but sometimes i wonder until which limit it is really worth...

This remark has a wide and deep reach and applies pretty much to any
life situation, individual or collective. In short, where is the limit
between 'self' and 'the rest of the world'?

Since there is no definite answer to this question then maybe, following
the traditional Smalltalk way, should we reify it and construct the
proper objects and protocols able to deal with its different specific
aspects- have classes reflecting the 'external world' so to say.

In other words, the implementation details of the external systems do
not have to enter the image, but there could be a (rather wide) layer of
abstraction within the image that tells the rest of the system what is
outside and how we are interfaced to it, with switches to allow, for a
given situation, either to retract in our autistic world (where backward
compatibility and resilience are ensured) or open up to a specific API,
(where the risk of obsolescence and deprecation exist, but are then made
explicit and dealable with).

Just hand-waving here...

Stef



Reply | Threaded
Open this post in threaded view
|

Re: FFI | Byte alignment

Eliot Miranda-2
In reply to this post by Nicolas Cellier
Hi Nicolas, Hi Marcel,


On May 29, 2020, at 8:02 AM, Nicolas Cellier <[hidden email]> wrote:


Try with this:

cat > test_align.c <<END
#include <stdio.h>

struct foo1 {int a; long long b; int d;};
struct foo2 {int a; double c; int d;};

Here’s a suggestion.  We add a primitive To the ThreadedFFIPlugin that answers structure alignment thusly:

- the primitive takes as it’s argument the numeric code for the integer or float type or a structure as used in the compiled type specs 
- the primitive answers the alignment of the data type in a struct that looks like this:

struct alignment_of_longlong {
     char pad_to_force_alignment;
     long long element;
}

return &((struct alignment_of_longlong *)0)-> element);

struct alignment_of_double {
     char pad_to_force_alignment;
     long long element;
}

return &((struct alignment_of_double *)0)-> element);


struct alignment_of_struct {
     char pad_to_force_alignment;
     struct { char element; } element;
}

return &((struct alignment_of_struct *)0)-> element);

etc

Then we can remember these alignments in class vars of, say, ExternalStructure, check them on start-up, and resize/update ExternalStructure if any of the values change.

int main() {
struct foo1 x;
struct foo2 y;
printf("size of x = %d\n",sizeof(x));
printf("size of y = %d\n",sizeof(y));
return 0;
}
END
i686-w64-mingw32-gcc test_align.c
 ./a.exe

size of x = 24
size of y = 24

so at leat in windows (mingw) the alignment is 8 bytes for both long long and double.
I have no 32bits linux handy, but it should be easy enough to pass the same test...

Le ven. 29 mai 2020 à 16:45, Nicolas Cellier <[hidden email]> a écrit :
Good catch!
Wikipedia page does not even agree on longlong... But it's not the ultimate normative reference, better check by ourselves!

Le ven. 29 mai 2020 à 15:53, Marcel Taeumel <[hidden email]> a écrit :
Hi, there.

In ExternalType class >> #initializeAtomicTypes, there is an interesting claim and a piece of dead code:

"On 32 bits Windows and MacOS, double and long have an alignment of 8. But on Linux, their alignment is 4"
(Smalltalk wordSize = 4 and: [Smalltalk platformName = 'unix']) ifTrue: [
(#('double longlong ulonglong') includes: typeName) ifTrue: [
byteAlignment := 4
]
].

As you can see, there are single quotes missing and so will the path "byteAlignment  := 4" never be reached.

I tried to figure out whether one should either fix the conditional or remove the entire passage. Maybe this got long fixed inside the FFI plugin?

Best,
Marcel




Reply | Threaded
Open this post in threaded view
|

Re: FFI | Byte alignment

marcel.taeumel
Hi Eliot,

those values are constant on a platform, right? So, we could make an external-shared-pool for it... :-)

I am almost done with the next iteration on FFI-Pools package. I am just cleaning up, adding some class comments, ...

Best,
Marcel

Am 31.05.2020 01:58:46 schrieb Eliot Miranda <[hidden email]>:

Hi Nicolas, Hi Marcel,


On May 29, 2020, at 8:02 AM, Nicolas Cellier <[hidden email]> wrote:


Try with this:

cat > test_align.c <<END
#include <stdio.h>

struct foo1 {int a; long long b; int d;};
struct foo2 {int a; double c; int d;};

Here’s a suggestion.  We add a primitive To the ThreadedFFIPlugin that answers structure alignment thusly:

- the primitive takes as it’s argument the numeric code for the integer or float type or a structure as used in the compiled type specs 
- the primitive answers the alignment of the data type in a struct that looks like this:

struct alignment_of_longlong {
     char pad_to_force_alignment;
     long long element;
}

return &((struct alignment_of_longlong *)0)-> element);

struct alignment_of_double {
     char pad_to_force_alignment;
     long long element;
}

return &((struct alignment_of_double *)0)-> element);


struct alignment_of_struct {
     char pad_to_force_alignment;
     struct { char element; } element;
}

return &((struct alignment_of_struct *)0)-> element);

etc

Then we can remember these alignments in class vars of, say, ExternalStructure, check them on start-up, and resize/update ExternalStructure if any of the values change.

int main() {
struct foo1 x;
struct foo2 y;
printf("size of x = %d\n",sizeof(x));
printf("size of y = %d\n",sizeof(y));
return 0;
}
END
i686-w64-mingw32-gcc test_align.c
 ./a.exe

size of x = 24
size of y = 24

so at leat in windows (mingw) the alignment is 8 bytes for both long long and double.
I have no 32bits linux handy, but it should be easy enough to pass the same test...

Le ven. 29 mai 2020 à 16:45, Nicolas Cellier <[hidden email]> a écrit :
Good catch!
Wikipedia page does not even agree on longlong... But it's not the ultimate normative reference, better check by ourselves!

Le ven. 29 mai 2020 à 15:53, Marcel Taeumel <[hidden email]> a écrit :
Hi, there.

In ExternalType class >> #initializeAtomicTypes, there is an interesting claim and a piece of dead code:

"On 32 bits Windows and MacOS, double and long have an alignment of 8. But on Linux, their alignment is 4"
(Smalltalk wordSize = 4 and: [Smalltalk platformName = 'unix']) ifTrue: [
(#('double longlong ulonglong') includes: typeName) ifTrue: [
byteAlignment := 4
]
].

As you can see, there are single quotes missing and so will the path "byteAlignment  := 4" never be reached.

I tried to figure out whether one should either fix the conditional or remove the entire passage. Maybe this got long fixed inside the FFI plugin?

Best,
Marcel




Reply | Threaded
Open this post in threaded view
|

Re: FFI | Byte alignment

Nicolas Cellier
Hi all,
I like the primitive idea, but if we can do without, that's also fine.
as long as we let C compiler do the job in both cases...

Le dim. 31 mai 2020 à 21:21, Marcel Taeumel <[hidden email]> a écrit :
Hi Eliot,

those values are constant on a platform, right? So, we could make an external-shared-pool for it... :-)

I am almost done with the next iteration on FFI-Pools package. I am just cleaning up, adding some class comments, ...

Best,
Marcel

Am 31.05.2020 01:58:46 schrieb Eliot Miranda <[hidden email]>:

Hi Nicolas, Hi Marcel,


On May 29, 2020, at 8:02 AM, Nicolas Cellier <[hidden email]> wrote:


Try with this:

cat > test_align.c <<END
#include <stdio.h>

struct foo1 {int a; long long b; int d;};
struct foo2 {int a; double c; int d;};

Here’s a suggestion.  We add a primitive To the ThreadedFFIPlugin that answers structure alignment thusly:

- the primitive takes as it’s argument the numeric code for the integer or float type or a structure as used in the compiled type specs 
- the primitive answers the alignment of the data type in a struct that looks like this:

struct alignment_of_longlong {
     char pad_to_force_alignment;
     long long element;
}

return &((struct alignment_of_longlong *)0)-> element);

struct alignment_of_double {
     char pad_to_force_alignment;
     long long element;
}

return &((struct alignment_of_double *)0)-> element);


struct alignment_of_struct {
     char pad_to_force_alignment;
     struct { char element; } element;
}

return &((struct alignment_of_struct *)0)-> element);

etc

Then we can remember these alignments in class vars of, say, ExternalStructure, check them on start-up, and resize/update ExternalStructure if any of the values change.

int main() {
struct foo1 x;
struct foo2 y;
printf("size of x = %d\n",sizeof(x));
printf("size of y = %d\n",sizeof(y));
return 0;
}
END
i686-w64-mingw32-gcc test_align.c
 ./a.exe

size of x = 24
size of y = 24

so at leat in windows (mingw) the alignment is 8 bytes for both long long and double.
I have no 32bits linux handy, but it should be easy enough to pass the same test...

Le ven. 29 mai 2020 à 16:45, Nicolas Cellier <[hidden email]> a écrit :
Good catch!
Wikipedia page does not even agree on longlong... But it's not the ultimate normative reference, better check by ourselves!

Le ven. 29 mai 2020 à 15:53, Marcel Taeumel <[hidden email]> a écrit :
Hi, there.

In ExternalType class >> #initializeAtomicTypes, there is an interesting claim and a piece of dead code:

"On 32 bits Windows and MacOS, double and long have an alignment of 8. But on Linux, their alignment is 4"
(Smalltalk wordSize = 4 and: [Smalltalk platformName = 'unix']) ifTrue: [
(#('double longlong ulonglong') includes: typeName) ifTrue: [
byteAlignment := 4
]
].

As you can see, there are single quotes missing and so will the path "byteAlignment  := 4" never be reached.

I tried to figure out whether one should either fix the conditional or remove the entire passage. Maybe this got long fixed inside the FFI plugin?

Best,
Marcel





Reply | Threaded
Open this post in threaded view
|

Re: FFI | Byte alignment

Eliot Miranda-2
In reply to this post by marcel.taeumel


On Sun, May 31, 2020 at 12:21 PM Marcel Taeumel <[hidden email]> wrote:
Hi Eliot,

those values are constant on a platform, right? So, we could make an external-shared-pool for it... :-)

Right.  I think that the primitive is a simple way of generating the data.  At least it is organized with the rest of he plugin.  Having a separate program for it is perhaps a maintenance problem.  So I think the primitive makes sense.


I am almost done with the next iteration on FFI-Pools package. I am just cleaning up, adding some class comments, ...

Cool!!
 

Best,
Marcel

Am 31.05.2020 01:58:46 schrieb Eliot Miranda <[hidden email]>:

Hi Nicolas, Hi Marcel,


On May 29, 2020, at 8:02 AM, Nicolas Cellier <[hidden email]> wrote:


Try with this:

cat > test_align.c <<END
#include <stdio.h>

struct foo1 {int a; long long b; int d;};
struct foo2 {int a; double c; int d;};

Here’s a suggestion.  We add a primitive To the ThreadedFFIPlugin that answers structure alignment thusly:

- the primitive takes as it’s argument the numeric code for the integer or float type or a structure as used in the compiled type specs 
- the primitive answers the alignment of the data type in a struct that looks like this:

struct alignment_of_longlong {
     char pad_to_force_alignment;
     long long element;
}

return &((struct alignment_of_longlong *)0)-> element);

struct alignment_of_double {
     char pad_to_force_alignment;
     long long element;
}

return &((struct alignment_of_double *)0)-> element);


struct alignment_of_struct {
     char pad_to_force_alignment;
     struct { char element; } element;
}

return &((struct alignment_of_struct *)0)-> element);

etc

Then we can remember these alignments in class vars of, say, ExternalStructure, check them on start-up, and resize/update ExternalStructure if any of the values change.

int main() {
struct foo1 x;
struct foo2 y;
printf("size of x = %d\n",sizeof(x));
printf("size of y = %d\n",sizeof(y));
return 0;
}
END
i686-w64-mingw32-gcc test_align.c
 ./a.exe

size of x = 24
size of y = 24

so at leat in windows (mingw) the alignment is 8 bytes for both long long and double.
I have no 32bits linux handy, but it should be easy enough to pass the same test...

Le ven. 29 mai 2020 à 16:45, Nicolas Cellier <[hidden email]> a écrit :
Good catch!
Wikipedia page does not even agree on longlong... But it's not the ultimate normative reference, better check by ourselves!

Le ven. 29 mai 2020 à 15:53, Marcel Taeumel <[hidden email]> a écrit :
Hi, there.

In ExternalType class >> #initializeAtomicTypes, there is an interesting claim and a piece of dead code:

"On 32 bits Windows and MacOS, double and long have an alignment of 8. But on Linux, their alignment is 4"
(Smalltalk wordSize = 4 and: [Smalltalk platformName = 'unix']) ifTrue: [
(#('double longlong ulonglong') includes: typeName) ifTrue: [
byteAlignment := 4
]
].

As you can see, there are single quotes missing and so will the path "byteAlignment  := 4" never be reached.

I tried to figure out whether one should either fix the conditional or remove the entire passage. Maybe this got long fixed inside the FFI plugin?

Best,
Marcel





--
_,,,^..^,,,_
best, Eliot