FFI | Byte alignment

And for your other question concerning the plugin:

1. it's been fixed only recently

2. it's not completely fixed with this regard!

See ThreadedFFIPlugin>>checkAlignmentOfStructSpec:OfLength:StartingAt:

We are always using the filedSize in case of atomic type as fieldAlignment...

So it ain't gonna work for double/longlong/ulonglong on 32 bits linux

(unless library is compiled with -malign-double or other exotic flags, but how could FFI guess that...)

Ideally, the alignment would be occupying some bits in the type spec.

Or we hardcode the well known exception in a second place.

Anyway, the plugin also had to duplicate the alignment algorithm as you can see...

We live in a world full of such accidental complexity...

At least, while mankind is taking care of such detail, this leaves less time for killing each other, so let's declare that it's a good thing ;)

Le ven. 29 mai 2020 à 17:37, Nicolas Cellier <[hidden email]> a écrit :

Found a 32bits debian VM:

st@debi9:~$ gcc test_align.c
st@debi9:~$ ./a.out
size of x = 16
size of y = 16

So keep the code, just correct it!

Le ven. 29 mai 2020 à 17:01, Nicolas Cellier <[hidden email]> a écrit :

Try with this:

cat > test_align.c <<END

#include <stdio.h>

struct foo1 {int a; long long b; int d;};
struct foo2 {int a; double c; int d;};

int main() {
struct foo1 x;
struct foo2 y;
printf("size of x = %d\n",sizeof(x));
printf("size of y = %d\n",sizeof(y));
return 0;
}

END

i686-w64-mingw32-gcc test_align.c

 ./a.exe

size of x = 24
size of y = 24

so at leat in windows (mingw) the alignment is 8 bytes for both long long and double.

I have no 32bits linux handy, but it should be easy enough to pass the same test...

Le ven. 29 mai 2020 à 16:45, Nicolas Cellier <[hidden email]> a écrit :

Good catch!

Wikipedia page does not even agree on longlong... But it's not the ultimate normative reference, better check by ourselves!

https://en.wikipedia.org/wiki/Data_structure_alignment

Le ven. 29 mai 2020 à 15:53, Marcel Taeumel <[hidden email]> a écrit :

Hi, there.

In ExternalType class >> #initializeAtomicTypes, there is an interesting claim and a piece of dead code:

"On 32 bits Windows and MacOS, double and long have an alignment of 8. But on Linux, their alignment is 4"

(Smalltalk wordSize = 4 and: [Smalltalk platformName = 'unix']) ifTrue: [

(#('double longlong ulonglong') includes: typeName) ifTrue: [

byteAlignment := 4

]

].

As you can see, there are single quotes missing and so will the path "byteAlignment  := 4" never be reached.

I tried to figure out whether one should either fix the conditional or remove the entire passage. Maybe this got long fixed inside the FFI plugin?

Best,

Marcel

Hi Eliot,

those values are constant on a platform, right? So, we could make an external-shared-pool for it... :-)

I am almost done with the next iteration on FFI-Pools package. I am just cleaning up, adding some class comments, ...

Best,

Marcel

Am 31.05.2020 01:58:46 schrieb Eliot Miranda <[hidden email]>:

Hi Nicolas, Hi Marcel,

On May 29, 2020, at 8:02 AM, Nicolas Cellier <[hidden email]> wrote:



Try with this:

cat > test_align.c <<END

#include <stdio.h>

struct foo1 {int a; long long b; int d;};
struct foo2 {int a; double c; int d;};

Here’s a suggestion.  We add a primitive To the ThreadedFFIPlugin that answers structure alignment thusly:

- the primitive takes as it’s argument the numeric code for the integer or float type or a structure as used in the compiled type specs 

- the primitive answers the alignment of the data type in a struct that looks like this:

struct alignment_of_longlong {

     char pad_to_force_alignment;

     long long element;

}

return &((struct alignment_of_longlong *)0)-> element);

struct alignment_of_double {

     char pad_to_force_alignment;

     long long element;

}

return &((struct alignment_of_double *)0)-> element);

struct alignment_of_struct {

     char pad_to_force_alignment;

     struct { char element; } element;

}

return &((struct alignment_of_struct *)0)-> element);

etc

Then we can remember these alignments in class vars of, say, ExternalStructure, check them on start-up, and resize/update ExternalStructure if any of the values change.

int main() {
struct foo1 x;
struct foo2 y;
printf("size of x = %d\n",sizeof(x));
printf("size of y = %d\n",sizeof(y));
return 0;
}

END

i686-w64-mingw32-gcc test_align.c

 ./a.exe

size of x = 24
size of y = 24

so at leat in windows (mingw) the alignment is 8 bytes for both long long and double.

I have no 32bits linux handy, but it should be easy enough to pass the same test...

Le ven. 29 mai 2020 à 16:45, Nicolas Cellier <[hidden email]> a écrit :

Good catch!

Wikipedia page does not even agree on longlong... But it's not the ultimate normative reference, better check by ourselves!

https://en.wikipedia.org/wiki/Data_structure_alignment

Le ven. 29 mai 2020 à 15:53, Marcel Taeumel <[hidden email]> a écrit :

Hi, there.

In ExternalType class >> #initializeAtomicTypes, there is an interesting claim and a piece of dead code:

"On 32 bits Windows and MacOS, double and long have an alignment of 8. But on Linux, their alignment is 4"

(Smalltalk wordSize = 4 and: [Smalltalk platformName = 'unix']) ifTrue: [

(#('double longlong ulonglong') includes: typeName) ifTrue: [

byteAlignment := 4

]

].

As you can see, there are single quotes missing and so will the path "byteAlignment  := 4" never be reached.

I tried to figure out whether one should either fix the conditional or remove the entire passage. Maybe this got long fixed inside the FFI plugin?

Best,

Marcel

Hi Eliot,

those values are constant on a platform, right? So, we could make an external-shared-pool for it... :-)

I am almost done with the next iteration on FFI-Pools package. I am just cleaning up, adding some class comments, ...

Best,

Marcel

Am 31.05.2020 01:58:46 schrieb Eliot Miranda <[hidden email]>:

Hi Nicolas, Hi Marcel,

On May 29, 2020, at 8:02 AM, Nicolas Cellier <[hidden email]> wrote:



Try with this:

cat > test_align.c <<END

#include <stdio.h>

struct foo1 {int a; long long b; int d;};
struct foo2 {int a; double c; int d;};

Here’s a suggestion.  We add a primitive To the ThreadedFFIPlugin that answers structure alignment thusly:

- the primitive takes as it’s argument the numeric code for the integer or float type or a structure as used in the compiled type specs 

- the primitive answers the alignment of the data type in a struct that looks like this:

struct alignment_of_longlong {

     char pad_to_force_alignment;

     long long element;

}

return &((struct alignment_of_longlong *)0)-> element);

struct alignment_of_double {

     char pad_to_force_alignment;

     long long element;

}

return &((struct alignment_of_double *)0)-> element);

struct alignment_of_struct {

     char pad_to_force_alignment;

     struct { char element; } element;

}

return &((struct alignment_of_struct *)0)-> element);

etc

Then we can remember these alignments in class vars of, say, ExternalStructure, check them on start-up, and resize/update ExternalStructure if any of the values change.

int main() {
struct foo1 x;
struct foo2 y;
printf("size of x = %d\n",sizeof(x));
printf("size of y = %d\n",sizeof(y));
return 0;
}

END

i686-w64-mingw32-gcc test_align.c

 ./a.exe

size of x = 24
size of y = 24

so at leat in windows (mingw) the alignment is 8 bytes for both long long and double.

I have no 32bits linux handy, but it should be easy enough to pass the same test...

Le ven. 29 mai 2020 à 16:45, Nicolas Cellier <[hidden email]> a écrit :

Good catch!

Wikipedia page does not even agree on longlong... But it's not the ultimate normative reference, better check by ourselves!

https://en.wikipedia.org/wiki/Data_structure_alignment

Le ven. 29 mai 2020 à 15:53, Marcel Taeumel <[hidden email]> a écrit :

Hi, there.

In ExternalType class >> #initializeAtomicTypes, there is an interesting claim and a piece of dead code:

"On 32 bits Windows and MacOS, double and long have an alignment of 8. But on Linux, their alignment is 4"

(Smalltalk wordSize = 4 and: [Smalltalk platformName = 'unix']) ifTrue: [

(#('double longlong ulonglong') includes: typeName) ifTrue: [

byteAlignment := 4

]

].

As you can see, there are single quotes missing and so will the path "byteAlignment  := 4" never be reached.

I tried to figure out whether one should either fix the conditional or remove the entire passage. Maybe this got long fixed inside the FFI plugin?

Best,

Marcel