latest sources removal platforms/unix/vm/sunos.h

 
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


Hello,

After removal of the sunos.h header file in

 platforms/unix/vm/sunos.h

this old file is gone now after some cleanup, it seems the Cog VM and Stack VM
still compile fine on Solaris 11.3.

I did not test yet on Solaris 11.4, but on Solaris 11.3 at least, it works.

The 64bit Cog VM and Stack VM compile fine (even after removal of sunos.h),
and they work.

HOWEVER ...  the 32bit version seems to compile fine but there's a new problem:

$ sqcogspursunosht/usr/bin/squeak Squeak6.0alpha-19843-32bit.image
sqImageFileSeek lseek: Invalid argument

also with the Stack VM in 32bit:

$ sqstkspursunosht/usr/bin/squeak Squeak6.0alpha-19843-32bit.image
sqImageFileSeek lseek: Invalid argument

The ckformat tool reports :

$ ckformat Squeak6.0alpha-19843-32bit.image
6521


What can be the reason / cause for this new "lseek Invalid argument" ?

So this problem does NOT happen on 64 bit.

For example with

$ ckformat Squeak5.3-19431-64bit.image
68021

the 64bit VM opens this image 68021 format just fine.

Regards,
David Stes

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJfpknkAAoJEAwpOKXMq1MaAo8IAJcv1N3ExsJgoLeYz/PrIxY+
saawdjwqVy+nVsKjVXHB29jDEgBnrC9vrIUxY/RJ4bJe4jItto8mlGVUUnxspzwa
jvApj2DFpaKXbzQboCpiwE2AYdfzsE07DHXppBvspC+xjCe4mG7lliiFyjxfGOne
iHBmCvJoaDvVyBCb8Uu5TShaRW+/ttcAARGmfbdmIz/zYKQLNc67IEfOPrIlS9Ks
VX4K6izd3Oim9KHAEBF5qC65RJSomI9VC8kI/QowCKEpkSQic+RsP1BdqXhDWKbP
k0k5znQ4ag96X/Sl8NjjqfP+hUEOPL5YGGo1QuxieY/nwbMX+rvzDS/UZ/U137g=
=u+R0
-----END PGP SIGNATURE-----

 

 

please remove me from the list

-------- Original Message --------
Subject: [Vm-dev] latest sources removal platforms/unix/vm/sunos.h
From: "[hidden email]" <[hidden email]>
Date: Sat, November 07, 2020 12:18 am
To: vm-dev <[hidden email]>


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


Hello,

After removal of the sunos.h header file in

platforms/unix/vm/sunos.h

this old file is gone now after some cleanup, it seems the Cog VM and Stack VM
still compile fine on Solaris 11.3.

I did not test yet on Solaris 11.4, but on Solaris 11.3 at least, it works.

The 64bit Cog VM and Stack VM compile fine (even after removal of sunos.h),
and they work.

HOWEVER ... the 32bit version seems to compile fine but there's a new problem:

$ sqcogspursunosht/usr/bin/squeak Squeak6.0alpha-19843-32bit.image
sqImageFileSeek lseek: Invalid argument

also with the Stack VM in 32bit:

$ sqstkspursunosht/usr/bin/squeak Squeak6.0alpha-19843-32bit.image
sqImageFileSeek lseek: Invalid argument

The ckformat tool reports :

$ ckformat Squeak6.0alpha-19843-32bit.image
6521


What can be the reason / cause for this new "lseek Invalid argument" ?

So this problem does NOT happen on 64 bit.

For example with

$ ckformat Squeak5.3-19431-64bit.image
68021

the 64bit VM opens this image 68021 format just fine.

Regards,
David Stes

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJfpknkAAoJEAwpOKXMq1MaAo8IAJcv1N3ExsJgoLeYz/PrIxY+
saawdjwqVy+nVsKjVXHB29jDEgBnrC9vrIUxY/RJ4bJe4jItto8mlGVUUnxspzwa
jvApj2DFpaKXbzQboCpiwE2AYdfzsE07DHXppBvspC+xjCe4mG7lliiFyjxfGOne
iHBmCvJoaDvVyBCb8Uu5TShaRW+/ttcAARGmfbdmIz/zYKQLNc67IEfOPrIlS9Ks
VX4K6izd3Oim9KHAEBF5qC65RJSomI9VC8kI/QowCKEpkSQic+RsP1BdqXhDWKbP
k0k5znQ4ag96X/Sl8NjjqfP+hUEOPL5YGGo1QuxieY/nwbMX+rvzDS/UZ/U137g=
=u+R0
-----END PGP SIGNATURE-----

 

 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


In the past there was a rather odd definition in:

platforms/unix/vm/sqPlatformSpecific.h

#ifndef __sun__
#define ftell(s) ftello(s)
#define fseek(s,o,w) fseeko(s,o,w)
#endif /* __sun__ */

I don't know what the reasoning behind that was.

I would prudently say that it looks suspicious to have such special cases,
in the source code (the presence of those __sun__ or sunos.h header file,
is perhaps going back to old work in porting or developing squeak on SunOS).

These functions ftell and ftello are:

       long ftell(FILE *stream);

       off_t ftello(FILE *stream);

where off_t is the type that is intended for 64bit file offsets I think.

Also there exists a lseek64() function.

I have no idea whether this is correct, but if I change:

- --- a/platforms/unix/vm/sqImageFileAccess.h
+++ b/platforms/unix/vm/sqImageFileAccess.h
@@ -108,7 +108,7 @@ sqImageFileWrite(void *ptr_arg, size_t sz, size_t count, sqImageFile f)
 static inline off_t
 sqImageFilePosition(sqImageFile f)
 {
- -       off_t pos = lseek(f, 0, SEEK_CUR);
+       off_t pos = lseek64(f, 0, SEEK_CUR);
        if (pos == (off_t)-1)
                perror("sqImageFilePosition lseek");
        return pos;
@@ -117,14 +117,14 @@ sqImageFilePosition(sqImageFile f)
 static inline void
 sqImageFileSeek(sqImageFile f,off_t pos)
 {
- -       if (lseek(f, pos, SEEK_SET) < 0)
+       if (lseek64(f, pos, SEEK_SET) < 0)
                perror("sqImageFileSeek lseek");
 }
 
 static inline void
 sqImageFileSeekEnd(sqImageFile f,off_t pos)
 {
- -       if (lseek(f, pos, SEEK_END) < 0)
+       if (lseek64(f, pos, SEEK_END) < 0)
                perror("sqImageFileSeekEnd lseek");
 }


Then the VM compiles in 32bit mode (as before) but now it succesfully loads
a 32bit VM image.

I'll have to think about this.

Maybe some "configure" script patch would be needed.

The old way to #undef lseek and so on seems ugly to me, personally.

(So I'm not unhappy about the cleanup that was done ...)

By the way, as I said before the 64bit VM on SunOS was and still is OK;
it is just after the recent change that the 32bit VM has this issue.

Regards,
David Stes


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJfp8FtAAoJEAwpOKXMq1MaVVYH/1Bz8MFEBs30KqfuFGk/oqjb
FNemHin321i/Elz0YZ04qn6Ch/1dV8PeotNPa1ZBZVczE6p9wBAJoSo9C/8uj83A
J+t2XwntLisUDgbEDb7Bo1Im8HlDn0ttuk4EHDMfhdHbXyF6F7XxSbT/T1H2UnAH
WH+qlaJUvqymTPJWUaDydlWR9AzJMk/wuXOzYqf5g1s/fWmD87U5OO3h5jhe2FcY
KAXdTc3Xeufy4kvAUmEnmVePE271oCIqEiQ1I+tcmwo92q1AUpzt3WJ3uPFp38/o
4GzMZIaKRyr1ddawM1L1R2zgV8XCM9iy/KVN+Mad2TyGv74GQokscGXuhcbZi28=
=WdhW
-----END PGP SIGNATURE-----

 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


After experimenting with lseek and lseek64, the following also could help:

bash-4.4$ cat fileoffsetbits.c

#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>

int main(int argc,char *argv[])
{
  printf("_FILE_OFFSET_BITS is %i\n",_FILE_OFFSET_BITS);
}

bash-4.4$ cc -m64 fileoffsetbits.c
bash-4.4$ ./a.out
_FILE_OFFSET_BITS is 64
bash-4.4$ cc -m32 fileoffsetbits.c
bash-4.4$ ./a.out
_FILE_OFFSET_BITS is 32
bash-4.4$ cc -m32 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 fileoffsetbits.c
bash-4.4$ ./a.out
_FILE_OFFSET_BITS is 64


The lfcompile manpage on Solaris says that one possibility is to compile with the above -D flags to support large files on 32bit executables.

This seems an issue that is also related to the configure.ac test
for AC_SYS_LARGEFILE

But that sets the _FILE_OFFSET_BITS in the "config.h".

So then it is an issue of having to #include config.h before the <sys/types.h>


Anyway, the SunOS 32bit VM used to work, but currently has this (minor) issue.

It is minor because it seems solvable by setting the -D options in the mvm script.

If I use the following patch it works:

- --- a/build.sunos32x86/squeak.stack.spur/build/mvm
+++ b/build.sunos32x86/squeak.stack.spur/build/mvm
@@ -10,7 +10,7 @@ esac
 # Stack Spur VM with VM profiler and threaded heartbeat
 INSTALLDIR=sqstkspursunosht/usr
 # Some gcc versions create a broken VM using -O2
- -OPT="-g -DAIO_DEBUG -DNDEBUG -DDEBUGVM=0"
+OPT="-g -DAIO_DEBUG -DNDEBUG -DDEBUGVM=0 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64"


That is probably the easiest way to get the old behavior back.


Regards,
David Stes



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJfp9PkAAoJEAwpOKXMq1MaZCcIAKzwPC+8nrluYmdLC/g+vJNf
oIF76MmaCIkaY3pfMNnjY+KyCJZk/wkM4tKXLrsEGGwUuwFXqK4ffBcmvsHCt8Zu
N+32QijgjkICqMuDX9B4kL3pQe1Jyx2e8A7NqTEL8cd0Omrw6eOmr8xnJ7f1JhC9
RSH+bGwzipwFZD9Lfn0ZlTkiYOhqZ+291OEhhvVqJuHHNUk0pcnaNcrc0iVDoTTp
BDE9mZKABzThr6MlCLyx4UmWn0cOUAikg9wf3kZYkF4qjZ4/zjQqJ/a+oZnyjqmn
3gEzqa4k2GZFa3H4xQtaZmXN9La6LgRMUv0G9y9dgu8nqBxdzb/15tjC9idi71E=
=MorK
-----END PGP SIGNATURE-----

 
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


I should add that I think normally the config.h header file already set
the 64 bit offset (via "configure").

The configure log shows:

configure:15085: cc -c  -g -DAIO_DEBUG -DNDEBUG -DDEBUGVM=0 -D_POSIX_PTHREAD_SEMANTICS -mt  -m32 conftest.c >&5
configure:15085: $? = 0
configure:15093: result: 64


So it sets correctly in config.h

#define _FILE_OFFSET_BITS 64



So depending on how to look at this, it may not be desired or needed to
- -D_FILE_OFFSET_BITS=64 in the mvm script.

That's because it is already set in the config.h.

But then there is an issue (possibly) with the #include order, whether
config.h is included before or after system headers.

David Stes


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJfp+mYAAoJEAwpOKXMq1MarYoIAImi+NL7Tk8gTNsfkqpjDYjk
lK3XS5TK0vZDNXz2GalO8i5x+6mec9BMrS5IzZn+I6JtteYlVZGPmhdV/e2iK+tR
c4CJuXlvcAaXOGysHQ3DjirnRUSsUeh17iMAeQW90eJrusFXOJ26lgzq3JgMbQOs
O69dGwT9s8ypp7j3UG3qEh36TELDZpI+3kL7T3DzmjhzsDlzNhVay5CqeGgn1bF6
cCKpyti7c/pnCFwAxzZOdXMriVUVbkCpbZF33pjUaPhfl4IUXGba3N37IQINgYuJ
TfGQ+3IHgNAZiBDgPPtFOMTcA6i6BYEQqGlo1aXsdbf7l7LQWapEctvX6ZRP/JM=
=/7eY
-----END PGP SIGNATURE-----

 
Hi David,

We should arrange that config.h is included first before anything else.  In this sense config.h is a substitute for/convenience for command line defines.

sqPlatformSpecific.h should be included after all (alas in practice most, because done system headers are pulled in before a specific function etc) system headers.

 
On 08/11/20 3:30 pm, [hidden email] wrote:

> #ifndef __sun__
> #define ftell(s) ftello(s)
> #define fseek(s,o,w) fseeko(s,o,w)
> #endif /* __sun__ */
>
> I don't know what the reasoning behind that was.

It dates back to a transitional period when 32-bit kernels just started
handling 64-bit file sizes. ISO C99 specified 32-bit offsets in
ftell/fseek while POSIX large file extensions mapped them to calls with
64-bit offsets. I doubt if these are required anymore because modern
toolchains use POSIX LFS by default.

See
http://www.gnu.org/software/libc/manual/html_node/Feature-Test-Macros.html

But it will use 32-bit offsets and can't handle images larger than 2GB :-(.

This shouldn't be required if compiles use -D_FILE_OFFSET_BITS=64
through autoconf or make scripts. The description for _FILE_OFFSET_BITS
in the URL above has more detail.

HTH .. Subbu

 
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


Thanks for the interesting link to

https://www.gnu.org/software/libc/manual/html_node/Feature-Test-Macros.html

To quote from that page:

"You should define these macros by using #define preprocessor directives at the top of your source code files. These directives must come before any #include of a system header file. It is best to make them the very first thing in the file, preceded only by comments. You could also use the -D option to GCC, but it's better if you make the source files indicate their own meaning in a self-contained way. "

Note the fact that it says:  "before any #include of a system header file".

So the change that I observe (because it used to work) is perhaps due to
reorganizing the source code so that the "before any #include" is no longer so.

Also this is of course very fragile, perhaps using the -D option in the mvm
script (which is an alternative that I tested and it works) can be done.

In fact the issue could be due to

AC_SYS_LARGEFILE

in the platforms/unix/config/configure.ac

which currently seems to set in the config.h.in the _FILE_OFFSET_BITS.

However perhaps AC_SYS_LARGEFILE could be instructed to not change the
config.h header but rather just add the -D option to the CFLAGS.

David Stes

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJfqPQ3AAoJEAwpOKXMq1MaFwIH/RP9Yg6cXLZl2smd+QlQGHoe
4iR0tDzkcSImSmIySY/2FS+4eHKvodMfNkIbRTRDhgvbFsV9ajKKi/m7oCAqgt6f
OTES0SXxZNZojDinM18IelVbnVZ0S+9MzPw1LXBrToqhKJ+PfPK+kHBnBbSMcpHF
/29pyFT4x3nhG7xzACCCHNiZ2NN2LJdf7u8TYf41kt6EL7SRvsaR3cwK7kRJmli2
dmtrUGBN0VJ4bxYIEi4ztvbtaOCCqLbIQRJ+5e9Qjuq7xjK+H5ZPDyJgU/aZHRrm
VW/i1ne7r/1zLqR4ulECLqX9rYGrm19ItcUka8cU/0dEi9d1HVX5C/zy1ut3QfI=
=ftSh
-----END PGP SIGNATURE-----

 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


Also see an interesting bug report for Debian Linux:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=158969

So arguably this is some sort of "bug" (or feature?) of "GNU configure"

and the AC_SYS_LARGEFILE macro.

The documentation says that it will append

         -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64

but this is apparently in my version of configure to "DEFS".

The Debian Linux guy correctly states that the variable is DEFS.

I just tested it and for me on Solaris 11.3 at least, the configure

AC_SYS_LARGEFILE

tests and expands than in the Makefile :

@DEFS@ expands to

- -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -D_FILE_OFFSET_BITS=64


Note that it appends the -D_FILE_OFFSET_BITS=64 to @DEFS@ !

So it is not so clear to me whether to patch the "mvm" script to set
- -D_FILE_OFFSET_BITS=64  or whether this is some sort of GNU configure bug ...

Anyway an easy solution would be to just patch the SunOS mvm script and set

  -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64

in the mvm script.

However that duplicates part of the autoconf configure script functionality
(the AC_SYS_LARGEFILE macro).

David Stes

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJfqR50AAoJEAwpOKXMq1Main4H/1uQ84SNt3ZJVEhO6PFe+GGP
qBReQhzrt2n0SAUkY/p/sn14lX9czBOumwXcDNwiE3AaE1+i/2r+F+XRMXfQp8Bn
WjyoehUDdO/qZIC580OBBDKTbiLMM7Zp5PJ7IWKrT7ue4QQZOmTbZvrroT9z3Ja4
dn8a44xesheY7e5JHi+rrZ0W0nr+vdhPdNNDZ6oU6N4SabIAYcys7QxnfTsfXYVr
Of7ZM0dYnDffoW7wJbaoxdcIr/qu+JmrCfcNC7jXDHhHppF2onnUibH0PNWlf6v4
H3tHFCdhlBCiZ0cb+SSjYWahSjJIVuC10d2u5oaC7a10WfhSaGv4sH7zlxKW9Es=
=TMvm
-----END PGP SIGNATURE-----

 
Has Squeak come full circle with this question?

A quarter century ago Dicky T was funnier:

http://lists.squeakfoundation.org/pipermail/squeak-dev/1998-August/009428.html

On 11/7/20 4:45 PM, Eliot Miranda wrote:
 > _,,,^..^,,,_ (phone)
 >
 >> On Nov 7, 2020, at 11:34 AM, [hidden email] wrote:
 >>
 >> 
 >> please remove me from the list
 >
 > Remove yourself.