In conjunction with getting Open Cobalt running on FreeBSD I have been looking into the problems with SqueakFFIPrims on FreeBSD and have reached a point that is beyond my understanding. I am not sure if the fix should come from the VM side or the FreeBSD port maintainer. I needed to get source code back into the ports tree so I did a make in /usr/ports/lang/squeak. I already had the 3.9 tarball, so that just repopulated the source and did the initial build. In the "files" folder I found patch files typical for a FreeBSD port. These patch source files from a generic tarball to work with FreeBSD. The one of interest was patch-platforms__unix__plugins__SqueakFFIPrims__ffi-config which contains ----x-----x----- --- platforms/unix/plugins/SqueakFFIPrims/ffi-config.org Wed Apr 26 20:27:53 2006 +++ platforms/unix/plugins/SqueakFFIPrims/ffi-config Wed Apr 26 20:29:00 2006 @@ -39,6 +39,7 @@ case ${abi} in linux) abi=sysv;; + freebsd*) abi=sysv;; darwin*) abi=darwin;; *) abi=libffi; lib="-lffi";; esac -----x-----x----- I then had the folder work/Squeak-3.9-7/platforms/unix/plugins/SqueakFFIPrims In it is a file 00README which explains how to test FFI. It says to run ffi-test-config but I have no such file. It says to type "make" to build a test suite but the build failed. It says to try with make CPU=any ABI=libffi LIB=-lffi which also failed. I traced the problem to the gcc search paths for includes and libraries -- turns out FreeBSD only searches /usr/include and /usr/lib when the required files are in /usr/local/include and /usr/local/lib. After some fiddling I finally had a "main" to test. It failed: ffi-test-main.c failed at line 361. Here is a bit from that file, ending with line 361: ffiInitialize(); for (ul= 0; ul < 15; ++ul) ffiPushSingleFloat(fa[ul]); GO(FFITypeSingleFloat, many); f= ffiReturnFloatValue(); ffiCleanup(); CHECK(f - ff < FLT_EPSILON); I worked out the CHECK macro but could not find what FLT_EPSILON is. At that point I decided to stop and come to you for help. Where do I go from here? -- Gary Dunn, Honolulu [hidden email] http://openslate.net/ http://e9erust.blogspot.com/ Sent from Slate001 |
On Thu, Nov 19, 2009 at 1:47 AM, Gary Dunn <[hidden email]> wrote:
define FLT_EPSILON as something like 1e-20 and rewrite the check as CHECK(fabs(f - ff) < FLT_EPSILON)?
In math epsilon is a small value tending towards zero. The intent here is to check that you get back something close to ff without assuming that the floating-point arithmetic involved in many is exact enough to return ff.
|
> define FLT_EPSILON as something like 1e-20 and rewrite the check as CHECK(fabs(f - ff) < FLT_EPSILON)? > > In math epsilon is a small value tending towards zero. The intent here is to check that you get back something close to ff without assuming that the floating-point arithmetic involved in many is exact enough to return ff. Ok, the implication then for testing is that ffiPushSingleFloat ffiReturnFloatValue modify the floating point value slightly, if that is acceptable then you need to epsilon test. also GCC seems to define #define FLT_EPSILON 1.19209290e-07F across all flavours of CPUs and float word lengths. perhaps via "float.h" -- =========================================================================== John M. McIntosh <[hidden email]> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com =========================================================================== |
On Thu, Nov 19, 2009 at 9:29 AM, John M McIntosh <[hidden email]> wrote:
No. Look at the body of many: static float many(float f1, float f2, float f3, float f4, float f5, float f6, float f7, float f8, float f9, float f10, float f11, float f12, float f13, float f14, float f15)
{ DPRINTF(("%f %f %f %f %f %f %f %f %f %f %f %f %f %f %f\n", (double)f1, (double)f2, (double)f3, (double)f4, (double)f5, (double)f6, (double)f7, (double)f8, (double)f9, (double)f10,
(double)f11, (double)f12, (double)f13, (double)f14, (double)f15)); return ((f1/f2+f3/f4+f5/f6+f7/f8+f9/f10+f11/f12+f13/f14) * f15); } There's lots of scope for floating-point rounding errors here, so the epsilon test is required. There's no implication that any float marshalling code alters the values involved. That would be a huge bug.
|
I can report progress but no solution. 1. Since ffi-test-main failed to build with a simple "make" I had assumed I needed to use libffi. I added a print statement to show the values at the point of failure at line 361, the CHECK statement: ffiInitialize(); for (ul= 0; ul < 15; ++ul) ffiPushSingleFloat(fa[ul]); GO(FFITypeSingleFloat, many); f= ffiReturnFloatValue(); ffiCleanup(); printf("%g %g %g %g\n", f, ff, f - ff, FLT_EPSILON); CHECK(f - ff < FLT_EPSILON); It gave f and ff to be identical, f - ff zero, and FLT_EPSILON a very small value. The strange thing was that the test did not fail. I commented out my print statement and it failed. Weird. I noticed I was not seeing the output from puts statements. Recalling that this is typically a macro I looked to see if it had been defined, and found this near the top of the file: #if 0 # define dprintf(ARGS) printf ARGS # define puts(ARG) puts(ARG) #else # define dprintf(ARGS) # define puts(ARG) #endif I changed the zero to one and the output was verbose. No failure, even with my print statement commented out. I suspect there is some tricky coding here that works on some versions of C and not others. The preprocessor, especially. So I got to wondering why make without args failed. I traced the problem to these lines from the Makefile CPU= $(shell ./ffi-config -cpu) ABI= $(shell ./ffi-config -abi) LIB= $(shell ./ffi-config -lib) Running them manually gave x86, sysv and blank, but OBJ= ffi-test-main.o ffi-test.o $(CPU)-$(ABI).o $(CPU)-$(ABI)-asm.o resolved to ffi-test-main.o ffi-test.o -.o --asm.o which caused make to complain that it did not know how to make -.o. I substituted like this #OBJ= ffi-test-main.o ffi-test.o $(CPU)-$(ABI).o $(CPU)-$(ABI)-asm.o OBJ= ffi-test-main.o ffi-test.o x86-sysv.o x86-sysv-asm.o and the build completed. Eventually I figured out that the shell construct works in gnu make. FreeBSD does not use gnu make. It is available as gmake. I put back the Makefile and used gmake and got a good build. The results are inconclusive. The output is slate01# ./main passed ffi assertions (0 failed) Problem passing 64bit structures Is that because I am not using a 64bit CPU? Is that a show stopper? Apparently not. The thing is, this does not fix the real problem. If I load my 3.9 image and install the FFI package from SqueakMap, then run the FFI-Tests in SUnit runner I get 21 run, 0 passes, 0 expected failures, 0 failures, 21 errors, 0 unexpected passes. When I click on a result and open a debugger I find Error: Unable to find function address invokeWithArguments: argArray "Manually invoke the receiver, representing an external function." <primitive: 'primitiveCalloutWithArgs' module:'SqueakFFIPrims'> ^self externalCallFailed externalCallFailed "Raise an error after a failed call to an external function" | errCode | errCode := ExternalFunction getLastError. "this allows us to look at the actual error code" ^self error: (ExternalFunction errorMessageFor: errCode). error: aString "Throw a generic Error exception." ^Error new signal: aString If the ffi-test-main.c tests are passing, why are the SUnit tests all failing? Is that 64bit thing a big deal? -- Gary Dunn, Honolulu [hidden email] http://openslate.net/ http://e9erust.blogspot.com/ Sent from Slate001 |
On Fri, Nov 20, 2009 at 3:26 AM, Gary Dunn <[hidden email]> wrote:
twice as big a deal as 32-bits... But seriously folks... Have you built a SqueakFFIPrims shared object? Have you installed it in the right directory with other plugins? Is the export table in SqueakFFIPrims.c correct (calls itself SqueakFFIPrims, includes both primitiveCallout and primitiveCalloutWithArgs as exports).
First try and understand invokeWithArguments: argArray "Manually invoke the receiver, representing an external function." <primitive: 'primitiveCalloutWithArgs' module:'SqueakFFIPrims'> ^self externalCallFailed and Error: Unable to find function address.
The first is a method that includes a spec for calling a plugin primitive, in the SqueakFFIPrims plugin, which could be in a shared object, a dll or a bundle depending on platform. You're on linux so it should be in either SqueakFFIPrims or SqueakFFIPrims.so depending on which version of the VM you're working on.
The second tells you that, probably the VM is finding the SqueakFFIPrims module but its not finding primitiveCalloutWithArgs within it. You can check by tracing execution through ioLoadExternalFunctionOfLengthFromModuleOfLength. Assuming the module is being found you need to find out why the function isn't being found.
At the end of SqueakFFIPrims.c you should see a table that looks something like
#ifdef SQUEAK_BUILTIN_PLUGIN void* SqueakFFIPrims_exports[][3] = { {"SqueakFFIPrims", "ffiLogCallsTo", (void*)ffiLogCallsTo}, {"SqueakFFIPrims", "getModuleName", (void*)getModuleName},
{"SqueakFFIPrims", "initialiseModule", (void*)initialiseModule}, {"SqueakFFIPrims", "primitiveCallout", (void*)primitiveCallout}, {"SqueakFFIPrims", "primitiveCalloutWithArgs", (void*)primitiveCalloutWithArgs},
{"SqueakFFIPrims", "primitiveFFIAllocate", (void*)primitiveFFIAllocate}, {"SqueakFFIPrims", "primitiveFFIDoubleAt", (void*)primitiveFFIDoubleAt}, {"SqueakFFIPrims", "primitiveFFIDoubleAtPut", (void*)primitiveFFIDoubleAtPut},
{"SqueakFFIPrims", "primitiveFFIFloatAt", (void*)primitiveFFIFloatAt}, {"SqueakFFIPrims", "primitiveFFIFloatAtPut", (void*)primitiveFFIFloatAtPut},
{"SqueakFFIPrims", "primitiveFFIFree", (void*)primitiveFFIFree}, {"SqueakFFIPrims", "primitiveFFIGetLastError", (void*)primitiveFFIGetLastError}, {"SqueakFFIPrims", "primitiveFFIIntegerAt", (void*)primitiveFFIIntegerAt},
{"SqueakFFIPrims", "primitiveFFIIntegerAtPut", (void*)primitiveFFIIntegerAtPut}, {"SqueakFFIPrims", "primitiveForceLoad", (void*)primitiveForceLoad},
{"SqueakFFIPrims", "primitiveLogCallsTo", (void*)primitiveLogCallsTo}, {"SqueakFFIPrims", "setInterpreter", (void*)setInterpreter}, {NULL, NULL, NULL}
}; #endif /* ifdef SQ_BUILTIN_PLUGIN */ If primitiveCalloutWithArgs is missing then SqueakFFIPrims.c didn't get generated correctly. Go back to VMMaker and the FFIPlugin source and try to figure out why. Note that the table shouldn't be being used because the plugin is typically used as an external plugin. If you're building it in then ignore what follows and reply saying "I want an internal plugin dammit!!"
If primitiveCalloutWithArgs isn't missing then is it being exported form the module? Use nm on whatever SqueakFFIPrims is compiled to and make sure its exported. It should have something like the following:
$ nm Fast.app/Contents/Resources/SqueakFFIPrims.bundle/Contents/MacOS/SqueakFFIPrims | grep primitiveC 00004200 T _primitiveCallout 00003750 T _primitiveCalloutWithArgs If there's a lowercase t there' figure out why it isn't being exported. e.g. EXPORT macro definition, linker commands, etc... If it is being exported then you need to start debugging ioLoadSymbolOfLengthFromModule, because that's failing to extract the address of primitiveCalloutWithArgs from the loaded module.
HTH Eliot
|
For those of you in or from the US, Happy Thanksgiving, and to everyone else, all the best. In the lull between the parade and the turkey I returned to my SqueakFFIPrims on FreeBSD mystery. On Fri, 2009-11-20 at 09:30 -0800, Eliot Miranda wrote: > > > On Fri, Nov 20, 2009 at 3:26 AM, Gary Dunn <[hidden email]> wrote: > [snip] > > The thing is, this does not fix the real problem. If I load my > 3.9 image > and install the FFI package from SqueakMap, then run the > FFI-Tests in > SUnit runner I get 21 run, 0 passes, 0 expected failures, 0 > failures, 21 > errors, 0 unexpected passes. When I click on a result and open > a > debugger I find > > Error: Unable to find function address > > invokeWithArguments: argArray > "Manually invoke the receiver, representing an external > function." > <primitive: 'primitiveCalloutWithArgs' > module:'SqueakFFIPrims'> > ^self externalCallFailed > > > > externalCallFailed > "Raise an error after a failed call to an external > function" > | errCode | > errCode := ExternalFunction getLastError. "this allows > us to look at > the actual error code" > ^self error: (ExternalFunction errorMessageFor: > errCode). > > > > error: aString > "Throw a generic Error exception." > > ^Error new signal: aString > > > If the ffi-test-main.c tests are passing, why are the SUnit > tests all > failing? Is that 64bit thing a big deal? > > > twice as big a deal as 32-bits... But seriously folks... > > > Have you built a SqueakFFIPrims shared object? Have you installed it > in the right directory with other plugins? Not there, but as you suggest later it might be 'built-in' The Makefile in /usr/ports/lang/squeak/work/Squeak-3.9-7/build begins with: MAKEFLAGS= --no-print-directory VM_MAJOR= 3 VM_MINOR= 9 VM_RELEASE= 7 VM_VERSION= 3.9-7 SQ_MAJOR= 3 SQ_MINOR= 9a SQ_UPDATE= 7024 SQ_VERSION= 3.9a-7024 DEVEL= @DEVEL@ host= i386-portbld-freebsd7.1 host_cpu= i386 host_vendor= portbld host_os= freebsd7.1 topdir= /usr/ports/lang/squeak/work/Squeak-3.9-7 cfgdir= /usr/ports/lang/squeak/work/Squeak-3.9-7/platforms/unix/config srcdir= /usr/ports/lang/squeak/work/Squeak-3.9-7/platforms/unix/src blddir= /usr/ports/lang/squeak/work/Squeak-3.9-7/build top_builddir= /usr/ports/lang/squeak/work/Squeak-3.9-7/build prefix=/usr/local exec_prefix= ${prefix} bindir= ${exec_prefix}/bin libdir= /usr/local/share datadir= ${prefix}/share mandir= /usr/local/man imgdir= /usr/local/share/squeak plgdir= /usr/local/share/squeak/3.9-7 docdir=/usr/local/share/doc/squeak SHELL= /bin/sh MKINSTALLDIRS= $(SHELL) $(cfgdir)/mkinstalldirs INSTALL= /usr/bin/install -c -o root -g wheel INSTALL_PROG= install -s -o root -g wheel -m 555 $(INSTALL_ARGS) INSTALL_DATA= install -o root -g wheel -m 644 $(INSTALL_ARGS) UNINSTALL= $(SHELL) $(cfgdir)/uninstall AS= as AWK= gawk RANLIB= ranlib LIBTOOL= $(SHELL) $(top_builddir)/libtool NM= /usr/bin/nm -B LN= ln LN_S= ln -s CC= cc DEFS= -DHAVE_CONFIG_H INCLUDES= -I/usr/ports/lang/squeak/work/Squeak-3.9-7/build -I/usr/ports/lang/squeak/work/Squeak-3.9-7/platforms/unix/vm -I/usr/ports/lang/squeak/work/Squeak-3.9-7/platforms/Cross/vm -I/usr/ports/lang/squeak/work/Squeak-3.9-7/platforms/unix/src/vm CPPFLAGS= -I/usr/local/include CFLAGS= -O2 -fno-strict-aliasing -pipe -fomit-frame-pointer -DLSB_FIRST=1 WFLAGS= -Wall -Wno-unknown-pragmas LDFLAGS= -L/usr/local/lib SOFLAGS= @SOFLAGS@ LIBM_CFLAGS= -O -fomit-frame-pointer LIBS= -lutil -lm -liconv X_CFLAGS= -O2 -fno-strict-aliasing -pipe -fomit-frame-pointer -DLSB_FIRST=1 X_INCLUDES= -I/usr/ports/lang/squeak/work/Squeak-3.9-7/build -I/usr/ports/lang/squeak/work/Squeak-3.9-7/platforms/unix/vm -I/usr/ports/lang/squeak/work/Squeak-3.9-7/platforms/Cross/vm -I/usr/ports/lang/squeak/work/Squeak-3.9-7/platforms/unix/src/vm -I/usr/local/include X_LIBS= -lGL -lXext -lSM -lICE -L/usr/local/lib -lm -lX11 LIB_UUID= FFI_DIR= /usr/ports/lang/squeak/work/Squeak-3.9-7/platforms/unix/plugins/SqueakFFIPrims FFI_C= x86-sysv FFI_S= x86-sysv-asm FFI_O= x86-sysv$o x86-sysv-asm$o ... I assume plgdir is the location of the plugins directory. Mine contains: $ cd /usr/local/share/squeak/3.9-7 $ ll total 18276 drwxr-xr-x 2 root wheel - 1024 Apr 9 2009 ./ drwxr-xr-x 3 root wheel - 512 Apr 9 2009 ../ -r-xr-xr-x 1 root wheel - 6228 Apr 9 2009 AioPlugin* -r-xr-xr-x 1 root wheel - 51232 Apr 9 2009 B3DAcceleratorPlugin* -r-xr-xr-x 1 root wheel - 8472 Apr 9 2009 PseudoTTYPlugin* -r-xr-xr-x 1 root wheel - 53456 Apr 9 2009 Squeak3D* -rw-r--r-- 1 root wheel - 17584742 Apr 9 2009 SqueakV39.sources -r-xr-xr-x 1 root wheel - 30388 Apr 9 2009 UnixOSProcessPlugin* -r-xr-xr-x 1 root wheel - 6420 Apr 9 2009 XDisplayControlPlugin* -rwxr-xr-x 1 root wheel - 18200 Apr 9 2009 npsqueak.so* -r-xr-xr-x 1 root wheel - 785476 Apr 9 2009 squeak* -r-xr-xr-x 1 root wheel - 68824 Apr 9 2009 vm-display-X11* -r-xr-xr-x 1 root wheel - 5128 Apr 9 2009 vm-display-null* -r-xr-xr-x 1 root wheel - 7552 Apr 9 2009 vm-sound-NAS* -r-xr-xr-x 1 root wheel - 19720 Apr 9 2009 vm-sound-OSS* -r-xr-xr-x 1 root wheel - 4136 Apr 9 2009 vm-sound-null* Nothing there about SqueakFFIPrims. > Is the export table in SqueakFFIPrims.c correct (calls itself > SqueakFFIPrims, includes both primitiveCallout and > primitiveCalloutWithArgs as exports). > I have /usr/ports/lang/squeak/work/Squeak-3.9-7/platforms/unix/src/vm/intplugins/SqueakFFIPrims which contains the single file SqueakFFIPrims.c Near the top it has /*** Constants ***/ #define FFIAtomicTypeMask 251658240 #define FFIAtomicTypeShift 24 #define FFIErrorAddressNotFound 13 which explains why the tests fail with error 13. At the end of the file is the table as described by Eliot. [snip] > > The second tells you that, probably the VM is finding the > SqueakFFIPrims module but its not finding primitiveCalloutWithArgs > within it. You can check by tracing execution through > ioLoadExternalFunctionOfLengthFromModuleOfLength. Never been there. I Googled and found sqUnixExternalPrims.c by Andreas Raab, /* Primitive entry point from the Interpreter. Answer the address of the * given function in the given module. Fail the primitive (setting * successFlag to false) if the function cannot be found. */ int ioLoadExternalFunctionOfLengthFromModuleOfLength( int functionNameIndex, int functionNameLength, int moduleNameIndex, int moduleNameLength) { ... I have that in /usr/ports/lang/squeak/work/Squeak-3.9-7/platforms/unix/src/vm How do I trace execution through that? (If I am getting in over my head please say so. I don't want to waste people's time.) [snip] > Note that the table shouldn't be being used because the plugin is > typically used as an external plugin. If you're building it in then > ignore what follows and reply saying "I want an internal plugin > dammit!!" > > > If primitiveCalloutWithArgs isn't missing then is it being exported > form the module? Use nm on whatever SqueakFFIPrims is compiled to and > make sure its exported. I can't find a SqueakFFIPrims.o file. Is there some other way to use nm? That's it for now, it's time to feast. I will keep on poking, and certainly appreciate any and all help. -- Gary Dunn, Honolulu [hidden email] http://openslate.net/ http://e9erust.blogspot.com/ Sent from Slate001 |
On 27.11.2009, at 01:44, Gary Dunn wrote: > > > [...] SqueakFFIPrims on FreeBSD mystery. > Not there, but as you suggest later it might be 'built-in' Unlikely. Under normal conditions it is built externally. > The Makefile in /usr/ports/lang/squeak/work/Squeak-3.9-7/build begins > with: Did you try the latest sources yet? I mean from squeakvm.org, not the ports. > That's it for now, it's time to feast. I will keep on poking, and > certainly appreciate any and all help. The ffi-test-main.c failing may be a red herring. Not sure anybody ever used that do it may suffer from bit rot. Try building am VM from the current source, and if it does not work (it is supposed to), report back with the logs from the build process and describe the failure. - Bert - |
When I was working on the 64bit mac platform support I wrote this test case to check that the byte/word/float accessors worked correctly. On Fri, Nov 27, 2009 at 7:13 AM, Bert Freudenberg <[hidden email]> wrote:
=========================================================================== John M. McIntosh. Corporate Smalltalk Consulting Ltd https://www.linkedin.com/in/smalltalk =========================================================================== FFITestCase.st (3K) Download Attachment |
Free forum by Nabble | Edit this page |