Call to Unix experts: how i can obtain a pointers to mmap/dlopen/dlsym functions?
Locked
Call to Unix experts: how i can obtain a pointers to mmap/dlopen/dlsym functions?
 
|
Hello, i am currently trying to implement a bootstrap functionality for NativeBoost on Linux. And i need to access the mmap() function in order to be able to play with virtual memory. Things was easy in Windows, because i were able to get pointer to any kernel function using interpreterProxy ioLoadModuleOfLength(char*, int) and void *ioLoadSymbolOfLengthFromModule(sqInt functionNameIndex, sqInt functionNameLength, void *moduleHandle) from Kernel32.dll. But on Linux, mmap() is a part of GLIBC library. Any idea, what module name i could use to get a pointer to mmap/dlopen/dlsym using interpreterProxy functions? By using pmap utility, i can see that squeak vm loads libc-2.10.1.so module: sig@sig-desktop:/proc/14514$ pmap -x 14514 14514: /home/sig/sqvm/build/squeakvm /home/sig/squeak/NativeBoost.image Address Kbytes RSS Anon Locked Mode Mapping 00110000 1272 - - - r-x-- libc-2.10.1.so 0024e000 8 - - - r---- libc-2.10.1.so 00250000 4 - - - rw--- libc-2.10.1.so but it is on my box. And i having a strong suspicion, that it won't be the same on different linux box :) so i can't simply call ioLoadModuleOfLength('libc-2.10.1.so', strlen('libc-2.10.1.so')) and be done. Is there alias of some sort, which identifies standard C library? I really don't like to adding primitives for getting pointers on these functions :( -- Best regards, Igor Stasenko AKA sig. |
|
|
On Saturday 25 Sep 2010 6:44:19 am Igor Stasenko wrote: > But on Linux, mmap() is a part of GLIBC library. > Any idea, what module name i could use to get a pointer to > mmap/dlopen/dlsym using interpreterProxy functions? libc.so.6 (for sixth generation C library). see man page for libc Subbu |
Re: Call to Unix experts: how i can obtain a pointers to mmap/dlopen/dlsym functions?
|
|
In reply to this post by Igor Stasenko
On 25.09.2010, at 03:14, Igor Stasenko wrote: > > Is there alias of some sort, which identifies standard C library? Yes: '' - Bert - |
Re: Call to Unix experts: how i can obtain a pointers to mmap/dlopen/dlsym functions?
|
|
On 25 September 2010 15:02, Bert Freudenberg <[hidden email]> wrote: > > On 25.09.2010, at 03:14, Igor Stasenko wrote: >> >> Is there alias of some sort, which identifies standard C library? > > Yes: '' > It actually identified main module (or global symbol space, if you want). But it doesn't works :( Or.. maybe it works, returning handle = 0, except that if i passing handle=0 to ioLoadSymbolOfLengthFromModule() its crashing the VM. I mean, since ioLoadSymbolOfLengthFromModule() ends up with dlsym() call, i tried to use RTLD_DEFAULT=0 constant as module handle. So, currently i'm end up with extending plugin with primitive, which returns me an address of dlsym() function. So, then i can discover anything else through it. > - Bert - > > -- Best regards, Igor Stasenko AKA sig. |
Re: Call to Unix experts: how i can obtain a pointers to mmap/dlopen/dlsym functions?
|
|
In reply to this post by K K Subbu
On 25 September 2010 10:32, K. K. Subramaniam <[hidden email]> wrote: > On Saturday 25 Sep 2010 6:44:19 am Igor Stasenko wrote: >> But on Linux, mmap() is a part of GLIBC library. >> Any idea, what module name i could use to get a pointer to >> mmap/dlopen/dlsym using interpreterProxy functions? > libc.so.6 (for sixth generation C library). see man page for libc > how i can be sure that squeak vm always using this ' sixth generation C library', and i'm not trying load a different version of it? Where the guarantees that all unix boxes using libc.so.6? > Subbu > -- Best regards, Igor Stasenko AKA sig. |
Re: Call to Unix experts: how i can obtain a pointers to mmap/dlopen/dlsym functions?
|
|
On Sat, Sep 25, 2010 at 7:30 AM, Igor Stasenko <[hidden email]> wrote:
Nothing. And on Mac OS X it will likely be libSystem.B.dylib and on Solaris will be something else again etc. You'll need to use the tool that displays the dynamic libraries against which the VM is linked. e.g.
ldd cogunix/bin/lib/squeak/3.9-7/squeak linux-gate.so.1 => (0x00765000) libutil.so.1 => /lib/libutil.so.1 (0x06b55000) libdl.so.2 => /lib/libdl.so.2 (0x00afd000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00b04000) libm.so.6 => /lib/libm.so.6 (0x00ad2000) libnsl.so.1 => /lib/libnsl.so.1 (0x064b5000) libc.so.6 => /lib/libc.so.6 (0x00b53000)
/lib/ld-linux.so.2 (0x00ab3000) otool -L Cog/oscogvm/macbuild/Fast.app/Contents/MacOS/Croquet Cog/oscogvm/macbuild/Fast.app/Contents/MacOS/Croquet: /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 476.19.0)
/System/Library/Frameworks/Carbon.framework/Versions/A/Carbon (compatibility version 2.0.0, current version 136.0.0) /System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL (compatibility version 1.0.0, current version 1.0.0)
/System/Library/Frameworks/AGL.framework/Versions/A/AGL (compatibility version 1.0.0, current version 1.0.0) /System/Library/Frameworks/QuickTime.framework/Versions/A/QuickTime (compatibility version 1.0.0, current version 1327.73.0)
/System/Library/Frameworks/CoreAudio.framework/Versions/A/CoreAudio (compatibility version 1.0.0, current version 1.0.0) /System/Library/Frameworks/AudioToolbox.framework/Versions/A/AudioToolbox (compatibility version 1.0.0, current version 1.0.0)
/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0) /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 677.26.0)
/System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration (compatibility version 1.0.0, current version 212.2.0) /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit (compatibility version 45.0.0, current version 949.54.0)
/usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.4) /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 227.0.0)
/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 32.0.0) /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices (compatibility version 1.0.0, current version 34.0.0)
It would be convenient if the VM could answer the basename of the C library against which it was linked, e.g. through a systemAttributeAt:, but I expect that would be implemented by using a manifest string constant in the VM. Clearly this info belongs in the VM, not a table in the image. But VM builders would still need to test it as it could easily get stale if its supplied from the build system. We'd need a test and e.g. one of my Alien tests is an invocation of qsort which is in the C library.
Igor, for now I would hard code this in the image. But we should add it to the VM soon. But this raises an important issue. I think the architecture of dynamic linking of both named primitives and FFI calls in Squeak is wrong. The VM does all the work of binding names to functions, taking a function name, library/plugin pair and resolving it to an address, which means for example that one can't substitute a symbolic name such as TheCLibrary in an FFI call since this name will be interpreted literally by the VM (and will fail).
If instead there were two primitives, openLibrary and lookupInLibrary (as there are in Alien) then the linking machinery could be moved up into the image where we could concretize symbolic names. This would be done either explicitly, e.g.
myPluginPrimitive <primitive: 'myPluginPrimitive' module: 'ThePlugin' error: errorCode> errorCode == #unlinked ifTrue: [^thisContext linkAndRetryExternalPluginInvocation]
... qsort: base elements: nel width: width comparator: compar <apicall: void 'qsort' (void* ulong ulong void*)
module: 'WhateverTheCLibraryIsOnThisPlatform' error: err> errorCode == #unlinked ifTrue:
[^thisContext linkAndRetryExternalPluginInvocation] ^self externalCallFailedWith: err or, as it is in VisualWorks, with a hidden callback (selector in the specialObjectsArray) that sends a message to the unlinked method whose arguments are the calls receiver and arguments. Personally I think the explicit method is better since its more flexible.
best Eliot
|
Re: Call to Unix experts: how i can obtain a pointers to mmap/dlopen/dlsym functions?
|
|
On 25 September 2010 19:30, Eliot Miranda <[hidden email]> wrote: > > > > On Sat, Sep 25, 2010 at 7:30 AM, Igor Stasenko <[hidden email]> wrote: >> >> On 25 September 2010 10:32, K. K. Subramaniam <[hidden email]> wrote: >> > On Saturday 25 Sep 2010 6:44:19 am Igor Stasenko wrote: >> >> But on Linux, mmap() is a part of GLIBC library. >> >> Any idea, what module name i could use to get a pointer to >> >> mmap/dlopen/dlsym using interpreterProxy functions? >> > libc.so.6 (for sixth generation C library). see man page for libc >> > >> how i can be sure that squeak vm always using this ' sixth generation >> C library', >> and i'm not trying load a different version of it? >> Where the guarantees that all unix boxes using libc.so.6? > > Nothing. And on Mac OS X it will likely be libSystem.B.dylib and on Solaris will be something else again etc. You'll need to use the tool that displays the dynamic libraries against which the VM is linked. e.g. > ldd cogunix/bin/lib/squeak/3.9-7/squeak > linux-gate.so.1 => (0x00765000) > libutil.so.1 => /lib/libutil.so.1 (0x06b55000) > libdl.so.2 => /lib/libdl.so.2 (0x00afd000) > libpthread.so.0 => /lib/libpthread.so.0 (0x00b04000) > libm.so.6 => /lib/libm.so.6 (0x00ad2000) > libnsl.so.1 => /lib/libnsl.so.1 (0x064b5000) > libc.so.6 => /lib/libc.so.6 (0x00b53000) > /lib/ld-linux.so.2 (0x00ab3000) > > otool -L Cog/oscogvm/macbuild/Fast.app/Contents/MacOS/Croquet > Cog/oscogvm/macbuild/Fast.app/Contents/MacOS/Croquet: > /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 476.19.0) > /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon (compatibility version 2.0.0, current version 136.0.0) > /System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL (compatibility version 1.0.0, current version 1.0.0) > /System/Library/Frameworks/AGL.framework/Versions/A/AGL (compatibility version 1.0.0, current version 1.0.0) > /System/Library/Frameworks/QuickTime.framework/Versions/A/QuickTime (compatibility version 1.0.0, current version 1327.73.0) > /System/Library/Frameworks/CoreAudio.framework/Versions/A/CoreAudio (compatibility version 1.0.0, current version 1.0.0) > /System/Library/Frameworks/AudioToolbox.framework/Versions/A/AudioToolbox (compatibility version 1.0.0, current version 1.0.0) > /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0) > /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 677.26.0) > /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration (compatibility version 1.0.0, current version 212.2.0) > /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit (compatibility version 45.0.0, current version 949.54.0) > /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0) > /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.4) > /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 227.0.0) > /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 32.0.0) > /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices (compatibility version 1.0.0, current version 34.0.0) > It would be convenient if the VM could answer the basename of the C library against which it was linked, e.g. through a systemAttributeAt:, but I expect that would be implemented by using a manifest string constant in the VM. Clearly this info belongs in the VM, not a table in the image. But VM builders would still need to test it as it could easily get stale if its supplied from the build system. We'd need a test and e.g. one of my Alien tests is an invocation of qsort which is in the C library. Exactly the same. I also using qsort to test callbacks. Obviously, its easy to run tool from command line to discover the library, and that was the reason why i can't google for solution, because all pages it finds, says: hey, just run this tool and you are done! Apparently not. I don't wanna tell user, who running squeak VM : hey pal, since you on linux, run this tool from command line and then copy the library name here and then you done. :) This is the case, when it is good to have a well-established standarts or conventions... > Igor, for now I would hard code this in the image. But we should add it to the VM soon. Well, i just added prim which returns a dlsym() function pointer. But it would be good if interpreterProxy->ioLoadSymbolOfLengthFromModule() could accept a default module handle, which corresponds to process's global symbol space. > But this raises an important issue. I think the architecture of dynamic linking of both named primitives and FFI calls in Squeak is wrong. The VM does all the work of binding names to functions, taking a function name, library/plugin pair and resolving it to an address, which means for example that one can't substitute a symbolic name such as TheCLibrary in an FFI call since this name will be interpreted literally by the VM (and will fail). > If instead there were two primitives, openLibrary and lookupInLibrary (as there are in Alien) then the linking machinery could be moved up into the image where we could concretize symbolic names. This would be done either explicitly, e.g. > myPluginPrimitive > <primitive: 'myPluginPrimitive' module: 'ThePlugin' error: errorCode> > errorCode == #unlinked ifTrue: > [^thisContext linkAndRetryExternalPluginInvocation] > ... > qsort: base elements: nel width: width comparator: compar > <apicall: void 'qsort' (void* ulong ulong void*) > module: 'WhateverTheCLibraryIsOnThisPlatform' > error: err> > errorCode == #unlinked ifTrue: > [^thisContext linkAndRetryExternalPluginInvocation] > ^self externalCallFailedWith: err > or, as it is in VisualWorks, with a hidden callback (selector in the specialObjectsArray) that sends a message to the unlinked method whose arguments are the calls receiver and arguments. Personally I think the explicit method is better since its more flexible. Yes. And i trying to add this to NativeBoost, so you can choose own way how to search for module and symbol in it. But i thought that for bootstrapping NativeBoost itself, i could reuse vm functionality to access requred bits (dlsym/dlopen), and then i free to do anything else from that point. > best > Eliot >> >> > Subbu >> > >> >> >> >> -- >> Best regards, >> Igor Stasenko AKA sig. > > > -- Best regards, Igor Stasenko AKA sig. |
Re: Call to Unix experts: how i can obtain a pointers to mmap/dlopen/dlsym functions?
|
|
On Sat, Sep 25, 2010 at 10:10 AM, Igor Stasenko <[hidden email]> wrote:
On Unix systems using dlopen et al that's dlsym(RTLD_DEFAULT,symbolname). I don't know what the equivalent, if any, there is on Windows.
|
Re: Call to Unix experts: how i can obtain a pointers to mmap/dlopen/dlsym functions?
|
|
On 25 September 2010 20:17, Eliot Miranda <[hidden email]> wrote: > > > > On Unix systems using dlopen et al that's dlsym(RTLD_DEFAULT,symbolname). I don't know what the equivalent, if any, there is on Windows. > On Windows i'm using crtdll.dll , which is C-run-time dymanically loadable library. And unless VM compiled using different compiler (not GNU one), i am pretty sure that this name will remain same. And there is no 'global symbol namespace' concept in windows. You always should supply a distinct module handle (be it dll, or process module). -- Best regards, Igor Stasenko AKA sig. |
Re: Call to Unix experts: how i can obtain a pointers to mmap/dlopen/dlsym functions?
|
|
On Sat, Sep 25, 2010 at 10:31 AM, Igor Stasenko <[hidden email]> wrote:
So what should the abstraction be? - a global space implemented directly by dlsym(RTLD_DEFAULT,...) on Unix and synthesized by enumerating al loaded modules on Windows
- provide a primitive that enumerates the loaded library names (using the standard lookup prims to search them)?
|
Re: Call to Unix experts: how i can obtain a pointers to mmap/dlopen/dlsym functions?
|
|
On 25 September 2010 20:36, Eliot Miranda <[hidden email]> wrote: > > > > On Sat, Sep 25, 2010 at 10:31 AM, Igor Stasenko <[hidden email]> wrote: >> >> On 25 September 2010 20:17, Eliot Miranda <[hidden email]> wrote: >> > >> > >> > >> > On Unix systems using dlopen et al that's dlsym(RTLD_DEFAULT,symbolname). I don't know what the equivalent, if any, there is on Windows. >> > >> >> On Windows i'm using crtdll.dll , which is C-run-time dymanically >> loadable library. >> And unless VM compiled using different compiler (not GNU one), i am >> pretty sure that this name will remain same. >> And there is no 'global symbol namespace' concept in windows. You >> always should supply a distinct module handle (be it dll, or >> process module). > > So what should the abstraction be? > - a global space implemented directly by dlsym(RTLD_DEFAULT,...) on Unix and synthesized by enumerating al loaded modules on Windows I'm not sure if its necessary for Windows. i'm not missing such functionality on Windows. Moreover, i can always implement it in image, because Kernel32.dll is always Kernel32.dll, and it is the place where i can find loadLibrary() and getProcAddres() and many other functions, like enum loaded modules etc, so i can discover all info i need about running environment. So, the absence of global symbol namespace on Windows its not an issue as to me. In constrast, on Unix, a dlsym(RTLD_DEFAULT,...) seems like the only way how you can hook-in into running environment for introspection purposes. So, i think we could leave things as it is, except modify linux version of void *ioLoadSymbolOfLengthFromModule(sqInt functionNameIndex, sqInt functionNameLength, void *moduleHandle) to accept null moduleHandle so, then it will lookup the symbol from global namespace. > - provide a primitive that enumerates the loaded library names (using the standard lookup prims to search them)? > Yes, that's would be userful. One really needs to know, what libraries is already loaded in order to avoid attempts to load different version(s) of them. >> >> >> -- >> Best regards, >> Igor Stasenko AKA sig. > > > -- Best regards, Igor Stasenko AKA sig. |
Re: Call to Unix experts: how i can obtain a pointers to mmap/dlopen/dlsym functions?
|
|
In reply to this post by Igor Stasenko
On 25.09.2010, at 16:27, Igor Stasenko wrote: > > On 25 September 2010 15:02, Bert Freudenberg <[hidden email]> wrote: >> >> On 25.09.2010, at 03:14, Igor Stasenko wrote: >>> >>> Is there alias of some sort, which identifies standard C library? >> >> Yes: '' >> > It actually identified main module (or global symbol space, if you want). > But it doesn't works :( It used to work. Guess we need to fix it? - Bert - > Or.. maybe it works, returning handle = 0, except that > if i passing handle=0 to ioLoadSymbolOfLengthFromModule() its crashing the VM. > > I mean, since ioLoadSymbolOfLengthFromModule() ends up with dlsym() call, > i tried to use RTLD_DEFAULT=0 constant as module handle. > > So, currently i'm end up with extending plugin with primitive, which > returns me an address > of dlsym() function. So, then i can discover anything else through it. > >> - Bert - >> >> > > > > -- > Best regards, > Igor Stasenko AKA sig. |
Re: Call to Unix experts: how i can obtain a pointers to mmap/dlopen/dlsym functions?
|
|
On 25 September 2010 21:08, Bert Freudenberg <[hidden email]> wrote: > > > On 25.09.2010, at 16:27, Igor Stasenko wrote: > >> >> On 25 September 2010 15:02, Bert Freudenberg <[hidden email]> wrote: >>> >>> On 25.09.2010, at 03:14, Igor Stasenko wrote: >>>> >>>> Is there alias of some sort, which identifies standard C library? >>> >>> Yes: '' >>> >> It actually identified main module (or global symbol space, if you want). >> But it doesn't works :( > > It used to work. Guess we need to fix it? > I could even attempt to do that, if someone could tell me, how to enable dprintf() to print, what is happening inside sqNamedPrims.c and sqUnixExternalPrims.c so, i can see it. > - Bert - > > >> Or.. maybe it works, returning handle = 0, except that >> if i passing handle=0 to ioLoadSymbolOfLengthFromModule() its crashing the VM. >> >> I mean, since ioLoadSymbolOfLengthFromModule() ends up with dlsym() call, >> i tried to use RTLD_DEFAULT=0 constant as module handle. >> >> So, currently i'm end up with extending plugin with primitive, which >> returns me an address >> of dlsym() function. So, then i can discover anything else through it. >> >>> - Bert - >>> >>> >> >> >> >> -- >> Best regards, >> Igor Stasenko AKA sig. > > -- Best regards, Igor Stasenko AKA sig. |
Re: Call to Unix experts: how i can obtain a pointers to mmap/dlopen/dlsym functions?
|
|
On 25 September 2010 21:14, Igor Stasenko <[hidden email]> wrote: > On 25 September 2010 21:08, Bert Freudenberg <[hidden email]> wrote: >> >> >> On 25.09.2010, at 16:27, Igor Stasenko wrote: >> >>> >>> On 25 September 2010 15:02, Bert Freudenberg <[hidden email]> wrote: >>>> >>>> On 25.09.2010, at 03:14, Igor Stasenko wrote: >>>>> >>>>> Is there alias of some sort, which identifies standard C library? >>>> >>>> Yes: '' >>>> >>> It actually identified main module (or global symbol space, if you want). >>> But it doesn't works :( >> >> It used to work. Guess we need to fix it? >> > > I could even attempt to do that, if someone could tell me, how to > enable dprintf() > to print, what is happening inside sqNamedPrims.c > and sqUnixExternalPrims.c > so, i can see it. > > Oh, it seems trivial :) /* ioLoadSymbolOfLengthFromModule This entry point is exclusively for the FFI. */ void *ioLoadSymbolOfLengthFromModule(sqInt functionNameIndex, sqInt functionNameLength, void *moduleHandle) { char *functionNamePointer= pointerForOop((usqInt)functionNameIndex); char functionName[256]; sqInt i; if(functionNameLength > 255) return 0; /* can't cope with those */ for(i=0; i< functionNameLength; i++) functionName[i] = functionNamePointer[i]; functionName[functionNameLength] = 0; if(moduleHandle) return ioFindExternalFunctionIn(functionName, moduleHandle); else return 0; } So, i suggest to remove if(moduleHandle), and modify platform-specific code to handle the null handle value. -- Best regards, Igor Stasenko AKA sig. |
«
Return to Squeak VM
|
1 view|%1 views
| Free forum by Nabble | Edit this page |