[squeak-dev] Alien & Squeak FFI issues on Snow Leopard.

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Alien & Squeak FFI issues on Snow Leopard.

johnmci
Someone finally managed to fire up a Sophie app http://www.opensophie.org/ 
  on Snow Leopard and immediately found an issue that also affects  
Squeak users who do fascinating things with FFI
The innocent FFI call was

apifstat: afileNo statBuffer: buffer
        <cdecl: long 'fstat' (long MacStat* ) module: 'libc.dylib'>
        ^self externalCallFailed

Now it seems BEFORE Snow Leopard *somehow* the Squeak external  
primitive call api which is part of the VM to assist in loading  
plugins or finding function entry points in executable binaries or  
libraries would
return the entry point for fstat by *somehow* finding libc.dylib which  
is via  /usr/lib/libc.dylib -> libSystem.dylib ->  libSystem.B.dylib

Interestingly this is found implicitly since the mac carbon vm lookup  
logic never looks in /usr/lib/

Now in Snow Leopard it seems they do less implicit searching, perhaps  
a security concern?  Then it fails.

In the past the Unix VM would search a hundred(sic) more places, but I  
understand less today, but it might still work? Maybe /usr/lib is in  
the hit list?

This issue also applies to Alien because it calls back to the VM for  
it to find the module and function entry point.

Since I have received a request to ensure that doing an explicit full  
path should be allowed in the squeak carbon macintosh vm.
Then a change would be:

apifstat: afileNo statBuffer: buffer
        <cdecl: long 'fstat' (long MacStat* ) module: '/usr/lib/libc.dylib'>
        ^self externalCallFailed

I'll look at doing this shortly.

However the workaround is to go to Sophie.app/Contents/Resources
and do a ln -s /usr/lib/libc.dylib libc.dylib
to create a symbolic link in the Resources directory for libc.dylib  
since we do explicitly look for libc.dylib in the Resources directory.

Obviously if you have a FFI call like above where you are calling a  
system library Oracle? for example.
You might find it no longer works with Snow Leopard on the Macintosh,  
so you should check.


--
=
=
=
========================================================================
John M. McIntosh <[hidden email]>   Twitter:  
squeaker68882
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
=
=
=
========================================================================





Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Re: [Pharo-project] Alien & Squeak FFI issues on Snow Leopard.

Andrey Larionov
I think it's related http://forums.newspeaklanguage.org/index.php?topic=88.0

On Wed, Sep 30, 2009 at 06:13, John M McIntosh
<[hidden email]> wrote:

> Someone finally managed to fire up a Sophie app http://www.opensophie.org/
>  on Snow Leopard and immediately found an issue that also affects
> Squeak users who do fascinating things with FFI
> The innocent FFI call was
>
> apifstat: afileNo statBuffer: buffer
>        <cdecl: long 'fstat' (long MacStat* ) module: 'libc.dylib'>
>        ^self externalCallFailed
>
> Now it seems BEFORE Snow Leopard *somehow* the Squeak external
> primitive call api which is part of the VM to assist in loading
> plugins or finding function entry points in executable binaries or
> libraries would
> return the entry point for fstat by *somehow* finding libc.dylib which
> is via  /usr/lib/libc.dylib -> libSystem.dylib ->  libSystem.B.dylib
>
> Interestingly this is found implicitly since the mac carbon vm lookup
> logic never looks in /usr/lib/
>
> Now in Snow Leopard it seems they do less implicit searching, perhaps
> a security concern?  Then it fails.
>
> In the past the Unix VM would search a hundred(sic) more places, but I
> understand less today, but it might still work? Maybe /usr/lib is in
> the hit list?
>
> This issue also applies to Alien because it calls back to the VM for
> it to find the module and function entry point.
>
> Since I have received a request to ensure that doing an explicit full
> path should be allowed in the squeak carbon macintosh vm.
> Then a change would be:
>
> apifstat: afileNo statBuffer: buffer
>        <cdecl: long 'fstat' (long MacStat* ) module: '/usr/lib/libc.dylib'>
>        ^self externalCallFailed
>
> I'll look at doing this shortly.
>
> However the workaround is to go to Sophie.app/Contents/Resources
> and do a ln -s /usr/lib/libc.dylib libc.dylib
> to create a symbolic link in the Resources directory for libc.dylib
> since we do explicitly look for libc.dylib in the Resources directory.
>
> Obviously if you have a FFI call like above where you are calling a
> system library Oracle? for example.
> You might find it no longer works with Snow Leopard on the Macintosh,
> so you should check.
>
>
> --
> =
> =
> =
> ========================================================================
> John M. McIntosh <[hidden email]>   Twitter:
> squeaker68882
> Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
> =
> =
> =
> ========================================================================
>
>
>
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: [Pharo-project] Alien & Squeak FFI issues on Snow Leopard.

Eliot Miranda-2
Craig Latta found a simple work-around which is to set the VM's Info.plist variable "SqueakPluginsBuiltInOrLocalOnly" to false, so that libraries are found on Snow Leopard.  Up until now things have been being found through vagaries if the Mac OS X dlopen call which would, I think, search the paths in an executable used to find dlls.  i.e. prior to Snow Leopard if you tried to run something in libSystem.B.dylib and you traced the logic in sqMacUnixExternalPrims.c you'd find that dlopen was "finding" libSystem.B.dylib in VM.app/Contents/Resources, even though it is not there.

Personally I think that sqMacUnixExternalPrims.c is ripe for a rewrite.  It is torturously complex.  But Craig's workaround works for us and takes the pressure off.

P.S.  I'm not on the Pharo list.  Can someone forward this for me?

On Wed, Sep 30, 2009 at 12:52 AM, Andrey Larionov <[hidden email]> wrote:
I think it's related http://forums.newspeaklanguage.org/index.php?topic=88.0

On Wed, Sep 30, 2009 at 06:13, John M McIntosh
<[hidden email]> wrote:
> Someone finally managed to fire up a Sophie app http://www.opensophie.org/
>  on Snow Leopard and immediately found an issue that also affects
> Squeak users who do fascinating things with FFI
> The innocent FFI call was
>
> apifstat: afileNo statBuffer: buffer
>        <cdecl: long 'fstat' (long MacStat* ) module: 'libc.dylib'>
>        ^self externalCallFailed
>
> Now it seems BEFORE Snow Leopard *somehow* the Squeak external
> primitive call api which is part of the VM to assist in loading
> plugins or finding function entry points in executable binaries or
> libraries would
> return the entry point for fstat by *somehow* finding libc.dylib which
> is via  /usr/lib/libc.dylib -> libSystem.dylib ->  libSystem.B.dylib
>
> Interestingly this is found implicitly since the mac carbon vm lookup
> logic never looks in /usr/lib/
>
> Now in Snow Leopard it seems they do less implicit searching, perhaps
> a security concern?  Then it fails.
>
> In the past the Unix VM would search a hundred(sic) more places, but I
> understand less today, but it might still work? Maybe /usr/lib is in
> the hit list?
>
> This issue also applies to Alien because it calls back to the VM for
> it to find the module and function entry point.
>
> Since I have received a request to ensure that doing an explicit full
> path should be allowed in the squeak carbon macintosh vm.
> Then a change would be:
>
> apifstat: afileNo statBuffer: buffer
>        <cdecl: long 'fstat' (long MacStat* ) module: '/usr/lib/libc.dylib'>
>        ^self externalCallFailed
>
> I'll look at doing this shortly.
>
> However the workaround is to go to Sophie.app/Contents/Resources
> and do a ln -s /usr/lib/libc.dylib libc.dylib
> to create a symbolic link in the Resources directory for libc.dylib
> since we do explicitly look for libc.dylib in the Resources directory.
>
> Obviously if you have a FFI call like above where you are calling a
> system library Oracle? for example.
> You might find it no longer works with Snow Leopard on the Macintosh,
> so you should check.
>
>
> --
> =
> =
> =
> ========================================================================
> John M. McIntosh <[hidden email]>   Twitter:
> squeaker68882
> Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
> =
> =
> =
> ========================================================================
>
>
>
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>




Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: [Pharo-project] Alien & Squeak FFI issues on Snow Leopard.

johnmci
The reason for the SqueakPluginsBuiltInOrLocalOnly was to prevent the  
VM from looking in about 100 places for a plugin before deciding to  
use the one built into the VM
this saved several 10 of ms in launch time. At the time there was also  
a transition period between PowerPC and Universal plugins, the  
historical algorithm would try to load powerpc
plugins lying loose in a developer's various libraries, versus using  
the newly complied ones supplied by the VM.

So setting SqueakPluginsBuiltInOrLocalOnly to NO might lead to other  
interesting issues, versus for example doing a 'ln -s /usr/lib/
libc.dylib   libc.dylib  in the VM's resource folder.

I will look at changing sqMacUnixExternalPrims.c to allow absolute  
paths, and also consider the changes Ian made lately to reduce the  
amount of searching that the original UnixExternalPrims.c did.

On 2009-09-30, at 9:04 AM, Eliot Miranda wrote:

> Craig Latta found a simple work-around which is to set the VM's  
> Info.plist variable "SqueakPluginsBuiltInOrLocalOnly" to false, so  
> that libraries are found on Snow Leopard.  Up until now things have  
> been being found through vagaries if the Mac OS X dlopen call which  
> would, I think, search the paths in an executable used to find  
> dlls.  i.e. prior to Snow Leopard if you tried to run something in  
> libSystem.B.dylib and you traced the logic in  
> sqMacUnixExternalPrims.c you'd find that dlopen was "finding"  
> libSystem.B.dylib in VM.app/Contents/Resources, even though it is  
> not there.
>
> Personally I think that sqMacUnixExternalPrims.c is ripe for a  
> rewrite.  It is torturously complex.  But Craig's workaround works  
> for us and takes the pressure off.
>
> P.S.  I'm not on the Pharo list.  Can someone forward this for me?
--
=
=
=
========================================================================
John M. McIntosh <[hidden email]>   Twitter:  
squeaker68882
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
=
=
=
========================================================================





Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: [Pharo-project] Alien & Squeak FFI issues on Snow Leopard.

Sean P. DeNigris
Administrator
What's the status of this issue?

Also, is there an FAQ someplace where this could go?  I was scratching my head because many of ffi's examples didn't work until I found this post.

Thanks.
Sean
Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: [Pharo-project] Alien & Squeak FFI issues on Snow Leopard.

Sean P. DeNigris
Administrator
Oops, forgot one relevant thing I found...

With the current implementation, module names must be fully written out, but with the SqueakPluginsBuiltInOrLocalOnly fix, you can abbreviate

e.g.
libSystem.B.dylib
libSystem.dylib
System.dylib
System

all work.

Maybe everyone already knew that, but as you can see, I did a little science experiment, lol

Sean
Cheers,
Sean