FFI and mpfr calls in squeak

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

FFI and mpfr calls in squeak

LawsonEnglish
So I tested a simple mandelbrot set rendering algorithm using the ArbitraryPrecisionFloat package and it works just fine, but terribly slowly.

So I had the idea of using mpfr via FFI and find that I can’t even begin.

I’m getting the dreaded External module not found error and just can’t get rid of it.

I thought I was taking the easy route and simply copying the mpfr library into the Squeak all-in-one Resources directory but no matter what I try, that error pops up.


initPrecision: aDefaultPrecision
        “initialize default precision"

<cdecl: void 'mpfr_init' (long ) module:'libmpfr.6.dylib'>
        ^self externalCallFailed

This is roughly the “hello world” of using mpfr, I thought, but can’t even do that.

Now, waiting in the wings is a bigger problem, I think: the mpfr library uses the gmp library. So how does one do THAT with squeak? Compile both libraries into a single target? (yikes!).

That might be an issue later on, but the impression I’m getting is that squeak simply can’t find libmpfr.6.dylib, so errors that result from calling a second library from the first haven’t even popped up yet.


Squeak5.3-19435-64bit, Mac OS X 10.15.6


Thanks


Lawson

Reply | Threaded
Open this post in threaded view
|

Re: FFI and mpfr calls in squeak

LawsonEnglish
That otool command told me what was wrong:

libmpfr.6.dylib:
/opt/local/lib/libmpfr.6.dylib (compatibility version 7.0.0, current version 7.2.0)
/opt/local/lib/libgmp.10.dylib (compatibility version 15.0.0, current version 15.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)

Macports hardcodes the directory that the lib will run in and the directory that it looks for the external dylib in.

I might be able to coerce Squeak to execute libmpfr.6.dylib to execute in the Resources directory, but the external lib MUST be in the original directory anyway.

My xcode-fu isn’t at the level (at all) to grab the source and compile it with options that redirect everything to work in the Resources directory, so’ll I need to use Craig Latta’s solution of pointing the directory path to the right library directory rather than the easier thing of copying the libraries into the Resources directory.


L

On Aug 13, 2020, at 4:35 AM, Tobias Pape <[hidden email]> wrote:

Hi

On 13.08.2020, at 13:21, Marcel Taeumel <[hidden email]> wrote:

Hi Lawson.

I think: the mpfr library uses the gmp library. So how does one do THAT with squeak?

You don't. Your operating system does that for you. Well, the same lookup rules for that dependent library apply, I suppose. Do you have the gmp library installed in your system?

Yes, not having all dependent libraries does raise that "external module not found" error, too. It can be confusing.

Working with FFI requires knowledge about working with shared libraries in your operating system. It's one of those "leaky abstractions" where Squeak is -- understandably -- not able to hold your hand for all kinds of configurations out there. :-)

You can try using a debug-build VM, which produces a more descriptive error log.

also, try 'otool -L ' on the library you are trying to load.


Best,
Marcel
Am 13.08.2020 12:11:53 schrieb LawsonEnglish <[hidden email]>:

So I tested a simple mandelbrot set rendering algorithm using the ArbitraryPrecisionFloat package and it works just fine, but terribly slowly.

So I had the idea of using mpfr via FFI and find that I can’t even begin.

How did you install mpfr?


I’m getting the dreaded External module not found error and just can’t get rid of it.

I thought I was taking the easy route and simply copying the mpfr library into the Squeak all-in-one Resources directory but no matter what I try, that error pops up.


initPrecision: aDefaultPrecision
“initialize default precision"


^self externalCallFailed

This is roughly the “hello world” of using mpfr, I thought, but can’t even do that.

Now, waiting in the wings is a bigger problem, I think: the mpfr library uses the gmp library. So how does one do THAT with squeak? Compile both libraries into a single target? (yikes!).

That might be an issue later on, but the impression I’m getting is that squeak simply can’t find libmpfr.6.dylib, so errors that result from calling a second library from the first haven’t even popped up yet.


Squeak5.3-19435-64bit, Mac OS X 10.15.6

Note that MacOS changes rapidly these days and we might simply have not catched up with the newest nonsense of Apple . . .

Best regards
-Tobias





Thanks


Lawson








Reply | Threaded
Open this post in threaded view
|

Re: FFI and mpfr calls in squeak

Tobias Pape
In reply to this post by LawsonEnglish
Hi


> On 13.08.2020, at 23:15, LawsonEnglish <[hidden email]> wrote:
>
> Yes, thanks. I’ve created a very simple test app using mpfr. However, in xcode, at least, I have to explicitly link in both libgmp.dylib and libmpfr.dylib , which are both aliases pointing to the actual libriaries containing the code.
>
> ALL the libraries of all types created by the MacPort and put into the macport directory have been copied into the resources via duplication rather than simply moving aliases around.
>
> I first referred to “libmpfr.dylib” — which is an aliias —  but starting using libmpfr.6.dylib (what the alias points to) on the outside chance that it was the alias causing problems.
>
> I don’t  see how the references to code in the gmp dylib  found in the mpfr dylib are going to automatically resolve if xcode requires one to explicitly link both dylibs in order for something to compile and run.
>

Linking is different from runtime-loading. If you show me the test-app code, I certainly can give you the way FFI would do things and show why things are as they are :)

> .
>
> My next test is to compile some absolutely trivial library without external references and see if I can get that to work.

That's a good start in any case :)

Best regards
        -Tobias

>
> Thanks for your input.
>
> L
>
>
>
>> On Aug 13, 2020, at 4:21 AM, Marcel Taeumel <[hidden email]> wrote:
>>
>> Hi Lawson.
>>
>>> I think: the mpfr library uses the gmp library. So how does one do THAT with squeak?
>>
>> You don't. Your operating system does that for you. Well, the same lookup rules for that dependent library apply, I suppose. Do you have the gmp library installed in your system?
>>
>> Yes, not having all dependent libraries does raise that "external module not found" error, too. It can be confusing.
>>
>> Working with FFI requires knowledge about working with shared libraries in your operating system. It's one of those "leaky abstractions" where Squeak is -- understandably -- not able to hold your hand for all kinds of configurations out there. :-)
>>
>> You can try using a debug-build VM, which produces a more descriptive error log.
>>
>> Best,
>> Marcel
>>> Am 13.08.2020 12:11:53 schrieb LawsonEnglish <[hidden email]>:
>>>
>>> So I tested a simple mandelbrot set rendering algorithm using the ArbitraryPrecisionFloat package and it works just fine, but terribly slowly.
>>>
>>> So I had the idea of using mpfr via FFI and find that I can’t even begin.
>>>
>>> I’m getting the dreaded External module not found error and just can’t get rid of it.
>>>
>>> I thought I was taking the easy route and simply copying the mpfr library into the Squeak all-in-one Resources directory but no matter what I try, that error pops up.
>>>
>>>
>>> initPrecision: aDefaultPrecision
>>> “initialize default precision"
>>>
>>>
>>> ^self externalCallFailed
>>>
>>> This is roughly the “hello world” of using mpfr, I thought, but can’t even do that.
>>>
>>> Now, waiting in the wings is a bigger problem, I think: the mpfr library uses the gmp library. So how does one do THAT with squeak? Compile both libraries into a single target? (yikes!).
>>>
>>> That might be an issue later on, but the impression I’m getting is that squeak simply can’t find libmpfr.6.dylib, so errors that result from calling a second library from the first haven’t even popped up yet.
>>>
>>>
>>> Squeak5.3-19435-64bit, Mac OS X 10.15.6
>>>
>>>
>>> Thanks
>>>
>>>
>>> Lawson
>>>
>>
>
>



Reply | Threaded
Open this post in threaded view
|

Re: FFI and mpfr calls in squeak

Tobias Pape
In reply to this post by LawsonEnglish

> On 14.08.2020, at 00:19, LawsonEnglish <[hidden email]> wrote:
>
> That otool command told me what was wrong:
>
> libmpfr.6.dylib:
> /opt/local/lib/libmpfr.6.dylib (compatibility version 7.0.0, current version 7.2.0)
> /opt/local/lib/libgmp.10.dylib (compatibility version 15.0.0, current version 15.0.0)
> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
>
> Macports hardcodes the directory that the lib will run in and the directory that it looks for the external dylib in.
>
> I might be able to coerce Squeak to execute libmpfr.6.dylib to execute in the Resources directory, but the external lib MUST be in the original directory anyway.
>
> My xcode-fu isn’t at the level (at all) to grab the source and compile it with options that redirect everything to work in the Resources directory, so’ll I need to use Craig Latta’s solution of pointing the directory path to the right library directory rather than the easier thing of copying the libraries into the Resources directory.

You can start the Squeak binary  with DYLD_LIBRARY_PATH="/opt/local/lib/" and see if that works. (Caveat Lector: 10.15 tightened the thumb screws, so that _might_ fail)

Also, a typical deployment thing is indeed:
 - copy all the necessary dylibs to Resources
 - use install_name_tool to change the respective names and such to @rpath names

Please read the manpage for dyld(1), especially the end for @executable_path/@loader_path/@rpath :)
This will help to understand why the dynamic loading might need these names that are shown with otool

Best regards
        -Tobias




>
>
> L
>
>> On Aug 13, 2020, at 4:35 AM, Tobias Pape <[hidden email]> wrote:
>>
>> Hi
>>
>>> On 13.08.2020, at 13:21, Marcel Taeumel <[hidden email]> wrote:
>>>
>>> Hi Lawson.
>>>
>>>> I think: the mpfr library uses the gmp library. So how does one do THAT with squeak?
>>>
>>> You don't. Your operating system does that for you. Well, the same lookup rules for that dependent library apply, I suppose. Do you have the gmp library installed in your system?
>>>
>>> Yes, not having all dependent libraries does raise that "external module not found" error, too. It can be confusing.
>>>
>>> Working with FFI requires knowledge about working with shared libraries in your operating system. It's one of those "leaky abstractions" where Squeak is -- understandably -- not able to hold your hand for all kinds of configurations out there. :-)
>>>
>>> You can try using a debug-build VM, which produces a more descriptive error log.
>>
>> also, try 'otool -L ' on the library you are trying to load.
>>
>>>
>>> Best,
>>> Marcel
>>>> Am 13.08.2020 12:11:53 schrieb LawsonEnglish <[hidden email]>:
>>>>
>>>> So I tested a simple mandelbrot set rendering algorithm using the ArbitraryPrecisionFloat package and it works just fine, but terribly slowly.
>>>>
>>>> So I had the idea of using mpfr via FFI and find that I can’t even begin.
>>
>> How did you install mpfr?
>>
>>>>
>>>> I’m getting the dreaded External module not found error and just can’t get rid of it.
>>>>
>>>> I thought I was taking the easy route and simply copying the mpfr library into the Squeak all-in-one Resources directory but no matter what I try, that error pops up.
>>>>
>>>>
>>>> initPrecision: aDefaultPrecision
>>>> “initialize default precision"
>>>>
>>>>
>>>> ^self externalCallFailed
>>>>
>>>> This is roughly the “hello world” of using mpfr, I thought, but can’t even do that.
>>>>
>>>> Now, waiting in the wings is a bigger problem, I think: the mpfr library uses the gmp library. So how does one do THAT with squeak? Compile both libraries into a single target? (yikes!).
>>>>
>>>> That might be an issue later on, but the impression I’m getting is that squeak simply can’t find libmpfr.6.dylib, so errors that result from calling a second library from the first haven’t even popped up yet.
>>>>
>>>>
>>>> Squeak5.3-19435-64bit, Mac OS X 10.15.6
>>
>> Note that MacOS changes rapidly these days and we might simply have not catched up with the newest nonsense of Apple . . .
>>
>> Best regards
>> -Tobias
>>
>>
>>
>>>>
>>>>
>>>> Thanks
>>>>
>>>>
>>>> Lawson
>>>>
>>>
>>
>>
>>
>
>