How to set library path for UFFI on Linux?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

How to set library path for UFFI on Linux?

Dan Wilczak
I've just installed the 64-bit version of Pharo 6.1 on OpenSuse linux (64
bit). The Athens demos fail because
UFFI looks for libcairo in /usr/lib rather the /usr/lib64. I tried setting
LD_LIBRARY_PATH but that has no effect.
How can I force it to use lib64?

Dan




--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Reply | Threaded
Open this post in threaded view
|

Re: How to set library path for UFFI on Linux?

Stephane Ducasse-3
Hi dan

We are sorry that you encounter a problem.

Not sure that this is the solution but check

ffiLibraryName
^ CairoLibrary


CairoLibrary >> unix32ModuleName
"On different flavors of linux the path to library may differ
depending on OS distro or whether system is 32 or 64 bit.
Packages such as snapcraft can also change the path.
Look in a number of known locations, or LD_LIBRARY_PATH."

#('/usr/lib/i386-linux-gnu' '/usr/lib32' '/usr/lib')
, (((OSEnvironment current getEnv: 'LD_LIBRARY_PATH') ifNil: [ '' ])
substrings: ':')
do: [ :path |
| libraryPath |
libraryPath := path asFileReference / 'libcairo.so.2'.
libraryPath exists
ifTrue: [ ^ libraryPath fullName ] ].
self error: 'Cannot locate cairo library. Please check if it installed
on your system'

Better


CairoLibrary >> unix64ModuleName
"On different flavors of linux the path to library may differ
depending on OS distro or whether system is 32 or 64 bit.
Packages such as snapcraft can also change the path.
Look in a number of known locations, or LD_LIBRARY_PATH."

#('/lib/x86_64-linux-gnu' '/usr/lib')
, (((OSEnvironment current getEnv: 'LD_LIBRARY_PATH') ifNil: [ '' ])
substrings: ':')
do: [ :path |
| libraryPath |
libraryPath := path asFileReference / 'libcairo.so.2'.
libraryPath exists
ifTrue: [ ^ libraryPath fullName ] ].
self error: 'Cannot locate cairo library. Please check if it installed
on your system'

On Fri, Sep 22, 2017 at 6:37 PM, Dan Wilczak <[hidden email]> wrote:

> I've just installed the 64-bit version of Pharo 6.1 on OpenSuse linux (64
> bit). The Athens demos fail because
> UFFI looks for libcairo in /usr/lib rather the /usr/lib64. I tried setting
> LD_LIBRARY_PATH but that has no effect.
> How can I force it to use lib64?
>
> Dan
>
>
>
>
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>

Reply | Threaded
Open this post in threaded view
|

Re: How to set library path for UFFI on Linux?

Dan Wilczak
That pointed me in the right direction, thanks. I've got it working now.

I think there's a bug there which may bite other people ---

If you have both 32-bit and 64-bit versions of the library installed,
Unix64ModuleName can find the
32-bit version first (depending on your LD_LIBRARY_PATH) and set that as the
lib to use. Then when
UFFI loads it, it recognizes it as 32-bit and gives up rather than looking
in any more directories.

Dan



--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Reply | Threaded
Open this post in threaded view
|

Re: How to set library path for UFFI on Linux?

Stephane Ducasse-3
Can you open a bug entry and propose your solution?

Stef

On Fri, Sep 22, 2017 at 9:56 PM, Dan Wilczak <[hidden email]> wrote:

> That pointed me in the right direction, thanks. I've got it working now.
>
> I think there's a bug there which may bite other people ---
>
> If you have both 32-bit and 64-bit versions of the library installed,
> Unix64ModuleName can find the
> 32-bit version first (depending on your LD_LIBRARY_PATH) and set that as the
> lib to use. Then when
> UFFI loads it, it recognizes it as 32-bit and gives up rather than looking
> in any more directories.
>
> Dan
>
>
>
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>

Reply | Threaded
Open this post in threaded view
|

Re: How to set library path for UFFI on Linux?

EstebanLM
hi,

this is not a bug, UFFI cannot know where a library is (because it can be anywhere).
but, not being a bug, this remains a problem :)
One solution is to search on LD_LIBRARY_PATH but this is not generalised (it should) for the moment. And of course, this is a solution that works for linux and not other platforms, so not perfect. Also we need to consider architecture, etc. etc. etc.

Esteban


> On 23 Sep 2017, at 09:16, Stephane Ducasse <[hidden email]> wrote:
>
> Can you open a bug entry and propose your solution?
>
> Stef
>
> On Fri, Sep 22, 2017 at 9:56 PM, Dan Wilczak <[hidden email]> wrote:
>> That pointed me in the right direction, thanks. I've got it working now.
>>
>> I think there's a bug there which may bite other people ---
>>
>> If you have both 32-bit and 64-bit versions of the library installed,
>> Unix64ModuleName can find the
>> 32-bit version first (depending on your LD_LIBRARY_PATH) and set that as the
>> lib to use. Then when
>> UFFI loads it, it recognizes it as 32-bit and gives up rather than looking
>> in any more directories.
>>
>> Dan
>>
>>
>>
>> --
>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>>
>


Reply | Threaded
Open this post in threaded view
|

Re: How to set library path for UFFI on Linux?

HilaireFernandes
What about shipping the dedicated libcairo with the Linux Virtual Image?

Hilaire


Le 23/09/2017 à 10:03, Esteban Lorenzano a écrit :
> hi,
>
> this is not a bug, UFFI cannot know where a library is (because it can be anywhere).
> but, not being a bug, this remains a problem:)
> One solution is to search on LD_LIBRARY_PATH but this is not generalised (it should) for the moment. And of course, this is a solution that works for linux and not other platforms, so not perfect. Also we need to consider architecture, etc. etc. etc.
>
> Esteban

--
Dr. Geo
http://drgeo.eu



Reply | Threaded
Open this post in threaded view
|

Re: How to set library path for UFFI on Linux?

Dan Wilczak
In reply to this post by EstebanLM
It does search on LD_LIBRARY_PATH now, it just fails because it finds the 32-bit libcairo before the 64-bit one.

The traceback displayed inside Pharo says "Error: External module not found" which is quite misleading.  A minimal fix would be to just provide a better error message.

There is a traceback that prints to the console if you launched pharo from it, and it does reveal the true failure, but only to someone who knows what ELFCLASS32 means:

ioLoadModule(//usr/lib/libcairo.so.2):
  //usr/lib/libcairo.so.2: wrong ELF class: ELFCLASS32
Error: External module not found

This shows that the information for a better error message is available.
 
A better fix would be to skip the 32-bit lib and keep searching. This looks like it would be messy because CairoLibrary provides the path but
​U​
FFI has to deal with it.
​ As far as I can see, there's no way to just check whether a library is 32- or 64-bit, other than trying to call something in it.

I'm a total Pharo novice (I wrote maybe 300 lines of Squeak code several years ago) but this design seems odd to me. Why is LD_LIBRARY_PATH being handled in​ Athens-Cairo rather than UFFI itself?

Dan






On Sat, Sep 23, 2017 at 4:03 AM, Esteban Lorenzano <[hidden email]> wrote:
hi,

this is not a bug, UFFI cannot know where a library is (because it can be anywhere).
but, not being a bug, this remains a problem :)
One solution is to search on LD_LIBRARY_PATH but this is not generalised (it should) for the moment. And of course, this is a solution that works for linux and not other platforms, so not perfect. Also we need to consider architecture, etc. etc. etc.

Esteban


> On 23 Sep 2017, at 09:16, Stephane Ducasse <[hidden email]> wrote:
>
> Can you open a bug entry and propose your solution?
>
> Stef
>
> On Fri, Sep 22, 2017 at 9:56 PM, Dan Wilczak <[hidden email]> wrote:
>> That pointed me in the right direction, thanks. I've got it working now.
>>
>> I think there's a bug there which may bite other people ---
>>
>> If you have both 32-bit and 64-bit versions of the library installed,
>> Unix64ModuleName can find the
>> 32-bit version first (depending on your LD_LIBRARY_PATH) and set that as the
>> lib to use. Then when
>> UFFI loads it, it recognizes it as 32-bit and gives up rather than looking
>> in any more directories.
>>
>> Dan
>>
>>
>>
>> --
>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>>
>



Reply | Threaded
Open this post in threaded view
|

Re: How to set library path for UFFI on Linux?

hernanmd
Hi Dan

2017-09-23 13:24 GMT-03:00 Dan Wilczak <[hidden email]>:
> It does search on LD_LIBRARY_PATH now, it just fails because it finds the
> 32-bit libcairo before the 64-bit one.
>
> The traceback displayed inside Pharo says "Error: External module not found"
> which is quite misleading.  A minimal fix would be to just provide a better
> error message.
>

Yes, better error messages saves a lot of time.
Have you opened an issue for this?

> There is a traceback that prints to the console if you launched pharo from
> it, and it does reveal the true failure, but only to someone who knows what
> ELFCLASS32 means:
>
> ioLoadModule(//usr/lib/libcairo.so.2):
>   //usr/lib/libcairo.so.2: wrong ELF class: ELFCLASS32
> Error: External module not found
>
> This shows that the information for a better error message is available.
>
> A better fix would be to skip the 32-bit lib and keep searching. This looks
> like it would be messy because CairoLibrary provides the path but
> U
> FFI has to deal with it.

You mean keep searching the whole file system?

> As far as I can see, there's no way to just check whether a library is 32-
> or 64-bit, other than trying to call something in it.
>

One can use a fork() or exec() with the "file" command in Linux based
OS, to find library type. It will answer something like:

ELF 32-bit LSB shared object, Intel80386, version 1, not stripped

or

ELF 64-bit LSB shared object, version 1, not stripped

> I'm a total Pharo novice (I wrote maybe 300 lines of Squeak code several
> years ago) but this design seems odd to me. Why is LD_LIBRARY_PATH being
> handled in Athens-Cairo rather than UFFI itself?
>
> Dan
>
>
>
>
>
>
> On Sat, Sep 23, 2017 at 4:03 AM, Esteban Lorenzano <[hidden email]>
> wrote:
>>
>> hi,
>>
>> this is not a bug, UFFI cannot know where a library is (because it can be
>> anywhere).
>> but, not being a bug, this remains a problem :)
>> One solution is to search on LD_LIBRARY_PATH but this is not generalised
>> (it should) for the moment. And of course, this is a solution that works for
>> linux and not other platforms, so not perfect. Also we need to consider
>> architecture, etc. etc. etc.
>>
>> Esteban
>>
>>
>> > On 23 Sep 2017, at 09:16, Stephane Ducasse <[hidden email]>
>> > wrote:
>> >
>> > Can you open a bug entry and propose your solution?
>> >
>> > Stef
>> >
>> > On Fri, Sep 22, 2017 at 9:56 PM, Dan Wilczak <[hidden email]>
>> > wrote:
>> >> That pointed me in the right direction, thanks. I've got it working
>> >> now.
>> >>
>> >> I think there's a bug there which may bite other people ---
>> >>
>> >> If you have both 32-bit and 64-bit versions of the library installed,
>> >> Unix64ModuleName can find the
>> >> 32-bit version first (depending on your LD_LIBRARY_PATH) and set that
>> >> as the
>> >> lib to use. Then when
>> >> UFFI loads it, it recognizes it as 32-bit and gives up rather than
>> >> looking
>> >> in any more directories.
>> >>
>> >> Dan
>> >>
>> >>
>> >>
>> >> --
>> >> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>> >>
>> >
>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: How to set library path for UFFI on Linux?

Dan Wilczak
Hernan -

I haven't opened an issue - how do I do it? (I'm very new to Pharo.)

About continuing the search - I only mean continuing the search of the
LD_LIBRARY_PATH directories, not the whole filesystem. Two changes would be
needed to accomplish this:

1) Athens-Cairo >> CairoLibrary would have to return a list of paths to all
the matching libraries rather than just the first one that it finds. This
part seems easy.

2) UFFI Libraries UnixDynamicLoader >> loadLibrary:flag: would have to take
the list of paths (rather than just one), check them for being 32-bit or
64-bit, and load the first correct one. I can't find any way to perform that
check in Pharo directly. How would you fork or exec the "file" command from
inside Pharo?

Dan
 



--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Reply | Threaded
Open this post in threaded view
|

Re: How to set library path for UFFI on Linux?

Stephane Ducasse-3
https://pharo.fogbugz.com

On Sat, Sep 30, 2017 at 6:53 PM, Dan Wilczak <[hidden email]> wrote:

> Hernan -
>
> I haven't opened an issue - how do I do it? (I'm very new to Pharo.)
>
> About continuing the search - I only mean continuing the search of the
> LD_LIBRARY_PATH directories, not the whole filesystem. Two changes would be
> needed to accomplish this:
>
> 1) Athens-Cairo >> CairoLibrary would have to return a list of paths to all
> the matching libraries rather than just the first one that it finds. This
> part seems easy.
>
> 2) UFFI Libraries UnixDynamicLoader >> loadLibrary:flag: would have to take
> the list of paths (rather than just one), check them for being 32-bit or
> 64-bit, and load the first correct one. I can't find any way to perform that
> check in Pharo directly. How would you fork or exec the "file" command from
> inside Pharo?
>
> Dan
>
>
>
>
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>

Reply | Threaded
Open this post in threaded view
|

Re: How to set library path for UFFI on Linux?

Herby Vojčík
Stephane Ducasse wrote:

Would you pls care to look at the sibling thread "How to make pharo find
SQLite", it seems that it struggles with similar problem, is it supposed
to created symlinks on linux, or it should be able to fund the library
but somehow fails?

Thanks, Herby

> https://pharo.fogbugz.com
>
> On Sat, Sep 30, 2017 at 6:53 PM, Dan Wilczak<[hidden email]>  wrote:
>> Hernan -
>>
>> I haven't opened an issue - how do I do it? (I'm very new to Pharo.)
>>
>> About continuing the search - I only mean continuing the search of the
>> LD_LIBRARY_PATH directories, not the whole filesystem. Two changes would be
>> needed to accomplish this:
>>
>> 1) Athens-Cairo>>  CairoLibrary would have to return a list of paths to all
>> the matching libraries rather than just the first one that it finds. This
>> part seems easy.
>>
>> 2) UFFI Libraries UnixDynamicLoader>>  loadLibrary:flag: would have to take
>> the list of paths (rather than just one), check them for being 32-bit or
>> 64-bit, and load the first correct one. I can't find any way to perform that
>> check in Pharo directly. How would you fork or exec the "file" command from
>> inside Pharo?
>>
>> Dan
>>
>>
>>
>>
>> --
>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>>
>


Reply | Threaded
Open this post in threaded view
|

Re: How to set library path for UFFI on Linux?

Herby Vojčík
In reply to this post by Dan Wilczak
Dan Wilczak wrote:
> Hernan -
>
> I haven't opened an issue - how do I do it? (I'm very new to Pharo.)
>
> About continuing the search - I only mean continuing the search of the
> LD_LIBRARY_PATH directories, not the whole filesystem. Two changes would be
> needed to accomplish this:

FWIW, I had problem with loading 'sqlite3' module. Crossposting the
solution:

It seems FFI for some reason struggles with 'lib' and/or '.so.0' things
in linux (even if LD_LIBRARY_PATH is properly set).

I had to do this:

TARGETDIR=`find . -type f -name SqueakSSL.so -print0 | xargs -0 dirname`
ln -s `/sbin/ldconfig -p | sed -e 's|[^/]*||' | grep sqlite3`
${TARGETDIR}/sqlite3.so

(so, link in plugin directory, and the name is plain 'sqlite3.so'). With
that, things work. Maybe, libsqlite.so would do the trick as well, but I
got no nerve to play with it more.

But, frankly, do not tell me this is what ppl need to do to load
external libs in linux. :-/

Herby

>
> 1) Athens-Cairo>>  CairoLibrary would have to return a list of paths to all
> the matching libraries rather than just the first one that it finds. This
> part seems easy.
>
> 2) UFFI Libraries UnixDynamicLoader>>  loadLibrary:flag: would have to take
> the list of paths (rather than just one), check them for being 32-bit or
> 64-bit, and load the first correct one. I can't find any way to perform that
> check in Pharo directly. How would you fork or exec the "file" command from
> inside Pharo?
>
> Dan
>
>
>
>
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>


Reply | Threaded
Open this post in threaded view
|

Re: How to set library path for UFFI on Linux?

philippeback
Usually there are a number of names to try but on Linux, there are less users and more distros, so people are not adding more libs to try.

So, yes, symlink in the plugin directory works usually.

Alternative would be to use LD_LIBRARY_PATH or pkg_config support to do this cleanly (not done at this point).

Phil

On Sat, Sep 30, 2017 at 11:19 PM, Herby Vojčík <[hidden email]> wrote:
Dan Wilczak wrote:
Hernan -

I haven't opened an issue - how do I do it? (I'm very new to Pharo.)

About continuing the search - I only mean continuing the search of the
LD_LIBRARY_PATH directories, not the whole filesystem. Two changes would be
needed to accomplish this:

FWIW, I had problem with loading 'sqlite3' module. Crossposting the solution:

It seems FFI for some reason struggles with 'lib' and/or '.so.0' things in linux (even if LD_LIBRARY_PATH is properly set).

I had to do this:

TARGETDIR=`find . -type f -name SqueakSSL.so -print0 | xargs -0 dirname`
ln -s `/sbin/ldconfig -p | sed -e 's|[^/]*||' | grep sqlite3` ${TARGETDIR}/sqlite3.so

(so, link in plugin directory, and the name is plain 'sqlite3.so'). With that, things work. Maybe, libsqlite.so would do the trick as well, but I got no nerve to play with it more.

But, frankly, do not tell me this is what ppl need to do to load external libs in linux. :-/

Herby


1) Athens-Cairo>>  CairoLibrary would have to return a list of paths to all
the matching libraries rather than just the first one that it finds. This
part seems easy.

2) UFFI Libraries UnixDynamicLoader>>  loadLibrary:flag: would have to take
the list of paths (rather than just one), check them for being 32-bit or
64-bit, and load the first correct one. I can't find any way to perform that
check in Pharo directly. How would you fork or exec the "file" command from
inside Pharo?

Dan




--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html





Reply | Threaded
Open this post in threaded view
|

Re: How to set library path for UFFI on Linux?

Herby Vojčík


On October 1, 2017 12:33:04 AM GMT+02:00, "[hidden email]" <[hidden email]> wrote:
>Usually there are a number of names to try but on Linux, there are less
>users and more distros, so people are not adding more libs to try.
>
>So, yes, symlink in the plugin directory works usually.
>
>Alternative would be to use LD_LIBRARY_PATH or pkg_config support to do
>this cleanly (not done at this point).

I did use LD_LIBRARY_PATH. But it did not work (it looks like Pharo vm us using it, as it is used and decorated in pharo runner script). Even link in the plugin dir did not work unless I specified simplified name. The library is libsqlite3.so.0. Using that name, it wasn't found ('lib' is probably tried, so it is likely '.0' part that somehow blocked Pharo).

Shouldn't this actually be solved somehow? So linux vm loads libs using linux conventions?

Or is it preinstalled wrongly in Ubuntu images on Digital Ocean when it does not have symlink named libsqlite3.so ?

Herby

>Phil
>
>On Sat, Sep 30, 2017 at 11:19 PM, Herby Vojčík <[hidden email]>
>wrote:
>
>> Dan Wilczak wrote:
>>
>>> Hernan -
>>>
>>> I haven't opened an issue - how do I do it? (I'm very new to Pharo.)
>>>
>>> About continuing the search - I only mean continuing the search of
>the
>>> LD_LIBRARY_PATH directories, not the whole filesystem. Two changes
>would
>>> be
>>> needed to accomplish this:
>>>
>>
>> FWIW, I had problem with loading 'sqlite3' module. Crossposting the
>> solution:
>>
>> It seems FFI for some reason struggles with 'lib' and/or '.so.0'
>things in
>> linux (even if LD_LIBRARY_PATH is properly set).
>>
>> I had to do this:
>>
>> TARGETDIR=`find . -type f -name SqueakSSL.so -print0 | xargs -0
>dirname`
>> ln -s `/sbin/ldconfig -p | sed -e 's|[^/]*||' | grep sqlite3`
>> ${TARGETDIR}/sqlite3.so
>>
>> (so, link in plugin directory, and the name is plain 'sqlite3.so').
>With
>> that, things work. Maybe, libsqlite.so would do the trick as well,
>but I
>> got no nerve to play with it more.
>>
>> But, frankly, do not tell me this is what ppl need to do to load
>external
>> libs in linux. :-/
>>
>> Herby
>>
>>
>>> 1) Athens-Cairo>>  CairoLibrary would have to return a list of paths
>to
>>> all
>>> the matching libraries rather than just the first one that it finds.
>This
>>> part seems easy.
>>>
>>> 2) UFFI Libraries UnixDynamicLoader>>  loadLibrary:flag: would have
>to
>>> take
>>> the list of paths (rather than just one), check them for being
>32-bit or
>>> 64-bit, and load the first correct one. I can't find any way to
>perform
>>> that
>>> check in Pharo directly. How would you fork or exec the "file"
>command
>>> from
>>> inside Pharo?
>>>
>>> Dan
>>>
>>>
>>>
>>>
>>> --
>>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>>>
>>>
>>
>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: How to set library path for UFFI on Linux?

Dan Wilczak
One obstacle to fixing these problems properly is that handling of library
search paths is not centralized, but it really should be. A Pharo package
for sqlite or cairo or whatever should provide only the library name; UFFI
or some associated package should deal with the search for it. Otherwise
there will just be more ad hoc code and fixes in the future.



--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html