Specifying library dependencies in UFFI

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Specifying library dependencies in UFFI

raffaello.giulietti
Hi,

say I want to directly use a library that depends on other libraries in
different folders. How does UFFI proceed to find these other libraries?

Is there a way to specify the search folders in UFFI?

I'm on Windows and in Pharo5/6.

Greetings
Raffaello

Reply | Threaded
Open this post in threaded view
|

Re: Specifying library dependencies in UFFI

philippeback

Yes there is. Search for the .so or .dll or .dylib strings with the finder and you will see a few and how they are found.

Also check https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgresshttps://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgress/lastSuccessfulBuild/artifact/book-result/UnifiedFFI/UnifiedFFI.html where there are examples like

Let's for example consider the library LibC already provided by uFFI:

FFILibrary subclass: #LibC
instanceVariableNames: ''
classVariableNames: ''
package: 'UnifiedFFI-Libraries'

LibC>>macModuleName
^ 'libc.dylib'

LibC>>unixModuleName
^ 'libc.so.6'

LibC>>win32ModuleName
"While this is not a 'libc' properly, msvcrt has the functions we are defining here"
^ 'msvcrt.dll'

This is for libraries that are in the path or library path.

One can come up with other strategies for loading these, with absolute path  like '/usr/lib/xxx.so'

Beware of the 32/64 bit thing< 32 bit pharo needs 32 bit lib.

I am using a 32 bits container on linux to build 32 bits libraries, much less headaches.


HTH


Phil
Reply | Threaded
Open this post in threaded view
|

Re: Specifying library dependencies in UFFI

Ben Coman
In reply to this post by raffaello.giulietti


On Tue, Mar 14, 2017 at 11:23 PM, Raffaello Giulietti <[hidden email]> wrote:
Hi,

say I want to directly use a library that depends on other libraries in different folders. How does UFFI proceed to find these other libraries?

Is there a way to specify the search folders in UFFI?

I'm on Windows and in Pharo5/6.

Greetings
Raffaello


I assume you mean specifying the search path outside of Pharo.
Perhaps call  SetDllDirectory  via Pharo FFI.
(YMMV disclaimer, my knowledge of this is a few minutes searching)

SetDllDirectory function

LoadLibrary function

Dynamic-Link Library Search Order

cheers -ben
Reply | Threaded
Open this post in threaded view
|

Re: Specifying library dependencies in UFFI

raffaello.giulietti
In reply to this post by philippeback
Hi Phil,

I'm familiar with all this, but it doesn't help.

Suppose my code directly accesses "my.dll" which, in turn, depends on
"her.dll" which my code does not need directly and suppose that library
is located in folder "<path-to-her-folder>". How can I enforce the UFFI
to perform the search there?



BTW, the URI you mention is not reachable. I usually use the following:

https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgress/lastSuccessfulBuild/artifact/book-result/UnifiedFFI/UnifiedFFI.pdf



On 2017-03-15 00:13, [hidden email] wrote:

>
> Yes there is. Search for the .so or .dll or .dylib strings with the
> finder and you will see a few and how they are found.
>
> Also check
> https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgresshttps://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgress/lastSuccessfulBuild/artifact/book-result/UnifiedFFI/UnifiedFFI.html
> where there are examples like
>
> Let's for example consider the library LibC already provided by uFFI:
>
> FFILibrary subclass: #LibC
> instanceVariableNames: ''
> classVariableNames: ''
> package: 'UnifiedFFI-Libraries'
>
> LibC>>macModuleName
> ^ 'libc.dylib'
>
> LibC>>unixModuleName
> ^ 'libc.so.6'
>
> LibC>>win32ModuleName
> "While this is not a 'libc' properly, msvcrt has the functions we are
> defining here"
> ^ 'msvcrt.dll'
>
> This is for libraries that are in the path or library path.
>
> One can come up with other strategies for loading these, with absolute
> path  like '/usr/lib/xxx.so'
>
> Beware of the 32/64 bit thing< 32 bit pharo needs 32 bit lib.
>
> I am using a 32 bits container on linux to build 32 bits libraries, much
> less headaches.
>
> See https://gist.github.com/philippeback/33717128d5fb24263e557922d5ac913e
>
> HTH
>
>
> Phil


Reply | Threaded
Open this post in threaded view
|

Re: Specifying library dependencies in UFFI

raffaello.giulietti
In reply to this post by Ben Coman
Hi Ben,

my understanding is that SetDllDirectory only affects the search path of
libraries that are loaded at *run-time* with LoadLibrary.

What I'm asking for is a mechanism that works at *load-time*, when a
library I'm accessing directly depends on another one which I'm not
targeting directly.

But I'll try whether SetDllDirectory also affects load-time searches.

Anyway, I hope future versions of the UFFI to offer setting the search
paths more easily.



Greetings
Raffaello



On 2017-03-15 07:22, Ben Coman wrote:

>
>
> On Tue, Mar 14, 2017 at 11:23 PM, Raffaello Giulietti
> <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi,
>
>     say I want to directly use a library that depends on other libraries
>     in different folders. How does UFFI proceed to find these other
>     libraries?
>
>     Is there a way to specify the search folders in UFFI?
>
>     I'm on Windows and in Pharo5/6.
>
>     Greetings
>     Raffaello
>
>
> I assume you mean specifying the search path outside of Pharo.
> Perhaps call  SetDllDirectory  via Pharo FFI.
> (YMMV disclaimer, my knowledge of this is a few minutes searching)
>
> SetDllDirectory function
> https://msdn.microsoft.com/en-us/library/windows/desktop/ms686203(v=vs.85).aspx
>
> LoadLibrary function
> https://msdn.microsoft.com/en-us/library/windows/desktop/ms684175(v=vs.85).aspx
>
> Dynamic-Link Library Search Order
> https://msdn.microsoft.com/en-us/library/windows/desktop/ms682586(v=vs.85).aspx
>
> cheers -ben


Reply | Threaded
Open this post in threaded view
|

Re: Specifying library dependencies in UFFI

Ben Coman


On Wed, Mar 15, 2017 at 4:42 PM, Raffaello Giulietti <[hidden email]> wrote:
Hi Ben,

my understanding is that SetDllDirectory only affects the search path of libraries that are loaded at *run-time* with LoadLibrary.

What I'm asking for is a mechanism that works at *load-time*, when a library I'm accessing directly depends on another one which I'm not targeting directly.

I don't quite follow.  I'm not intimate with Windows dynamic loading, so these questions are an opportunity for me to learn... 

You can't mean when Pharo loads, because it wouldn't know of OTHER.DLL ?
    
Do you mean that LoadLibrary is not called explicitly by YOUR.DLL before its uses the function from OTHER.DLL?  

Do you mean you link OTHER.DLL to YOUR.DLL at compile time?  
But still, YOUR.DLL it not loaded until called via Pharo FFI, so neither is OTHER.DLL,
and Dynamic-Link Library Search Order [1] says... "If a DLL with the same module name 
is already loaded in memory, the system uses the loaded DLL, no matter which directory it is in. 
The system does not search for the DLL."

So maybe before calling any of YOUR.DLL,
do a SetDllDirectory followed by a LoadLibrary(OTHER.DLL) 
and later when you call into YOUR.DLL, 
OTHER.DLL is already loaded.

Otherwise it would seems(?) your options are either Manifests
or Dynamic-Link Library Redirection, both done outside of Pharo.
 
 
cheers -ben


But I'll try whether SetDllDirectory also affects load-time searches.

Anyway, I hope future versions of the UFFI to offer setting the search paths more easily.



Greetings
Raffaello




On 2017-03-15 07:22, Ben Coman wrote:


On Tue, Mar 14, 2017 at 11:23 PM, Raffaello Giulietti
<[hidden email]
<mailto:[hidden email]>> wrote:

    Hi,

    say I want to directly use a library that depends on other libraries
    in different folders. How does UFFI proceed to find these other
    libraries?

    Is there a way to specify the search folders in UFFI?

    I'm on Windows and in Pharo5/6.

    Greetings
    Raffaello


I assume you mean specifying the search path outside of Pharo.
Perhaps call  SetDllDirectory  via Pharo FFI.
(YMMV disclaimer, my knowledge of this is a few minutes searching)

SetDllDirectory function
https://msdn.microsoft.com/en-us/library/windows/desktop/ms686203(v=vs.85).aspx

LoadLibrary function
https://msdn.microsoft.com/en-us/library/windows/desktop/ms684175(v=vs.85).aspx

Dynamic-Link Library Search Order
https://msdn.microsoft.com/en-us/library/windows/desktop/ms682586(v=vs.85).aspx

cheers -ben



Reply | Threaded
Open this post in threaded view
|

Re: Specifying library dependencies in UFFI

raffaello.giulietti
On 2017-03-15 13:51, Ben Coman wrote:

>
>
> On Wed, Mar 15, 2017 at 4:42 PM, Raffaello Giulietti
> <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi Ben,
>
>     my understanding is that SetDllDirectory only affects the search
>     path of libraries that are loaded at *run-time* with LoadLibrary.
>
>     What I'm asking for is a mechanism that works at *load-time*, when a
>     library I'm accessing directly depends on another one which I'm not
>     targeting directly.
>
>
> I don't quite follow.  I'm not intimate with Windows dynamic loading, so
> these questions are an opportunity for me to learn...
>
> You can't mean when Pharo loads, because it wouldn't know of OTHER.DLL ?
>
> Do you mean that LoadLibrary is not called explicitly by YOUR.DLL before
> its uses the function from OTHER.DLL?
>

During the build of YOUR.DLL, you can specify that it should depend on
OTHER.LIB (kind of binary which lists function headers and static data
declarations for OTHER.DLL). Linking resolution is then done at
load-time of YOUR.DLL, not by invoking LoadLibrary("OTHER.DLL").

When YOUR.DLL is loaded by UFFI at run-time (here by means of
LoadLibrary(), I guess), Windows (not the UFFI) also attempts to load
OTHER.DLL because of the linking information gathered during the build.
Notice that the linking information does *not* include a specific path
for OTHER.DLL.

(You don't have to mention standard *.LIBs like Kernel32.lib because
they are included by default. But of course, Windows knows where to find
the corresponding .dll.)

OTHER.DLL is searched using the strategy described in the MSDN page you
mention. My expectation is/was that UFFI has/had an API for specifying
the search paths where to find OTHER.DLL in a platform-independent way.
In other words, I would expect the UFFI to do the call of
SetDllDirectory() or whatever is needed on Windows for me or to use
other mechanisms on other platforms.



> Do you mean you link OTHER.DLL to YOUR.DLL at compile time?
> But still, YOUR.DLL it not loaded until called via Pharo FFI, so neither
> is OTHER.DLL,
> and Dynamic-Link Library Search Order [1] says... "If a DLL with the
> same module name
> is already loaded in memory, the system uses the loaded DLL, no matter
> which directory it is in.
> The system does not search for the DLL."
>
> So maybe before calling any of YOUR.DLL,
> do a SetDllDirectory followed by a LoadLibrary(OTHER.DLL)
> and later when you call into YOUR.DLL,
> OTHER.DLL is already loaded.
>

Yes, this might work but is rather contrived.



> Otherwise it would seems(?) your options are either Manifests
> or Dynamic-Link Library Redirection, both done outside of Pharo.
>

What if neither YOUR.DLL nor OTHER.DLL are under my control? The only
thing I know is where the .dll and the corresponding .lib + .h are.



Thanks for your interest
Raffaello




>     But I'll try whether SetDllDirectory also affects load-time searches.
>
>     Anyway, I hope future versions of the UFFI to offer setting the
>     search paths more easily.
>
>
>
>     Greetings
>     Raffaello
>


Reply | Threaded
Open this post in threaded view
|

Re: Specifying library dependencies in UFFI

EstebanLM
Hi,

UFFI cannot do what you want.
If I understand well, you have:

Pharo -> libA.dll -> libB.dll

Pharo does a LoadLibrary(libA.dll), but has no control on how libA calls libB… and is not possible for us to influence it more than the predefined platform ways.

One posible workaround (just possible, no idea if it will work) is to perform a load of libB before is required by libB. Then hopefully the loader will understand is same library and will answer same handler (but not sure, because if it discriminates by path… then you’re in the same position). Anyway, if you want to try it, it would be something like this:

DynamicLoader loadLibrary: 'path/to/libB.dll’.
DynamicLoader loadLibrary: 'path/to/libA.dll’.

then you can try to execute functions of libA…

cheers,
Esteban


> On 15 Mar 2017, at 14:32, Raffaello Giulietti <[hidden email]> wrote:
>
> On 2017-03-15 13:51, Ben Coman wrote:
>>
>>
>> On Wed, Mar 15, 2017 at 4:42 PM, Raffaello Giulietti
>> <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>>    Hi Ben,
>>
>>    my understanding is that SetDllDirectory only affects the search
>>    path of libraries that are loaded at *run-time* with LoadLibrary.
>>
>>    What I'm asking for is a mechanism that works at *load-time*, when a
>>    library I'm accessing directly depends on another one which I'm not
>>    targeting directly.
>>
>>
>> I don't quite follow.  I'm not intimate with Windows dynamic loading, so
>> these questions are an opportunity for me to learn...
>>
>> You can't mean when Pharo loads, because it wouldn't know of OTHER.DLL ?
>>
>> Do you mean that LoadLibrary is not called explicitly by YOUR.DLL before
>> its uses the function from OTHER.DLL?
>>
>
> During the build of YOUR.DLL, you can specify that it should depend on OTHER.LIB (kind of binary which lists function headers and static data declarations for OTHER.DLL). Linking resolution is then done at load-time of YOUR.DLL, not by invoking LoadLibrary("OTHER.DLL").
>
> When YOUR.DLL is loaded by UFFI at run-time (here by means of LoadLibrary(), I guess), Windows (not the UFFI) also attempts to load OTHER.DLL because of the linking information gathered during the build. Notice that the linking information does *not* include a specific path for OTHER.DLL.
>
> (You don't have to mention standard *.LIBs like Kernel32.lib because they are included by default. But of course, Windows knows where to find the corresponding .dll.)
>
> OTHER.DLL is searched using the strategy described in the MSDN page you mention. My expectation is/was that UFFI has/had an API for specifying the search paths where to find OTHER.DLL in a platform-independent way. In other words, I would expect the UFFI to do the call of SetDllDirectory() or whatever is needed on Windows for me or to use other mechanisms on other platforms.
>
>
>
>> Do you mean you link OTHER.DLL to YOUR.DLL at compile time?
>> But still, YOUR.DLL it not loaded until called via Pharo FFI, so neither
>> is OTHER.DLL,
>> and Dynamic-Link Library Search Order [1] says... "If a DLL with the
>> same module name
>> is already loaded in memory, the system uses the loaded DLL, no matter
>> which directory it is in.
>> The system does not search for the DLL."
>>
>> So maybe before calling any of YOUR.DLL,
>> do a SetDllDirectory followed by a LoadLibrary(OTHER.DLL)
>> and later when you call into YOUR.DLL,
>> OTHER.DLL is already loaded.
>>
>
> Yes, this might work but is rather contrived.
>
>
>
>> Otherwise it would seems(?) your options are either Manifests
>> or Dynamic-Link Library Redirection, both done outside of Pharo.
>>
>
> What if neither YOUR.DLL nor OTHER.DLL are under my control? The only thing I know is where the .dll and the corresponding .lib + .h are.
>
>
>
> Thanks for your interest
> Raffaello
>
>
>
>
>>    But I'll try whether SetDllDirectory also affects load-time searches.
>>
>>    Anyway, I hope future versions of the UFFI to offer setting the
>>    search paths more easily.
>>
>>
>>
>>    Greetings
>>    Raffaello
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Specifying library dependencies in UFFI

raffaello.giulietti
Hi Esteban,

I understand this is the current status of the UFFI, so I can certainly
use the workarounds discussed below.

But I hope UFFI will soon offer a more "declarative" way to specify the
search folders, kind of LD_LIBRARY_PATH mechanism but in the UFFI.



Greetings
Raffaello




On 2017-03-15 14:52, Esteban Lorenzano wrote:

> Hi,
>
> UFFI cannot do what you want.
> If I understand well, you have:
>
> Pharo -> libA.dll -> libB.dll
>
> Pharo does a LoadLibrary(libA.dll), but has no control on how libA calls libB… and is not possible for us to influence it more than the predefined platform ways.
>
> One posible workaround (just possible, no idea if it will work) is to perform a load of libB before is required by libB. Then hopefully the loader will understand is same library and will answer same handler (but not sure, because if it discriminates by path… then you’re in the same position). Anyway, if you want to try it, it would be something like this:
>
> DynamicLoader loadLibrary: 'path/to/libB.dll’.
> DynamicLoader loadLibrary: 'path/to/libA.dll’.
>
> then you can try to execute functions of libA…
>
> cheers,
> Esteban
>
>
>> On 15 Mar 2017, at 14:32, Raffaello Giulietti <[hidden email]> wrote:
>>
>> On 2017-03-15 13:51, Ben Coman wrote:
>>>
>>>
>>> On Wed, Mar 15, 2017 at 4:42 PM, Raffaello Giulietti
>>> <[hidden email]
>>> <mailto:[hidden email]>> wrote:
>>>
>>>    Hi Ben,
>>>
>>>    my understanding is that SetDllDirectory only affects the search
>>>    path of libraries that are loaded at *run-time* with LoadLibrary.
>>>
>>>    What I'm asking for is a mechanism that works at *load-time*, when a
>>>    library I'm accessing directly depends on another one which I'm not
>>>    targeting directly.
>>>
>>>
>>> I don't quite follow.  I'm not intimate with Windows dynamic loading, so
>>> these questions are an opportunity for me to learn...
>>>
>>> You can't mean when Pharo loads, because it wouldn't know of OTHER.DLL ?
>>>
>>> Do you mean that LoadLibrary is not called explicitly by YOUR.DLL before
>>> its uses the function from OTHER.DLL?
>>>
>>
>> During the build of YOUR.DLL, you can specify that it should depend on OTHER.LIB (kind of binary which lists function headers and static data declarations for OTHER.DLL). Linking resolution is then done at load-time of YOUR.DLL, not by invoking LoadLibrary("OTHER.DLL").
>>
>> When YOUR.DLL is loaded by UFFI at run-time (here by means of LoadLibrary(), I guess), Windows (not the UFFI) also attempts to load OTHER.DLL because of the linking information gathered during the build. Notice that the linking information does *not* include a specific path for OTHER.DLL.
>>
>> (You don't have to mention standard *.LIBs like Kernel32.lib because they are included by default. But of course, Windows knows where to find the corresponding .dll.)
>>
>> OTHER.DLL is searched using the strategy described in the MSDN page you mention. My expectation is/was that UFFI has/had an API for specifying the search paths where to find OTHER.DLL in a platform-independent way. In other words, I would expect the UFFI to do the call of SetDllDirectory() or whatever is needed on Windows for me or to use other mechanisms on other platforms.
>>
>>
>>
>>> Do you mean you link OTHER.DLL to YOUR.DLL at compile time?
>>> But still, YOUR.DLL it not loaded until called via Pharo FFI, so neither
>>> is OTHER.DLL,
>>> and Dynamic-Link Library Search Order [1] says... "If a DLL with the
>>> same module name
>>> is already loaded in memory, the system uses the loaded DLL, no matter
>>> which directory it is in.
>>> The system does not search for the DLL."
>>>
>>> So maybe before calling any of YOUR.DLL,
>>> do a SetDllDirectory followed by a LoadLibrary(OTHER.DLL)
>>> and later when you call into YOUR.DLL,
>>> OTHER.DLL is already loaded.
>>>
>>
>> Yes, this might work but is rather contrived.
>>
>>
>>
>>> Otherwise it would seems(?) your options are either Manifests
>>> or Dynamic-Link Library Redirection, both done outside of Pharo.
>>>
>>
>> What if neither YOUR.DLL nor OTHER.DLL are under my control? The only thing I know is where the .dll and the corresponding .lib + .h are.
>>
>>
>>
>> Thanks for your interest
>> Raffaello
>>
>>
>>
>>
>>>    But I'll try whether SetDllDirectory also affects load-time searches.
>>>
>>>    Anyway, I hope future versions of the UFFI to offer setting the
>>>    search paths more easily.
>>>
>>>
>>>
>>>    Greetings
>>>    Raffaello
>>>
>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Specifying library dependencies in UFFI

EstebanLM

> On 15 Mar 2017, at 15:08, Raffaello Giulietti <[hidden email]> wrote:
>
> Hi Esteban,
>
> I understand this is the current status of the UFFI, so I can certainly use the workarounds discussed below.
>
> But I hope UFFI will soon offer a more "declarative" way to specify the search folders, kind of LD_LIBRARY_PATH mechanism but in the UFFI.

that will NOT solve the problem of dependencies of libraries indirectly references.
There is no way to do that (indirect reference solving) on an executing program that I know.
So is not “current status of UFFI”: it will never provide that.

All that, unless someone come and says: "look, Esteban is like this”.

I spent last three weeks (not full time of course) trying to do exactly that for our current linux vm, to force libgit2 to use the version of libssh2 we want (and not the version on system)… and I failed. I solved some cases, but not all. So at the end I opted to change zeroconf to define LD_LIBRARY_PATH before calling pharo. Only way I found I can force the lookup paths.

cheers,
Esteban

>
>
>
> Greetings
> Raffaello
>
>
>
>
> On 2017-03-15 14:52, Esteban Lorenzano wrote:
>> Hi,
>>
>> UFFI cannot do what you want.
>> If I understand well, you have:
>>
>> Pharo -> libA.dll -> libB.dll
>>
>> Pharo does a LoadLibrary(libA.dll), but has no control on how libA calls libB… and is not possible for us to influence it more than the predefined platform ways.
>>
>> One posible workaround (just possible, no idea if it will work) is to perform a load of libB before is required by libB. Then hopefully the loader will understand is same library and will answer same handler (but not sure, because if it discriminates by path… then you’re in the same position). Anyway, if you want to try it, it would be something like this:
>>
>> DynamicLoader loadLibrary: 'path/to/libB.dll’.
>> DynamicLoader loadLibrary: 'path/to/libA.dll’.
>>
>> then you can try to execute functions of libA…
>>
>> cheers,
>> Esteban
>>
>>
>>> On 15 Mar 2017, at 14:32, Raffaello Giulietti <[hidden email]> wrote:
>>>
>>> On 2017-03-15 13:51, Ben Coman wrote:
>>>>
>>>>
>>>> On Wed, Mar 15, 2017 at 4:42 PM, Raffaello Giulietti
>>>> <[hidden email]
>>>> <mailto:[hidden email]>> wrote:
>>>>
>>>>   Hi Ben,
>>>>
>>>>   my understanding is that SetDllDirectory only affects the search
>>>>   path of libraries that are loaded at *run-time* with LoadLibrary.
>>>>
>>>>   What I'm asking for is a mechanism that works at *load-time*, when a
>>>>   library I'm accessing directly depends on another one which I'm not
>>>>   targeting directly.
>>>>
>>>>
>>>> I don't quite follow.  I'm not intimate with Windows dynamic loading, so
>>>> these questions are an opportunity for me to learn...
>>>>
>>>> You can't mean when Pharo loads, because it wouldn't know of OTHER.DLL ?
>>>>
>>>> Do you mean that LoadLibrary is not called explicitly by YOUR.DLL before
>>>> its uses the function from OTHER.DLL?
>>>>
>>>
>>> During the build of YOUR.DLL, you can specify that it should depend on OTHER.LIB (kind of binary which lists function headers and static data declarations for OTHER.DLL). Linking resolution is then done at load-time of YOUR.DLL, not by invoking LoadLibrary("OTHER.DLL").
>>>
>>> When YOUR.DLL is loaded by UFFI at run-time (here by means of LoadLibrary(), I guess), Windows (not the UFFI) also attempts to load OTHER.DLL because of the linking information gathered during the build. Notice that the linking information does *not* include a specific path for OTHER.DLL.
>>>
>>> (You don't have to mention standard *.LIBs like Kernel32.lib because they are included by default. But of course, Windows knows where to find the corresponding .dll.)
>>>
>>> OTHER.DLL is searched using the strategy described in the MSDN page you mention. My expectation is/was that UFFI has/had an API for specifying the search paths where to find OTHER.DLL in a platform-independent way. In other words, I would expect the UFFI to do the call of SetDllDirectory() or whatever is needed on Windows for me or to use other mechanisms on other platforms.
>>>
>>>
>>>
>>>> Do you mean you link OTHER.DLL to YOUR.DLL at compile time?
>>>> But still, YOUR.DLL it not loaded until called via Pharo FFI, so neither
>>>> is OTHER.DLL,
>>>> and Dynamic-Link Library Search Order [1] says... "If a DLL with the
>>>> same module name
>>>> is already loaded in memory, the system uses the loaded DLL, no matter
>>>> which directory it is in.
>>>> The system does not search for the DLL."
>>>>
>>>> So maybe before calling any of YOUR.DLL,
>>>> do a SetDllDirectory followed by a LoadLibrary(OTHER.DLL)
>>>> and later when you call into YOUR.DLL,
>>>> OTHER.DLL is already loaded.
>>>>
>>>
>>> Yes, this might work but is rather contrived.
>>>
>>>
>>>
>>>> Otherwise it would seems(?) your options are either Manifests
>>>> or Dynamic-Link Library Redirection, both done outside of Pharo.
>>>>
>>>
>>> What if neither YOUR.DLL nor OTHER.DLL are under my control? The only thing I know is where the .dll and the corresponding .lib + .h are.
>>>
>>>
>>>
>>> Thanks for your interest
>>> Raffaello
>>>
>>>
>>>
>>>
>>>>   But I'll try whether SetDllDirectory also affects load-time searches.
>>>>
>>>>   Anyway, I hope future versions of the UFFI to offer setting the
>>>>   search paths more easily.
>>>>
>>>>
>>>>
>>>>   Greetings
>>>>   Raffaello
>>>>
>>>
>>>
>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Specifying library dependencies in UFFI

raffaello.giulietti
On 2017-03-15 15:20, Esteban Lorenzano wrote:

>
>> On 15 Mar 2017, at 15:08, Raffaello Giulietti <[hidden email]> wrote:
>>
>> Hi Esteban,
>>
>> I understand this is the current status of the UFFI, so I can certainly use the workarounds discussed below.
>>
>> But I hope UFFI will soon offer a more "declarative" way to specify the search folders, kind of LD_LIBRARY_PATH mechanism but in the UFFI.
>
> that will NOT solve the problem of dependencies of libraries indirectly references.
> There is no way to do that (indirect reference solving) on an executing program that I know.
> So is not “current status of UFFI”: it will never provide that.
>

Do you mean that SetDllDirectory() has no influence when invoked between
Pharo's start and the LoadLibrary() call done by the UFFI?




> All that, unless someone come and says: "look, Esteban is like this”.
>
> I spent last three weeks (not full time of course) trying to do exactly that for our current linux vm, to force libgit2 to use the version of libssh2 we want (and not the version on system)… and I failed. I solved some cases, but not all. So at the end I opted to change zeroconf to define LD_LIBRARY_PATH before calling pharo. Only way I found I can force the lookup paths.
>
> cheers,
> Esteban
>
>>
>>
>>
>> Greetings
>> Raffaello
>>
>>
>>
>>
>> On 2017-03-15 14:52, Esteban Lorenzano wrote:
>>> Hi,
>>>
>>> UFFI cannot do what you want.
>>> If I understand well, you have:
>>>
>>> Pharo -> libA.dll -> libB.dll
>>>
>>> Pharo does a LoadLibrary(libA.dll), but has no control on how libA calls libB… and is not possible for us to influence it more than the predefined platform ways.
>>>
>>> One posible workaround (just possible, no idea if it will work) is to perform a load of libB before is required by libB. Then hopefully the loader will understand is same library and will answer same handler (but not sure, because if it discriminates by path… then you’re in the same position). Anyway, if you want to try it, it would be something like this:
>>>
>>> DynamicLoader loadLibrary: 'path/to/libB.dll’.
>>> DynamicLoader loadLibrary: 'path/to/libA.dll’.
>>>
>>> then you can try to execute functions of libA…
>>>
>>> cheers,
>>> Esteban
>>>
>>>
>>>> On 15 Mar 2017, at 14:32, Raffaello Giulietti <[hidden email]> wrote:
>>>>
>>>> On 2017-03-15 13:51, Ben Coman wrote:
>>>>>
>>>>>
>>>>> On Wed, Mar 15, 2017 at 4:42 PM, Raffaello Giulietti
>>>>> <[hidden email]
>>>>> <mailto:[hidden email]>> wrote:
>>>>>
>>>>>   Hi Ben,
>>>>>
>>>>>   my understanding is that SetDllDirectory only affects the search
>>>>>   path of libraries that are loaded at *run-time* with LoadLibrary.
>>>>>
>>>>>   What I'm asking for is a mechanism that works at *load-time*, when a
>>>>>   library I'm accessing directly depends on another one which I'm not
>>>>>   targeting directly.
>>>>>
>>>>>
>>>>> I don't quite follow.  I'm not intimate with Windows dynamic loading, so
>>>>> these questions are an opportunity for me to learn...
>>>>>
>>>>> You can't mean when Pharo loads, because it wouldn't know of OTHER.DLL ?
>>>>>
>>>>> Do you mean that LoadLibrary is not called explicitly by YOUR.DLL before
>>>>> its uses the function from OTHER.DLL?
>>>>>
>>>>
>>>> During the build of YOUR.DLL, you can specify that it should depend on OTHER.LIB (kind of binary which lists function headers and static data declarations for OTHER.DLL). Linking resolution is then done at load-time of YOUR.DLL, not by invoking LoadLibrary("OTHER.DLL").
>>>>
>>>> When YOUR.DLL is loaded by UFFI at run-time (here by means of LoadLibrary(), I guess), Windows (not the UFFI) also attempts to load OTHER.DLL because of the linking information gathered during the build. Notice that the linking information does *not* include a specific path for OTHER.DLL.
>>>>
>>>> (You don't have to mention standard *.LIBs like Kernel32.lib because they are included by default. But of course, Windows knows where to find the corresponding .dll.)
>>>>
>>>> OTHER.DLL is searched using the strategy described in the MSDN page you mention. My expectation is/was that UFFI has/had an API for specifying the search paths where to find OTHER.DLL in a platform-independent way. In other words, I would expect the UFFI to do the call of SetDllDirectory() or whatever is needed on Windows for me or to use other mechanisms on other platforms.
>>>>
>>>>
>>>>
>>>>> Do you mean you link OTHER.DLL to YOUR.DLL at compile time?
>>>>> But still, YOUR.DLL it not loaded until called via Pharo FFI, so neither
>>>>> is OTHER.DLL,
>>>>> and Dynamic-Link Library Search Order [1] says... "If a DLL with the
>>>>> same module name
>>>>> is already loaded in memory, the system uses the loaded DLL, no matter
>>>>> which directory it is in.
>>>>> The system does not search for the DLL."
>>>>>
>>>>> So maybe before calling any of YOUR.DLL,
>>>>> do a SetDllDirectory followed by a LoadLibrary(OTHER.DLL)
>>>>> and later when you call into YOUR.DLL,
>>>>> OTHER.DLL is already loaded.
>>>>>
>>>>
>>>> Yes, this might work but is rather contrived.
>>>>
>>>>
>>>>
>>>>> Otherwise it would seems(?) your options are either Manifests
>>>>> or Dynamic-Link Library Redirection, both done outside of Pharo.
>>>>>
>>>>
>>>> What if neither YOUR.DLL nor OTHER.DLL are under my control? The only thing I know is where the .dll and the corresponding .lib + .h are.
>>>>
>>>>
>>>>
>>>> Thanks for your interest
>>>> Raffaello
>>>>
>>>>
>>>>
>>>>
>>>>>   But I'll try whether SetDllDirectory also affects load-time searches.
>>>>>
>>>>>   Anyway, I hope future versions of the UFFI to offer setting the
>>>>>   search paths more easily.
>>>>>
>>>>>
>>>>>
>>>>>   Greetings
>>>>>   Raffaello
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Specifying library dependencies in UFFI

EstebanLM

> On 15 Mar 2017, at 15:44, Raffaello Giulietti <[hidden email]> wrote:
>
> On 2017-03-15 15:20, Esteban Lorenzano wrote:
>>
>>> On 15 Mar 2017, at 15:08, Raffaello Giulietti <[hidden email]> wrote:
>>>
>>> Hi Esteban,
>>>
>>> I understand this is the current status of the UFFI, so I can certainly use the workarounds discussed below.
>>>
>>> But I hope UFFI will soon offer a more "declarative" way to specify the search folders, kind of LD_LIBRARY_PATH mechanism but in the UFFI.
>>
>> that will NOT solve the problem of dependencies of libraries indirectly references.
>> There is no way to do that (indirect reference solving) on an executing program that I know.
>> So is not “current status of UFFI”: it will never provide that.
>>
>
> Do you mean that SetDllDirectory() has no influence when invoked between Pharo's start and the LoadLibrary() call done by the UFFI?

I really have no idea. You can try to add it to your app.

Esteban

>
>
>
>
>> All that, unless someone come and says: "look, Esteban is like this”.
>>
>> I spent last three weeks (not full time of course) trying to do exactly that for our current linux vm, to force libgit2 to use the version of libssh2 we want (and not the version on system)… and I failed. I solved some cases, but not all. So at the end I opted to change zeroconf to define LD_LIBRARY_PATH before calling pharo. Only way I found I can force the lookup paths.
>>
>> cheers,
>> Esteban
>>
>>>
>>>
>>>
>>> Greetings
>>> Raffaello
>>>
>>>
>>>
>>>
>>> On 2017-03-15 14:52, Esteban Lorenzano wrote:
>>>> Hi,
>>>>
>>>> UFFI cannot do what you want.
>>>> If I understand well, you have:
>>>>
>>>> Pharo -> libA.dll -> libB.dll
>>>>
>>>> Pharo does a LoadLibrary(libA.dll), but has no control on how libA calls libB… and is not possible for us to influence it more than the predefined platform ways.
>>>>
>>>> One posible workaround (just possible, no idea if it will work) is to perform a load of libB before is required by libB. Then hopefully the loader will understand is same library and will answer same handler (but not sure, because if it discriminates by path… then you’re in the same position). Anyway, if you want to try it, it would be something like this:
>>>>
>>>> DynamicLoader loadLibrary: 'path/to/libB.dll’.
>>>> DynamicLoader loadLibrary: 'path/to/libA.dll’.
>>>>
>>>> then you can try to execute functions of libA…
>>>>
>>>> cheers,
>>>> Esteban
>>>>
>>>>
>>>>> On 15 Mar 2017, at 14:32, Raffaello Giulietti <[hidden email]> wrote:
>>>>>
>>>>> On 2017-03-15 13:51, Ben Coman wrote:
>>>>>>
>>>>>>
>>>>>> On Wed, Mar 15, 2017 at 4:42 PM, Raffaello Giulietti
>>>>>> <[hidden email]
>>>>>> <mailto:[hidden email]>> wrote:
>>>>>>
>>>>>>  Hi Ben,
>>>>>>
>>>>>>  my understanding is that SetDllDirectory only affects the search
>>>>>>  path of libraries that are loaded at *run-time* with LoadLibrary.
>>>>>>
>>>>>>  What I'm asking for is a mechanism that works at *load-time*, when a
>>>>>>  library I'm accessing directly depends on another one which I'm not
>>>>>>  targeting directly.
>>>>>>
>>>>>>
>>>>>> I don't quite follow.  I'm not intimate with Windows dynamic loading, so
>>>>>> these questions are an opportunity for me to learn...
>>>>>>
>>>>>> You can't mean when Pharo loads, because it wouldn't know of OTHER.DLL ?
>>>>>>
>>>>>> Do you mean that LoadLibrary is not called explicitly by YOUR.DLL before
>>>>>> its uses the function from OTHER.DLL?
>>>>>>
>>>>>
>>>>> During the build of YOUR.DLL, you can specify that it should depend on OTHER.LIB (kind of binary which lists function headers and static data declarations for OTHER.DLL). Linking resolution is then done at load-time of YOUR.DLL, not by invoking LoadLibrary("OTHER.DLL").
>>>>>
>>>>> When YOUR.DLL is loaded by UFFI at run-time (here by means of LoadLibrary(), I guess), Windows (not the UFFI) also attempts to load OTHER.DLL because of the linking information gathered during the build. Notice that the linking information does *not* include a specific path for OTHER.DLL.
>>>>>
>>>>> (You don't have to mention standard *.LIBs like Kernel32.lib because they are included by default. But of course, Windows knows where to find the corresponding .dll.)
>>>>>
>>>>> OTHER.DLL is searched using the strategy described in the MSDN page you mention. My expectation is/was that UFFI has/had an API for specifying the search paths where to find OTHER.DLL in a platform-independent way. In other words, I would expect the UFFI to do the call of SetDllDirectory() or whatever is needed on Windows for me or to use other mechanisms on other platforms.
>>>>>
>>>>>
>>>>>
>>>>>> Do you mean you link OTHER.DLL to YOUR.DLL at compile time?
>>>>>> But still, YOUR.DLL it not loaded until called via Pharo FFI, so neither
>>>>>> is OTHER.DLL,
>>>>>> and Dynamic-Link Library Search Order [1] says... "If a DLL with the
>>>>>> same module name
>>>>>> is already loaded in memory, the system uses the loaded DLL, no matter
>>>>>> which directory it is in.
>>>>>> The system does not search for the DLL."
>>>>>>
>>>>>> So maybe before calling any of YOUR.DLL,
>>>>>> do a SetDllDirectory followed by a LoadLibrary(OTHER.DLL)
>>>>>> and later when you call into YOUR.DLL,
>>>>>> OTHER.DLL is already loaded.
>>>>>>
>>>>>
>>>>> Yes, this might work but is rather contrived.
>>>>>
>>>>>
>>>>>
>>>>>> Otherwise it would seems(?) your options are either Manifests
>>>>>> or Dynamic-Link Library Redirection, both done outside of Pharo.
>>>>>>
>>>>>
>>>>> What if neither YOUR.DLL nor OTHER.DLL are under my control? The only thing I know is where the .dll and the corresponding .lib + .h are.
>>>>>
>>>>>
>>>>>
>>>>> Thanks for your interest
>>>>> Raffaello
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>  But I'll try whether SetDllDirectory also affects load-time searches.
>>>>>>
>>>>>>  Anyway, I hope future versions of the UFFI to offer setting the
>>>>>>  search paths more easily.
>>>>>>
>>>>>>
>>>>>>
>>>>>>  Greetings
>>>>>>  Raffaello
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Specifying library dependencies in UFFI

kilon.alios
Please note the moment you use a FFI of any language you step to a C territory. VMs tend not to touch the code because the code is already in machine code format.

There is zero reason for UFFI to worry about the dependencies of a library loaded, this must happen at the library level that UFFI does not focus on.

What you describe here is standard practice because the vast majority of libraries depend on other libraries. In the end UFFI or any FFI in any language is not there to replace proper C coding but rather to allow you to utilize C libraries. You cannot hope to achieve performance comparable to C if you try to use a language that is not C to do standard things like dependency management.

If the library fail to load it wont be because of UFFI it will be because the library is buggy at handling its dependencies. That means that the library will be buggy even if you use it from C directly.

Even you do not know what you doing at C level, you going to crash Pharo countless times anyway and the least of your worries will be handling dependencies which far more obvious than some other nasty C bugs.

I think its a great idea that UFFI does zero hand holding. This way it gives the clear message that if you do not understand C then you should not be using the UFFI in the first place.

On Wed, Mar 15, 2017 at 4:51 PM Esteban Lorenzano <[hidden email]> wrote:

> On 15 Mar 2017, at 15:44, Raffaello Giulietti <[hidden email]> wrote:
>
> On 2017-03-15 15:20, Esteban Lorenzano wrote:
>>
>>> On 15 Mar 2017, at 15:08, Raffaello Giulietti <[hidden email]> wrote:
>>>
>>> Hi Esteban,
>>>
>>> I understand this is the current status of the UFFI, so I can certainly use the workarounds discussed below.
>>>
>>> But I hope UFFI will soon offer a more "declarative" way to specify the search folders, kind of LD_LIBRARY_PATH mechanism but in the UFFI.
>>
>> that will NOT solve the problem of dependencies of libraries indirectly references.
>> There is no way to do that (indirect reference solving) on an executing program that I know.
>> So is not “current status of UFFI”: it will never provide that.
>>
>
> Do you mean that SetDllDirectory() has no influence when invoked between Pharo's start and the LoadLibrary() call done by the UFFI?

I really have no idea. You can try to add it to your app.

Esteban

>
>
>
>
>> All that, unless someone come and says: "look, Esteban is like this”.
>>
>> I spent last three weeks (not full time of course) trying to do exactly that for our current linux vm, to force libgit2 to use the version of libssh2 we want (and not the version on system)… and I failed. I solved some cases, but not all. So at the end I opted to change zeroconf to define LD_LIBRARY_PATH before calling pharo. Only way I found I can force the lookup paths.
>>
>> cheers,
>> Esteban
>>
>>>
>>>
>>>
>>> Greetings
>>> Raffaello
>>>
>>>
>>>
>>>
>>> On 2017-03-15 14:52, Esteban Lorenzano wrote:
>>>> Hi,
>>>>
>>>> UFFI cannot do what you want.
>>>> If I understand well, you have:
>>>>
>>>> Pharo -> libA.dll -> libB.dll
>>>>
>>>> Pharo does a LoadLibrary(libA.dll), but has no control on how libA calls libB… and is not possible for us to influence it more than the predefined platform ways.
>>>>
>>>> One posible workaround (just possible, no idea if it will work) is to perform a load of libB before is required by libB. Then hopefully the loader will understand is same library and will answer same handler (but not sure, because if it discriminates by path… then you’re in the same position). Anyway, if you want to try it, it would be something like this:
>>>>
>>>> DynamicLoader loadLibrary: 'path/to/libB.dll’.
>>>> DynamicLoader loadLibrary: 'path/to/libA.dll’.
>>>>
>>>> then you can try to execute functions of libA…
>>>>
>>>> cheers,
>>>> Esteban
>>>>
>>>>
>>>>> On 15 Mar 2017, at 14:32, Raffaello Giulietti <[hidden email]> wrote:
>>>>>
>>>>> On 2017-03-15 13:51, Ben Coman wrote:
>>>>>>
>>>>>>
>>>>>> On Wed, Mar 15, 2017 at 4:42 PM, Raffaello Giulietti
>>>>>> <[hidden email]
>>>>>> <mailto:[hidden email]>> wrote:
>>>>>>
>>>>>>  Hi Ben,
>>>>>>
>>>>>>  my understanding is that SetDllDirectory only affects the search
>>>>>>  path of libraries that are loaded at *run-time* with LoadLibrary.
>>>>>>
>>>>>>  What I'm asking for is a mechanism that works at *load-time*, when a
>>>>>>  library I'm accessing directly depends on another one which I'm not
>>>>>>  targeting directly.
>>>>>>
>>>>>>
>>>>>> I don't quite follow.  I'm not intimate with Windows dynamic loading, so
>>>>>> these questions are an opportunity for me to learn...
>>>>>>
>>>>>> You can't mean when Pharo loads, because it wouldn't know of OTHER.DLL ?
>>>>>>
>>>>>> Do you mean that LoadLibrary is not called explicitly by YOUR.DLL before
>>>>>> its uses the function from OTHER.DLL?
>>>>>>
>>>>>
>>>>> During the build of YOUR.DLL, you can specify that it should depend on OTHER.LIB (kind of binary which lists function headers and static data declarations for OTHER.DLL). Linking resolution is then done at load-time of YOUR.DLL, not by invoking LoadLibrary("OTHER.DLL").
>>>>>
>>>>> When YOUR.DLL is loaded by UFFI at run-time (here by means of LoadLibrary(), I guess), Windows (not the UFFI) also attempts to load OTHER.DLL because of the linking information gathered during the build. Notice that the linking information does *not* include a specific path for OTHER.DLL.
>>>>>
>>>>> (You don't have to mention standard *.LIBs like Kernel32.lib because they are included by default. But of course, Windows knows where to find the corresponding .dll.)
>>>>>
>>>>> OTHER.DLL is searched using the strategy described in the MSDN page you mention. My expectation is/was that UFFI has/had an API for specifying the search paths where to find OTHER.DLL in a platform-independent way. In other words, I would expect the UFFI to do the call of SetDllDirectory() or whatever is needed on Windows for me or to use other mechanisms on other platforms.
>>>>>
>>>>>
>>>>>
>>>>>> Do you mean you link OTHER.DLL to YOUR.DLL at compile time?
>>>>>> But still, YOUR.DLL it not loaded until called via Pharo FFI, so neither
>>>>>> is OTHER.DLL,
>>>>>> and Dynamic-Link Library Search Order [1] says... "If a DLL with the
>>>>>> same module name
>>>>>> is already loaded in memory, the system uses the loaded DLL, no matter
>>>>>> which directory it is in.
>>>>>> The system does not search for the DLL."
>>>>>>
>>>>>> So maybe before calling any of YOUR.DLL,
>>>>>> do a SetDllDirectory followed by a LoadLibrary(OTHER.DLL)
>>>>>> and later when you call into YOUR.DLL,
>>>>>> OTHER.DLL is already loaded.
>>>>>>
>>>>>
>>>>> Yes, this might work but is rather contrived.
>>>>>
>>>>>
>>>>>
>>>>>> Otherwise it would seems(?) your options are either Manifests
>>>>>> or Dynamic-Link Library Redirection, both done outside of Pharo.
>>>>>>
>>>>>
>>>>> What if neither YOUR.DLL nor OTHER.DLL are under my control? The only thing I know is where the .dll and the corresponding .lib + .h are.
>>>>>
>>>>>
>>>>>
>>>>> Thanks for your interest
>>>>> Raffaello
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>  But I'll try whether SetDllDirectory also affects load-time searches.
>>>>>>
>>>>>>  Anyway, I hope future versions of the UFFI to offer setting the
>>>>>>  search paths more easily.
>>>>>>
>>>>>>
>>>>>>
>>>>>>  Greetings
>>>>>>  Raffaello
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Specifying library dependencies in UFFI

raffaello.giulietti
Hi Dimitris,

I don't get your point about proper C coding, C performance or Pharo crashes. It is not the focus of this thread.

How dependencies between libraries are resolved at load-time is dictated by the OS. If the OS offers a working mechanism to specify folders that the OS uses to search for libraries at load-time, well, I think that Pharo, hence the UFFI, should offer it as well and in a more fashionable format.
UFFI is quite elegant in many respects but I wouldn't call it a minimalist design (zero hand holding, if I understand you correctly). But I'm happy the extra elegance is there because it makes life more comfortable.

Again, I still have to experiment with SetDllDirectory() in the case of Windows, so I'm not in a position to say something authoritative yet.







On 2017-03-15 17:41, Dimitris Chloupis wrote:
Please note the moment you use a FFI of any language you step to a C territory. VMs tend not to touch the code because the code is already in machine code format.

There is zero reason for UFFI to worry about the dependencies of a library loaded, this must happen at the library level that UFFI does not focus on.

What you describe here is standard practice because the vast majority of libraries depend on other libraries. In the end UFFI or any FFI in any language is not there to replace proper C coding but rather to allow you to utilize C libraries. You cannot hope to achieve performance comparable to C if you try to use a language that is not C to do standard things like dependency management.

If the library fail to load it wont be because of UFFI it will be because the library is buggy at handling its dependencies. That means that the library will be buggy even if you use it from C directly.

Even you do not know what you doing at C level, you going to crash Pharo countless times anyway and the least of your worries will be handling dependencies which far more obvious than some other nasty C bugs.

I think its a great idea that UFFI does zero hand holding. This way it gives the clear message that if you do not understand C then you should not be using the UFFI in the first place.

On Wed, Mar 15, 2017 at 4:51 PM Esteban Lorenzano <[hidden email]> wrote:

> On 15 Mar 2017, at 15:44, Raffaello Giulietti <[hidden email]> wrote:
>
> On 2017-03-15 15:20, Esteban Lorenzano wrote:
>>
>>> On 15 Mar 2017, at 15:08, Raffaello Giulietti <[hidden email]> wrote:
>>>
>>> Hi Esteban,
>>>
>>> I understand this is the current status of the UFFI, so I can certainly use the workarounds discussed below.
>>>
>>> But I hope UFFI will soon offer a more "declarative" way to specify the search folders, kind of LD_LIBRARY_PATH mechanism but in the UFFI.
>>
>> that will NOT solve the problem of dependencies of libraries indirectly references.
>> There is no way to do that (indirect reference solving) on an executing program that I know.
>> So is not “current status of UFFI”: it will never provide that.
>>
>
> Do you mean that SetDllDirectory() has no influence when invoked between Pharo's start and the LoadLibrary() call done by the UFFI?

I really have no idea. You can try to add it to your app.

Esteban

>
>
>
>
>> All that, unless someone come and says: "look, Esteban is like this”.
>>
>> I spent last three weeks (not full time of course) trying to do exactly that for our current linux vm, to force libgit2 to use the version of libssh2 we want (and not the version on system)… and I failed. I solved some cases, but not all. So at the end I opted to change zeroconf to define LD_LIBRARY_PATH before calling pharo. Only way I found I can force the lookup paths.
>>
>> cheers,
>> Esteban
>>
>>>
>>>
>>>
>>> Greetings
>>> Raffaello
>>>
>>>
>>>
>>>
>>> On 2017-03-15 14:52, Esteban Lorenzano wrote:
>>>> Hi,
>>>>
>>>> UFFI cannot do what you want.
>>>> If I understand well, you have:
>>>>
>>>> Pharo -> libA.dll -> libB.dll
>>>>
>>>> Pharo does a LoadLibrary(libA.dll), but has no control on how libA calls libB… and is not possible for us to influence it more than the predefined platform ways.
>>>>
>>>> One posible workaround (just possible, no idea if it will work) is to perform a load of libB before is required by libB. Then hopefully the loader will understand is same library and will answer same handler (but not sure, because if it discriminates by path… then you’re in the same position). Anyway, if you want to try it, it would be something like this:
>>>>
>>>> DynamicLoader loadLibrary: 'path/to/libB.dll’.
>>>> DynamicLoader loadLibrary: 'path/to/libA.dll’.
>>>>
>>>> then you can try to execute functions of libA…
>>>>
>>>> cheers,
>>>> Esteban
>>>>
>>>>
>>>>> On 15 Mar 2017, at 14:32, Raffaello Giulietti <[hidden email]> wrote:
>>>>>
>>>>> On 2017-03-15 13:51, Ben Coman wrote:
>>>>>>
>>>>>>
>>>>>> On Wed, Mar 15, 2017 at 4:42 PM, Raffaello Giulietti
>>>>>> <[hidden email]
>>>>>> <mailto:[hidden email]>> wrote:
>>>>>>
>>>>>>  Hi Ben,
>>>>>>
>>>>>>  my understanding is that SetDllDirectory only affects the search
>>>>>>  path of libraries that are loaded at *run-time* with LoadLibrary.
>>>>>>
>>>>>>  What I'm asking for is a mechanism that works at *load-time*, when a
>>>>>>  library I'm accessing directly depends on another one which I'm not
>>>>>>  targeting directly.
>>>>>>
>>>>>>
>>>>>> I don't quite follow.  I'm not intimate with Windows dynamic loading, so
>>>>>> these questions are an opportunity for me to learn...
>>>>>>
>>>>>> You can't mean when Pharo loads, because it wouldn't know of OTHER.DLL ?
>>>>>>
>>>>>> Do you mean that LoadLibrary is not called explicitly by YOUR.DLL before
>>>>>> its uses the function from OTHER.DLL?
>>>>>>
>>>>>
>>>>> During the build of YOUR.DLL, you can specify that it should depend on OTHER.LIB (kind of binary which lists function headers and static data declarations for OTHER.DLL). Linking resolution is then done at load-time of YOUR.DLL, not by invoking LoadLibrary("OTHER.DLL").
>>>>>
>>>>> When YOUR.DLL is loaded by UFFI at run-time (here by means of LoadLibrary(), I guess), Windows (not the UFFI) also attempts to load OTHER.DLL because of the linking information gathered during the build. Notice that the linking information does *not* include a specific path for OTHER.DLL.
>>>>>
>>>>> (You don't have to mention standard *.LIBs like Kernel32.lib because they are included by default. But of course, Windows knows where to find the corresponding .dll.)
>>>>>
>>>>> OTHER.DLL is searched using the strategy described in the MSDN page you mention. My expectation is/was that UFFI has/had an API for specifying the search paths where to find OTHER.DLL in a platform-independent way. In other words, I would expect the UFFI to do the call of SetDllDirectory() or whatever is needed on Windows for me or to use other mechanisms on other platforms.
>>>>>
>>>>>
>>>>>
>>>>>> Do you mean you link OTHER.DLL to YOUR.DLL at compile time?
>>>>>> But still, YOUR.DLL it not loaded until called via Pharo FFI, so neither
>>>>>> is OTHER.DLL,
>>>>>> and Dynamic-Link Library Search Order [1] says... "If a DLL with the
>>>>>> same module name
>>>>>> is already loaded in memory, the system uses the loaded DLL, no matter
>>>>>> which directory it is in.
>>>>>> The system does not search for the DLL."
>>>>>>
>>>>>> So maybe before calling any of YOUR.DLL,
>>>>>> do a SetDllDirectory followed by a LoadLibrary(OTHER.DLL)
>>>>>> and later when you call into YOUR.DLL,
>>>>>> OTHER.DLL is already loaded.
>>>>>>
>>>>>
>>>>> Yes, this might work but is rather contrived.
>>>>>
>>>>>
>>>>>
>>>>>> Otherwise it would seems(?) your options are either Manifests
>>>>>> or Dynamic-Link Library Redirection, both done outside of Pharo.
>>>>>>
>>>>>
>>>>> What if neither YOUR.DLL nor OTHER.DLL are under my control? The only thing I know is where the .dll and the corresponding .lib + .h are.
>>>>>
>>>>>
>>>>>
>>>>> Thanks for your interest
>>>>> Raffaello
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>  But I'll try whether SetDllDirectory also affects load-time searches.
>>>>>>
>>>>>>  Anyway, I hope future versions of the UFFI to offer setting the
>>>>>>  search paths more easily.
>>>>>>
>>>>>>
>>>>>>
>>>>>>  Greetings
>>>>>>  Raffaello
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>



Reply | Threaded
Open this post in threaded view
|

Re: Specifying library dependencies in UFFI

bestlem
Yes the calling process can alter where these indirect DLLs are loaded
from and so should be able to be done in UFFI and directly from Smalltalk

The OS will load the DLL specified then it runs DllMain and doing that
causes the OS to load any DLLs that it calls

MSDN https://msdn.microsoft.com/en-us/library/7d83bc18.aspx says

With both implicit and explicit linking, Windows first searches for
"known DLLs", such as Kernel32.dll and User32.dll. Windows then searches
for the DLLs in the following sequence:
The directory where the executable module for the current process is
located.
The current directory.
The Windows system directory. The GetSystemDirectory function retrieves
the path of this directory.
The Windows directory. The GetWindowsDirectory function retrieves the
path of this directory.
The directories listed in the PATH environment variable.


This can be changed using the call LoadLibraryEx instead of LoadLibrary
Flags can be set to change the directories used

https://msdn.microsoft.com/en-us/library/windows/desktop/ms684179(v=vs.85).aspx


Also see AddDllDirectory
https://msdn.microsoft.com/en-us/library/windows/desktop/hh310513(v=vs.85).aspx

Mark

On 15/03/2017 17:22, Raffaello Giulietti wrote:

> Hi Dimitris,
>
> I don't get your point about proper C coding, C performance or Pharo
> crashes. It is not the focus of this thread.
>
> How dependencies between libraries are resolved at load-time is dictated
> by the OS. If the OS offers a working mechanism to specify folders that
> the OS uses to search for libraries at load-time, well, I think that
> Pharo, hence the UFFI, should offer it as well and in a more fashionable
> format.
> UFFI is quite elegant in many respects but I wouldn't call it a
> minimalist design (zero hand holding, if I understand you correctly).
> But I'm happy the extra elegance is there because it makes life more
> comfortable.
>
> Again, I still have to experiment with SetDllDirectory() in the case of
> Windows, so I'm not in a position to say something authoritative yet.
>
>
>
>
>
>
>
> On 2017-03-15 17:41, Dimitris Chloupis wrote:
>> Please note the moment you use a FFI of any language you step to a C
>> territory. VMs tend not to touch the code because the code is already
>> in machine code format.
>>
>> There is zero reason for UFFI to worry about the dependencies of a
>> library loaded, this must happen at the library level that UFFI does
>> not focus on.
>>
>> What you describe here is standard practice because the vast majority
>> of libraries depend on other libraries. In the end UFFI or any FFI in
>> any language is not there to replace proper C coding but rather to
>> allow you to utilize C libraries. You cannot hope to achieve
>> performance comparable to C if you try to use a language that is not C
>> to do standard things like dependency management.
>>
>> If the library fail to load it wont be because of UFFI it will be
>> because the library is buggy at handling its dependencies. That means
>> that the library will be buggy even if you use it from C directly.
>>
>> Even you do not know what you doing at C level, you going to crash
>> Pharo countless times anyway and the least of your worries will be
>> handling dependencies which far more obvious than some other nasty C
>> bugs.
>>
>> I think its a great idea that UFFI does zero hand holding. This way it
>> gives the clear message that if you do not understand C then you
>> should not be using the UFFI in the first place.
>>
>> On Wed, Mar 15, 2017 at 4:51 PM Esteban Lorenzano
>> <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>>
>>     > On 15 Mar 2017, at 15:44, Raffaello Giulietti
>>     <[hidden email]
>>     <mailto:[hidden email]>>
>>     wrote:
>>     >
>>     > On 2017-03-15 15:20, Esteban Lorenzano wrote:
>>     >>
>>     >>> On 15 Mar 2017, at 15:08, Raffaello Giulietti
>>     <[hidden email]
>>     <mailto:[hidden email]>>
>>     wrote:
>>     >>>
>>     >>> Hi Esteban,
>>     >>>
>>     >>> I understand this is the current status of the UFFI, so I can
>>     certainly use the workarounds discussed below.
>>     >>>
>>     >>> But I hope UFFI will soon offer a more "declarative" way to
>>     specify the search folders, kind of LD_LIBRARY_PATH mechanism but
>>     in the UFFI.
>>     >>
>>     >> that will NOT solve the problem of dependencies of libraries
>>     indirectly references.
>>     >> There is no way to do that (indirect reference solving) on an
>>     executing program that I know.
>>     >> So is not “current status of UFFI”: it will never provide that.
>>     >>
>>     >
>>     > Do you mean that SetDllDirectory() has no influence when invoked
>>     between Pharo's start and the LoadLibrary() call done by the UFFI?
>>
>>     I really have no idea. You can try to add it to your app.
>>
>>     Esteban
>>
>>     >
>>     >
>>     >
>>     >
>>     >> All that, unless someone come and says: "look, Esteban is like
>>     this”.
>>     >>
>>     >> I spent last three weeks (not full time of course) trying to do
>>     exactly that for our current linux vm, to force libgit2 to use the
>>     version of libssh2 we want (and not the version on system)… and I
>>     failed. I solved some cases, but not all. So at the end I opted to
>>     change zeroconf to define LD_LIBRARY_PATH before calling pharo.
>>     Only way I found I can force the lookup paths.
>>     >>
>>     >> cheers,
>>     >> Esteban
>>     >>
>>     >>>
>>     >>>
>>     >>>
>>     >>> Greetings
>>     >>> Raffaello
>>     >>>
>>     >>>
>>     >>>
>>     >>>
>>     >>> On 2017-03-15 14:52, Esteban Lorenzano wrote:
>>     >>>> Hi,
>>     >>>>
>>     >>>> UFFI cannot do what you want.
>>     >>>> If I understand well, you have:
>>     >>>>
>>     >>>> Pharo -> libA.dll -> libB.dll
>>     >>>>
>>     >>>> Pharo does a LoadLibrary(libA.dll), but has no control on how
>>     libA calls libB… and is not possible for us to influence it more
>>     than the predefined platform ways.
>>     >>>>
>>     >>>> One posible workaround (just possible, no idea if it will
>>     work) is to perform a load of libB before is required by libB.
>>     Then hopefully the loader will understand is same library and will
>>     answer same handler (but not sure, because if it discriminates by
>>     path… then you’re in the same position). Anyway, if you want to
>>     try it, it would be something like this:
>>     >>>>
>>     >>>> DynamicLoader loadLibrary: 'path/to/libB.dll’.
>>     >>>> DynamicLoader loadLibrary: 'path/to/libA.dll’.
>>     >>>>
>>     >>>> then you can try to execute functions of libA…
>>     >>>>
>>     >>>> cheers,
>>     >>>> Esteban
>>     >>>>
>>     >>>>
>>     >>>>> On 15 Mar 2017, at 14:32, Raffaello Giulietti
>>     <[hidden email]
>>     <mailto:[hidden email]>>
>>     wrote:
>>     >>>>>
>>     >>>>> On 2017-03-15 13:51, Ben Coman wrote:
>>     >>>>>>
>>     >>>>>>
>>     >>>>>> On Wed, Mar 15, 2017 at 4:42 PM, Raffaello Giulietti
>>     >>>>>>
>>     <[hidden email]
>>     <mailto:[hidden email]>
>>     >>>>>>
>>     <mailto:[hidden email] <mailto:[hidden email]>>>
>>     wrote:
>>     >>>>>>
>>     >>>>>>  Hi Ben,
>>     >>>>>>
>>     >>>>>>  my understanding is that SetDllDirectory only affects the
>>     search
>>     >>>>>>  path of libraries that are loaded at *run-time* with
>>     LoadLibrary.
>>     >>>>>>
>>     >>>>>>  What I'm asking for is a mechanism that works at
>>     *load-time*, when a
>>     >>>>>>  library I'm accessing directly depends on another one
>>     which I'm not
>>     >>>>>>  targeting directly.
>>     >>>>>>
>>     >>>>>>
>>     >>>>>> I don't quite follow.  I'm not intimate with Windows
>>     dynamic loading, so
>>     >>>>>> these questions are an opportunity for me to learn...
>>     >>>>>>
>>     >>>>>> You can't mean when Pharo loads, because it wouldn't know
>>     of OTHER.DLL ?
>>     >>>>>>
>>     >>>>>> Do you mean that LoadLibrary is not called explicitly by
>>     YOUR.DLL before
>>     >>>>>> its uses the function from OTHER.DLL?
>>     >>>>>>
>>     >>>>>
>>     >>>>> During the build of YOUR.DLL, you can specify that it should
>>     depend on OTHER.LIB (kind of binary which lists function headers
>>     and static data declarations for OTHER.DLL). Linking resolution is
>>     then done at load-time of YOUR.DLL, not by invoking
>>     LoadLibrary("OTHER.DLL").
>>     >>>>>
>>     >>>>> When YOUR.DLL is loaded by UFFI at run-time (here by means
>>     of LoadLibrary(), I guess), Windows (not the UFFI) also attempts
>>     to load OTHER.DLL because of the linking information gathered
>>     during the build. Notice that the linking information does *not*
>>     include a specific path for OTHER.DLL.
>>     >>>>>
>>     >>>>> (You don't have to mention standard *.LIBs like Kernel32.lib
>>     because they are included by default. But of course, Windows knows
>>     where to find the corresponding .dll.)
>>     >>>>>
>>     >>>>> OTHER.DLL is searched using the strategy described in the
>>     MSDN page you mention. My expectation is/was that UFFI has/had an
>>     API for specifying the search paths where to find OTHER.DLL in a
>>     platform-independent way. In other words, I would expect the UFFI
>>     to do the call of SetDllDirectory() or whatever is needed on
>>     Windows for me or to use other mechanisms on other platforms.
>>     >>>>>
>>     >>>>>
>>     >>>>>
>>     >>>>>> Do you mean you link OTHER.DLL to YOUR.DLL at compile time?
>>     >>>>>> But still, YOUR.DLL it not loaded until called via Pharo
>>     FFI, so neither
>>     >>>>>> is OTHER.DLL,
>>     >>>>>> and Dynamic-Link Library Search Order [1] says... "If a DLL
>>     with the
>>     >>>>>> same module name
>>     >>>>>> is already loaded in memory, the system uses the loaded
>>     DLL, no matter
>>     >>>>>> which directory it is in.
>>     >>>>>> The system does not search for the DLL."
>>     >>>>>>
>>     >>>>>> So maybe before calling any of YOUR.DLL,
>>     >>>>>> do a SetDllDirectory followed by a LoadLibrary(OTHER.DLL)
>>     >>>>>> and later when you call into YOUR.DLL,
>>     >>>>>> OTHER.DLL is already loaded.
>>     >>>>>>
>>     >>>>>
>>     >>>>> Yes, this might work but is rather contrived.
>>     >>>>>
>>     >>>>>
>>     >>>>>
>>     >>>>>> Otherwise it would seems(?) your options are either Manifests
>>     >>>>>> or Dynamic-Link Library Redirection, both done outside of
>>     Pharo.
>>     >>>>>>
>>     >>>>>
>>     >>>>> What if neither YOUR.DLL nor OTHER.DLL are under my control?
>>     The only thing I know is where the .dll and the corresponding .lib
>>     + .h are.
>>     >>>>>
>>     >>>>>
>>     >>>>>
>>     >>>>> Thanks for your interest
>>     >>>>> Raffaello
>>     >>>>>
>>     >>>>>
>>     >>>>>
>>     >>>>>
>>     >>>>>>  But I'll try whether SetDllDirectory also affects
>>     load-time searches.
>>     >>>>>>
>>     >>>>>>  Anyway, I hope future versions of the UFFI to offer
>>     setting the
>>     >>>>>>  search paths more easily.
>>     >>>>>>
>>     >>>>>>
>>     >>>>>>
>>     >>>>>>  Greetings
>>     >>>>>>  Raffaello
>>     >>>>>>
>>     >>>>>
>>     >>>>>
>>     >>>>
>>     >>>>
>>     >>>
>>     >>>
>>     >>
>>     >>
>>     >
>>     >
>>
>>
>


--
Mark


Reply | Threaded
Open this post in threaded view
|

Re: Specifying library dependencies in UFFI

raffaello.giulietti
Indeed, experimenting using some combination of AddDllDirectory(),
SetDllDirectory() and LoadLibraryEx() was in my plan for next week.

Thanks



On 16/03/17 11:59, Mark Bestley wrote:

> Yes the calling process can alter where these indirect DLLs are loaded
> from and so should be able to be done in UFFI and directly from Smalltalk
>
> The OS will load the DLL specified then it runs DllMain and doing that
> causes the OS to load any DLLs that it calls
>
> MSDN https://msdn.microsoft.com/en-us/library/7d83bc18.aspx says
>
> With both implicit and explicit linking, Windows first searches for
> "known DLLs", such as Kernel32.dll and User32.dll. Windows then searches
> for the DLLs in the following sequence:
> The directory where the executable module for the current process is
> located.
> The current directory.
> The Windows system directory. The GetSystemDirectory function retrieves
> the path of this directory.
> The Windows directory. The GetWindowsDirectory function retrieves the
> path of this directory.
> The directories listed in the PATH environment variable.
>
>
> This can be changed using the call LoadLibraryEx instead of LoadLibrary
> Flags can be set to change the directories used
>
> https://msdn.microsoft.com/en-us/library/windows/desktop/ms684179(v=vs.85).aspx
>
>
>
> Also see AddDllDirectory
> https://msdn.microsoft.com/en-us/library/windows/desktop/hh310513(v=vs.85).aspx
>
>
> Mark
>
> On 15/03/2017 17:22, Raffaello Giulietti wrote:
>> Hi Dimitris,
>>
>> I don't get your point about proper C coding, C performance or Pharo
>> crashes. It is not the focus of this thread.
>>
>> How dependencies between libraries are resolved at load-time is dictated
>> by the OS. If the OS offers a working mechanism to specify folders that
>> the OS uses to search for libraries at load-time, well, I think that
>> Pharo, hence the UFFI, should offer it as well and in a more fashionable
>> format.
>> UFFI is quite elegant in many respects but I wouldn't call it a
>> minimalist design (zero hand holding, if I understand you correctly).
>> But I'm happy the extra elegance is there because it makes life more
>> comfortable.
>>
>> Again, I still have to experiment with SetDllDirectory() in the case of
>> Windows, so I'm not in a position to say something authoritative yet.
>>
>>
>>
>>
>>
>>
>>
>> On 2017-03-15 17:41, Dimitris Chloupis wrote:
>>> Please note the moment you use a FFI of any language you step to a C
>>> territory. VMs tend not to touch the code because the code is already
>>> in machine code format.
>>>
>>> There is zero reason for UFFI to worry about the dependencies of a
>>> library loaded, this must happen at the library level that UFFI does
>>> not focus on.
>>>
>>> What you describe here is standard practice because the vast majority
>>> of libraries depend on other libraries. In the end UFFI or any FFI in
>>> any language is not there to replace proper C coding but rather to
>>> allow you to utilize C libraries. You cannot hope to achieve
>>> performance comparable to C if you try to use a language that is not C
>>> to do standard things like dependency management.
>>>
>>> If the library fail to load it wont be because of UFFI it will be
>>> because the library is buggy at handling its dependencies. That means
>>> that the library will be buggy even if you use it from C directly.
>>>
>>> Even you do not know what you doing at C level, you going to crash
>>> Pharo countless times anyway and the least of your worries will be
>>> handling dependencies which far more obvious than some other nasty C
>>> bugs.
>>>
>>> I think its a great idea that UFFI does zero hand holding. This way it
>>> gives the clear message that if you do not understand C then you
>>> should not be using the UFFI in the first place.
>>>
>>> On Wed, Mar 15, 2017 at 4:51 PM Esteban Lorenzano
>>> <[hidden email]
>>> <mailto:[hidden email]>> wrote:
>>>
>>>
>>>     > On 15 Mar 2017, at 15:44, Raffaello Giulietti
>>>     <[hidden email]
>>>     <mailto:[hidden email]>>
>>>     wrote:
>>>     >
>>>     > On 2017-03-15 15:20, Esteban Lorenzano wrote:
>>>     >>
>>>     >>> On 15 Mar 2017, at 15:08, Raffaello Giulietti
>>>     <[hidden email]
>>>     <mailto:[hidden email]>>
>>>     wrote:
>>>     >>>
>>>     >>> Hi Esteban,
>>>     >>>
>>>     >>> I understand this is the current status of the UFFI, so I can
>>>     certainly use the workarounds discussed below.
>>>     >>>
>>>     >>> But I hope UFFI will soon offer a more "declarative" way to
>>>     specify the search folders, kind of LD_LIBRARY_PATH mechanism but
>>>     in the UFFI.
>>>     >>
>>>     >> that will NOT solve the problem of dependencies of libraries
>>>     indirectly references.
>>>     >> There is no way to do that (indirect reference solving) on an
>>>     executing program that I know.
>>>     >> So is not “current status of UFFI”: it will never provide that.
>>>     >>
>>>     >
>>>     > Do you mean that SetDllDirectory() has no influence when invoked
>>>     between Pharo's start and the LoadLibrary() call done by the UFFI?
>>>
>>>     I really have no idea. You can try to add it to your app.
>>>
>>>     Esteban
>>>
>>>     >
>>>     >
>>>     >
>>>     >
>>>     >> All that, unless someone come and says: "look, Esteban is like
>>>     this”.
>>>     >>
>>>     >> I spent last three weeks (not full time of course) trying to do
>>>     exactly that for our current linux vm, to force libgit2 to use the
>>>     version of libssh2 we want (and not the version on system)… and I
>>>     failed. I solved some cases, but not all. So at the end I opted to
>>>     change zeroconf to define LD_LIBRARY_PATH before calling pharo.
>>>     Only way I found I can force the lookup paths.
>>>     >>
>>>     >> cheers,
>>>     >> Esteban
>>>     >>
>>>     >>>
>>>     >>>
>>>     >>>
>>>     >>> Greetings
>>>     >>> Raffaello
>>>     >>>
>>>     >>>
>>>     >>>
>>>     >>>
>>>     >>> On 2017-03-15 14:52, Esteban Lorenzano wrote:
>>>     >>>> Hi,
>>>     >>>>
>>>     >>>> UFFI cannot do what you want.
>>>     >>>> If I understand well, you have:
>>>     >>>>
>>>     >>>> Pharo -> libA.dll -> libB.dll
>>>     >>>>
>>>     >>>> Pharo does a LoadLibrary(libA.dll), but has no control on how
>>>     libA calls libB… and is not possible for us to influence it more
>>>     than the predefined platform ways.
>>>     >>>>
>>>     >>>> One posible workaround (just possible, no idea if it will
>>>     work) is to perform a load of libB before is required by libB.
>>>     Then hopefully the loader will understand is same library and will
>>>     answer same handler (but not sure, because if it discriminates by
>>>     path… then you’re in the same position). Anyway, if you want to
>>>     try it, it would be something like this:
>>>     >>>>
>>>     >>>> DynamicLoader loadLibrary: 'path/to/libB.dll’.
>>>     >>>> DynamicLoader loadLibrary: 'path/to/libA.dll’.
>>>     >>>>
>>>     >>>> then you can try to execute functions of libA…
>>>     >>>>
>>>     >>>> cheers,
>>>     >>>> Esteban
>>>     >>>>
>>>     >>>>
>>>     >>>>> On 15 Mar 2017, at 14:32, Raffaello Giulietti
>>>     <[hidden email]
>>>     <mailto:[hidden email]>>
>>>     wrote:
>>>     >>>>>
>>>     >>>>> On 2017-03-15 13:51, Ben Coman wrote:
>>>     >>>>>>
>>>     >>>>>>
>>>     >>>>>> On Wed, Mar 15, 2017 at 4:42 PM, Raffaello Giulietti
>>>     >>>>>>
>>>     <[hidden email]
>>>     <mailto:[hidden email]>
>>>     >>>>>>
>>>     <mailto:[hidden email]
>>> <mailto:[hidden email]>>>
>>>     wrote:
>>>     >>>>>>
>>>     >>>>>>  Hi Ben,
>>>     >>>>>>
>>>     >>>>>>  my understanding is that SetDllDirectory only affects the
>>>     search
>>>     >>>>>>  path of libraries that are loaded at *run-time* with
>>>     LoadLibrary.
>>>     >>>>>>
>>>     >>>>>>  What I'm asking for is a mechanism that works at
>>>     *load-time*, when a
>>>     >>>>>>  library I'm accessing directly depends on another one
>>>     which I'm not
>>>     >>>>>>  targeting directly.
>>>     >>>>>>
>>>     >>>>>>
>>>     >>>>>> I don't quite follow.  I'm not intimate with Windows
>>>     dynamic loading, so
>>>     >>>>>> these questions are an opportunity for me to learn...
>>>     >>>>>>
>>>     >>>>>> You can't mean when Pharo loads, because it wouldn't know
>>>     of OTHER.DLL ?
>>>     >>>>>>
>>>     >>>>>> Do you mean that LoadLibrary is not called explicitly by
>>>     YOUR.DLL before
>>>     >>>>>> its uses the function from OTHER.DLL?
>>>     >>>>>>
>>>     >>>>>
>>>     >>>>> During the build of YOUR.DLL, you can specify that it should
>>>     depend on OTHER.LIB (kind of binary which lists function headers
>>>     and static data declarations for OTHER.DLL). Linking resolution is
>>>     then done at load-time of YOUR.DLL, not by invoking
>>>     LoadLibrary("OTHER.DLL").
>>>     >>>>>
>>>     >>>>> When YOUR.DLL is loaded by UFFI at run-time (here by means
>>>     of LoadLibrary(), I guess), Windows (not the UFFI) also attempts
>>>     to load OTHER.DLL because of the linking information gathered
>>>     during the build. Notice that the linking information does *not*
>>>     include a specific path for OTHER.DLL.
>>>     >>>>>
>>>     >>>>> (You don't have to mention standard *.LIBs like Kernel32.lib
>>>     because they are included by default. But of course, Windows knows
>>>     where to find the corresponding .dll.)
>>>     >>>>>
>>>     >>>>> OTHER.DLL is searched using the strategy described in the
>>>     MSDN page you mention. My expectation is/was that UFFI has/had an
>>>     API for specifying the search paths where to find OTHER.DLL in a
>>>     platform-independent way. In other words, I would expect the UFFI
>>>     to do the call of SetDllDirectory() or whatever is needed on
>>>     Windows for me or to use other mechanisms on other platforms.
>>>     >>>>>
>>>     >>>>>
>>>     >>>>>
>>>     >>>>>> Do you mean you link OTHER.DLL to YOUR.DLL at compile time?
>>>     >>>>>> But still, YOUR.DLL it not loaded until called via Pharo
>>>     FFI, so neither
>>>     >>>>>> is OTHER.DLL,
>>>     >>>>>> and Dynamic-Link Library Search Order [1] says... "If a DLL
>>>     with the
>>>     >>>>>> same module name
>>>     >>>>>> is already loaded in memory, the system uses the loaded
>>>     DLL, no matter
>>>     >>>>>> which directory it is in.
>>>     >>>>>> The system does not search for the DLL."
>>>     >>>>>>
>>>     >>>>>> So maybe before calling any of YOUR.DLL,
>>>     >>>>>> do a SetDllDirectory followed by a LoadLibrary(OTHER.DLL)
>>>     >>>>>> and later when you call into YOUR.DLL,
>>>     >>>>>> OTHER.DLL is already loaded.
>>>     >>>>>>
>>>     >>>>>
>>>     >>>>> Yes, this might work but is rather contrived.
>>>     >>>>>
>>>     >>>>>
>>>     >>>>>
>>>     >>>>>> Otherwise it would seems(?) your options are either Manifests
>>>     >>>>>> or Dynamic-Link Library Redirection, both done outside of
>>>     Pharo.
>>>     >>>>>>
>>>     >>>>>
>>>     >>>>> What if neither YOUR.DLL nor OTHER.DLL are under my control?
>>>     The only thing I know is where the .dll and the corresponding .lib
>>>     + .h are.
>>>     >>>>>
>>>     >>>>>
>>>     >>>>>
>>>     >>>>> Thanks for your interest
>>>     >>>>> Raffaello
>>>     >>>>>
>>>     >>>>>
>>>     >>>>>
>>>     >>>>>
>>>     >>>>>>  But I'll try whether SetDllDirectory also affects
>>>     load-time searches.
>>>     >>>>>>
>>>     >>>>>>  Anyway, I hope future versions of the UFFI to offer
>>>     setting the
>>>     >>>>>>  search paths more easily.
>>>     >>>>>>
>>>     >>>>>>
>>>     >>>>>>
>>>     >>>>>>  Greetings
>>>     >>>>>>  Raffaello
>>>     >>>>>>
>>>     >>>>>
>>>     >>>>>
>>>     >>>>
>>>     >>>>
>>>     >>>
>>>     >>>
>>>     >>
>>>     >>
>>>     >
>>>     >
>>>
>>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Specifying library dependencies in UFFI

kilon.alios
My point was that anything you do in Pharo will lower performance quite a bit, but it wont be noticable anyway unless its in a big loop or something. 

Second that Pharo cannot debug such issues which is a huge deal when you work with OS stuff and OS calls. 

You are far better doing this outside UFFI and straight inside a C IDE because you will have an army of tools that will make your life a ton easier. 

Hence why I talked about performance, C coding and Pharo crashes. 

In my case I had to write and test the code in C to port it to UFFI because it was impossible to resolve the bugs from inside Pharo. Also crashing Pharo all the time got really annoying and I managed to corrupt a couple of images too in the process. 

Of course debugging C code from a C IDE is quite similar to how you work with Pharo. 

I also got an experience with loading libraries because I worked with Shared memory mapped files which is the technology OS uses to load DLLs. Its really low level stuff that come directly from OS kernel. It was a ton of fun. 

Also C is easier because obviously there is a ton of docs and all you have to do is copy paste. In case of UFFI you have to first figure out the C code and then how to convert it to UFFI so its double the effort for little gain. Its much better thing to just wrap C code and minimize the usage of UFFI. 

The benefits of Pharo are lost anyway as soon as you drop so low level.  

On Thu, Mar 16, 2017 at 3:05 PM <[hidden email]> wrote:
Indeed, experimenting using some combination of AddDllDirectory(),
SetDllDirectory() and LoadLibraryEx() was in my plan for next week.

Thanks



On 16/03/17 11:59, Mark Bestley wrote:
> Yes the calling process can alter where these indirect DLLs are loaded
> from and so should be able to be done in UFFI and directly from Smalltalk
>
> The OS will load the DLL specified then it runs DllMain and doing that
> causes the OS to load any DLLs that it calls
>
> MSDN https://msdn.microsoft.com/en-us/library/7d83bc18.aspx says
>
> With both implicit and explicit linking, Windows first searches for
> "known DLLs", such as Kernel32.dll and User32.dll. Windows then searches
> for the DLLs in the following sequence:
> The directory where the executable module for the current process is
> located.
> The current directory.
> The Windows system directory. The GetSystemDirectory function retrieves
> the path of this directory.
> The Windows directory. The GetWindowsDirectory function retrieves the
> path of this directory.
> The directories listed in the PATH environment variable.
>
>
> This can be changed using the call LoadLibraryEx instead of LoadLibrary
> Flags can be set to change the directories used
>
> https://msdn.microsoft.com/en-us/library/windows/desktop/ms684179(v=vs.85).aspx
>
>
>
> Also see AddDllDirectory
> https://msdn.microsoft.com/en-us/library/windows/desktop/hh310513(v=vs.85).aspx
>
>
> Mark
>
> On 15/03/2017 17:22, Raffaello Giulietti wrote:
>> Hi Dimitris,
>>
>> I don't get your point about proper C coding, C performance or Pharo
>> crashes. It is not the focus of this thread.
>>
>> How dependencies between libraries are resolved at load-time is dictated
>> by the OS. If the OS offers a working mechanism to specify folders that
>> the OS uses to search for libraries at load-time, well, I think that
>> Pharo, hence the UFFI, should offer it as well and in a more fashionable
>> format.
>> UFFI is quite elegant in many respects but I wouldn't call it a
>> minimalist design (zero hand holding, if I understand you correctly).
>> But I'm happy the extra elegance is there because it makes life more
>> comfortable.
>>
>> Again, I still have to experiment with SetDllDirectory() in the case of
>> Windows, so I'm not in a position to say something authoritative yet.
>>
>>
>>
>>
>>
>>
>>
>> On 2017-03-15 17:41, Dimitris Chloupis wrote:
>>> Please note the moment you use a FFI of any language you step to a C
>>> territory. VMs tend not to touch the code because the code is already
>>> in machine code format.
>>>
>>> There is zero reason for UFFI to worry about the dependencies of a
>>> library loaded, this must happen at the library level that UFFI does
>>> not focus on.
>>>
>>> What you describe here is standard practice because the vast majority
>>> of libraries depend on other libraries. In the end UFFI or any FFI in
>>> any language is not there to replace proper C coding but rather to
>>> allow you to utilize C libraries. You cannot hope to achieve
>>> performance comparable to C if you try to use a language that is not C
>>> to do standard things like dependency management.
>>>
>>> If the library fail to load it wont be because of UFFI it will be
>>> because the library is buggy at handling its dependencies. That means
>>> that the library will be buggy even if you use it from C directly.
>>>
>>> Even you do not know what you doing at C level, you going to crash
>>> Pharo countless times anyway and the least of your worries will be
>>> handling dependencies which far more obvious than some other nasty C
>>> bugs.
>>>
>>> I think its a great idea that UFFI does zero hand holding. This way it
>>> gives the clear message that if you do not understand C then you
>>> should not be using the UFFI in the first place.
>>>
>>> On Wed, Mar 15, 2017 at 4:51 PM Esteban Lorenzano
>>> <[hidden email]
>>> <mailto:[hidden email]>> wrote:
>>>
>>>
>>>     > On 15 Mar 2017, at 15:44, Raffaello Giulietti
>>>     <[hidden email]
>>>     <mailto:[hidden email]>>
>>>     wrote:
>>>     >
>>>     > On 2017-03-15 15:20, Esteban Lorenzano wrote:
>>>     >>
>>>     >>> On 15 Mar 2017, at 15:08, Raffaello Giulietti
>>>     <[hidden email]
>>>     <mailto:[hidden email]>>
>>>     wrote:
>>>     >>>
>>>     >>> Hi Esteban,
>>>     >>>
>>>     >>> I understand this is the current status of the UFFI, so I can
>>>     certainly use the workarounds discussed below.
>>>     >>>
>>>     >>> But I hope UFFI will soon offer a more "declarative" way to
>>>     specify the search folders, kind of LD_LIBRARY_PATH mechanism but
>>>     in the UFFI.
>>>     >>
>>>     >> that will NOT solve the problem of dependencies of libraries
>>>     indirectly references.
>>>     >> There is no way to do that (indirect reference solving) on an
>>>     executing program that I know.
>>>     >> So is not “current status of UFFI”: it will never provide that.
>>>     >>
>>>     >
>>>     > Do you mean that SetDllDirectory() has no influence when invoked
>>>     between Pharo's start and the LoadLibrary() call done by the UFFI?
>>>
>>>     I really have no idea. You can try to add it to your app.
>>>
>>>     Esteban
>>>
>>>     >
>>>     >
>>>     >
>>>     >
>>>     >> All that, unless someone come and says: "look, Esteban is like
>>>     this”.
>>>     >>
>>>     >> I spent last three weeks (not full time of course) trying to do
>>>     exactly that for our current linux vm, to force libgit2 to use the
>>>     version of libssh2 we want (and not the version on system)… and I
>>>     failed. I solved some cases, but not all. So at the end I opted to
>>>     change zeroconf to define LD_LIBRARY_PATH before calling pharo.
>>>     Only way I found I can force the lookup paths.
>>>     >>
>>>     >> cheers,
>>>     >> Esteban
>>>     >>
>>>     >>>
>>>     >>>
>>>     >>>
>>>     >>> Greetings
>>>     >>> Raffaello
>>>     >>>
>>>     >>>
>>>     >>>
>>>     >>>
>>>     >>> On 2017-03-15 14:52, Esteban Lorenzano wrote:
>>>     >>>> Hi,
>>>     >>>>
>>>     >>>> UFFI cannot do what you want.
>>>     >>>> If I understand well, you have:
>>>     >>>>
>>>     >>>> Pharo -> libA.dll -> libB.dll
>>>     >>>>
>>>     >>>> Pharo does a LoadLibrary(libA.dll), but has no control on how
>>>     libA calls libB… and is not possible for us to influence it more
>>>     than the predefined platform ways.
>>>     >>>>
>>>     >>>> One posible workaround (just possible, no idea if it will
>>>     work) is to perform a load of libB before is required by libB.
>>>     Then hopefully the loader will understand is same library and will
>>>     answer same handler (but not sure, because if it discriminates by
>>>     path… then you’re in the same position). Anyway, if you want to
>>>     try it, it would be something like this:
>>>     >>>>
>>>     >>>> DynamicLoader loadLibrary: 'path/to/libB.dll’.
>>>     >>>> DynamicLoader loadLibrary: 'path/to/libA.dll’.
>>>     >>>>
>>>     >>>> then you can try to execute functions of libA…
>>>     >>>>
>>>     >>>> cheers,
>>>     >>>> Esteban
>>>     >>>>
>>>     >>>>
>>>     >>>>> On 15 Mar 2017, at 14:32, Raffaello Giulietti
>>>     <[hidden email]
>>>     <mailto:[hidden email]>>
>>>     wrote:
>>>     >>>>>
>>>     >>>>> On 2017-03-15 13:51, Ben Coman wrote:
>>>     >>>>>>
>>>     >>>>>>
>>>     >>>>>> On Wed, Mar 15, 2017 at 4:42 PM, Raffaello Giulietti
>>>     >>>>>>
>>>     <[hidden email]
>>>     <mailto:[hidden email]>
>>>     >>>>>>
>>>     <mailto:[hidden email]
>>> <mailto:[hidden email]>>>
>>>     wrote:
>>>     >>>>>>
>>>     >>>>>>  Hi Ben,
>>>     >>>>>>
>>>     >>>>>>  my understanding is that SetDllDirectory only affects the
>>>     search
>>>     >>>>>>  path of libraries that are loaded at *run-time* with
>>>     LoadLibrary.
>>>     >>>>>>
>>>     >>>>>>  What I'm asking for is a mechanism that works at
>>>     *load-time*, when a
>>>     >>>>>>  library I'm accessing directly depends on another one
>>>     which I'm not
>>>     >>>>>>  targeting directly.
>>>     >>>>>>
>>>     >>>>>>
>>>     >>>>>> I don't quite follow.  I'm not intimate with Windows
>>>     dynamic loading, so
>>>     >>>>>> these questions are an opportunity for me to learn...
>>>     >>>>>>
>>>     >>>>>> You can't mean when Pharo loads, because it wouldn't know
>>>     of OTHER.DLL ?
>>>     >>>>>>
>>>     >>>>>> Do you mean that LoadLibrary is not called explicitly by
>>>     YOUR.DLL before
>>>     >>>>>> its uses the function from OTHER.DLL?
>>>     >>>>>>
>>>     >>>>>
>>>     >>>>> During the build of YOUR.DLL, you can specify that it should
>>>     depend on OTHER.LIB (kind of binary which lists function headers
>>>     and static data declarations for OTHER.DLL). Linking resolution is
>>>     then done at load-time of YOUR.DLL, not by invoking
>>>     LoadLibrary("OTHER.DLL").
>>>     >>>>>
>>>     >>>>> When YOUR.DLL is loaded by UFFI at run-time (here by means
>>>     of LoadLibrary(), I guess), Windows (not the UFFI) also attempts
>>>     to load OTHER.DLL because of the linking information gathered
>>>     during the build. Notice that the linking information does *not*
>>>     include a specific path for OTHER.DLL.
>>>     >>>>>
>>>     >>>>> (You don't have to mention standard *.LIBs like Kernel32.lib
>>>     because they are included by default. But of course, Windows knows
>>>     where to find the corresponding .dll.)
>>>     >>>>>
>>>     >>>>> OTHER.DLL is searched using the strategy described in the
>>>     MSDN page you mention. My expectation is/was that UFFI has/had an
>>>     API for specifying the search paths where to find OTHER.DLL in a
>>>     platform-independent way. In other words, I would expect the UFFI
>>>     to do the call of SetDllDirectory() or whatever is needed on
>>>     Windows for me or to use other mechanisms on other platforms.
>>>     >>>>>
>>>     >>>>>
>>>     >>>>>
>>>     >>>>>> Do you mean you link OTHER.DLL to YOUR.DLL at compile time?
>>>     >>>>>> But still, YOUR.DLL it not loaded until called via Pharo
>>>     FFI, so neither
>>>     >>>>>> is OTHER.DLL,
>>>     >>>>>> and Dynamic-Link Library Search Order [1] says... "If a DLL
>>>     with the
>>>     >>>>>> same module name
>>>     >>>>>> is already loaded in memory, the system uses the loaded
>>>     DLL, no matter
>>>     >>>>>> which directory it is in.
>>>     >>>>>> The system does not search for the DLL."
>>>     >>>>>>
>>>     >>>>>> So maybe before calling any of YOUR.DLL,
>>>     >>>>>> do a SetDllDirectory followed by a LoadLibrary(OTHER.DLL)
>>>     >>>>>> and later when you call into YOUR.DLL,
>>>     >>>>>> OTHER.DLL is already loaded.
>>>     >>>>>>
>>>     >>>>>
>>>     >>>>> Yes, this might work but is rather contrived.
>>>     >>>>>
>>>     >>>>>
>>>     >>>>>
>>>     >>>>>> Otherwise it would seems(?) your options are either Manifests
>>>     >>>>>> or Dynamic-Link Library Redirection, both done outside of
>>>     Pharo.
>>>     >>>>>>
>>>     >>>>>
>>>     >>>>> What if neither YOUR.DLL nor OTHER.DLL are under my control?
>>>     The only thing I know is where the .dll and the corresponding .lib
>>>     + .h are.
>>>     >>>>>
>>>     >>>>>
>>>     >>>>>
>>>     >>>>> Thanks for your interest
>>>     >>>>> Raffaello
>>>     >>>>>
>>>     >>>>>
>>>     >>>>>
>>>     >>>>>
>>>     >>>>>>  But I'll try whether SetDllDirectory also affects
>>>     load-time searches.
>>>     >>>>>>
>>>     >>>>>>  Anyway, I hope future versions of the UFFI to offer
>>>     setting the
>>>     >>>>>>  search paths more easily.
>>>     >>>>>>
>>>     >>>>>>
>>>     >>>>>>
>>>     >>>>>>  Greetings
>>>     >>>>>>  Raffaello
>>>     >>>>>>
>>>     >>>>>
>>>     >>>>>
>>>     >>>>
>>>     >>>>
>>>     >>>
>>>     >>>
>>>     >>
>>>     >>
>>>     >
>>>     >
>>>
>>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Specifying library dependencies in UFFI

raffaello.giulietti
Hi Dimitris,

I think I understand what you mean.

You are probably saying that, when facing multiple libraries that need
to be glued together, it is far better to implement a C "gluing" library
and then targeting that one from the UFFI rather than trying to do the
gluing code using Smalltalk and the UFFI directly, right?

I can only agree with you that, except for the most simplest cases, a
gluing C bridge library is far better and simpler to get right.

Personally, I've got no problems in using the UFFI for now, except for C
varargs (the ... in a C function header), for invoking functions through
pointers rather than by name and for indirectly handling C++ exceptions.
For that I had to resort to a thin gluing C library written just for
that purpose.

Greetings
Raffaello



On 16/03/17 15:48, Dimitris Chloupis wrote:

> My point was that anything you do in Pharo will lower performance quite
> a bit, but it wont be noticable anyway unless its in a big loop or
> something.
>
> Second that Pharo cannot debug such issues which is a huge deal when you
> work with OS stuff and OS calls.
>
> You are far better doing this outside UFFI and straight inside a C IDE
> because you will have an army of tools that will make your life a ton
> easier.
>
> Hence why I talked about performance, C coding and Pharo crashes.
>
> In my case I had to write and test the code in C to port it to UFFI
> because it was impossible to resolve the bugs from inside Pharo. Also
> crashing Pharo all the time got really annoying and I managed to corrupt
> a couple of images too in the process.
>
> Of course debugging C code from a C IDE is quite similar to how you work
> with Pharo.
>
> I also got an experience with loading libraries because I worked with
> Shared memory mapped files which is the technology OS uses to load DLLs.
> Its really low level stuff that come directly from OS kernel. It was a
> ton of fun.
>
> Also C is easier because obviously there is a ton of docs and all you
> have to do is copy paste. In case of UFFI you have to first figure out
> the C code and then how to convert it to UFFI so its double the effort
> for little gain. Its much better thing to just wrap C code and minimize
> the usage of UFFI.
>
> The benefits of Pharo are lost anyway as soon as you drop so low level.  
>
> On Thu, Mar 16, 2017 at 3:05 PM <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Indeed, experimenting using some combination of AddDllDirectory(),
>     SetDllDirectory() and LoadLibraryEx() was in my plan for next week.
>
>     Thanks
>
>
>
>     On 16/03/17 11:59, Mark Bestley wrote:
>     > Yes the calling process can alter where these indirect DLLs are loaded
>     > from and so should be able to be done in UFFI and directly from
>     Smalltalk
>     >
>     > The OS will load the DLL specified then it runs DllMain and doing that
>     > causes the OS to load any DLLs that it calls
>     >
>     > MSDN https://msdn.microsoft.com/en-us/library/7d83bc18.aspx says
>     >
>     > With both implicit and explicit linking, Windows first searches for
>     > "known DLLs", such as Kernel32.dll and User32.dll. Windows then
>     searches
>     > for the DLLs in the following sequence:
>     > The directory where the executable module for the current process is
>     > located.
>     > The current directory.
>     > The Windows system directory. The GetSystemDirectory function
>     retrieves
>     > the path of this directory.
>     > The Windows directory. The GetWindowsDirectory function retrieves the
>     > path of this directory.
>     > The directories listed in the PATH environment variable.
>     >
>     >
>     > This can be changed using the call LoadLibraryEx instead of
>     LoadLibrary
>     > Flags can be set to change the directories used
>     >
>     >
>     https://msdn.microsoft.com/en-us/library/windows/desktop/ms684179(v=vs.85).aspx
>     >
>     >
>     >
>     > Also see AddDllDirectory
>     >
>     https://msdn.microsoft.com/en-us/library/windows/desktop/hh310513(v=vs.85).aspx
>     >
>     >
>     > Mark
>     >
>     > On 15/03/2017 17:22, Raffaello Giulietti wrote:
>     >> Hi Dimitris,
>     >>
>     >> I don't get your point about proper C coding, C performance or Pharo
>     >> crashes. It is not the focus of this thread.
>     >>
>     >> How dependencies between libraries are resolved at load-time is
>     dictated
>     >> by the OS. If the OS offers a working mechanism to specify
>     folders that
>     >> the OS uses to search for libraries at load-time, well, I think that
>     >> Pharo, hence the UFFI, should offer it as well and in a more
>     fashionable
>     >> format.
>     >> UFFI is quite elegant in many respects but I wouldn't call it a
>     >> minimalist design (zero hand holding, if I understand you correctly).
>     >> But I'm happy the extra elegance is there because it makes life more
>     >> comfortable.
>     >>
>     >> Again, I still have to experiment with SetDllDirectory() in the
>     case of
>     >> Windows, so I'm not in a position to say something authoritative yet.
>     >>
>     >>
>     >>
>     >>
>     >>
>     >>
>     >>
>     >> On 2017-03-15 17:41, Dimitris Chloupis wrote:
>     >>> Please note the moment you use a FFI of any language you step to a C
>     >>> territory. VMs tend not to touch the code because the code is
>     already
>     >>> in machine code format.
>     >>>
>     >>> There is zero reason for UFFI to worry about the dependencies of a
>     >>> library loaded, this must happen at the library level that UFFI does
>     >>> not focus on.
>     >>>
>     >>> What you describe here is standard practice because the vast
>     majority
>     >>> of libraries depend on other libraries. In the end UFFI or any
>     FFI in
>     >>> any language is not there to replace proper C coding but rather to
>     >>> allow you to utilize C libraries. You cannot hope to achieve
>     >>> performance comparable to C if you try to use a language that is
>     not C
>     >>> to do standard things like dependency management.
>     >>>
>     >>> If the library fail to load it wont be because of UFFI it will be
>     >>> because the library is buggy at handling its dependencies. That
>     means
>     >>> that the library will be buggy even if you use it from C directly.
>     >>>
>     >>> Even you do not know what you doing at C level, you going to crash
>     >>> Pharo countless times anyway and the least of your worries will be
>     >>> handling dependencies which far more obvious than some other nasty C
>     >>> bugs.
>     >>>
>     >>> I think its a great idea that UFFI does zero hand holding. This
>     way it
>     >>> gives the clear message that if you do not understand C then you
>     >>> should not be using the UFFI in the first place.
>     >>>
>     >>> On Wed, Mar 15, 2017 at 4:51 PM Esteban Lorenzano
>     >>> <[hidden email] <mailto:[hidden email]>
>     >>> <mailto:[hidden email] <mailto:[hidden email]>>> wrote:
>     >>>
>     >>>
>     >>>     > On 15 Mar 2017, at 15:44, Raffaello Giulietti
>     >>>     <[hidden email]
>     <mailto:[hidden email]>
>     >>>     <mailto:[hidden email]
>     <mailto:[hidden email]>>>
>     >>>     wrote:
>     >>>     >
>     >>>     > On 2017-03-15 15:20, Esteban Lorenzano wrote:
>     >>>     >>
>     >>>     >>> On 15 Mar 2017, at 15:08, Raffaello Giulietti
>     >>>     <[hidden email]
>     <mailto:[hidden email]>
>     >>>     <mailto:[hidden email]
>     <mailto:[hidden email]>>>
>     >>>     wrote:
>     >>>     >>>
>     >>>     >>> Hi Esteban,
>     >>>     >>>
>     >>>     >>> I understand this is the current status of the UFFI, so
>     I can
>     >>>     certainly use the workarounds discussed below.
>     >>>     >>>
>     >>>     >>> But I hope UFFI will soon offer a more "declarative" way to
>     >>>     specify the search folders, kind of LD_LIBRARY_PATH
>     mechanism but
>     >>>     in the UFFI.
>     >>>     >>
>     >>>     >> that will NOT solve the problem of dependencies of libraries
>     >>>     indirectly references.
>     >>>     >> There is no way to do that (indirect reference solving) on an
>     >>>     executing program that I know.
>     >>>     >> So is not “current status of UFFI”: it will never provide
>     that.
>     >>>     >>
>     >>>     >
>     >>>     > Do you mean that SetDllDirectory() has no influence when
>     invoked
>     >>>     between Pharo's start and the LoadLibrary() call done by the
>     UFFI?
>     >>>
>     >>>     I really have no idea. You can try to add it to your app.
>     >>>
>     >>>     Esteban
>     >>>
>     >>>     >
>     >>>     >
>     >>>     >
>     >>>     >
>     >>>     >> All that, unless someone come and says: "look, Esteban is
>     like
>     >>>     this”.
>     >>>     >>
>     >>>     >> I spent last three weeks (not full time of course) trying
>     to do
>     >>>     exactly that for our current linux vm, to force libgit2 to
>     use the
>     >>>     version of libssh2 we want (and not the version on system)…
>     and I
>     >>>     failed. I solved some cases, but not all. So at the end I
>     opted to
>     >>>     change zeroconf to define LD_LIBRARY_PATH before calling pharo.
>     >>>     Only way I found I can force the lookup paths.
>     >>>     >>
>     >>>     >> cheers,
>     >>>     >> Esteban
>     >>>     >>
>     >>>     >>>
>     >>>     >>>
>     >>>     >>>
>     >>>     >>> Greetings
>     >>>     >>> Raffaello
>     >>>     >>>
>     >>>     >>>
>     >>>     >>>
>     >>>     >>>
>     >>>     >>> On 2017-03-15 14:52, Esteban Lorenzano wrote:
>     >>>     >>>> Hi,
>     >>>     >>>>
>     >>>     >>>> UFFI cannot do what you want.
>     >>>     >>>> If I understand well, you have:
>     >>>     >>>>
>     >>>     >>>> Pharo -> libA.dll -> libB.dll
>     >>>     >>>>
>     >>>     >>>> Pharo does a LoadLibrary(libA.dll), but has no control
>     on how
>     >>>     libA calls libB… and is not possible for us to influence it more
>     >>>     than the predefined platform ways.
>     >>>     >>>>
>     >>>     >>>> One posible workaround (just possible, no idea if it will
>     >>>     work) is to perform a load of libB before is required by libB.
>     >>>     Then hopefully the loader will understand is same library
>     and will
>     >>>     answer same handler (but not sure, because if it
>     discriminates by
>     >>>     path… then you’re in the same position). Anyway, if you want to
>     >>>     try it, it would be something like this:
>     >>>     >>>>
>     >>>     >>>> DynamicLoader loadLibrary: 'path/to/libB.dll’.
>     >>>     >>>> DynamicLoader loadLibrary: 'path/to/libA.dll’.
>     >>>     >>>>
>     >>>     >>>> then you can try to execute functions of libA…
>     >>>     >>>>
>     >>>     >>>> cheers,
>     >>>     >>>> Esteban
>     >>>     >>>>
>     >>>     >>>>
>     >>>     >>>>> On 15 Mar 2017, at 14:32, Raffaello Giulietti
>     >>>     <[hidden email]
>     <mailto:[hidden email]>
>     >>>     <mailto:[hidden email]
>     <mailto:[hidden email]>>>
>     >>>     wrote:
>     >>>     >>>>>
>     >>>     >>>>> On 2017-03-15 13:51, Ben Coman wrote:
>     >>>     >>>>>>
>     >>>     >>>>>>
>     >>>     >>>>>> On Wed, Mar 15, 2017 at 4:42 PM, Raffaello Giulietti
>     >>>     >>>>>>
>     >>>     <[hidden email]
>     <mailto:[hidden email]>
>     >>>     <mailto:[hidden email]
>     <mailto:[hidden email]>>
>     >>>     >>>>>>
>     >>>     <mailto:[hidden email]
>     <mailto:[hidden email]>
>     >>> <mailto:[hidden email]
>     <mailto:[hidden email]>>>>
>     >>>     wrote:
>     >>>     >>>>>>
>     >>>     >>>>>>  Hi Ben,
>     >>>     >>>>>>
>     >>>     >>>>>>  my understanding is that SetDllDirectory only
>     affects the
>     >>>     search
>     >>>     >>>>>>  path of libraries that are loaded at *run-time* with
>     >>>     LoadLibrary.
>     >>>     >>>>>>
>     >>>     >>>>>>  What I'm asking for is a mechanism that works at
>     >>>     *load-time*, when a
>     >>>     >>>>>>  library I'm accessing directly depends on another one
>     >>>     which I'm not
>     >>>     >>>>>>  targeting directly.
>     >>>     >>>>>>
>     >>>     >>>>>>
>     >>>     >>>>>> I don't quite follow.  I'm not intimate with Windows
>     >>>     dynamic loading, so
>     >>>     >>>>>> these questions are an opportunity for me to learn...
>     >>>     >>>>>>
>     >>>     >>>>>> You can't mean when Pharo loads, because it wouldn't know
>     >>>     of OTHER.DLL ?
>     >>>     >>>>>>
>     >>>     >>>>>> Do you mean that LoadLibrary is not called explicitly by
>     >>>     YOUR.DLL before
>     >>>     >>>>>> its uses the function from OTHER.DLL?
>     >>>     >>>>>>
>     >>>     >>>>>
>     >>>     >>>>> During the build of YOUR.DLL, you can specify that it
>     should
>     >>>     depend on OTHER.LIB (kind of binary which lists function headers
>     >>>     and static data declarations for OTHER.DLL). Linking
>     resolution is
>     >>>     then done at load-time of YOUR.DLL, not by invoking
>     >>>     LoadLibrary("OTHER.DLL").
>     >>>     >>>>>
>     >>>     >>>>> When YOUR.DLL is loaded by UFFI at run-time (here by means
>     >>>     of LoadLibrary(), I guess), Windows (not the UFFI) also attempts
>     >>>     to load OTHER.DLL because of the linking information gathered
>     >>>     during the build. Notice that the linking information does *not*
>     >>>     include a specific path for OTHER.DLL.
>     >>>     >>>>>
>     >>>     >>>>> (You don't have to mention standard *.LIBs like
>     Kernel32.lib
>     >>>     because they are included by default. But of course, Windows
>     knows
>     >>>     where to find the corresponding .dll.)
>     >>>     >>>>>
>     >>>     >>>>> OTHER.DLL is searched using the strategy described in the
>     >>>     MSDN page you mention. My expectation is/was that UFFI
>     has/had an
>     >>>     API for specifying the search paths where to find OTHER.DLL in a
>     >>>     platform-independent way. In other words, I would expect the
>     UFFI
>     >>>     to do the call of SetDllDirectory() or whatever is needed on
>     >>>     Windows for me or to use other mechanisms on other platforms.
>     >>>     >>>>>
>     >>>     >>>>>
>     >>>     >>>>>
>     >>>     >>>>>> Do you mean you link OTHER.DLL to YOUR.DLL at compile
>     time?
>     >>>     >>>>>> But still, YOUR.DLL it not loaded until called via Pharo
>     >>>     FFI, so neither
>     >>>     >>>>>> is OTHER.DLL,
>     >>>     >>>>>> and Dynamic-Link Library Search Order [1] says... "If
>     a DLL
>     >>>     with the
>     >>>     >>>>>> same module name
>     >>>     >>>>>> is already loaded in memory, the system uses the loaded
>     >>>     DLL, no matter
>     >>>     >>>>>> which directory it is in.
>     >>>     >>>>>> The system does not search for the DLL."
>     >>>     >>>>>>
>     >>>     >>>>>> So maybe before calling any of YOUR.DLL,
>     >>>     >>>>>> do a SetDllDirectory followed by a LoadLibrary(OTHER.DLL)
>     >>>     >>>>>> and later when you call into YOUR.DLL,
>     >>>     >>>>>> OTHER.DLL is already loaded.
>     >>>     >>>>>>
>     >>>     >>>>>
>     >>>     >>>>> Yes, this might work but is rather contrived.
>     >>>     >>>>>
>     >>>     >>>>>
>     >>>     >>>>>
>     >>>     >>>>>> Otherwise it would seems(?) your options are either
>     Manifests
>     >>>     >>>>>> or Dynamic-Link Library Redirection, both done outside of
>     >>>     Pharo.
>     >>>     >>>>>>
>     >>>     >>>>>
>     >>>     >>>>> What if neither YOUR.DLL nor OTHER.DLL are under my
>     control?
>     >>>     The only thing I know is where the .dll and the
>     corresponding .lib
>     >>>     + .h are.
>     >>>     >>>>>
>     >>>     >>>>>
>     >>>     >>>>>
>     >>>     >>>>> Thanks for your interest
>     >>>     >>>>> Raffaello
>     >>>     >>>>>
>     >>>     >>>>>
>     >>>     >>>>>
>     >>>     >>>>>
>     >>>     >>>>>>  But I'll try whether SetDllDirectory also affects
>     >>>     load-time searches.
>     >>>     >>>>>>
>     >>>     >>>>>>  Anyway, I hope future versions of the UFFI to offer
>     >>>     setting the
>     >>>     >>>>>>  search paths more easily.
>     >>>     >>>>>>
>     >>>     >>>>>>
>     >>>     >>>>>>
>     >>>     >>>>>>  Greetings
>     >>>     >>>>>>  Raffaello
>     >>>     >>>>>>
>     >>>     >>>>>
>     >>>     >>>>>
>     >>>     >>>>
>     >>>     >>>>
>     >>>     >>>
>     >>>     >>>
>     >>>     >>
>     >>>     >>
>     >>>     >
>     >>>     >
>     >>>
>     >>>
>     >>
>     >
>     >
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Specifying library dependencies in UFFI

bestlem
In general I would agree do in C or C++ to cut down the interface and
pass as little data between the two languages as possible. So hiding of
C++ objects and exceptions and the others listed in you last paragraph
is reasonable

However the OS functions for loading need to be controlled outside the
function called and so control should be in the UFFI layer. Also
calling DLLs from Pharo will have a different environment from DLL
called from a C main() call so can't be debugged in the C IDE. (My
experience in this comes from DLLs called from Excel  so a similar case,
you end up running the excel executable under the C debugger and set C
breakpoints and even then that differs from production use)

Mark

On 16/03/2017 16:19,
[hidden email] wrote:

> Hi Dimitris,
>
> I think I understand what you mean.
>
> You are probably saying that, when facing multiple libraries that need
> to be glued together, it is far better to implement a C "gluing" library
> and then targeting that one from the UFFI rather than trying to do the
> gluing code using Smalltalk and the UFFI directly, right?
>
> I can only agree with you that, except for the most simplest cases, a
> gluing C bridge library is far better and simpler to get right.
>
> Personally, I've got no problems in using the UFFI for now, except for C
> varargs (the ... in a C function header), for invoking functions through
> pointers rather than by name and for indirectly handling C++ exceptions.
> For that I had to resort to a thin gluing C library written just for
> that purpose.
>
> Greetings
> Raffaello
>
>
>
> On 16/03/17 15:48, Dimitris Chloupis wrote:
>> My point was that anything you do in Pharo will lower performance quite
>> a bit, but it wont be noticable anyway unless its in a big loop or
>> something.
>>
>> Second that Pharo cannot debug such issues which is a huge deal when you
>> work with OS stuff and OS calls.
>>
>> You are far better doing this outside UFFI and straight inside a C IDE
>> because you will have an army of tools that will make your life a ton
>> easier.
>>
>> Hence why I talked about performance, C coding and Pharo crashes.
>>
>> In my case I had to write and test the code in C to port it to UFFI
>> because it was impossible to resolve the bugs from inside Pharo. Also
>> crashing Pharo all the time got really annoying and I managed to corrupt
>> a couple of images too in the process.
>>
>> Of course debugging C code from a C IDE is quite similar to how you work
>> with Pharo.
>>
>> I also got an experience with loading libraries because I worked with
>> Shared memory mapped files which is the technology OS uses to load DLLs.
>> Its really low level stuff that come directly from OS kernel. It was a
>> ton of fun.
>>
>> Also C is easier because obviously there is a ton of docs and all you
>> have to do is copy paste. In case of UFFI you have to first figure out
>> the C code and then how to convert it to UFFI so its double the effort
>> for little gain. Its much better thing to just wrap C code and minimize
>> the usage of UFFI.
>>
>> The benefits of Pharo are lost anyway as soon as you drop so low level.
>>
>> On Thu, Mar 16, 2017 at 3:05 PM <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>>     Indeed, experimenting using some combination of AddDllDirectory(),
>>     SetDllDirectory() and LoadLibraryEx() was in my plan for next week.
>>
>>     Thanks
>>
>>
>>
>>     On 16/03/17 11:59, Mark Bestley wrote:
>>     > Yes the calling process can alter where these indirect DLLs are loaded
>>     > from and so should be able to be done in UFFI and directly from
>>     Smalltalk
>>     >
>>     > The OS will load the DLL specified then it runs DllMain and doing that
>>     > causes the OS to load any DLLs that it calls
>>     >
>>     > MSDN https://msdn.microsoft.com/en-us/library/7d83bc18.aspx says
>>     >
>>     > With both implicit and explicit linking, Windows first searches for
>>     > "known DLLs", such as Kernel32.dll and User32.dll. Windows then
>>     searches
>>     > for the DLLs in the following sequence:
>>     > The directory where the executable module for the current process is
>>     > located.
>>     > The current directory.
>>     > The Windows system directory. The GetSystemDirectory function
>>     retrieves
>>     > the path of this directory.
>>     > The Windows directory. The GetWindowsDirectory function retrieves the
>>     > path of this directory.
>>     > The directories listed in the PATH environment variable.
>>     >
>>     >
>>     > This can be changed using the call LoadLibraryEx instead of
>>     LoadLibrary
>>     > Flags can be set to change the directories used
>>     >
>>     >
>>     https://msdn.microsoft.com/en-us/library/windows/desktop/ms684179(v=vs.85).aspx
>>     >
>>     >
>>     >
>>     > Also see AddDllDirectory
>>     >
>>     https://msdn.microsoft.com/en-us/library/windows/desktop/hh310513(v=vs.85).aspx
>>     >
>>     >
>>     > Mark
>>     >
>>     > On 15/03/2017 17:22, Raffaello Giulietti wrote:
>>     >> Hi Dimitris,
>>     >>
>>     >> I don't get your point about proper C coding, C performance or Pharo
>>     >> crashes. It is not the focus of this thread.
>>     >>
>>     >> How dependencies between libraries are resolved at load-time is
>>     dictated
>>     >> by the OS. If the OS offers a working mechanism to specify
>>     folders that
>>     >> the OS uses to search for libraries at load-time, well, I think that
>>     >> Pharo, hence the UFFI, should offer it as well and in a more
>>     fashionable
>>     >> format.
>>     >> UFFI is quite elegant in many respects but I wouldn't call it a
>>     >> minimalist design (zero hand holding, if I understand you correctly).
>>     >> But I'm happy the extra elegance is there because it makes life more
>>     >> comfortable.
>>     >>
>>     >> Again, I still have to experiment with SetDllDirectory() in the
>>     case of
>>     >> Windows, so I'm not in a position to say something authoritative yet.
>>     >>
>>     >>
>>     >>
>>     >>
>>     >>
>>     >>
>>     >>
>>     >> On 2017-03-15 17:41, Dimitris Chloupis wrote:
>>     >>> Please note the moment you use a FFI of any language you step to a C
>>     >>> territory. VMs tend not to touch the code because the code is
>>     already
>>     >>> in machine code format.
>>     >>>
>>     >>> There is zero reason for UFFI to worry about the dependencies of a
>>     >>> library loaded, this must happen at the library level that UFFI does
>>     >>> not focus on.
>>     >>>
>>     >>> What you describe here is standard practice because the vast
>>     majority
>>     >>> of libraries depend on other libraries. In the end UFFI or any
>>     FFI in
>>     >>> any language is not there to replace proper C coding but rather to
>>     >>> allow you to utilize C libraries. You cannot hope to achieve
>>     >>> performance comparable to C if you try to use a language that is
>>     not C
>>     >>> to do standard things like dependency management.
>>     >>>
>>     >>> If the library fail to load it wont be because of UFFI it will be
>>     >>> because the library is buggy at handling its dependencies. That
>>     means
>>     >>> that the library will be buggy even if you use it from C directly.
>>     >>>
>>     >>> Even you do not know what you doing at C level, you going to crash
>>     >>> Pharo countless times anyway and the least of your worries will be
>>     >>> handling dependencies which far more obvious than some other nasty C
>>     >>> bugs.
>>     >>>
>>     >>> I think its a great idea that UFFI does zero hand holding. This
>>     way it
>>     >>> gives the clear message that if you do not understand C then you
>>     >>> should not be using the UFFI in the first place.
>>     >>>
>>     >>> On Wed, Mar 15, 2017 at 4:51 PM Esteban Lorenzano
>>     >>> <[hidden email] <mailto:[hidden email]>
>>     >>> <mailto:[hidden email] <mailto:[hidden email]>>> wrote:
>>     >>>
>>     >>>
>>     >>>     > On 15 Mar 2017, at 15:44, Raffaello Giulietti
>>     >>>     <[hidden email]
>>     <mailto:[hidden email]>
>>     >>>     <mailto:[hidden email]
>>     <mailto:[hidden email]>>>
>>     >>>     wrote:
>>     >>>     >
>>     >>>     > On 2017-03-15 15:20, Esteban Lorenzano wrote:
>>     >>>     >>
>>     >>>     >>> On 15 Mar 2017, at 15:08, Raffaello Giulietti
>>     >>>     <[hidden email]
>>     <mailto:[hidden email]>
>>     >>>     <mailto:[hidden email]
>>     <mailto:[hidden email]>>>
>>     >>>     wrote:
>>     >>>     >>>
>>     >>>     >>> Hi Esteban,
>>     >>>     >>>
>>     >>>     >>> I understand this is the current status of the UFFI, so
>>     I can
>>     >>>     certainly use the workarounds discussed below.
>>     >>>     >>>
>>     >>>     >>> But I hope UFFI will soon offer a more "declarative" way to
>>     >>>     specify the search folders, kind of LD_LIBRARY_PATH
>>     mechanism but
>>     >>>     in the UFFI.
>>     >>>     >>
>>     >>>     >> that will NOT solve the problem of dependencies of libraries
>>     >>>     indirectly references.
>>     >>>     >> There is no way to do that (indirect reference solving) on an
>>     >>>     executing program that I know.
>>     >>>     >> So is not “current status of UFFI”: it will never provide
>>     that.
>>     >>>     >>
>>     >>>     >
>>     >>>     > Do you mean that SetDllDirectory() has no influence when
>>     invoked
>>     >>>     between Pharo's start and the LoadLibrary() call done by the
>>     UFFI?
>>     >>>
>>     >>>     I really have no idea. You can try to add it to your app.
>>     >>>
>>     >>>     Esteban
>>     >>>
>>     >>>     >
>>     >>>     >
>>     >>>     >
>>     >>>     >
>>     >>>     >> All that, unless someone come and says: "look, Esteban is
>>     like
>>     >>>     this”.
>>     >>>     >>
>>     >>>     >> I spent last three weeks (not full time of course) trying
>>     to do
>>     >>>     exactly that for our current linux vm, to force libgit2 to
>>     use the
>>     >>>     version of libssh2 we want (and not the version on system)…
>>     and I
>>     >>>     failed. I solved some cases, but not all. So at the end I
>>     opted to
>>     >>>     change zeroconf to define LD_LIBRARY_PATH before calling pharo.
>>     >>>     Only way I found I can force the lookup paths.
>>     >>>     >>
>>     >>>     >> cheers,
>>     >>>     >> Esteban
>>     >>>     >>
>>     >>>     >>>
>>     >>>     >>>
>>     >>>     >>>
>>     >>>     >>> Greetings
>>     >>>     >>> Raffaello
>>     >>>     >>>
>>     >>>     >>>
>>     >>>     >>>
>>     >>>     >>>
>>     >>>     >>> On 2017-03-15 14:52, Esteban Lorenzano wrote:
>>     >>>     >>>> Hi,
>>     >>>     >>>>
>>     >>>     >>>> UFFI cannot do what you want.
>>     >>>     >>>> If I understand well, you have:
>>     >>>     >>>>
>>     >>>     >>>> Pharo -> libA.dll -> libB.dll
>>     >>>     >>>>
>>     >>>     >>>> Pharo does a LoadLibrary(libA.dll), but has no control
>>     on how
>>     >>>     libA calls libB… and is not possible for us to influence it more
>>     >>>     than the predefined platform ways.
>>     >>>     >>>>
>>     >>>     >>>> One posible workaround (just possible, no idea if it will
>>     >>>     work) is to perform a load of libB before is required by libB.
>>     >>>     Then hopefully the loader will understand is same library
>>     and will
>>     >>>     answer same handler (but not sure, because if it
>>     discriminates by
>>     >>>     path… then you’re in the same position). Anyway, if you want to
>>     >>>     try it, it would be something like this:
>>     >>>     >>>>
>>     >>>     >>>> DynamicLoader loadLibrary: 'path/to/libB.dll’.
>>     >>>     >>>> DynamicLoader loadLibrary: 'path/to/libA.dll’.
>>     >>>     >>>>
>>     >>>     >>>> then you can try to execute functions of libA…
>>     >>>     >>>>
>>     >>>     >>>> cheers,
>>     >>>     >>>> Esteban
>>     >>>     >>>>
>>     >>>     >>>>
>>     >>>     >>>>> On 15 Mar 2017, at 14:32, Raffaello Giulietti
>>     >>>     <[hidden email]
>>     <mailto:[hidden email]>
>>     >>>     <mailto:[hidden email]
>>     <mailto:[hidden email]>>>
>>     >>>     wrote:
>>     >>>     >>>>>
>>     >>>     >>>>> On 2017-03-15 13:51, Ben Coman wrote:
>>     >>>     >>>>>>
>>     >>>     >>>>>>
>>     >>>     >>>>>> On Wed, Mar 15, 2017 at 4:42 PM, Raffaello Giulietti
>>     >>>     >>>>>>
>>     >>>     <[hidden email]
>>     <mailto:[hidden email]>
>>     >>>     <mailto:[hidden email]
>>     <mailto:[hidden email]>>
>>     >>>     >>>>>>
>>     >>>     <mailto:[hidden email]
>>     <mailto:[hidden email]>
>>     >>> <mailto:[hidden email]
>>     <mailto:[hidden email]>>>>
>>     >>>     wrote:
>>     >>>     >>>>>>
>>     >>>     >>>>>>  Hi Ben,
>>     >>>     >>>>>>
>>     >>>     >>>>>>  my understanding is that SetDllDirectory only
>>     affects the
>>     >>>     search
>>     >>>     >>>>>>  path of libraries that are loaded at *run-time* with
>>     >>>     LoadLibrary.
>>     >>>     >>>>>>
>>     >>>     >>>>>>  What I'm asking for is a mechanism that works at
>>     >>>     *load-time*, when a
>>     >>>     >>>>>>  library I'm accessing directly depends on another one
>>     >>>     which I'm not
>>     >>>     >>>>>>  targeting directly.
>>     >>>     >>>>>>
>>     >>>     >>>>>>
>>     >>>     >>>>>> I don't quite follow.  I'm not intimate with Windows
>>     >>>     dynamic loading, so
>>     >>>     >>>>>> these questions are an opportunity for me to learn...
>>     >>>     >>>>>>
>>     >>>     >>>>>> You can't mean when Pharo loads, because it wouldn't know
>>     >>>     of OTHER.DLL ?
>>     >>>     >>>>>>
>>     >>>     >>>>>> Do you mean that LoadLibrary is not called explicitly by
>>     >>>     YOUR.DLL before
>>     >>>     >>>>>> its uses the function from OTHER.DLL?
>>     >>>     >>>>>>
>>     >>>     >>>>>
>>     >>>     >>>>> During the build of YOUR.DLL, you can specify that it
>>     should
>>     >>>     depend on OTHER.LIB (kind of binary which lists function headers
>>     >>>     and static data declarations for OTHER.DLL). Linking
>>     resolution is
>>     >>>     then done at load-time of YOUR.DLL, not by invoking
>>     >>>     LoadLibrary("OTHER.DLL").
>>     >>>     >>>>>
>>     >>>     >>>>> When YOUR.DLL is loaded by UFFI at run-time (here by means
>>     >>>     of LoadLibrary(), I guess), Windows (not the UFFI) also attempts
>>     >>>     to load OTHER.DLL because of the linking information gathered
>>     >>>     during the build. Notice that the linking information does *not*
>>     >>>     include a specific path for OTHER.DLL.
>>     >>>     >>>>>
>>     >>>     >>>>> (You don't have to mention standard *.LIBs like
>>     Kernel32.lib
>>     >>>     because they are included by default. But of course, Windows
>>     knows
>>     >>>     where to find the corresponding .dll.)
>>     >>>     >>>>>
>>     >>>     >>>>> OTHER.DLL is searched using the strategy described in the
>>     >>>     MSDN page you mention. My expectation is/was that UFFI
>>     has/had an
>>     >>>     API for specifying the search paths where to find OTHER.DLL in a
>>     >>>     platform-independent way. In other words, I would expect the
>>     UFFI
>>     >>>     to do the call of SetDllDirectory() or whatever is needed on
>>     >>>     Windows for me or to use other mechanisms on other platforms.
>>     >>>     >>>>>
>>     >>>     >>>>>
>>     >>>     >>>>>
>>     >>>     >>>>>> Do you mean you link OTHER.DLL to YOUR.DLL at compile
>>     time?
>>     >>>     >>>>>> But still, YOUR.DLL it not loaded until called via Pharo
>>     >>>     FFI, so neither
>>     >>>     >>>>>> is OTHER.DLL,
>>     >>>     >>>>>> and Dynamic-Link Library Search Order [1] says... "If
>>     a DLL
>>     >>>     with the
>>     >>>     >>>>>> same module name
>>     >>>     >>>>>> is already loaded in memory, the system uses the loaded
>>     >>>     DLL, no matter
>>     >>>     >>>>>> which directory it is in.
>>     >>>     >>>>>> The system does not search for the DLL."
>>     >>>     >>>>>>
>>     >>>     >>>>>> So maybe before calling any of YOUR.DLL,
>>     >>>     >>>>>> do a SetDllDirectory followed by a LoadLibrary(OTHER.DLL)
>>     >>>     >>>>>> and later when you call into YOUR.DLL,
>>     >>>     >>>>>> OTHER.DLL is already loaded.
>>     >>>     >>>>>>
>>     >>>     >>>>>
>>     >>>     >>>>> Yes, this might work but is rather contrived.
>>     >>>     >>>>>
>>     >>>     >>>>>
>>     >>>     >>>>>
>>     >>>     >>>>>> Otherwise it would seems(?) your options are either
>>     Manifests
>>     >>>     >>>>>> or Dynamic-Link Library Redirection, both done outside of
>>     >>>     Pharo.
>>     >>>     >>>>>>
>>     >>>     >>>>>
>>     >>>     >>>>> What if neither YOUR.DLL nor OTHER.DLL are under my
>>     control?
>>     >>>     The only thing I know is where the .dll and the
>>     corresponding .lib
>>     >>>     + .h are.
>>     >>>     >>>>>
>>     >>>     >>>>>
>>     >>>     >>>>>
>>     >>>     >>>>> Thanks for your interest
>>     >>>     >>>>> Raffaello
>>     >>>     >>>>>
>>     >>>     >>>>>
>>     >>>     >>>>>
>>     >>>     >>>>>
>>     >>>     >>>>>>  But I'll try whether SetDllDirectory also affects
>>     >>>     load-time searches.
>>     >>>     >>>>>>
>>     >>>     >>>>>>  Anyway, I hope future versions of the UFFI to offer
>>     >>>     setting the
>>     >>>     >>>>>>  search paths more easily.
>>     >>>     >>>>>>
>>     >>>     >>>>>>
>>     >>>     >>>>>>
>>     >>>     >>>>>>  Greetings
>>     >>>     >>>>>>  Raffaello
>>     >>>     >>>>>>
>>     >>>     >>>>>
>>     >>>     >>>>>
>>     >>>     >>>>
>>     >>>     >>>>
>>     >>>     >>>
>>     >>>     >>>
>>     >>>     >>
>>     >>>     >>
>>     >>>     >
>>     >>>     >
>>     >>>
>>     >>>
>>     >>
>>     >
>>     >
>>
>>
>
>
>


--
Mark


Reply | Threaded
Open this post in threaded view
|

Re: Specifying library dependencies in UFFI

kilon.alios
In reply to this post by raffaello.giulietti
A utility to view DLLs is this one


It helped a lot in my project. 

also C compilers have several checks for DLL loading 

Visual studio also has several tools for low level detection of memory if you are familiar with Assembly it has also disassembling tools. 

Anyway, your time, your project, your choice. 

On Thu, Mar 16, 2017 at 6:20 PM <[hidden email]> wrote:
Hi Dimitris,

I think I understand what you mean.

You are probably saying that, when facing multiple libraries that need
to be glued together, it is far better to implement a C "gluing" library
and then targeting that one from the UFFI rather than trying to do the
gluing code using Smalltalk and the UFFI directly, right?

I can only agree with you that, except for the most simplest cases, a
gluing C bridge library is far better and simpler to get right.

Personally, I've got no problems in using the UFFI for now, except for C
varargs (the ... in a C function header), for invoking functions through
pointers rather than by name and for indirectly handling C++ exceptions.
For that I had to resort to a thin gluing C library written just for
that purpose.

Greetings
Raffaello



On 16/03/17 15:48, Dimitris Chloupis wrote:
> My point was that anything you do in Pharo will lower performance quite
> a bit, but it wont be noticable anyway unless its in a big loop or
> something.
>
> Second that Pharo cannot debug such issues which is a huge deal when you
> work with OS stuff and OS calls.
>
> You are far better doing this outside UFFI and straight inside a C IDE
> because you will have an army of tools that will make your life a ton
> easier.
>
> Hence why I talked about performance, C coding and Pharo crashes.
>
> In my case I had to write and test the code in C to port it to UFFI
> because it was impossible to resolve the bugs from inside Pharo. Also
> crashing Pharo all the time got really annoying and I managed to corrupt
> a couple of images too in the process.
>
> Of course debugging C code from a C IDE is quite similar to how you work
> with Pharo.
>
> I also got an experience with loading libraries because I worked with
> Shared memory mapped files which is the technology OS uses to load DLLs.
> Its really low level stuff that come directly from OS kernel. It was a
> ton of fun.
>
> Also C is easier because obviously there is a ton of docs and all you
> have to do is copy paste. In case of UFFI you have to first figure out
> the C code and then how to convert it to UFFI so its double the effort
> for little gain. Its much better thing to just wrap C code and minimize
> the usage of UFFI.
>
> The benefits of Pharo are lost anyway as soon as you drop so low level.
>
> On Thu, Mar 16, 2017 at 3:05 PM <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Indeed, experimenting using some combination of AddDllDirectory(),
>     SetDllDirectory() and LoadLibraryEx() was in my plan for next week.
>
>     Thanks
>
>
>
>     On 16/03/17 11:59, Mark Bestley wrote:
>     > Yes the calling process can alter where these indirect DLLs are loaded
>     > from and so should be able to be done in UFFI and directly from
>     Smalltalk
>     >
>     > The OS will load the DLL specified then it runs DllMain and doing that
>     > causes the OS to load any DLLs that it calls
>     >
>     > MSDN https://msdn.microsoft.com/en-us/library/7d83bc18.aspx says
>     >
>     > With both implicit and explicit linking, Windows first searches for
>     > "known DLLs", such as Kernel32.dll and User32.dll. Windows then
>     searches
>     > for the DLLs in the following sequence:
>     > The directory where the executable module for the current process is
>     > located.
>     > The current directory.
>     > The Windows system directory. The GetSystemDirectory function
>     retrieves
>     > the path of this directory.
>     > The Windows directory. The GetWindowsDirectory function retrieves the
>     > path of this directory.
>     > The directories listed in the PATH environment variable.
>     >
>     >
>     > This can be changed using the call LoadLibraryEx instead of
>     LoadLibrary
>     > Flags can be set to change the directories used
>     >
>     >
>     https://msdn.microsoft.com/en-us/library/windows/desktop/ms684179(v=vs.85).aspx
>     >
>     >
>     >
>     > Also see AddDllDirectory
>     >
>     https://msdn.microsoft.com/en-us/library/windows/desktop/hh310513(v=vs.85).aspx
>     >
>     >
>     > Mark
>     >
>     > On 15/03/2017 17:22, Raffaello Giulietti wrote:
>     >> Hi Dimitris,
>     >>
>     >> I don't get your point about proper C coding, C performance or Pharo
>     >> crashes. It is not the focus of this thread.
>     >>
>     >> How dependencies between libraries are resolved at load-time is
>     dictated
>     >> by the OS. If the OS offers a working mechanism to specify
>     folders that
>     >> the OS uses to search for libraries at load-time, well, I think that
>     >> Pharo, hence the UFFI, should offer it as well and in a more
>     fashionable
>     >> format.
>     >> UFFI is quite elegant in many respects but I wouldn't call it a
>     >> minimalist design (zero hand holding, if I understand you correctly).
>     >> But I'm happy the extra elegance is there because it makes life more
>     >> comfortable.
>     >>
>     >> Again, I still have to experiment with SetDllDirectory() in the
>     case of
>     >> Windows, so I'm not in a position to say something authoritative yet.
>     >>
>     >>
>     >>
>     >>
>     >>
>     >>
>     >>
>     >> On 2017-03-15 17:41, Dimitris Chloupis wrote:
>     >>> Please note the moment you use a FFI of any language you step to a C
>     >>> territory. VMs tend not to touch the code because the code is
>     already
>     >>> in machine code format.
>     >>>
>     >>> There is zero reason for UFFI to worry about the dependencies of a
>     >>> library loaded, this must happen at the library level that UFFI does
>     >>> not focus on.
>     >>>
>     >>> What you describe here is standard practice because the vast
>     majority
>     >>> of libraries depend on other libraries. In the end UFFI or any
>     FFI in
>     >>> any language is not there to replace proper C coding but rather to
>     >>> allow you to utilize C libraries. You cannot hope to achieve
>     >>> performance comparable to C if you try to use a language that is
>     not C
>     >>> to do standard things like dependency management.
>     >>>
>     >>> If the library fail to load it wont be because of UFFI it will be
>     >>> because the library is buggy at handling its dependencies. That
>     means
>     >>> that the library will be buggy even if you use it from C directly.
>     >>>
>     >>> Even you do not know what you doing at C level, you going to crash
>     >>> Pharo countless times anyway and the least of your worries will be
>     >>> handling dependencies which far more obvious than some other nasty C
>     >>> bugs.
>     >>>
>     >>> I think its a great idea that UFFI does zero hand holding. This
>     way it
>     >>> gives the clear message that if you do not understand C then you
>     >>> should not be using the UFFI in the first place.
>     >>>
>     >>> On Wed, Mar 15, 2017 at 4:51 PM Esteban Lorenzano
>     >>> <[hidden email] <mailto:[hidden email]>
>     >>> <mailto:[hidden email] <mailto:[hidden email]>>> wrote:
>     >>>
>     >>>
>     >>>     > On 15 Mar 2017, at 15:44, Raffaello Giulietti
>     >>>     <[hidden email]
>     <mailto:[hidden email]>
>     >>>     <mailto:[hidden email]
>     <mailto:[hidden email]>>>
>     >>>     wrote:
>     >>>     >
>     >>>     > On 2017-03-15 15:20, Esteban Lorenzano wrote:
>     >>>     >>
>     >>>     >>> On 15 Mar 2017, at 15:08, Raffaello Giulietti
>     >>>     <[hidden email]
>     <mailto:[hidden email]>
>     >>>     <mailto:[hidden email]
>     <mailto:[hidden email]>>>
>     >>>     wrote:
>     >>>     >>>
>     >>>     >>> Hi Esteban,
>     >>>     >>>
>     >>>     >>> I understand this is the current status of the UFFI, so
>     I can
>     >>>     certainly use the workarounds discussed below.
>     >>>     >>>
>     >>>     >>> But I hope UFFI will soon offer a more "declarative" way to
>     >>>     specify the search folders, kind of LD_LIBRARY_PATH
>     mechanism but
>     >>>     in the UFFI.
>     >>>     >>
>     >>>     >> that will NOT solve the problem of dependencies of libraries
>     >>>     indirectly references.
>     >>>     >> There is no way to do that (indirect reference solving) on an
>     >>>     executing program that I know.
>     >>>     >> So is not “current status of UFFI”: it will never provide
>     that.
>     >>>     >>
>     >>>     >
>     >>>     > Do you mean that SetDllDirectory() has no influence when
>     invoked
>     >>>     between Pharo's start and the LoadLibrary() call done by the
>     UFFI?
>     >>>
>     >>>     I really have no idea. You can try to add it to your app.
>     >>>
>     >>>     Esteban
>     >>>
>     >>>     >
>     >>>     >
>     >>>     >
>     >>>     >
>     >>>     >> All that, unless someone come and says: "look, Esteban is
>     like
>     >>>     this”.
>     >>>     >>
>     >>>     >> I spent last three weeks (not full time of course) trying
>     to do
>     >>>     exactly that for our current linux vm, to force libgit2 to
>     use the
>     >>>     version of libssh2 we want (and not the version on system)…
>     and I
>     >>>     failed. I solved some cases, but not all. So at the end I
>     opted to
>     >>>     change zeroconf to define LD_LIBRARY_PATH before calling pharo.
>     >>>     Only way I found I can force the lookup paths.
>     >>>     >>
>     >>>     >> cheers,
>     >>>     >> Esteban
>     >>>     >>
>     >>>     >>>
>     >>>     >>>
>     >>>     >>>
>     >>>     >>> Greetings
>     >>>     >>> Raffaello
>     >>>     >>>
>     >>>     >>>
>     >>>     >>>
>     >>>     >>>
>     >>>     >>> On 2017-03-15 14:52, Esteban Lorenzano wrote:
>     >>>     >>>> Hi,
>     >>>     >>>>
>     >>>     >>>> UFFI cannot do what you want.
>     >>>     >>>> If I understand well, you have:
>     >>>     >>>>
>     >>>     >>>> Pharo -> libA.dll -> libB.dll
>     >>>     >>>>
>     >>>     >>>> Pharo does a LoadLibrary(libA.dll), but has no control
>     on how
>     >>>     libA calls libB… and is not possible for us to influence it more
>     >>>     than the predefined platform ways.
>     >>>     >>>>
>     >>>     >>>> One posible workaround (just possible, no idea if it will
>     >>>     work) is to perform a load of libB before is required by libB.
>     >>>     Then hopefully the loader will understand is same library
>     and will
>     >>>     answer same handler (but not sure, because if it
>     discriminates by
>     >>>     path… then you’re in the same position). Anyway, if you want to
>     >>>     try it, it would be something like this:
>     >>>     >>>>
>     >>>     >>>> DynamicLoader loadLibrary: 'path/to/libB.dll’.
>     >>>     >>>> DynamicLoader loadLibrary: 'path/to/libA.dll’.
>     >>>     >>>>
>     >>>     >>>> then you can try to execute functions of libA…
>     >>>     >>>>
>     >>>     >>>> cheers,
>     >>>     >>>> Esteban
>     >>>     >>>>
>     >>>     >>>>
>     >>>     >>>>> On 15 Mar 2017, at 14:32, Raffaello Giulietti
>     >>>     <[hidden email]
>     <mailto:[hidden email]>
>     >>>     <mailto:[hidden email]
>     <mailto:[hidden email]>>>
>     >>>     wrote:
>     >>>     >>>>>
>     >>>     >>>>> On 2017-03-15 13:51, Ben Coman wrote:
>     >>>     >>>>>>
>     >>>     >>>>>>
>     >>>     >>>>>> On Wed, Mar 15, 2017 at 4:42 PM, Raffaello Giulietti
>     >>>     >>>>>>
>     >>>     <[hidden email]
>     <mailto:[hidden email]>
>     >>>     <mailto:[hidden email]
>     <mailto:[hidden email]>>
>     >>>     >>>>>>
>     >>>     <mailto:[hidden email]
>     <mailto:[hidden email]>
>     >>> <mailto:[hidden email]
>     <mailto:[hidden email]>>>>
>     >>>     wrote:
>     >>>     >>>>>>
>     >>>     >>>>>>  Hi Ben,
>     >>>     >>>>>>
>     >>>     >>>>>>  my understanding is that SetDllDirectory only
>     affects the
>     >>>     search
>     >>>     >>>>>>  path of libraries that are loaded at *run-time* with
>     >>>     LoadLibrary.
>     >>>     >>>>>>
>     >>>     >>>>>>  What I'm asking for is a mechanism that works at
>     >>>     *load-time*, when a
>     >>>     >>>>>>  library I'm accessing directly depends on another one
>     >>>     which I'm not
>     >>>     >>>>>>  targeting directly.
>     >>>     >>>>>>
>     >>>     >>>>>>
>     >>>     >>>>>> I don't quite follow.  I'm not intimate with Windows
>     >>>     dynamic loading, so
>     >>>     >>>>>> these questions are an opportunity for me to learn...
>     >>>     >>>>>>
>     >>>     >>>>>> You can't mean when Pharo loads, because it wouldn't know
>     >>>     of OTHER.DLL ?
>     >>>     >>>>>>
>     >>>     >>>>>> Do you mean that LoadLibrary is not called explicitly by
>     >>>     YOUR.DLL before
>     >>>     >>>>>> its uses the function from OTHER.DLL?
>     >>>     >>>>>>
>     >>>     >>>>>
>     >>>     >>>>> During the build of YOUR.DLL, you can specify that it
>     should
>     >>>     depend on OTHER.LIB (kind of binary which lists function headers
>     >>>     and static data declarations for OTHER.DLL). Linking
>     resolution is
>     >>>     then done at load-time of YOUR.DLL, not by invoking
>     >>>     LoadLibrary("OTHER.DLL").
>     >>>     >>>>>
>     >>>     >>>>> When YOUR.DLL is loaded by UFFI at run-time (here by means
>     >>>     of LoadLibrary(), I guess), Windows (not the UFFI) also attempts
>     >>>     to load OTHER.DLL because of the linking information gathered
>     >>>     during the build. Notice that the linking information does *not*
>     >>>     include a specific path for OTHER.DLL.
>     >>>     >>>>>
>     >>>     >>>>> (You don't have to mention standard *.LIBs like
>     Kernel32.lib
>     >>>     because they are included by default. But of course, Windows
>     knows
>     >>>     where to find the corresponding .dll.)
>     >>>     >>>>>
>     >>>     >>>>> OTHER.DLL is searched using the strategy described in the
>     >>>     MSDN page you mention. My expectation is/was that UFFI
>     has/had an
>     >>>     API for specifying the search paths where to find OTHER.DLL in a
>     >>>     platform-independent way. In other words, I would expect the
>     UFFI
>     >>>     to do the call of SetDllDirectory() or whatever is needed on
>     >>>     Windows for me or to use other mechanisms on other platforms.
>     >>>     >>>>>
>     >>>     >>>>>
>     >>>     >>>>>
>     >>>     >>>>>> Do you mean you link OTHER.DLL to YOUR.DLL at compile
>     time?
>     >>>     >>>>>> But still, YOUR.DLL it not loaded until called via Pharo
>     >>>     FFI, so neither
>     >>>     >>>>>> is OTHER.DLL,
>     >>>     >>>>>> and Dynamic-Link Library Search Order [1] says... "If
>     a DLL
>     >>>     with the
>     >>>     >>>>>> same module name
>     >>>     >>>>>> is already loaded in memory, the system uses the loaded
>     >>>     DLL, no matter
>     >>>     >>>>>> which directory it is in.
>     >>>     >>>>>> The system does not search for the DLL."
>     >>>     >>>>>>
>     >>>     >>>>>> So maybe before calling any of YOUR.DLL,
>     >>>     >>>>>> do a SetDllDirectory followed by a LoadLibrary(OTHER.DLL)
>     >>>     >>>>>> and later when you call into YOUR.DLL,
>     >>>     >>>>>> OTHER.DLL is already loaded.
>     >>>     >>>>>>
>     >>>     >>>>>
>     >>>     >>>>> Yes, this might work but is rather contrived.
>     >>>     >>>>>
>     >>>     >>>>>
>     >>>     >>>>>
>     >>>     >>>>>> Otherwise it would seems(?) your options are either
>     Manifests
>     >>>     >>>>>> or Dynamic-Link Library Redirection, both done outside of
>     >>>     Pharo.
>     >>>     >>>>>>
>     >>>     >>>>>
>     >>>     >>>>> What if neither YOUR.DLL nor OTHER.DLL are under my
>     control?
>     >>>     The only thing I know is where the .dll and the
>     corresponding .lib
>     >>>     + .h are.
>     >>>     >>>>>
>     >>>     >>>>>
>     >>>     >>>>>
>     >>>     >>>>> Thanks for your interest
>     >>>     >>>>> Raffaello
>     >>>     >>>>>
>     >>>     >>>>>
>     >>>     >>>>>
>     >>>     >>>>>
>     >>>     >>>>>>  But I'll try whether SetDllDirectory also affects
>     >>>     load-time searches.
>     >>>     >>>>>>
>     >>>     >>>>>>  Anyway, I hope future versions of the UFFI to offer
>     >>>     setting the
>     >>>     >>>>>>  search paths more easily.
>     >>>     >>>>>>
>     >>>     >>>>>>
>     >>>     >>>>>>
>     >>>     >>>>>>  Greetings
>     >>>     >>>>>>  Raffaello
>     >>>     >>>>>>
>     >>>     >>>>>
>     >>>     >>>>>
>     >>>     >>>>
>     >>>     >>>>
>     >>>     >>>
>     >>>     >>>
>     >>>     >>
>     >>>     >>
>     >>>     >
>     >>>     >
>     >>>
>     >>>
>     >>
>     >
>     >
>
>


12