How to make pharo find sqlite?

Hello!

I try to deploy UDBCSQLite-using image in a 32bit ubuntu 16.04.3.

I do have libsqlite3:

root@32bit-agent:~# find / -name '*libsqlite*' -type f 2>>/dev/null
/usr/lib/i386-linux-gnu/libsqlite3.so.0.8.6
/var/lib/dpkg/info/libsqlite0.list
/var/lib/dpkg/info/libsqlite3-0:i386.postinst
/var/lib/dpkg/info/libsqlite3-0:i386.md5sums
/var/lib/dpkg/info/libsqlite3-0:i386.shlibs
/var/lib/dpkg/info/libsqlite0.postrm
/var/lib/dpkg/info/libsqlite3-0:i386.symbols
/var/lib/dpkg/info/libsqlite3-0:i386.list
/var/lib/dpkg/info/libsqlite3-0:i386.triggers
/var/cache/apt/archives/libsqlite0_2.8.17-12fakesync1_i386.deb

but I get this in the output of the CI:

17:16:54.233 + ../pharo/pharo ./filmtower.image conf/run-tests.st
17:16:54.508 pthread_setschedparam failed: Operation not permitted
17:16:54.509 This VM uses a separate heartbeat thread to update its internal clock
17:16:54.509 and handle events.  For best operation, this thread should run at a
17:16:54.509 higher priority, however the VM was unable to change the priority.  The
17:16:54.509 effect is that heavily loaded systems may experience some latency
17:16:54.509 issues.  If this occurs, please create the appropriate configuration
17:16:54.509 file in /etc/security/limits.d/ as shown below:
17:16:54.509
17:16:54.509 cat <<END | sudo tee /etc/security/limits.d/pharo.conf
17:16:54.509 *      hard    rtprio  2
17:16:54.509 *      soft    rtprio  2
17:16:54.509 END
17:16:54.509
17:16:54.509 and report to the pharo mailing list whether this improves behaviour.
17:16:54.512
17:16:54.512 You will need to log out and log back in for the limits to take effect.
17:16:54.512 For more information please see
17:16:54.512 https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux
17:16:54.785
17:16:54.786 TowergameSyncTests
17:16:54.831 Error: External module not found
17:16:54.832 ExternalLibraryFunction(Object)>>error:
17:16:54.832 ExternalLibraryFunction(Object)>>externalCallFailed
17:16:54.833 ExternalLibraryFunction(ExternalFunction)>>invokeWithArguments:
17:16:54.833 UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData:
17:16:54.834 FFICalloutAPI>>function:module:
17:16:54.834 UDBCSQLite3Library(Object)>>ffiCall:module:
17:16:54.835 UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData:
17:16:54.836 FFIExternalResourceExecutor>>finalize
17:16:54.836 WeakFinalizerItem>>finalizeValues
17:16:54.845 [ each finalizeValues ] in [ :each | [ each finalizeValues ] on: Exception fork: [ :ex | ex pass ] ] in WeakRegistry>>finalizeValues in Block: [ each finalizeValues ]
17:16:54.846 BlockClosure>>on:do:
17:16:54.852 [ Processor terminateActive ] in [ :ex |
17:16:54.852 | copy onDoCtx process handler bottom thisCtx |
17:16:54.852 onDoCtx := thisContext.
17:16:54.852 thisCtx := onDoCtx home.
17:16:54.852
17:16:54.852 "find the context on stack for which this method's is sender"
17:16:54.852 [ onDoCtx sender == thisCtx ]
17:16:54.852    whileFalse: [ onDoCtx := onDoCtx sender.
17:16:54.852            onDoCtx
17:16:54.852                    ifNil: [ "Can't find our home context. seems like we're already forked
17:16:54.852                            and handling another exception in new thread. In this case, just pass it through handler." ^ handlerAction cull: ex ] ].
17:16:54.852 bottom := [ Processor terminateActive ] asContext.
17:16:54.853 onDoCtx privSender: bottom.
17:16:54.853 handler := [ handlerAction cull: ex ] asContext.
17:16:54.853 handler privSender: thisContext sender.
17:16:54.853 (Process forContext: handler priority: Processor activePriority)
17:16:54.853    resume.
17:16:54.853
17:16:54.853 "cut the stack of current process"
17:16:54.853 thisContext privSender: thisCtx.
17:16:54.853 nil ] in BlockClosure>>on:fork: in Block: [ Processor terminateActive ]
17:16:54.989

Look like pharo was not able to find the sqlite3 lib.

Any help?

Thanks, Herby

[hidden email] wrote:

What about

LD_LIBRARY_PATH=<sqlite3place>;$LD_LIBRARYPATH  ./pharo-ui some.image

Phil

Thanks for answer, did not help.

In fact it must be something different. As can be seen in the stack, it fails during finalizers, and as can be seen by looking at UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: code, the method it calls is sqlite close. It is hardly the first method that is should call...

I suspect something around image save / load. Again. Lots of errors in those parts. But may be something else, as it kicks in only when SQLite-using tests starts to run. :-(

Herby

P.S.: I saw there is a similar thread out there, but it has problems with 32bit loaded by 64bit vm; but here, I have 32bit linux, so the vm installed should be 32bit.

On Thu, Sep 28, 2017 at 7:40 PM, Herby Vojčík <[hidden email]
<mailto:[hidden email]>> wrote:

    Hello!

    I try to deploy UDBCSQLite-using image in a 32bit ubuntu 16.04.3.

    I do have libsqlite3:

    root@32bit-agent:~# find / -name '*libsqlite*' -type f 2>>/dev/null
    /usr/lib/i386-linux-gnu/libsqlite3.so.0.8.6
    /var/lib/dpkg/info/libsqlite0.list
    /var/lib/dpkg/info/libsqlite3-0:i386.postinst
    /var/lib/dpkg/info/libsqlite3-0:i386.md5sums
    /var/lib/dpkg/info/libsqlite3-0:i386.shlibs
    /var/lib/dpkg/info/libsqlite0.postrm
    /var/lib/dpkg/info/libsqlite3-0:i386.symbols
    /var/lib/dpkg/info/libsqlite3-0:i386.list
    /var/lib/dpkg/info/libsqlite3-0:i386.triggers
    /var/cache/apt/archives/libsqlite0_2.8.17-12fakesync1_i386.deb

    but I get this in the output of the CI:

    17:16:54.233 + ../pharo/pharo ./filmtower.image conf/run-tests.st
    <http://run-tests.st>
    17:16:54.508 pthread_setschedparam failed: Operation not permitted
    17:16:54.509 This VM uses a separate heartbeat thread to update its
    internal clock
    17:16:54.509 and handle events.  For best operation, this thread
    should run at a
    17:16:54.509 higher priority, however the VM was unable to change
    the priority.  The
    17:16:54.509 effect is that heavily loaded systems may experience
    some latency
    17:16:54.509 issues.  If this occurs, please create the appropriate
    configuration
    17:16:54.509 file in /etc/security/limits.d/ as shown below:
    17:16:54.509
    17:16:54.509 cat <<END | sudo tee /etc/security/limits.d/pharo.conf
    17:16:54.509 *      hard    rtprio  2
    17:16:54.509 *      soft    rtprio  2
    17:16:54.509 END
    17:16:54.509
    17:16:54.509 and report to the pharo mailing list whether this
    improves behaviour.
    17:16:54.512
    17:16:54.512 You will need to log out and log back in for the limits
    to take effect.
    17:16:54.512 For more information please see
    17:16:54.512
    https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux
    <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux>
    17:16:54.785
    17:16:54.786 TowergameSyncTests
    17:16:54.831 Error: External module not found
    17:16:54.832 ExternalLibraryFunction(Object)>>error:
    17:16:54.832 ExternalLibraryFunction(Object)>>externalCallFailed
    17:16:54.833
    ExternalLibraryFunction(ExternalFunction)>>invokeWithArguments:
    17:16:54.833 UDBCSQLite3DatabaseExternalObject
    class>>finalizeResourceData:
    17:16:54.834 FFICalloutAPI>>function:module:
    17:16:54.834 UDBCSQLite3Library(Object)>>ffiCall:module:
    17:16:54.835 UDBCSQLite3DatabaseExternalObject
    class>>finalizeResourceData:
    17:16:54.836 FFIExternalResourceExecutor>>finalize
    17:16:54.836 WeakFinalizerItem>>finalizeValues
    17:16:54.845 [ each finalizeValues ] in [ :each | [ each
    finalizeValues ] on: Exception fork: [ :ex | ex pass ] ] in
    WeakRegistry>>finalizeValues in Block: [ each finalizeValues ]
    17:16:54.846 BlockClosure>>on:do:
    17:16:54.852 [ Processor terminateActive ] in [ :ex |
    17:16:54.852 | copy onDoCtx process handler bottom thisCtx |
    17:16:54.852 onDoCtx := thisContext.
    17:16:54.852 thisCtx := onDoCtx home.
    17:16:54.852
    17:16:54.852 "find the context on stack for which this method's is
    sender"
    17:16:54.852 [ onDoCtx sender == thisCtx ]
    17:16:54.852    whileFalse: [ onDoCtx := onDoCtx sender.
    17:16:54.852            onDoCtx
    17:16:54.852                    ifNil: [ "Can't find our home
    context. seems like we're already forked
    17:16:54.852                            and handling another
    exception in new thread. In this case, just pass it through
    handler." ^ handlerAction cull: ex ] ].
    17:16:54.852 bottom := [ Processor terminateActive ] asContext.
    17:16:54.853 onDoCtx privSender: bottom.
    17:16:54.853 handler := [ handlerAction cull: ex ] asContext.
    17:16:54.853 handler privSender: thisContext sender.
    17:16:54.853 (Process forContext: handler priority: Processor
    activePriority)
    17:16:54.853    resume.
    17:16:54.853
    17:16:54.853 "cut the stack of current process"
    17:16:54.853 thisContext privSender: thisCtx.
    17:16:54.853 nil ] in BlockClosure>>on:fork: in Block: [ Processor
    terminateActive ]
    17:16:54.989

    Look like pharo was not able to find the sqlite3 lib.

    Any help?

    Thanks, Herby

[hidden email] wrote:

What about

LD_LIBRARY_PATH=<sqlite3place>;$LD_LIBRARYPATH  ./pharo-ui some.image

Phil

Thanks for answer, did not help.

In fact it must be something different. As can be seen in the stack, it fails during finalizers, and as can be seen by looking at UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: code, the method it calls is sqlite close. It is hardly the first method that is should call...

I suspect something around image save / load. Again. Lots of errors in those parts. But may be something else, as it kicks in only when SQLite-using tests starts to run. :-(

Herby

P.S.: I saw there is a similar thread out there, but it has problems with 32bit loaded by 64bit vm; but here, I have 32bit linux, so the vm installed should be 32bit.

On Thu, Sep 28, 2017 at 7:40 PM, Herby Vojčík <[hidden email]
<mailto:[hidden email]>> wrote:

    Hello!

    I try to deploy UDBCSQLite-using image in a 32bit ubuntu 16.04.3.

    I do have libsqlite3:

    root@32bit-agent:~# find / -name '*libsqlite*' -type f 2>>/dev/null
    /usr/lib/i386-linux-gnu/libsqlite3.so.0.8.6
    /var/lib/dpkg/info/libsqlite0.list
    /var/lib/dpkg/info/libsqlite3-0:i386.postinst
    /var/lib/dpkg/info/libsqlite3-0:i386.md5sums
    /var/lib/dpkg/info/libsqlite3-0:i386.shlibs
    /var/lib/dpkg/info/libsqlite0.postrm
    /var/lib/dpkg/info/libsqlite3-0:i386.symbols
    /var/lib/dpkg/info/libsqlite3-0:i386.list
    /var/lib/dpkg/info/libsqlite3-0:i386.triggers
    /var/cache/apt/archives/libsqlite0_2.8.17-12fakesync1_i386.deb

    but I get this in the output of the CI:

    17:16:54.233 + ../pharo/pharo ./filmtower.image conf/run-tests.st
    <http://run-tests.st>
    17:16:54.508 pthread_setschedparam failed: Operation not permitted
    17:16:54.509 This VM uses a separate heartbeat thread to update its
    internal clock
    17:16:54.509 and handle events.  For best operation, this thread
    should run at a
    17:16:54.509 higher priority, however the VM was unable to change
    the priority.  The
    17:16:54.509 effect is that heavily loaded systems may experience
    some latency
    17:16:54.509 issues.  If this occurs, please create the appropriate
    configuration
    17:16:54.509 file in /etc/security/limits.d/ as shown below:
    17:16:54.509
    17:16:54.509 cat <<END | sudo tee /etc/security/limits.d/pharo.conf
    17:16:54.509 *      hard    rtprio  2
    17:16:54.509 *      soft    rtprio  2
    17:16:54.509 END
    17:16:54.509
    17:16:54.509 and report to the pharo mailing list whether this
    improves behaviour.
    17:16:54.512
    17:16:54.512 You will need to log out and log back in for the limits
    to take effect.
    17:16:54.512 For more information please see
    17:16:54.512
    https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux
    <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux>
    17:16:54.785
    17:16:54.786 TowergameSyncTests
    17:16:54.831 Error: External module not found
    17:16:54.832 ExternalLibraryFunction(Object)>>error:
    17:16:54.832 ExternalLibraryFunction(Object)>>externalCallFailed
    17:16:54.833
    ExternalLibraryFunction(ExternalFunction)>>invokeWithArguments:
    17:16:54.833 UDBCSQLite3DatabaseExternalObject
    class>>finalizeResourceData:
    17:16:54.834 FFICalloutAPI>>function:module:
    17:16:54.834 UDBCSQLite3Library(Object)>>ffiCall:module:
    17:16:54.835 UDBCSQLite3DatabaseExternalObject
    class>>finalizeResourceData:
    17:16:54.836 FFIExternalResourceExecutor>>finalize
    17:16:54.836 WeakFinalizerItem>>finalizeValues
    17:16:54.845 [ each finalizeValues ] in [ :each | [ each
    finalizeValues ] on: Exception fork: [ :ex | ex pass ] ] in
    WeakRegistry>>finalizeValues in Block: [ each finalizeValues ]
    17:16:54.846 BlockClosure>>on:do:
    17:16:54.852 [ Processor terminateActive ] in [ :ex |
    17:16:54.852 | copy onDoCtx process handler bottom thisCtx |
    17:16:54.852 onDoCtx := thisContext.
    17:16:54.852 thisCtx := onDoCtx home.
    17:16:54.852
    17:16:54.852 "find the context on stack for which this method's is
    sender"
    17:16:54.852 [ onDoCtx sender == thisCtx ]
    17:16:54.852    whileFalse: [ onDoCtx := onDoCtx sender.
    17:16:54.852            onDoCtx
    17:16:54.852                    ifNil: [ "Can't find our home
    context. seems like we're already forked
    17:16:54.852                            and handling another
    exception in new thread. In this case, just pass it through
    handler." ^ handlerAction cull: ex ] ].
    17:16:54.852 bottom := [ Processor terminateActive ] asContext.
    17:16:54.853 onDoCtx privSender: bottom.
    17:16:54.853 handler := [ handlerAction cull: ex ] asContext.
    17:16:54.853 handler privSender: thisContext sender.
    17:16:54.853 (Process forContext: handler priority: Processor
    activePriority)
    17:16:54.853    resume.
    17:16:54.853
    17:16:54.853 "cut the stack of current process"
    17:16:54.853 thisContext privSender: thisCtx.
    17:16:54.853 nil ] in BlockClosure>>on:fork: in Block: [ Processor
    terminateActive ]
    17:16:54.989

    Look like pharo was not able to find the sqlite3 lib.

    Any help?

    Thanks, Herby

[hidden email] wrote:

Also, did you try with this VM:

http://get.pharo.org/vmTLatest60

18:51:46.191 + curl get.pharo.org/vmTLatest60
18:51:46.207   % Total    % Received % Xferd  Average Speed   Time Time     Time  Current
18:51:46.208                                  Dload  Upload   Total Spent    Left  Speed
18:51:46.208
18:51:46.242   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
18:51:46.242 100  6126  100  6126    0     0   172k      0 --:--:-- --:--:-- --:--:--  175k
18:51:46.253 Downloading the latest pharoVM:
18:51:46.253 http://files.pharo.org/get-files/60/pharo-linux-threaded-latest.zip
18:51:46.305 [pharo-vm/vm.zip]
18:51:46.305   End-of-central-directory signature not found.  Either this file is not
18:51:46.305   a zipfile, or it constitutes one disk of a multi-part archive.  In the
18:51:46.305   latter case the central directory and zipfile comment will be found on
18:51:46.305   the last disk(s) of this archive.
18:51:46.305 unzip:  cannot find zipfile directory in one of pharo-vm/vm.zip or
18:51:46.305         pharo-vm/vm.zip.zip, and cannot find pharo-vm/vm.zip.ZIP, period.


It probably does not exist any more (I tried 70+vm and it failed in other aspects, it wasn't able to load git repo).

Tried both 61+vmT and 61+vmI both; in 32vm/32os and 64vm/64os combinations. Always ended with same result.

Must be some error in UDBCSQLiteLibrary itself. :-/

Although the missing transcript output is scary and shows that vm may be culprit as well.

Herby

Phil

On Sat, Sep 30, 2017 at 1:28 PM, Herby Vojčík <[hidden email]
<mailto:[hidden email]>> wrote:

    [hidden email] <mailto:[hidden email]> wrote:

        What about

        LD_LIBRARY_PATH=<sqlite3place>;$LD_LIBRARYPATH  ./pharo-ui
        some.image

        Phil


    Thanks for answer, did not help.

    In fact it must be something different. As can be seen in the stack,
    it fails during finalizers, and as can be seen by looking at
    UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: code,
    the method it calls is sqlite close. It is hardly the first method
    that is should call...

    I suspect something around image save / load. Again. Lots of errors
    in those parts. But may be something else, as it kicks in only when
    SQLite-using tests starts to run. :-(

    Herby

    P.S.: I saw there is a similar thread out there, but it has problems
    with 32bit loaded by 64bit vm; but here, I have 32bit linux, so the
    vm installed should be 32bit.

        On Thu, Sep 28, 2017 at 7:40 PM, Herby Vojčík <[hidden email]
        <mailto:[hidden email]>
        <mailto:[hidden email] <mailto:[hidden email]>>> wrote:

             Hello!

             I try to deploy UDBCSQLite-using image in a 32bit ubuntu
        16.04.3.

             I do have libsqlite3:

             root@32bit-agent:~# find / -name '*libsqlite*' -type f
        2>>/dev/null
             /usr/lib/i386-linux-gnu/libsqlite3.so.0.8.6
             /var/lib/dpkg/info/libsqlite0.list
             /var/lib/dpkg/info/libsqlite3-0:i386.postinst
             /var/lib/dpkg/info/libsqlite3-0:i386.md5sums
             /var/lib/dpkg/info/libsqlite3-0:i386.shlibs
             /var/lib/dpkg/info/libsqlite0.postrm
             /var/lib/dpkg/info/libsqlite3-0:i386.symbols
             /var/lib/dpkg/info/libsqlite3-0:i386.list
             /var/lib/dpkg/info/libsqlite3-0:i386.triggers
             /var/cache/apt/archives/libsqlite0_2.8.17-12fakesync1_i386.deb

             but I get this in the output of the CI:

             17:16:54.233 + ../pharo/pharo ./filmtower.image
        conf/run-tests.st <http://run-tests.st>
        <http://run-tests.st>
             17:16:54.508 pthread_setschedparam failed: Operation not
        permitted
             17:16:54.509 This VM uses a separate heartbeat thread to
        update its
             internal clock
             17:16:54.509 and handle events.  For best operation, this
        thread
             should run at a
             17:16:54.509 higher priority, however the VM was unable to
        change
             the priority.  The
             17:16:54.509 effect is that heavily loaded systems may
        experience
             some latency
             17:16:54.509 issues.  If this occurs, please create the
        appropriate
             configuration
             17:16:54.509 file in /etc/security/limits.d/ as shown below:
             17:16:54.509
             17:16:54.509 cat <<END | sudo tee
        /etc/security/limits.d/pharo.conf
             17:16:54.509 *      hard    rtprio  2
             17:16:54.509 *      soft    rtprio  2
             17:16:54.509 END
             17:16:54.509
             17:16:54.509 and report to the pharo mailing list whether this
             improves behaviour.
             17:16:54.512
             17:16:54.512 You will need to log out and log back in for
        the limits
             to take effect.
             17:16:54.512 For more information please see
             17:16:54.512
        https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux
        <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux>
        <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux
        <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux>>
             17:16:54.785
             17:16:54.786 TowergameSyncTests
             17:16:54.831 Error: External module not found
             17:16:54.832 ExternalLibraryFunction(Object)>>error:
             17:16:54.832
        ExternalLibraryFunction(Object)>>externalCallFailed
             17:16:54.833
             ExternalLibraryFunction(ExternalFunction)>>invokeWithArguments:
             17:16:54.833 UDBCSQLite3DatabaseExternalObject
             class>>finalizeResourceData:
             17:16:54.834 FFICalloutAPI>>function:module:
             17:16:54.834 UDBCSQLite3Library(Object)>>ffiCall:module:
             17:16:54.835 UDBCSQLite3DatabaseExternalObject
             class>>finalizeResourceData:
             17:16:54.836 FFIExternalResourceExecutor>>finalize
             17:16:54.836 WeakFinalizerItem>>finalizeValues
             17:16:54.845 [ each finalizeValues ] in [ :each | [ each
             finalizeValues ] on: Exception fork: [ :ex | ex pass ] ] in
             WeakRegistry>>finalizeValues in Block: [ each finalizeValues ]
             17:16:54.846 BlockClosure>>on:do:
             17:16:54.852 [ Processor terminateActive ] in [ :ex |
             17:16:54.852 | copy onDoCtx process handler bottom thisCtx |
             17:16:54.852 onDoCtx := thisContext.
             17:16:54.852 thisCtx := onDoCtx home.
             17:16:54.852
             17:16:54.852 "find the context on stack for which this
        method's is
             sender"
             17:16:54.852 [ onDoCtx sender == thisCtx ]
             17:16:54.852    whileFalse: [ onDoCtx := onDoCtx sender.
             17:16:54.852            onDoCtx
             17:16:54.852                    ifNil: [ "Can't find our home
             context. seems like we're already forked
             17:16:54.852                            and handling another
             exception in new thread. In this case, just pass it through
             handler." ^ handlerAction cull: ex ] ].
             17:16:54.852 bottom := [ Processor terminateActive ] asContext.
             17:16:54.853 onDoCtx privSender: bottom.
             17:16:54.853 handler := [ handlerAction cull: ex ] asContext.
             17:16:54.853 handler privSender: thisContext sender.
             17:16:54.853 (Process forContext: handler priority: Processor
             activePriority)
             17:16:54.853    resume.
             17:16:54.853
             17:16:54.853 "cut the stack of current process"
             17:16:54.853 thisContext privSender: thisCtx.
             17:16:54.853 nil ] in BlockClosure>>on:fork: in Block: [
        Processor
             terminateActive ]
             17:16:54.989

             Look like pharo was not able to find the sqlite3 lib.

             Any help?

             Thanks, Herby

[hidden email] wrote:

Is https://pharo.fogbugz.com/f/cases/19990 showing again?

What is the module being loaded ?

This seems to be the important question. It seems FFI for some reason struggles with 'lib' and/or '.so.0' things in linux (even if LD_LIBRARY_PATH is properly set).

I had to do this:

TARGETDIR=`find . -type f -name SqueakSSL.so -print0 | xargs -0 dirname`
ln -s `/sbin/ldconfig -p | sed -e 's|[^/]*||' | grep sqlite3` ${TARGETDIR}/sqlite3.so

(so, link in plugin directory, and the name is plain 'sqlite3.so'). With that, things work. Maybe, libsqlite.so would do the trick as well, but I got no nerve to play with it more.

But, frankly, do not tell me this is what ppl need to do to load external libs in linux. :-/

Herby

Phil


On Sat, Sep 30, 2017 at 1:28 PM, Herby Vojčík <[hidden email]
<mailto:[hidden email]>> wrote:

    [hidden email] <mailto:[hidden email]> wrote:

        What about

        LD_LIBRARY_PATH=<sqlite3place>;$LD_LIBRARYPATH  ./pharo-ui
        some.image

        Phil


    Thanks for answer, did not help.

    In fact it must be something different. As can be seen in the stack,
    it fails during finalizers, and as can be seen by looking at
    UDBCSQLite3DatabaseExternalObject class>>finalizeResourceData: code,
    the method it calls is sqlite close. It is hardly the first method
    that is should call...

    I suspect something around image save / load. Again. Lots of errors
    in those parts. But may be something else, as it kicks in only when
    SQLite-using tests starts to run. :-(

    Herby

    P.S.: I saw there is a similar thread out there, but it has problems
    with 32bit loaded by 64bit vm; but here, I have 32bit linux, so the
    vm installed should be 32bit.

        On Thu, Sep 28, 2017 at 7:40 PM, Herby Vojčík <[hidden email]
        <mailto:[hidden email]>

        <mailto:[hidden email] <mailto:[hidden email]>>> wrote:

             Hello!

             I try to deploy UDBCSQLite-using image in a 32bit ubuntu
        16.04.3.

             I do have libsqlite3:

             root@32bit-agent:~# find / -name '*libsqlite*' -type f
        2>>/dev/null
             /usr/lib/i386-linux-gnu/libsqlite3.so.0.8.6
             /var/lib/dpkg/info/libsqlite0.list
             /var/lib/dpkg/info/libsqlite3-0:i386.postinst
             /var/lib/dpkg/info/libsqlite3-0:i386.md5sums
             /var/lib/dpkg/info/libsqlite3-0:i386.shlibs
             /var/lib/dpkg/info/libsqlite0.postrm
             /var/lib/dpkg/info/libsqlite3-0:i386.symbols
             /var/lib/dpkg/info/libsqlite3-0:i386.list
             /var/lib/dpkg/info/libsqlite3-0:i386.triggers
             /var/cache/apt/archives/libsqlite0_2.8.17-12fakesync1_i386.deb

             but I get this in the output of the CI:

             17:16:54.233 + ../pharo/pharo ./filmtower.image
        conf/run-tests.st <http://run-tests.st>
        <http://run-tests.st>
             17:16:54.508 pthread_setschedparam failed: Operation not
        permitted
             17:16:54.509 This VM uses a separate heartbeat thread to
        update its
             internal clock
             17:16:54.509 and handle events.  For best operation, this
        thread
             should run at a
             17:16:54.509 higher priority, however the VM was unable to
        change
             the priority.  The
             17:16:54.509 effect is that heavily loaded systems may
        experience
             some latency
             17:16:54.509 issues.  If this occurs, please create the
        appropriate
             configuration
             17:16:54.509 file in /etc/security/limits.d/ as shown below:
             17:16:54.509
             17:16:54.509 cat <<END | sudo tee
        /etc/security/limits.d/pharo.conf
             17:16:54.509 *      hard    rtprio  2
             17:16:54.509 *      soft    rtprio  2
             17:16:54.509 END
             17:16:54.509
             17:16:54.509 and report to the pharo mailing list whether this
             improves behaviour.
             17:16:54.512
             17:16:54.512 You will need to log out and log back in for
        the limits
             to take effect.
             17:16:54.512 For more information please see
             17:16:54.512
        https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux
        <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux>
        <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux
        <https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/r3732#linux>>
             17:16:54.785
             17:16:54.786 TowergameSyncTests
             17:16:54.831 Error: External module not found
             17:16:54.832 ExternalLibraryFunction(Object)>>error:
             17:16:54.832
        ExternalLibraryFunction(Object)>>externalCallFailed
             17:16:54.833
             ExternalLibraryFunction(ExternalFunction)>>invokeWithArguments:
             17:16:54.833 UDBCSQLite3DatabaseExternalObject
             class>>finalizeResourceData:
             17:16:54.834 FFICalloutAPI>>function:module:
             17:16:54.834 UDBCSQLite3Library(Object)>>ffiCall:module:
             17:16:54.835 UDBCSQLite3DatabaseExternalObject
             class>>finalizeResourceData:
             17:16:54.836 FFIExternalResourceExecutor>>finalize
             17:16:54.836 WeakFinalizerItem>>finalizeValues
             17:16:54.845 [ each finalizeValues ] in [ :each | [ each
             finalizeValues ] on: Exception fork: [ :ex | ex pass ] ] in
             WeakRegistry>>finalizeValues in Block: [ each finalizeValues ]
             17:16:54.846 BlockClosure>>on:do:
             17:16:54.852 [ Processor terminateActive ] in [ :ex |
             17:16:54.852 | copy onDoCtx process handler bottom thisCtx |
             17:16:54.852 onDoCtx := thisContext.
             17:16:54.852 thisCtx := onDoCtx home.
             17:16:54.852
             17:16:54.852 "find the context on stack for which this
        method's is
             sender"
             17:16:54.852 [ onDoCtx sender == thisCtx ]
             17:16:54.852    whileFalse: [ onDoCtx := onDoCtx sender.
             17:16:54.852            onDoCtx
             17:16:54.852                    ifNil: [ "Can't find our home
             context. seems like we're already forked
             17:16:54.852                            and handling another
             exception in new thread. In this case, just pass it through
             handler." ^ handlerAction cull: ex ] ].
             17:16:54.852 bottom := [ Processor terminateActive ] asContext.
             17:16:54.853 onDoCtx privSender: bottom.
             17:16:54.853 handler := [ handlerAction cull: ex ] asContext.
             17:16:54.853 handler privSender: thisContext sender.
             17:16:54.853 (Process forContext: handler priority: Processor
             activePriority)
             17:16:54.853    resume.
             17:16:54.853
             17:16:54.853 "cut the stack of current process"
             17:16:54.853 thisContext privSender: thisCtx.
             17:16:54.853 nil ] in BlockClosure>>on:fork: in Block: [
        Processor
             terminateActive ]
             17:16:54.989

             Look like pharo was not able to find the sqlite3 lib.

             Any help?

             Thanks, Herby

Renaud de Villemeur wrote:

Hi

[...snip...]

The reason it pass under windows is because the method library return by
default sqlite3, which is the dll name you put under pharo VM directory
to get it work.

Not true. It is not in pharo vm directory. It finds it on %PATH%.

On linux, unless you link your library in the VM folder, the image has
no clue where to find sqlite3.

And this should be fixed, IMNSHO. It probably _partly_ works, but it should be made to work better.

Herby

Hope this helps.
Renaud

.

On Sat, Sep 30, 2017 at 10:16:47PM +0200, Herby Vojčík wrote:
> case the library name used is the plain 'sqlite3', I don't know - am
> I supposed to make a symlink on the same directory as the image? Is
> it the thing that is normally needed / done routinely?

Below is how I've been doing it since early days of Pharo. Desktop OS = Linux
Mint. Current server OS = Ubuntu 16.04.

Firstly, I don't use the distro-provided libsqlite3.so. My self-built copy
includes features like FTS, JSON, etc.

I place libsqlite3.so in the VM directory, say /pkg/pharo5vm/. I have a script
/pkg/pharo5vm/gopharo:

  #!/bin/sh
  PHAROVMPATH=$(dirname `readlink -f "$0"`)
  LD_LIBRARY_PATH="$PHAROVMPATH" exec "$PHAROVMPATH/pharo" $@ &

On desktop, I start Pharo using gopharo. On server, I use daemontools. Say my
application directory on the server is /pkg/app1. Here's the daemontools run
file /pkg/app1/run:

  #!/bin/sh
  /usr/bin/setuidgid app1 \
      /pkg/pharo5vm/gopharo -vm-display-null -vm-sound-null smallcms1.image --no-quit

The other files in /pkg/app1 are PharoV*.sources and the application
image/changes files.

/pkg/pharo5vm contains the pharo executable, vm-*, and the *so* files/symlinks
that come with Pharo plus any others that I custom build myself such as
libsqlite3.so and libshacrypt.so.

HTH.

Pierce