Re: UUID as internal (was 4.0.3.2196 for Unix)

On Mon, Apr 12, 2010 at 01:48:32PM +0200, laurent laffont wrote:
>
> I tend to think that if it's really a libuuid problem the VM would crash
> whether UUIDPlugin is built Internal or External.

Hi Laurent,

This bug is documented on Mantis
 http://bugs.squeak.org/view.php?id=7358

Can you confirm that the bug does *not* exist when you build the plugin
as internal, and that it *does" exist if you build it external on the
same platform. It would be a big help to know if this is the case.

My impression up until now is that this is a bug in the runtime libuuid
library, and that it has something to do with running a program with
pthreads (i.e. it does not happen with a simple test program). But I
have never actually seen the bug on my own system, so I'm guessing based
on some google searches and the information that I read on the Pharo list.

Thanks!

Dave

On Tue, Apr 13, 2010 at 08:12:26AM +0200, laurent laffont wrote:
>
> On Tue, Apr 13, 2010 at 1:36 AM, David T. Lewis <[hidden email]> wrote:
> >

> > Can you confirm that the bug does *not* exist when you build the plugin
> > as internal, and that it *does" exist if you build it external on the
> > same platform. It would be a big help to know if this is the case.
> >
> > My impression up until now is that this is a bug in the runtime libuuid
> > library, and that it has something to do with running a program with
> > pthreads (i.e. it does not happen with a simple test program). But I
> > have never actually seen the bug on my own system, so I'm guessing based
> > on some google searches and the information that I read on the Pharo list.
>

> I have just built last release (Squeak-4.0.3.2200-src.tar.gz) with UUID as
> internal and it works like a charm. All Pharo 1.0-RC4 tests green ! Thank
> you all.
>
> So there's a real problem when UUID is external.

Hi Laurent,

Thanks for the update. I added this to the Mantis 7358 issue also.

I have one more request if you have the time. Could you please download
Ian's latest build to your system, and try running the Pharo test using
Ian's build? The reason that I ask is that neither Ian nor I can reproduce
the UUID bug on our computers.  So if you can run Ian's VM on your computer
without problems for the UUID bug, this would indicate that it is safe
to distribute the VM with the internal UUID plugin for Squeak users.

You can download the VM from:
 http://squeakvm.org/unix/release/Squeak-4.0.3.2200-linux_i386.tar.gz

You do not need to install Ian's VM to test it, just unpack it into a
temporary directory, and run it from the temporary directory.

 $ tar xzvf Squeak-4.0.3.2200-linux_i386.tar.gz
 $ cd Squeak-4.0.3.2200-linux_i386/bin/squeak
 $ ./squeak ../../pharo.image

Thanks a lot, this is a big help :)

Dave

 

I have just built last release (Squeak-4.0.3.2200-src.tar.gz) with UUID as internal and it works like a charm. All Pharo 1.0-RC4 tests green ! Thank you all.

So there's a real problem when UUID is external. 

Cheers,

Laurent Laffont

On Tue, Apr 13, 2010 at 1:36 AM, David T. Lewis <[hidden email]> wrote:

On Mon, Apr 12, 2010 at 01:48:32PM +0200, laurent laffont wrote:
>
> I tend to think that if it's really a libuuid problem the VM would crash
> whether UUIDPlugin is built Internal or External.

Hi Laurent,

This bug is documented on Mantis
 http://bugs.squeak.org/view.php?id=7358

Can you confirm that the bug does *not* exist when you build the plugin
as internal, and that it *does" exist if you build it external on the
same platform. It would be a big help to know if this is the case.

My impression up until now is that this is a bug in the runtime libuuid
library, and that it has something to do with running a program with
pthreads (i.e. it does not happen with a simple test program). But I
have never actually seen the bug on my own system, so I'm guessing based
on some google searches and the information that I read on the Pharo list.

Thanks!

Dave

 
Hi Eliot,

I've built Squeak-4.0.3.2196-src (with UUID as External). My configure command was:

../unix/cmake/configure --CFLAGS="-Wl,-Bsymbolic"

VM crashes too.

Laurent Laffont

On Tue, Apr 13, 2010 at 7:20 PM, Eliot Miranda <[hidden email]> wrote:

 

On Mon, Apr 12, 2010 at 11:12 PM, laurent laffont <[hidden email]> wrote:

 

I have just built last release (Squeak-4.0.3.2200-src.tar.gz) with UUID as internal and it works like a charm. All Pharo 1.0-RC4 tests green ! Thank you all.

So there's a real problem when UUID is external. 

Has anybody looked at the issue of symbol scope in linux as being a possible cause here?  The default linux binding rules are that symbols in libraries are process-wide and the first loaded definition of a symbol is used in all subsequent library loads to resolve references.  The result is that if, say, one links an external plugin against some utility code that also exists in the VM, then on loading the external plugin its references to the utility code will be resolved to the definitions in the VM, not its own copy.  This may be fine until you have two external plugins linked against utility libraries that expect to have their own copies of that utility code.  When the second external plugin is loaded it gets linked against the first plugin's utility code and they end up sharing one copy which can cause confusion and delay ;)

In Cog on linux for external plugins I use the -Bsymbolic link flag to override the default binding rule.  This form the ld man page:

       -Bsymbolic

           When creating a shared library, bind references to  global  symbols

           to  the definition within the shared library, if any.  Normally, it

           is possible for a program linked against a shared library to  over-

           ride the definition within the shared library.  This option is only

           meaningful on ELF platforms which support shared libraries.

So when I link an external plugin I use libtool ....  -Wl,-Bsymbolic ...

If this looks at all relevant then could someone who has a system on which the external UUID plugin fails, try rebuilding the external plugin using the -Wl,-Bsymbolic flag and see if it fixes things?  I'm imagining that the symbol visiblity rules could affect whether variables used in the uuid seed are shared or not or initialized or not, etc.

HTH

Eliot

Cheers,

Laurent Laffont

On Tue, Apr 13, 2010 at 1:36 AM, David T. Lewis <[hidden email]> wrote:

On Mon, Apr 12, 2010 at 01:48:32PM +0200, laurent laffont wrote:
>
> I tend to think that if it's really a libuuid problem the VM would crash
> whether UUIDPlugin is built Internal or External.

Hi Laurent,

This bug is documented on Mantis
 http://bugs.squeak.org/view.php?id=7358

Can you confirm that the bug does *not* exist when you build the plugin
as internal, and that it *does" exist if you build it external on the
same platform. It would be a big help to know if this is the case.

My impression up until now is that this is a bug in the runtime libuuid
library, and that it has something to do with running a program with
pthreads (i.e. it does not happen with a simple test program). But I
have never actually seen the bug on my own system, so I'm guessing based
on some google searches and the information that I read on the Pharo list.

Thanks!

Dave

On Apr 14, 2010, at 9:08 AM, Eliot Miranda wrote:

I wouldn't have thought that CFLAGS was where this belongs.  LDFLAGS or some such.  Can you check that the libtool

libtool (along with all auto* tools) was removed from the Unix build process in September of last year.

invocation includes  -Wl,-Bsymbolic?

"configure --CFLAGS=x" sets CMAKE_C_FLAGS=x and the latter should indeed be passed to the link command.

Laurent: to confirm this, go to your build directory, delete squeakvm, then run "make VERBOSE=1" and scroll up to the link command (which will be just under a line highlighted in red, assuming your terminal does colour).

Cheers,
Ian

 

On Wed, Apr 14, 2010 at 6:29 PM, Ian Piumarta <[hidden email]> wrote:

On Apr 14, 2010, at 9:08 AM, Eliot Miranda wrote:

I wouldn't have thought that CFLAGS was where this belongs.  LDFLAGS or some such.  Can you check that the libtool

libtool (along with all auto* tools) was removed from the Unix build process in September of last year.

invocation includes  -Wl,-Bsymbolic?

"configure --CFLAGS=x" sets CMAKE_C_FLAGS=x and the latter should indeed be passed to the link command.

Laurent: to confirm this, go to your build directory, delete squeakvm, then run "make VERBOSE=1" and scroll up to the link command (which will be just under a line highlighted in red, assuming your terminal does colour).

Cheers,
Ian

Linking C executable squeakvm

/usr/bin/cmake -E cmake_link_script CMakeFiles/squeakvm.dir/link.txt --verbose=1                                                                                                   

/usr/bin/gcc  -Wl,-Bsymbolic  -Wl,--export-dynamic CMakeFiles/squeakvm.dir/gnu-interp.c.o CMakeFiles/squeakvm.dir/vm/aio.c.o CMakeFiles/squeakvm.dir/vm/debug.c.o CMakeFiles/squeakvm.dir/vm/osExports.c.o CMakeFiles/squeakvm.dir/vm/sqUnixCharConv.c.o CMakeFiles/squeakvm.dir/vm/sqUnixExternalPrims.c.o CMakeFiles/squeakvm.dir/vm/sqUnixMain.c.o CMakeFiles/squeakvm.dir/vm/sqUnixMemory.c.o CMakeFiles/squeakvm.dir/home/lol/sandbox/squeakvm/Squeak-4.0.3.2196-src/Cross/vm/sqNamedPrims.c.o CMakeFiles/squeakvm.dir/home/lol/sandbox/squeakvm/Squeak-4.0.3.2196-src/Cross/vm/sqVirtualMachine.c.o CMakeFiles/squeakvm.dir/version.c.o CMakeFiles/squeakvm.dir/disabledPlugins.c.o  -o squeakvm -rdynamic -lm -ldl ADPCMCodecPlugin/libADPCMCodecPlugin.a AsynchFilePlugin/libAsynchFilePlugin.a BMPReadWriterPlugin/libBMPReadWriterPlugin.a B2DPlugin/libB2DPlugin.a BitBltPlugin/libBitBltPlugin.a CroquetPlugin/libCroquetPlugin.a DSAPrims/libDSAPrims.a ZipPlugin/libZipPlugin.a DropPlugin/libDropPlugin.a FFTPlugin/libFFTPlugin.a FilePlugin/libFilePlugin.a FloatArrayPlugin/libFloatArrayPlugin.a FloatMathPlugin/libFloatMathPlugin.a GeniePlugin/libGeniePlugin.a JPEGReadWriter2Plugin/libJPEGReadWriter2Plugin.a JPEGReaderPlugin/libJPEGReaderPlugin.a JoystickTabletPlugin/libJoystickTabletPlugin.a Klatt/libKlatt.a LargeIntegers/libLargeIntegers.a LocalePlugin/libLocalePlugin.a Matrix2x3Plugin/libMatrix2x3Plugin.a MiscPrimitivePlugin/libMiscPrimitivePlugin.a RePlugin/libRePlugin.a SecurityPlugin/libSecurityPlugin.a SerialPlugin/libSerialPlugin.a SocketPlugin/libSocketPlugin.a SoundCodecPrims/libSoundCodecPrims.a SoundGenerationPlugin/libSoundGenerationPlugin.a SoundPlugin/libSoundPlugin.a StarSqueakPlugin/libStarSqueakPlugin.a SurfacePlugin/libSurfacePlugin.a -lnsl

Laurent Laffont

Hi Eliot,

I know you don't often read anything I write, but please read the next paragraph...

On Apr 14, 2010, at 11:55 AM, Eliot Miranda wrote:

but what's the libtool command for making the external UUID plugin?

Maybe you are using the word "libtool" 'generically'?  Since September last year there is NO libtool, nor any other autotools, in the Unix build process.

To answer your question: the link command on Linux for shared libraries in the VM looks like this

/usr/bin/gcc  -fPIC  YOUR-CMAKE_C_FLAGS-HERE   -shared -Wl,-soname,YOUR-MODULE-NAME-HERE -o YOUR-MODULE-NAME-HERE  YOUR-OBJECT-FILES-HERE  -lYOUR-REQUIRED-LIBS-HERE...

Whatever Laurent passes to --CFLAGS= will appear in the YOUR-CMAKE_C_FLAGS-HERE.  -Wl,-Bsymbolic, for example.

Regards,
Ian