RFC: Unix 3.11.3-2116 VM

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
26 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: RFC: Unix 3.11.3-2116 VM

Andreas Wacknitz
 
Hello Ian,

Am 31.08.2009 um 11:08 schrieb Ian Piumarta:

> Unix VM hackers,
>
> I've published a preview of the 3.11.3 Unix VM for the amusement of  
> alpha testers everywhere.  Lots of things are bound to be broken and/
> or forgotten and/or incomplete and/or begging for improvement.  I'm  
> not quite finished tweaking but comments and suggestions are  
> definitely welcome at this stage!
>
> Grab sources from the trunk of Subversion, or look in
>
>  http://squeakvm.org/unix/alpha
>
> for archives of sources and binaries for i386 (Linux, NetBSD,  
> FreeBSD and Solaris) and powerpc (Darwin).  (Files ending in '.sh'  
> are self-extracting archives; run as shell script and follow  
> instructions.)
>
> The noticeable changes since 3.10 are:
>
> 0. Plugins have been regenerated from latest releases
>
> At least for those on squeaksource.  Any hiding in more obscure  
> places might still be stale.
>
> 1. New build process based on CMake
>
> See README in the source archive or platforms/unix/README.CMake in  
> the repository.
>
> 2. Different plugin and FFI library search strategy
>
> The default plugin directory is now the VM binary directory.  This  
> is correct for installed VMs where the binary lives alongside the  
> plugins in /usr/.../lib/squeak/version/squeakvm.  A launcher script  
> installed as /usr/.../bin/squeak figures out where the VM is (either  
> installed or running from a build directory) and adds a -plugins  
> option appropriately.  The script also does the pulse audio OSS  
> kludge if necessary.  (Another script, squeak.sh, is intended to  
> launch Squeak from a menu.  It's begging for improvement.)
>
> SQUEAK_PLUGINS (or the argument to -plugins) can be set to a colon-
> separated list of places to look for plugins.  (Dave can debug one  
> plugin at a time using 'squeakvm -plugins bld/%n:/usr/lib/squeak/
> <version>'.)
>
> Libraries are searched only by whatever mechanisms dlopen() uses.  
> Applications that used to rely on the VM searching every nook and  
> cranny for FFI callouts may have to set LD_LIBRARY_PATH, or whatever  
> local equivalent is provided, to ensure their external functions'  
> libraries are found.
>
> 3. New numbering scheme
>
> The VM is now numbered X.Y.Z-R where X.Y.Z are the version of  
> VMMaker used to generate the Interpreter and plugins, and R is the  
> Subversion repository revision of the Unix support code.  (The  
> archives are named X.Y.Z.R, just because rpmbuild is grumpy  
> otherwise.)
>
> Cheers,
> Ian

I have compiled the actual sources (2123) for Solaris SPARC today. I  
had two things to fix in order to compile under Solaris 10:
   1. unix/plugins/PseudoTTYPlugin/sqUnixPseudoTTYPlugin.c includes  
"sqUnixAsynchFile.h" but this file is not in the search path.
   2. unix/plugins/UUIDPlugin/sqUnixUUID.c includes <uuid.h>. I needed  
to change it to include <uuid/uuid.h>.

The next thing is that "gmake install" installs the plugins under .../
lib/squeak/3.10-6/. Shouldn't that be 3.11.3?
At the moment the squeak executable cannot find the plugins and dumps  
core. But that has nothing to do with the version number.

Regards
Andreas

Reply | Threaded
Open this post in threaded view
|

Re: Re: RFC: Unix 3.11.3-2116 VM

David Farber
In reply to this post by Andreas.Raab
 

On Sep 1, 2009, at 10:12 AM, Andreas Raab wrote:

> David Farber wrote:
>> On Sep 1, 2009, at 9:28 AM, Ian Piumarta wrote:
>>> Is enabling IMAGE_DUMP in sqUnixMain.c sufficient for what you  
>>> need?  We could also have an option that names an image dump  
>>> file, disabling the dump if no name is given but still printing  
>>> the stack.
>> I was looking at this code earlier this year.  I couldn't convince  
>> myself that the resulting image would actually be usable.  If the  
>> VM just dumps the image, then won't you (at the very least) lose  
>> all your file handles including the handle to the changes file?  
>> And if you've lost your handle to the changes file then the image  
>> won't, for any practical purposes be usable.
>> Am I missing something?
>
> The simulator. It can be used to inspect the contents of an image  
> file regardless of whether you run it or it. Useful for post-mortem  
> analysis.

So how would you use the simulator to do a post-mortem?

Here is what happened to me back at the end of March.  I was running  
a Seaside/Pier site headless [1] on CentOS 5.  For persistency, Pier  
was supposed to snapshot the image after any relevant changes.  
Somewhere, somehow an error arose in the snapshot codepath, so Pier  
stopped snapshotting the image (even though the rest of the app ran  
fine and was accumulating data).  I tried to manually snapshot the  
image (and save precious data) but, because the error was in the  
snapshot codepath, all I managed to do was hang the web interface.  
Out of the box, Seaside doesn't do any error logging [2] and I wasn't  
running a VM that had IMAGE_DUMP or stack-printing enabled[3].  So I  
was stuck with an image that was running (with unsaved data) but  
completely incommunicado.

I was able to core dump the running image [4] and manually  
reconstruct an image file (i.e. what I would have had if my VM had  
IMAGE_DUMP enabled).  I loaded the image into the simulator, but I  
wasn't able to really do anything with it.  Specifically, I didn't  
see any way to debug why the image was failing to snapshot.  I was  
able to recover the data in the image[5].  But I still don't know  
what killed my image.  How could the simulator help me figure out  
what when wrong?

David

[1]  I wasn't running the image under any kind of remote X setup,  
which seems to be popular amongst Seaside deployers.
[2]  I won't make the mistake of deploying a Seaside app without  
error logging again.
[3]  I won't make the mistake of deploying a Seaside app on a VM  
without stack printing again.
[4]  On Linux, gcore will give you a core dump without terminating  
the process.  A version for OS X is at
   http://osxbook.com/book/bonus/chapter8/core/
[5]  I wrote code that will recover an object tree and write it to a  
ReferenceStream.  At some point I should package and release it.

Reply | Threaded
Open this post in threaded view
|

Triggering image save (was: Re: RFC: Unix 3.11.3-2116 VM)

David T. Lewis
 
On Tue, Sep 01, 2009 at 02:21:02PM -0600, David Farber wrote:

>
> Here is what happened to me back at the end of March.  I was running  
> a Seaside/Pier site headless [1] on CentOS 5.  For persistency, Pier  
> was supposed to snapshot the image after any relevant changes.  
> Somewhere, somehow an error arose in the snapshot codepath, so Pier  
> stopped snapshotting the image (even though the rest of the app ran  
> fine and was accumulating data).  I tried to manually snapshot the  
> image (and save precious data) but, because the error was in the  
> snapshot codepath, all I managed to do was hang the web interface.  
> Out of the box, Seaside doesn't do any error logging [2] and I wasn't  
> running a VM that had IMAGE_DUMP or stack-printing enabled[3].  So I  
> was stuck with an image that was running (with unsaved data) but  
> completely incommunicado.

This is probably redundant with what Ian previously described, but here
is another way to set up your image to do a save when you send SIGUSR2
to the vm. The image save has little or no impact on the running image
(i.e. no pause while the files are being written) because it is done
in a background OS process.

  sigUsr2Semaphore := OSProcess accessor forwardSigUsr2.
  sigWatcher := [[sigUsr2Semaphore wait.
    UnixProcess saveImageInBackgroundNicely.
    (Delay forSeconds: 30) wait] repeat] fork.

This would need to be run at image startup, because #forwardSigUsr2
sets the signal handler only for the duration of the current session.

I guess you could also use cron to schedule periodic image saves.

Dave

Reply | Threaded
Open this post in threaded view
|

Re: Triggering image save (was: Re: RFC: Unix 3.11.3-2116 VM)

David Farber
 

On Sep 1, 2009, at 6:23 PM, David T. Lewis wrote:

> This is probably redundant with what Ian previously described, but  
> here
> is another way to set up your image to do a save when you send SIGUSR2
> to the vm. The image save has little or no impact on the running image
> (i.e. no pause while the files are being written) because it is done
> in a background OS process.
>
>   sigUsr2Semaphore := OSProcess accessor forwardSigUsr2.
>   sigWatcher := [[sigUsr2Semaphore wait.
>     UnixProcess saveImageInBackgroundNicely.
>     (Delay forSeconds: 30) wait] repeat] fork.
>
> This would need to be run at image startup, because #forwardSigUsr2
> sets the signal handler only for the duration of the current session.
>
> I guess you could also use cron to schedule periodic image saves.

Thanks Dave.  I've seen your previous posts where you've recommended  
saveImageInBackgroundNicely and had already planned to take that  
route next time I deploy a Seaside app.  And yes, I'll use cron to  
fire off nightly backups.

David

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Unix 3.11.3-2116 VM

David T. Lewis
In reply to this post by Ian Piumarta
 
I would be much too embarassed to admit how much time I spent poking around
in gdb trying to figure out why the vm-display-X11 module was not loading on
my 64 bit linux before it finally occurred to me that I should read the instructions
and run platforms/unix/cmake/configure instead of platforms/unix/config/configure.

So I'm not going to admit to that, it probably never happened. But just on the off
chance that there might actually be *other* people dumb enough to do such a thing,
I'm going to suggest adding this to the top of platforms/unix/config/configure:

  echo platforms/unix/config/configure is obsolete, please use platforms/unix/cmake/configure
  exit

sheepishly,
Dave

Reply | Threaded
Open this post in threaded view
|

Re: RFC: Unix 3.11.3-2116 VM

David T. Lewis
In reply to this post by Bert Freudenberg
 
On 31.08.2009, at 11:08, Ian Piumarta wrote:
>
> Unix VM hackers,
>
> I've published a preview of the 3.11.3 Unix VM for the amusement of  
> alpha testers everywhere.  Lots of things are bound to be broken and/
> or forgotten and/or incomplete and/or begging for improvement.  I'm  
> not quite finished tweaking but comments and suggestions are  
> definitely welcome at this stage!

Hi Ian,

Attached is an updated platforms/cmake/configure script that does a
better job of building in a directory outside of the platforms tree.
It fixes the generation of VM_VERSION (and hence also the name of the
target install directory). It also causes --src=<directory> to do the
right thing for both VM_VERSION and for the cmake argument list when
specifying a source directory outside of the platforms tree. If the
"--src=<directory" option is not specified, it defaults to the sources
in platforms/unix/src.

For background, here is why this was a problem for me. My customary
directory layout is:

  ./SVN/squeak-svn-source-2127/platforms
  ./platforms -> ./SVN/squeak-svn-source-2127/platforms
  ./src32
  ./src64
  ./src -> ./src32
  ./build32
  ./build64

When using the config/configure script, I would work in the ./build32
(or ./build64) directory. When running configure without arguments,
the default ../src source location would follow my symlink to ../src32,
hence "cd build32; ..platforms/unix/config/configure; make; make install"
to build a VM.

When running the new cmake/configure script, I was expecting it to
find ../src32 but instead it used ../platforms/unix/src. In addition,
cmake had CMAKE_VERBOSE_MAKEFILE set to FALSE, so it cheerfully built
my VM from the wrong source directory, and I was none the wiser until
I noticed that changes to a plugin that I was debugging were not getting
included in the build.

Phew, how did I get off on that tangent? Oh, yes, I was going to look
into that "[Vm-dev] Bug in BitBlt. Need help" thing that Juan posted ;)

Dave


configure (3K) Download Attachment
platforms-unix-cmake-configure.diff (2K) Download Attachment
12