OSProcess fork issue with Debian built VM

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
44 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: OSProcess fork issue with Debian built VM

Nicolas Cellier
 
Hi Holger,
thanks for reminding this.
So we must regenerate source from  VMConstruction-Plugins-OSProcessPlugin.oscog-dtl.56
I just did that for nice.55 (for a 2nd time), a pity that I missed the more recent one...
It's better to send such message to vm-dev as you did, I thnk audience is broader for VM concerns than pharo-dev

2017-06-01 11:32 GMT+02:00 Holger Freyther <[hidden email]>:


> On 16. May 2017, at 20:57, Max Leske <[hidden email]> wrote:
>
>
> Hi,

Hi!


> We're building the pharo.cog.spur VM on Debian (32 bits, itimer, on 64 with compat libs). Unfortunately, we're running into one particular problem with OSProcess. Some primitives like primitiveFileStat work but forking a child process (e.g. with #waitForCommand) produces a segfault. We've tried to find a difference between our build and the VM downloaded from bintray but we weren't very successful. One difference we do see is that our libc6 version is 2.19, while the one used in the travis build is 2.15 (although both binaries use the 2.19 version at runtime).
>
> I'm aware that OSProcess hasn't been tagged as officially ready for Pharo 6 (which is where I'm using it). However, all of my tests with the prebuilt VM's have been successful.


I wanted to look at the issue and it seems that the compiler warning about out of bound array access[1] in restoreDefaultSignalHandlers remains unfixed? I reported it in February and the plugin code doesn't seem to be regenerated since then. Some of my PRs are still unreviewed so does somebody else want to try her or his luck?

I think this[2] and that[3] are candidates to be merged?


holger




[1] http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/2017-February/127495.html
[2] http://www.squeaksource.com/@ZzgxlHh7fhZ1n9Zo/rdfZ1xy3
[3] http://www.squeaksource.com/@ZzgxlHh7fhZ1n9Zo/wVMJyP1K

Reply | Threaded
Open this post in threaded view
|

Re: OSProcess fork issue with Debian built VM

Nicolas Cellier
 

2017-06-01 14:38 GMT+02:00 Nicolas Cellier <[hidden email]>:
Hi Holger,
thanks for reminding this.
So we must regenerate source from  VMConstruction-Plugins-OSProcessPlugin.oscog-dtl.56
I just did that for nice.55 (for a 2nd time), a pity that I missed the more recent one...
It's better to send such message to vm-dev as you did, I thnk audience is broader for VM concerns than pharo-dev

2017-06-01 11:32 GMT+02:00 Holger Freyther <[hidden email]>:


> On 16. May 2017, at 20:57, Max Leske <[hidden email]> wrote:
>
>
> Hi,

Hi!


> We're building the pharo.cog.spur VM on Debian (32 bits, itimer, on 64 with compat libs). Unfortunately, we're running into one particular problem with OSProcess. Some primitives like primitiveFileStat work but forking a child process (e.g. with #waitForCommand) produces a segfault. We've tried to find a difference between our build and the VM downloaded from bintray but we weren't very successful. One difference we do see is that our libc6 version is 2.19, while the one used in the travis build is 2.15 (although both binaries use the 2.19 version at runtime).
>
> I'm aware that OSProcess hasn't been tagged as officially ready for Pharo 6 (which is where I'm using it). However, all of my tests with the prebuilt VM's have been successful.


I wanted to look at the issue and it seems that the compiler warning about out of bound array access[1] in restoreDefaultSignalHandlers remains unfixed? I reported it in February and the plugin code doesn't seem to be regenerated since then. Some of my PRs are still unreviewed so does somebody else want to try her or his luck?

I think this[2] and that[3] are candidates to be merged?


holger




[1] http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/2017-February/127495.html
[2] http://www.squeaksource.com/@ZzgxlHh7fhZ1n9Zo/rdfZ1xy3
[3] http://www.squeaksource.com/@ZzgxlHh7fhZ1n9Zo/wVMJyP1K


Reply | Threaded
Open this post in threaded view
|

Re: OSProcess fork issue with Debian built VM

Eliot Miranda-2
In reply to this post by Nicolas Cellier
 
Hi David,

     what would help here is if all the VMConstruction-Plugins- packages lived in a single repository.  So instead of <a href="http://www.squeaksource.com/{AioPlugin,OSProcessPlugin,XDCP}">http://www.squeaksource.com/{AioPlugin,OSProcessPlugin,XDCP} there was http://www.squeaksource.com/OSConnectivity or http://www.squeaksource.com/AdditionalPlugins or some such.  Checking lots of different repositories for changes is tedious.  If there was one I could keep a repository inspector open and refresh.

Alternatively we need Update URLs instead of Update URL.

On Thu, Jun 1, 2017 at 5:38 AM, Nicolas Cellier <[hidden email]> wrote:
 
Hi Holger,
thanks for reminding this.
So we must regenerate source from  VMConstruction-Plugins-OSProcessPlugin.oscog-dtl.56
I just did that for nice.55 (for a 2nd time), a pity that I missed the more recent one...
It's better to send such message to vm-dev as you did, I thnk audience is broader for VM concerns than pharo-dev

2017-06-01 11:32 GMT+02:00 Holger Freyther <[hidden email]>:


> On 16. May 2017, at 20:57, Max Leske <[hidden email]> wrote:
>
>
> Hi,

Hi!


> We're building the pharo.cog.spur VM on Debian (32 bits, itimer, on 64 with compat libs). Unfortunately, we're running into one particular problem with OSProcess. Some primitives like primitiveFileStat work but forking a child process (e.g. with #waitForCommand) produces a segfault. We've tried to find a difference between our build and the VM downloaded from bintray but we weren't very successful. One difference we do see is that our libc6 version is 2.19, while the one used in the travis build is 2.15 (although both binaries use the 2.19 version at runtime).
>
> I'm aware that OSProcess hasn't been tagged as officially ready for Pharo 6 (which is where I'm using it). However, all of my tests with the prebuilt VM's have been successful.


I wanted to look at the issue and it seems that the compiler warning about out of bound array access[1] in restoreDefaultSignalHandlers remains unfixed? I reported it in February and the plugin code doesn't seem to be regenerated since then. Some of my PRs are still unreviewed so does somebody else want to try her or his luck?

I think this[2] and that[3] are candidates to be merged?


holger




[1] http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/2017-February/127495.html
[2] http://www.squeaksource.com/@ZzgxlHh7fhZ1n9Zo/rdfZ1xy3
[3] http://www.squeaksource.com/@ZzgxlHh7fhZ1n9Zo/wVMJyP1K





--
_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|

Re: OSProcess fork issue with Debian built VM

David T. Lewis
 
On Thu, Jun 01, 2017 at 04:50:40PM -0700, Eliot Miranda wrote:

> Hi David,
>
>      what would help here is if all the VMConstruction-Plugins- packages
> lived in a single repository.  So instead of
> http://www.squeaksource.com/{AioPlugin,OSProcessPlugin,XDCP} there was
> http://www.squeaksource.com/OSConnectivity or
> http://www.squeaksource.com/AdditionalPlugins or some such.  Checking lots
> of different repositories for changes is tedious.  If there was one I could
> keep a repository inspector open and refresh.
>
> Alternatively we need Update URLs instead of Update URL.
>

Hi Eliot,

Try using update maps. There is already an update stream for VMMaker (as opposed
to VMMaker.oscog), so you can name your update maps "update.oscog" to
distinguish them from the existing "update" maps.

Then do something like this:

  VMMaker class>>updateFromServer
 
      "VMMaker updateFromServer"
 
      (Smalltalk hasClassNamed: #MCMcmUpdater)
          ifTrue: [ | updater |
              updater := Smalltalk at: #MCMcmUpdater.
              (updater respondsTo: #updateFromRepository:baseName: )
                  ifTrue: [ "newer MCMcmUpdater supports multiple update streams per repository"
                      updater perform: #updateFromRepository:baseName:
                              withArguments: { 'http://source.squeak.org/VMMaker' . 'update.oscog' } ]
                  ifFalse: [ "older versions of MCMcmUpdater"
                          updater updateFromRepositories: #('http://source.squeak.org/VMMaker') ] ]
          ifFalse: [self notify: 'MonticelloConfigurations not installed in this image']

I don't mind moving the plugins I have written, but I would really like to
encourage more people to write and contribute their own VM plugins. For
that I think it is very helpful for someone to be able to start their own
small plugin experiment in squeaksource/whatever, and offer it for other
people to build and use, and maybe later for inclusion in supported VMs if
it turns out to be of general interest. I might be a little biased here
because that is how I got involved in this community myself, and I do think
that it is a good thing to encourage.

Dave


> On Thu, Jun 1, 2017 at 5:38 AM, Nicolas Cellier <
> [hidden email]> wrote:
>
> >
> > Hi Holger,
> > thanks for reminding this.
> > So we must regenerate source from
> > *VMConstruction-Plugins-OSProcessPlugin.oscog-dtl.56*
> > I just did that for nice.55 (for a 2nd time), a pity that I missed the
> > more recent one...
> > It's better to send such message to vm-dev as you did, I thnk audience is
> > broader for VM concerns than pharo-dev
> >
> > 2017-06-01 11:32 GMT+02:00 Holger Freyther <[hidden email]>:
> >
> >>
> >>
> >> > On 16. May 2017, at 20:57, Max Leske <[hidden email]> wrote:
> >> >
> >> >
> >> > Hi,
> >>
> >> Hi!
> >>
> >>
> >> > We're building the pharo.cog.spur VM on Debian (32 bits, itimer, on 64
> >> with compat libs). Unfortunately, we're running into one particular problem
> >> with OSProcess. Some primitives like primitiveFileStat work but forking a
> >> child process (e.g. with #waitForCommand) produces a segfault. We've tried
> >> to find a difference between our build and the VM downloaded from bintray
> >> but we weren't very successful. One difference we do see is that our libc6
> >> version is 2.19, while the one used in the travis build is 2.15 (although
> >> both binaries use the 2.19 version at runtime).
> >> >
> >> > I'm aware that OSProcess hasn't been tagged as officially ready for
> >> Pharo 6 (which is where I'm using it). However, all of my tests with the
> >> prebuilt VM's have been successful.
> >>
> >>
> >> I wanted to look at the issue and it seems that the compiler warning
> >> about out of bound array access[1] in restoreDefaultSignalHandlers remains
> >> unfixed? I reported it in February and the plugin code doesn't seem to be
> >> regenerated since then. Some of my PRs are still unreviewed so does
> >> somebody else want to try her or his luck?
> >>
> >> I think this[2] and that[3] are candidates to be merged?
> >>
> >>
> >> holger
> >>
> >>
> >>
> >>
> >> [1] http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/
> >> 2017-February/127495.html
> >> [2] http://www.squeaksource.com/@ZzgxlHh7fhZ1n9Zo/rdfZ1xy3
> >> [3] http://www.squeaksource.com/@ZzgxlHh7fhZ1n9Zo/wVMJyP1K
> >
> >
> >
> >
>
>
> --
> _,,,^..^,,,_
> best, Eliot
Reply | Threaded
Open this post in threaded view
|

Re: OSProcess fork issue with Debian built VM

EstebanLM
 
Hi,

I’m sorry, but all of this is a mess.
Why not simply require that all plugins that are part of the VM lives in the same place?

In an ideal world, that unified place would be VMMaker repository itself, but if not… why not a /VMPlugins project?

cheers,
Esteban

On 2 Jun 2017, at 02:59, David T. Lewis <[hidden email]> wrote:


On Thu, Jun 01, 2017 at 04:50:40PM -0700, Eliot Miranda wrote:
Hi David,

   what would help here is if all the VMConstruction-Plugins- packages
lived in a single repository.  So instead of
http://www.squeaksource.com/{AioPlugin,OSProcessPlugin,XDCP} there was
http://www.squeaksource.com/OSConnectivity or
http://www.squeaksource.com/AdditionalPlugins or some such.  Checking lots
of different repositories for changes is tedious.  If there was one I could
keep a repository inspector open and refresh.

Alternatively we need Update URLs instead of Update URL.


Hi Eliot,

Try using update maps. There is already an update stream for VMMaker (as opposed
to VMMaker.oscog), so you can name your update maps "update.oscog" to
distinguish them from the existing "update" maps.

Then do something like this:

VMMaker class>>updateFromServer

    "VMMaker updateFromServer"

    (Smalltalk hasClassNamed: #MCMcmUpdater)
        ifTrue: [ | updater |
            updater := Smalltalk at: #MCMcmUpdater.
            (updater respondsTo: #updateFromRepository:baseName: )
                ifTrue: [ "newer MCMcmUpdater supports multiple update streams per repository"
                    updater perform: #updateFromRepository:baseName:
                            withArguments: { 'http://source.squeak.org/VMMaker' . 'update.oscog' } ]
                ifFalse: [ "older versions of MCMcmUpdater"
                        updater updateFromRepositories: #('http://source.squeak.org/VMMaker') ] ]
        ifFalse: [self notify: 'MonticelloConfigurations not installed in this image']

I don't mind moving the plugins I have written, but I would really like to
encourage more people to write and contribute their own VM plugins. For
that I think it is very helpful for someone to be able to start their own
small plugin experiment in squeaksource/whatever, and offer it for other
people to build and use, and maybe later for inclusion in supported VMs if
it turns out to be of general interest. I might be a little biased here
because that is how I got involved in this community myself, and I do think
that it is a good thing to encourage.

Dave


On Thu, Jun 1, 2017 at 5:38 AM, Nicolas Cellier <
[hidden email]> wrote:


Hi Holger,
thanks for reminding this.
So we must regenerate source from
*VMConstruction-Plugins-OSProcessPlugin.oscog-dtl.56*
I just did that for nice.55 (for a 2nd time), a pity that I missed the
more recent one...
It's better to send such message to vm-dev as you did, I thnk audience is
broader for VM concerns than pharo-dev

2017-06-01 11:32 GMT+02:00 Holger Freyther <[hidden email]>:



On 16. May 2017, at 20:57, Max Leske <[hidden email]> wrote:


Hi,

Hi!


We're building the pharo.cog.spur VM on Debian (32 bits, itimer, on 64
with compat libs). Unfortunately, we're running into one particular problem
with OSProcess. Some primitives like primitiveFileStat work but forking a
child process (e.g. with #waitForCommand) produces a segfault. We've tried
to find a difference between our build and the VM downloaded from bintray
but we weren't very successful. One difference we do see is that our libc6
version is 2.19, while the one used in the travis build is 2.15 (although
both binaries use the 2.19 version at runtime).

I'm aware that OSProcess hasn't been tagged as officially ready for
Pharo 6 (which is where I'm using it). However, all of my tests with the
prebuilt VM's have been successful.


I wanted to look at the issue and it seems that the compiler warning
about out of bound array access[1] in restoreDefaultSignalHandlers remains
unfixed? I reported it in February and the plugin code doesn't seem to be
regenerated since then. Some of my PRs are still unreviewed so does
somebody else want to try her or his luck?

I think this[2] and that[3] are candidates to be merged?


holger




[1] http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/
2017-February/127495.html
[2] http://www.squeaksource.com/@ZzgxlHh7fhZ1n9Zo/rdfZ1xy3
[3] http://www.squeaksource.com/@ZzgxlHh7fhZ1n9Zo/wVMJyP1K






--
_,,,^..^,,,_
best, Eliot

Reply | Threaded
Open this post in threaded view
|

Re: OSProcess fork issue with Debian built VM

K K Subbu
 
On Friday 02 June 2017 10:10 AM, Esteban Lorenzano wrote:
> I’m sorry, but all of this is a mess.
> Why not simply require that all plugins that are part of the VM lives in
> the same place?
>
> In an ideal world, that unified place would be VMMaker repository
> itself, but if not… why not a /VMPlugins project?

+1. I would go further and propose that plugins, other than those which
are integral to VM[1], be moved into separate projects and built in
their own tree with no references to vm source paths. They should be
free to organize their platform-specific and platform-independent code
and makefiles.

I suppose this requires factoring essential VM functions into libstvm
(runtime+header files) and then generate integral plugins using this
library. The main VM can then parse options, open image file and invoke
the interpreter main loop. It could also dynamically load a matching
interpreter lib for the image instead of bailing out with an unknown
image error.

To start with, libstvm components can be exported into
$BUILD/lib/libstvm.so and $BUILD/lib/include/sq.h ...

Has anyone tried this before?

[1] like vm-display-null, vm-sound-null, fileplugin, osprocess etc.

Regards .. Subbu
Reply | Threaded
Open this post in threaded view
|

Re: OSProcess fork issue with Debian built VM

Holger Freyther
 

> On 2. Jun 2017, at 18:43, K K Subbu <[hidden email]> wrote:
>

Hey!


> +1. I would go further and propose that plugins, other than those which are integral to VM[1], be moved into separate projects and built in their own tree with no references to vm source paths. They should be free to organize their platform-specific and platform-independent code and makefiles.

why? How many plugins are used that don't ship with the pharo-vm (or squeakvm)? I have to say I like the pharo-vm approach a lot where the VMMaker* packages and generated sources sit next to each other.

Let's compare two options with the OSProcess timeline at hand


What happened:

* I compile with a new compiler, look at the compiler warning and agree with it
* David Lewis agrees with my analysis and fixes it
...
* Max runs into a crash
* Max debugs a crash.. hours? days?
* Max involves others
...
* I have to debug the crash myself..
* I notice that the fix is not there yet
* I have to wait for someone privileged to regenerate the source (black magic?)


My preferred option

* I compile with a new compiler, look at the compiler warning and agree with it
* David Lewis agrees with my analysis and commits/pushes the fix to mc/ directory
* New VM will be build and regenerates sources on the fly
* Max never runs into the problem

Let's do it the Smalltalk way? Make simple things simple and complex (allow externally maintained plugins) possible?

have a nice weekend

        holger
Reply | Threaded
Open this post in threaded view
|

Re: OSProcess fork issue with Debian built VM

Max Leske
In reply to this post by Max Leske
 

On 2 Jun 2017, at 14:00, [hidden email] wrote:

Hey!


+1. I would go further and propose that plugins, other than those which are integral to VM[1], be moved into separate projects and built in their own tree with no references to vm source paths. They should be free to organize their platform-specific and platform-independent code and makefiles.

why? How many plugins are used that don't ship with the pharo-vm (or squeakvm)? I have to say I like the pharo-vm approach a lot where the VMMaker* packages and generated sources sit next to each other.

Let's compare two options with the OSProcess timeline at hand


What happened:

* I compile with a new compiler, look at the compiler warning and agree with it
* David Lewis agrees with my analysis and fixes it
...
* Max runs into a crash
* Max debugs a crash.. hours? days?
* Max involves others
...
* I have to debug the crash myself..
* I notice that the fix is not there yet
* I have to wait for someone privileged to regenerate the source (black magic?)


My preferred option

* I compile with a new compiler, look at the compiler warning and agree with it
* David Lewis agrees with my analysis and commits/pushes the fix to mc/ directory
* New VM will be build and regenerates sources on the fly
* Max never runs into the problem

(Dancing to "Don't know much") ...Oh what a wonderful world this would be!...


Let's do it the Smalltalk way? Make simple things simple and complex (allow externally maintained plugins) possible?

have a nice weekend

holger

Reply | Threaded
Open this post in threaded view
|

Re: OSProcess fork issue with Debian built VM

Eliot Miranda-2
In reply to this post by Holger Freyther
 
Hi Holger,

    I have no disagreement with your time lines, but I do disagree with the black magic accusation, and you introduce your own piece of magic (see below).

> On Jun 2, 2017, at 4:20 AM, Holger Freyther <[hidden email]> wrote:
>
>
>
>> On 2. Jun 2017, at 18:43, K K Subbu <[hidden email]> wrote:
>>
>
> Hey!
>
>
>> +1. I would go further and propose that plugins, other than those which are integral to VM[1], be moved into separate projects and built in their own tree with no references to vm source paths. They should be free to organize their platform-specific and platform-independent code and makefiles.
>
> why? How many plugins are used that don't ship with the pharo-vm (or squeakvm)? I have to say I like the pharo-vm approach a lot where the VMMaker* packages and generated sources sit next to each other.
>
> Let's compare two options with the OSProcess timeline at hand
>
>
> What happened:
>
> * I compile with a new compiler, look at the compiler warning and agree with it
> * David Lewis agrees with my analysis and fixes it
> ...
> * Max runs into a crash
> * Max debugs a crash.. hours? days?
> * Max involves others
> ...
> * I have to debug the crash myself..
> * I notice that the fix is not there yet
> * I have to wait for someone privileged to regenerate the source (black magic?)

One builds a VMMaker image, which will pull in the new source, and therein evaluates

        VMMaker generateVMPlugins

and then checks in the new sources.  David should have been able to do this.  The process is not documented.

The souce generation process is also set up to generate new source for plugins for those classes that have been changed since last generation, on the basis of a time stamp (it has ever been so).  This is a problem; if one builds a VMMaker image just to regenerate source for plugins then all plugins will be generated.  In this case there's no harm done because the check in script (scripts/gitci) reverts any plugins whose code hasn't changed.  But if some new vm source needed to be generated (say one method in one JIT backend was changed) then all source would be regenerated by

    [VMMaker generateAllConfigurationsUnderVersionControl] valueAnswering: false

(The false says don't generate new source for header files and classes that haven't changed).  The time stamp scheme works for me because I keep a VMMaker image (it's my life) and generate when required.  It doesn't work for someone wanting to build an image and regenerate.

So the time stamp stuff needs more thought.  If, on building a VMMaker image, class timestamps were derived from method timestamps then the classes would be accurate.  Then if one applied the scripts/gitrevert, which uses touch -t to set last modification times to check in times, to the generated source files in the source tree their time stamps would be accurate.  The trouble is that I believe at least the Smalltalk method timestamps are in local time, not UTC.

>
>
> My preferred option
>
> * I compile with a new compiler, look at the compiler warning and agree with it
> * David Lewis agrees with my analysis and commits/pushes the fix to mc/ directory

You're glossing over some steps here.  Note that the fix is in the Smalltalk source, upstream of the C source.

The missing step is that the plugin source is regenerated .  As a Smalltalk-side-of-the-business person David knows how to regenerate source.  David, why didn't you regenerate? [honest Q, not an accusation]

One *mustn't* manually fix the generated source because doing so means
- the fix is lost the next time source is regenerated from the Smalltalk
- the C source has a misleading version stamp so is effectively untracked, masquerading as something else

One *must* regenerate from an unmodified checked in Monticello package because doing so means
- the C source is version stamped as originating from that package and so one can locate the correct version of the Smalltalk from which it derived

> * New VM will be build and regenerates sources on the fly

How does this work? Seems like magic to me.  Source has to be generated from a VMMaker image.  We need to regenerate source only for that which has changed, not for the entire generated source.  But right now that's problematic because of the time issues I mention above.

> * Max never runs into the problem
>
> Let's do it the Smalltalk way? Make simple things simple and complex (allow externally maintained plugins) possible?

Agreed.  No need to complicate source generation.  But we still need people who fix the Smalltalk to be able to regenerate source.  Several of us, Ronie, Clément, Nicolas, myself.  David, you should feel free to do so also.

>
> have a nice weekend
>
>    holger
Reply | Threaded
Open this post in threaded view
|

Re: OSProcess fork issue with Debian built VM

K K Subbu
In reply to this post by Holger Freyther
 
On Friday 02 June 2017 04:50 PM, Holger Freyther wrote:

>
>> +1. I would go further and propose that plugins, other than those
>> which are integral to VM[1], be moved into separate projects and
>> built in their own tree with no references to vm source paths. They
>> should be free to organize their platform-specific and
>> platform-independent code and makefiles.
>
> why? How many plugins are used that don't ship with the pharo-vm (or
> squeakvm)? I have to say I like the pharo-vm approach a lot where the
> VMMaker* packages and generated sources sit next to each other.

My proposal is to loosen build time coupling, not packaging. Nothing
prevents plugins built out of tree from being built, installed and
shipped together. Supporting out of tree plugins allows others to extend
a VM even long after its release.

Eliot has already responded to your example in detail, so I will not
dwell upon it further, except to point out that we need a command line
tool to generate a .c file directly from the plugin's mcz file. Then we
can create a Make rule to compile a .o and get rid of .c file. e.g.

  FooPlugin.o : FooPlugin.mcz
        squeak -headless vmmaker.image gencc.st FooPlugin.mcz
        cc -c ... FooPlugin.c
        rm -f FooPlugin.c

where st2c should generate the .c file for FooPlugin from its Slang code
in its package or from a .st file. There is no need for .c file to be
human readable or be subject to version control. That will be very messy.

Regards .. Subbu
Reply | Threaded
Open this post in threaded view
|

Re: OSProcess fork issue with Debian built VM

David T. Lewis
In reply to this post by Eliot Miranda-2
 
Hi Eliot,

>
> The missing step is that the plugin source is regenerated .  As a
> Smalltalk-side-of-the-business person David knows how to regenerate
> source.  David, why didn't you regenerate? [honest Q, not an accusation]

To be honest, I just didn't think of it because I assumed that you were
doing the code generation.

>
> Agreed.  No need to complicate source generation.  But we still need
> people who fix the Smalltalk to be able to regenerate source.  Several of
> us, Ronie, Clément, Nicolas, myself.  David, you should feel free to do
> so also.
>

Noted, will do :-)

Thanks,
Dave


Reply | Threaded
Open this post in threaded view
|

Re: OSProcess fork issue with Debian built VM

Eliot Miranda-2
In reply to this post by K K Subbu
 


On Fri, Jun 2, 2017 at 9:54 AM, K K Subbu <[hidden email]> wrote:

On Friday 02 June 2017 04:50 PM, Holger Freyther wrote:

+1. I would go further and propose that plugins, other than those
which are integral to VM[1], be moved into separate projects and
built in their own tree with no references to vm source paths. They
should be free to organize their platform-specific and
platform-independent code and makefiles.

why? How many plugins are used that don't ship with the pharo-vm (or
squeakvm)? I have to say I like the pharo-vm approach a lot where the
VMMaker* packages and generated sources sit next to each other.

My proposal is to loosen build time coupling, not packaging. Nothing prevents plugins built out of tree from being built, installed and shipped together. Supporting out of tree plugins allows others to extend a VM even long after its release.

Eliot has already responded to your example in detail, so I will not dwell upon it further, except to point out that we need a command line tool to generate a .c file directly from the plugin's mcz file. Then we can create a Make rule to compile a .o and get rid of .c file. e.g.

 FooPlugin.o : FooPlugin.mcz
        squeak -headless vmmaker.image gencc.st FooPlugin.mcz
        cc -c ... FooPlugin.c
        rm -f FooPlugin.c

Over my dead body.  The make must *NEVER* be done like this.  The problems are many:
- the VM so produced is undebuggable.  There is no source for FooPlugin.c; it has been deleted
- the build is slow; building a VMMaker image (the necessary prerequisite for st2c) takes a long time, mush longer than the st2c step
- unless FooPlugin.mcz is versioned then the VM so produced is unversioned and if  FooPlugin.mcz /is/ versioned then there's no reason why the plugin can't be produced using the normal workflow, generate C source, check it in to opensmalltalk/vm, build, and that way we have a versioned entity and break the dependency of builds requiring VMMaker

Look, I just spent several years persuading the Paro community that generating source and then building was a really bad idea and they should shelve it and now someone is trying to bring it back.  No, no, and thrice times, no.

 

where st2c should generate the .c file for FooPlugin from its Slang code in its package or from a .st file. There is no need for .c file to be human readable or be subject to version control. That will be very messy.

_yes_.  _there_. _is_.
 

Regards .. Subbu



--
_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|

Re: OSProcess fork issue with Debian built VM

Ben Coman
 


On Sat, Jun 3, 2017 at 5:57 AM, Eliot Miranda <[hidden email]> wrote:
 


On Fri, Jun 2, 2017 at 9:54 AM, K K Subbu <[hidden email]> wrote:

On Friday 02 June 2017 04:50 PM, Holger Freyther wrote:

+1. I would go further and propose that plugins, other than those
which are integral to VM[1], be moved into separate projects and
built in their own tree with no references to vm source paths. They
should be free to organize their platform-specific and
platform-independent code and makefiles.

why? How many plugins are used that don't ship with the pharo-vm (or
squeakvm)? I have to say I like the pharo-vm approach a lot where the
VMMaker* packages and generated sources sit next to each other.

My proposal is to loosen build time coupling, not packaging. Nothing prevents plugins built out of tree from being built, installed and shipped together. Supporting out of tree plugins allows others to extend a VM even long after its release.

Eliot has already responded to your example in detail, so I will not dwell upon it further, except to point out that we need a command line tool to generate a .c file directly from the plugin's mcz file. Then we can create a Make rule to compile a .o and get rid of .c file. e.g.

 FooPlugin.o : FooPlugin.mcz
        squeak -headless vmmaker.image gencc.st FooPlugin.mcz
        cc -c ... FooPlugin.c
        rm -f FooPlugin.c

Over my dead body.  The make must *NEVER* be done like this.  The problems are many:
- the VM so produced is undebuggable.  There is no source for FooPlugin.c; it has been deleted
- the build is slow; building a VMMaker image (the necessary prerequisite for st2c) takes a long time, mush longer than the st2c step
- unless FooPlugin.mcz is versioned then the VM so produced is unversioned and if  FooPlugin.mcz /is/ versioned then there's no reason why the plugin can't be produced using the normal workflow, generate C source, check it in to opensmalltalk/vm, build, and that way we have a versioned entity and break the dependency of builds requiring VMMaker


And I remember it mentioned a while ago, the bug is sometimes in the generation-code, so you want historical access to both mcz and generated-C to help debug that. 

 
Look, I just spent several years persuading the Paro community that generating source and then building was a really bad idea and they should shelve it and now someone is trying to bring it back.  No, no, and thrice times, no.



I think Pharo has not completely abandoned generating C-sources, although its now a sideline activity...
On Wed, May 31, 2017 at 2:48 PM, Esteban Lorenzano <[hidden email]> wrote:

In any case, in the “pharo side” of things, we keep a parallel CI process who does this: 

1) there is a development branch that keep all sources we need: the osvm *and* all monticello packages (filetree format) acting as a mirror that merges changes and executes all building process (including generating sources) and testing (we run all pharo tests as a way to test the VM). This is done just for being sure the CI is green and no artifact is created. This is how I have early reported problem in VM generation before.
2) in parallel the build follows the “normal” process on opensmalltalk-vm and everything is build there. This generates a “latest” build which is published in pharo servers so people can test (but this binary could be wrong as is not something declared as “stable”


I do wonder from time to time is why the C-generation can't be done as part of an opensmalltalk-vm CI process?  The manual process is a bit opaque, and a CI process would be better than documentation.  One possibility of doing it via CI could be generating from both Squeak and Pharo and checking the output is identical (assuming that might be useful).

IIRC Eliot didn't want the generated-C updated in the Cog branch for every VMMaker commit.  A way to conform to this would be the CI updating the generated C in a side branch, and only when deemed worthy it could be integrated into the Cog branch via a simple merge rather than a one-time manual generation run.  

Eliot,  If you're receptive to this idea, I'd be interested in working on this.  What concerns would you have with it?

cheers -ben
 
 

where st2c should generate the .c file for FooPlugin from its Slang code in its package or from a .st file. There is no need for .c file to be human readable or be subject to version control. That will be very messy.

_yes_.  _there_. _is_.
 

Regards .. Subbu

--
_,,,^..^,,,_
best, Eliot


Reply | Threaded
Open this post in threaded view
|

Re: OSProcess fork issue with Debian built VM

Eliot Miranda-2
In reply to this post by David T. Lewis
 
Hi David,

> On Jun 2, 2017, at 10:51 AM, David T. Lewis <[hidden email]> wrote:
>
>
> Hi Eliot,
>
>>
>> The missing step is that the plugin source is regenerated .  As a
>> Smalltalk-side-of-the-business person David knows how to regenerate
>> source.  David, why didn't you regenerate? [honest Q, not an accusation]
>
> To be honest, I just didn't think of it because I assumed that you were
> doing the code generation.

Ah ok.  Well I'm happy to generate an occasional plugin for people as if doesn't take much time (but would if lots of people were working on different plugins).  But you'd have to send me a personal email.  I try and keep up with the commit emails but I live in a blizzard of email so sometimes things go unnoticed.

I think it's freeing that people who make changes in the Smalltalk side of things generate the new source from their own changes.

>
>>
>> Agreed.  No need to complicate source generation.  But we still need
>> people who fix the Smalltalk to be able to regenerate source.  Several of
>> us, Ronie, Clément, Nicolas, myself.  David, you should feel free to do
>> so also.
>>
>
> Noted, will do :-)

Cool, thanks!

>
> Thanks,
> Dave

_,,,^..^,,,_ (phone)
Reply | Threaded
Open this post in threaded view
|

Re: OSProcess fork issue with Debian built VM

K K Subbu
In reply to this post by Eliot Miranda-2
 
On Saturday 03 June 2017 03:27 AM, Eliot Miranda wrote:
>
> Over my dead body.  The make must *NEVER* be done like this.  The
> problems are many:

Ouch! :-). Please don't take my example too literally. Smalltalk does
not have a concept of a file-based compilation unit like C, so I had to
use an extension to denote a collection of st code that goes into
generating a .c compilation unit. I may well have used a .st or .zip for
source code.

> - the VM so produced is undebuggable.  There is no source for
> FooPlugin.c; it has been deleted

This is not the first time C-code gets generated on the fly. This
technique has been used by Lex, Yacc etc. since early 70s. Bison
generates a FSM interpreter in C from a Yacc spec. My example makefile
segment deleted the file, but it can also cache it for debugging.

> - the build is slow; building a VMMaker image (the necessary
> prerequisite for st2c) takes a long time, mush longer than the st2c step

The image is just a smalltalk app that translates slang to C. It does
not have to be regenerated in every build anymore than gcc or cmake. In
earlier posts, I had floated the idea of a bootstrap VM - a st machine
with a built-in image to build and launch larger VMs. It can even be a
separate project.

The .st code has to be translated to C at some time. If it is done
within a single VM, then we won't be able to exploit multi-cores. With
an st2c app, we can translate in parallel for a quicker build.

> - unless FooPlugin.mcz is versioned then the VM so produced is
> unversioned and if  FooPlugin.mcz /is/ versioned then there's no reason
> why the plugin can't be produced using the normal workflow, generate C
> source, check it in to opensmalltalk/vm, build, and that way we have a
> versioned entity and break the dependency of builds requiring VMMaker

Anyway, my point is that only the smalltalk "compilation unit" should be
under version control. If a .c file is generated from it, there is no
need to subject it to version control. It can continue to remain in a
cache. Like the way we preserve .i (preprocessed) or .s (assembly) files
for debugging .c code but never check-in these i. or .s files.

Okay, I had no idea I was stepping into a minefield of raw nerves here
;-). I will step back from these threads and do some prototyping and
come up with specific code changes. The code seems to have grown too
large for a lone hobbyist but it is still worth a try.

Regards .. Subbu
Reply | Threaded
Open this post in threaded view
|

Re: OSProcess fork issue with Debian built VM

Eliot Miranda-2
In reply to this post by Ben Coman
 
Hi Ben,

On Jun 3, 2017, at 2:22 AM, Ben Coman <[hidden email]> wrote:



On Sat, Jun 3, 2017 at 5:57 AM, Eliot Miranda <[hidden email]> wrote:
 


On Fri, Jun 2, 2017 at 9:54 AM, K K Subbu <[hidden email]> wrote:

On Friday 02 June 2017 04:50 PM, Holger Freyther wrote:

+1. I would go further and propose that plugins, other than those
which are integral to VM[1], be moved into separate projects and
built in their own tree with no references to vm source paths. They
should be free to organize their platform-specific and
platform-independent code and makefiles.

why? How many plugins are used that don't ship with the pharo-vm (or
squeakvm)? I have to say I like the pharo-vm approach a lot where the
VMMaker* packages and generated sources sit next to each other.

My proposal is to loosen build time coupling, not packaging. Nothing prevents plugins built out of tree from being built, installed and shipped together. Supporting out of tree plugins allows others to extend a VM even long after its release.

Eliot has already responded to your example in detail, so I will not dwell upon it further, except to point out that we need a command line tool to generate a .c file directly from the plugin's mcz file. Then we can create a Make rule to compile a .o and get rid of .c file. e.g.

 FooPlugin.o : FooPlugin.mcz
        squeak -headless vmmaker.image gencc.st FooPlugin.mcz
        cc -c ... FooPlugin.c
        rm -f FooPlugin.c

Over my dead body.  The make must *NEVER* be done like this.  The problems are many:
- the VM so produced is undebuggable.  There is no source for FooPlugin.c; it has been deleted
- the build is slow; building a VMMaker image (the necessary prerequisite for st2c) takes a long time, mush longer than the st2c step
- unless FooPlugin.mcz is versioned then the VM so produced is unversioned and if  FooPlugin.mcz /is/ versioned then there's no reason why the plugin can't be produced using the normal workflow, generate C source, check it in to opensmalltalk/vm, build, and that way we have a versioned entity and break the dependency of builds requiring VMMaker


And I remember it mentioned a while ago, the bug is sometimes in the generation-code, so you want historical access to both mcz and generated-C to help debug that. 

+1


Look, I just spent several years persuading the Paro community that generating source and then building was a really bad idea and they should shelve it and now someone is trying to bring it back.  No, no, and thrice times, no.



I think Pharo has not completely abandoned generating C-sources, although its now a sideline activity...

It's a CI activity to test the current state of the VMMaker package.  This is useful.

On Wed, May 31, 2017 at 2:48 PM, Esteban Lorenzano <[hidden email]> wrote:

In any case, in the “pharo side” of things, we keep a parallel CI process who does this: 

1) there is a development branch that keep all sources we need: the osvm *and* all monticello packages (filetree format) acting as a mirror that merges changes and executes all building process (including generating sources) and testing (we run all pharo tests as a way to test the VM). This is done just for being sure the CI is green and no artifact is created. This is how I have early reported problem in VM generation before.
2) in parallel the build follows the “normal” process on opensmalltalk-vm and everything is build there. This generates a “latest” build which is published in pharo servers so people can test (but this binary could be wrong as is not something declared as “stable”


I do wonder from time to time is why the C-generation can't be done as part of an opensmalltalk-vm CI process?  The manual process is a bit opaque, and a CI process would be better than documentation.  One possibility of doing it via CI could be generating from both Squeak and Pharo and checking the output is identical (assuming that might be useful).

Yes this would be useful.  Also there's still at least one place in the CoInterpreter source that instability in the type inference code that flips the type of a variable between sqInt & usqInt and automation generation might help track that down.

IIRC Eliot didn't want the generated-C updated in the Cog branch for every VMMaker commit. 

Right.  No point doing so for changes to simulation only code, or to experimental code.  Or on every commit to a package that contains a translated primitive, a form of primitives we need to eliminate by rewriting as conventional ones.

But the real issue is in not committing changes to generated code that hasn't changed.  See scripts/revertUnchangedPlugins.

If there is, say, a Slang change that has limited effects (only a subset of plugins, or only the interpreter file, etc) we don't want to commit new versions of the essentially unchanged code because it obscures which commit leads to a bug.

Plus there's your observation about Slang bugs.  I want to eyeball the changes in at least one representative generated file to look for unintended consequences and assure myself that source generation is sane.

A way to conform to this would be the CI updating the generated C in a side branch, and only when deemed worthy it could be integrated into the Cog branch via a simple merge rather than a one-time manual generation run.  

Eliot,  If you're receptive to this idea, I'd be interested in working on this.  What concerns would you have with it?

- the process identifies files that only change in version stamp (if essentially unchanged) and /not/ commit this subset.  Right now that happens with a script on plugins (cuz they're relatively simple) and by my VMMaker image that tracks commits only producing source for classes whose timestamps have changed

- that the process maintains the separation between generation followed by check in from build from checked in source.

- that people who work on the Smalltalk side of the business not get lazy in eyeballing the generation process cuz Slang issues can be subtle and tedious to track down.  IME, human supervision remains useful here 

cheers -ben
 
 

where st2c should generate the .c file for FooPlugin from its Slang code in its package or from a .st file. There is no need for .c file to be human readable or be subject to version control. That will be very messy.

_yes_.  _there_. _is_.
 

Regards .. Subbu

--
_,,,^..^,,,_
best, Eliot


Reply | Threaded
Open this post in threaded view
|

Re: OSProcess fork issue with Debian built VM

Eliot Miranda-2
In reply to this post by K K Subbu
 
Hi Subbu,


> On Jun 3, 2017, at 9:47 AM, K K Subbu <[hidden email]> wrote:
>
>> On Saturday 03 June 2017 03:27 AM, Eliot Miranda wrote:
>>
>> Over my dead body.  The make must *NEVER* be done like this.  The
>> problems are many:
>
> Ouch! :-). Please don't take my example too literally. Smalltalk does not have a concept of a file-based compilation unit like C, so I had to use an extension to denote a collection of st code that goes into generating a .c compilation unit. I may well have used a .st or .zip for source code.
>
>> - the VM so produced is undebuggable.  There is no source for
>> FooPlugin.c; it has been deleted
>
> This is not the first time C-code gets generated on the fly. This technique has been used by Lex, Yacc etc. since early 70s. Bison generates a FSM interpreter in C from a Yacc spec. My example makefile segment deleted the file, but it can also cache it for debugging.

Caching it is not good enough.  If there are bugs in generation then there's no guarantee that regenerating it at a later time produces the same source.  Since the generated source contains time and version stamps it won't be the same anyway.

The inputs to any production vm /must/ be fully versioned at the point the source is consumed by the production compilation system.

The time taken to rebuild after a change should be as short as possible to keep debugging productivity high.  Mandating a translation step defeats this.

>
>> - the build is slow; building a VMMaker image (the necessary
>> prerequisite for st2c) takes a long time, mush longer than the st2c step
>
> The image is just a smalltalk app that translates slang to C. It does not have to be regenerated in every build anymore than gcc or cmake. In earlier posts, I had floated the idea of a bootstrap VM - a st machine with a built-in image to build and launch larger VMs. It can even be a separate project.

Sure.  This is also what my work image is.

>
> The .st code has to be translated to C at some time. If it is done within a single VM, then we won't be able to exploit multi-cores. With an st2c app, we can translate in parallel for a quicker build.

That could be nice.

>
>> - unless FooPlugin.mcz is versioned then the VM so produced is
>> unversioned and if  FooPlugin.mcz /is/ versioned then there's no reason
>> why the plugin can't be produced using the normal workflow, generate C
>> source, check it in to opensmalltalk/vm, build, and that way we have a
>> versioned entity and break the dependency of builds requiring VMMaker
>
> Anyway, my point is that only the smalltalk "compilation unit" should be under version control. If a .c file is generated from it, there is no need to subject it to version control. It can continue to remain in a cache. Like the way we preserve .i (preprocessed) or .s (assembly) files for debugging .c code but never check-in these i. or .s files.

And my point is that input source to compilers producing the VM (e.g. .c .h .cpp .m and .s) whether generated or manually prepared /must/ be versioned, and must change only when changes are made to them (e.g. regenerated plugin source is only checked in when either the Plugin Smalltalk has changed or Slang has changed in a way that affects the plugin).

> Okay, I had no idea I was stepping into a minefield of raw nerves here ;-). I will step back from these threads and do some prototyping and come up with specific code changes. The code seems to have grown too large for a lone hobbyist but it is still worth a try.

Supporting hobby projects, or rather, keeping the system open for people to add plugins, platforms, new forms of garbage collection, new vm configurations such as packaging as a DLL, or what ever else is /absolutely/ something we should support and encourage.  But that doesn't mean throwing away reproducibility and versioning.  It probably implies much better documentation.  But things are in a good state to improve:

- anyone can clone the opensmalltalk/vm repository

- there is a script in images to build a current VMMaker image so anyone can easily produce a VMMaker image in which to simulate, author and generate.  And the generation is fine to the right place in the opensmalltalk/v clone for it to be committed when ready

- because generation and check-in are separate steps one can get all the bits together (e.g. plugin generated under src/plugins, plugins.int or plugins.ext files updated on relevant platforms, special plugin majefiles containing support library references, etc) and make one commit instead of many noisy failing intermediate steps

So perhaps take a step back and ask what is the support you need, both in terms of documentation, code and infrastructure, that will help you experiment.

> Regards .. Subbu

best,
Eliot
Reply | Threaded
Open this post in threaded view
|

Re: OSProcess fork issue with Debian built VM

Holger Freyther
In reply to this post by Eliot Miranda-2
 

On 2. Jun 2017, at 22:40, Eliot Miranda <[hidden email]> wrote:


Hi Holger,

Hey Eliot!


   I have no disagreement with your time lines, but I do disagree with the black magic accusation, and you introduce your own piece of magic (see below).

in the end it is a word and a very subjective thing. I used to spend a lot of time working on OpenEmbedded to build Linux distributions. If one didn't understand what needs to be done to build a distribution and didn't know how the system worked, it seemed like black magic.


and then checks in the new sources.  David should have been able to do this.  The process is not documented.

Let's say I saw the code was not regenerated, what could I have done? Figure out the not documented process, make a PR and hope it is being reviewed within a month or two? It might not be black magic but quite a high barrier and maybe higher than necessary?



My preferred option

* I compile with a new compiler, look at the compiler warning and agree with it
* David Lewis agrees with my analysis and commits/pushes the fix to mc/ directory

You're glossing over some steps here.  Note that the fix is in the Smalltalk source, upstream of the C source.

I commit my Smalltalk code to the mc/ directory and the CI will build a VMMaker image and generate the C sources.


The missing step is that the plugin source is regenerated .  As a Smalltalk-side-of-the-business person David knows how to regenerate source. David, why didn't you regenerate? [honest Q, not an accusation]

Done by the CI in pharo-vm. And besides being done by the CI, the script that builds the VMMaker image and generates the C sources provide an executable documentation of the process. So anyone capable of committing Smalltalk code to the mc/ directory can generate new sources.


One *mustn't* manually fix the generated source because doing so means

I didn't argue about that. That would be the beginning of the end.



One *must* regenerate from an unmodified checked in Monticello package because doing so means
- the C source is version stamped as originating from that package and so one can locate the correct version of the Smalltalk from which it derived 

* The CI works by definition from an unmodified checked in Monticello package.
* tags, source uploads, debian source packages can provide as 1:1 mapping from version to archived source

skipping stuff here to keep the answer short.




* New VM will be build and regenerates sources on the fly

How does this work? Seems like magic to me.  Source has to be generated from a VMMaker image.  We need to regenerate source only for that which has changed, not for the entire generated source.  But right now that's problematic because of the time issues I mention above.

We are in the phase of computing where human time is (once again) more expensive that computing time. So the magic is that  some labor can be outsourced to a shell script. The nice thing with a shell script is one can observe each step (set -x) it is doing. For "stable" code generation review one can still check-in and compare the generated code.

I do my deployments on Debian with debian packages. Since a couple of weeks commits to the pharo-vm will generate a debian source package with generated C source and upload it. At the place below one can see the uploads and the difference in generated code from one version to another.


have a nice day

holger


Reply | Threaded
Open this post in threaded view
|

Re: OSProcess fork issue with Debian built VM

Luke Gorrie
In reply to this post by Max Leske
 
Hi Max,

On 18 May 2017 at 13:32, Max Leske <[hidden email]> wrote:
 
We managed to figure out that OSProcess works when we use gcc <= 4.8 on Debian. We are happy to use 4.8 for now, so we're good. It would of course be super cool if we could use the series 6 gcc as that will soon ship with Debian 9 (stretch) but it's probably not trivial to just move to a new compiler version (as seems evident from the fact that a minor version change can mess up compilation).

Thanks for tracking this problem down and sharing the details. I came upon exactly the same problem with the Nix package and was able to work-around that by also downgrading to gcc 4.8. I am sure it would have taken a long time to work this out without the benefit of your experience.

Relatedly: Is there a suitable automated test that we can use to check that our VMs are basically okay? I am pushing the switch to gcc 4.8 to resolve this problem but I don't know whether this is causing other regressions or what other obvious problems I have missed. I'd love a bit of test coverage for the builds.


Reply | Threaded
Open this post in threaded view
|

Re: OSProcess fork issue with Debian built VM

Eliot Miranda-2
 
Hi Luke,

On Sun, Jul 9, 2017 at 8:57 PM, Luke Gorrie <[hidden email]> wrote:
 
Hi Max,

On 18 May 2017 at 13:32, Max Leske <[hidden email]> wrote:
 
We managed to figure out that OSProcess works when we use gcc <= 4.8 on Debian. We are happy to use 4.8 for now, so we're good. It would of course be super cool if we could use the series 6 gcc as that will soon ship with Debian 9 (stretch) but it's probably not trivial to just move to a new compiler version (as seems evident from the fact that a minor version change can mess up compilation).

Thanks for tracking this problem down and sharing the details. I came upon exactly the same problem with the Nix package and was able to work-around that by also downgrading to gcc 4.8. I am sure it would have taken a long time to work this out without the benefit of your experience.

Relatedly: Is there a suitable automated test that we can use to check that our VMs are basically okay? I am pushing the switch to gcc 4.8 to resolve this problem but I don't know whether this is causing other regressions or what other obvious problems I have missed. I'd love a bit of test coverage for the builds.

Tests that stress the VM:
- Running the complete test suite
- running update to Squeak trunk tip from Squeak 5.0
- recompiling the entire system
- building a Moose image

Having these run automatically by a CI server would be very nice.

There are others (Cadence run an aggressive regression suite every night) but they are not publicly available.

_,,,^..^,,,_
best, Eliot
123