[OSProcess] #safeToForkSqueak when running headless

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[OSProcess] #safeToForkSqueak when running headless

Max Leske
Hi folks.

I’m running into an issue when forking images with OSProcess in headless images. In older versions of OSProcess #forkSqueak would try to fork in any case, independent of the presence of or the connection state to XDisplayControlPlugin.
In dtl.57 #forkSqueak was changed to explicitly check if XDisplayControlPlugin is present and can be controlled. This is obviously a good thing for when the image runs headful. In my case however I can no longer fork my headless image, even though I don’t need an X session.

Important: apparently #safeToForkSqueak answers true if the XDisplayControlPlugin is available, even in a headless image. But since we don’t need windows, we don’t install X libraries on our server and so #safeToForkSqueak answers false. So the whole thing boils down to a situation where Squeak / Pharo is run in an environment where the XDisplayPlugin can’t be loaded but where forking is still possible.

I don’t know how to do this in a cross platform way but for Pharo we could extend #safeToForkSqueak with a headless check like this:

safeToForkSqueak
        "The forkSqueak method is currently supported only for X11 displays with
        XDisplayControlPlugin available. If the plugin is not available, the child Squeak
        VM will continue interacting with the X display, which leads to errors in the
        parent Squeak, including VM crashes."

        ^self canControlXDisplay
                or: [ Smalltalk isHeadless ]


I’m going to dirty patch this for my images because I really need it but I think some version of this patch would be a good thing to integrate into OSProcess.

Cheers,
Max
Reply | Threaded
Open this post in threaded view
|

Re: [OSProcess] #safeToForkSqueak when running headless

David T. Lewis
On Wed, Aug 13, 2014 at 09:42:50AM +0200, Max Leske wrote:

> Hi folks.
>
> I?m running into an issue when forking images with OSProcess in headless images. In older versions of OSProcess #forkSqueak would try to fork in any case, independent of the presence of or the connection state to XDisplayControlPlugin.
> In dtl.57 #forkSqueak was changed to explicitly check if XDisplayControlPlugin is present and can be controlled. This is obviously a good thing for when the image runs headful. In my case however I can no longer fork my headless image, even though I don?t need an X session.
>
> Important: apparently #safeToForkSqueak answers true if the XDisplayControlPlugin is available, even in a headless image. But since we don?t need windows, we don?t install X libraries on our server and so #safeToForkSqueak answers false. So the whole thing boils down to a situation where Squeak / Pharo is run in an environment where the XDisplayPlugin can?t be loaded but where forking is still possible.
>
> I don?t know how to do this in a cross platform way but for Pharo we could extend #safeToForkSqueak with a headless check like this:
>
> safeToForkSqueak
> "The forkSqueak method is currently supported only for X11 displays with
> XDisplayControlPlugin available. If the plugin is not available, the child Squeak
> VM will continue interacting with the X display, which leads to errors in the
> parent Squeak, including VM crashes."
>
> ^self canControlXDisplay
> or: [ Smalltalk isHeadless ]
>
>
> I?m going to dirty patch this for my images because I really need it but I think some version of this patch would be a good thing to integrate into OSProcess.
>

Hi Max,

Thanks for this fix. I'll add it to the OSP repository as soon as I have a
chance (but maybe not until Sunday).

As a side note, I have also found that an active RFB (VNC) connection presents
a similar issue. If I have a headless image (such as a SqueakSource server), and
if I have a VNC session connected to that image, then a forkSqueak does bad things
because it results in two instances of the VM/image connected to the same socket.
This is the same kind of issue as with the X11 server socket connection, so I should
probably find a way to handle this in the safeToForkSqueak checks.

Dave


Reply | Threaded
Open this post in threaded view
|

Re: [OSProcess] #safeToForkSqueak when running headless

Max Leske
In reply to this post by Max Leske

On 13.08.2014, at 14:00, [hidden email] wrote:

Date: Wed, 13 Aug 2014 07:45:08 -0400
From: "David T. Lewis" <[hidden email]>
Subject: Re: [squeak-dev] [OSProcess] #safeToForkSqueak when running
headless
To: The general-purpose Squeak developers list
<[hidden email]>
Message-ID: <[hidden email]>
Content-Type: text/plain; charset=us-ascii

On Wed, Aug 13, 2014 at 09:42:50AM +0200, Max Leske wrote:
Hi folks.

I?m running into an issue when forking images with OSProcess in headless images. In older versions of OSProcess #forkSqueak would try to fork in any case, independent of the presence of or the connection state to XDisplayControlPlugin.
In dtl.57 #forkSqueak was changed to explicitly check if XDisplayControlPlugin is present and can be controlled. This is obviously a good thing for when the image runs headful. In my case however I can no longer fork my headless image, even though I don?t need an X session.

Important: apparently #safeToForkSqueak answers true if the XDisplayControlPlugin is available, even in a headless image. But since we don?t need windows, we don?t install X libraries on our server and so #safeToForkSqueak answers false. So the whole thing boils down to a situation where Squeak / Pharo is run in an environment where the XDisplayPlugin can?t be loaded but where forking is still possible.

I don?t know how to do this in a cross platform way but for Pharo we could extend #safeToForkSqueak with a headless check like this:

safeToForkSqueak
"The forkSqueak method is currently supported only for X11 displays with
XDisplayControlPlugin available. If the plugin is not available, the child Squeak
VM will continue interacting with the X display, which leads to errors in the
parent Squeak, including VM crashes."

^self canControlXDisplay
or: [ Smalltalk isHeadless ]


I?m going to dirty patch this for my images because I really need it but I think some version of this patch would be a good thing to integrate into OSProcess.


Hi Max,

Thanks for this fix. I'll add it to the OSP repository as soon as I have a
chance (but maybe not until Sunday).

As a side note, I have also found that an active RFB (VNC) connection presents
a similar issue. If I have a headless image (such as a SqueakSource server), and
if I have a VNC session connected to that image, then a forkSqueak does bad things
because it results in two instances of the VM/image connected to the same socket.
This is the same kind of issue as with the X11 server socket connection, so I should
probably find a way to handle this in the safeToForkSqueak checks.

We explicitly check for that and forbid forking if there’s an RFB session.
We also do a couple of other things before forking to make sure we don’t run into problems. One of the most important things for us is to stop WAKom before the fork and restart it afterwards. Otherwise the child would sometimes steal the socket or would crash etc.
Forking is realy a bit of a can of worms and I’m really happy that it still works so reliably. So thanks for that :)

Max



Dave




Reply | Threaded
Open this post in threaded view
|

Re: [OSProcess] #safeToForkSqueak when running headless

Eliot Miranda-2



On Wed, Aug 13, 2014 at 5:23 AM, Max Leske <[hidden email]> wrote:

On 13.08.2014, at 14:00, [hidden email] wrote:

Date: Wed, 13 Aug 2014 07:45:08 -0400
From: "David T. Lewis" <[hidden email]>
Subject: Re: [squeak-dev] [OSProcess] #safeToForkSqueak when running
headless
To: The general-purpose Squeak developers list
<[hidden email]>
Message-ID: <[hidden email]>
Content-Type: text/plain; charset=us-ascii

On Wed, Aug 13, 2014 at 09:42:50AM +0200, Max Leske wrote:
Hi folks.

I?m running into an issue when forking images with OSProcess in headless images. In older versions of OSProcess #forkSqueak would try to fork in any case, independent of the presence of or the connection state to XDisplayControlPlugin.
In dtl.57 #forkSqueak was changed to explicitly check if XDisplayControlPlugin is present and can be controlled. This is obviously a good thing for when the image runs headful. In my case however I can no longer fork my headless image, even though I don?t need an X session.

Important: apparently #safeToForkSqueak answers true if the XDisplayControlPlugin is available, even in a headless image. But since we don?t need windows, we don?t install X libraries on our server and so #safeToForkSqueak answers false. So the whole thing boils down to a situation where Squeak / Pharo is run in an environment where the XDisplayPlugin can?t be loaded but where forking is still possible.

I don?t know how to do this in a cross platform way but for Pharo we could extend #safeToForkSqueak with a headless check like this:

safeToForkSqueak
"The forkSqueak method is currently supported only for X11 displays with
XDisplayControlPlugin available. If the plugin is not available, the child Squeak
VM will continue interacting with the X display, which leads to errors in the
parent Squeak, including VM crashes."

^self canControlXDisplay
or: [ Smalltalk isHeadless ]


I?m going to dirty patch this for my images because I really need it but I think some version of this patch would be a good thing to integrate into OSProcess.


Hi Max,

Thanks for this fix. I'll add it to the OSP repository as soon as I have a
chance (but maybe not until Sunday).

As a side note, I have also found that an active RFB (VNC) connection presents
a similar issue. If I have a headless image (such as a SqueakSource server), and
if I have a VNC session connected to that image, then a forkSqueak does bad things
because it results in two instances of the VM/image connected to the same socket.
This is the same kind of issue as with the X11 server socket connection, so I should
probably find a way to handle this in the safeToForkSqueak checks.

We explicitly check for that and forbid forking if there’s an RFB session.
We also do a couple of other things before forking to make sure we don’t run into problems. One of the most important things for us is to stop WAKom before the fork and restart it afterwards. Otherwise the child would sometimes steal the socket or would crash etc.
Forking is realy a bit of a can of worms and I’m really happy that it still works so reliably. So thanks for that :)

Should it be doing something to the effect of Smalltalk processShutDownList:/processStartUpList: around the fork?  I can see this is problematic cuz one won't want to shut down the kernel (Delay etc) or OSProcess itself.  But an arbitrary set of things may need to shutDown and restart and the start-up and shut-down lists are the pluggable mechanism.
--
best,
Eliot


Reply | Threaded
Open this post in threaded view
|

Re: [OSProcess] #safeToForkSqueak when running headless

David T. Lewis
On Wed, Aug 13, 2014 at 06:28:10AM -0700, Eliot Miranda wrote:

> On Wed, Aug 13, 2014 at 5:23 AM, Max Leske <[hidden email]> wrote:
>
> >
> > On 13.08.2014, at 14:00, [hidden email]
> > wrote:
> >
> > Date: Wed, 13 Aug 2014 07:45:08 -0400
> > From: "David T. Lewis" <[hidden email]>
> > Subject: Re: [squeak-dev] [OSProcess] #safeToForkSqueak when running
> > headless
> > To: The general-purpose Squeak developers list
> > <[hidden email]>
> > Message-ID: <[hidden email]>
> > Content-Type: text/plain; charset=us-ascii
> >
> > On Wed, Aug 13, 2014 at 09:42:50AM +0200, Max Leske wrote:
> >
> > Hi folks.
> >
> > I?m running into an issue when forking images with OSProcess in headless
> > images. In older versions of OSProcess #forkSqueak would try to fork in any
> > case, independent of the presence of or the connection state to
> > XDisplayControlPlugin.
> > In dtl.57 #forkSqueak was changed to explicitly check if
> > XDisplayControlPlugin is present and can be controlled. This is obviously a
> > good thing for when the image runs headful. In my case however I can no
> > longer fork my headless image, even though I don?t need an X session.
> >
> > Important: apparently #safeToForkSqueak answers true if the
> > XDisplayControlPlugin is available, even in a headless image. But since we
> > don?t need windows, we don?t install X libraries on our server and so
> > #safeToForkSqueak answers false. So the whole thing boils down to a
> > situation where Squeak / Pharo is run in an environment where the
> > XDisplayPlugin can?t be loaded but where forking is still possible.
> >
> > I don?t know how to do this in a cross platform way but for Pharo we could
> > extend #safeToForkSqueak with a headless check like this:
> >
> > safeToForkSqueak
> > "The forkSqueak method is currently supported only for X11 displays with
> > XDisplayControlPlugin available. If the plugin is not available, the child
> > Squeak
> > VM will continue interacting with the X display, which leads to errors in
> > the
> > parent Squeak, including VM crashes."
> >
> > ^self canControlXDisplay
> >  or: [ Smalltalk isHeadless ]
> >
> >
> > I?m going to dirty patch this for my images because I really need it but I
> > think some version of this patch would be a good thing to integrate into
> > OSProcess.
> >
> >
> > Hi Max,
> >
> > Thanks for this fix. I'll add it to the OSP repository as soon as I have a
> > chance (but maybe not until Sunday).
> >
> >
> > As a side note, I have also found that an active RFB (VNC) connection
> > presents
> > a similar issue. If I have a headless image (such as a SqueakSource
> > server), and
> > if I have a VNC session connected to that image, then a forkSqueak does
> > bad things
> > because it results in two instances of the VM/image connected to the same
> > socket.
> > This is the same kind of issue as with the X11 server socket connection,
> > so I should
> > probably find a way to handle this in the safeToForkSqueak checks.
> >
> >
> > We explicitly check for that and forbid forking if there???s an RFB session.
> > We also do a couple of other things before forking to make sure we don???t
> > run into problems. One of the most important things for us is to stop WAKom
> > before the fork and restart it afterwards. Otherwise the child would
> > sometimes steal the socket or would crash etc.
> > Forking is realy a bit of a can of worms and I???m really happy that it
> > still works so reliably. So thanks for that :)
> >
>
> Should it be doing something to the effect of Smalltalk
> processShutDownList:/processStartUpList: around the fork?  I can see this
> is problematic cuz one won't want to shut down the kernel (Delay etc) or
> OSProcess itself.  But an arbitrary set of things may need to shutDown and
> restart and the start-up and shut-down lists are the pluggable mechanism.

That's a very good idea.

Dave


Reply | Threaded
Open this post in threaded view
|

Re: [OSProcess] #safeToForkSqueak when running headless

David T. Lewis
In reply to this post by Max Leske
On Wed, Aug 13, 2014 at 09:42:50AM +0200, Max Leske wrote:

> Hi folks.
>
> I?m running into an issue when forking images with OSProcess in headless images. In older versions of OSProcess #forkSqueak would try to fork in any case, independent of the presence of or the connection state to XDisplayControlPlugin.
> In dtl.57 #forkSqueak was changed to explicitly check if XDisplayControlPlugin is present and can be controlled. This is obviously a good thing for when the image runs headful. In my case however I can no longer fork my headless image, even though I don?t need an X session.
>
> Important: apparently #safeToForkSqueak answers true if the XDisplayControlPlugin is available, even in a headless image. But since we don?t need windows, we don?t install X libraries on our server and so #safeToForkSqueak answers false. So the whole thing boils down to a situation where Squeak / Pharo is run in an environment where the XDisplayPlugin can?t be loaded but where forking is still possible.
>
> I don?t know how to do this in a cross platform way but for Pharo we could extend #safeToForkSqueak with a headless check like this:
>
> safeToForkSqueak
> "The forkSqueak method is currently supported only for X11 displays with
> XDisplayControlPlugin available. If the plugin is not available, the child Squeak
> VM will continue interacting with the X display, which leads to errors in the
> parent Squeak, including VM crashes."
>
> ^self canControlXDisplay
> or: [ Smalltalk isHeadless ]
>
>
> I?m going to dirty patch this for my images because I really need it but I think some version of this patch would be a good thing to integrate into OSProcess.
>

Thanks Max,

I added your patch to the OSP repository. I added a check so that it will work
on classic Squeak 3.8 as well as the latest Squeak trunk and Pharo.

Dave