competition for SIGTERM

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

competition for SIGTERM

Chris Muller-4
OSProcess provides the ability to set up and wait on any of several
semaphores which will be signaled if the VM process running the
current image gets a posix signal.

A few years ago I added a handler for SqueakSource's startUp: to set
up a response to TERM, like this:

restartSigTermHandler
SigTermHandler ifNotNil: [ SigTermHandler terminate ].
SigTermHandler :=
          [ OSProcess accessor restoreSigTerm.
          OSProcess accessor forwardSigTerm wait.
          SSRepository current log: 'SIGTERM received, shutting down
gracefully.'.
          self shutDown.
          Smalltalk snapshot: false andQuit: true ] newProcess
                name: 'Waiting for SIGTERM for graceful shutdown' ;
                resume

But now I want Magma servers to be able to respond similarly whether
SqueakSource is loaded in the same image or not.  But if I were to
introduce the above into Magma, then if SqueakSource was also loaded,
there would either be two processes waiting to shutdown or one of them
would run and the other wouldn't.

So this led me to ask, what is the best way to handle this in a
general fashion?  This is at the process level, so the first answer I
came up with was that it would make sense for the out-of-the-box
behavior to respond to TERM with a "Smalltalk snapshot: false andQuit:
true" so that each framework can do their own proper thing in
#shutDown:...   Is this a good idea or what is?

Reply | Threaded
Open this post in threaded view
|

Re: competition for SIGTERM

David T. Lewis
On Sun, Apr 26, 2015 at 06:57:51PM -0500, Chris Muller wrote:

> OSProcess provides the ability to set up and wait on any of several
> semaphores which will be signaled if the VM process running the
> current image gets a posix signal.
>
> A few years ago I added a handler for SqueakSource's startUp: to set
> up a response to TERM, like this:
>
> restartSigTermHandler
> SigTermHandler ifNotNil: [ SigTermHandler terminate ].
> SigTermHandler :=
>           [ OSProcess accessor restoreSigTerm.
>           OSProcess accessor forwardSigTerm wait.
>           SSRepository current log: 'SIGTERM received, shutting down
> gracefully.'.
>           self shutDown.
>           Smalltalk snapshot: false andQuit: true ] newProcess
>                 name: 'Waiting for SIGTERM for graceful shutdown' ;
>                 resume
>
> But now I want Magma servers to be able to respond similarly whether
> SqueakSource is loaded in the same image or not.  But if I were to
> introduce the above into Magma, then if SqueakSource was also loaded,
> there would either be two processes waiting to shutdown or one of them
> would run and the other wouldn't.
>
> So this led me to ask, what is the best way to handle this in a
> general fashion?  This is at the process level, so the first answer I
> came up with was that it would make sense for the out-of-the-box
> behavior to respond to TERM with a "Smalltalk snapshot: false andQuit:
> true" so that each framework can do their own proper thing in
> #shutDown:...   Is this a good idea or what is?

I guess that if you wanted to generalize this, you might follow the
pattern of the system startUp: and shutDown: processing, and maybe have
a class that was responsible for holding on to the processes that wait
for signals. A class could register itself as being interested in SIGTERM
if it implemented a method #sigTerm that did something to respond to
that signal.

I would be cautious about saving the image in response to a SIGTERM,
except in the special case of a server image that wanted to do so. It
is possible to open several VMs on the same image, either on purpose
or by accident, and you would not want all those VMs to be trying to
save the same image when you do "killall squeak". And I think that
SIGTERM is sent to all processes when the system is being rebooted,
so you would not want to be half way through saving your image when
the system goes down.

Dave


Reply | Threaded
Open this post in threaded view
|

Re: competition for SIGTERM

Chris Muller-3
On Mon, Apr 27, 2015 at 6:53 PM, David T. Lewis <[hidden email]> wrote:

> On Sun, Apr 26, 2015 at 06:57:51PM -0500, Chris Muller wrote:
>> OSProcess provides the ability to set up and wait on any of several
>> semaphores which will be signaled if the VM process running the
>> current image gets a posix signal.
>>
>> A few years ago I added a handler for SqueakSource's startUp: to set
>> up a response to TERM, like this:
>>
>> restartSigTermHandler
>> SigTermHandler ifNotNil: [ SigTermHandler terminate ].
>> SigTermHandler :=
>>           [ OSProcess accessor restoreSigTerm.
>>           OSProcess accessor forwardSigTerm wait.
>>           SSRepository current log: 'SIGTERM received, shutting down
>> gracefully.'.
>>           self shutDown.
>>           Smalltalk snapshot: false andQuit: true ] newProcess
>>                 name: 'Waiting for SIGTERM for graceful shutdown' ;
>>                 resume
>>
>> But now I want Magma servers to be able to respond similarly whether
>> SqueakSource is loaded in the same image or not.  But if I were to
>> introduce the above into Magma, then if SqueakSource was also loaded,
>> there would either be two processes waiting to shutdown or one of them
>> would run and the other wouldn't.
>>
>> So this led me to ask, what is the best way to handle this in a
>> general fashion?  This is at the process level, so the first answer I
>> came up with was that it would make sense for the out-of-the-box
>> behavior to respond to TERM with a "Smalltalk snapshot: false andQuit:
>> true" so that each framework can do their own proper thing in
>> #shutDown:...   Is this a good idea or what is?
>
> I guess that if you wanted to generalize this, you might follow the
> pattern of the system startUp: and shutDown: processing, and maybe have
> a class that was responsible for holding on to the processes that wait
> for signals. A class could register itself as being interested in SIGTERM
> if it implemented a method #sigTerm that did something to respond to
> that signal.

Yes, that's exactly what I've done.  I put the state in my own class
but it has the same problem -- competition.

That's why I think it makes sense for the VM to call a method or
provide some hook for the image to respond to SIGTERM with a Smalltalk
snapshot: false andQuit: true.  I think it should do this even without
OSProcess loaded.

That way, all frameworks within the image only have to implement their
normal startUp: / shutDown: code, and they get the graceful SIGTERM
shutdown for "free".

> I would be cautious about saving the image in response to a SIGTERM,
> except in the special case of a server image that wanted to do so. It
> is possible to open several VMs on the same image, either on purpose
> or by accident, and you would not want all those VMs to be trying to
> save the same image when you do "killall squeak". And I think that
> SIGTERM is sent to all processes when the system is being rebooted,
> so you would not want to be half way through saving your image when
> the system goes down.

Agreed.  Right now I'm only interested in quitting without saving so
we can support graceful shutdown of Squeak-based servers via SIGTERM.

Reply | Threaded
Open this post in threaded view
|

Re: competition for SIGTERM

David T. Lewis
On Mon, Apr 27, 2015 at 08:50:02PM -0500, Chris Muller wrote:

> On Mon, Apr 27, 2015 at 6:53 PM, David T. Lewis <[hidden email]> wrote:
> > On Sun, Apr 26, 2015 at 06:57:51PM -0500, Chris Muller wrote:
> >> OSProcess provides the ability to set up and wait on any of several
> >> semaphores which will be signaled if the VM process running the
> >> current image gets a posix signal.
> >>
> >> A few years ago I added a handler for SqueakSource's startUp: to set
> >> up a response to TERM, like this:
> >>
> >> restartSigTermHandler
> >> SigTermHandler ifNotNil: [ SigTermHandler terminate ].
> >> SigTermHandler :=
> >>           [ OSProcess accessor restoreSigTerm.
> >>           OSProcess accessor forwardSigTerm wait.
> >>           SSRepository current log: 'SIGTERM received, shutting down
> >> gracefully.'.
> >>           self shutDown.
> >>           Smalltalk snapshot: false andQuit: true ] newProcess
> >>                 name: 'Waiting for SIGTERM for graceful shutdown' ;
> >>                 resume
> >>
> >> But now I want Magma servers to be able to respond similarly whether
> >> SqueakSource is loaded in the same image or not.  But if I were to
> >> introduce the above into Magma, then if SqueakSource was also loaded,
> >> there would either be two processes waiting to shutdown or one of them
> >> would run and the other wouldn't.
> >>
> >> So this led me to ask, what is the best way to handle this in a
> >> general fashion?  This is at the process level, so the first answer I
> >> came up with was that it would make sense for the out-of-the-box
> >> behavior to respond to TERM with a "Smalltalk snapshot: false andQuit:
> >> true" so that each framework can do their own proper thing in
> >> #shutDown:...   Is this a good idea or what is?
> >
> > I guess that if you wanted to generalize this, you might follow the
> > pattern of the system startUp: and shutDown: processing, and maybe have
> > a class that was responsible for holding on to the processes that wait
> > for signals. A class could register itself as being interested in SIGTERM
> > if it implemented a method #sigTerm that did something to respond to
> > that signal.
>
> Yes, that's exactly what I've done.  I put the state in my own class
> but it has the same problem -- competition.
>
> That's why I think it makes sense for the VM to call a method or
> provide some hook for the image to respond to SIGTERM with a Smalltalk
> snapshot: false andQuit: true.  I think it should do this even without
> OSProcess loaded.

But why should the VM need to figure this out? That was the whole point
of the signal forwarding mechanism in OSProcess - forward the signal
directly to the image, and let the image figure out what to do with it.

   UnixOSProcessAccessor forwardSignal: signalNumber
        "Set a signal handler in the VM which will signal a Smalltalk semaphore
        at semaphoreIndex whenever an external signal sigNum is received.
        Answer a new Semaphore, or nil if unable to set the handler (possibly
        because it has already been set). A Smalltalk process can wait on the
        Semaphore, and take action when a signal is detected. See man(7) signal
        for signal number definitions on your unix system."

The VM support (in the OSPP plugin) consists of signaling a registered
Smalltalk semaphore when an OS signal (such as SIGTERM) is detected. That
is all that should be required. If there are issues with how, what, when,
and by whom that event should be handled, then you are much better off
figuring that out in the image rather than in the VM.

The basic signal forwarding mechanism could be incorporated into the VM,
but I don't see any problem with keeping it in a plugin. The mechanism
is specific to unix systems, so to me it seems like a good thing to keep
it out of the VM proper.

Dave


Reply | Threaded
Open this post in threaded view
|

Re: competition for SIGTERM

Chris Muller-4
Squeak 4.6 will currently respond to USR1 by printing all stacks.
That's a nice OOTB feature.  One more would be if it could somehow
respond to SIGTERM with a graceful shutdown instead of an ungraceful
one.

I'm simply interested in Squeak 4.6 having that feature OOTB.  Do
other Linux programs normally respond to SIGTERM by running their exit
code?  Sometimes I notice some of them print a nice exit message when
I press control+c.

If we brought OSProcess into the image we could do it..   :)

On Mon, Apr 27, 2015 at 9:17 PM, David T. Lewis <[hidden email]> wrote:

> On Mon, Apr 27, 2015 at 08:50:02PM -0500, Chris Muller wrote:
>> On Mon, Apr 27, 2015 at 6:53 PM, David T. Lewis <[hidden email]> wrote:
>> > On Sun, Apr 26, 2015 at 06:57:51PM -0500, Chris Muller wrote:
>> >> OSProcess provides the ability to set up and wait on any of several
>> >> semaphores which will be signaled if the VM process running the
>> >> current image gets a posix signal.
>> >>
>> >> A few years ago I added a handler for SqueakSource's startUp: to set
>> >> up a response to TERM, like this:
>> >>
>> >> restartSigTermHandler
>> >> SigTermHandler ifNotNil: [ SigTermHandler terminate ].
>> >> SigTermHandler :=
>> >>           [ OSProcess accessor restoreSigTerm.
>> >>           OSProcess accessor forwardSigTerm wait.
>> >>           SSRepository current log: 'SIGTERM received, shutting down
>> >> gracefully.'.
>> >>           self shutDown.
>> >>           Smalltalk snapshot: false andQuit: true ] newProcess
>> >>                 name: 'Waiting for SIGTERM for graceful shutdown' ;
>> >>                 resume
>> >>
>> >> But now I want Magma servers to be able to respond similarly whether
>> >> SqueakSource is loaded in the same image or not.  But if I were to
>> >> introduce the above into Magma, then if SqueakSource was also loaded,
>> >> there would either be two processes waiting to shutdown or one of them
>> >> would run and the other wouldn't.
>> >>
>> >> So this led me to ask, what is the best way to handle this in a
>> >> general fashion?  This is at the process level, so the first answer I
>> >> came up with was that it would make sense for the out-of-the-box
>> >> behavior to respond to TERM with a "Smalltalk snapshot: false andQuit:
>> >> true" so that each framework can do their own proper thing in
>> >> #shutDown:...   Is this a good idea or what is?
>> >
>> > I guess that if you wanted to generalize this, you might follow the
>> > pattern of the system startUp: and shutDown: processing, and maybe have
>> > a class that was responsible for holding on to the processes that wait
>> > for signals. A class could register itself as being interested in SIGTERM
>> > if it implemented a method #sigTerm that did something to respond to
>> > that signal.
>>
>> Yes, that's exactly what I've done.  I put the state in my own class
>> but it has the same problem -- competition.
>>
>> That's why I think it makes sense for the VM to call a method or
>> provide some hook for the image to respond to SIGTERM with a Smalltalk
>> snapshot: false andQuit: true.  I think it should do this even without
>> OSProcess loaded.
>
> But why should the VM need to figure this out? That was the whole point
> of the signal forwarding mechanism in OSProcess - forward the signal
> directly to the image, and let the image figure out what to do with it.
>
>    UnixOSProcessAccessor forwardSignal: signalNumber
>         "Set a signal handler in the VM which will signal a Smalltalk semaphore
>         at semaphoreIndex whenever an external signal sigNum is received.
>         Answer a new Semaphore, or nil if unable to set the handler (possibly
>         because it has already been set). A Smalltalk process can wait on the
>         Semaphore, and take action when a signal is detected. See man(7) signal
>         for signal number definitions on your unix system."
>
> The VM support (in the OSPP plugin) consists of signaling a registered
> Smalltalk semaphore when an OS signal (such as SIGTERM) is detected. That
> is all that should be required. If there are issues with how, what, when,
> and by whom that event should be handled, then you are much better off
> figuring that out in the image rather than in the VM.
>
> The basic signal forwarding mechanism could be incorporated into the VM,
> but I don't see any problem with keeping it in a plugin. The mechanism
> is specific to unix systems, so to me it seems like a good thing to keep
> it out of the VM proper.
>
> Dave
>

Reply | Threaded
Open this post in threaded view
|

Re: competition for SIGTERM

David T. Lewis
Hi Chris,

A few comments below, just personal opinions.

On Mon, Apr 27, 2015 at 09:59:00PM -0500, Chris Muller wrote:
> Squeak 4.6 will currently respond to USR1 by printing all stacks.
> That's a nice OOTB feature.  One more would be if it could somehow
> respond to SIGTERM with a graceful shutdown instead of an ungraceful
> one.

This is a good feature, and the stack print from Cog is very useful. There
is a similar stack print in the interpreter VM, though not as good as the
Cog one, and it requires OSProcess to activate it.

        "OSProcess accessor setPrintAllStacksOnSigUsr1"
        "OSProcess accessor clearPrintAllStacksOnSigUsr1"

In terms of out of the box behavior, neither approach is very good.

Requiring OSProcess to activate the handler in the VM is bad, because
nobody (including me) will ever remember to set that up, so it will never
be there when you really need it (stuck image, want to look at the stack).

Having it hard wired to SIGUSR1 in the VM is not good either, because
SIGUSR1 might be needed for some other purpose (e.g. a third party library
linked to a VM plugin).

A better approach would be to have it active by default, but control it
with a command line option to the VM (for all Unix VMs). That way the
feature can be available for everyone out of the box, but it can be
disabled or reconfigured to use a different signal number in cases where
SIGUSR1 might be needed for some other purpose.

>
> I'm simply interested in Squeak 4.6 having that feature OOTB.  Do
> other Linux programs normally respond to SIGTERM by running their exit
> code?  Sometimes I notice some of them print a nice exit message when
> I press control+c.

It's a good idea, but proceed with caution. If someone sent a SIGTERM to
you, it means they want you to terminate real soon now. If you ignore
that advice, you do so at your own peril.

>
> If we brought OSProcess into the image we could do it..   :)

I appreciate the sentiment, but I really think that OSProcess is best
maintained as a separate package :-) Bits and pieces might be adopted
where useful, but I really would not want to see Squeak turn into a YAUK
(yet another unix klone). Besides, we have SqueakMap so it is easy to
load packages like this.

Dave

>
> On Mon, Apr 27, 2015 at 9:17 PM, David T. Lewis <[hidden email]> wrote:
> > On Mon, Apr 27, 2015 at 08:50:02PM -0500, Chris Muller wrote:
> >> On Mon, Apr 27, 2015 at 6:53 PM, David T. Lewis <[hidden email]> wrote:
> >> > On Sun, Apr 26, 2015 at 06:57:51PM -0500, Chris Muller wrote:
> >> >> OSProcess provides the ability to set up and wait on any of several
> >> >> semaphores which will be signaled if the VM process running the
> >> >> current image gets a posix signal.
> >> >>
> >> >> A few years ago I added a handler for SqueakSource's startUp: to set
> >> >> up a response to TERM, like this:
> >> >>
> >> >> restartSigTermHandler
> >> >> SigTermHandler ifNotNil: [ SigTermHandler terminate ].
> >> >> SigTermHandler :=
> >> >>           [ OSProcess accessor restoreSigTerm.
> >> >>           OSProcess accessor forwardSigTerm wait.
> >> >>           SSRepository current log: 'SIGTERM received, shutting down
> >> >> gracefully.'.
> >> >>           self shutDown.
> >> >>           Smalltalk snapshot: false andQuit: true ] newProcess
> >> >>                 name: 'Waiting for SIGTERM for graceful shutdown' ;
> >> >>                 resume
> >> >>
> >> >> But now I want Magma servers to be able to respond similarly whether
> >> >> SqueakSource is loaded in the same image or not.  But if I were to
> >> >> introduce the above into Magma, then if SqueakSource was also loaded,
> >> >> there would either be two processes waiting to shutdown or one of them
> >> >> would run and the other wouldn't.
> >> >>
> >> >> So this led me to ask, what is the best way to handle this in a
> >> >> general fashion?  This is at the process level, so the first answer I
> >> >> came up with was that it would make sense for the out-of-the-box
> >> >> behavior to respond to TERM with a "Smalltalk snapshot: false andQuit:
> >> >> true" so that each framework can do their own proper thing in
> >> >> #shutDown:...   Is this a good idea or what is?
> >> >
> >> > I guess that if you wanted to generalize this, you might follow the
> >> > pattern of the system startUp: and shutDown: processing, and maybe have
> >> > a class that was responsible for holding on to the processes that wait
> >> > for signals. A class could register itself as being interested in SIGTERM
> >> > if it implemented a method #sigTerm that did something to respond to
> >> > that signal.
> >>
> >> Yes, that's exactly what I've done.  I put the state in my own class
> >> but it has the same problem -- competition.
> >>
> >> That's why I think it makes sense for the VM to call a method or
> >> provide some hook for the image to respond to SIGTERM with a Smalltalk
> >> snapshot: false andQuit: true.  I think it should do this even without
> >> OSProcess loaded.
> >
> > But why should the VM need to figure this out? That was the whole point
> > of the signal forwarding mechanism in OSProcess - forward the signal
> > directly to the image, and let the image figure out what to do with it.
> >
> >    UnixOSProcessAccessor forwardSignal: signalNumber
> >         "Set a signal handler in the VM which will signal a Smalltalk semaphore
> >         at semaphoreIndex whenever an external signal sigNum is received.
> >         Answer a new Semaphore, or nil if unable to set the handler (possibly
> >         because it has already been set). A Smalltalk process can wait on the
> >         Semaphore, and take action when a signal is detected. See man(7) signal
> >         for signal number definitions on your unix system."
> >
> > The VM support (in the OSPP plugin) consists of signaling a registered
> > Smalltalk semaphore when an OS signal (such as SIGTERM) is detected. That
> > is all that should be required. If there are issues with how, what, when,
> > and by whom that event should be handled, then you are much better off
> > figuring that out in the image rather than in the VM.
> >
> > The basic signal forwarding mechanism could be incorporated into the VM,
> > but I don't see any problem with keeping it in a plugin. The mechanism
> > is specific to unix systems, so to me it seems like a good thing to keep
> > it out of the VM proper.
> >
> > Dave
> >