Starting processes from Squeak makes Raspbian unwilling to start another process

Hi,

I managed to make Raspbian unwilling to open any new process. Neither in
the X session, nor from Squeak nor from a SSH session.
It seems I create Zombie processes (what is that, in the SSH session I
ran top and it reports thousands of zombies).

What I do: I poll a GPIO pin every 100ms. When it is high I query I2C via:
(PipeableOSProcess command: 'i2cget -y ', busNo, ' ', address) output

and when I find a certain bit set, I switch on some LEDs via:
current := ExternalUnixOSProcess command: 'i2cset -y ', busNo, ' ',
address, ' ', aString.
OSProcess thisOSProcess unregisterChildProcess: current. "This doesn't
unregister the process?????"
current := nil.

I found 2700++ instances of ExternalOSProcess, emptied the Dictionary
via UnixProcess>>unregisterChildProcess and it stopped when it
accumulated to another 900+ instances.

This ran for half a day until It can't start another process. First I
thought this was the Raspbian update for the PasPi B2 but now I
reproduced it on a Pi A+ last updated mid January.

Having read Wikipedia on Zombie Process it seems, that I should try to
kill each process I started?
What do I do to the PipeableOSProcess? closePipes? release?

Thanks,

Herbert

Hi,

now that I noticed the zombies I can experiment much faster so here some
further information:

Stopping Squeak removes all Zombies. Each sensor event creates 3
Zombies, two for the I2Csends, one for the I2Cread.
Sending the PipeableOSProcess instances #closePipes then #release
doesn't help.
Sending the ExternalUnixProcess instances #terminate doesn't help either.
Sending an additional PipeableOSProcess>>unixCommandPipeLine doesn't add
a Zombie but returns ''.
PipeableOSProcess>>tkExample works fine and doesn't add Zombies.

So is it a problem of I2Ctools? Should I try other Linux commands?

Thanks,

Herbert


Am 07.02.2015 um 18:11 schrieb Herbert König:

Hi Herbert,

On Sat, Feb 07, 2015 at 07:46:43PM +0100, Herbert K??nig wrote:
This looks like some kind of problem in OSProcess. I am not sure what
is wrong, so let me try to explain how it works.

In Unix, a zombie process is a process that has exited, but its parent
process has not yet acknowledged that it has exited. When you run a
command with OSProcess, it starts a new process that is a child of the
VM process. When that external process exits, it is the responsibility
of OSProcess (running in the VM process) to notice this, and to acknowledge
the exit by calling the Unix waitpid() system call.

When things are working right, you should be able to open a process browser
in Squeak, and see an active process labeled "the child OSProcess watcher".
That process is normally blocking on a Semaphore, and it wake up whenever
the semaphore is signalled (which occurs when the OSProcess plugin detects
a SIGCHLD signal from the operating system). Each time it wakes up, it
calls a primitive (UnixOSProcessPlugin>>primitiveReapChildProcess). That
primitive uses a call to waitpid() to obtain the exit status of the child
process, and as soon as that happens the process is no longer in a zombie
state.

I am not sure where this is going wrong, but one thing you can try is
to restart the child watcher process, and see if that makes the zombies
go away:

   "OSProcess accessor restartChildWatcherProcess"

The actual child watcher process is UnixOSProcessAccessor>>grimReaperProcess,
so you may want to look at that and see if it gives you any other ideas.

Please let us know if you figure out the problem. Whatever is going wrong
here, I definitely want to get it fixed.

Thanks,
Dave

Hi David,
Am 08.02.2015 um 21:04 schrieb David T. Lewis:
I see that process while producing Zombies
leaves the number of Zombies constant and continues to produce new ones
on each call of ExternalUnixOSProcess>>command

> The actual child watcher process is
> UnixOSProcessAccessor>>grimReaperProcess, so you may want to look at that

That looked like more than I can handle before I'm off again so I tried
something different. On Saturday I had bookmarked:
http://en.wikipedia.org/wiki/Zombie_process
where the fourth paragraph of the overview section talks about sending
SIGCHLD so I tried that.
It works but only adds to the mystery.

For each event in the real world (me moving in front of a PIR sensor) I
send one
PipeableOSProcess command: 'i2cget -y 1 0x20) and two
ExternalUnixOSProcess command: 'i2cset -y 1 0x20', commandByte

This gave me three new Zombies per event. PipeableOSProcess does not
understand sigchld so I only sent sigchld to the two
ExternalUnixOSProcess  end expected a reduction to one Zombie per event.
But no more Zombies are produced. We called that "Fehler unerkannt
entkommen" (roughly "the error escaped incognito") and I'm very
interested what happened here.

> and see if it gives you any other ideas. Please let us know if you
> figure out the problem. Whatever is going wrong here, I definitely
> want to get it fixed. Thanks, Dave


Later I'll try to step through grimReaperProcess to see what I can
learn. And then I'll also look at the version of OSProcess you recently
uploaded.
Also the other mystery: When I overflowed the process table I couldn't
from a terminal do 'ls | grep something' but I could do that via
ExternalUnixOSProcess  without creating additional Zombies. So which
commands create Zombies and which don't?

Cheers,

Herbert

Hi,

just a short follow up.

Am 09.02.2015 um 15:38 schrieb Herbert König:
> Hi David,
> Am 08.02.2015 um 21:04 schrieb David T. Lewis:
>> This looks like some kind of problem in OSProcess. I am not sure what
>> is wrong, so let me try to explain how it works.

several updates of Raspbian later I do not need to send sigchld any
more. So I assume it was a bug in Raspbian introduced by the transition
to the Pi 2. Nothing to do with OSProcess.

Also I now use the version of OSProcess which limits the number of
children it holds on to.

This has run uninterrupted now for two weeks without a single more
Zombie and Squeak's memory consumption being constant.

Thanks,

Herbert

On Mon, Feb 23, 2015 at 08:27:42PM +0100, Herbert K?nig wrote:
> Hi,
>
> just a short follow up.

Hi Herbert,

Thank you for the follow up message, much appreciated!

Dave