How to get rid of instances of ExternalUnixOSProcess

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

How to get rid of instances of ExternalUnixOSProcess

Herbert König
Hi,

I start external OS processes via the class side method command:
The returned instances correctly show runState #complete but the
instances won't get garbage collected.
Pointer Finder gets me ThisOSProvess allMyChildren.

I start the external program every few seconds so the instances accumulate.

So should I manually send unregisterChildProcess: or should I reuse one
instance of ExternalUnixOSProcess via repeated forkChild  to restart the
program? Or something completely different?

Thanks,

Herbert

Reply | Threaded
Open this post in threaded view
|

Re: How to get rid of instances of ExternalUnixOSProcess

JohnReed Maffeo
> Sent: Tuesday, February 03, 2015 at 6:49 AM
> From: "Herbert König" <[hidden email]>
> To: "The general-purpose Squeak developers list" <[hidden email]>
> Subject: [squeak-dev] How to get rid of instances of ExternalUnixOSProcess
>
> Hi,
>
> I start external OS processes via the class side method command:
> The returned instances correctly show runState #complete but the
> instances won't get garbage collected.
> Pointer Finder gets me ThisOSProvess allMyChildren.
>
> I start the external program every few seconds so the instances accumulate.
>
> So should I manually send unregisterChildProcess: or should I reuse one
> instance of ExternalUnixOSProcess via repeated forkChild  to restart the
> program? Or something completely different?
>
> Thanks,
>
> Herbert

David T. Lewis wrote to the beginners list,
"Just an FYI - I just made some updates to PipeableOSProcess class>>command:
so that it will do a better job of cleaning up pipe handles. You'll still need
to keep an eye out for pipe handle leaks, but expressions like this should now
close all pipes after processing is completed:

(PipeableOSProcess command: '/bin/echo this is a test') output

The updates are in the latest version of CommandShell on SqueakSource."


jrm

Reply | Threaded
Open this post in threaded view
|

Re: How to get rid of instances of ExternalUnixOSProcess

Herbert König
Hi JohnReed,

thanks for your hint but PipeableOSProcess is in a different class
hierarchy than ExternalUnixOSProcess.
I updated to the latest versions of OSProcess and CommandShell and it
doesn't help.

I think I use   it the wrong way. Do you know of documentation other
than the examples in the code?

Cheers,

Herbert

Am 03.02.2015 um 16:20 schrieb JohnReed Maffeo:

>> Sent: Tuesday, February 03, 2015 at 6:49 AM
>> From: "Herbert König" <[hidden email]>
>> To: "The general-purpose Squeak developers list" <[hidden email]>
>> Subject: [squeak-dev] How to get rid of instances of ExternalUnixOSProcess
>>
>> Hi,
>>
>> I start external OS processes via the class side method command:
>> The returned instances correctly show runState #complete but the
>> instances won't get garbage collected.
>> Pointer Finder gets me ThisOSProvess allMyChildren.
>>
>> I start the external program every few seconds so the instances accumulate.
>>
>> So should I manually send unregisterChildProcess: or should I reuse one
>> instance of ExternalUnixOSProcess via repeated forkChild  to restart the
>> program? Or something completely different?
>>
>> Thanks,
>>
>> Herbert
> David T. Lewis wrote to the beginners list,
> "Just an FYI - I just made some updates to PipeableOSProcess class>>command:
> so that it will do a better job of cleaning up pipe handles. You'll still need
> to keep an eye out for pipe handle leaks, but expressions like this should now
> close all pipes after processing is completed:
>
> (PipeableOSProcess command: '/bin/echo this is a test') output
>
> The updates are in the latest version of CommandShell on SqueakSource."
>
>
> jrm
>


Reply | Threaded
Open this post in threaded view
|

Re: How to get rid of instances of ExternalUnixOSProcess

David T. Lewis
In reply to this post by Herbert König
On Tue, Feb 03, 2015 at 02:49:54PM +0100, Herbert K??nig wrote:

> Hi,
>
> I start external OS processes via the class side method command:
> The returned instances correctly show runState #complete but the
> instances won't get garbage collected.
> Pointer Finder gets me ThisOSProvess allMyChildren.
>
> I start the external program every few seconds so the instances accumulate.
>
> So should I manually send unregisterChildProcess: or should I reuse one
> instance of ExternalUnixOSProcess via repeated forkChild  to restart the
> program? Or something completely different?

All of the child processes that run in your current Squeak session (beginning
when you start the image) are saved in a dictionary. This is just to make
them browsable (OSProcess thisOSProcess allMyChildren).

Unregistering each child process, as you suggest above, will work fine, at
least after fixing the bug that you found (thank you for that, it is updated
on squeaksource now).

But you can also clean up all of the terminated processes like this:

  OSProcess thisOSProcess discardExitedChildren

Or you can empty the dictionary completely:

  OSProcess thisOSProcess resetChildProcessDictionary

I never implemented anything to automatically clean up the child process
dictionary because it did not really seem necessary based on how I was
using OSProcess. But for the kind of thing you are doing (restart a new
process every few seconds, maybe for a long time) I can see that it might
be a real problem. Do you think that this should be changed?

Thanks,

Dave


Reply | Threaded
Open this post in threaded view
|

Re: How to get rid of instances of ExternalUnixOSProcess

Herbert König
Hi Dave,

 

> All of the child processes that run in your current Squeak session (beginning
> when you start the image) are saved in a dictionary. This is just to make
> them browsable (OSProcess thisOSProcess allMyChildren).
>
> Unregistering each child process, as you suggest above, will work fine, at
> least after fixing the bug that you found (thank you for that, it is updated
> on squeaksource now).
>
> But you can also clean up all of the terminated processes like this:
>
>    OSProcess thisOSProcess discardExitedChildren
>
> Or you can empty the dictionary completely:
>
>    OSProcess thisOSProcess resetChildProcessDictionary

thanks, these helped a lot.

> process every few seconds, maybe for a long time) I can see that it might
> be a real problem. Do you think that this should be changed?

I have been doing embedded with scarce RAM and now I play with home
automation (in a very broad sense) which is supposed to run 24/7 me
being abroad and my family not programming.

 From this perspective my immediate instinct is to not collect the
instances at all.  Now that I know about the fact I prefer to remove
them myself. But I'm sure other users have a different perspective on
this and you might break somebody's code if you wouldn't collect them at
all.

I former uses I collected the processes myself (to kill them, query
their state) and for that purpose wouldn't want to pick them out of a
dictionary.

I'm very interested in learning when this dictionary comes in handy or
is needed because I see many methods referencing allMyChildren but most
of them are not sent in my image so I can't find out.

Cheers,

Herbert

Reply | Threaded
Open this post in threaded view
|

Re: How to get rid of instances of ExternalUnixOSProcess

David T. Lewis
On Wed, Feb 04, 2015 at 11:04:06AM +0100, Herbert K?nig wrote:

> Hi Dave,
>
> >All of the child processes that run in your current Squeak session (beginning
> >when you start the image) are saved in a dictionary. This is just to make
> >them browsable (OSProcess thisOSProcess allMyChildren).
> >
> >Unregistering each child process, as you suggest above, will work fine, at
> >least after fixing the bug that you found (thank you for that, it is updated
> >on squeaksource now).
> >
> >But you can also clean up all of the terminated processes like this:
> >
> >   OSProcess thisOSProcess discardExitedChildren
> >
> >Or you can empty the dictionary completely:
> >
> >   OSProcess thisOSProcess resetChildProcessDictionary
>
> thanks, these helped a lot.
>
> >process every few seconds, maybe for a long time) I can see that it might
> >be a real problem. Do you think that this should be changed?
>
> I have been doing embedded with scarce RAM and now I play with home
> automation (in a very broad sense) which is supposed to run 24/7 me
> being abroad and my family not programming.

I see what you mean! It would be rediculous to accumulate all of the
process proxies in this case.

>
> From this perspective my immediate instinct is to not collect the
> instances at all.  Now that I know about the fact I prefer to remove
> them myself. But I'm sure other users have a different perspective on
> this and you might break somebody's code if you wouldn't collect them at
> all.
>
> I former uses I collected the processes myself (to kill them, query
> their state) and for that purpose wouldn't want to pick them out of a
> dictionary.
>

As a quick and dirty workaround, you might want to just modify
UnixProcess>unregisterChildProcess: to make it do nothing at all.
That would be a hack, but you have no real need for the child process
registry anyway, so that would be an easy way to make the problem go
away for your home automation application.

Meanwhile, I'll see if I can come up with more sensible solution.

> I'm very interested in learning when this dictionary comes in handy or
> is needed because I see many methods referencing allMyChildren but most
> of them are not sent in my image so I can't find out.
>

Well, when I wrote OSProcess/CommandShell, it was sort of an experiment
to figure out if I could wrap various functions of common operating
systems (not just Unix) in such a way that they could be accessible
from Squeak. I wanted to be able to browse the objects that represented
those OS functions, and I wanted to be able to watch the proxy objects
change state as external processes ran and then exited. I figured that
an object that represents the current OS process (the one in which your
Squeak VM is running) should be able to know about the child processes
that it creates. So I added a child process registry to keep track of
this. When the image started, the registry would be empty, and as time
goes on and you run various child processes using OSProcess, the proxies
for those child processes are maintained in the registry dictionary.

None of this has any practical application that I know of, but it fit in
with my idea of having an abstract representation of OS process functions
that can be viewed and manipulated directly from the Squeak image.

Dave


Reply | Threaded
Open this post in threaded view
|

Re: How to get rid of instances of ExternalUnixOSProcess

David T. Lewis
In reply to this post by Herbert König
On Wed, Feb 04, 2015 at 11:04:06AM +0100, Herbert K?nig wrote:

> Hi Dave,
>
> >All of the child processes that run in your current Squeak session (beginning
> >when you start the image) are saved in a dictionary. This is just to make
> >them browsable (OSProcess thisOSProcess allMyChildren).
> >
> >Unregistering each child process, as you suggest above, will work fine, at
> >least after fixing the bug that you found (thank you for that, it is updated
> >on squeaksource now).
> >
> >But you can also clean up all of the terminated processes like this:
> >
> >   OSProcess thisOSProcess discardExitedChildren
> >
> >Or you can empty the dictionary completely:
> >
> >   OSProcess thisOSProcess resetChildProcessDictionary
>
> thanks, these helped a lot.
>
> >process every few seconds, maybe for a long time) I can see that it might
> >be a real problem. Do you think that this should be changed?
>
> I have been doing embedded with scarce RAM and now I play with home
> automation (in a very broad sense) which is supposed to run 24/7 me
> being abroad and my family not programming.
>
> From this perspective my immediate instinct is to not collect the
> instances at all.  Now that I know about the fact I prefer to remove
> them myself. But I'm sure other users have a different perspective on
> this and you might break somebody's code if you wouldn't collect them at
> all.
>
> I former uses I collected the processes myself (to kill them, query
> their state) and for that purpose wouldn't want to pick them out of a
> dictionary.
>
> I'm very interested in learning when this dictionary comes in handy or
> is needed because I see many methods referencing allMyChildren but most
> of them are not sent in my image so I can't find out.


Hi Herbert,

I think I finally came up with a way to handle this problem in a better way.
I changed the child process registry to be a list of recent child processes,
and set it up so the list gets pruned to a reasonable size every time a new
child process is registered. I kept "OSProcess thisOSProcess allMyChildren"
as a way to get a dictionary of recent child processes, so this is still easy
to browse as before.

For your application with home automation, you should see the child process
list maintained at a maximum size of 20, trimmed continuously as each new
process is started.

The update on squeaksource.com is:

  Name: OSProcess-dtl.95
  Author: dtl
  Time: 6 February 2015, 9:02:45.087 pm
  UUID: aaffb554-75d2-406d-be2b-f9d83428f6a8
  Ancestors: OSProcess-dtl.94
 
  OSProcess 4.5.16
 
  The allMyChildren registry is convenient for browsing, but should not
  be allowed to accumulate exited children without limit. Change it from a
  dictionary to an ordered collection, retaining #allMyChildren as a method
  that answers a dictionary of process proxies by pid. Internally keep a
  list of recent children, and prune the list to a reasonable size each
  time a new child process is registered.

Dave


Reply | Threaded
Open this post in threaded view
|

Re: How to get rid of instances of ExternalUnixOSProcess

Herbert König
Hi Dave,

Am 07.02.2015 um 03:58 schrieb David T. Lewis:

> Hi Herbert, I think I finally came up with a way to handle this
> problem in a better way. I changed the child process registry to be a
> list of recent child processes, and set it up so the list gets pruned
> to a reasonable size every time a new child process is registered. I
> kept "OSProcess thisOSProcess allMyChildren" as a way to get a
> dictionary of recent child processes, so this is still easy to browse
> as before. For your application with home automation, you should see
> the child process list maintained at a maximum size of 20, trimmed
> continuously as each new process is started. The update on
> squeaksource.com is: Name: OSProcess-dtl.95 Author: dtl Time: 6
> February 2015, 9:02:45.087 pm UUID:
> aaffb554-75d2-406d-be2b-f9d83428f6a8 Ancestors: OSProcess-dtl.94
> OSProcess 4.5.16 The allMyChildren registry is convenient for
> browsing, but should not be allowed to accumulate exited children
> without limit. Change it from a dictionary to an ordered collection,
> retaining #allMyChildren as a method that answers a dictionary of
> process proxies by pid. Internally keep a list of recent children, and
> prune the list to a reasonable size each time a new child process is
> registered.
> Dave

I'll try this next, it sounds good. Especially as unregistering the
processes failed though I tried to unregister them with the changed
version of the method. But my problems are not from a growing
allMyChildren so I'll open another thread for this.


Thanks,

Herbert