[squeak-dev] debugging PipeableOSProcess

Hello,

I am having a very strange problem using PipeableOSProcess.

I created a method similar to below. It worked fine. Still does.

updateLocalDirectory
    | c d p |
    c := 'wget http://www.url.com/dir1'.
    d := '/home/jimmie/VL/dl/'.
    p := PipeableOSProcess command: c environment: nil workingDir: d
input: nil output: nil error: nil errorPipelineStream: nil.
    p closePipes


I attempted to use the same method to download repositories for other
directories on the same website. The only change was from dir1 to dir2.

Running the command in a shell with dir2 works just fine. Copying the
command from Squeak and pasting in the shell, works fine.

But when I run the method in a workspace. Squeak just sits and waits on
wget which at some point just stops. It doesn't exit or finish.

Sometimes if I kill the wget process, Squeak will resume working.
Sometimes not.

Is there a way to debug such a bizarre situation. With the Squeak UI
locked up until the method exits, I am at a loss on anything I can do.

Any help greatly appreciated.

And thanks for PipeableOSProcess. Its been interesting doing this in
Squeak as opposed to Python.

Jimmie

On Wed, Mar 05, 2008 at 04:51:17PM -0600, Jimmie Houchin wrote:
Hi Jimmie,

If an external process is running and never exits, even though you
expect it to exit, it is probably because the external process is
blocked trying to write a lot of data to its stdout or stderr pipes
but nobody is reading data from the pipes. Once you start reading
the data, the external process will be able to proceed to completion.

Since the stdout and stderr pipes are connected to Squeak by means
of a PipeableOSProcess, this means that you will need to read the
pipes from Squeak in order to relieve the constipation. When you
run a command from a CommandShell, this happens automatically. But
if you create your own instance of PipeableOSProcess, you may need
to read the data out of these pipes in order for the external
process to reach completion.

HTH,

Dave

On Wed, Mar 05, 2008 at 04:51:17PM -0600, Jimmie Houchin wrote:
> I am having a very strange problem using PipeableOSProcess.
<snip>
> Is there a way to debug such a bizarre situation. With the Squeak UI
> locked up until the method exits, I am at a loss on anything I can do.

I forgot to mention: Of course you should be able to just to use
<alt>. to interrupt Squeak and get into a debugger. From there,
you can explore the PipeableOSProcess, find its output and error
pipes, read from them, etc.

Dave

David T. Lewis wrote:
Hello Dave,

Thanks for the help.

As per your other message, I had already tried alt-. to attempt to
interrupt the UI and enter a debugger, but the image does not respond.
It just stays locked. :)

I tried searching the image for how to use the other parts of the
PipeableOSProcess command, input: output: errorPipelineStream:, but
didn't find any. So I do not know how to do what you describe.

While writing this I went ahead and killed wget. Then Squeak interrupted
the UI and had a debugger. It had:

in UnixOSProcessAccesser>>grimReaperProcess

Any help on how to use the stdout and stderr pipes would be greatly
appreciated.

Thanks.

Jimmie

On Wed, Mar 05, 2008 at 11:30:31PM -0600, Jimmie Houchin wrote:
One note on the grimReaperProcess. If you accidentally kill this
process (by stopping it in a debugger and exiting the debugger for
example), the external process handling will stop working. You will
need to restart it, which can be done by stopping and start Squeak.

> Any help on how to use the stdout and stderr pipes would be greatly
> appreciated.

Jimmie,

Try using an object explorer. You can "drill down" into this to see
the pipes, the process proxy, and so forth. The pipes are implemented
by ExternalPipe, OSPipe and InternalPipe, which are all implementations
of Stream.

I'm not sure what is causing your wget process to hang up, but just
as a suggestion you might try using ProxyPipeline class>>command: rather
than PipeableOSProcess class>>command: to run the wget command. The
difference is that the latter uses a real Unix shell (/bin/bash) to
run the wget command, while ProxyPipeline uses a Smalltalk simulation
of the shell to directly run the wget program with no /bin/bash
involved at all. It might be worth a try to see if taking the extra
/bin/bash process out of the equation makes a difference.

(Note, ProxyPipeline is a container for one or more connected
processes, so use an object explorer to see how it is working.
For your wget example, it will be a container with one instance
of PipeableOSProcess, and that instance of PipeableOSProcess
will be running the wget OS process rather than running a
/bin/bash process. Also, ProxyPipeline implements streaming
protocol itself, so you can read from its output and error pipes
directly as if it were a PipeableOSProcess.)

Dave

On Thursday 06 March 2008 4:21:17 am Jimmie Houchin wrote: Are you using any web-proxy in your shell environment ($http_proxy)? You will
have to use the same setting in the environment: too. You may also want to
use the -q option (quiet) when running wget this way.

Subbu

David T. Lewis wrote:
> On Wed, Mar 05, 2008 at 11:30:31PM -0600, Jimmie Houchin wrote:
[snip]
> Jimmie,
>
> Try using an object explorer. You can "drill down" into this to see
> the pipes, the process proxy, and so forth. The pipes are implemented
> by ExternalPipe, OSPipe and InternalPipe, which are all implementations
> of Stream.

If I understand you correctly, I can't do that when the UI is locked?
And so far the UI has remained locked until I kill wget. :)

> I'm not sure what is causing your wget process to hang up, but just
> as a suggestion you might try using ProxyPipeline class>>command: rather
> than PipeableOSProcess class>>command: to run the wget command. The
> difference is that the latter uses a real Unix shell (/bin/bash) to
> run the wget command, while ProxyPipeline uses a Smalltalk simulation
> of the shell to directly run the wget program with no /bin/bash
> involved at all. It might be worth a try to see if taking the extra
> /bin/bash process out of the equation makes a difference.

Thanks for this suggestion. It worked beautifully. I changed everyplace
I was using PipeableOSProcess to use ProxyPipeline and everything is
working as desired.

With wget I am only wanting the files desired. But I have written
wrappers around other commandline tools where I require the output in
order to process the output to use for arguments to other commandline
tools. Heretofore I was using PipeableOSProcess and doing a simple
substitution to ProxyPipeline has gone without problem.

The only thing that I miss somewhat with PipeableOSProcess that I don't
see with ProxyPipeline is the ability to set a current working directory
for the process to run in. I presume it uses the directory of the Squeak
image I am using.

> (Note, ProxyPipeline is a container for one or more connected
> processes, so use an object explorer to see how it is working.
> For your wget example, it will be a container with one instance
> of PipeableOSProcess, and that instance of PipeableOSProcess
> will be running the wget OS process rather than running a
> /bin/bash process. Also, ProxyPipeline implements streaming
> protocol itself, so you can read from its output and error pipes
> directly as if it were a PipeableOSProcess.)

Thanks for the information.

Thank you again for all of this help. It has saved my day.

And a big thank you, thank you, thank you for writing this package in
the first place. It saves me from having to write wrappers in Python
when I would much rather spend my time using and learning more of Squeak.

Have a great day.

Jimmie

K. K. Subramaniam wrote:
Hello,

I have done no customizing of my shell. It is simply the default bash
which comes with Ubuntu Gutsy Gibbon.

Dave got me going again.

Thanks for the reply.

Jimmie