Running in background with OSProcess

How would I run '/usr/bin/java -jar "/path/to/jenkins.war" &' with OSProcess? Specifically I want to know the process number of the java child process.

I unsuccessfully tried:
* PipeableOSProcess command: '/usr/bin/java -jar "/path/to/jenkins.war" &', which successfully launches java, but I don't know its pid because the returned process is the completed parent "sh" that launched it
* PipeableOSProcess command: '/usr/bin/java -jar "/path/to/jenkins.war"' (without ampersand), which also launches successfully, and leaves the parent "sh" process running as well (not as clean), but I still don't know the java pid

I then succeeded with:
  p := PipeableOSProcess command: '/usr/bin/java -jar "/path/to/jenkins.war" & { echo $! ; }'.
  p output lines first.

But this seems convoluted and I'm assuming there must be a simpler way.

What is the best way to do this?
I.e.:
* launch a long running child process
* without keeping the sh process open
* accessing the pid of the child

Thanks,
Sean

On 07/05/12 2:52 PM, Sean P. DeNigris wrote:
> But this seems convoluted and I'm assuming there must be a simpler way.
>
> What is the best way to do this?
> I.e.:
> * launch a long running child process
> * without keeping the sh process open
> * accessing the pid of the child

How about:

        process := UnixProcess
                forkJob: job
                arguments: args
                environment: env
                descriptors: nil.
        ^ process pid

Note that no "sh" is run, so you may need to set some environment
variables yourself.

Am 07.05.2012 um 20:52 schrieb Sean P. DeNigris:
If this is all shell handling then you should be able to exec your command. An

exec /usr/bin/java -jar "/path/to/jenkins.war"

(if it works) will replace the shell with the execution of the java process. Therefor the pid is the pid of the java process. But Yannis approach looks a lot better from here :)

Norbert

On Mon, May 07, 2012 at 11:46:30PM +0200, Norbert Hartl wrote:
That's an excellent suggestion and it will do exactly what Sean is looking
to do. But it may a bit confusing because the process proxy will think that
it is running a shell even though that shell has done an exec to replace
itself with the /usr/bin/java executable. So I'll try to add some more
explanation here, using the /bin/sleep program as an example rather that
/usr/bin/java. For this example, the "long running process" is /bin/sleep
running for ten seconds, so /bin/sleep takes the place of /usr/bin/java
for the example.

Here is what Sean was originally doing:
        PipeableOSProcess command: '/bin/sleep 10'

This uses a unix shell to run the command, and lets the shell do the argument
parsing (to pass the argument "10" to the /bin/sleep program). The problem
then is that the actual /bin/sleep program is forked by the unix shell,
so you have no direct access to that subprocess from your image, and you
do not know the pid of the process that is actually running your program.

By adding the exec shell command, you get similar results except that the
/bin/sleep program replaces /bin/sh (or bash or whatever the shell was),
such that the pid of the process running /bin/sleep remains the same as
that for /bin/sh (because it is actually the same process).
        PipeableOSProcess command: 'exec /bin/sleep 10'

So this does what is wanted. However, it might be confusing when you inspect
the process proxy in your image, because your image still thinks that a
shell is being run, even though the /bin/sh executable has now been replaced
by /bin/sleep running in that same external process. When you inspect
the process proxy, it will show the program name '/bin/sh' even though
that executable program has now been replaced by /bin/sleep as a result
of the exec.

One way to avoid this confusion is to do the unix shell processing in
Smalltalk rather than relying on an external unix shell. That is what
CommandShell does, so you could run the /bin/sleep program like this and
let CommandShell do all the work that would otherwise be done by /bin/sh.
        CommandShell command: '/bin/sleep 10'

But this is still not quite right, because CommandShell will also open a user
interface window, and that is not what you want.

CommandShell itself uses a ProxyPipeline to represent one or more process
proxies from a command line, so you can use that class directly to evaluate
a command line:
        ProxyPipeline command: '/bin/sleep 10'

This gives you exactly what you expect to see. The command line will be parsed
in a manner similar to a unix shell, and it will set up a single PipeableOSProcess
that runs the /bin/sleep program, with a process proxy that gives both the
pid of the process for /bin/sleep, and the actual program name of the
/bin/sleep program that is being run.

HTH,
Dave

David T. Lewis wrote

        ProxyPipeline command: '/bin/sleep 10'
This gives you exactly what you expect to see.

Okay, it worked! However, I'm confused by the string escaping. Because I'm on a Mac, and only later discovered that meant I'm on Unix, I have lots of paths with spaces and even [gasp] apostrophes in them (the second makes Smalltalk strings a lot of fun). One thing that I loved about OSProcess is that I felt like I had more control over strings (e.g. paths) in commands because I could construct my paths in Smalltalk and then just double-quote the final product.

However, using ProxyPipeline, it seems that CommandShell has layered on its own escaping rules that are different from Bash's. So, although I can write:
    p := PipeableOSProcess command: '/usr/bin/java -DJENKINS_HOME="/path/with/a space/and/an/apostrophe/isn''t/fun/Pharo-1.4/.jenkins"...'

with ProxyPipeline, I apparently have to write:
    p2 := ProxyPipeline command: '/usr/bin/java -DJENKINS_HOME=/path/with/a\ space/and/an/apostrophe/isn\''t/fun/Pharo-1.4/.jenkins...'

I.e. "enclose in double-quotes" vs. "escape single-quotes and spaces with preceding backslash"

Can I somehow circumvent these extra rules so I don't have to adapt all my commands? What is their purpose?

Getting closer and closer to having this work...

Thanks.
Sean

On Mon, May 07, 2012 at 10:05:12PM -0700, Sean P. DeNigris wrote:
Hi Sean,

I'm afraid that the simulation of unix shell behavior is far from perfect,
and many things that you might expect from a real shell (notably including
shell variables and some of the substitution behaviour you are experiencing
here) are either missing from CommandShell, or behave differently for
various reasons.

In this case you are seeing a mashup of Smalltalk string escaping in a
unix-like command shell. You are first composing a string in Smalltalk,
so the "enclose in double-quotes" rules apply. Then you are (quite reasonably)
expected that preceeding-backslash quoting should work in a unix shell,
but it does not work here because I never put that into the shell syntax
processing.

Yes, this is confusing. No, there is no easy workaround or fix. I'm afraid
that doing a complete and faithful reproduction of bash would be a fool's
errand, just way too much work to be worth the trouble. Adding backslash
escaping would not be hard to do but ... well, when it comes to adding
features, where do you stop?  ;)

BTW, I would welcome patches to improve things like this, as long as
they are supported with unit tests.

Dave

David T. Lewis wrote

I'm afraid that the simulation of unix shell behavior is far from perfect
...
BTW, I would welcome patches to improve things like this, as long as
they are supported with unit tests.

I'd def be willing to take a look...

But, for my current pressing problem, I'm looking for the opposite from OSP. Instead of being able to pass any valid Bash syntax and have OSP parse it, I'm willing to restrict myself as long as I can do the parse myself. I don't want globbing or any other fancy processing. I will do all that myself before hand and only be supplying absolute paths. How do I circumvent all the processing and tell OSP what args to actually pass to the underlying system?

Thanks for all the help!

Sean

Sean P. DeNigris wrote

How do I circumvent all the processing and tell OSP what args to actually pass to the underlying system?

I found a solution:
  env := CommandShell new environment.
  pwd := '/'.
args := Array
        with: '-DJENKINS_HOME=/path with spaces/.jenkins'
        with: '-jar'
        with: '/path/with/another space/jenkins.war'
        with: '--httpPort=####'
        with: '--ajp13Port=####'.
desc := (Array with: nil with: nil with: nil).
p := PipeableOSProcess
        new: '/usr/bin/java'
        arguments: args
        environment: env
        descriptors: desc
        workingDir: pwd
        errorPipelineStream: nil.

Since I'm not doing any redirecting, this works. If you have to redirect, it looks like the best move would be to go through ProxyPipeline, like Dave suggested, but define and plugin a custom ShellSyntax that does not do any string processing...

Sean

On Tue, May 08, 2012 at 08:20:38AM -0700, Sean P. DeNigris wrote:
Sean,

This is a good solution. It does exactly what you want, and the command
parameters are clearly specified without relying on either /bin/sh or
ShellSyntax to do the parsing.

One last pointer: When Jenkins is done and your external process exits,
it is good practice to close any open file descriptors (p closePipes).
This will prevent file handle leaks if you run Jenkins on a regularly
scheduled basis.

Dave