[OT] Perl or Python to start an image?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[OT] Perl or Python to start an image?

Chris Cunnington

What is the best language for starting an image? I have two programs
here: one in perl [1]; and, the other in python [2]. The perl works. The
python does not. Both are attempts to write an "external rewriting
program" for Apache [3]

The aim of the program is to isolate a subdomain from a request; use it
to return a port number from a list of subdomains; check to see if the
port is open with netcat; if closed, start an image on the port number
and then print the port number; if open, print the port number. Both
programs turn off buffering.

I have no idea what type lives in $_ and I don't seem to have to care.
With the python, whatever type readline() is putting into the variable
input, I can neither concatenate nor format with another string. Every
step produces some wrinkle I do not see.

I finally fed input into netcat by adding '\r'. The reason the input
variable cannot be formatted into a string and then fed to Popen defies
me utterly.

What's going on? I mean, I could use the perl program, but sort of want
to invest in python.

Chris

[1]

#!/usr/bin/perl -w
# ERP - External Rewrite Program
# take a subdomain and return a port number

$| = 1;

while(<STDIN>) {

$answer=`nc -z localhost $_`;

if ($answer eq '') {

     $path = sprintf("/home/ooplatin/coglinux/lib/squeak/4.0-2585/squeak
-vm display=none /home/ooplatin/squeak.org/Squeak4.4alpha.image
/home/ooplating/dirfarm/%u/startparameters.st&", $_);

     system($path);

     sleep(3);

     print $_;

} else {

     print $_;
}

}

[2]

#!/usr/bin/env python
# ERP - External Rewrite Program
# take a subdomain and return a port number

import sys, commands, time, subprocess

while True:

     input = sys.stdin.readline()
     ncstring = '/usr/bin/nc -z localhost ' + input + '\r'
     myoutput = commands.getoutput(ncstring)

     if len(myoutput) < 40:
         startstring =
"/home/ooplatin/coglinux/lib/squeak/4.0-2585/squeak -vm display-none
/home/ooplatin/squeak.org/Squeak4.4alpha.image
/home/ooplatin/dirfarm/%s/startparams.st&" % (input)
         subprocess.Popen(startstring, shell=True, stdout=subprocess.PIPE)
         time.sleep(3)
         print input
     else:
         print input



[3]

     ServerName www.ooplatin.com
     ServerAlias *.ooplatin.com
     DocumentRoot /home/ooplatin
     RewriteLog /etc/httpd/logs/rewrite_log
     RewriteLogLevel 9
     RewriteEngine On
     TransferLog /home/ooplatin/access_log
     ErrorLog /home/ooplatin/error_log
     RewriteMap ERP prg:/home/ooplatin/ERP.py
     RewriteMap SUBDOMAINS txt:/home/ooplatin/subdomains
     RewriteCond %{HTTP_HOST} ^([^.]+)\.ooplatin.com
     RewriteRule ^(.*) <a href="http://%1.ooplatin.com:$">http://%1.ooplatin.com:${ERP:${SUBDOMAINS:%1}} [P]


Reply | Threaded
Open this post in threaded view
|

Re: [OT] Perl or Python to start an image?

ccrraaiigg

Hi Chris--

     Off your off-topic topic, but... did you consider using FastCGI (or
whatever) to hand off the request to an already-running Smalltalk, and
use Smalltalk to start the requested image, instead of either Perl or
Python?


-C

--
Craig Latta
www.netjam.org/resume
+31   6 2757 7177 (SMS ok)
+ 1 415 287 3547 (no SMS)


Reply | Threaded
Open this post in threaded view
|

Re: [OT] Perl or Python to start an image?

Chris Cunnington
On 2013-01-23 7:10 PM, Craig Latta wrote:

> Hi Chris--
>
>       Off your off-topic topic, but... did you consider using FastCGI (or
> whatever) to hand off the request to an already-running Smalltalk, and
> use Smalltalk to start the requested image, instead of either Perl or
> Python?
>
>
> -C
>
> --
> Craig Latta
> www.netjam.org/resume
> +31   6 2757 7177 (SMS ok)
> + 1 415 287 3547 (no SMS)
>
>
Hmm, no, I hadn't thought of that. That's an interesting idea.

Chris

Reply | Threaded
Open this post in threaded view
|

Re: [OT] Perl or Python to start an image?

David T. Lewis
In reply to this post by Chris Cunnington
On Wed, Jan 23, 2013 at 06:11:55PM -0500, Chris Cunnington wrote:
>
> What is the best language for starting an image? I have two programs
> here: one in perl [1]; and, the other in python [2]. The perl works. The
> python does not.

I'm probably missing some of the finer nuances here, but if it were me, I'd
go with the one that works.

I also know from experience that, given a choice, most people prefer to
use technology that does not work. So in the end I guess it's really just
a matter of personal preference.

;-)

Dave


Reply | Threaded
Open this post in threaded view
|

Re: [OT] Perl or Python to start an image?

Chris Cunnington
On 2013-01-23 7:36 PM, David T. Lewis wrote:
> most people prefer to
> use technology that does not work.
Yikes. That's an insight I think I'll have to reflect on. Clearly, I
figure I "ought" to make the python one work. Hmm...

Chris

Reply | Threaded
Open this post in threaded view
|

Re: [OT] Perl or Python to start an image?

Colin Putney-3
In reply to this post by ccrraaiigg



On Wed, Jan 23, 2013 at 4:10 PM, Craig Latta <[hidden email]> wrote:

Hi Chris--

     Off your off-topic topic, but... did you consider using FastCGI (or
whatever) to hand off the request to an already-running Smalltalk, and
use Smalltalk to start the requested image, instead of either Perl or
Python?

If I understand correctly, Chris wants to use mod_rewrite's <RewriteMap> with the 'prg' option[1]. That runs the specified binary/script and communicates with it over stdin/stdout. There's no way to get it to use the FastCGI protocol[2]. 

However, mod_rewrite doesn't run the rewrite program for each request like a CGI. It invokes it once, and sends it urls on stdin as requests come in. So it might be possible prepare an image that starts a rewrite server on startup. The tricky bit is synchronization[3].

For DabbleDB we wrote a ruby program to do this—each server had a few thousand images, each configured to run Seaside on a particular port. When a request came in for that image, the ruby program would look up the port, see if the image was running, and if not, start it, then rewrite the request URL so that it would get proxied through to the image. The images would automatically shut down if they didn't receive any requests for a while. 

[2] Short of writing an Apache module to do it
[3] Note the "But be very careful:" section at the bottom of [1]

Colin


Reply | Threaded
Open this post in threaded view
|

Re: [OT] Perl or Python to start an image?

Chris Cunnington
On 2013-01-23 9:07 PM, Colin Putney wrote:
> For DabbleDB we wrote a ruby program to do this—each server had a few
> thousand images, each configured to run Seaside on a particular port.
> When a request came in for that image, the ruby program would look up
> the port, see if the image was running, and if not, start it, then
> rewrite the request URL so that it would get proxied through to the
> image. The images would automatically shut down if they didn't receive
> any requests for a while.
>
That's exactly what I'm riffing on. Thank you Ramon Leon for outlining
how DabbleDB did it. [1]

And after going to the gym, I've realized I forgot the
sys.stdout.flush() required after print to actually turn off buffering
in the Python script. Otherwise the two processes sharing an anonymous
pipe will deadlock waiting forever for the other to send something.
("Programming Python", p. 224)

It's almost there. Just airing the problem actually does a lot of good.
I'll sleep on it and it'll be clearer tomorrow.

Thanks,
Chris


[1]
http://onsmalltalk.com/scaling-seaside-more-advanced-load-balancing-and-publishing

Reply | Threaded
Open this post in threaded view
|

Re: [OT] Perl or Python to start an image?

ccrraaiigg
In reply to this post by Colin Putney-3

> If I understand correctly, Chris wants to use mod_rewrite's
> <RewriteMap> with the 'prg' option[1].

     Sure, by suggesting FastCGI (or some other way of communicating
with a running Smalltalk system, I have no special affinity for FastCGI
in particular) I was suggesting using another Apache module entirely
instead of mod_rewrite. In the case of FastCGI, it'd probably be mod_fcgid.

     Carry on, the point of my suggestion was only to consider Smalltalk
along with not-Smalltalk.


-C

--
Craig Latta
www.netjam.org/resume
+31   6 2757 7177 (SMS ok)
+ 1 415 287 3547 (no SMS)