What is the best language for starting an image? I have two programs here: one in perl [1]; and, the other in python [2]. The perl works. The python does not. Both are attempts to write an "external rewriting program" for Apache [3] The aim of the program is to isolate a subdomain from a request; use it to return a port number from a list of subdomains; check to see if the port is open with netcat; if closed, start an image on the port number and then print the port number; if open, print the port number. Both programs turn off buffering. I have no idea what type lives in $_ and I don't seem to have to care. With the python, whatever type readline() is putting into the variable input, I can neither concatenate nor format with another string. Every step produces some wrinkle I do not see. I finally fed input into netcat by adding '\r'. The reason the input variable cannot be formatted into a string and then fed to Popen defies me utterly. What's going on? I mean, I could use the perl program, but sort of want to invest in python. Chris [1] #!/usr/bin/perl -w # ERP - External Rewrite Program # take a subdomain and return a port number $| = 1; while(<STDIN>) { $answer=`nc -z localhost $_`; if ($answer eq '') { $path = sprintf("/home/ooplatin/coglinux/lib/squeak/4.0-2585/squeak -vm display=none /home/ooplatin/squeak.org/Squeak4.4alpha.image /home/ooplating/dirfarm/%u/startparameters.st&", $_); system($path); sleep(3); print $_; } else { print $_; } } [2] #!/usr/bin/env python # ERP - External Rewrite Program # take a subdomain and return a port number import sys, commands, time, subprocess while True: input = sys.stdin.readline() ncstring = '/usr/bin/nc -z localhost ' + input + '\r' myoutput = commands.getoutput(ncstring) if len(myoutput) < 40: startstring = "/home/ooplatin/coglinux/lib/squeak/4.0-2585/squeak -vm display-none /home/ooplatin/squeak.org/Squeak4.4alpha.image /home/ooplatin/dirfarm/%s/startparams.st&" % (input) subprocess.Popen(startstring, shell=True, stdout=subprocess.PIPE) time.sleep(3) print input else: print input [3] ServerName www.ooplatin.com ServerAlias *.ooplatin.com DocumentRoot /home/ooplatin RewriteLog /etc/httpd/logs/rewrite_log RewriteLogLevel 9 RewriteEngine On TransferLog /home/ooplatin/access_log ErrorLog /home/ooplatin/error_log RewriteMap ERP prg:/home/ooplatin/ERP.py RewriteMap SUBDOMAINS txt:/home/ooplatin/subdomains RewriteCond %{HTTP_HOST} ^([^.]+)\.ooplatin.com RewriteRule ^(.*) <a href="http://%1.ooplatin.com:$">http://%1.ooplatin.com:${ERP:${SUBDOMAINS:%1}} [P] |
Hi Chris-- Off your off-topic topic, but... did you consider using FastCGI (or whatever) to hand off the request to an already-running Smalltalk, and use Smalltalk to start the requested image, instead of either Perl or Python? -C -- Craig Latta www.netjam.org/resume +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS) |
On 2013-01-23 7:10 PM, Craig Latta wrote:
> Hi Chris-- > > Off your off-topic topic, but... did you consider using FastCGI (or > whatever) to hand off the request to an already-running Smalltalk, and > use Smalltalk to start the requested image, instead of either Perl or > Python? > > > -C > > -- > Craig Latta > www.netjam.org/resume > +31 6 2757 7177 (SMS ok) > + 1 415 287 3547 (no SMS) > > Chris |
In reply to this post by Chris Cunnington
On Wed, Jan 23, 2013 at 06:11:55PM -0500, Chris Cunnington wrote:
> > What is the best language for starting an image? I have two programs > here: one in perl [1]; and, the other in python [2]. The perl works. The > python does not. I'm probably missing some of the finer nuances here, but if it were me, I'd go with the one that works. I also know from experience that, given a choice, most people prefer to use technology that does not work. So in the end I guess it's really just a matter of personal preference. ;-) Dave |
On 2013-01-23 7:36 PM, David T. Lewis wrote:
> most people prefer to > use technology that does not work. Yikes. That's an insight I think I'll have to reflect on. Clearly, I figure I "ought" to make the python one work. Hmm... Chris |
In reply to this post by ccrraaiigg
On Wed, Jan 23, 2013 at 4:10 PM, Craig Latta <[hidden email]> wrote:
If I understand correctly, Chris wants to use mod_rewrite's <RewriteMap> with the 'prg' option[1]. That runs the specified binary/script and communicates with it over stdin/stdout. There's no way to get it to use the FastCGI protocol[2].
However, mod_rewrite doesn't run the rewrite program for each request like a CGI. It invokes it once, and sends it urls on stdin as requests come in. So it might be possible prepare an image that starts a rewrite server on startup. The tricky bit is synchronization[3]. For DabbleDB we wrote a ruby program to do this—each server had a few thousand images, each configured to run Seaside on a particular port. When a request came in for that image, the ruby program would look up the port, see if the image was running, and if not, start it, then rewrite the request URL so that it would get proxied through to the image. The images would automatically shut down if they didn't receive any requests for a while.
[2] Short of writing an Apache module to do it
[3] Note the "But be very careful:" section at the bottom of [1] Colin |
On 2013-01-23 9:07 PM, Colin Putney wrote:
> For DabbleDB we wrote a ruby program to do this—each server had a few > thousand images, each configured to run Seaside on a particular port. > When a request came in for that image, the ruby program would look up > the port, see if the image was running, and if not, start it, then > rewrite the request URL so that it would get proxied through to the > image. The images would automatically shut down if they didn't receive > any requests for a while. > That's exactly what I'm riffing on. Thank you Ramon Leon for outlining how DabbleDB did it. [1] And after going to the gym, I've realized I forgot the sys.stdout.flush() required after print to actually turn off buffering in the Python script. Otherwise the two processes sharing an anonymous pipe will deadlock waiting forever for the other to send something. ("Programming Python", p. 224) It's almost there. Just airing the problem actually does a lot of good. I'll sleep on it and it'll be clearer tomorrow. Thanks, Chris [1] http://onsmalltalk.com/scaling-seaside-more-advanced-load-balancing-and-publishing |
In reply to this post by Colin Putney-3
> If I understand correctly, Chris wants to use mod_rewrite's > <RewriteMap> with the 'prg' option[1]. Sure, by suggesting FastCGI (or some other way of communicating with a running Smalltalk system, I have no special affinity for FastCGI in particular) I was suggesting using another Apache module entirely instead of mod_rewrite. In the case of FastCGI, it'd probably be mod_fcgid. Carry on, the point of my suggestion was only to consider Smalltalk along with not-Smalltalk. -C -- Craig Latta www.netjam.org/resume +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS) |
Free forum by Nabble | Edit this page |