Gemstone 3106 with Pier and crawler
Locked
Gemstone 3106 with Pier and crawler
 
|
Ciao,
i have a Gemstone 3.1.0.6 environment with Seaside and Pier framework. The Gemstone stone and the relative Fast-CGI service is managed by Daemontools. Everything works for a few years. It's been a few days since the home server is continuously submitted to requests (one every 5 seconds) made by ip 46.4.122.196 ( a crawler ?! ) Now it happens that in some situations the service relative to Fast-CGI go down but is not managed ( restart )from daemontools. These are the daemontool service entry /etc/service/gs_seaside-9060: up (pid 25393) 142648 seconds /etc/service/gs_seaside-9061: up (pid 7106) 15059 seconds /etc/service/gs_seaside-9062: up (pid 20118) 194917 seconds Every Fast-CGI service create 3 entry. I report the 9062 entry with: lsof -n -i -P | grep 20118 topaz 20118 dario 6u IPv6 236275 0t0 TCP [::1]:40117->[::1]:42453 (ESTABLISHED) topaz 20118 dario 8u IPv6 236276 0t0 TCP [::1]:40503->[::1]:43399 (ESTABLISHED) topaz 20118 dario 12u IPv4 236281 0t0 TCP *:9062 (LISTEN) When the 20118 Fast-CGI IPv4 go down the : topaz 20118 dario 6u IPv6 236275 0t0 TCP [::1]:40117->[::1]:42453 (ESTABLISHED) topaz 20118 dario 8u IPv6 236276 0t0 TCP [::1]:40503->[::1]:43399 (ESTABLISHED) remain active and therefore i believe that the daemontools does not consider service ( 20118 ) to be dead. But the web request ( when all the Fast-CGI are down )is not managed by any TCP *:906[ 0 - 2 ] (LISTEN) because all are down. Any idea about it? Another question: i can intercept the requests made by 46.4.122.196 before creating the related WASessions and replying ...????.... - do not reply? Thanks for every consideration, Dario _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: Gemstone 3106 with Pier and crawler
|
|
You can block the IP with nginx (or apache if you're using it) if you want:
https://help.dreamhost.com/hc/en-us/articles/216456127-Blocking-IPs-with-Nginx and I'm sure you could google an article that covers how to do it with Apache. For the "daemontools thinks its working but its really not" problem you could add an additional daemontools monitored process that checks a url that is served by each fastcgi process e.g. every 30 seconds or so and then kill the gem if it returns the wrong thing. GLASS mailing list wrote > Ciao, > > i have a Gemstone 3.1.0.6 environment with Seaside and Pier framework. > > The Gemstone stone and the relative Fast-CGI service is managed by > Daemontools. > > Everything works for a few years. > > It's been a few days since the home server is continuously submitted to > requests (one every 5 seconds) made by ip 46.4.122.196 ( a crawler ?! ) > > Now it happens that in some situations the service relative to Fast-CGI > go down but is not managed ( restart )from daemontools. > > These are the daemontool service entry > /etc/service/gs_seaside-9060: up (pid 25393) 142648 seconds > /etc/service/gs_seaside-9061: up (pid 7106) 15059 seconds > /etc/service/gs_seaside-9062: up (pid 20118) 194917 seconds > > Every Fast-CGI service create 3 entry. > > I report the 9062 entry with: lsof -n -i -P | grep 20118 > > topaz 20118 dario 6u IPv6 236275 0t0 TCP > [::1]:40117->[::1]:42453 (ESTABLISHED) > topaz 20118 dario 8u IPv6 236276 0t0 TCP > [::1]:40503->[::1]:43399 (ESTABLISHED) > topaz 20118 dario 12u IPv4 236281 0t0 TCP *:9062 (LISTEN) > > When the 20118 Fast-CGI IPv4 go down the : > > topaz 20118 dario 6u IPv6 236275 0t0 TCP > [::1]:40117->[::1]:42453 (ESTABLISHED) > topaz 20118 dario 8u IPv6 236276 0t0 TCP > [::1]:40503->[::1]:43399 (ESTABLISHED) > > remain active and therefore i believe that the daemontools does not > consider service ( 20118 ) to be dead. > > But the web request ( when all the Fast-CGI are down )is not managed by > any TCP *:906[ 0 - 2 ] (LISTEN) because all are down. > > Any idea about it? > > Another question: > > i can intercept the requests made by 46.4.122.196 before creating the > related WASessions and replying ...????.... - do not reply? > > Thanks for every consideration, > > Dario > _______________________________________________ > Glass mailing list > Glass@.gemtalksystems > http://lists.gemtalksystems.com/mailman/listinfo/glass -- Sent from: http://forum.world.st/GLASS-f1460844.html _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: Gemstone 3106 with Pier and crawler
|
|
Ciao, thanks.
> You can block the IP with nginx (or apache if you're using it) if you want: > > https://help.dreamhost.com/hc/en-us/articles/216456127-Blocking-IPs-with-Nginx > > and I'm sure you could google an article that covers how to do it with > Apache. OK, i use Lighttpd server and i found the relative data entry to do it. Would it be interesting, at a first level, to directly manage the request in Seaside without having to restart the web server? > > > For the "daemontools thinks its working but its really not" problem you > could add an additional daemontools monitored process that checks a url that > is served by each fastcgi process e.g. every 30 seconds or so and then kill > the gem if it returns the wrong thing. Are you saying that as I configured the system does not work? When the system has been subjected to a greater load of requests, everything is degraded! Does anyone have a Gemstone Pier site with a certain load of requests? > > GLASS mailing list wrote >> Ciao, >> >> i have a Gemstone 3.1.0.6 environment with Seaside and Pier framework. >> >> The Gemstone stone and the relative Fast-CGI service is managed by >> Daemontools. >> >> Everything works for a few years. >> >> It's been a few days since the home server is continuously submitted to >> requests (one every 5 seconds) made by ip 46.4.122.196 ( a crawler ?! ) >> >> Now it happens that in some situations the service relative to Fast-CGI >> go down but is not managed ( restart )from daemontools. >> >> These are the daemontool service entry >> /etc/service/gs_seaside-9060: up (pid 25393) 142648 seconds >> /etc/service/gs_seaside-9061: up (pid 7106) 15059 seconds >> /etc/service/gs_seaside-9062: up (pid 20118) 194917 seconds >> >> Every Fast-CGI service create 3 entry. >> >> I report the 9062 entry with: lsof -n -i -P | grep 20118 >> >> topaz 20118 dario 6u IPv6 236275 0t0 TCP >> [::1]:40117->[::1]:42453 (ESTABLISHED) >> topaz 20118 dario 8u IPv6 236276 0t0 TCP >> [::1]:40503->[::1]:43399 (ESTABLISHED) >> topaz 20118 dario 12u IPv4 236281 0t0 TCP *:9062 (LISTEN) >> >> When the 20118 Fast-CGI IPv4 go down the : >> >> topaz 20118 dario 6u IPv6 236275 0t0 TCP >> [::1]:40117->[::1]:42453 (ESTABLISHED) >> topaz 20118 dario 8u IPv6 236276 0t0 TCP >> [::1]:40503->[::1]:43399 (ESTABLISHED) >> >> remain active and therefore i believe that the daemontools does not >> consider service ( 20118 ) to be dead. So this consideration is true? >> >> But the web request ( when all the Fast-CGI are down )is not managed by >> any TCP *:906[ 0 - 2 ] (LISTEN) because all are down. >> >> Any idea about it? >> >> Another question: >> >> i can intercept the requests made by 46.4.122.196 before creating the >> related WASessions and replying ...????.... - do not reply? Thanks, Dario _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: Gemstone 3106 with Pier and crawler
|
|
Hi Dario,
GLASS mailing list wrote > Ciao, thanks. > >> You can block the IP with nginx (or apache if you're using it) if you >> want: >> >> https://help.dreamhost.com/hc/en-us/articles/216456127-Blocking-IPs-with-Nginx >> >> and I'm sure you could google an article that covers how to do it with >> Apache. > > OK, i use Lighttpd server and i found the relative data entry to do it. > > Would it be interesting, at a first level, > > to directly manage the request in Seaside without having to restart the > web server? I'm sure there is but I don't know exactly where and why you'd want Seaside checking every request to see if its from an IP if the webserver can do it faster. I'd probably start looking in WAServerAdaptor to figure out how. GLASS mailing list wrote >> >> >> For the "daemontools thinks its working but its really not" problem you >> could add an additional daemontools monitored process that checks a url >> that >> is served by each fastcgi process e.g. every 30 seconds or so and then >> kill >> the gem if it returns the wrong thing. > > Are you saying that as I configured the system does not work? > > When the system has been subjected to a greater load of requests, > everything is degraded! > > Does anyone have a Gemstone Pier site with a certain load of requests? I just think sometimes there are errors in the gems that aren't enough to kill them, so daemontools doesn't know they're faulty, but the gem would/could still benefit from a restart. It should be able to handle the extra load as what you've described isn't much I'm not sure why performance was degraded. Are you using SSDs ? Did you run out of disk where you're storing stats or tranlogs? GLASS mailing list wrote >> >> GLASS mailing list wrote >>> Ciao, >>> >>> i have a Gemstone 3.1.0.6 environment with Seaside and Pier framework. >>> >>> The Gemstone stone and the relative Fast-CGI service is managed by >>> Daemontools. >>> >>> Everything works for a few years. >>> >>> It's been a few days since the home server is continuously submitted to >>> requests (one every 5 seconds) made by ip 46.4.122.196 ( a crawler ?! ) >>> >>> Now it happens that in some situations the service relative to Fast-CGI >>> go down but is not managed ( restart )from daemontools. >>> >>> These are the daemontool service entry >>> /etc/service/gs_seaside-9060: up (pid 25393) 142648 seconds >>> /etc/service/gs_seaside-9061: up (pid 7106) 15059 seconds >>> /etc/service/gs_seaside-9062: up (pid 20118) 194917 seconds >>> >>> Every Fast-CGI service create 3 entry. >>> >>> I report the 9062 entry with: lsof -n -i -P | grep 20118 >>> >>> topaz 20118 dario 6u IPv6 236275 0t0 TCP >>> [::1]:40117->[::1]:42453 (ESTABLISHED) >>> topaz 20118 dario 8u IPv6 236276 0t0 TCP >>> [::1]:40503->[::1]:43399 (ESTABLISHED) >>> topaz 20118 dario 12u IPv4 236281 0t0 TCP *:9062 (LISTEN) >>> >>> When the 20118 Fast-CGI IPv4 go down the : >>> >>> topaz 20118 dario 6u IPv6 236275 0t0 TCP >>> [::1]:40117->[::1]:42453 (ESTABLISHED) >>> topaz 20118 dario 8u IPv6 236276 0t0 TCP >>> [::1]:40503->[::1]:43399 (ESTABLISHED) >>> >>> remain active and therefore i believe that the daemontools does not >>> consider service ( 20118 ) to be dead. > > So this consideration is true? Yes I think you're right. Daemontools just makes sure the gem process is running, not whether the gem can handle tcp connections (or do anything else for that matter). Hope this helps. GLASS mailing list wrote >>> >>> But the web request ( when all the Fast-CGI are down )is not managed by >>> any TCP *:906[ 0 - 2 ] (LISTEN) because all are down. >>> >>> Any idea about it? >>> >>> Another question: >>> >>> i can intercept the requests made by 46.4.122.196 before creating >>> the >>> related WASessions and replying ...????.... - do not reply? > > Thanks, > > Dario > > _______________________________________________ > Glass mailing list > Glass@.gemtalksystems > http://lists.gemtalksystems.com/mailman/listinfo/glass -- Sent from: http://forum.world.st/GLASS-f1460844.html _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: Gemstone 3106 with Pier and crawler
|
|
Ciao, the FASTCGI daemon_server-9060.log report:
In the GLASS mailing list i found: [1] http://forum.world.st/Glass-Fastcgi-Socket-issues-td4816974.html At a certain point we read: Maybe the heavy load makes it reproduce it because it's only when the SPC gets full enough to flush the GsSocket instance to disk... And I think it's the problem I had when the crawler made many requests, and the system is degraded. Now as I write this email, the system has created 32 new WASessions in one minute and the system is degraded again, and i need to restart ( kill ) the Fastcgi daemon entry. The solution for my old environment Gemstone 3.1.0.6 is to comment the last lines **** of code in :
as reported at the end of [ 1 ] ? Thanks for every consideration, Dario
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: Gemstone 3106 with Pier and crawler
|
|
What version of Seaside are you running?
This reminds me of https://github.com/GsDevKit/Seaside31/issues/64 Johan
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: Gemstone 3106 with Pier and crawler
|
|
Ciao,
Seaside 3.0.13 Magritte3 3.0 Magritte3AddOns 3.0.0
Pier3 3.0.0
Pier3AddOns 3.0.3 Thanks, Dario
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: Gemstone 3106 with Pier and crawler
|
|
Dario,
I think upgrading Seaside would help out with the bug you are seeing. This because of the familiar signature of the bug that was fixed in 3.1.4 (which I referenced in my previous email). If you do not want to upgrade, you might want to try patching with the diff referenced from within the bug issues. Cheers Johan
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
«
Return to GLASS
|
1 view|%1 views
| Free forum by Nabble | Edit this page |