OO languages are spoiling us...

And we forget to close files. :-)

Stupid me for suggesting that Stefan tracked accept system calls, but
not opens!

Index: Core/RequestHandlers/FileHandler.st
===================================================================
--- Core/RequestHandlers/FileHandler.st (revision 1408)
+++ Core/RequestHandlers/FileHandler.st (working copy)
@@ -526,8 +526,10 @@ RequestHandler subclass: FileHandler [

      produceResponse [
  <category: 'responding'>
- self respond: [:response |
-    response nextPutAll: file readStream binary contentsStream.
+ self respond: [:response || stream |
+    stream := file readStream.
+    [response nextPutAll: stream binary contentsStream]
+ ensure: [stream close].
     response contentType: (self mimeTypeFor: file filename).
     self addAllowHeaderTo: response.
     self session useCookies ifTrue: [


With this in place, I have pretty cool results:

1) the number of sockets open while running Stefan's mechanize script is
exactly "number of concurrent requests" * 2.  No CLOSE_WAIT sockets.

2) the file descriptors are never higher than 20, which means that my
connection thresholding system is working nicely (it did have a couple
of bugs).  Without the pool, Swazoo was using about twice as many file
descriptors.  The speed was more or less the same.

With the pool, the beginning of the strace (strace -e accept -e
signal=none) is absolutely awesome:

accept(4, {sa_family=AF_INET, sin_port=htons(57504), ...) = 5
accept(4, {sa_family=AF_INET, sin_port=htons(57505), ...) = 6
accept(4, {sa_family=AF_INET, sin_port=htons(57506), ...) = 7
accept(4, {sa_family=AF_INET, sin_port=htons(57507), ...) = 8
accept(4, {sa_family=AF_INET, sin_port=htons(57508), ...) = 9
accept(4, {sa_family=AF_INET, sin_port=htons(57509), ...) = 10
accept(4, {sa_family=AF_INET, sin_port=htons(57510), ...) = 11
accept(4, {sa_family=AF_INET, sin_port=htons(57511), ...) = 12
accept(4, {sa_family=AF_INET, sin_port=htons(57512), ...) = 13
accept(4, {sa_family=AF_INET, sin_port=htons(57513), ...) = 14
accept(4, {sa_family=AF_INET, sin_port=htons(57514), ...) = 15
accept(4, {sa_family=AF_INET, sin_port=htons(57515), ...) = 16
accept(4, {sa_family=AF_INET, sin_port=htons(57516), ...) = 17
accept(4, {sa_family=AF_INET, sin_port=htons(57517), ...) = 18
accept(4, {sa_family=AF_INET, sin_port=htons(57518), ...) = 19
accept(4, {sa_family=AF_INET, sin_port=htons(57519), ...) = 20
accept(4, {sa_family=AF_INET, sin_port=htons(57520), ...) = 5
accept(4, {sa_family=AF_INET, sin_port=htons(57521), ...) = 6
accept(4, {sa_family=AF_INET, sin_port=htons(57522), ...) = 7
accept(4, {sa_family=AF_INET, sin_port=htons(57523), ...) = 8
accept(4, {sa_family=AF_INET, sin_port=htons(57524), ...) = 9
accept(4, {sa_family=AF_INET, sin_port=htons(57525), ...) = 10
accept(4, {sa_family=AF_INET, sin_port=htons(57526), ...) = 11
accept(4, {sa_family=AF_INET, sin_port=htons(57527), ...) = 12

:-)

3) forty clients consume 70 megabytes, eighty consume 110 megabytes.
and this is on a 64-bit machine; a 32-bit machine would consume half
approximately.  The buffers for the files probably were consuming some
memory; anyway remember that we were at 110 megabytes for 25 clients
when we started.

Only negative point, garbage is not reclaimed well so running "ruby
mech.rb 80" twice will take 174 megabytes.  I didn't check whether
compacting between the two runs would affect the memory consumption.

4) I had one exception that I'll look into:

Object: nil error: did not understand #goodness:
MessageNotUnderstood(Exception)>>signal (AnsiExcept.st:216)
UndefinedObject(Object)>>doesNotUnderstand: #goodness: (AnsiExcept.st:1556)
optimized [] in BlockClosure class>>exceptionHandlerSearchBlock
(BlkClosure.st:13)
[] in Kernel.CoreException>>instantiateNextHandler:from:
(ExcHandling.st:344)
MethodContext(ContextPart)>>scanBacktraceForAttribute:do:
(ContextPart.st:424)
Kernel.CoreException>>instantiateNextHandler:from: (ExcHandling.st:345)
MessageNotUnderstood(Exception)>>signal (AnsiExcept.st:216)
UndefinedObject(Object)>>doesNotUnderstand: #key (AnsiExcept.st:1556)
Delay class>>unscheduleDelay: (Delay.st:142)
Delay class>>handleDelayEvent (Delay.st:91)
optimized [] in Delay class>>runDelayProcess (Delay.st:116)

but GNU Smalltalk, Swazoo, Iliad continued working like a charm!!!

I'm amazed.

Paolo


_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

On 07/25/2009 03:26 PM, nicolas petton wrote:
>
> Yes, it's a lot better. Just for a comparison, do you know how much
> memory does Seaside use per client?

Well, the only way to know would be to reimplement Stefan's app.

 > Do you have an idea why it isn't?

No, I didn't check that.

Paolo


_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

I'm forwarding an obvious point from Stefan on memory usage, that he
wrote to me offlist:

>> >  Only negative point, garbage is not reclaimed well so running "ruby
>> >  mech.rb 80" twice will take 174 megabytes.  I didn't check whether
>> >  compacting between the two runs would affect the memory consumption.
>
> That's not too bad, since the web app does not know that the
> first 80 clients are gone for good. Running "mech.rb 80" twice
> should (now) be only slightly better than running "mech.rb 160".

Paolo


_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

On Sat, 25 Jul 2009 19:39:32 +0200
Paolo Bonzini <[hidden email]> wrote:

just ran mech.rb 80 twice: RSIZE reached 228052 kB on my box.
then mech.rb 160: RSIZE 230416

Running
        gst-remote --eval "ObjectMemory compact"
after mech.rb 160 finished had an interesting effect: RSIZE 238792

s.


_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk