Few Aida on Swazoo benchmarks, Pharo and VW

Hi guys,

Encouraged by recent Philippe's benchmarking I did similar on Aida with
Swazoo web server, adding the comparison of Pharo with VisualWorks.

Results are interesting, Pharo is now only 3x slower than VW on dynamic
page generation but on network layer it is the same or even slighty
faster, while it was around 10x slower on Squeak back in 2008 [1],

Results (requests/s)
--------------------

1. Dynamically generated 3KB login page  (/admin?view=login)

       VW:  120,  Pharo:   40   ratio: 3x

2. Aida ping with zero size response (/ping.html)

local: VW: 4700,  Pharo: 4500   ratio: 1,04x
lan:   VW: 4500   Pharo: 4600   ratio: 0,97x
inet:  VW: 4000,  Pharo: 3000   ratio: 1,33x

3. Aida ping with 8K response

local: VW: 2600,   Pharo: 3100  ratio: 0,84x
lan:   VW: 1400,   Pharo: 1400  ratio: 1,00x
inet:  VW:  140,   Pharo:  140  ratio: 1,00x

Note: network saturation!

Legend:

  local: client run on local machine
  lan:   client run on machine over 1Gb/s LAN netwrok
  inet:  client run on machine over 100/10Mb/s internet connection



Test environment
----------------

ab (ApacheBech) testing tool

Test machine:
Intel Core i7 860 (2 cores) at 2.80GHz, 8GB RAM
openSuse 11.4 64bit Linux

Pharo 1.3 13320 OneClick (with CogVM)
VisualWorks 7.8
Aida 6.4 (right after installation)

Swazoo patch to allow Keep Alive (ab -k) over HTTP 1.0:

HTTPConnection>>getAndDispatchMessages
  ...
  (self task request isHttp10 and: [self task request isKeepAlive not])
        ifTrue: [self close].
  ...


[1] Swazoo vs. Apache Benchmark (2008)
http://www.swazoo.org/benchmarks/swazoo-vs-apache.html

Best regards
Janko


--
Janko Mivšek
Aida/Web
Smalltalk Web Application Server
http://www.aidaweb.si

On 02/27/2012 01:56 PM, Janko Mivšek wrote:
> ...
> Results are interesting, Pharo is now only 3x slower than VW on dynamic
> page generation but on network layer it is the same or even slighty
> faster, ...

Do you have any explanation for this? I'm amazed at how close the number
for static content are. Sure, it's the same Smalltalk code but different
VMs. At least in the AJP case the image maxes out one core so I would
expect the VM to play a big role. The numbers for dynamic content are
more what I would expect.

Cheers
Philippe

S, Philippe Marschall piše:
> Janko Mivšek wrote:

>> Results are interesting, Pharo is now only 3x slower than VW on dynamic
>> page generation but on network layer it is the same or even slighty
>> faster, ...

> Do you have any explanation for this? I'm amazed at how close the number
> for static content are. Sure, it's the same Smalltalk code but different
> VMs. At least in the AJP case the image maxes out one core so I would
> expect the VM to play a big role. The numbers for dynamic content are
> more what I would expect.

It was surprise for me too. Ok, for 8K responses not too much, because
this is almost a plain block copying by primitive in VM and here Pharo
VM cannot be much slower to VW, but for zero size responses?

I just rechecked once again, same results.

I checked with Wireshark network sniffer that request and responses were
composed correctly and keep-alive worked. There are differences between
both traces but not so much to be worth studying how both VMs assemble
and dissasemble TCP packets.

CPU was saturated 100% in VW and 110%(?) in Pharo (top - looking just VM
proces). This is two core machine.

It seems that network layer works most in both cases while image
processing much less, that's why differences in speed are not so
evident, IMHO. Bust still, how can I check that?

Best regards
Janko





>
> Cheers
> Philippe
>
>
>

--
Janko Mivšek
Aida/Web
Smalltalk Web Application Server
http://www.aidaweb.si