Smalltalk › Frameworks & Tools › Swazoo

Re: Yes, it works - and fast too! (was [Q] File Upload/Download Server, Comanche or Swazoo)

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

2 messages Options

Janko Mivšek

Re: Yes, it works - and fast too! (was [Q] File Upload/Download Server, Comanche or Swazoo)

Hi all,

Göran's work encouraged me to run benchmarks more carefully and
repeatably to minimize impact of OS file system performance and as you
can see, Kom and Swazoo are actually close in upload performance, about
5-6MB/s.

Big apology to Squeak community for that original 1.5MB/s figure, which
obviously came out from only one test, instead of repeatable to avoid
other impacts. In new tests I didn't change Swazoo code, I just made an
additional upload demo on Aida.

Here are results in seconds of repeatable test for two files, 572MB and
102MB ones:

572MB file (600.037.00 bytes)

Squeak:
Kom/Seaside 92 98 117 (avg 5.9MB/s)
Swazoo/Aida 108 111 107 (avg 5.6MB/s)
VW:
Swazoo/Aida 67 68 66 (avg 9MB/s)

102MB file (106.491.000 bytes)

Squeak:
Kom/Seaside 18 35 25 32 (avg 3.9MB/s)
Swazoo/Aida 16 16 17 16 (avg 6.6MB/s)
VW:
Swazoo/Aida 5 5 6 5 (avg 20MB/s)

As you can see for larger files we are very close while for smaller file
Swazoo is still faster. Also you can see that Kom results vary a lot
while Swazoo ones are stable. It seems that this is due to bigger amount
of garbage generation by Kom (this can also be seen from memory
consumption variations) while Swazoo garbage is minimal.

In any case we can conclude with quite certainty that on Squeak we
reached the upload speed limit near 6MB/sec. To be faster we will need
to optimize TCP socket and file VM modules.

Tests were done on Damien's squeak-dev 3.9 with both Aida and Seaside
installed, with Swazoo 2.2beta and Göran's Kom patch. Done on openSuse
10.0 64bit on 3.2GHz PC, bogomips: 6400.

Best regards
Janko

Göran Krampe wrote:

> Ok, I just tested this again using this setup on my Dell D420 (small
> Dell laptop, bogomips: 2394 according to /proc/cpuinfo) using:
>
> - 3.8.1-6747 image
> - gokr@yoda:~$ squeakvm -version
> 3.9-8 #1 Tue Mar 25 22:39:11 UTC 2008 gcc 4.2.3
> Squeak3.9alpha of 4 July 2005 [latest update: #7021]
> Linux vernadsky 2.6.15.7 #1 SMP
> - Xubuntu Linux yoda 2.6.24-19-generic #1 SMP
>
> I tested by loading my changeset into a Gjallar image and then using a
> firefox 3 browser on:
> http://localhost:8080/seaside/tests/alltests
>
> ...where there is an upload test. Ok the numbers:
>
> Local test with a file of size 731654144 took circa 138 seconds to
> upload giving approximately 5Mb/sec upload speed. Since Janko mentions
> 1.5Mb/sec for Swazoo this seems kinda good. :)
>
> The image size does not move an inch during this - but the CPU shows 95%
> load in top - looking closer at this it seems to be a side effect
> because Process browser tells me that the actual Process in Squeak doing
> the work is showing about 3% time! It is instead Morphic/idle that goes
> up. Squeak is also fully responsive as is the OS during the upload.
>
> I am sure this is related to other mechanisms in Squeak. And oh, yes,
> also tried a file of size 1463308288 (double size) giving eh, slightly
> larger upload time (not sure why).
>
> Did not succeed with a 2Gb+ file, not sure why, haven't investigated
> further. Code attached, it downloads files into ./seasidetmp
>
> regards, Göran

--
Janko Mivšek
AIDA/Web
Smalltalk Web Application Server
http://www.aidaweb.si

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Swazoo-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/swazoo-devel

Janko Mivšek

Re: Yes, it works - and fast too! (was [Q] File Upload/Download Server, Comanche or Swazoo)

Dear all,

New tests, with Göran's new Kom patch with direct copying and updated
Swazoo with large buffer support. Here I tested impact of buffer size to
the performance, with 8K (only Swazoo), 1M and 10M buffers.

572MB file (600.037.000 bytes)

Squeak:

Kom/Seaside 1M 110 104 104 (avg 5.7MB/s)
10M 100 104 98 (avg 5.9MB/s) (+4%)

Swazoo/Aida 8K 102 107 105 (avg 5.7MB/s)
1M 93 92 93 (avg 6.5MB/s) (+20%)
10M 90 89 91 (avg 6.7MB/s
VW:
Swazoo/Aida 8K 63 63 63 (avg 9.5MB/s)
1M 75 72 73 (avg 8.2MB/s) (-15%)
10M 70 70 70 (avg 8.6MB/s

102MB file (106.491.000 bytes)

Squeak:

Kom/Seaside 1M 22 20 21 (avg 5MB/s)
10M 16 21 16 (avg 6MB/s) (+20%)

Swazoo/Aida 8K 14 15 14 (avg 7.5MB/s)
1M 11 11 12 (avg 9MB/s) (+20%)
10M 12 11 12 (avg 9MB/s)
VW:
Swazoo/Aida 8K 5 5 6 (avg 20MB/s)
1M 6 6 6 (avg 18MB/s) (-10%)
10M 6 6 6 (avg 18MB/s)

Kom results are now more stable from run to run and better for 100MB
file where impact of large buffer is also biggest, +20%.

Swazoo on Squeak is 20% better with 1M buffer while 10M one improves
upload just slightly. Interesting are VisualWorks results, which are
with bigger buffer 10% worse !

Above tests were done with server and browser on the same machine, now
it is a time to rerun them on the real network. Buffer sizes can have
different impact then!

Best regards
Janko

Göran Krampe wrote:

> Janko Mivšek wrote:

>> Göran's work encouraged me to run benchmarks more carefully and
>> repeatably to minimize impact of OS file system performance and as you
>> can see, Kom and Swazoo are actually close in upload performance, about
>> 5-6MB/s.
>>
>> Big apology to Squeak community for that original 1.5MB/s figure, which
>> obviously came out from only one test, instead of repeatable to avoid
>> other impacts. In new tests I didn't change Swazoo code, I just made an
>> additional upload demo on Aida.
>>
>> Here are results in seconds of repeatable test for two files, 572MB and
>> 102MB ones:
>>
>> 572MB file (600.037.00 bytes)
>>
>> Squeak:
>> Kom/Seaside 92 98 117 (avg 5.9MB/s)
>> Swazoo/Aida 108 111 107 (avg 5.6MB/s)
>> VW:
>> Swazoo/Aida 67 68 66 (avg 9MB/s)
>>
>>
>> 102MB file (106.491.000 bytes)
>>
>> Squeak:
>> Kom/Seaside 18 35 25 32 (avg 3.9MB/s)
>> Swazoo/Aida 16 16 17 16 (avg 6.6MB/s)
>> VW:
>> Swazoo/Aida 5 5 6 5 (avg 20MB/s)
>>
>> As you can see for larger files we are very close while for smaller file
>> Swazoo is still faster. Also you can see that Kom results vary a lot
>> while Swazoo ones are stable. It seems that this is due to bigger amount
>> of garbage generation by Kom (this can also be seen from memory
>> consumption variations) while Swazoo garbage is minimal.
>>
>> In any case we can conclude with quite certainty that on Squeak we
>> reached the upload speed limit near 6MB/sec. To be faster we will need
>> to optimize TCP socket and file VM modules.
>
> Hmmm, ok, I did two more things:
> - Refactored away one buffer copy. Now the Kom code copies straight from
> the SocketStream inBuffer into the FileStream.
> - Raised the buffer size from 1 Mb to 10 (or 20, 30, it does improve
> numbers with a large buffer).
>
> This gave the following numbers on my box for some files:
>
> Time: 11.613 Size: 81601394 MB/s: 6.70121016453518
> Time: 0.064 Size: 642560 MB/s: 9.57489013671875
> Time: 12.024 Size: 81601394 MB/s: 6.4721518330628
> Time: 0.111 Size: 828738 MB/s: 7.12023554621516
> Time: 130.331 Size: 731654144 MB/s: 5.353751337939554
> Time: 126.742 Size: 731654144 MB/s: 5.50535549087911
>
> I also profiled this a bit and now there is very little gc going on and
> I can't really easily see how to improve this much further. Small files
> seem to generally be faster, but as you see the 81Mb file lands
> regularly between 6.5 and 7 MB/s. Peaks around 10Mb for small files.
>
> Latest version attached, this was tested with Seaside 2.8.
>
> regards, Göran
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside