Final try for Kom/Seaside file upload tuning

Hi all!

Ok, the little "competition" between me and Janko regarding Swazoo/Aida
vs Kom/Seaside regarding receiving HTTP file uploads has produced some
faster code in both codebases - good!

This post summarizes my last effort, and I really don't know how to
squeeze more juice out of it now, profiling just shows primitives taking
all the time and it looks like it behaves "as it should".

This code was tested in a 3.10.2-7179 image with Seaside installed via
SM. I also used the very latest VM from squeakvm.org, manually compiled
on my Xubuntu box.

I hope Janko will run and post one final test comparing numbers - note
Janko that SocketStream now starts with a 8kb inBuffer and grows it
dynamically to a maximum of 8Mb IIRC.

Note that the Kom/Seaside code uses the base SocketStream included in
Squeak. This changeset modifies mainly SocketStream. I also tried using
AsyncFile for doing file writing asynchronously - and it works fine -
but it gave no visible performance advantage. The main change from last
changeset is the grow/move behavior of the inBuffer - now it should do
much less copying in memory.

Two things still make me itch:

1. My 81 Mb test file can be uploaded in 5-10 Mb/sec. Mostly 9-10, but
also often around 5-6. No idea why! And no, there are only very few full
gcs going on, so that is not it.

2. My 731 Mb test file uploads considerably slower - but it seems to me
that SocketStream behaves quite in the same way - typically it grows the
inBuffer dynamically to 8Mb and then it chews the data in 2-4 Mb chunks.
Can it be that writing at the end of a large file is slower? Are there
file seeks being done or something?

If you want to test this yourself, file into Seaside 2.8, use the upload
file test in Seaside (http://localhost:8080/seaside/tests/alltests) and
then check Transcript for a line showing file size and Mb/sec.

regards, Göran

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

2008/8/5 Göran Krampe <[hidden email]>: Could you upgrade the SqueakSource package of Kom?

If there is enough interest I can backport the upload streaming
changes from Seaside 2.9 to Seaside 2.8.

Thanks
Philippe

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Hi!

Philippe Marschall wrote:
> Could you upgrade the SqueakSource package of Kom?
>
> If there is enough interest I can backport the upload streaming
> changes from Seaside 2.9 to Seaside 2.8.

I can and will, but not today :)

regards, Göran
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

I think everybody wants it on 2.8 :-) Thank you.

----- Original Message -----
   From: Philippe Marschall <[hidden email]>
   To: Seaside - general discussion <[hidden email]>
   Sent: 08-08-05 22:42:17
   Subject: Re: [Seaside] Final try for Kom/Seaside file upload tuning

  2008/8/5 Göran Krampe <[hidden email]>: Could you upgrade the SqueakSource package of Kom?

If there is enough interest I can backport the upload streaming
changes from Seaside 2.9 to Seaside 2.8.

Thanks
Philippe

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

I think everybody wants it on 2.8 :-) Thank you.

----- Original Message -----
   From: Philippe Marschall <[hidden email]>
   To: Seaside - general discussion <[hidden email]>
   Sent: 08-08-05 22:42:17
   Subject: Re: [Seaside] Final try for Kom/Seaside file upload tuning

  2008/8/5 Göran Krampe <[hidden email]>: Could you upgrade the SqueakSource package of Kom?

If there is enough interest I can backport the upload streaming
changes from Seaside 2.9 to Seaside 2.8.

Thanks
Philippe

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

2008/8/5 Göran Krampe <[hidden email]>:
>
> If you want to test this yourself, file into Seaside 2.8, use the upload
> file test in Seaside (http://localhost:8080/seaside/tests/alltests) and
> then check Transcript for a line showing file size and Mb/sec.
>

Hi Goran,

I'm having trouble integrating this patch in the main KomHttpServer codeline.
It seems to me that you took the opportunity to do some refactoring to
SocketStream
as well. Can you please split the patch into two separate patches, one
for Kom+the
relevant extensions to SocketStream, and one with the other
performance changes to SocketStream?

        Ciao,

                Giovanni
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside