Hi folks!
Ok, this came up once again on IRC - how to do chunkwise uploading and thus continously write to a file and handle LARGE files. I googled and first I found Colin Curtin writing code for this (and also streamed downloading code) back in 2004! Might be worth looking at more closely. I also found Colin Putney's code from aug 2007, which seems to originate from DabbleDB (I guess) - Squeak 3.7. I more or less took Colin's approach, rewrote the low level bit (so that it does not end up doing bytewise searching as I presume Colin's code does) and more or less copied WATempFile as is. The multipartFormFieldsDo: code now ALWAYS uses WATempFile, regardless if it has a file name or not - and regardless of how small they are. This could of course be discussed. But the MAIN THING is this: Upload of 295Mb file (locally) on a box with tinyBenchmarks ('117972350 bytecodes/sec; 4788828 sends/sec') took around 45 seconds. Uploading squeak.image which was 16Mb took 2 seconds or so. CPU load kinda high (profile shows that it is the copying from the buffer into FileStream) but no visible memory consumption. I also noted that this is being recently discussed too - but hey, I had already started coding and wanted to finish it :). regards, Göran _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside StreamedUpload.4.cs.gz (3K) Download Attachment |
Looks interesting. If Giovanni can take care of the Kom side it can
integrate it into the Squeak side of Seaside 2.9. Cheers Philippe 2008/4/7, Göran Krampe <[hidden email]>: > Hi folks! > > Ok, this came up once again on IRC - how to do chunkwise uploading and > thus continously write to a file and handle LARGE files. > > I googled and first I found Colin Curtin writing code for this (and also > streamed downloading code) back in 2004! Might be worth looking at more > closely. I also found Colin Putney's code from aug 2007, which seems to > originate from DabbleDB (I guess) - Squeak 3.7. > > I more or less took Colin's approach, rewrote the low level bit (so that > it does not end up doing bytewise searching as I presume Colin's code > does) and more or less copied WATempFile as is. The multipartFormFieldsDo: > code now ALWAYS uses WATempFile, regardless if it has a file name or not - > and regardless of how small they are. This could of course be discussed. > > But the MAIN THING is this: > > Upload of 295Mb file (locally) on a box with tinyBenchmarks ('117972350 > bytecodes/sec; 4788828 sends/sec') took around 45 seconds. Uploading > squeak.image which was 16Mb took 2 seconds or so. CPU load kinda high > (profile shows that it is the copying from the buffer into FileStream) but > no visible memory consumption. > > I also noted that this is being recently discussed too - but hey, I had > already started coding and wanted to finish it :). > > regards, Göran > _______________________________________________ > seaside mailing list > [hidden email] > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside > > > _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Hi!
> Looks interesting. If Giovanni can take care of the Kom side it can > integrate it into the Squeak side of Seaside 2.9. > > Cheers > Philippe Yeah, the ideal approach is probably: 1. Add new method in SocketStream as a Kom class extension for now and modify the method in MultipartChunk that uses it. This is easy. I can make this happen and talk to Giovanni etc. 2. Add/modify the bits in Seaside 2.9. Then we can eventually move the new method in SocketStream into Squeak, where it belongs. I can extent the file upload test example for this, and add some class comments to show how to use WATempFile etc, thus making this more polished. regards, Göran _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Free forum by Nabble | Edit this page |