Streaming of large files

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Streaming of large files

Janko Mivšek
Hello all,

I just like to inform you what I'm currently doing and ask you for comments.

Streaming of large files, both for upload and for download is starting
to be really needed, so I started to look at Swazoo to add streaming
support.

Streaming means that content of (say) file is not copied first into
memory then send forward, but it is streamed directly from source to
destination. For file upload that means that a file is streamed directly
from TCP socket to a destination file, without any copying of entire
file in Smalltlak memory as it happens now. Currently such a file is
copied about 4 times!

I'll make input streaming of HTTRequests by deferring parsing of POST
data (where file upload resides) until a web framework provides a output
stream for contents. Then a POST data will be streamed directly to that
output stream, which can be just a file writeStream.

That is an idea so far and currently I'm changing a POST parser to be
more stream friendly (HTTPPost multipartDataFrom: partFromBytes: ).

Do you also fell that streaming can be important for you. Any comments,
suggestions, wishes appreciated.

Best regards
Janko

--
Janko Miv?ek
AIDA/Web
Smalltalk Web Application Server
http://www.aidaweb.si