RAM-efficient alternative to ZnMultiPartFormDataEntity

Hello,

I'm searching for an alternative implementation of ZnMultiPartFormDataEntity which dumps it's parts  to Filesystem as files saving RAM.

Is there such an addon or it should be developed?


Thanks,
Mike

Hi Mike,

> On 29 Jun 2015, at 04:43, mikefilonov <[hidden email]> wrote:
>
> Hello,
>
> I'm searching for an alternative implementation of ZnMultiPartFormDataEntity
> which dumps it's parts  to Filesystem as files saving RAM.
>
> Is there such an addon or it should be developed?

No, there is no such a thing, and it could be useful, although I am not sure what you have in mind exactly. Please propose something we can discuss.

Sven

> Thanks,
> Mike
>
>
>
> --
> View this message in context: http://forum.world.st/RAM-efficient-alternative-to-ZnMultiPartFormDataEntity-tp4834528.html
> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>

Hi Sven,

Thank you for a prompt reply.

I have in mind something similar to the followin nginx plugin:
http://www.grid.net.ru/nginx/upload.en.html

We could dump entity body directly from socket stream to a file  (buffered reader/writer) and add a header to Request so backend could just have a file name (or FileReference?) instead of a ByteArray.

I think this could help lowering RAM requirements for web-services which accepts files.

Have you seen:

http://www.squeaksource.com/fileupload.html

Yes, that would be doable with the stuff that is already available.

Good luck with it !

Thank you for the link. It is a useful example of how ajax upload can be implemented in Seaside.

However, by looking at the code which works without nginx plugin I see the project does not change  the mechanism of request reading, meaning if you upload 1Gb file you still need 1Gb of RAM to store WAFile rawContent.

mikefilonov wrote

Thank you for the link. It is a useful example of how ajax upload can be implemented in Seaside.

However, by looking at the code which works without nginx plugin I see the project does not change  the mechanism of request reading, meaning if you upload 1Gb file you still need 1Gb of RAM to store WAFile rawContent.

Oh.  I thought NAFileUploadNginx was code to use nginx to store uploaded data directly to disk which would prevent the image from seeing the uploaded data and thereby not change the RAM requirements really at all.  Which was what you were considering using in your second message in the thread.  

On 29/06/15 10:00, mikefilonov wrote:
> Hi Sven,
>
> Thank you for a prompt reply.
>
> I have in mind something similar to the followin nginx plugin:
> http://www.grid.net.ru/nginx/upload.en.html

That is in http://www.squeaksource.com/fileupload

Stephan

On 30/06/15 06:01, mikefilonov wrote:
> Thank you for the link. It is a useful example of how ajax upload can be
> implemented in Seaside.
>
> However, by looking at the code which works without nginx plugin I see the
> project does not change  the mechanism of request reading, meaning if you
> upload 1Gb file you still need 1Gb of RAM to store WAFile rawContent.

Yes, you want the code that uses the nginx plugin.

Stephan

On 01/07/15 23:07, Stephan Eggermont wrote:
> On 29/06/15 10:00, mikefilonov wrote:
>> Hi Sven,
>>
>> Thank you for a prompt reply.
>>
>> I have in mind something similar to the followin nginx plugin:
>> http://www.grid.net.ru/nginx/upload.en.html
>
> That is in http://www.squeaksource.com/fileupload

I should read the whole thread before posting.
Large files in web services provide many more
challenges than ram efficiency. Oh, the enterprise
buses I've crashed...

Stephan