support for streamed files with Zinc adaptor?

> Hi Sven el all
>
> I'm using Zinc-Adaptor and I need to upload files. I remember Comanche adaptor had support for streamed files (they were derived to a temporary file instead load everything in the image).
> While I could configure nginx or apache to do that, I would like to be able to do it  using plain Seaside too.
> It is possible with Zinc-Adaptor?
>
> thanks,
> Esteban_______________________________________________
> seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

> Hi Esteban,
>
> On 19 Jun 2012, at 15:31, Esteban Lorenzano wrote:
>
>> Hi Sven el all
>>
>> I'm using Zinc-Adaptor and I need to upload files. I remember Comanche adaptor had support for streamed files (they were derived to a temporary file instead load everything in the image).
>> While I could configure nginx or apache to do that, I would like to be able to do it  using plain Seaside too.
>> It is possible with Zinc-Adaptor?
>>
>> thanks,
>> Esteban_______________________________________________
>> seaside mailing list
>> [hidden email]
>> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>
> I am (still) confused as to what exactly is expected of a 'streaming' Seaside adaptor.
>
> Zn can do streaming uploads and downloads for clients and servers through ZnStreamingEntity objects.
>
> Setting this up and configuring this depends on what is expected.
>
> Sven
>
> --
> Sven Van Caekenberghe
> http://stfx.eu
> Smalltalk is the Red Pill
>
> _______________________________________________
> seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

>
> On 19 Jun 2012, at 16:28, Esteban Lorenzano wrote:
>
>> most probably I'm not using the right english words :)
>> What I want is a way to not upload to the image a 50m file. With the comanche adaptor, I can set an option and get, instead a WAFile, a WATemporaryFile, which is a reference to an external file (so the multipart moves the file to a local file, and I can use the reference).
>
> OK, I think I understand.
>
> Is there a Seaside image that I can download that contains the comanche adaptor with that functionality ?_______________________________________________
> seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

> I think any image with comanche has it, just install  Seaside-Adaptors-Comanche package and look for references of WATemporaryFile.
>
> On Jun 19, 2012, at 4:42 PM, Sven Van Caekenberghe wrote:
>
>>
>> On 19 Jun 2012, at 16:28, Esteban Lorenzano wrote:
>>
>>> most probably I'm not using the right english words :)
>>> What I want is a way to not upload to the image a 50m file. With the comanche adaptor, I can set an option and get, instead a WAFile, a WATemporaryFile, which is a reference to an external file (so the multipart moves the file to a local file, and I can use the reference).
>>
>> OK, I think I understand.
>>
>> Is there a Seaside image that I can download that contains the comanche adaptor with that functionality ?_______________________________________________
>> seaside mailing list
>> [hidden email]
>> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>
> _______________________________________________
> seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

> Hi Esteban,
>
> On 19 Jun 2012, at 15:31, Esteban Lorenzano wrote:
>
>> Hi Sven el all
>>
>> I'm using Zinc-Adaptor and I need to upload files. I remember Comanche adaptor had support for streamed files (they were derived to a temporary file instead load everything in the image).
>> While I could configure nginx or apache to do that, I would like to be able to do it  using plain Seaside too.
>> It is possible with Zinc-Adaptor?
>>
>> thanks,
>> Esteban_______________________________________________
>> seaside mailing list
>> [hidden email]
>> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>
> I am (still) confused as to what exactly is expected of a 'streaming' Seaside adaptor.

> On Tue, Jun 19, 2012 at 4:12 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>> Hi Esteban,
>>
>> On 19 Jun 2012, at 15:31, Esteban Lorenzano wrote:
>>
>>> Hi Sven el all
>>>
>>> I'm using Zinc-Adaptor and I need to upload files. I remember Comanche adaptor had support for streamed files (they were derived to a temporary file instead load everything in the image).
>>> While I could configure nginx or apache to do that, I would like to be able to do it  using plain Seaside too.
>>> It is possible with Zinc-Adaptor?
>>>
>>> thanks,
>>> Esteban_______________________________________________
>>
>> I am (still) confused as to what exactly is expected of a 'streaming' Seaside adaptor.
>
> There are two different issues, file upload streaming and response streaming.
>
> What I believe is wanted here is file upload streaming. The idea is
> that instead of loading the whole contents of a file upload into
> memory (and copying it several times along the way) you stream the
> contents into a file on disk and just get a "handle" to the file.
> Dealing with multipart mime makes this a bit tricky, you don't want to
> do this for normal upload fields. So you need to hook into your mime
> parser. It's basically the same as mod_upload does [2]. Looking for
> references to WATemporaryFile in an up to date Seaside image [1]
> should get you started. Note that the Comanche implementation relies
> on the fact that the request is dispatched after the headers are
> parsed before the body is fully parsed.

> Hey Philippe,
>
> On 19 Jun 2012, at 20:15, Philippe Marschall wrote:
>
>> On Tue, Jun 19, 2012 at 4:12 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>>> Hi Esteban,
>>>
>>> On 19 Jun 2012, at 15:31, Esteban Lorenzano wrote:
>>>
>>>> Hi Sven el all
>>>>
>>>> I'm using Zinc-Adaptor and I need to upload files. I remember Comanche adaptor had support for streamed files (they were derived to a temporary file instead load everything in the image).
>>>> While I could configure nginx or apache to do that, I would like to be able to do it  using plain Seaside too.
>>>> It is possible with Zinc-Adaptor?
>>>>
>>>> thanks,
>>>> Esteban_______________________________________________
>>>
>>> I am (still) confused as to what exactly is expected of a 'streaming' Seaside adaptor.
>>
>> There are two different issues, file upload streaming and response streaming.
>>
>> What I believe is wanted here is file upload streaming. The idea is
>> that instead of loading the whole contents of a file upload into
>> memory (and copying it several times along the way) you stream the
>> contents into a file on disk and just get a "handle" to the file.
>> Dealing with multipart mime makes this a bit tricky, you don't want to
>> do this for normal upload fields. So you need to hook into your mime
>> parser. It's basically the same as mod_upload does [2]. Looking for
>> references to WATemporaryFile in an up to date Seaside image [1]
>> should get you started. Note that the Comanche implementation relies
>> on the fact that the request is dispatched after the headers are
>> parsed before the body is fully parsed.
>
> Thanks for the excellent, to the point explanation.
> I browsed the code and found what I was looking for.
>
> This won't be easy as it depends on, first a streaming multipart parser (the one currently used is based on your AJP parser that works on a byte array, do you have another one ?),

>> The other thing is response streaming, sending data to the client
>> before the response is fully computed. An example for this is to send
>> the html header when you start rendering the body so that the user
>> agent can start downloading linked resources. If you don't know how
>> long the response is going to be (because it still being build) you'll
>> have to resort to chunking. See WAComboResponse >> #flush to get you
>> started.
>
> Very interesting as well.
> Is this working ? Using which adaptor ?

> On Tue, Jun 19, 2012 at 10:06 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>> Hey Philippe,
>>
>> On 19 Jun 2012, at 20:15, Philippe Marschall wrote:
>>
>>> On Tue, Jun 19, 2012 at 4:12 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>>>> Hi Esteban,
>>>>
>>>> On 19 Jun 2012, at 15:31, Esteban Lorenzano wrote:
>>>>
>>>>> Hi Sven el all
>>>>>
>>>>> I'm using Zinc-Adaptor and I need to upload files. I remember Comanche adaptor had support for streamed files (they were derived to a temporary file instead load everything in the image).
>>>>> While I could configure nginx or apache to do that, I would like to be able to do it  using plain Seaside too.
>>>>> It is possible with Zinc-Adaptor?
>>>>>
>>>>> thanks,
>>>>> Esteban_______________________________________________
>>>>
>>>> I am (still) confused as to what exactly is expected of a 'streaming' Seaside adaptor.
>>>
>>> There are two different issues, file upload streaming and response streaming.
>>>
>>> What I believe is wanted here is file upload streaming. The idea is
>>> that instead of loading the whole contents of a file upload into
>>> memory (and copying it several times along the way) you stream the
>>> contents into a file on disk and just get a "handle" to the file.
>>> Dealing with multipart mime makes this a bit tricky, you don't want to
>>> do this for normal upload fields. So you need to hook into your mime
>>> parser. It's basically the same as mod_upload does [2]. Looking for
>>> references to WATemporaryFile in an up to date Seaside image [1]
>>> should get you started. Note that the Comanche implementation relies
>>> on the fact that the request is dispatched after the headers are
>>> parsed before the body is fully parsed.
>>
>> Thanks for the excellent, to the point explanation.
>> I browsed the code and found what I was looking for.
>>
>> This won't be easy as it depends on, first a streaming multipart parser (the one currently used is based on your AJP parser that works on a byte array, do you have another one ?),
>
> No, as AJP works only behind a front end server you'd probably do this
> is with a module there instead of in the Smalltalk image. So I don't
> have a need for it.
>
>> second in Zn currently only generic byte streams can be seen as streaming entities, not interpreted/parsed ones like multipart, I'll have to think about that.
>
> If you encode the multipart boundary with the request encoding you
> should be able to do the parsing on byte streams, in theory, I haven't
> tested it.