zinc streaming post

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

zinc streaming post

Camillo Bruni-3
we're having our weekly git hacking session and are working on the http protocol.

The smart git protocol simply runs over http, nothing too fancy once you found the
documentation (which is the hardest part).

Now we want to post a request and wanted to directly stream it to the server, but
we only figure out on how to do it using an intermediate string. The code looks
something like this:


        requestContents := self createGitRequestEntity.
        client := ZnClient new
                url: 'https://github.com/pharo-project/phar-core';
                entity: (requestContents);
                post.

Now I would like to do something like this.


        postStream := ZnClient new
                url: 'https://github.com/pharo-project/phar-core';
                postStream.
        self writeGitRequestOn: postStream


How can I do that in Zn? I already skimmed the documentation but it only shows
the example of files, where we copy over from a fully existing source.

best
cami
Reply | Threaded
Open this post in threaded view
|

Re: zinc streaming post

Sven Van Caekenberghe
Hi Camillo !

On 26 Feb 2013, at 15:13, Camillo Bruni <[hidden email]> wrote:

> we're having our weekly git hacking session and are working on the http protocol.

Great !

> The smart git protocol simply runs over http, nothing too fancy once you found the
> documentation (which is the hardest part).
>
> Now we want to post a request and wanted to directly stream it to the server, but
> we only figure out on how to do it using an intermediate string. The code looks
> something like this:
>
>
> requestContents := self createGitRequestEntity.
> client := ZnClient new
> url: 'https://github.com/pharo-project/phar-core';
> entity: (requestContents);
> post.
>
> Now I would like to do something like this.
>
>
> postStream := ZnClient new
> url: 'https://github.com/pharo-project/phar-core';
> postStream.
> self writeGitRequestOn: postStream
>
>
> How can I do that in Zn? I already skimmed the documentation but it only shows
> the example of files, where we copy over from a fully existing source.
>
> best
> cami
OK.

When you do a POST, at one point the entity is sent #writeOn: so that it gets transferred to the wire. The special thing about a ZnStreamingEntity is that is wraps an open stream and then in its #writeOn: copies from that stream to the wire. That is basically it.

In the case of a file upload, you open the file stream, put it in a ZnStreamingEntity, set the content type and length (length is almost always required) and your good.

Where does your input stream come from in this particular case ?

If you have your data already in memory, it makes no sense to want to do streaming.
Have a look at #entityStreamContents: which is half right.

Sven

--
Powering the T3 Platform - http://t3-platform.net
Sven Van Caekenberghe - mailto:[hidden email]
Beta Nine - software engineering - http://www.beta9.be


smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: zinc streaming post

Camillo Bruni-3

On 2013-02-26, at 15:24, Sven Van Caekenberghe <[hidden email]> wrote:

> Hi Camillo !
>
> On 26 Feb 2013, at 15:13, Camillo Bruni <[hidden email]> wrote:
>
>> we're having our weekly git hacking session and are working on the http protocol.
>
> Great !
>
>> The smart git protocol simply runs over http, nothing too fancy once you found the
>> documentation (which is the hardest part).
>>
>> Now we want to post a request and wanted to directly stream it to the server, but
>> we only figure out on how to do it using an intermediate string. The code looks
>> something like this:
>>
>>
>> requestContents := self createGitRequestEntity.
>> client := ZnClient new
>> url: 'https://github.com/pharo-project/phar-core';
>> entity: (requestContents);
>> post.
>>
>> Now I would like to do something like this.
>>
>>
>> postStream := ZnClient new
>> url: 'https://github.com/pharo-project/phar-core';
>> postStream.
>> self writeGitRequestOn: postStream
>>
>>
>> How can I do that in Zn? I already skimmed the documentation but it only shows
>> the example of files, where we copy over from a fully existing source.
>>
>> best
>> cami
>
> OK.
>
> When you do a POST, at one point the entity is sent #writeOn: so that it gets transferred to the wire. The special thing about a ZnStreamingEntity is that is wraps an open stream and then in its #writeOn: copies from that stream to the wire. That is basically it.
>
> In the case of a file upload, you open the file stream, put it in a ZnStreamingEntity, set the content type and length (length is almost always required) and your good.
>
> Where does your input stream come from in this particular case ?

right now we generate the git request on the fly, so it is not present as a whole
data structure at the tim we send the request.

Later we will have more issue when we upload stuff to git. We will send a lot of
data but we generate it on the fly piece wise. So here it makes sense not to generate
an intermediate buffer to store the full post request.

> If you have your data already in memory, it makes no sense to want to do streaming.
> Have a look at #entityStreamContents: which is half right.

ok, looks more or less what we want ;), yet you fully buffer the request first, which I
would like to avoid in the future.
Reply | Threaded
Open this post in threaded view
|

Re: zinc streaming post

Sven Van Caekenberghe-2

On 26 Feb 2013, at 15:31, Camillo Bruni <[hidden email]> wrote:

> On 2013-02-26, at 15:24, Sven Van Caekenberghe <[hidden email]> wrote:
>
>> Hi Camillo !
>>
>> On 26 Feb 2013, at 15:13, Camillo Bruni <[hidden email]> wrote:
>>
>>> we're having our weekly git hacking session and are working on the http protocol.
>>
>> Great !
>>
>>> The smart git protocol simply runs over http, nothing too fancy once you found the
>>> documentation (which is the hardest part).
>>>
>>> Now we want to post a request and wanted to directly stream it to the server, but
>>> we only figure out on how to do it using an intermediate string. The code looks
>>> something like this:
>>>
>>>
>>> requestContents := self createGitRequestEntity.
>>> client := ZnClient new
>>> url: 'https://github.com/pharo-project/phar-core';
>>> entity: (requestContents);
>>> post.
>>>
>>> Now I would like to do something like this.
>>>
>>>
>>> postStream := ZnClient new
>>> url: 'https://github.com/pharo-project/phar-core';
>>> postStream.
>>> self writeGitRequestOn: postStream
>>>
>>>
>>> How can I do that in Zn? I already skimmed the documentation but it only shows
>>> the example of files, where we copy over from a fully existing source.
>>>
>>> best
>>> cami
>>
>> OK.
>>
>> When you do a POST, at one point the entity is sent #writeOn: so that it gets transferred to the wire. The special thing about a ZnStreamingEntity is that is wraps an open stream and then in its #writeOn: copies from that stream to the wire. That is basically it.
>>
>> In the case of a file upload, you open the file stream, put it in a ZnStreamingEntity, set the content type and length (length is almost always required) and your good.
>>
>> Where does your input stream come from in this particular case ?
>
> right now we generate the git request on the fly, so it is not present as a whole
> data structure at the tim we send the request.
>
> Later we will have more issue when we upload stuff to git. We will send a lot of
> data but we generate it on the fly piece wise. So here it makes sense not to generate
> an intermediate buffer to store the full post request.

But it is simple: I assume you already have a stream that can be read for the data ? You know the mime-type and length ? Just take these 3 elements and make a ZnStreamingEntity and use/POST it. When the time comes, a piece wise copy from your stream to the HTTP wire will be done, directly.

>> If you have your data already in memory, it makes no sense to want to do streaming.
>> Have a look at #entityStreamContents: which is half right.
>
> ok, looks more or less what we want ;), yet you fully buffer the request first, which I
> would like to avoid in the future.

Of course, that example is not good(™) ;-)

Sven