Writing into stream from another stream

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Writing into stream from another stream

Denis Kudriashov
Hi.

Do we have simple method to write data between streams?

I want something like:

writeStream nextPutAllFrom: readStream 

I always used loop for this. But maybe we have something suitable. Otherwise lets introduce it.

Best regards,
Denis
Reply | Threaded
Open this post in threaded view
|

Re: Writing into stream from another stream

Sven Van Caekenberghe-2
ZnUtils class>>#streamFrom:to: ?

> On 29 Mar 2018, at 11:22, Denis Kudriashov <[hidden email]> wrote:
>
> Hi.
>
> Do we have simple method to write data between streams?
>
> I want something like:
>
> writeStream nextPutAllFrom: readStream
>
> I always used loop for this. But maybe we have something suitable. Otherwise lets introduce it.
>
> Best regards,
> Denis


Reply | Threaded
Open this post in threaded view
|

Re: Writing into stream from another stream

Peter Uhnak
I use

writeStream nextPutAll: readStream contents

but it is quite silly...to convert stream -> string -> stream

On Thu, Mar 29, 2018 at 11:45 AM, Sven Van Caekenberghe <[hidden email]> wrote:
ZnUtils class>>#streamFrom:to: ?

> On 29 Mar 2018, at 11:22, Denis Kudriashov <[hidden email]> wrote:
>
> Hi.
>
> Do we have simple method to write data between streams?
>
> I want something like:
>
> writeStream nextPutAllFrom: readStream
>
> I always used loop for this. But maybe we have something suitable. Otherwise lets introduce it.
>
> Best regards,
> Denis



Reply | Threaded
Open this post in threaded view
|

Re: Writing into stream from another stream

Denis Kudriashov
In reply to this post by Sven Van Caekenberghe-2

2018-03-29 11:45 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:
ZnUtils class>>#streamFrom:to: ?

Yes, it gives feature which I want. But it's too Java style. 
What you think to move it to stream itself? And with what message name?
 

> On 29 Mar 2018, at 11:22, Denis Kudriashov <[hidden email]> wrote:
>
> Hi.
>
> Do we have simple method to write data between streams?
>
> I want something like:
>
> writeStream nextPutAllFrom: readStream
>
> I always used loop for this. But maybe we have something suitable. Otherwise lets introduce it.
>
> Best regards,
> Denis



Reply | Threaded
Open this post in threaded view
|

Re: Writing into stream from another stream

Sven Van Caekenberghe-2


> On 29 Mar 2018, at 14:34, Denis Kudriashov <[hidden email]> wrote:
>
>
> 2018-03-29 11:45 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:
> ZnUtils class>>#streamFrom:to: ?
>
> Yes, it gives feature which I want. But it's too Java style.
> What you think to move it to stream itself? And with what message name?

This was written in a specific context, to be efficient. I don't know if it is general enough for broad use. Yes it could be more OO to not make it a utility method, but on the other hand, where would you put it ? I am hesitant to make the Stream API bigger ;-)

> > On 29 Mar 2018, at 11:22, Denis Kudriashov <[hidden email]> wrote:
> >
> > Hi.
> >
> > Do we have simple method to write data between streams?
> >
> > I want something like:
> >
> > writeStream nextPutAllFrom: readStream
> >
> > I always used loop for this. But maybe we have something suitable. Otherwise lets introduce it.
> >
> > Best regards,
> > Denis
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Writing into stream from another stream

Denis Kudriashov
2018-03-29 14:40 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:


> On 29 Mar 2018, at 14:34, Denis Kudriashov <[hidden email]> wrote:
>
>
> 2018-03-29 11:45 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:
> ZnUtils class>>#streamFrom:to: ?
>
> Yes, it gives feature which I want. But it's too Java style.
> What you think to move it to stream itself? And with what message name?

This was written in a specific context, to be efficient. I don't know if it is general enough for broad use. Yes it could be more OO to not make it a utility method, but on the other hand, where would you put it ?

In read streams we already have:

readInto: aCollection startingAt: startIndex count: n

So I would put new method in ReadStream (#readIntoStream: ?).
 
I am hesitant to make the Stream API bigger ;-)

We already have this method in Utils class. So it is here but in strange hidden place. 


> > On 29 Mar 2018, at 11:22, Denis Kudriashov <[hidden email]> wrote:
> >
> > Hi.
> >
> > Do we have simple method to write data between streams?
> >
> > I want something like:
> >
> > writeStream nextPutAllFrom: readStream
> >
> > I always used loop for this. But maybe we have something suitable. Otherwise lets introduce it.
> >
> > Best regards,
> > Denis
>
>
>



Reply | Threaded
Open this post in threaded view
|

Re: Writing into stream from another stream

Nicolas Cellier
There are two different questions:
- expressiveness
- efficiency

Real efficiency of transfer (without any intermediate copy) is only guaranteed in very rare cases
(if the target collection of write stream is same species as source collection of read stream, and there are no transform in any of the intermediate wrapper streams, then a direct bulk transfer of the form  targetCollection replaceFrom:to:with: sourceCollection startingAt: is possible)
I'm not at all convinced that we should focus too much on such optimization.

The fact that we have at least two choices, let the readStream drive the operation, or let the writeStream do it, is a smell.
It sounds like we could provide the two:
    aReadStream writeContentsTo: aWriteStream.
    aWriteStream readContentsFrom: aReadStream.
... and then let the programmer decide (maybe there is a more optimal path).
... or provide a way to negotiate the transfer (size of chunked buffers, etc...)

Unless there is a more symetrical solution.
It's a long time since I did not check Xtreams, but wasn't there a reified Buffer that could/would be the effective actor of such transfer?

Nicolas

2018-03-29 15:02 GMT+02:00 Denis Kudriashov <[hidden email]>:
2018-03-29 14:40 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:


> On 29 Mar 2018, at 14:34, Denis Kudriashov <[hidden email]> wrote:
>
>
> 2018-03-29 11:45 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:
> ZnUtils class>>#streamFrom:to: ?
>
> Yes, it gives feature which I want. But it's too Java style.
> What you think to move it to stream itself? And with what message name?

This was written in a specific context, to be efficient. I don't know if it is general enough for broad use. Yes it could be more OO to not make it a utility method, but on the other hand, where would you put it ?

In read streams we already have:

readInto: aCollection startingAt: startIndex count: n

So I would put new method in ReadStream (#readIntoStream: ?).
 
I am hesitant to make the Stream API bigger ;-)

We already have this method in Utils class. So it is here but in strange hidden place. 


> > On 29 Mar 2018, at 11:22, Denis Kudriashov <[hidden email]> wrote:
> >
> > Hi.
> >
> > Do we have simple method to write data between streams?
> >
> > I want something like:
> >
> > writeStream nextPutAllFrom: readStream
> >
> > I always used loop for this. But maybe we have something suitable. Otherwise lets introduce it.
> >
> > Best regards,
> > Denis
>
>
>




Reply | Threaded
Open this post in threaded view
|

Re: Writing into stream from another stream

Stephane Ducasse-3
In reply to this post by Denis Kudriashov
I agree we should not have Utils class.


On Thu, Mar 29, 2018 at 3:02 PM, Denis Kudriashov <[hidden email]> wrote:

> 2018-03-29 14:40 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:
>>
>>
>>
>> > On 29 Mar 2018, at 14:34, Denis Kudriashov <[hidden email]> wrote:
>> >
>> >
>> > 2018-03-29 11:45 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:
>> > ZnUtils class>>#streamFrom:to: ?
>> >
>> > Yes, it gives feature which I want. But it's too Java style.
>> > What you think to move it to stream itself? And with what message name?
>>
>> This was written in a specific context, to be efficient. I don't know if
>> it is general enough for broad use. Yes it could be more OO to not make it a
>> utility method, but on the other hand, where would you put it ?
>
>
> In read streams we already have:
>
>
> readInto: aCollection startingAt: startIndex count: n
>
>
> So I would put new method in ReadStream (#readIntoStream: ?).
>
>>
>> I am hesitant to make the Stream API bigger ;-)
>
>
> We already have this method in Utils class. So it is here but in strange
> hidden place.
>
>>
>> > > On 29 Mar 2018, at 11:22, Denis Kudriashov <[hidden email]>
>> > > wrote:
>> > >
>> > > Hi.
>> > >
>> > > Do we have simple method to write data between streams?
>> > >
>> > > I want something like:
>> > >
>> > > writeStream nextPutAllFrom: readStream
>> > >
>> > > I always used loop for this. But maybe we have something suitable.
>> > > Otherwise lets introduce it.
>> > >
>> > > Best regards,
>> > > Denis
>> >
>> >
>> >
>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Writing into stream from another stream

Stephane Ducasse-3
In reply to this post by Nicolas Cellier
Hi nicolas

I like your proposal.

Stef

On Thu, Mar 29, 2018 at 5:44 PM, Nicolas Cellier
<[hidden email]> wrote:

> There are two different questions:
> - expressiveness
> - efficiency
>
> Real efficiency of transfer (without any intermediate copy) is only
> guaranteed in very rare cases
> (if the target collection of write stream is same species as source
> collection of read stream, and there are no transform in any of the
> intermediate wrapper streams, then a direct bulk transfer of the form
> targetCollection replaceFrom:to:with: sourceCollection startingAt: is
> possible)
> I'm not at all convinced that we should focus too much on such optimization.
>
> The fact that we have at least two choices, let the readStream drive the
> operation, or let the writeStream do it, is a smell.
> It sounds like we could provide the two:
>     aReadStream writeContentsTo: aWriteStream.
>     aWriteStream readContentsFrom: aReadStream.
> ... and then let the programmer decide (maybe there is a more optimal path).
> ... or provide a way to negotiate the transfer (size of chunked buffers,
> etc...)
>
> Unless there is a more symetrical solution.
> It's a long time since I did not check Xtreams, but wasn't there a reified
> Buffer that could/would be the effective actor of such transfer?
>
> Nicolas
>
>
> 2018-03-29 15:02 GMT+02:00 Denis Kudriashov <[hidden email]>:
>>
>> 2018-03-29 14:40 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:
>>>
>>>
>>>
>>> > On 29 Mar 2018, at 14:34, Denis Kudriashov <[hidden email]>
>>> > wrote:
>>> >
>>> >
>>> > 2018-03-29 11:45 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:
>>> > ZnUtils class>>#streamFrom:to: ?
>>> >
>>> > Yes, it gives feature which I want. But it's too Java style.
>>> > What you think to move it to stream itself? And with what message name?
>>>
>>> This was written in a specific context, to be efficient. I don't know if
>>> it is general enough for broad use. Yes it could be more OO to not make it a
>>> utility method, but on the other hand, where would you put it ?
>>
>>
>> In read streams we already have:
>>
>>
>> readInto: aCollection startingAt: startIndex count: n
>>
>>
>> So I would put new method in ReadStream (#readIntoStream: ?).
>>
>>>
>>> I am hesitant to make the Stream API bigger ;-)
>>
>>
>> We already have this method in Utils class. So it is here but in strange
>> hidden place.
>>
>>>
>>> > > On 29 Mar 2018, at 11:22, Denis Kudriashov <[hidden email]>
>>> > > wrote:
>>> > >
>>> > > Hi.
>>> > >
>>> > > Do we have simple method to write data between streams?
>>> > >
>>> > > I want something like:
>>> > >
>>> > > writeStream nextPutAllFrom: readStream
>>> > >
>>> > > I always used loop for this. But maybe we have something suitable.
>>> > > Otherwise lets introduce it.
>>> > >
>>> > > Best regards,
>>> > > Denis
>>> >
>>> >
>>> >
>>>
>>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Writing into stream from another stream

Sven Van Caekenberghe-2
In reply to this post by Denis Kudriashov


> On 29 Mar 2018, at 15:02, Denis Kudriashov <[hidden email]> wrote:
>
> 2018-03-29 14:40 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:
>
>
> > On 29 Mar 2018, at 14:34, Denis Kudriashov <[hidden email]> wrote:
> >
> >
> > 2018-03-29 11:45 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:
> > ZnUtils class>>#streamFrom:to: ?
> >
> > Yes, it gives feature which I want. But it's too Java style.
> > What you think to move it to stream itself? And with what message name?
>
> This was written in a specific context, to be efficient. I don't know if it is general enough for broad use. Yes it could be more OO to not make it a utility method, but on the other hand, where would you put it ?
>
> In read streams we already have:
>
> readInto: aCollection startingAt: startIndex count: n
>
> So I would put new method in ReadStream (#readIntoStream: ?).
>  
> I am hesitant to make the Stream API bigger ;-)
>
> We already have this method in Utils class. So it is here but in strange hidden place.

Not all streams inherit from ReadStream, far from it. So if you add it there, you add it to 'the stream API' that every other stream-like class has to implement. Hence it is a burden, for limited gain.

I also do not specifically like Util classes, but this is a more special case. The operation belongs to both parties involved. You could define a StreamCopier to do the job.

The way it is implemented now, it does not extend the stream API, that is worth something too.

Anyway, we are just discussing ideas.

> > > On 29 Mar 2018, at 11:22, Denis Kudriashov <[hidden email]> wrote:
> > >
> > > Hi.
> > >
> > > Do we have simple method to write data between streams?
> > >
> > > I want something like:
> > >
> > > writeStream nextPutAllFrom: readStream
> > >
> > > I always used loop for this. But maybe we have something suitable. Otherwise lets introduce it.
> > >
> > > Best regards,
> > > Denis
> >
> >
> >


Reply | Threaded
Open this post in threaded view
|

Re: Writing into stream from another stream

Denis Kudriashov
In reply to this post by Nicolas Cellier
Hi Nicolas

2018-03-29 17:44 GMT+02:00 Nicolas Cellier <[hidden email]>:
There are two different questions:
- expressiveness
- efficiency

Real efficiency of transfer (without any intermediate copy) is only guaranteed in very rare cases
(if the target collection of write stream is same species as source collection of read stream, and there are no transform in any of the intermediate wrapper streams, then a direct bulk transfer of the form  targetCollection replaceFrom:to:with: sourceCollection startingAt: is possible)
I'm not at all convinced that we should focus too much on such optimization.

The fact that we have at least two choices, let the readStream drive the operation, or let the writeStream do it, is a smell.
It sounds like we could provide the two:
    aReadStream writeContentsTo: aWriteStream.
    aWriteStream readContentsFrom: aReadStream.
... and then let the programmer decide (maybe there is a more optimal path).
... or provide a way to negotiate the transfer (size of chunked buffers, etc...)

I like this idea.
 

Unless there is a more symetrical solution.
It's a long time since I did not check Xtreams, but wasn't there a reified Buffer that could/would be the effective actor of such transfer?

Xtreams provides nice API to "pipe" multiple streams.  But as I remember the Buffer was internal detail hidden from your. Idea was to reuse internal stream arrays to minimize garbage (or something like this).
 


Nicolas


2018-03-29 15:02 GMT+02:00 Denis Kudriashov <[hidden email]>:
2018-03-29 14:40 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:


> On 29 Mar 2018, at 14:34, Denis Kudriashov <[hidden email]> wrote:
>
>
> 2018-03-29 11:45 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:
> ZnUtils class>>#streamFrom:to: ?
>
> Yes, it gives feature which I want. But it's too Java style.
> What you think to move it to stream itself? And with what message name?

This was written in a specific context, to be efficient. I don't know if it is general enough for broad use. Yes it could be more OO to not make it a utility method, but on the other hand, where would you put it ?

In read streams we already have:

readInto: aCollection startingAt: startIndex count: n

So I would put new method in ReadStream (#readIntoStream: ?).
 
I am hesitant to make the Stream API bigger ;-)

We already have this method in Utils class. So it is here but in strange hidden place. 


> > On 29 Mar 2018, at 11:22, Denis Kudriashov <[hidden email]> wrote:
> >
> > Hi.
> >
> > Do we have simple method to write data between streams?
> >
> > I want something like:
> >
> > writeStream nextPutAllFrom: readStream
> >
> > I always used loop for this. But maybe we have something suitable. Otherwise lets introduce it.
> >
> > Best regards,
> > Denis
>
>
>