Smalltalk › Cincom › VisualWorks

Xtreams compressing

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

7 messages Options

Karsten Kusche

Xtreams compressing

Hi List,

i’ve just had a look at Xtreams because i wanted to read files from disk and send them over the network. I thought Xtreams would be quite handy in that scenario because it allows for transforming the data as it is read or written.

A closer look at the #compressing mechanism however showed that this is only useful if you want to read the decompressed data of a compressed stream or if you want to write compressed data of an uncompressed stream. In my case i’d want the opposite: reading compressed data from a uncompressed stream. I’ve got an uncompressed file and would like to read gzip-compressed bytes from that file, so that i can hand the read-stream directly to the request.

As i see it, i’d have to somehow get my hands on the writestream of the request, send #compressing to it and then write the uncompressed read-stream to it.

Am I getting that wrong or are there other ways of achieving that?

Kind Regards

Karsten

Karsten Kusche - Dipl. Inf. (FH) - [hidden email]

Georg Heeg eK - Köthen

Handelsregister: Amtsgericht Dortmund A 12812

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Michael Lucas-Smith-2

Re: Xtreams compressing

Hi Karsten,

Martin and I voted to leave out the reverse variants from the package because we felt they weren’t that common and if someone needed them, they would be trivial to implement. I’d recommend you implement the reverse read and write streams for compression.

Cheers,

Michael

On 24 Oct 2014, at 3:40 am, Karsten Kusche <[hidden email]> wrote:

Hi List,

i’ve just had a look at Xtreams because i wanted to read files from disk and send them over the network. I thought Xtreams would be quite handy in that scenario because it allows for transforming the data as it is read or written.
A closer look at the #compressing mechanism however showed that this is only useful if you want to read the decompressed data of a compressed stream or if you want to write compressed data of an uncompressed stream. In my case i’d want the opposite: reading compressed data from a uncompressed stream. I’ve got an uncompressed file and would like to read gzip-compressed bytes from that file, so that i can hand the read-stream directly to the request.
As i see it, i’d have to somehow get my hands on the writestream of the request, send #compressing to it and then write the uncompressed read-stream to it.

Am I getting that wrong or are there other ways of achieving that?

Kind Regards
Karsten

--
Karsten Kusche - Dipl. Inf. (FH) - [hidden email]
Georg Heeg eK - Köthen
Handelsregister: Amtsgericht Dortmund A 12812

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Karsten Kusche

Re: Xtreams compressing

Hi Michael,

it looked indeed more complicated than it was at the end. It’s basically a matter of initializing the compression instvar for deflate instead of inflate.

A bigger problem though is the naming. The #compressing method is already taken, even though it actually performs the decompression.

The same problem applies to #encoding:. The assumption here is that it’s always sent to a ReadStream of bytes and then produces Characters out of it. That’s actually decoding the ByteArray. Encoding would take Characters and produce Bytes. As far as i can tell, this direction is also not implemented at the moment. If i understand the classes in the Transformation package correctly, the same problem may apply to all of those classes.

It may not be the best idea to implement both ways for all kinds of transformations for reading and writing because then you’d always end up with 4 classes. I wonder if there’s some way to reuse the implementations of the write-streams as read-streams and vise versa.

Something like:

ReadStream>>encodingToBytes: anEncoding

^EncodeWriteStream forReading on: aWriteStream encoding: anEncoding

Not sure if that’s feasible but would allow for using the already existing classes in the opposite direction.

Kind regards

Karsten

Karsten Kusche - Dipl. Inf. (FH) - [hidden email]

Georg Heeg eK - Köthen

Handelsregister: Amtsgericht Dortmund A 12812

Am Freitag, 24. Oktober 2014 um 01:04 schrieb Michael Lucas-Smith:

Hi Karsten,

Martin and I voted to leave out the reverse variants from the package because we felt they weren’t that common and if someone needed them, they would be trivial to implement. I’d recommend you implement the reverse read and write streams for compression.

Cheers,
Michael

On 24 Oct 2014, at 3:40 am, Karsten Kusche <[hidden email]> wrote:

Hi List,

i’ve just had a look at Xtreams because i wanted to read files from disk and send them over the network. I thought Xtreams would be quite handy in that scenario because it allows for transforming the data as it is read or written.
A closer look at the #compressing mechanism however showed that this is only useful if you want to read the decompressed data of a compressed stream or if you want to write compressed data of an uncompressed stream. In my case i’d want the opposite: reading compressed data from a uncompressed stream. I’ve got an uncompressed file and would like to read gzip-compressed bytes from that file, so that i can hand the read-stream directly to the request.
As i see it, i’d have to somehow get my hands on the writestream of the request, send #compressing to it and then write the uncompressed read-stream to it.

Am I getting that wrong or are there other ways of achieving that?

Kind Regards
Karsten

--
Karsten Kusche - Dipl. Inf. (FH) - [hidden email]
Georg Heeg eK - Köthen
Handelsregister: Amtsgericht Dortmund A 12812

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Karsten Kusche

Re: Xtreams compressing

Thinking about the problem a bit more, i think the design of the Compression-Classes really helps to solve the problem. It’s possible to change the CompressReadStream to actually compress by implementing a #compress method that reinitializes the compression-instvar. The following code is then possible:

((('Hello World' asByteArrayEncoding:#utf8) reading compressing compress rest) compressing encoding: #utf8) rest

If the same design was implemented in the encode-classes, too the first part of the code could be changed to:

('Hello World' reading encoding: #utf8) toBytes compressing compress rest ...etc..

Instead of sending #encoding: and then #toBytes it would probably be a lot easier to understand if the methods #encodeToBytes: or #encodeFromBytes: were introduced (or even better #encoding: and #decoding:).

Kind Regards

Karsten

Karsten Kusche - Dipl. Inf. (FH) - [hidden email]

Georg Heeg eK - Köthen

Handelsregister: Amtsgericht Dortmund A 12812

Am Freitag, 24. Oktober 2014 um 09:28 schrieb Karsten Kusche:

Hi Michael,

it looked indeed more complicated than it was at the end. It’s basically a matter of initializing the compression instvar for deflate instead of inflate.
A bigger problem though is the naming. The #compressing method is already taken, even though it actually performs the decompression.

The same problem applies to #encoding:. The assumption here is that it’s always sent to a ReadStream of bytes and then produces Characters out of it. That’s actually decoding the ByteArray. Encoding would take Characters and produce Bytes. As far as i can tell, this direction is also not implemented at the moment. If i understand the classes in the Transformation package correctly, the same problem may apply to all of those classes.

It may not be the best idea to implement both ways for all kinds of transformations for reading and writing because then you’d always end up with 4 classes. I wonder if there’s some way to reuse the implementations of the write-streams as read-streams and vise versa.
Something like:

ReadStream>>encodingToBytes: anEncoding
^EncodeWriteStream forReading on: aWriteStream encoding: anEncoding

Not sure if that’s feasible but would allow for using the already existing classes in the opposite direction.

Kind regards
Karsten

--
Karsten Kusche - Dipl. Inf. (FH) - [hidden email]
Georg Heeg eK - Köthen
Handelsregister: Amtsgericht Dortmund A 12812

Am Freitag, 24. Oktober 2014 um 01:04 schrieb Michael Lucas-Smith:

Hi Karsten,

Martin and I voted to leave out the reverse variants from the package because we felt they weren’t that common and if someone needed them, they would be trivial to implement. I’d recommend you implement the reverse read and write streams for compression.

Cheers,
Michael

On 24 Oct 2014, at 3:40 am, Karsten Kusche <[hidden email]> wrote:

Hi List,

i’ve just had a look at Xtreams because i wanted to read files from disk and send them over the network. I thought Xtreams would be quite handy in that scenario because it allows for transforming the data as it is read or written.
A closer look at the #compressing mechanism however showed that this is only useful if you want to read the decompressed data of a compressed stream or if you want to write compressed data of an uncompressed stream. In my case i’d want the opposite: reading compressed data from a uncompressed stream. I’ve got an uncompressed file and would like to read gzip-compressed bytes from that file, so that i can hand the read-stream directly to the request.
As i see it, i’d have to somehow get my hands on the writestream of the request, send #compressing to it and then write the uncompressed read-stream to it.

Am I getting that wrong or are there other ways of achieving that?

Kind Regards
Karsten

--
Karsten Kusche - Dipl. Inf. (FH) - [hidden email]
Georg Heeg eK - Köthen
Handelsregister: Amtsgericht Dortmund A 12812

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

mkobetic

Re: Xtreams compressing

In reply to this post by Karsten Kusche

The problem with making the initialization a two step process is that you lose control over when the second stage happens and it will open up the possibility of invoking the second stage later, when the stream has already been used in the other mode. The classic example is the #binary/#text message on classic streams. Invoking those half way through the stream simply cannot work reasonably in general case and makes for bad API IMO.

My inclination would be to not try to fit all possible use-cases into the -ing pattern and just resolve to using suitable class methods on the corresponding stream class for the ones that are less common, e.g.:

(CompressReadStream on: aReadStream compress: aBoolean) ...

or even just

(CompressReadStream on: aReadStream compression: aCompression) ...

The later would lend itself to using other compression algorithms or implementations nicely as well.
This form would break the stream stack creation flow, but maybe it's good to highlight that something unusual is happening here.

Also note that not all the transform streams are so easily reversible. IIRC the encode streams are written for the specific direction, implementing the reverse would be completely different code (we had that earlier, the code is probably still somewhere in Store). The compress streams benefit from the fact that zlib hides that complexity and provides a uniform API in both directions. Anyway I suspect that implementing reverse transformation for most/all streams would likely a be a lot of work (and subsequent maintenance) for benefit of rare use-cases.

Cheers!

Martin

"Karsten Kusche"<[hidden email]> wrote:

> Date: October 24, 2014 4:45:27 AM
> From: Karsten Kusche <[hidden email]>
> To: "Michael Lucas-Smith"<[hidden email]>
> Cc: "VW NC"<[hidden email]>
> Subject: Re: [vwnc] Xtreams compressing
>
> Thinking about the problem a bit more, i think the design of the Compression-Classes really helps to solve the problem. It’s possible to change the CompressReadStream to actually compress by implementing a #compress method that reinitializes the compression-instvar. The following code is then possible:
>
> ((('Hello World' asByteArrayEncoding:#utf8) reading compressing compress rest) compressing encoding: #utf8) rest
>
> If the same design was implemented in the encode-classes, too the first part of the code could be changed to:
> ('Hello World' reading encoding: #utf8) toBytes compressing compress rest ...etc..
>
> Instead of sending #encoding: and then #toBytes it would probably be a lot easier to understand if the methods #encodeToBytes: or #encodeFromBytes: were introduced (or even better #encoding: and #decoding:).
>
> Kind Regards
> Karsten
>
>
> --
> Karsten Kusche - Dipl. Inf. (FH) - [hidden email] (mailto:[hidden email])
> Georg Heeg eK - Köthen
> Handelsregister: Amtsgericht Dortmund A 12812
>
>
> Am Freitag, 24. Oktober 2014 um 09:28 schrieb Karsten Kusche:
>
> > Hi Michael,
> >
> > it looked indeed more complicated than it was at the end. It’s basically a matter of initializing the compression instvar for deflate instead of inflate.
> > A bigger problem though is the naming. The #compressing method is already taken, even though it actually performs the decompression.
> >
> > The same problem applies to #encoding:. The assumption here is that it’s always sent to a ReadStream of bytes and then produces Characters out of it. That’s actually decoding the ByteArray. Encoding would take Characters and produce Bytes. As far as i can tell, this direction is also not implemented at the moment. If i understand the classes in the Transformation package correctly, the same problem may apply to all of those classes.
> >
> > It may not be the best idea to implement both ways for all kinds of transformations for reading and writing because then you’d always end up with 4 classes. I wonder if there’s some way to reuse the implementations of the write-streams as read-streams and vise versa.
> > Something like:
> >
> > ReadStream>>encodingToBytes: anEncoding
> > ^EncodeWriteStream forReading on: aWriteStream encoding: anEncoding
> >
> > Not sure if that’s feasible but would allow for using the already existing classes in the opposite direction.
> >
> > Kind regards
> > Karsten
> >
> >
> >
> >
> > --
> > Karsten Kusche - Dipl. Inf. (FH) - [hidden email] (mailto:[hidden email])
> > Georg Heeg eK - Köthen
> > Handelsregister: Amtsgericht Dortmund A 12812
> >
> >
> > Am Freitag, 24. Oktober 2014 um 01:04 schrieb Michael Lucas-Smith:
> >
> > > Hi Karsten,
> > >
> > > Martin and I voted to leave out the reverse variants from the package because we felt they weren’t that common and if someone needed them, they would be trivial to implement. I’d recommend you implement the reverse read and write streams for compression.
> > >
> > > Cheers,
> > > Michael
> > >
> > > > On 24 Oct 2014, at 3:40 am, Karsten Kusche <[hidden email] (mailto:[hidden email])> wrote:
> > > > Hi List,
> > > >
> > > > i’ve just had a look at Xtreams because i wanted to read files from disk and send them over the network. I thought Xtreams would be quite handy in that scenario because it allows for transforming the data as it is read or written.
> > > > A closer look at the #compressing mechanism however showed that this is only useful if you want to read the decompressed data of a compressed stream or if you want to write compressed data of an uncompressed stream. In my case i’d want the opposite: reading compressed data from a uncompressed stream. I’ve got an uncompressed file and would like to read gzip-compressed bytes from that file, so that i can hand the read-stream directly to the request.
> > > > As i see it, i’d have to somehow get my hands on the writestream of the request, send #compressing to it and then write the uncompressed read-stream to it.
> > > >
> > > > Am I getting that wrong or are there other ways of achieving that?
> > > >
> > > > Kind Regards
> > > > Karsten
> > > >
> > > >
> > > >
> > > > --
> > > > Karsten Kusche - Dipl. Inf. (FH) - [hidden email] (mailto:[hidden email])
> > > > Georg Heeg eK - Köthen
> > > > Handelsregister: Amtsgericht Dortmund A 12812
> > > >
> > > > _______________________________________________
> > > > vwnc mailing list
> > > > [hidden email] (mailto:[hidden email])
> > > > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
> > >
> >
> > _______________________________________________
> > vwnc mailing list
> > [hidden email] (mailto:[hidden email])
> > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
> >
> >
>
>
>
> _______________________________________________
> vwnc mailing list
> [hidden email]
> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
>

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Hildebrant, Richard

Why does window shrink when opened?

Hello,
When building a user interface with UIPainter, its window can be sized up to its max, which can be specified in the settings tool. The window will open to that specified size when it is run, but then shrinks in height a bit when it is moved. Is there some other setting on the window’s size that I’m missing? Any suggesting/comments appreciated.

Thanks,
Rick Hildebrant
________________________________
Notice: This email and any attachments may contain proprietary (Draper non-public) and/or export-controlled information of Draper Laboratory. If you are not the intended recipient of this email, please immediately notify the sender by replying to this email and immediately destroy all copies of this email.
________________________________

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Dave & Vicki Stevenson

Re: Why does window shrink when opened?

It sounds to me as though your default size is just a bit taller than your max size. I think max size is only consulted when opening the window via user placement. If automatic placement is used, I think the default size is used regardless of max size. When the window is moved or resized, the max size is consulted, which slightly reduces the height of the window.

Vicki & Dave Stevenson
[hidden email]

On Monday, October 27, 2014 11:58 AM, "Hildebrant, Richard" <[hidden email]> wrote:

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc