DeflateStream unable to write binary streams?

I was playing with GZipWriteStream and ZLibWriteStream, trying
to write compressed binary data in the middle of a chunk file,
like so:

writeDelta: aDelta
    stream nextPutAll: '!DSBinStReader new!'.
    (DataStream on: (GZipWriteStream on: stream)) nextPut: aDelta

This didn't work at all. First, GZipWriteStream wasn't even
writing to the stream, but to it's own internal collection.
DeflateStreams seem to try to seize control of the entire
underlying stream or collection.

Second, it wouldn't accept the binary data that DataStream was
feeding it. It seems to only accept character data, as sending
the message #binary to a DeflateStream does nothing.

So, should DeflateStreams be general purpose streams, or is
there a good reason they are text-only and not embeddable?

--
Matthew Fulmer -- http://mtfulmer.wordpress.com/
Help improve Squeak Documentation: http://wiki.squeak.org/squeak/808

Matthew Fulmer wrote:
Well, let's be careful with the claims here. A couple of examples:

1) Simple compression to an existing stream:

(GZipWriteStream on: (FileStream newFileNamed: 'test.gz'))
        nextPutAll: 'Hello World';
        close.

2) Embedding compressed data:

        stream := String new writeStream.
        stream nextPutAll: 'Binary data: '.
        (GZipWriteStream on: stream) nextPutAll: 'Hello World'; finish.
        stream nextPutAll: 'More data follows'.
        stream contents.


3) Embedding compressed binary data in the midst of text:

        stream := RWBinaryOrTextStream on: String new.
        stream nextPutAll: 'Binary data: '.
        stream binary.
        (GZipWriteStream on: stream) nextPut: 123; nextPut: 234; finish.
        stream ascii.
        stream nextPutAll: 'More data follows' asByteArray.
        stream contents.

All of the above examples work fine.

> Second, it wouldn't accept the binary data that DataStream was
> feeding it. It seems to only accept character data, as sending
> the message #binary to a DeflateStream does nothing.

DeflateStreams accept binary data if and only if the target stream is
binary. They do not allow switching between binary and non-binary in the
middle of things.

> So, should DeflateStreams be general purpose streams, or is
> there a good reason they are text-only and not embeddable?

Besides it being not true (as the third example above illustrates) it
wouldn't be a problem if it were. You could trivially solve the problem
by using:

writeDelta: aDelta
     stream nextPutAll: '!DSBinStReader new!'.
     stream nextPutAll: (DataStream streamedRepresentationOf: aDelta)
asString zipped.

which is not significantly more complex than your solution and actually
works - yours doesn't because you either need to #close or #finish the
compressor to write the last block (remember LZ is block based). Note
that #flush is *not* enough here since the LZ family of compressors
marks the last block specifically (#flush will write a block but not
mark it as the last block and upset the decompressor on reading it).

Cheers,
   - Andreas

On Tue, Sep 25, 2007 at 11:33:26PM -0700, Andreas Raab wrote:
> 3) Embedding compressed binary data in the midst of text:
>
> stream := RWBinaryOrTextStream on: String new.
> stream nextPutAll: 'Binary data: '.
> stream binary.
> (GZipWriteStream on: stream) nextPut: 123; nextPut: 234; finish.
> stream ascii.
> stream nextPutAll: 'More data follows' asByteArray.
> stream contents.

Ah. I needed to switch the stream *before* I start writing. I
also didn't know about the #finish message.

> All of the above examples work fine.
>
> >Second, it wouldn't accept the binary data that DataStream was
> >feeding it. It seems to only accept character data, as sending
> >the message #binary to a DeflateStream does nothing.
>
> DeflateStreams accept binary data if and only if the target stream is
> binary. They do not allow switching between binary and non-binary in the
> middle of things.

Didn't know that. Thank you

> >So, should DeflateStreams be general purpose streams, or is
> >there a good reason they are text-only and not embeddable?
>
> Besides it being not true (as the third example above illustrates) it
> wouldn't be a problem if it were. You could trivially solve the problem
> by using:

Sorry for the false assertion.

Thank you very much. I guess the problem was me not finding or
reading any comments about how to use DeflateStreams.

PS: what is the difference between GZipWriteStream,
ZLibWriteStream, and ZipWriteStream? ZipWriteStream has the
concept of files in an archive, but I couldn't figure out the
first two

--
Matthew Fulmer -- http://mtfulmer.wordpress.com/
Help improve Squeak Documentation: http://wiki.squeak.org/squeak/808

Matthew Fulmer wrote:
> PS: what is the difference between GZipWriteStream,
> ZLibWriteStream, and ZipWriteStream? ZipWriteStream has the
> concept of files in an archive, but I couldn't figure out the
> first two

ZipWriteStream is the superclass for both GZipWriteStream and
ZLibWriteStream and implements the full LZ77 algorithm (I think you were
talking about ZipArchive which has files in an archive; ZipWriteStream
doesn't). The difference between GZip and ZLib is minimal: Different
headers, footers and checksum algorithm. Other than that they are identical.

Cheers,
   - Andreas

I had to fix Inflate streams for rio (there are some tests too)
so that binary and text setting of the wrapped stream is honoured
If I remember, binary uses a ByteArray and Text uses a String for the
internal buffer
At some point it fails to ask the stream whether it is a binary stream
and initializes its buffer as a ByteArray.

Have a look at rios's class extensions for details

regards

Keith