OS.ZLib.ZLibInterface, accessing additional functions?

I am trying to compress ByteArrays using the zlib library linked into the VisualWorks VM (on Windows 2000). I already have code written for a homegrown interface to an external zlib DLL which uses the function compress(). This function is not part of the interface of OS.ZLib.ZLibInterface, therefore I added the following method to ZLibInterface:

compress: dest with: destLen with: source with: sourceLen
        <C: int compress(Byte * dest, uLong * destLen, Byte * source, uLong sourceLen)>
        ^self externalAccessFailedWith: _errorCode

The same method worked with the homegrown zlib interface. When I use it, it raises an Error with the text "External object not found". The reason is not that there is no type definition for Byte - I added that, too. ;-)

Does anybody know if (and how) it is possible to add interface methods for the zlib functions which are not yet part of ZLibInterface, or is it restricted to what is already there?

Best regards,
Joachim Geidel

Hi Joachim,

[hidden email] wrote:

| I am trying to compress ByteArrays using the zlib library linked into the VisualWorks VM (on Windows 2000). I already have code written for a homegrown interface to an external zlib DLL which uses the function compress(). This function is not part of the interface of OS.ZLib.ZLibInterface, therefore I added the following method to ZLibInterface:

| compress: dest with: destLen with: source with: sourceLen
| <C: int compress(Byte * dest, uLong * destLen, Byte * source, uLong sourceLen)>
| ^self externalAccessFailedWith: _errorCode

| The same method worked with the homegrown zlib interface. When I use it, it raises an Error with the text "External object not found". The reason is not that there is no type definition for Byte - I added that, too. ;-)

| Does anybody know if (and how) it is possible to add interface methods for the zlib functions which are not yet part of ZLibInterface, or is it restricted to what is already there?

Thaty certainly was the intention.  But I just checked the relevant code and fond I screwed up; the current VMs don't export compress, compress2 or uncompress.

The god news is that this is easily fixed and coincidentally I am doing a new VM product build, vw7.4d, today or tomorrow.  So a VM with this problem fixed should be available within a week.

Apologies.
---
Eliot Miranda                 ,,,^..^,,,                mailto:[hidden email]
VisualWorks Engineering, Cincom  Smalltalk: scene not herd  Tel +1 408 216 4581
3350 Scott Blvd, Bldg 36 Suite B, Santa Clara, CA 95054 USA Fax +1 408 216 4500

Try to use the (GZipWrite /GzipRead)  Stream.

Here is example as one  can store arbitrary objects as compressed
byteArrays.  You probably need only first line and the two last as you
already have your byteArray ready.

| encodedStream newStr theResult intStr |
        encodedStream := OS.ZLib.GZipWriteStream
                                                bestCompressionOn: (theResult := ByteArray new writeStream).
                        newStr := ByteString new readWriteStream.
                        self original storeOn: newStr.
                        intStr := ReadStream on: newStr contents asByteArray.
                        intStr reset.
                        [intStr atEnd] whileFalse: [encodedStream nextPutBufferFrom: intStr].
                        encodedStream close.
                        myObject := theResult contents.


HTH

--Mark Pirogovsky

[hidden email] wrote:

Hi Mark,

>Try to use the (GZipWrite /GzipRead)  Stream.

Thanks for the suggestion. Unfortunately, using GZipWriteStream
seems to produce different results than using compress().
E.g., compressing an empty ByteArray has these results:

compress()
#[120 156 3 0 0 0 0 1]

GZipWriteStream, default compression
#[31 139 8 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0]

As we have to ensure backward compatibility, I'll wait for the
new VM.

Joachim

Joahim,

That is good to know.  I was using those classes on the external streams
ant it works reasonably well.

If you look at the way the GZipWriteStream designed and implemented - it
is for writing out compressed files. Which one can open later with other
programs such as winZip.  For that purpose it adds the gzip signature
header to every compressed stream.  The header is  10 bytes in length.
(see the #writeHeader) and then it adds 8 bytes at the tail to tell some
of the size and crc information (#finishOutput).

Apparently the compress() also does make some header and tail, albeit
differently -- the head is 2 bytes and tail is 4 .

The middle part is the same #[3 0].

--Mark



[hidden email] wrote:

Hi Joachim,

        fixed VMs are now available at
http://www.cincomsmalltalk.com/CincomSmalltalkWiki/VisualWorks+7.4d+engines

[hidden email] wrote:

| I am trying to compress ByteArrays using the zlib library linked into the VisualWorks VM (on Windows 2000). I already have code written for a homegrown interface to an external zlib DLL which uses the function compress(). This function is not part of the interface of OS.ZLib.ZLibInterface, therefore I added the following method to ZLibInterface:

| compress: dest with: destLen with: source with: sourceLen
| <C: int compress(Byte * dest, uLong * destLen, Byte * source, uLong sourceLen)>
| ^self externalAccessFailedWith: _errorCode

| The same method worked with the homegrown zlib interface. When I use it, it raises an Error with the text "External object not found". The reason is not that there is no type definition for Byte - I added that, too. ;-)

| Does anybody know if (and how) it is possible to add interface methods for the zlib functions which are not yet part of ZLibInterface, or is it restricted to what is already there?

| Best regards,
| Joachim Geidel
---
Eliot Miranda                 ,,,^..^,,,                mailto:[hidden email]
VisualWorks Engineering, Cincom  Smalltalk: scene not herd  Tel +1 408 216 4581
3350 Scott Blvd, Bldg 36 Suite B, Santa Clara, CA 95054 USA Fax +1 408 216 4500

Many Thanks Eliot,

I have two questions:

1. In the list of fixes there is a line:

  50990: Moving low-level event dispatch from the VM up into the VI

does it require some VI code modification as well ?

2. 51363: ZLib code fails to export all functions in the zlib interface.

Is there image level code which allow us to utilize those missing
functions ?  I hate to reinvent the wheel and write my own, when most
likely it is already available...

Thanks,

--Mark Pirogovsky

[hidden email] wrote:

Mark Pirogovsky <[hidden email]> wrote:

| Many Thanks Eliot,

| I have two questions:

| 1. In the list of fixes there is a line:

|   50990: Moving low-level event dispatch from the VM up into the VI

| does it require some VI code modification as well ?

yes.  Talk to the OS8 folks, Andreas Hiltner ([hidden email]), Mark Grinnell ([hidden email]), Georg Heeg ([hidden email]), Jörg Belger ([hidden email]) for details.

| 2. 51363: ZLib code fails to export all functions in the zlib interface.

| Is there image level code which allow us to utilize those missing
| functions ?  I hate to reinvent the wheel and write my own, when most
| likely it is already available...

Ask Joachim ([hidden email]).  I just responded to his problem report.

Cheers
---
Eliot Miranda                 ,,,^..^,,,                mailto:[hidden email]
VisualWorks Engineering, Cincom  Smalltalk: scene not herd  Tel +1 408 216 4581
3350 Scott Blvd, Bldg 36 Suite B, Santa Clara, CA 95054 USA Fax +1 408 216 4500

There is a bug in OS.ZLib.ZLibInterface>>cInflateSync:. It calls
inflateReset() instead of inflateSync():

cInflateSync: zstreamp
        <C: int inflateReset( z_streamp )>
        ^self externalAccessFailedWith: _errorCode

There should be a pair of methods like these instead:

cInflateSync: zstreamp
        <C: int inflateSync( z_streamp )>
        ^self externalAccessFailedWith: _errorCode

cInflateReset: zstreamp
        <C: int inflateReset( z_streamp )>
        ^self externalAccessFailedWith: _errorCode

Best regards,
Joachim Geidel

> Mark Pirogovsky <[hidden email]> wrote:
> | 2. 51363: ZLib code fails to export all functions in the zlib interface.
>
> | Is there image level code which allow us to utilize those missing
> | functions ?  I hate to reinvent the wheel and write my own, when most
> | likely it is already available...

Hi Mark, hi Eliot,

I have published a package Compression-ZLib-Extensions and an
accompanying test package based on SUnitToo to the public repository. It
contains wrapper methods for compress(), compress2(), and uncompress(),
i.e. the in-memory-compression utility methods of ZLib.

The package also contains
- an override for cInflateSync: such that it calls inflateSync() instead
of inflateReset(), and
- interface methods for some functions not present in Compression-ZLib:
inflateReset(), gzerror(), gzgetc(), gzputc(), gzgets(), gzputs(), and
gzprintf().
These methods are untested, but I suspect nobody will ever use them anyway.

Do with it whatever you want - but at your own risk, of course. ;-)

BTW, are there any plans to upgrade to zlib 1.2.3? The authors claim
that inflating is about 20% faster, which might be a good thing when
working with compressed image files.

Best regards,
Joachim Geidel