File/Stream changes: one Integer decoder/encoder to rule them all

Hi,

After the File and Stream changes in Pharo 7, a binary read, resp. write stream from/to a file is actually a ZnBuffered(Read|Write)Stream on a BinaryFileStream. You access these using #binary(Read|Write)Stream[Do:] sent to a FileReference.

As minimal streams the API of ZnBuffered(Read|Write)Stream was different from what existed before.

Specifically, a number of Integer decoding/encoding methods were missing. It is probably best to add those (an alternative would be a subclass).

When I looked at what was available, I thought I could improve upon the current situation. In fact, I think all existing ones can be written in terms of just one key method, one for reading and one for writing. The existing methods than become simple aliases, all while offering more functionality.

(Incidentally this would be an ideal use of a Trait, can we use them again/still ?)

The key method is #nextIntegerOfSize: numberOfBytes signed: signed bigEndian: bigEndian [put: value] and can be found in the latest version of Zinc-CharacterEncoding-Core with a comprehensive set of unit tests.

For example,

int32
  ^ self nextIntegerOfSize: 4 signed: true bigEndian: true

nextWord
  ^ self nextIntegerOfSize: 2 signed: false bigEndian: true

or non-aliases ones like

  binaryStream nextIntegerOfSize: 3 signed: true bigEndian: false

I think I nailed it, but I would love a second pair of eyes to check what I did.

The same functionality could be added to ByteArray in a slight variation (with offsets), where similar integer encoding/decoding methods exist.

Sven

On Fri, Apr 20, 2018 at 5:24 PM, Sven Van Caekenberghe <[hidden email]> wrote:

Hi,

After the File and Stream changes in Pharo 7, a binary read, resp. write stream from/to a file is actually a ZnBuffered(Read|Write)Stream on a BinaryFileStream. You access these using #binary(Read|Write)Stream[Do:] sent to a FileReference.

As minimal streams the API of ZnBuffered(Read|Write)Stream was different from what existed before.

Specifically, a number of Integer decoding/encoding methods were missing. It is probably best to add those (an alternative would be a subclass).

When I looked at what was available, I thought I could improve upon the current situation. In fact, I think all existing ones can be written in terms of just one key method, one for reading and one for writing. The existing methods than become simple aliases, all while offering more functionality.

(Incidentally this would be an ideal use of a Trait, can we use them again/still ?)

The key method is #nextIntegerOfSize: numberOfBytes signed: signed bigEndian: bigEndian [put: value] and can be found in the latest version of Zinc-CharacterEncoding-Core with a comprehensive set of unit tests.

For example,

int32
  ^ self nextIntegerOfSize: 4 signed: true bigEndian: true

nextWord
  ^ self nextIntegerOfSize: 2 signed: false bigEndian: true

or non-aliases ones like

  binaryStream nextIntegerOfSize: 3 signed: true bigEndian: false

I think I nailed it, but I would love a second pair of eyes to check what I did.

Sorry I arrive late.

I'm thinking that this is only valid for binary streams, putting it in buffered stream overloads the API... We can leave with it but... Two thoughts that come to my mind about this:

 - first, when we are decoding like that from a stream, we are probably parsing a binary format, so we want in general to have utility methods to read different sizes. Maybe we could include a Reader class with methods such as

   * readSignedInt32

   * readUnsignedInt32

   * readSignedInt64

   * readUnsignedInt64

   * readFloat64

... and so on?

- Second, that Reader may be (or not) a new decorator alternative. ZnBinaryDecoder? Wrapping a binary (buffered or not) stream. And this would not overload the API ^^.

What do you think?

The same functionality could be added to ByteArray in a slight variation (with offsets), where similar integer encoding/decoding methods exist.

Sven

--

Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - http://www.cnrs.fr Web: http://guillep.github.io Phone: +33 06 52 70 66 13

>
>
> For ByteArray, I didn't check recently in Pharo, but I know that it's quite a mess in Squeak because some methods are in Alien, other in FFI, other in base Squeak...
> IMO all methods should be in core Squeak/Pharo because of general usage (like decoding binary data).
>
I want to move the methods from FFI to the base since years… I was told that this is impossible to ever happen because it is not important to do changes like that.
 

        Marcus

On 14 May 2018, at 15:09, Nicolas Cellier <[hidden email]> wrote:

2018-05-14 14:44 GMT+02:00 Marcus Denker <[hidden email]>:

>
>
> For ByteArray, I didn't check recently in Pharo, but I know that it's quite a mess in Squeak because some methods are in Alien, other in FFI, other in base Squeak...
> IMO all methods should be in core Squeak/Pharo because of general usage (like decoding binary data).
>
I want to move the methods from FFI to the base since years… I was told that this is impossible to ever happen because it is not important to do changes like that.
 

        Marcus

Hi Marcus,

we all know that those un-important and not-so-much-rewarding changes makes a big difference at the end.
like commenting classes, classifying methods, deprecating duplicates, reducing package inter-dependencies, etc...

I thought that the defenders of statu quo had two reasons in mind:

- the will to keep a FFI-less (locked-in) image

- the fear to loose backward compatibility

The idea that “only big important changes” are worth doing… you know, it’s “inventing the future” business, not “cleaning up”!

(Of course revolutionary changes while staying 100% backward compatible… because think about the Children!)

It would be funny if it would not be sad…

But's it the past, now you are free :)

Not really, because we share that package with Squeak. If I would be free to change it, it would have happened a loong time ago…

Marcus