Smalltalk › Pharo › Pharo Smalltalk Users

What is your preferred way for storing ByteArray's as text?

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

11 messages Options

Esteban A. Maringolo

What is your preferred way for storing ByteArray's as text?

Hello all,

As some already know I'm working in the new GLORP port from VW to Pharo.

And as part of minor modifications I'm making to the code, I'd like to know what is your preferred way of storing ByteArrays as text representation/CLOB, etc...

Option A) The printString of ByteArray, it is #[23 12 253 ...]

Option B) A Base64 encoded version of the ByteArray

Option C) Some other encoding

I posted the same question in the #databases channel at Pharo's Slack team (https://pharoproject.slack.com/archives/databases/p1446130304000022)

Regards!

Esteban A. Maringolo

Sven Van Caekenberghe-2

Re: What is your preferred way for storing ByteArray's as text?

> On 29 Oct 2015, at 16:35, Esteban A. Maringolo <[hidden email]> wrote:
>
> Hello all,
>
> As some already know I'm working in the new GLORP port from VW to Pharo.
>
> And as part of minor modifications I'm making to the code, I'd like to know what is your preferred way of storing ByteArrays as text representation/CLOB, etc...
>
> Option A) The printString of ByteArray, it is #[23 12 253 ...]
> Option B) A Base64 encoded version of the ByteArray
> Option C) Some other encoding

Does that not dependent, at least in part, on the actual underlying SQL DB ?
I think HEX and Base64 are make the most sense.

> I posted the same question in the #databases channel at Pharo's Slack team (https://pharoproject.slack.com/archives/databases/p1446130304000022)
>
> Regards!
>
>
> Esteban A. Maringolo

Esteban A. Maringolo

Re: What is your preferred way for storing ByteArray's as text?

2015-10-29 12:44 GMT-03:00 Sven Van Caekenberghe <[hidden email]>:

>
>
> > On 29 Oct 2015, at 16:35, Esteban A. Maringolo <[hidden email]> wrote:
> >
> > Hello all,
> >
> > As some already know I'm working in the new GLORP port from VW to Pharo.
> >
> > And as part of minor modifications I'm making to the code, I'd like to know what is your preferred way of storing ByteArrays as text representation/CLOB, etc...
> >
> > Option A) The printString of ByteArray, it is #[23 12 253 ...]
> > Option B) A Base64 encoded version of the ByteArray
> > Option C) Some other encoding
>
> Does that not dependent, at least in part, on the actual underlying SQL DB ?

Yes it does, but certain backends (like SQLite) doesn't provide BLOB
storage, and that's why I'm talking about CLOB (character instead of
byte based) or text representations for it. I'd like to have a lowest
common denominator to use as a fallback.

> I think HEX and Base64 are make the most sense.

Is Base64 more compact than Hex? (I guess it is, given it's 64vs16 :D)

Regards!

Esteban A. Maringolo

Sven Van Caekenberghe-2

Re: What is your preferred way for storing ByteArray's as text?

> On 29 Oct 2015, at 17:02, Esteban A. Maringolo <[hidden email]> wrote:
>
> 2015-10-29 12:44 GMT-03:00 Sven Van Caekenberghe <[hidden email]>:
>>
>>
>>> On 29 Oct 2015, at 16:35, Esteban A. Maringolo <[hidden email]> wrote:
>>>
>>> Hello all,
>>>
>>> As some already know I'm working in the new GLORP port from VW to Pharo.
>>>
>>> And as part of minor modifications I'm making to the code, I'd like to know what is your preferred way of storing ByteArrays as text representation/CLOB, etc...
>>>
>>> Option A) The printString of ByteArray, it is #[23 12 253 ...]
>>> Option B) A Base64 encoded version of the ByteArray
>>> Option C) Some other encoding
>>
>> Does that not dependent, at least in part, on the actual underlying SQL DB ?
>
> Yes it does, but certain backends (like SQLite) doesn't provide BLOB
> storage, and that's why I'm talking about CLOB (character instead of
> byte based) or text representations for it. I'd like to have a lowest
> common denominator to use as a fallback.
>
>> I think HEX and Base64 are make the most sense.
>
> Is Base64 more compact than Hex? (I guess it is, given it's 64vs16 :D)

(0 to: 255) asByteArray base64Encoded size. "348"
(0 to: 255) asByteArray hex size. "512"

Beware the newlines in Base64 (better disable them)

> Regards!
>
> Esteban A. Maringolo

Peter Uhnak

Re: What is your preferred way for storing ByteArray's as text?

>>> Option A) The printString of ByteArray, it is #[23 12 253 ...]

>>> Option B) A Base64 encoded version of the ByteArray

I like this one, because it is not Pharo-specific format. ByteArray is not pharo-specific, but the syntax is --- I can copy paste base64 from/to outside world.

Plus base64 has better tool support in the wild.

Esteban A. Maringolo

Re: What is your preferred way for storing ByteArray's as text?

2015-10-29 15:21 GMT-03:00 Peter Uhnák <[hidden email]>:
>> >>> Option A) The printString of ByteArray, it is #[23 12 253 ...]
>>
>> >>> Option B) A Base64 encoded version of the ByteArray
>
>
> I like this one, because it is not Pharo-specific format. ByteArray is not
> pharo-specific, but the syntax is --- I can copy paste base64 from/to
> outside world.
> Plus base64 has better tool support in the wild.

I also prefer base64 even over native binary storage.

I ran a small comparison[1] of speed and size and these were my results.

ByteArray size: 19448860

Speed:
Base64MimeConverter: ''0.592 per second''
#hex: ''0.666 per second''
#asString: ''6.898 per second''
ZnBase64Encoder ''0.459 per second''

Size:
Base64MimeConverter: 26291980
#hex: 38897720
#asString: 19448860
ZnBase64Encoder 25931816

Esteban A. Maringolo

[1]: http://ws.stfx.eu/47PV2GUP9ITS

vonbecmann

Re: What is your preferred way for storing ByteArray's as text?

Esteban,

Could you explain line 8 ? why did you use a Base64MimeConverter?

thanks.

line 8:

stream nextPutAll: 'ZnBase64Encoder '; nextPutAll: [ (Base64MimeConverter mimeEncode: ba readStream multiLine: false) ] bench printString; cr.

On Thu, Oct 29, 2015 at 3:43 PM, Esteban A. Maringolo <[hidden email]> wrote:

2015-10-29 15:21 GMT-03:00 Peter Uhnák <[hidden email]>:
>> >>> Option A) The printString of ByteArray, it is #[23 12 253 ...]
>>
>> >>> Option B) A Base64 encoded version of the ByteArray
>
>
> I like this one, because it is not Pharo-specific format. ByteArray is not
> pharo-specific, but the syntax is --- I can copy paste base64 from/to
> outside world.
> Plus base64 has better tool support in the wild.

I also prefer base64 even over native binary storage.

I ran a small comparison[1] of speed and size and these were my results.

ByteArray size: 19448860

Speed:
Base64MimeConverter: ''0.592 per second''
#hex: ''0.666 per second''
#asString: ''6.898 per second''
ZnBase64Encoder ''0.459 per second''

Size:
Base64MimeConverter: 26291980
#hex: 38897720
#asString: 19448860
ZnBase64Encoder 25931816

Esteban A. Maringolo

[1]: http://ws.stfx.eu/47PV2GUP9ITS

Bernardo E.C.

Sent from a cheap desktop computer in South America.

Esteban A. Maringolo

Re: What is your preferred way for storing ByteArray's as text?

2015-10-29 17:07 GMT-03:00 Bernardo Ezequiel Contreras <[hidden email]>:
> Esteban,
> Could you explain line 8 ? why did you use a Base64MimeConverter?
> thanks.
>
> line 8:
>
> stream nextPutAll: 'ZnBase64Encoder '; nextPutAll: [ (Base64MimeConverter
> mimeEncode: ba readStream multiLine: false) ] bench printString; cr.

Because it is the class used by ByteArray>>#base64Encoded.

Esteban A. Maringolo

vonbecmann

Re: What is your preferred way for storing ByteArray's as text?

thanks, so the title should be Base64MimeConverter instead of ZnBase64Encoder.

On Thu, Oct 29, 2015 at 5:18 PM, Esteban A. Maringolo <[hidden email]> wrote:

2015-10-29 17:07 GMT-03:00 Bernardo Ezequiel Contreras <[hidden email]>:
> Esteban,
> Could you explain line 8 ? why did you use a Base64MimeConverter?
> thanks.
>
> line 8:
>
> stream nextPutAll: 'ZnBase64Encoder '; nextPutAll: [ (Base64MimeConverter
> mimeEncode: ba readStream multiLine: false) ] bench printString; cr.

Because it is the class used by ByteArray>>#base64Encoded.

Esteban A. Maringolo

Bernardo E.C.

Sent from a cheap desktop computer in South America.

Esteban A. Maringolo

Re: What is your preferred way for storing ByteArray's as text?

2015-10-29 17:59 GMT-03:00 Bernardo Ezequiel Contreras <[hidden email]>:

> On Thu, Oct 29, 2015 at 5:18 PM, Esteban A. Maringolo <[hidden email]>
> wrote:
>>
>> 2015-10-29 17:07 GMT-03:00 Bernardo Ezequiel Contreras
>> <[hidden email]>:
>> > Esteban,
>> > Could you explain line 8 ? why did you use a Base64MimeConverter?
>> > thanks.
>> >
>> > line 8:
>> >
>> > stream nextPutAll: 'ZnBase64Encoder '; nextPutAll: [
>> > (Base64MimeConverter
>> > mimeEncode: ba readStream multiLine: false) ] bench printString; cr.
>>
>> Because it is the class used by ByteArray>>#base64Encoded.

> thanks, so the title should be Base64MimeConverter instead of
> ZnBase64Encoder.

No... I got it wrong, it was a copy & paste mistake. It should be
Zn... because I put them both in the tests to compare.

Thank you for noticing.

Regards!

Esteban A. Maringolo

Stephan Eggermont-3

Re: What is your preferred way for storing ByteArray's as text?

In reply to this post by Esteban A. Maringolo

On 29/10/15 16:35, Esteban A. Maringolo wrote:
> As some already know I'm working in the new GLORP port from VW to Pharo.

Are you generating the port with rewrite rules?

Stephan