[Glass] How to properly write bytes objects into binary streams?
Locked
[Glass] How to properly write bytes objects into binary streams?
 
|
Hi guys,
In Pharo (yes, sorry if many of my threads start this way), we are able to directly write a byte object, that is, variableByteSubclass:... , into a binary stream. Example:
| buffer position byteObject | buffer := ByteArray new: 100. position := 5. byteObject := 8932479837423648732648237642. buffer replaceFrom: position + 1 to: position + byteObject size with: byteObject.
Now buffer is: #[0 0 0 0 0 74 234 94 86 234 171 88 69 46 198 220 28 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
In the same way I wrote there a large positive integer, I can also write any other byte-like object. This is efficient since I avoid a kind of #asByteArray and mostly, because I have a unified way to write a byte object to a binary stream, and then read it back.
In GemStone this doesn't work. I thought I could do a #asByteArray and then a #nextPutAll: That works, for example, for LargeInteger, but not for Float or Character. So....I wonder...is there a unified way to get a sequence of bytes from byte-like objects and then to instantiate them with such sequence of bytes?
Thanks in advance, _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] How to properly write bytes objects into binary streams?
|
|
Mariano, Is this still an issue? I see from later messages that you discovered #isByteOrSpecialand I assume that it's related... Dale On Tue, Sep 2, 2014 at 7:09 AM, Mariano Martinez Peck via Glass <[hidden email]> wrote:
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] How to properly write bytes objects into binary streams?
|
|
On Tue, Sep 2, 2014 at 12:30 PM, Dale Henrichs <[hidden email]> wrote:
Yes, still an issue. #isByteOrSpecial helped me to discover those strange guys. But I still don't know what I ask in this thread. Thanks
Mariano http://marianopeck.wordpress.com _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] How to properly write bytes objects into binary streams?
|
|
In reply to this post by GLASS mailing list
On 09/02/2014 07:09 AM, Mariano Martinez Peck via Glass wrote:
> Hi guys, > > In Pharo (yes, sorry if many of my threads start this way), we are able > to directly write a byte object, that is, variableByteSubclass:... , > into a binary stream. Example: > > | buffer position byteObject | > buffer := ByteArray new: 100. > position := 5. > byteObject := 8932479837423648732648237642. > buffer replaceFrom: position + 1 to: position + byteObject size with: > byteObject. > > Now buffer is: > > #[0 0 0 0 0 74 234 94 86 234 171 88 69 46 198 220 28 0 0 0 0 0 0 0 0 0 > 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 > 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] > > In the same way I wrote there a large positive integer, I can also write > any other byte-like object. This is efficient since I avoid a kind of > #asByteArray and mostly, because I have a unified way to write a byte > object to a binary stream, and then read it back. > > In GemStone this doesn't work. I thought I could do a #asByteArray and > then a #nextPutAll: That works, for example, for LargeInteger, but not > for Float or Character. > > So....I wonder...is there a unified way to get a sequence of bytes from > byte-like objects and then to instantiate them with such sequence of bytes? > Hi Mariano, I'm afraid I need to answer your question with more questions... Should the byte object you write out from a GemStone server be readable into a non-GemStone Smalltalk (for instance, Pharo) Should the object be readable into a GemStone server of opposite endianness from the server that wrote it out? If the answer to either of these is "yes," then things get a bit more complicated. Regards, -Martin _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] How to properly write bytes objects into binary streams?
|
|
On Tue, Sep 2, 2014 at 6:40 PM, Martin McClure <[hidden email]> wrote:
Hi Martin, Should the byte object you write out from a GemStone server be readable into a non-GemStone Smalltalk (for instance, Pharo) Well....as a very very first step, I am hacking a little to see how much effort would be to have Fuel working in Gemstone (and I already have like 50% tests working!). And as a first step gemstone-to-gemstone would be ok. If I note that Fuel does work correctly and stable in GemStone, then I might analyze how to do gemstone-pharo and vice-versa, but this looks like a much more complex project.
Should the object be readable into a GemStone server of opposite endianness from the server that wrote it out? This has less important I think since nowadays we are almost most of the time in the same order, aren't we? In any case, in Fuel's Pharo version, we do take care of that. The encoder stores the endianess, and then at materialization time we compare against current image. If opposite, we do a:
Bitmap swapBytesIn: aWordObject from: 1 to: aWordObject basicSize. But again, I want to go step by step.
If the answer to either of these is "yes," then things get a bit more complicated. Even if they would be desired later, right now, I am fine with a "No" answer. :) Thanks, Regards, Mariano http://marianopeck.wordpress.com _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] How to properly write bytes objects into binary streams?
|
|
On 09/03/2014 06:58 AM, Mariano Martinez Peck wrote:
> [...] > > But again, I want to go step by step. > > If the answer to either of these is "yes," then things get a bit > more complicated. > > > Even if they would be desired later, right now, I am fine with a "No" > answer. > OK, for the limited gs-to-gs case, same endianness... Since in GemStone #replaceFrom:to:with: expects to be copying from a collection, not (for instance) a LargeInteger, the most generic way to copy the bytes of a byte object is probably to write a loop that uses #_basicSize and #_basicAt: to access the bytes. Regards, -Martin _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] How to properly write bytes objects into binary streams?
|
|
On Wed, Sep 3, 2014 at 1:12 PM, Martin McClure <[hidden email]> wrote: On 09/03/2014 06:58 AM, Mariano Martinez Peck wrote: OK. Thanks. But then how can I create an empty instance of one of them, in a general way, and with a certain size? I tried: LargeInteger _basicNew: 10 But it fails. Thanks,
Mariano http://marianopeck.wordpress.com _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] How to properly write bytes objects into binary streams?
|
|
Take a look at Integer class>>_new:neg: ... the implication is that you might need to store "digits" instead of bytes .... Dale On Wed, Sep 3, 2014 at 11:43 AM, Mariano Martinez Peck via Glass <[hidden email]> wrote:
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] How to properly write bytes objects into binary streams?
|
|
On Wed, Sep 3, 2014 at 3:48 PM, Dale Henrichs <[hidden email]> wrote:
Thanks Dale. But that only works for Integers. I am looking for a polimorphic way for all type of byte-objects that are not SequenceableCollection subclasses. Float, Character, LargeInteger, Integer etc. OK, I do have the polimorphic way to get all bytes and write it to the binary stream using #_basicSize and #basicAt: . But then I am missing the materialization part. Actually, I miss the instance creation method to create an instance with a concrete size, so that I can then do the #_basicAt:put:
Thanks,
Mariano http://marianopeck.wordpress.com _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] How to properly write bytes objects into binary streams?
|
|
My point is that LargeIntegers are not polymorphic byte-objects (no methods for accessing the bytes for a LargeInteger) ... in GemStone Dale
On Wed, Sep 3, 2014 at 11:58 AM, Mariano Martinez Peck <[hidden email]> wrote:
_______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] How to properly write bytes objects into binary streams?
|
|
On Wed, Sep 3, 2014 at 4:10 PM, Dale Henrichs <[hidden email]> wrote:
OK. Thanks. Point taken. I could make a special serialization for them. But I was hoping to find a general way for all the other guys. These are the problematic ones:
Smalltalk allClasses select: [:each | each isBytesOrSpecial and: [(each includesBehavior: SequenceableCollection) not]] -> anArray( Boolean, Character, DecimalFloat, SmallInteger, UndefinedObject, ObsLargePositiveInteger, ObsLargeNegativeInteger, AbstractCharacter, JISCharacter, ObsFloat, ObsSmallFloat, ObsDoubleByteString, ObsDoubleByteSymbol, SmallDouble, ObsQuadByteString, ObsQuadByteSymbol, Float, LargeInteger, SmallFloat, TrueClass, FalseClass, BitSet, GsNativeCode, MCMockClassH)
Why? Because since they are not SequenceableCollection subclasses, #replaceFrom:to:with: won't work. So I was wondering a way to get bytes out of them, and then a way to create an empty instance for the correct size and set one by one...
But....taking a deeper look to that list, it seems that if I do a special serialization for AbstractCharacter (and subclasses), Float (and subclasses), Integer (and subclasses) , and Booleans (this is already solved) I am actually serializing most of the things I want. Right?
Mariano http://marianopeck.wordpress.com _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] How to properly write bytes objects into binary streams?
|
|
On 09/03/2014 12:26 PM, Mariano Martinez Peck wrote:
> > > > On Wed, Sep 3, 2014 at 4:10 PM, Dale Henrichs > <[hidden email] > <mailto:[hidden email]>> wrote: > > My point is that LargeIntegers are not polymorphic byte-objects (no > methods for accessing the bytes for a LargeInteger) ... in GemStone > > > OK. Thanks. Point taken. I could make a special serialization for them. > But I was hoping to find a general way for all the other guys. These are > the problematic ones: > #_basicAt: and #_basicAt:put: do work to access the bytes of large integers (and I think all byte objects). The trick is to create an instance of these class classes. For LargeInteger, for instance, this is currently quite restricted. The best I've come up with so far is to create one the correct size like this: | size int1 int2| size := <size in bytes you want> int1 := (2 raisedToInteger: (size - 4 * 8)) - 1. "int1 is the right size, but immutable" int2 := int1 copy. "int2 is mutable, and can be modified via #_basicAt:put:" Regards, -Martin _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] How to properly write bytes objects into binary streams?
|
|
On Wed, Sep 3, 2014 at 5:07 PM, Martin McClure <[hidden email]> wrote:
mmmmm I cannot access those in SmallDouble. I need to do a _asFloat. Maybe because it is "special" (immediate object?). Anyway, this doesn't work:
| byte1 byte2 | byte1 := (1.1 _asFloat _basicAt: 1). byte2 := (1.1 _asFloat _basicAt: 2 ). (Float basicNew) _basicAt: 1 put: byte1; _basicAt: 2 put: byte2 ; yourself
_basicAt: is answering strange numbers I think. The trick is to create an instance of these class classes. For LargeInteger, for instance, this is currently quite restricted. The best I've come up with so far is to create one the correct size like this: OK, I see. THe way I found for Integers and LargeIntegers is to do a #asByteArray when serializing and #asInteger when materializing and use a regular #nextPutAll. Thanks! Regards, Mariano http://marianopeck.wordpress.com _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Re: [Glass] How to properly write bytes objects into binary streams?
|
|
On 09/03/2014 01:22 PM, Mariano Martinez Peck wrote:
> > > #_basicAt: and #_basicAt:put: do work to access the bytes of large > integers (and I think all byte objects). > > > mmmmm I cannot access those in SmallDouble. I need to do a _asFloat. > Maybe because it is "special" (immediate object?). Right. SmallDoubles are not byte objects, they're immediate. > > Anyway, this doesn't work: > > | byte1 byte2 | > byte1 := (1.1 _asFloat _basicAt: 1). > byte2 := (1.1 _asFloat _basicAt: 2 ). > (Float basicNew) _basicAt: 1 put: byte1; _basicAt: 2 put: byte2 ; yourself > > _basicAt: is answering strange numbers I think. It should work. I think the problem is that you ended up with a two-byte Float, and Floats are eight bytes. Try this: | sourceFloat destFloat | sourceFloat := 1.1. sourceFloat := sourceFloat _asFloat. "Must convert possible SmallDouble to full Float." destFloat := Float basicNew: 8. 1 to: 8 do: [:i | destFloat _basicAt: i put: (sourceFloat basicAt: i)]. ^destFloat + 0.0. "Add 0 to possibly convert to SmallDouble." > > The trick is to create an instance of these class classes. For > LargeInteger, for instance, this is currently quite restricted. The > best I've come up with so far is to create one the correct size like > this: > > | size int1 int2| > size := <size in bytes you want> > int1 := (2 raisedToInteger: (size - 4 * 8)) - 1. > "int1 is the right size, but immutable" > int2 := int1 copy. > "int2 is mutable, and can be modified via #_basicAt:put:" I'd missed Dale's previous message about #_new:neg:, which is a better way than what I wrote above. > > > OK, I see. THe way I found for Integers and LargeIntegers is to do a > #asByteArray when serializing and #asInteger when materializing and use > a regular #nextPutAll. You're using the printString of the large integer to represent it, rather than the actual bytes, then? Regards, -Martin _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
«
Return to GLASS
|
1 view|%1 views
| Free forum by Nabble | Edit this page |