Is there any serializer (Fuel/StOMP/Ma object
serialization/BOSS/SRP/SIXX??) which let me attach an external compressor/decompressor for specific Strings to the serialization/deserialization process? With external I mean it could be in a separate binary (I could arrange the binding though). Thanks Hernán _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Hernán,
There are several ways to do it with SRP. The encoding supports embedded data (variable and fixed width), so you could do inline serialization. It is clever enough that it is even possible to do this with positional access even to base64 encoded data. Tricks like that are used for example to update individual values in a persistent index. There are simpler techniques that should work for any marshaler... All marshalers should have some form of pre-save and post-load exchange. SRP has many ways to pre-save and post-load while maintaining relative references. SRP can even do post-load exchanges after the rest of the object graph is loaded and still keep relative references (without using a #become:). Save a proxy in place of the object. The proxy would be a class that you create to do what you want as a post-load actions. You can create the proxy as part of a pre-save action. The proxy can have the encoded binary ByteArray you want embedded in the data. The post-load of the proxy would typically unmarshal the embedded data. Here is an example of a class-based exchange: SpecialSaveObject>>srpSavePreMap: aContext ^MyOtherMarshalProxy newFor: self MyOtherMarshalerProxy.binData MyOtherMarshalerProxy class>>newFor: anObject ^self new save: anObject MyOtherMarshalProxy>>save: anObject binData := self marshaler saveAsBinary: anObject. MyOtherMarshalerProxy>>srpLoadPostMap: aContext ^OtherMarshaler loadFromBinary: binData Depending on your needs, you might want your object to remember the embedded data. In that case, you could store the encoded data in an inst var that is lazy-initialized as a pre-save action and save only that inst var. A post-load action could perhaps read the other attributes from the encoded data that you saved. SRP also allows any object to answer back "extended attributes" that are saved with an object (#srpExtendedAttributes) and can be restored after loading (#srpExtendedAttributes:). Extended attributes are typically data you want associated with an object that is not directly referenced by the object. You could save/load encoded data along with the SRP saved object. Paul Baumann -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Hernán Morales Durand Sent: Monday, October 10, 2011 15:36 To: Pharo Development; The general-purpose Squeak developers list; vwnc mailing list Subject: [vwnc] Any serializer can customize serialization strategy? Is there any serializer (Fuel/StOMP/Ma object serialization/BOSS/SRP/SIXX??) which let me attach an external compressor/decompressor for specific Strings to the serialization/deserialization process? With external I mean it could be in a separate binary (I could arrange the binding though). Thanks Hernán _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired. _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Hi Paul,
Thanks for your detailed reply. What you describe seems suitable for my needs (I'm interested in simplest techniques, my problem is complex enough to try premature improvements at the tool level :) however I cannot comment more until I do some real experiments with SRP. Thanks again Hernán 2011/10/12 Paul Baumann <[hidden email]>: > Hernán, > > There are several ways to do it with SRP. > > The encoding supports embedded data (variable and fixed width), so you could do inline serialization. It is clever enough that it is even possible to do this with positional access even to base64 encoded data. Tricks like that are used for example to update individual values in a persistent index. There are simpler techniques that should work for any marshaler... > > All marshalers should have some form of pre-save and post-load exchange. SRP has many ways to pre-save and post-load while maintaining relative references. SRP can even do post-load exchanges after the rest of the object graph is loaded and still keep relative references (without using a #become:). > > Save a proxy in place of the object. The proxy would be a class that you create to do what you want as a post-load actions. You can create the proxy as part of a pre-save action. The proxy can have the encoded binary ByteArray you want embedded in the data. The post-load of the proxy would typically unmarshal the embedded data. Here is an example of a class-based exchange: > > SpecialSaveObject>>srpSavePreMap: aContext > ^MyOtherMarshalProxy newFor: self > > MyOtherMarshalerProxy.binData > > MyOtherMarshalerProxy class>>newFor: anObject > ^self new save: anObject > > MyOtherMarshalProxy>>save: anObject > binData := self marshaler saveAsBinary: anObject. > > MyOtherMarshalerProxy>>srpLoadPostMap: aContext > ^OtherMarshaler loadFromBinary: binData > > Depending on your needs, you might want your object to remember the embedded data. In that case, you could store the encoded data in an inst var that is lazy-initialized as a pre-save action and save only that inst var. A post-load action could perhaps read the other attributes from the encoded data that you saved. > > SRP also allows any object to answer back "extended attributes" that are saved with an object (#srpExtendedAttributes) and can be restored after loading (#srpExtendedAttributes:). Extended attributes are typically data you want associated with an object that is not directly referenced by the object. You could save/load encoded data along with the SRP saved object. > > Paul Baumann > > > > -----Original Message----- > From: [hidden email] [mailto:[hidden email]] On Behalf Of Hernán Morales Durand > Sent: Monday, October 10, 2011 15:36 > To: Pharo Development; The general-purpose Squeak developers list; vwnc mailing list > Subject: [vwnc] Any serializer can customize serialization strategy? > > Is there any serializer (Fuel/StOMP/Ma object > serialization/BOSS/SRP/SIXX??) which let me attach an external > compressor/decompressor for specific Strings to the > serialization/deserialization process? With external I mean it could > be in a separate binary (I could arrange the binding though). > Thanks > > Hernán > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired. > _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Hernán,
Here are a few more SRP specific tips that may be helpful... A relatively new addition to SRP is the ability to marshal "distinct" object graphs embedded within a larger marshaling context. Look at the #Distinct related code in #storeTypeArrays. Class and inst var store types can be declared through #srpDataTypeMap declarations. Your SrpConfiguration can declare the proxy to save and marshaler to save the distinct object with (SRP is the default). SRP sends #srpSavedAs: with the object (a proxy in this case) that SRP saved as. It is possible to note the stream position that your distinct object is saved at. SRP also has a marshal directive for saving Proxy type objects. Proxies can be declared by class or inst var (by way of #srpDataTypeMap declarations) or by one of several other pre-save mapping options. You'd customize your SrpConfiguration to answer the proxy (#proxyFor:context:). You could for example save an object as a proxy that would do an SQL query to get the data from an RDB. Either SrpContext.userData or SrpConfiguration.userData can be used to manage your proxies. Either saving a #Proxy or #Distinct may serve your needs, or at least show a pattern that you can customize. The SRP encoding is designed to be infinitely extendable and portable. When using SRP, you should always start with a SrpConfiguration that you'd customize for your needs or for different marshaling purposes. The default SRP configurations are intended for greatest portability. SRP encodes metastate and state in as many bytes as other typical marshaling frameworks encode just state. If you need even faster and tighter encoding then use SrpMetastateTables to externalize your most commonly used metastates. Paul Baumann -----Original Message----- From: Hernán Morales Durand [mailto:[hidden email]] Sent: Friday, October 14, 2011 01:26 To: Paul Baumann Cc: Pharo Development; The general-purpose Squeak developers list; vwnc mailing list Subject: Re: [vwnc] Any serializer can customize serialization strategy? Hi Paul, Thanks for your detailed reply. What you describe seems suitable for my needs (I'm interested in simplest techniques, my problem is complex enough to try premature improvements at the tool level :) however I cannot comment more until I do some real experiments with SRP. Thanks again Hernán 2011/10/12 Paul Baumann <[hidden email]>: > Hernán, > > There are several ways to do it with SRP. > > The encoding supports embedded data (variable and fixed width), so you could do inline serialization. It is clever enough that it is even possible to do this with positional access even to base64 encoded data. Tricks like that are used for example to update individual values in a persistent index. There are simpler techniques that should work for any marshaler... > > All marshalers should have some form of pre-save and post-load exchange. SRP has many ways to pre-save and post-load while maintaining relative references. SRP can even do post-load exchanges after the rest of the object graph is loaded and still keep relative references (without using a #become:). > > Save a proxy in place of the object. The proxy would be a class that you create to do what you want as a post-load actions. You can create the proxy as part of a pre-save action. The proxy can have the encoded binary ByteArray you want embedded in the data. The post-load of the proxy would typically unmarshal the embedded data. Here is an example of a class-based exchange: > > SpecialSaveObject>>srpSavePreMap: aContext > ^MyOtherMarshalProxy newFor: self > > MyOtherMarshalerProxy.binData > > MyOtherMarshalerProxy class>>newFor: anObject > ^self new save: anObject > > MyOtherMarshalProxy>>save: anObject > binData := self marshaler saveAsBinary: anObject. > > MyOtherMarshalerProxy>>srpLoadPostMap: aContext > ^OtherMarshaler loadFromBinary: binData > > Depending on your needs, you might want your object to remember the embedded data. In that case, you could store the encoded data in an inst var that is lazy-initialized as a pre-save action and save only that inst var. A post-load action could perhaps read the other attributes from the encoded data that you saved. > > SRP also allows any object to answer back "extended attributes" that are saved with an object (#srpExtendedAttributes) and can be restored after loading (#srpExtendedAttributes:). Extended attributes are typically data you want associated with an object that is not directly referenced by the object. You could save/load encoded data along with the SRP saved object. > > Paul Baumann > > > > -----Original Message----- > From: [hidden email] [mailto:[hidden email]] On Behalf Of Hernán Morales Durand > Sent: Monday, October 10, 2011 15:36 > To: Pharo Development; The general-purpose Squeak developers list; vwnc mailing list > Subject: [vwnc] Any serializer can customize serialization strategy? > > Is there any serializer (Fuel/StOMP/Ma object > serialization/BOSS/SRP/SIXX??) which let me attach an external > compressor/decompressor for specific Strings to the > serialization/deserialization process? With external I mean it could > be in a separate binary (I could arrange the binding though). > Thanks > > Hernán > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired. > This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired. _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
In reply to this post by hernanmd
Hi,
Hernán Morales Durand <[hidden email]> wrote: >Is there any serializer (Fuel/StOMP/Ma object >serialization/BOSS/SRP/SIXX??) which let me attach an external >compressor/decompressor for specific Strings to the >serialization/deserialization process? Mariano Martinez Peck <[hidden email]> wrote: > I think StOMP should support this as well: > http://stomp.smalltalk-users.jp/home/how-to-use-stomp/hook-methods Yes. StOMP does support such kind of custom serialization. Pattern 1: Using Memento. StOMP has two basic hook methods (stompWriteValue/stompReadValue). With a memento pattern, you can customize serialization/deserialization behavior easily. SpecificString>> stompWriteValue ^ StringCompressedMemento on: self StringCompressedMemento>> stompReadValue ^ SpecificString on: self uncompressToString SpecificString is a mere wrapper class (holding original string). And StringCompressedMemento should implement compress/uncompress logic. Pattern 2: Subclassing Writer/Reader Pattern 1 is usable, but it is basically suitable for pointer objects. For String-like objects, defining a new String wrapper and Memento is overwhelming. You can also subclass StompWriter/Reader for adding custom serialization/deserialization behaviors. MyStompWriter>>writeString: aString (self isSpecificString: aString) ifTrue: [^super writeString: (self compressString: aString)]. ^ super writeString: string MyStompReader>>readByteString | rawString | rawString := self basicReadObject asString. ^ (self isCompressedString: rawString) ifTrue: [self uncompressString: rawString] ifFalse: [rawString] You have to add some marking headers to the original string for determining it should be compressed/uncompressed. (Yes. this is a little awkward, but works well). I've attached working sample codes. So please see for details. Best regards, -- [:masashi | ^umezawa] _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Free forum by Nabble | Edit this page |