Smalltalk › Pharo › Pharo Smalltalk Developers

binary serialization

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

29 messages Options

Robert Withers

binary serialization

Does Pharo have stream classes to binary de/serialize an object, such
that the protocol accepts an object as an argument and converts it to a
byteArray?

--
thanks,
Robert

Sven Van Caekenberghe-2

Re: binary serialization

Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer.

> On 13 Oct 2015, at 06:59, Robert Withers <[hidden email]> wrote:
>
> Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray?
>
> --
> thanks,
> Robert
>

Robert Withers

Re: binary serialization

Those are the most beautiful words I've heard today.

thanks to you, brother,
Robert

On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote:

> Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer.
>
>> On 13 Oct 2015, at 06:59, Robert Withers <[hidden email]> wrote:
>>
>> Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray?
>>
>> --
>> thanks,
>> Robert
>>
>
>

Robert Withers

Re: binary serialization

In reply to this post by Sven Van Caekenberghe-2

Sven and Torsten, that's a binary serialization library! It will take
time to learn it and how to use mappers.

What is the format; is it language neutral?

thanks,
Robert

On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote:

Max Leske

Re: binary serialization

> On 13 Oct 2015, at 09:40, Robert Withers <[hidden email]> wrote:
>
> Sven and Torsten, that's a binary serialization library! It will take time to learn it and how to use mappers.
>
> What is the format; is it language neutral?

For quick serialization you don’t need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff.

Documentation: http://rmod.inria.fr/web/software/Fuel.

>
> thanks,
> Robert
>
> On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote:
>> Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer.
>>
>>> On 13 Oct 2015, at 06:59, Robert Withers <[hidden email]> wrote:
>>>
>>> Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray?
>>>
>>> --
>>> thanks,
>>> Robert
>>>
>>
>>
>

Robert Withers

Re: binary serialization

Yes, I have to do object substitutions. Thanks for the link!

thanks,
Robert

On 10/13/2015 03:43 AM, Max Leske wrote:

>
>> On 13 Oct 2015, at 09:40, Robert Withers <[hidden email]> wrote:
>>
>> Sven and Torsten, that's a binary serialization library! It will take time to learn it and how to use mappers.
>>
>> What is the format; is it language neutral?
>
> For quick serialization you don’t need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff.
>
> Documentation: http://rmod.inria.fr/web/software/Fuel.
>
>
>>
>> thanks,
>> Robert
>>
>> On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote:
>>> Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer.
>>>
>>>> On 13 Oct 2015, at 06:59, Robert Withers <[hidden email]> wrote:
>>>>
>>>> Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray?
>>>>
>>>> --
>>>> thanks,
>>>> Robert
>>>>
>>>
>>>
>>
>
>

Torsten Bergmann

Re: binary serialization

Hi Robert,

Also checkout the chapter on Fuel in Pharo Enterprise book:

https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/EnterprisePharo-A4.pdf

Bye
Torsten

> Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr
> Von: "Robert Withers" <[hidden email]>
> An: [hidden email]
> Betreff: Re: [Pharo-dev] binary serialization
>
> Yes, I have to do object substitutions. Thanks for the link!
>
> thanks,
> Robert
>
> On 10/13/2015 03:43 AM, Max Leske wrote:
> >
> >> On 13 Oct 2015, at 09:40, Robert Withers <[hidden email]> wrote:
> >>
> >> Sven and Torsten, that's a binary serialization library! It will take time to learn it and how to use mappers.
> >>
> >> What is the format; is it language neutral?
> >
> > For quick serialization you don’t need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff.
> >
> > Documentation: http://rmod.inria.fr/web/software/Fuel.
> >
> >
> >>
> >> thanks,
> >> Robert
> >>
> >> On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote:
> >>> Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer.
> >>>
> >>>> On 13 Oct 2015, at 06:59, Robert Withers <[hidden email]> wrote:
> >>>>
> >>>> Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray?
> >>>>
> >>>> --
> >>>> thanks,
> >>>> Robert
> >>>>
> >>>
> >>>
> >>
> >
> >
>
>

Mariano Martinez Peck

Re: binary serialization

Hi Robert,

As for the documentation, you have LOTS of tests, you have the chapter Torsten pasted, you have this documentation: http://rmod.inria.fr/web/software/Fuel

But also, as for internals, there is a journal paper we wrote: http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf

Let us know how it goes,

On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <[hidden email]> wrote:

Hi Robert,

Also checkout the chapter on Fuel in Pharo Enterprise book:

https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/EnterprisePharo-A4.pdf

Bye
Torsten

> Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr
> Von: "Robert Withers" <[hidden email]>
> An: [hidden email]
> Betreff: Re: [Pharo-dev] binary serialization

>
> Yes, I have to do object substitutions. Thanks for the link!
>
> thanks,
> Robert
>
> On 10/13/2015 03:43 AM, Max Leske wrote:
> >
> >> On 13 Oct 2015, at 09:40, Robert Withers <[hidden email]> wrote:
> >>
> >> Sven and Torsten, that's a binary serialization library! It will take time to learn it and how to use mappers.
> >>
> >> What is the format; is it language neutral?
> >
> > For quick serialization you don’t need to do anything. It works for (almost) all objects. Only if you want to exclude things or treat some objects in a special way, you will need to do some stuff.
> >
> > Documentation: http://rmod.inria.fr/web/software/Fuel.
> >
> >
> >>
> >> thanks,
> >> Robert
> >>
> >> On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote:
> >>> Yes, it is called FUEL and it is a standard part of the image. See FLSerializer and FLMaterializer.
> >>>
> >>>> On 13 Oct 2015, at 06:59, Robert Withers <[hidden email]> wrote:
> >>>>
> >>>> Does Pharo have stream classes to binary de/serialize an object, such that the protocol accepts an object as an argument and converts it to a byteArray?
> >>>>
> >>>> --
> >>>> thanks,
> >>>> Robert
> >>>>
> >>>
> >>>
> >>
> >
> >
>
>

Mariano
http://marianopeck.wordpress.com

Robert Withers

Re: binary serialization

Every extra source helps, thank you. I see how to do non-stream
substitutions on materializations, but the documentation did not
indicate a way to do non-stream substitutions on serialization. Is it
possible?

thanks,
Robert

On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:

> Hi Robert,
>
> As for the documentation, you have LOTS of tests, you have the chapter
> Torsten pasted, you have this documentation:
> http://rmod.inria.fr/web/software/Fuel
>
> But also, as for internals, there is a journal paper we wrote:
> http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
>
> Let us know how it goes,
>
>
> On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <[hidden email]
> <mailto:[hidden email]>> wrote:
>
> Hi Robert,
>
> Also checkout the chapter on Fuel in Pharo Enterprise book:
>
> https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/EnterprisePharo-A4.pdf
>
> Bye
> Torsten
>
> > Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr
> > Von: "Robert Withers" <[hidden email]
> <mailto:[hidden email]>>
> > An: [hidden email] <mailto:[hidden email]>
> > Betreff: Re: [Pharo-dev] binary serialization
> >
> > Yes, I have to do object substitutions. Thanks for the link!
> >
> > thanks,
> > Robert
> >
> > On 10/13/2015 03:43 AM, Max Leske wrote:
> > >
> > >> On 13 Oct 2015, at 09:40, Robert Withers
> <[hidden email] <mailto:[hidden email]>> wrote:
> > >>
> > >> Sven and Torsten, that's a binary serialization library! It
> will take time to learn it and how to use mappers.
> > >>
> > >> What is the format; is it language neutral?
> > >
> > > For quick serialization you don’t need to do anything. It works
> for (almost) all objects. Only if you want to exclude things or
> treat some objects in a special way, you will need to do some stuff.
> > >
> > > Documentation: http://rmod.inria.fr/web/software/Fuel.
> > >
> > >
> > >>
> > >> thanks,
> > >> Robert
> > >>
> > >> On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote:
> > >>> Yes, it is called FUEL and it is a standard part of the
> image. See FLSerializer and FLMaterializer.
> > >>>
> > >>>> On 13 Oct 2015, at 06:59, Robert Withers
> <[hidden email] <mailto:[hidden email]>> wrote:
> > >>>>
> > >>>> Does Pharo have stream classes to binary de/serialize an
> object, such that the protocol accepts an object as an argument and
> converts it to a byteArray?
> > >>>>
> > >>>> --
> > >>>> thanks,
> > >>>> Robert
> > >>>>
> > >>>
> > >>>
> > >>
> > >
> > >
> >
> >
>
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com

Max Leske

Re: binary serialization

> On 13 Oct 2015, at 17:16, Robert Withers <[hidden email]> wrote:
>
> Every extra source helps, thank you. I see how to do non-stream substitutions on materializations, but the documentation did not indicate a way to do non-stream substitutions on serialization. Is it possible?

I don’t understand what you mean by “non-stream”. Could you give an example?

>
> thanks,
> Robert
>
> On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
>> Hi Robert,
>>
>> As for the documentation, you have LOTS of tests, you have the chapter
>> Torsten pasted, you have this documentation:
>> http://rmod.inria.fr/web/software/Fuel
>>
>> But also, as for internals, there is a journal paper we wrote:
>> http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
>>
>> Let us know how it goes,
>>
>>
>> On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>> Hi Robert,
>>
>> Also checkout the chapter on Fuel in Pharo Enterprise book:
>>
>> https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/EnterprisePharo-A4.pdf
>>
>> Bye
>> Torsten
>>
>> > Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr
>> > Von: "Robert Withers" <[hidden email]
>> <mailto:[hidden email]>>
>> > An: [hidden email] <mailto:[hidden email]>
>> > Betreff: Re: [Pharo-dev] binary serialization
>> >
>> > Yes, I have to do object substitutions. Thanks for the link!
>> >
>> > thanks,
>> > Robert
>> >
>> > On 10/13/2015 03:43 AM, Max Leske wrote:
>> > >
>> > >> On 13 Oct 2015, at 09:40, Robert Withers
>> <[hidden email] <mailto:[hidden email]>> wrote:
>> > >>
>> > >> Sven and Torsten, that's a binary serialization library! It
>> will take time to learn it and how to use mappers.
>> > >>
>> > >> What is the format; is it language neutral?
>> > >
>> > > For quick serialization you don’t need to do anything. It works
>> for (almost) all objects. Only if you want to exclude things or
>> treat some objects in a special way, you will need to do some stuff.
>> > >
>> > > Documentation: http://rmod.inria.fr/web/software/Fuel.
>> > >
>> > >
>> > >>
>> > >> thanks,
>> > >> Robert
>> > >>
>> > >> On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote:
>> > >>> Yes, it is called FUEL and it is a standard part of the
>> image. See FLSerializer and FLMaterializer.
>> > >>>
>> > >>>> On 13 Oct 2015, at 06:59, Robert Withers
>> <[hidden email] <mailto:[hidden email]>> wrote:
>> > >>>>
>> > >>>> Does Pharo have stream classes to binary de/serialize an
>> object, such that the protocol accepts an object as an argument and
>> converts it to a byteArray?
>> > >>>>
>> > >>>> --
>> > >>>> thanks,
>> > >>>> Robert
>> > >>>>
>> > >>>
>> > >>>
>> > >>
>> > >
>> > >
>> >
>> >
>>
>>
>>
>>
>> --
>> Mariano
>> http://marianopeck.wordpress.com
>

Robert Withers

Re: binary serialization

Hi Max,

Yes, I was referring to this page:
http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingGraph,
though I had it backwards (and thought I might have) and got confused as
the example is about a stream. I meant that this page demonstrates a
way to register a substitution rule with the analyzer on serialization,
rather than implement #fuelAccepts: with a mapper.

I do not see a use with the analyzer on materialization. Is there such?

thanks,
Robert

On 10/13/2015 11:40 AM, Max Leske wrote:

>
>> On 13 Oct 2015, at 17:16, Robert Withers <[hidden email]> wrote:
>>
>> Every extra source helps, thank you. I see how to do non-stream substitutions on materializations, but the documentation did not indicate a way to do non-stream substitutions on serialization. Is it possible?
>
> I don’t understand what you mean by “non-stream”. Could you give an example?
>
>>
>> thanks,
>> Robert
>>
>> On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
>>> Hi Robert,
>>>
>>> As for the documentation, you have LOTS of tests, you have the chapter
>>> Torsten pasted, you have this documentation:
>>> http://rmod.inria.fr/web/software/Fuel
>>>
>>> But also, as for internals, there is a journal paper we wrote:
>>> http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
>>>
>>> Let us know how it goes,
>>>
>>>
>>> On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <[hidden email]
>>> <mailto:[hidden email]>> wrote:
>>>
>>> Hi Robert,
>>>
>>> Also checkout the chapter on Fuel in Pharo Enterprise book:
>>>
>>> https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/EnterprisePharo-A4.pdf
>>>
>>> Bye
>>> Torsten
>>>
>>> > Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr
>>> > Von: "Robert Withers" <[hidden email]
>>> <mailto:[hidden email]>>
>>> > An: [hidden email] <mailto:[hidden email]>
>>> > Betreff: Re: [Pharo-dev] binary serialization
>>> >
>>> > Yes, I have to do object substitutions. Thanks for the link!
>>> >
>>> > thanks,
>>> > Robert
>>> >
>>> > On 10/13/2015 03:43 AM, Max Leske wrote:
>>> > >
>>> > >> On 13 Oct 2015, at 09:40, Robert Withers
>>> <[hidden email] <mailto:[hidden email]>> wrote:
>>> > >>
>>> > >> Sven and Torsten, that's a binary serialization library! It
>>> will take time to learn it and how to use mappers.
>>> > >>
>>> > >> What is the format; is it language neutral?
>>> > >
>>> > > For quick serialization you don’t need to do anything. It works
>>> for (almost) all objects. Only if you want to exclude things or
>>> treat some objects in a special way, you will need to do some stuff.
>>> > >
>>> > > Documentation: http://rmod.inria.fr/web/software/Fuel.
>>> > >
>>> > >
>>> > >>
>>> > >> thanks,
>>> > >> Robert
>>> > >>
>>> > >> On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote:
>>> > >>> Yes, it is called FUEL and it is a standard part of the
>>> image. See FLSerializer and FLMaterializer.
>>> > >>>
>>> > >>>> On 13 Oct 2015, at 06:59, Robert Withers
>>> <[hidden email] <mailto:[hidden email]>> wrote:
>>> > >>>>
>>> > >>>> Does Pharo have stream classes to binary de/serialize an
>>> object, such that the protocol accepts an object as an argument and
>>> converts it to a byteArray?
>>> > >>>>
>>> > >>>> --
>>> > >>>> thanks,
>>> > >>>> Robert
>>> > >>>>
>>> > >>>
>>> > >>>
>>> > >>
>>> > >
>>> > >
>>> >
>>> >
>>>
>>>
>>>
>>>
>>> --
>>> Mariano
>>> http://marianopeck.wordpress.com
>>
>
>

Robert Withers

Re: binary serialization

In reply to this post by Max Leske

Yes, I meant dynamic substitution on materialization, to use the correct
terminology.

thanks,
Robert

On 10/13/2015 11:40 AM, Max Leske wrote:

Mariano Martinez Peck

Re: binary serialization

No, unfortunately, as far as I can remember, we do not have that. There are another hooks you may use but only in certain scenarios (#fuelNew, #fuelAfterMaterialization, global sends, etc). But everything is listed in http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingGraph so if you didn't find anything of help in there there are chances there isn't anything.

Cheers,

On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers <[hidden email]> wrote:

Yes, I meant dynamic substitution on materialization, to use the correct terminology.

thanks,
Robert

On 10/13/2015 11:40 AM, Max Leske wrote:

On 13 Oct 2015, at 17:16, Robert Withers <[hidden email]> wrote:

Every extra source helps, thank you. I see how to do non-stream substitutions on materializations, but the documentation did not indicate a way to do non-stream substitutions on serialization. Is it possible?

I don’t understand what you mean by “non-stream”. Could you give an example?

thanks,
Robert

On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:

Hi Robert,

As for the documentation, you have LOTS of tests, you have the chapter
Torsten pasted, you have this documentation:
http://rmod.inria.fr/web/software/Fuel

But also, as for internals, there is a journal paper we wrote:
http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf

Let us know how it goes,

On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann <[hidden email]
<mailto:[hidden email]>> wrote:

Hi Robert,

Also checkout the chapter on Fuel in Pharo Enterprise book:

https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/EnterprisePharo-A4.pdf

Bye
Torsten

> Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr
> Von: "Robert Withers" <[hidden email]
<mailto:[hidden email]>>
> An: [hidden email] <mailto:[hidden email]>
> Betreff: Re: [Pharo-dev] binary serialization
>
> Yes, I have to do object substitutions. Thanks for the link!
>
> thanks,
> Robert
>
> On 10/13/2015 03:43 AM, Max Leske wrote:
> >
> >> On 13 Oct 2015, at 09:40, Robert Withers
<[hidden email] <mailto:[hidden email]>> wrote:
> >>
> >> Sven and Torsten, that's a binary serialization library! It
will take time to learn it and how to use mappers.
> >>
> >> What is the format; is it language neutral?
> >
> > For quick serialization you don’t need to do anything. It works
for (almost) all objects. Only if you want to exclude things or
treat some objects in a special way, you will need to do some stuff.
> >
> > Documentation: http://rmod.inria.fr/web/software/Fuel.
> >
> >
> >>
> >> thanks,
> >> Robert
> >>
> >> On 10/13/2015 01:21 AM, Sven Van Caekenberghe wrote:
> >>> Yes, it is called FUEL and it is a standard part of the
image. See FLSerializer and FLMaterializer.
> >>>
> >>>> On 13 Oct 2015, at 06:59, Robert Withers
<[hidden email] <mailto:[hidden email]>> wrote:
> >>>>
> >>>> Does Pharo have stream classes to binary de/serialize an
object, such that the protocol accepts an object as an argument and
converts it to a byteArray?
> >>>>
> >>>> --
> >>>> thanks,
> >>>> Robert
> >>>>
> >>>
> >>>
> >>
> >
> >
>
>

--
Mariano
http://marianopeck.wordpress.com

Mariano
http://marianopeck.wordpress.com

Robert Withers

Re: binary serialization

Hi Mariano,

This presents me with a big challenge, then. I read the docs and
explored the code and the only other aspect not mentioned, beyond
instance creation (#fuelNew #fuelNew:) and postMaterialization
(#fuelAfterMaterialization), is migrations. However, migration only
allows for instanceVar mappings, no code blocks.

#fuelAccept: is a serialization side method.

If Fuel supports substitution on serialization, I don't understand why
no substitution support on materialization.

I am definitely going to use the world-class Fuel binary serialization
system. However, I find myself needing to extend Fuel to support
substitution on materialization. Perhaps the solution is a custom decoder.

No, a bit more. It looks like I need a new
FLSubstitutePointerObjectCluster, write them on serialization with the
substitute, then do unsubstitution on materialization, since the cluster
controls materialization and not the decoder.

Does this approach seem sound to you, from a you know architecture and
design approach?

thanks so much ^^
Robert

On 10/13/2015 04:49 PM, Mariano Martinez Peck wrote:

> No, unfortunately, as far as I can remember, we do not have that. There
> are another hooks you may use but only in certain scenarios (#fuelNew,
> #fuelAfterMaterialization, global sends, etc). But everything is listed
> in
> http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingGraph
> so if you didn't find anything of help in there there are chances
> there isn't anything.
>
> Cheers,
>
> On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers
> <[hidden email] <mailto:[hidden email]>> wrote:
>
> Yes, I meant dynamic substitution on materialization, to use the
> correct terminology.
>
> thanks,
> Robert
>
>
> On 10/13/2015 11:40 AM, Max Leske wrote:
>
>
> On 13 Oct 2015, at 17:16, Robert Withers
> <[hidden email]
> <mailto:[hidden email]>> wrote:
>
> Every extra source helps, thank you. I see how to do
> non-stream substitutions on materializations, but the
> documentation did not indicate a way to do non-stream
> substitutions on serialization. Is it possible?
>
>
> I don’t understand what you mean by “non-stream”. Could you give
> an example?
>
>
> thanks,
> Robert
>
> On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
>
> Hi Robert,
>
> As for the documentation, you have LOTS of tests, you
> have the chapter
> Torsten pasted, you have this documentation:
> http://rmod.inria.fr/web/software/Fuel
>
> But also, as for internals, there is a journal paper we
> wrote:
> http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
>
> Let us know how it goes,
>
>
> On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann
> <[hidden email] <mailto:[hidden email]>
> <mailto:[hidden email] <mailto:[hidden email]>>> wrote:
>
> Hi Robert,
>
> Also checkout the chapter on Fuel in Pharo
> Enterprise book:
>
> https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/EnterprisePharo-A4.pdf
>
> Bye
> Torsten
>
> > Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr
> > Von: "Robert Withers"
> <[hidden email]
> <mailto:[hidden email]>
> <mailto:[hidden email]
> <mailto:[hidden email]>>>
> > An: [hidden email]
> <mailto:[hidden email]>
> <mailto:[hidden email]
> <mailto:[hidden email]>>
> > Betreff: Re: [Pharo-dev] binary serialization
> >
> > Yes, I have to do object substitutions. Thanks
> for the link!
> >
> > thanks,
> > Robert
> >
> > On 10/13/2015 03:43 AM, Max Leske wrote:
> > >
> > >> On 13 Oct 2015, at 09:40, Robert Withers
> <[hidden email]
> <mailto:[hidden email]>
> <mailto:[hidden email]
> <mailto:[hidden email]>>> wrote:
> > >>
> > >> Sven and Torsten, that's a binary
> serialization library! It
> will take time to learn it and how to use mappers.
> > >>
> > >> What is the format; is it language neutral?
> > >
> > > For quick serialization you don’t need to do
> anything. It works
> for (almost) all objects. Only if you want to
> exclude things or
> treat some objects in a special way, you will need
> to do some stuff.
> > >
> > > Documentation:
> http://rmod.inria.fr/web/software/Fuel.
> > >
> > >
> > >>
> > >> thanks,
> > >> Robert
> > >>
> > >> On 10/13/2015 01:21 AM, Sven Van Caekenberghe
> wrote:
> > >>> Yes, it is called FUEL and it is a standard
> part of the
> image. See FLSerializer and FLMaterializer.
> > >>>
> > >>>> On 13 Oct 2015, at 06:59, Robert Withers
> <[hidden email]
> <mailto:[hidden email]>
> <mailto:[hidden email]
> <mailto:[hidden email]>>> wrote:
> > >>>>
> > >>>> Does Pharo have stream classes to binary
> de/serialize an
> object, such that the protocol accepts an object as
> an argument and
> converts it to a byteArray?
> > >>>>
> > >>>> --
> > >>>> thanks,
> > >>>> Robert
> > >>>>
> > >>>
> > >>>
> > >>
> > >
> > >
> >
> >
>
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
>
>
>
>
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com

Mariano Martinez Peck

Re: binary serialization

On Tue, Oct 13, 2015 at 10:33 PM, Robert Withers <[hidden email]> wrote:

Hi Mariano,

This presents me with a big challenge, then. I read the docs and explored the code and the only other aspect not mentioned, beyond instance creation (#fuelNew #fuelNew:) and postMaterialization (#fuelAfterMaterialization), is migrations. However, migration only allows for instanceVar mappings, no code blocks.

What do you mean that migrations only allows instVar mappings , and no code blocks? I mean, what do you mean by code blocks?

We also support class renames. This is here: http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Migration?_s=Pvs4DYqPRjfEy8sg&_k=X6ltJu7FDSxumm4X&_n&22

Which kind of migration example you have in mind that would not be supported? An example would help.

#fuelAccept: is a serialization side method.

If Fuel supports substitution on serialization, I don't understand why no substitution support on materialization.

There was a reason, which I cannot remember completely. Maybe Martin or Max can remember.

I am definitely going to use the world-class Fuel binary serialization system. However, I find myself needing to extend Fuel to support substitution on materialization. Perhaps the solution is a custom decoder.

I have made custom clusters for example for my Ghost proxies of Marea system. It was a perfect example of how I could extent Fuel besides the common hooks. Fuel provides many places for extending , like clusters, analyzer, etc

No, a bit more. It looks like I need a new FLSubstitutePointerObjectCluster, write them on serialization with the substitute, then do unsubstitution on materialization, since the cluster controls materialization and not the decoder.

Does this approach seem sound to you, from a you know architecture and design approach?

There was an issue. Hope other can remember. If not, I will try to explan what I remember tomorrow.

thanks so much ^^
Robert

On 10/13/2015 04:49 PM, Mariano Martinez Peck wrote:

No, unfortunately, as far as I can remember, we do not have that. There
are another hooks you may use but only in certain scenarios (#fuelNew,
#fuelAfterMaterialization, global sends, etc). But everything is listed
in
http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingGraph
so if you didn't find anything of help in there there are chances
there isn't anything.

Cheers,

On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers
<[hidden email] <mailto:[hidden email]>> wrote:

Yes, I meant dynamic substitution on materialization, to use the
correct terminology.

thanks,
Robert

On 10/13/2015 11:40 AM, Max Leske wrote:

On 13 Oct 2015, at 17:16, Robert Withers
<[hidden email]
<mailto:[hidden email]>> wrote:

Every extra source helps, thank you. I see how to do
non-stream substitutions on materializations, but the
documentation did not indicate a way to do non-stream
substitutions on serialization. Is it possible?

I don’t understand what you mean by “non-stream”. Could you give
an example?

thanks,
Robert

On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:

Hi Robert,

As for the documentation, you have LOTS of tests, you
have the chapter
Torsten pasted, you have this documentation:
http://rmod.inria.fr/web/software/Fuel

But also, as for internals, there is a journal paper we
wrote:
http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf

Let us know how it goes,

On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann
<[hidden email] <mailto:[hidden email]>
<mailto:[hidden email] <mailto:[hidden email]>>> wrote:

Hi Robert,

Also checkout the chapter on Fuel in Pharo
Enterprise book:

https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/EnterprisePharo-A4.pdf

Bye
Torsten

> Gesendet: Dienstag, 13. Oktober 2015 um 09:44 Uhr
> Von: "Robert Withers"
<[hidden email]
<mailto:[hidden email]>
<mailto:[hidden email]
<mailto:[hidden email]>>>
> An: [hidden email]
<mailto:[hidden email]>
<mailto:[hidden email]
<mailto:[hidden email]>>
> Betreff: Re: [Pharo-dev] binary serialization
>
> Yes, I have to do object substitutions. Thanks
for the link!
>
> thanks,
> Robert
>
> On 10/13/2015 03:43 AM, Max Leske wrote:
> >
> >> On 13 Oct 2015, at 09:40, Robert Withers
<[hidden email]
<mailto:[hidden email]>
<mailto:[hidden email]
<mailto:[hidden email]>>> wrote:
> >>
> >> Sven and Torsten, that's a binary
serialization library! It
will take time to learn it and how to use mappers.
> >>
> >> What is the format; is it language neutral?
> >
> > For quick serialization you don’t need to do
anything. It works
for (almost) all objects. Only if you want to
exclude things or
treat some objects in a special way, you will need
to do some stuff.
> >
> > Documentation:
http://rmod.inria.fr/web/software/Fuel.
> >
> >
> >>
> >> thanks,
> >> Robert
> >>
> >> On 10/13/2015 01:21 AM, Sven Van Caekenberghe
wrote:
> >>> Yes, it is called FUEL and it is a standard
part of the
image. See FLSerializer and FLMaterializer.
> >>>
> >>>> On 13 Oct 2015, at 06:59, Robert Withers
<[hidden email]
<mailto:[hidden email]>
<mailto:[hidden email]
<mailto:[hidden email]>>> wrote:
> >>>>
> >>>> Does Pharo have stream classes to binary
de/serialize an
object, such that the protocol accepts an object as
an argument and
converts it to a byteArray?
> >>>>
> >>>> --
> >>>> thanks,
> >>>> Robert
> >>>>
> >>>
> >>>
> >>
> >
> >
>
>

--
Mariano
http://marianopeck.wordpress.com

--
Mariano
http://marianopeck.wordpress.com

Mariano
http://marianopeck.wordpress.com

Robert Withers

Re: binary serialization

On 10/13/2015 09:43 PM, Mariano Martinez Peck wrote:

>
>
> On Tue, Oct 13, 2015 at 10:33 PM, Robert Withers
> <[hidden email] <mailto:[hidden email]>> wrote:
>
> Hi Mariano,
>
> This presents me with a big challenge, then. I read the docs and
> explored the code and the only other aspect not mentioned, beyond
> instance creation (#fuelNew #fuelNew:) and postMaterialization
> (#fuelAfterMaterialization), is migrations. However, migration only
> allows for instanceVar mappings, no code blocks.
>
>
> What do you mean that migrations only allows instVar mappings , and no
> code blocks? I mean, what do you mean by code blocks?
> We also support class renames. This is here:
> http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Migration?_s=Pvs4DYqPRjfEy8sg&_k=X6ltJu7FDSxumm4X&_n&22

> Which kind of migration example you have in mind that would not be
> supported? An example would help.

Well, my pics will demonstrate. I am interested in doing more than
mappping ivars or a class rename. I want to do a total substitution,
then a further substitution on the receiving, import side:

Vat1: anObject (Class A) ---> On wire: desc (Descriptor) ---> Vat2:
aProxy (Class FarERef)

A desc is substituted for a PassByProxy object, then a FarERef is
substituted for the desc.

>
> #fuelAccept: is a serialization side method.
>
> If Fuel supports substitution on serialization, I don't understand
> why no substitution support on materialization.
>
>
> There was a reason, which I cannot remember completely. Maybe Martin or
> Max can remember.

It seems your focus was pickling to disk then back. My focus is
distributed proxy graphs, which has different use cases.

>
>
> I am definitely going to use the world-class Fuel binary
> serialization system. However, I find myself needing to extend Fuel
> to support substitution on materialization. Perhaps the solution is
> a custom decoder.
>
>
> I have made custom clusters for example for my Ghost proxies of Marea
> system. It was a perfect example of how I could extent Fuel besides the
> common hooks. Fuel provides many places for extending , like clusters,
> analyzer, etc

Right on, exactly! Could you tell me more about your Ghost proxies and
Marea, please? As well, could you mention how you select a custom
cluster on the serialization side?

thanks so much ^^
Robert

>
> No, a bit more. It looks like I need a new
> FLSubstitutePointerObjectCluster, write them on serialization with
> the substitute, then do unsubstitution on materialization, since the
> cluster controls materialization and not the decoder.
>
> Does this approach seem sound to you, from a you know architecture
> and design approach?
>
>
> There was an issue. Hope other can remember. If not, I will try to
> explan what I remember tomorrow.
>
>
> thanks so much ^^
> Robert
>
> On 10/13/2015 04:49 PM, Mariano Martinez Peck wrote:
>
> No, unfortunately, as far as I can remember, we do not have
> that. There
> are another hooks you may use but only in certain scenarios
> (#fuelNew,
> #fuelAfterMaterialization, global sends, etc). But everything is
> listed
> in
> http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingGraph
> so if you didn't find anything of help in there there are chances
> there isn't anything.
>
> Cheers,
>
> On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers
> <[hidden email] <mailto:[hidden email]>
> <mailto:[hidden email]
> <mailto:[hidden email]>>> wrote:
>
> Yes, I meant dynamic substitution on materialization, to
> use the
> correct terminology.
>
> thanks,
> Robert
>
>
> On 10/13/2015 11:40 AM, Max Leske wrote:
>
>
> On 13 Oct 2015, at 17:16, Robert Withers
> <[hidden email]
> <mailto:[hidden email]>
> <mailto:[hidden email]
> <mailto:[hidden email]>>> wrote:
>
> Every extra source helps, thank you. I see how to do
> non-stream substitutions on materializations, but the
> documentation did not indicate a way to do non-stream
> substitutions on serialization. Is it possible?
>
>
> I don’t understand what you mean by “non-stream”. Could
> you give
> an example?
>
>
> thanks,
> Robert
>
> On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
>
> Hi Robert,
>
> As for the documentation, you have LOTS of
> tests, you
> have the chapter
> Torsten pasted, you have this documentation:
> http://rmod.inria.fr/web/software/Fuel
>
> But also, as for internals, there is a journal
> paper we
> wrote:
> http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
>
> Let us know how it goes,
>
>
> On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann
> <[hidden email] <mailto:[hidden email]>
> <mailto:[hidden email] <mailto:[hidden email]>>
> <mailto:[hidden email] <mailto:[hidden email]>
> <mailto:[hidden email] <mailto:[hidden email]>>>> wrote:
>
> Hi Robert,
>
> Also checkout the chapter on Fuel in Pharo
> Enterprise book:
>
> https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/EnterprisePharo-A4.pdf
>
> Bye
> Torsten
>
> > Gesendet: Dienstag, 13. Oktober 2015 um
> 09:44 Uhr
> > Von: "Robert Withers"
> <[hidden email]
> <mailto:[hidden email]>
> <mailto:[hidden email]
> <mailto:[hidden email]>>
> <mailto:[hidden email]
> <mailto:[hidden email]>
> <mailto:[hidden email]
> <mailto:[hidden email]>>>>
> > An: [hidden email]
> <mailto:[hidden email]>
> <mailto:[hidden email]
> <mailto:[hidden email]>>
> <mailto:[hidden email]
> <mailto:[hidden email]>
> <mailto:[hidden email]
> <mailto:[hidden email]>>>
> > Betreff: Re: [Pharo-dev] binary
> serialization
> >
> > Yes, I have to do object substitutions.
> Thanks
> for the link!
> >
> > thanks,
> > Robert
> >
> > On 10/13/2015 03:43 AM, Max Leske wrote:
> > >
> > >> On 13 Oct 2015, at 09:40, Robert Withers
> <[hidden email]
> <mailto:[hidden email]>
> <mailto:[hidden email]
> <mailto:[hidden email]>>
> <mailto:[hidden email]
> <mailto:[hidden email]>
> <mailto:[hidden email]
> <mailto:[hidden email]>>>> wrote:
> > >>
> > >> Sven and Torsten, that's a binary
> serialization library! It
> will take time to learn it and how to use
> mappers.
> > >>
> > >> What is the format; is it language
> neutral?
> > >
> > > For quick serialization you don’t
> need to do
> anything. It works
> for (almost) all objects. Only if you want to
> exclude things or
> treat some objects in a special way, you
> will need
> to do some stuff.
> > >
> > > Documentation:
> http://rmod.inria.fr/web/software/Fuel.
> > >
> > >
> > >>
> > >> thanks,
> > >> Robert
> > >>
> > >> On 10/13/2015 01:21 AM, Sven Van
> Caekenberghe
> wrote:
> > >>> Yes, it is called FUEL and it is a
> standard
> part of the
> image. See FLSerializer and FLMaterializer.
> > >>>
> > >>>> On 13 Oct 2015, at 06:59, Robert
> Withers
> <[hidden email]
> <mailto:[hidden email]>
> <mailto:[hidden email]
> <mailto:[hidden email]>>
> <mailto:[hidden email]
> <mailto:[hidden email]>
> <mailto:[hidden email]
> <mailto:[hidden email]>>>> wrote:
> > >>>>
> > >>>> Does Pharo have stream classes to
> binary
> de/serialize an
> object, such that the protocol accepts an
> object as
> an argument and
> converts it to a byteArray?
> > >>>>
> > >>>> --
> > >>>> thanks,
> > >>>> Robert
> > >>>>
> > >>>
> > >>>
> > >>
> > >
> > >
> >
> >
>
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
>
>
>
>
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
>
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com

Exporting Vat.jpg (57K) Download Attachment

Importing Vat.jpg (55K) Download Attachment

Max Leske

Re: binary serialization

> On 14 Oct 2015, at 04:39, Robert Withers <[hidden email]> wrote:
>
>
> On 10/13/2015 09:43 PM, Mariano Martinez Peck wrote:
>>
>>
>> On Tue, Oct 13, 2015 at 10:33 PM, Robert Withers
>> <[hidden email] <mailto:[hidden email]>> wrote:
>>
>> Hi Mariano,
>>
>> This presents me with a big challenge, then. I read the docs and
>> explored the code and the only other aspect not mentioned, beyond
>> instance creation (#fuelNew #fuelNew:) and postMaterialization
>> (#fuelAfterMaterialization), is migrations. However, migration only
>> allows for instanceVar mappings, no code blocks.
>>
>>
>> What do you mean that migrations only allows instVar mappings , and no
>> code blocks? I mean, what do you mean by code blocks?

Sounds to me like this (see FuelOutStackDebuAction):

serializeTestFailureContext: aContext toFileNamed: aFilename
| serializer |

serializer := FLSerializer newDefault.
self encodeDebugInformationOn: serializer.
serializer addPostMaterializationAction: [ :materialization |
Smalltalk tools debugger
openOn: Processor activeProcess
context: materialization root
label: 'External stack'
contents: nil
fullView: false ].

serializer
" use the sender context, generally the current context is not interesting"
serialize: aContext
toFileNamed: aFilename

This stores a block in the serialization which is evaluated after materialization. The only requirement is that it’s a clean block (no closure!).

>> We also support class renames. This is here:
>> http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Migration?_s=Pvs4DYqPRjfEy8sg&_k=X6ltJu7FDSxumm4X&_n&22
> >
> > Which kind of migration example you have in mind that would not be
> > supported? An example would help.
>
> Well, my pics will demonstrate. I am interested in doing more than mappping ivars or a class rename. I want to do a total substitution, then a further substitution on the receiving, import side:
>
> Vat1: anObject (Class A) ---> On wire: desc (Descriptor) ---> Vat2: aProxy (Class FarERef)
>
> A desc is substituted for a PassByProxy object, then a FarERef is substituted for the desc.
>
>>
>> #fuelAccept: is a serialization side method.
>>
>> If Fuel supports substitution on serialization, I don't understand
>> why no substitution support on materialization.
>>
>>
>> There was a reason, which I cannot remember completely. Maybe Martin or
>> Max can remember.
>
> It seems your focus was pickling to disk then back. My focus is distributed proxy graphs, which has different use cases.
>
>>
>>
>> I am definitely going to use the world-class Fuel binary
>> serialization system. However, I find myself needing to extend Fuel
>> to support substitution on materialization. Perhaps the solution is
>> a custom decoder.
>>
>>
>> I have made custom clusters for example for my Ghost proxies of Marea
>> system. It was a perfect example of how I could extent Fuel besides the
>> common hooks. Fuel provides many places for extending , like clusters,
>> analyzer, etc
>
> Right on, exactly! Could you tell me more about your Ghost proxies and Marea, please? As well, could you mention how you select a custom cluster on the serialization side?
>
>
> thanks so much ^^
> Robert
>
>
>>
>> No, a bit more. It looks like I need a new
>> FLSubstitutePointerObjectCluster, write them on serialization with
>> the substitute, then do unsubstitution on materialization, since the
>> cluster controls materialization and not the decoder.
>>
>> Does this approach seem sound to you, from a you know architecture
>> and design approach?
>>
>>
>> There was an issue. Hope other can remember. If not, I will try to
>> explan what I remember tomorrow.
>>
>>
>> thanks so much ^^
>> Robert
>>
>> On 10/13/2015 04:49 PM, Mariano Martinez Peck wrote:
>>
>> No, unfortunately, as far as I can remember, we do not have
>> that. There
>> are another hooks you may use but only in certain scenarios
>> (#fuelNew,
>> #fuelAfterMaterialization, global sends, etc). But everything is
>> listed
>> in
>> http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingGraph
>> so if you didn't find anything of help in there there are chances
>> there isn't anything.
>>
>> Cheers,
>>
>> On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers
>> <[hidden email] <mailto:[hidden email]>
>> <mailto:[hidden email]
>> <mailto:[hidden email]>>> wrote:
>>
>> Yes, I meant dynamic substitution on materialization, to
>> use the
>> correct terminology.
>>
>> thanks,
>> Robert
>>
>>
>> On 10/13/2015 11:40 AM, Max Leske wrote:
>>
>>
>> On 13 Oct 2015, at 17:16, Robert Withers
>> <[hidden email]
>> <mailto:[hidden email]>
>> <mailto:[hidden email]
>> <mailto:[hidden email]>>> wrote:
>>
>> Every extra source helps, thank you. I see how to do
>> non-stream substitutions on materializations, but the
>> documentation did not indicate a way to do non-stream
>> substitutions on serialization. Is it possible?
>>
>>
>> I don’t understand what you mean by “non-stream”. Could
>> you give
>> an example?
>>
>>
>> thanks,
>> Robert
>>
>> On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
>>
>> Hi Robert,
>>
>> As for the documentation, you have LOTS of
>> tests, you
>> have the chapter
>> Torsten pasted, you have this documentation:
>> http://rmod.inria.fr/web/software/Fuel
>>
>> But also, as for internals, there is a journal
>> paper we
>> wrote:
>> http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
>>
>> Let us know how it goes,
>>
>>
>> On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann
>> <[hidden email] <mailto:[hidden email]>
>> <mailto:[hidden email] <mailto:[hidden email]>>
>> <mailto:[hidden email] <mailto:[hidden email]>
>> <mailto:[hidden email] <mailto:[hidden email]>>>> wrote:
>>
>> Hi Robert,
>>
>> Also checkout the chapter on Fuel in Pharo
>> Enterprise book:
>>
>> https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/EnterprisePharo-A4.pdf
>>
>> Bye
>> Torsten
>>
>> > Gesendet: Dienstag, 13. Oktober 2015 um
>> 09:44 Uhr
>> > Von: "Robert Withers"
>> <[hidden email]
>> <mailto:[hidden email]>
>> <mailto:[hidden email]
>> <mailto:[hidden email]>>
>> <mailto:[hidden email]
>> <mailto:[hidden email]>
>> <mailto:[hidden email]
>> <mailto:[hidden email]>>>>
>> > An: [hidden email]
>> <mailto:[hidden email]>
>> <mailto:[hidden email]
>> <mailto:[hidden email]>>
>> <mailto:[hidden email]
>> <mailto:[hidden email]>
>> <mailto:[hidden email]
>> <mailto:[hidden email]>>>
>> > Betreff: Re: [Pharo-dev] binary
>> serialization
>> >
>> > Yes, I have to do object substitutions.
>> Thanks
>> for the link!
>> >
>> > thanks,
>> > Robert
>> >
>> > On 10/13/2015 03:43 AM, Max Leske wrote:
>> > >
>> > >> On 13 Oct 2015, at 09:40, Robert Withers
>> <[hidden email]
>> <mailto:[hidden email]>
>> <mailto:[hidden email]
>> <mailto:[hidden email]>>
>> <mailto:[hidden email]
>> <mailto:[hidden email]>
>> <mailto:[hidden email]
>> <mailto:[hidden email]>>>> wrote:
>> > >>
>> > >> Sven and Torsten, that's a binary
>> serialization library! It
>> will take time to learn it and how to use
>> mappers.
>> > >>
>> > >> What is the format; is it language
>> neutral?
>> > >
>> > > For quick serialization you don’t
>> need to do
>> anything. It works
>> for (almost) all objects. Only if you want to
>> exclude things or
>> treat some objects in a special way, you
>> will need
>> to do some stuff.
>> > >
>> > > Documentation:
>> http://rmod.inria.fr/web/software/Fuel.
>> > >
>> > >
>> > >>
>> > >> thanks,
>> > >> Robert
>> > >>
>> > >> On 10/13/2015 01:21 AM, Sven Van
>> Caekenberghe
>> wrote:
>> > >>> Yes, it is called FUEL and it is a
>> standard
>> part of the
>> image. See FLSerializer and FLMaterializer.
>> > >>>
>> > >>>> On 13 Oct 2015, at 06:59, Robert
>> Withers
>> <[hidden email]
>> <mailto:[hidden email]>
>> <mailto:[hidden email]
>> <mailto:[hidden email]>>
>> <mailto:[hidden email]
>> <mailto:[hidden email]>
>> <mailto:[hidden email]
>> <mailto:[hidden email]>>>> wrote:
>> > >>>>
>> > >>>> Does Pharo have stream classes to
>> binary
>> de/serialize an
>> object, such that the protocol accepts an
>> object as
>> an argument and
>> converts it to a byteArray?
>> > >>>>
>> > >>>> --
>> > >>>> thanks,
>> > >>>> Robert
>> > >>>>
>> > >>>
>> > >>>
>> > >>
>> > >
>> > >
>> >
>> >
>>
>>
>>
>>
>> --
>> Mariano
>> http://marianopeck.wordpress.com
>>
>>
>>
>>
>>
>>
>>
>>
>> --
>> Mariano
>> http://marianopeck.wordpress.com
>>
>>
>>
>>
>>
>> --
>> Mariano
>> http://marianopeck.wordpress.com
> <Exporting Vat.jpg><Importing Vat.jpg>

Max Leske

Re: binary serialization

BTW, there is a dedicated Fuel mailing list: [hidden email]

Max

On 14 Oct 2015, at 09:45, Max Leske <[hidden email]> wrote:

On 14 Oct 2015, at 04:39, Robert Withers <[hidden email]> wrote:

On 10/13/2015 09:43 PM, Mariano Martinez Peck wrote:

On Tue, Oct 13, 2015 at 10:33 PM, Robert Withers
<[hidden email] <[hidden email]>> wrote:

  Hi Mariano,

  This presents me with a big challenge, then. I read the docs and
  explored the code and the only other aspect not mentioned, beyond
  instance creation (#fuelNew #fuelNew:) and postMaterialization
  (#fuelAfterMaterialization), is migrations. However, migration only
  allows for instanceVar mappings, no code blocks.

What do you mean that migrations only allows instVar mappings , and no
code blocks? I mean, what do you mean by code blocks?

Sounds to me like this (see FuelOutStackDebuAction):

serializeTestFailureContext: aContext toFileNamed: aFilename
| serializer |

serializer := FLSerializer newDefault.
self encodeDebugInformationOn: serializer.
serializer addPostMaterializationAction: [ :materialization |
Smalltalk tools debugger
openOn: Processor activeProcess
context: materialization root
label: 'External stack'
contents: nil
fullView: false ].

serializer
" use the sender context, generally the current context is not interesting"
serialize: aContext
toFileNamed: aFilename

This stores a block in the serialization which is evaluated after materialization. The only requirement is that it’s a clean block (no closure!).

We also support class renames. This is here:
http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Migration?_s=Pvs4DYqPRjfEy8sg&_k=X6ltJu7FDSxumm4X&_n&22

Which kind of migration example you have in mind that would not be
supported? An example would help.

Well, my pics will demonstrate. I am interested in doing more than mappping ivars or a class rename. I want to do a total substitution, then a further substitution on the receiving, import side:

Vat1: anObject (Class A) ---> On wire: desc (Descriptor) ---> Vat2: aProxy (Class FarERef)

A desc is substituted for a PassByProxy object, then a FarERef is substituted for the desc.

  #fuelAccept: is a serialization side method.

  If Fuel supports substitution on serialization, I don't understand
  why no substitution support on materialization.

There was a reason, which I cannot remember completely. Maybe Martin or
Max can remember.

It seems your focus was pickling to disk then back. My focus is distributed proxy graphs, which has different use cases.

  I am definitely going to use the world-class Fuel binary
  serialization system. However, I find myself needing to extend Fuel
  to support substitution on materialization. Perhaps the solution is
  a custom decoder.

I have made custom clusters for example for my Ghost proxies of Marea
system. It was a perfect example of how I could extent Fuel besides the
common hooks. Fuel provides many places for extending , like clusters,
analyzer, etc

Right on, exactly! Could you tell me more about your Ghost proxies and Marea, please? As well, could you mention how you select a custom cluster on the serialization side?

thanks so much ^^
Robert

  No, a bit more. It looks like I need a new
  FLSubstitutePointerObjectCluster, write them on serialization with
  the substitute, then do unsubstitution on materialization, since the
  cluster controls materialization and not the decoder.

  Does this approach seem sound to you, from a you know architecture
  and design approach?

There was an issue. Hope other can remember. If not, I will try to
explan what I remember tomorrow.

  thanks so much ^^
  Robert

  On 10/13/2015 04:49 PM, Mariano Martinez Peck wrote:

      No, unfortunately, as far as I can remember, we do not have
      that. There
      are another hooks you may use but only in certain scenarios
      (#fuelNew,
      #fuelAfterMaterialization, global sends, etc). But everything is
      listed
      in
      http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingGraph
         so if you didn't find anything of help in there there are chances
      there isn't anything.

      Cheers,

      On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers
      <[hidden email] <[hidden email]>
      <[hidden email]
      <[hidden email]>>> wrote:

           Yes, I meant dynamic substitution on materialization, to
      use the
           correct terminology.

           thanks,
           Robert

           On 10/13/2015 11:40 AM, Max Leske wrote:

                   On 13 Oct 2015, at 17:16, Robert Withers
                   <[hidden email]
      <[hidden email]>
                   <[hidden email]
      <[hidden email]>>> wrote:

                   Every extra source helps, thank you. I see how to do
                   non-stream substitutions on materializations, but the
                   documentation did not indicate a way to do non-stream
                   substitutions on serialization. Is it possible?

               I don’t understand what you mean by “non-stream”. Could
      you give
               an example?

                   thanks,
                   Robert

                   On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:

                       Hi Robert,

                       As for the documentation, you have LOTS of
      tests, you
                       have the chapter
                       Torsten pasted, you have this documentation:
      http://rmod.inria.fr/web/software/Fuel

                       But also, as for internals, there is a journal
      paper we
                       wrote:
      http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf

                       Let us know how it goes,

                       On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann
                       <[hidden email] <[hidden email]>
      <[hidden email] <[hidden email]>>
                       <[hidden email] <[hidden email]>
      <[hidden email] <[hidden email]>>>> wrote:

                            Hi Robert,

                            Also checkout the chapter on Fuel in Pharo
                       Enterprise book:

      https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/EnterprisePharo-A4.pdf

                            Bye
                            Torsten

Gesendet: Dienstag, 13. Oktober 2015 um
      09:44 Uhr
Von: "Robert Withers"
                       <[hidden email]
      <[hidden email]>
                       <[hidden email]
      <[hidden email]>>
                            <[hidden email]
      <[hidden email]>
                       <[hidden email]
      <[hidden email]>>>>
An: [hidden email]
      <[hidden email]>
                       <[hidden email]
      <[hidden email]>>
                       <[hidden email]
      <[hidden email]>
                       <[hidden email]
      <[hidden email]>>>
Betreff: Re: [Pharo-dev] binary
      serialization

Yes, I have to do object substitutions.
      Thanks
                       for the link!

thanks,
Robert

On 10/13/2015 03:43 AM, Max Leske wrote:

On 13 Oct 2015, at 09:40, Robert Withers
                            <[hidden email]
      <[hidden email]>
                       <[hidden email]
      <[hidden email]>>
                       <[hidden email]
      <[hidden email]>
                       <[hidden email]
      <[hidden email]>>>> wrote:

Sven and Torsten, that's a binary
                       serialization library! It
                            will take time to learn it and how to use
      mappers.

What is the format; is it language
      neutral?

For quick serialization you don’t
      need to do
                       anything. It works
                            for (almost) all objects. Only if you want to
                       exclude things or
                            treat some objects in a special way, you
      will need
                       to do some stuff.

Documentation:
      http://rmod.inria.fr/web/software/Fuel.

thanks,
Robert

On 10/13/2015 01:21 AM, Sven Van
      Caekenberghe
                       wrote:
Yes, it is called FUEL and it is a
      standard
                       part of the
                            image. See FLSerializer and FLMaterializer.

On 13 Oct 2015, at 06:59, Robert
      Withers
                            <[hidden email]
      <[hidden email]>
                       <[hidden email]
      <[hidden email]>>
                       <[hidden email]
      <[hidden email]>
                       <[hidden email]
      <[hidden email]>>>> wrote:

Does Pharo have stream classes to
      binary
                       de/serialize an
                            object, such that the protocol accepts an
      object as
                       an argument and
                            converts it to a byteArray?

--
thanks,
Robert

                       --
                       Mariano
      http://marianopeck.wordpress.com

      --
      Mariano
      http://marianopeck.wordpress.com

--
Mariano
http://marianopeck.wordpress.com
<Exporting Vat.jpg><Importing Vat.jpg>

Robert Withers

Re: binary serialization

Good morning, Max. Thank you for the example. I got a little confused,
between migrations and substitutions. My issue with no-arg blocks, I
believe, is the inability to access my scope object to maintain the
object tables.

I'm attempting to write my own Materializer, Decoder and
Materialization. At the moment, I'm just going to walk the graph,
testing and do #becomes:. See how well that works when I get a test.

It's really good to know about that other list.

thanks so much ^^
Robert

On 10/14/2015 04:15 AM, Max Leske wrote:

> BTW, there is a dedicated Fuel mailing list:
> [hidden email] <mailto:[hidden email]>
>
> Max
>
>
>> On 14 Oct 2015, at 09:45, Max Leske <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>>>
>>> On 14 Oct 2015, at 04:39, Robert Withers <[hidden email]
>>> <mailto:[hidden email]>> wrote:
>>>
>>>
>>> On 10/13/2015 09:43 PM, Mariano Martinez Peck wrote:
>>>>
>>>>
>>>> On Tue, Oct 13, 2015 at 10:33 PM, Robert Withers
>>>> <[hidden email] <mailto:[hidden email]>
>>>> <mailto:[hidden email]>> wrote:
>>>>
>>>> Hi Mariano,
>>>>
>>>> This presents me with a big challenge, then. I read the docs and
>>>> explored the code and the only other aspect not mentioned, beyond
>>>> instance creation (#fuelNew #fuelNew:) and postMaterialization
>>>> (#fuelAfterMaterialization), is migrations. However, migration only
>>>> allows for instanceVar mappings, no code blocks.
>>>>
>>>>
>>>> What do you mean that migrations only allows instVar mappings , and no
>>>> code blocks? I mean, what do you mean by code blocks?
>>
>> Sounds to me like this (see FuelOutStackDebuAction):
>>
>> serializeTestFailureContext: aContext toFileNamed: aFilename
>> | serializer |
>>
>> serializer := FLSerializer newDefault.
>> self encodeDebugInformationOn: serializer.
>> serializer addPostMaterializationAction: [ :materialization |
>> Smalltalk tools debugger
>> openOn: Processor activeProcess
>> context: materialization root
>> label: 'External stack'
>> contents: nil
>> fullView: false ].
>>
>> serializer
>> " use the sender context, generally the current context is not
>> interesting"
>> serialize: aContext
>> toFileNamed: aFilename
>>
>> This stores a block in the serialization which is evaluated after
>> materialization. The only requirement is that it’s a clean block (no
>> closure!).
>>
>>>> We also support class renames. This is here:
>>>> http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Migration?_s=Pvs4DYqPRjfEy8sg&_k=X6ltJu7FDSxumm4X&_n&22
>>>>
>>>> Which kind of migration example you have in mind that would not be
>>>> supported? An example would help.
>>>
>>> Well, my pics will demonstrate. I am interested in doing more than
>>> mappping ivars or a class rename. I want to do a total substitution,
>>> then a further substitution on the receiving, import side:
>>>
>>> Vat1: anObject (Class A) ---> On wire: desc (Descriptor) ---> Vat2:
>>> aProxy (Class FarERef)
>>>
>>> A desc is substituted for a PassByProxy object, then a FarERef is
>>> substituted for the desc.
>>>
>>>>
>>>> #fuelAccept: is a serialization side method.
>>>>
>>>> If Fuel supports substitution on serialization, I don't understand
>>>> why no substitution support on materialization.
>>>>
>>>>
>>>> There was a reason, which I cannot remember completely. Maybe Martin or
>>>> Max can remember.
>>>
>>> It seems your focus was pickling to disk then back. My focus is
>>> distributed proxy graphs, which has different use cases.
>>>
>>>>
>>>>
>>>> I am definitely going to use the world-class Fuel binary
>>>> serialization system. However, I find myself needing to extend Fuel
>>>> to support substitution on materialization. Perhaps the solution is
>>>> a custom decoder.
>>>>
>>>>
>>>> I have made custom clusters for example for my Ghost proxies of Marea
>>>> system. It was a perfect example of how I could extent Fuel besides the
>>>> common hooks. Fuel provides many places for extending , like clusters,
>>>> analyzer, etc
>>>
>>> Right on, exactly! Could you tell me more about your Ghost proxies
>>> and Marea, please? As well, could you mention how you select a custom
>>> cluster on the serialization side?
>>>
>>>
>>> thanks so much ^^
>>> Robert
>>>
>>>
>>>>
>>>> No, a bit more. It looks like I need a new
>>>> FLSubstitutePointerObjectCluster, write them on serialization with
>>>> the substitute, then do unsubstitution on materialization, since the
>>>> cluster controls materialization and not the decoder.
>>>>
>>>> Does this approach seem sound to you, from a you know architecture
>>>> and design approach?
>>>>
>>>>
>>>> There was an issue. Hope other can remember. If not, I will try to
>>>> explan what I remember tomorrow.
>>>>
>>>>
>>>> thanks so much ^^
>>>> Robert
>>>>
>>>> On 10/13/2015 04:49 PM, Mariano Martinez Peck wrote:
>>>>
>>>> No, unfortunately, as far as I can remember, we do not have
>>>> that. There
>>>> are another hooks you may use but only in certain scenarios
>>>> (#fuelNew,
>>>> #fuelAfterMaterialization, global sends, etc). But everything is
>>>> listed
>>>> in
>>>> http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingGraph
>>>> so if you didn't find anything of help in there there are
>>>> chances
>>>> there isn't anything.
>>>>
>>>> Cheers,
>>>>
>>>> On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers
>>>> <[hidden email]
>>>> <mailto:[hidden email]> <mailto:[hidden email]>
>>>> <mailto:[hidden email]
>>>> <mailto:[hidden email]>>> wrote:
>>>>
>>>> Yes, I meant dynamic substitution on materialization, to
>>>> use the
>>>> correct terminology.
>>>>
>>>> thanks,
>>>> Robert
>>>>
>>>>
>>>> On 10/13/2015 11:40 AM, Max Leske wrote:
>>>>
>>>>
>>>> On 13 Oct 2015, at 17:16, Robert Withers
>>>> <[hidden email]
>>>> <mailto:[hidden email]>
>>>> <mailto:[hidden email]>
>>>> <mailto:[hidden email]
>>>> <mailto:[hidden email]>>> wrote:
>>>>
>>>> Every extra source helps, thank you. I see how to do
>>>> non-stream substitutions on materializations, but the
>>>> documentation did not indicate a way to do non-stream
>>>> substitutions on serialization. Is it possible?
>>>>
>>>>
>>>> I don’t understand what you mean by “non-stream”. Could
>>>> you give
>>>> an example?
>>>>
>>>>
>>>> thanks,
>>>> Robert
>>>>
>>>> On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:
>>>>
>>>> Hi Robert,
>>>>
>>>> As for the documentation, you have LOTS of
>>>> tests, you
>>>> have the chapter
>>>> Torsten pasted, you have this documentation:
>>>> http://rmod.inria.fr/web/software/Fuel
>>>>
>>>> But also, as for internals, there is a journal
>>>> paper we
>>>> wrote:
>>>> http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf
>>>>
>>>> Let us know how it goes,
>>>>
>>>>
>>>> On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann
>>>> <[hidden email] <mailto:[hidden email]>
>>>> <mailto:[hidden email]>
>>>> <mailto:[hidden email] <mailto:[hidden email]>>
>>>> <mailto:[hidden email] <mailto:[hidden email]>
>>>> <mailto:[hidden email] <mailto:[hidden email]>>>> wrote:
>>>>
>>>> Hi Robert,
>>>>
>>>> Also checkout the chapter on Fuel in Pharo
>>>> Enterprise book:
>>>>
>>>> https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/EnterprisePharo-A4.pdf
>>>>
>>>> Bye
>>>> Torsten
>>>>
>>>>> Gesendet: Dienstag, 13. Oktober 2015 um
>>>> 09:44 Uhr
>>>>> Von: "Robert Withers"
>>>> <[hidden email]
>>>> <mailto:[hidden email]>
>>>> <mailto:[hidden email]>
>>>> <mailto:[hidden email]
>>>> <mailto:[hidden email]>>
>>>> <mailto:[hidden email]
>>>> <mailto:[hidden email]>
>>>> <mailto:[hidden email]
>>>> <mailto:[hidden email]>>>>
>>>>> An: [hidden email] <mailto:[hidden email]>
>>>> <mailto:[hidden email]>
>>>> <mailto:[hidden email]
>>>> <mailto:[hidden email]>>
>>>> <mailto:[hidden email]
>>>> <mailto:[hidden email]>
>>>> <mailto:[hidden email]
>>>> <mailto:[hidden email]>>>
>>>>> Betreff: Re: [Pharo-dev] binary
>>>> serialization
>>>>>
>>>>> Yes, I have to do object substitutions.
>>>> Thanks
>>>> for the link!
>>>>>
>>>>> thanks,
>>>>> Robert
>>>>>
>>>>> On 10/13/2015 03:43 AM, Max Leske wrote:
>>>>>>
>>>>>>> On 13 Oct 2015, at 09:40, Robert Withers
>>>> <[hidden email]
>>>> <mailto:[hidden email]>
>>>> <mailto:[hidden email]>
>>>> <mailto:[hidden email]
>>>> <mailto:[hidden email]>>
>>>> <mailto:[hidden email]
>>>> <mailto:[hidden email]>
>>>> <mailto:[hidden email]
>>>> <mailto:[hidden email]>>>> wrote:
>>>>>>>
>>>>>>> Sven and Torsten, that's a binary
>>>> serialization library! It
>>>> will take time to learn it and how to use
>>>> mappers.
>>>>>>>
>>>>>>> What is the format; is it language
>>>> neutral?
>>>>>>
>>>>>> For quick serialization you don’t
>>>> need to do
>>>> anything. It works
>>>> for (almost) all objects. Only if you
>>>> want to
>>>> exclude things or
>>>> treat some objects in a special way, you
>>>> will need
>>>> to do some stuff.
>>>>>>
>>>>>> Documentation:
>>>> http://rmod.inria.fr/web/software/Fuel.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> thanks,
>>>>>>> Robert
>>>>>>>
>>>>>>> On 10/13/2015 01:21 AM, Sven Van
>>>> Caekenberghe
>>>> wrote:
>>>>>>>> Yes, it is called FUEL and it is a
>>>> standard
>>>> part of the
>>>> image. See FLSerializer and FLMaterializer.
>>>>>>>>
>>>>>>>>> On 13 Oct 2015, at 06:59, Robert
>>>> Withers
>>>> <[hidden email]
>>>> <mailto:[hidden email]>
>>>> <mailto:[hidden email]>
>>>> <mailto:[hidden email]
>>>> <mailto:[hidden email]>>
>>>> <mailto:[hidden email]
>>>> <mailto:[hidden email]>
>>>> <mailto:[hidden email]
>>>> <mailto:[hidden email]>>>> wrote:
>>>>>>>>>
>>>>>>>>> Does Pharo have stream classes to
>>>> binary
>>>> de/serialize an
>>>> object, such that the protocol accepts an
>>>> object as
>>>> an argument and
>>>> converts it to a byteArray?
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> thanks,
>>>>>>>>> Robert
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Mariano
>>>> http://marianopeck.wordpress.com
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Mariano
>>>> http://marianopeck.wordpress.com
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Mariano
>>>> http://marianopeck.wordpress.com
>>> <Exporting Vat.jpg><Importing Vat.jpg>
>

Mariano Martinez Peck

Re: binary serialization

Robert,

As far as I can remember, the problem of substitutions at materialziation time was because... as you may have read, Fuel first serializes "objects" and then the references. At materialization, it first creates the objects and then, on a second step, sets the references betwen objects. So the problem was WHERE to place the hook, at which point in time. If we do it just after objects were created, then the substitution code would NOT have access to any of the instVars of that object. If we do it AFTER objects creation and after objects references, then there is no easy way to "substitute" which doesn't involve #become: (because the graph is already constructed) And you know that #become: is still terrible slow because it scans full memory. That will change in Spur.

As for Marea and Ghost,

Ghost proxies paper: https://hal.inria.fr/hal-01081236/document

Current Ghost repo: http://smalltalkhub.com/#!/~CAR/Ghost

Marea paper: http://www.jot.fm/issues/issue_2013_01/article2.pdf

Current repo: http://ss3.gemstone.com/ss/Marea.html

And finally, my PhD thesis: https://tel.archives-ouvertes.fr/tel-00764991/document

In Marea I needed custom clusters for my proxies because the serializer itself sends messages to the objects being serialized. My proxies would bring back graphs from a secondary memory. So if I was serializing a graph that had proxies already, I didn't want that. So I hooked my custom cluster for proxies that send only a few special messages to the proxy that these understand and answer rather than intercept those messages.

As for how to extend Fuel for this, I recommend to check the code of Marea. See categories 'Marea-Serializers' and 'Marea-Proxies'.

I have a Marea one click here: https://www.dropbox.com/sh/xp8jzyypmz0898j/AACRdHno6V7UfhaJ1ofTPPXva?dl=0

Cheers,

On Wed, Oct 14, 2015 at 11:40 AM, Robert Withers <[hidden email]> wrote:

Good morning, Max. Thank you for the example. I got a little confused, between migrations and substitutions. My issue with no-arg blocks, I believe, is the inability to access my scope object to maintain the object tables.

I'm attempting to write my own Materializer, Decoder and Materialization. At the moment, I'm just going to walk the graph, testing and do #becomes:. See how well that works when I get a test.

It's really good to know about that other list.

thanks so much ^^
Robert

On 10/14/2015 04:15 AM, Max Leske wrote:

BTW, there is a dedicated Fuel mailing list:
[hidden email] <mailto:[hidden email]>

Max

On 14 Oct 2015, at 09:45, Max Leske <[hidden email]
<mailto:[hidden email]>> wrote:

On 14 Oct 2015, at 04:39, Robert Withers <[hidden email]
<mailto:[hidden email]>> wrote:

On 10/13/2015 09:43 PM, Mariano Martinez Peck wrote:

On Tue, Oct 13, 2015 at 10:33 PM, Robert Withers
<[hidden email] <mailto:[hidden email]>
<mailto:[hidden email]>> wrote:

Hi Mariano,

This presents me with a big challenge, then. I read the docs and
explored the code and the only other aspect not mentioned, beyond
instance creation (#fuelNew #fuelNew:) and postMaterialization
(#fuelAfterMaterialization), is migrations. However, migration only
allows for instanceVar mappings, no code blocks.

What do you mean that migrations only allows instVar mappings , and no
code blocks? I mean, what do you mean by code blocks?

Sounds to me like this (see FuelOutStackDebuAction):

serializeTestFailureContext: aContext toFileNamed: aFilename
| serializer |

serializer := FLSerializer newDefault.
self encodeDebugInformationOn: serializer.
serializer addPostMaterializationAction: [ :materialization |
Smalltalk tools debugger
openOn: Processor activeProcess
context: materialization root
label: 'External stack'
contents: nil
fullView: false ].

serializer
" use the sender context, generally the current context is not
interesting"
serialize: aContext
toFileNamed: aFilename

This stores a block in the serialization which is evaluated after
materialization. The only requirement is that it’s a clean block (no
closure!).

We also support class renames. This is here:
http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Migration?_s=Pvs4DYqPRjfEy8sg&_k=X6ltJu7FDSxumm4X&_n&22

Which kind of migration example you have in mind that would not be
supported? An example would help.

Well, my pics will demonstrate. I am interested in doing more than
mappping ivars or a class rename. I want to do a total substitution,
then a further substitution on the receiving, import side:

Vat1: anObject (Class A) ---> On wire: desc (Descriptor) ---> Vat2:
aProxy (Class FarERef)

A desc is substituted for a PassByProxy object, then a FarERef is
substituted for the desc.

#fuelAccept: is a serialization side method.

If Fuel supports substitution on serialization, I don't understand
why no substitution support on materialization.

There was a reason, which I cannot remember completely. Maybe Martin or
Max can remember.

It seems your focus was pickling to disk then back. My focus is
distributed proxy graphs, which has different use cases.

I am definitely going to use the world-class Fuel binary
serialization system. However, I find myself needing to extend Fuel
to support substitution on materialization. Perhaps the solution is
a custom decoder.

I have made custom clusters for example for my Ghost proxies of Marea
system. It was a perfect example of how I could extent Fuel besides the
common hooks. Fuel provides many places for extending , like clusters,
analyzer, etc

Right on, exactly! Could you tell me more about your Ghost proxies
and Marea, please? As well, could you mention how you select a custom
cluster on the serialization side?

thanks so much ^^
Robert

No, a bit more. It looks like I need a new
FLSubstitutePointerObjectCluster, write them on serialization with
the substitute, then do unsubstitution on materialization, since the
cluster controls materialization and not the decoder.

Does this approach seem sound to you, from a you know architecture
and design approach?

There was an issue. Hope other can remember. If not, I will try to
explan what I remember tomorrow.

thanks so much ^^
Robert

On 10/13/2015 04:49 PM, Mariano Martinez Peck wrote:

No, unfortunately, as far as I can remember, we do not have
that. There
are another hooks you may use but only in certain scenarios
(#fuelNew,
#fuelAfterMaterialization, global sends, etc). But everything is
listed
in
http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingGraph
so if you didn't find anything of help in there there are
chances
there isn't anything.

Cheers,

On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers
<[hidden email]
<mailto:[hidden email]> <mailto:[hidden email]>
<mailto:[hidden email]
<mailto:[hidden email]>>> wrote:

Yes, I meant dynamic substitution on materialization, to
use the
correct terminology.

thanks,
Robert

On 10/13/2015 11:40 AM, Max Leske wrote:

On 13 Oct 2015, at 17:16, Robert Withers
<[hidden email]
<mailto:[hidden email]>
<mailto:[hidden email]>
<mailto:[hidden email]
<mailto:[hidden email]>>> wrote:

Every extra source helps, thank you. I see how to do
non-stream substitutions on materializations, but the
documentation did not indicate a way to do non-stream
substitutions on serialization. Is it possible?

I don’t understand what you mean by “non-stream”. Could
you give
an example?

thanks,
Robert

On 10/13/2015 09:00 AM, Mariano Martinez Peck wrote:

Hi Robert,

As for the documentation, you have LOTS of
tests, you
have the chapter
Torsten pasted, you have this documentation:
http://rmod.inria.fr/web/software/Fuel

But also, as for internals, there is a journal
paper we
wrote:
http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf

Let us know how it goes,

On Tue, Oct 13, 2015 at 6:00 AM, Torsten Bergmann
<[hidden email] <mailto:[hidden email]>
<mailto:[hidden email]>
<mailto:[hidden email] <mailto:[hidden email]>>
<mailto:[hidden email] <mailto:[hidden email]>
<mailto:[hidden email] <mailto:[hidden email]>>>> wrote:

Hi Robert,

Also checkout the chapter on Fuel in Pharo
Enterprise book:

https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/EnterprisePharo-A4.pdf

Bye
Torsten

Gesendet: Dienstag, 13. Oktober 2015 um

09:44 Uhr

Von: "Robert Withers"

<[hidden email]
<mailto:[hidden email]>
<mailto:[hidden email]>
<mailto:[hidden email]
<mailto:[hidden email]>>
<mailto:[hidden email]
<mailto:[hidden email]>
<mailto:[hidden email]
<mailto:[hidden email]>>>>

An: [hidden email] <mailto:[hidden email]>

<mailto:[hidden email]>
<mailto:[hidden email]
<mailto:[hidden email]>>
<mailto:[hidden email]
<mailto:[hidden email]>
<mailto:[hidden email]
<mailto:[hidden email]>>>

Betreff: Re: [Pharo-dev] binary

serialization

Yes, I have to do object substitutions.

Thanks
for the link!

thanks,
Robert

On 10/13/2015 03:43 AM, Max Leske wrote:

On 13 Oct 2015, at 09:40, Robert Withers

<[hidden email]
<mailto:[hidden email]>
<mailto:[hidden email]>
<mailto:[hidden email]
<mailto:[hidden email]>>
<mailto:[hidden email]
<mailto:[hidden email]>
<mailto:[hidden email]
<mailto:[hidden email]>>>> wrote:

Sven and Torsten, that's a binary

serialization library! It
will take time to learn it and how to use
mappers.

What is the format; is it language

neutral?

For quick serialization you don’t

need to do
anything. It works
for (almost) all objects. Only if you
want to
exclude things or
treat some objects in a special way, you
will need
to do some stuff.

Documentation:

http://rmod.inria.fr/web/software/Fuel.

thanks,
Robert

On 10/13/2015 01:21 AM, Sven Van

Caekenberghe
wrote:

Yes, it is called FUEL and it is a

standard
part of the
image. See FLSerializer and FLMaterializer.

On 13 Oct 2015, at 06:59, Robert

Withers
<[hidden email]
<mailto:[hidden email]>
<mailto:[hidden email]>
<mailto:[hidden email]
<mailto:[hidden email]>>
<mailto:[hidden email]
<mailto:[hidden email]>
<mailto:[hidden email]
<mailto:[hidden email]>>>> wrote:

Does Pharo have stream classes to

binary
de/serialize an
object, such that the protocol accepts an
object as
an argument and
converts it to a byteArray?

--
thanks,
Robert

--
Mariano
http://marianopeck.wordpress.com

--
Mariano
http://marianopeck.wordpress.com

--
Mariano
http://marianopeck.wordpress.com

<Exporting Vat.jpg><Importing Vat.jpg>

Mariano
http://marianopeck.wordpress.com