Other serialisation formats -- SIXX serialisation format, JSON, Fuel?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Other serialisation formats -- SIXX serialisation format, JSON, Fuel?

Hannes Hirzel
Hello

Note:
The SIXX serialisation format has been around for a long time (on
SqueakMap for 5.1)
http://www.mars.dti.ne.jp/~umejava/smalltalk/sixx/
It has as well Pharo, Cuis and other implementations.

We might consider JSON or Fuel might as well options for a format to
save projects (with ImageMorphs, Browsers, Workspaces, BookMorphs for
example).

--Hannes

Reply | Threaded
Open this post in threaded view
|

Re: Other serialisation formats -- SIXX serialisation format, JSON, Fuel?

fniephaus
Would the STON format be suitable for projects as well?
https://github.com/svenvc/ston/blob/master/ston-paper.md

Fabio

On Wed, 31 Aug 2016 at 08:33, H. Hirzel <[hidden email]> wrote:
Hello

Note:
The SIXX serialisation format has been around for a long time (on
SqueakMap for 5.1)
http://www.mars.dti.ne.jp/~umejava/smalltalk/sixx/
It has as well Pharo, Cuis and other implementations.

We might consider JSON or Fuel might as well options for a format to
save projects (with ImageMorphs, Browsers, Workspaces, BookMorphs for
example).

--Hannes



Reply | Threaded
Open this post in threaded view
|

Re: Other serialisation formats -- SIXX serialisation format, JSON, Fuel?

Hannes Hirzel
A note by Tim

On 8/29/16, tim Rowledge <[hidden email]> wrote:
...
> ....the deep-copy code seems awfully like (much of) a serialiser

On 8/31/16, Fabio Niephaus <[hidden email]> wrote:

> Would the STON format be suitable for projects as well?
> https://github.com/svenvc/ston/blob/master/ston-paper.md
>
> Fabio
>
> On Wed, 31 Aug 2016 at 08:33, H. Hirzel <[hidden email]> wrote:
>
>> Hello
>>
>> Note:
>> The SIXX serialisation format has been around for a long time (on
>> SqueakMap for 5.1)
>> http://www.mars.dti.ne.jp/~umejava/smalltalk/sixx/
>> It has as well Pharo, Cuis and other implementations.
>>
>> We might consider JSON or Fuel might as well options for a format to
>> save projects (with ImageMorphs, Browsers, Workspaces, BookMorphs for
>> example).
>>
>> --Hannes
>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Other serialisation formats -- SIXX serialisation format, JSON, Fuel?

Bert Freudenberg
In reply to this post by Hannes Hirzel
On Wednesday, 31 August 2016, H. Hirzel <[hidden email]> wrote:

We might consider JSON or Fuel might as well options for a format to
save projects (with ImageMorphs, Browsers, Workspaces, BookMorphs for
example).

If someone wants to take a serious look I'd suggest Fuel. Being a replacement for ImageSegments was one of its design goals, if I remember correctly.

- Bert - 


Reply | Threaded
Open this post in threaded view
|

Re: Other serialisation formats -- SIXX serialisation format, JSON, Fuel?

Tobias Pape

On 31.08.2016, at 11:14, Bert Freudenberg <[hidden email]> wrote:

> On Wednesday, 31 August 2016, H. Hirzel <[hidden email]> wrote:
>
> We might consider JSON or Fuel might as well options for a format to
> save projects (with ImageMorphs, Browsers, Workspaces, BookMorphs for
> example).
>
> If someone wants to take a serious look I'd suggest Fuel. Being a replacement for ImageSegments was one of its design goals, if I remember correctly.
>
> - Bert -

+1

Also, Max Leske always made an effort that Fuel easily loads in Squeak.

best regards
        -Tobias


Reply | Threaded
Open this post in threaded view
|

Re: Other serialisation formats -- SIXX serialisation format, JSON, Fuel?

Hannes Hirzel
https://github.com/theseion/Fuel

says about Fuel

- No special VM-support needed.
- Can serialize/materialize not only plain objects but also classes,
traits, methods, closures, contexts, packages, etc.
- Very customizable: ignore certain instance variables, substitute
objects by others, pre and post serialization and materialization
actions, etc.
- Supports class rename and class reshape.
- Good test coverage (almost 600 unit tests).

On 8/31/16, Tobias Pape <[hidden email]> wrote:

>
> On 31.08.2016, at 11:14, Bert Freudenberg <[hidden email]> wrote:
>
>> On Wednesday, 31 August 2016, H. Hirzel <[hidden email]> wrote:
>>
>> We might consider JSON or Fuel might as well options for a format to
>> save projects (with ImageMorphs, Browsers, Workspaces, BookMorphs for
>> example).
>>
>> If someone wants to take a serious look I'd suggest Fuel. Being a
>> replacement for ImageSegments was one of its design goals, if I remember
>> correctly.
>>
>> - Bert -
>
> +1
>
> Also, Max Leske always made an effort that Fuel easily loads in Squeak.
>
> best regards
> -Tobias
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Other serialisation formats -- SIXX serialisation format, JSON, Fuel?

Edgar De Cleene
In reply to this post by Hannes Hirzel
+1 JSON as is used for thousands


On 8/31/16, 3:33 AM, "H. Hirzel" <[hidden email]> wrote:

> Hello
>
> Note:
> The SIXX serialisation format has been around for a long time (on
> SqueakMap for 5.1)
> http://www.mars.dti.ne.jp/~umejava/smalltalk/sixx/
> It has as well Pharo, Cuis and other implementations.
>
> We might consider JSON or Fuel might as well options for a format to
> save projects (with ImageMorphs, Browsers, Workspaces, BookMorphs for
> example).
>
> --Hannes



Reply | Threaded
Open this post in threaded view
|

Re: Other serialisation formats -- SIXX serialisation format, JSON, Fuel?

fniephaus
In reply to this post by Hannes Hirzel
+1 Fuel

Max also runs all CI tests on different Squeak versions:
Therefore, Fuel should be compatible to Squeak versions later than 4.4.

--

On Wed, Aug 31, 2016 at 1:00 PM H. Hirzel <[hidden email]> wrote:
https://github.com/theseion/Fuel

says about Fuel

- No special VM-support needed.
- Can serialize/materialize not only plain objects but also classes,
traits, methods, closures, contexts, packages, etc.
- Very customizable: ignore certain instance variables, substitute
objects by others, pre and post serialization and materialization
actions, etc.
- Supports class rename and class reshape.
- Good test coverage (almost 600 unit tests).

On 8/31/16, Tobias Pape <[hidden email]> wrote:
>
> On 31.08.2016, at 11:14, Bert Freudenberg <[hidden email]> wrote:
>
>> On Wednesday, 31 August 2016, H. Hirzel <[hidden email]> wrote:
>>
>> We might consider JSON or Fuel might as well options for a format to
>> save projects (with ImageMorphs, Browsers, Workspaces, BookMorphs for
>> example).
>>
>> If someone wants to take a serious look I'd suggest Fuel. Being a
>> replacement for ImageSegments was one of its design goals, if I remember
>> correctly.
>>
>> - Bert -
>
> +1
>
> Also, Max Leske always made an effort that Fuel easily loads in Squeak.
>
> best regards
>       -Tobias
>
>
>



Reply | Threaded
Open this post in threaded view
|

Re: Other serialisation formats -- SIXX serialisation format, JSON, Fuel?

Hannes Hirzel
A test of Fuel with Squeak 5.1 ran three hours ago.

https://travis-ci.org/theseion/Fuel/jobs/156426655

SmalltalkCI: 515 Tests, 0 Failures, 0 Errors in 5.713s

On 8/31/16, Fabio Niephaus <[hidden email]> wrote:

> +1 Fuel
>
> Max also runs all CI tests on different Squeak versions:
> https://travis-ci.org/theseion/Fuel
> Therefore, Fuel should be compatible to Squeak versions later than 4.4.
>
> --
>
> On Wed, Aug 31, 2016 at 1:00 PM H. Hirzel <[hidden email]> wrote:
>
>> https://github.com/theseion/Fuel
>>
>> says about Fuel
>>
>> - No special VM-support needed.
>> - Can serialize/materialize not only plain objects but also classes,
>> traits, methods, closures, contexts, packages, etc.
>> - Very customizable: ignore certain instance variables, substitute
>> objects by others, pre and post serialization and materialization
>> actions, etc.
>> - Supports class rename and class reshape.
>> - Good test coverage (almost 600 unit tests).
>>
>> On 8/31/16, Tobias Pape <[hidden email]> wrote:
>> >
>> > On 31.08.2016, at 11:14, Bert Freudenberg <[hidden email]> wrote:
>> >
>> >> On Wednesday, 31 August 2016, H. Hirzel <[hidden email]>
>> wrote:
>> >>
>> >> We might consider JSON or Fuel might as well options for a format to
>> >> save projects (with ImageMorphs, Browsers, Workspaces, BookMorphs for
>> >> example).
>> >>
>> >> If someone wants to take a serious look I'd suggest Fuel. Being a
>> >> replacement for ImageSegments was one of its design goals, if I
>> >> remember
>> >> correctly.
>> >>
>> >> - Bert -
>> >
>> > +1
>> >
>> > Also, Max Leske always made an effort that Fuel easily loads in Squeak.
>> >
>> > best regards
>> >       -Tobias
>> >
>> >
>> >
>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Other serialisation formats -- SIXX serialisation format, JSON, Fuel?

Eliot Miranda-2
In reply to this post by Bert Freudenberg


On Aug 31, 2016, at 11:14 AM, Bert Freudenberg <[hidden email]> wrote:

On Wednesday, 31 August 2016, H. Hirzel <[hidden email]> wrote:

We might consider JSON or Fuel might as well options for a format to
save projects (with ImageMorphs, Browsers, Workspaces, BookMorphs for
example).

If someone wants to take a serious look I'd suggest Fuel. Being a replacement for ImageSegments was one of its design goals, if I remember correctly.

+1.  It also has a very performant architecture.  It is very similar to VW's parcel format which priced to be significantly faster than other formats at PPD.


- Bert - 



Reply | Threaded
Open this post in threaded view
|

Re: Other serialisation formats -- SIXX serialisation format, JSON, Fuel?

Chris Muller-3
I would, of course, love for Ma Serializer to be considered.  Its
mature and proven, and has a lot of hooks and I just know the scrutiny
and brilliance of this community would benefit it tremendously, and
since Magma uses it, would make Magma fundamentally better, too.

The Fuel hysteria appears to have already garnered everyone's vote
before I saw this thread to get myself on the ballot..    I once took
at look at trying to use Fuel for Magma, but its much too limited (and
not nearly as much faster than Ma Serializer as reported in the Fuel
paper).  Its is good for UC1) Save an object and UC2) Load an object,
but not much else.  :(

On Wed, Aug 31, 2016 at 6:40 AM, Eliot Miranda <[hidden email]> wrote:

>
>
> On Aug 31, 2016, at 11:14 AM, Bert Freudenberg <[hidden email]> wrote:
>
> On Wednesday, 31 August 2016, H. Hirzel <[hidden email]> wrote:
>>
>>
>> We might consider JSON or Fuel might as well options for a format to
>> save projects (with ImageMorphs, Browsers, Workspaces, BookMorphs for
>> example).
>
>
> If someone wants to take a serious look I'd suggest Fuel. Being a
> replacement for ImageSegments was one of its design goals, if I remember
> correctly.
>
>
> +1.  It also has a very performant architecture.  It is very similar to VW's
> parcel format which priced to be significantly faster than other formats at
> PPD.
>
>
> - Bert -
>
>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Other serialisation formats -- SIXX serialisation format, JSON, Fuel?

Hannes Hirzel
To widen the discussion.

I think there was no discussion yet if the serialisation format should

a) be a binary format
b) a text format
c) or that we need both.

The prominent use case is saving and loading of projects, see thread

'Vaidotas, Squeak 5.1 saving of Morphic projects is broken'

and the SISS thread.

Speed of course is an issue.
But then as well portability between different versions of images.

If the question is put as

Which serialisation format should succeed image segments then then the
choice is probably limited to

- Fuel
- Ma Serializer


On 8/31/16, Chris Muller <[hidden email]> wrote:

> I would, of course, love for Ma Serializer to be considered.  Its
> mature and proven, and has a lot of hooks and I just know the scrutiny
> and brilliance of this community would benefit it tremendously, and
> since Magma uses it, would make Magma fundamentally better, too.
>
> The Fuel hysteria appears to have already garnered everyone's vote
> before I saw this thread to get myself on the ballot..    I once took
> at look at trying to use Fuel for Magma, but its much too limited (and
> not nearly as much faster than Ma Serializer as reported in the Fuel
> paper).  Its is good for UC1) Save an object and UC2) Load an object,
> but not much else.  :(
>
> On Wed, Aug 31, 2016 at 6:40 AM, Eliot Miranda <[hidden email]>
> wrote:
>>
>>
>> On Aug 31, 2016, at 11:14 AM, Bert Freudenberg <[hidden email]>
>> wrote:
>>
>> On Wednesday, 31 August 2016, H. Hirzel <[hidden email]> wrote:
>>>
>>>
>>> We might consider JSON or Fuel might as well options for a format to
>>> save projects (with ImageMorphs, Browsers, Workspaces, BookMorphs for
>>> example).
>>
>>
>> If someone wants to take a serious look I'd suggest Fuel. Being a
>> replacement for ImageSegments was one of its design goals, if I remember
>> correctly.
>>
>>
>> +1.  It also has a very performant architecture.  It is very similar to
>> VW's
>> parcel format which priced to be significantly faster than other formats
>> at
>> PPD.
>>
>>
>> - Bert -
>>
>>
>>
>>
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Other serialisation formats -- SIXX serialisation format, JSON, Fuel?

Hannes Hirzel
I have started using Fuel and Ma Serializer to store PasteUpMorphs
stored in a project. Actually it is a collection of PastUpMorphs used
as a slide show. I used as well SIXX for some tests.

It is possible to keep all the serialization packages in the same image.

Fuel and Ma Serializer work fine so far to save and restore
PasteUpMorphs (with content).
To use SIXX I had to go for a description and then restore from the
description. Interestingly SIXX was about as space efficient as the
other ones if I used compression on the the resulting file. As SISS is
modeled after SIXX the result is probably the same (I did not verify
this).

For the tests I did so far Ma Serializer was more space efficient. And
the claim is that it spans versions wherase Fuel description
explicitly states that it does not.

There is a need to agree on some test cases and criteria.

It is as well worth considering supporting different formats.

The API to do so is very thin.

E.g. for the Fuel test to restore the slide collection I just had to do.

slides := FLMaterializer materializeFromFileNamed:
'/home/user/sq5.1test-Fuel/documentation/Kopie_Lit_01.FL'.

slides reverseDo: [:s | s openInWorld]

--Hannes



On 9/1/16, H. Hirzel <[hidden email]> wrote:

> To widen the discussion.
>
> I think there was no discussion yet if the serialisation format should
>
> a) be a binary format
> b) a text format
> c) or that we need both.
>
> The prominent use case is saving and loading of projects, see thread
>
> 'Vaidotas, Squeak 5.1 saving of Morphic projects is broken'
>
> and the SISS thread.
>
> Speed of course is an issue.
> But then as well portability between different versions of images.
>
> If the question is put as
>
> Which serialisation format should succeed image segments then then the
> choice is probably limited to
>
> - Fuel
> - Ma Serializer
>
>
> On 8/31/16, Chris Muller <[hidden email]> wrote:
>> I would, of course, love for Ma Serializer to be considered.  Its
>> mature and proven, and has a lot of hooks and I just know the scrutiny
>> and brilliance of this community would benefit it tremendously, and
>> since Magma uses it, would make Magma fundamentally better, too.
>>
>> The Fuel hysteria appears to have already garnered everyone's vote
>> before I saw this thread to get myself on the ballot..    I once took
>> at look at trying to use Fuel for Magma, but its much too limited (and
>> not nearly as much faster than Ma Serializer as reported in the Fuel
>> paper).  Its is good for UC1) Save an object and UC2) Load an object,
>> but not much else.  :(
>>
>> On Wed, Aug 31, 2016 at 6:40 AM, Eliot Miranda <[hidden email]>
>> wrote:
>>>
>>>
>>> On Aug 31, 2016, at 11:14 AM, Bert Freudenberg <[hidden email]>
>>> wrote:
>>>
>>> On Wednesday, 31 August 2016, H. Hirzel <[hidden email]> wrote:
>>>>
>>>>
>>>> We might consider JSON or Fuel might as well options for a format to
>>>> save projects (with ImageMorphs, Browsers, Workspaces, BookMorphs for
>>>> example).
>>>
>>>
>>> If someone wants to take a serious look I'd suggest Fuel. Being a
>>> replacement for ImageSegments was one of its design goals, if I remember
>>> correctly.
>>>
>>>
>>> +1.  It also has a very performant architecture.  It is very similar to
>>> VW's
>>> parcel format which priced to be significantly faster than other formats
>>> at
>>> PPD.
>>>
>>>
>>> - Bert -
>>>
>>>
>>>
>>>
>>>
>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Other serialisation formats -- SIXX serialisation format, JSON, Fuel?

Mariano Martinez Peck


On Mon, Sep 19, 2016 at 5:48 PM, H. Hirzel <[hidden email]> wrote:
I have started using Fuel and Ma Serializer to store PasteUpMorphs
stored in a project. Actually it is a collection of PastUpMorphs used
as a slide show. I used as well SIXX for some tests.

It is possible to keep all the serialization packages in the same image.

Fuel and Ma Serializer work fine so far to save and restore
PasteUpMorphs (with content).
To use SIXX I had to go for a description and then restore from the
description. Interestingly SIXX was about as space efficient as the
other ones if I used compression on the the resulting file. As SISS is
modeled after SIXX the result is probably the same (I did not verify
this).

For the tests I did so far Ma Serializer was more space efficient. And
the claim is that it spans versions wherase Fuel description
explicitly states that it does not.

There is a need to agree on some test cases and criteria.

It is as well worth considering supporting different formats.

The API to do so is very thin.

E.g. for the Fuel test to restore the slide collection I just had to do.

slides := FLMaterializer materializeFromFileNamed:
'/home/user/sq5.1test-Fuel/documentation/Kopie_Lit_01.FL'.

slides reverseDo: [:s | s openInWorld]


You can even add the #openInWorld as a post materialization action which is serialized in a header. That way, the #materialize will automatically evaluate your post actions.
 
--Hannes



On 9/1/16, H. Hirzel <[hidden email]> wrote:
> To widen the discussion.
>
> I think there was no discussion yet if the serialisation format should
>
> a) be a binary format
> b) a text format
> c) or that we need both.
>
> The prominent use case is saving and loading of projects, see thread
>
> 'Vaidotas, Squeak 5.1 saving of Morphic projects is broken'
>
> and the SISS thread.
>
> Speed of course is an issue.
> But then as well portability between different versions of images.
>
> If the question is put as
>
> Which serialisation format should succeed image segments then then the
> choice is probably limited to
>
> - Fuel
> - Ma Serializer
>
>
> On 8/31/16, Chris Muller <[hidden email]> wrote:
>> I would, of course, love for Ma Serializer to be considered.  Its
>> mature and proven, and has a lot of hooks and I just know the scrutiny
>> and brilliance of this community would benefit it tremendously, and
>> since Magma uses it, would make Magma fundamentally better, too.
>>
>> The Fuel hysteria appears to have already garnered everyone's vote
>> before I saw this thread to get myself on the ballot..    I once took
>> at look at trying to use Fuel for Magma, but its much too limited (and
>> not nearly as much faster than Ma Serializer as reported in the Fuel
>> paper).  Its is good for UC1) Save an object and UC2) Load an object,
>> but not much else.  :(
>>
>> On Wed, Aug 31, 2016 at 6:40 AM, Eliot Miranda <[hidden email]>
>> wrote:
>>>
>>>
>>> On Aug 31, 2016, at 11:14 AM, Bert Freudenberg <[hidden email]>
>>> wrote:
>>>
>>> On Wednesday, 31 August 2016, H. Hirzel <[hidden email]> wrote:
>>>>
>>>>
>>>> We might consider JSON or Fuel might as well options for a format to
>>>> save projects (with ImageMorphs, Browsers, Workspaces, BookMorphs for
>>>> example).
>>>
>>>
>>> If someone wants to take a serious look I'd suggest Fuel. Being a
>>> replacement for ImageSegments was one of its design goals, if I remember
>>> correctly.
>>>
>>>
>>> +1.  It also has a very performant architecture.  It is very similar to
>>> VW's
>>> parcel format which priced to be significantly faster than other formats
>>> at
>>> PPD.
>>>
>>>
>>> - Bert -
>>>
>>>
>>>
>>>
>>>
>>
>>
>




--