Using Magritte-Json to transfer magritte-based model data

Hi all,

We have been looking at ways to transfer data between Pharo and Gemstone. We decided to use Magritte-Json to export and import our model.
- Magritte-Json allows control over what to export: using the flag “isJsonSerialized”.  This allows us to simply set this flag where needed.
- Magritte-Json has no loop detection. It has several ways to control what is serialized. Mainly the flags: “isJsonSerialized” and “useJsonKey"
We needed to modify old implementation
- It did not make proper use of the JsonReader and JsonWriter you could set. We have some references to other objects. From the magritte-description we know the expected type. These objects need to be serialised as keys, instead of the object itself. On materialising we need to make a lookup. We made a KeyWriter for this.
- We do better detection of what class is read. The old implementation simply took the first class from the magritte-description, but this is not always correct.
- We improved the option serialisation. On reading, when there is a possible match, we make this match.
- We improved the relation serialisation. On writing it can write a key instead of the entire object. On reading it looks the key up in the collection (ownedIn) to create the exact value.

We took a look at the following alternatives:
- Fuel. It would be great is Fuel would be available for Gemstone as well, as it is fast and the transfer file is compact. Current implementation does not load in Gemstone, because it is using primitives. The following things need to be done to make this run in Gemstone:
— Splitting Fuel up into packages, with a Pharo specific part and a general part.
— Implementing the core export functionality in Gemstone (only the methods to read/write basic types like boolean, integer, etc. need implementing).
— Mapping basic types where needed.
We decided that especially the first (splitting up Fuel) was too much effort for us, as up until now we are only users of Fuel.

- Ston. We hoped Ston would use magritte descriptions to determine what to export. It turned out to use metaprogramming  and Ston does not handle loops. In order to make this work for us, we needed to rip our model to remove all back links. A better alternative would have been Sixx. Sixx is XML based and a bit larger. Also it would involve some extra packages loaded into production.

How to use Magritte-Json:
First you need to remove the loops from your model. The simple situation is when you have a tree. If this is the case, you don’t need to do anything.

If you have other references you need to decide who the primary owner is. The owner writes the object, all other references write keys. There two cases we have implemented:
- There is a parent / child relation. The parent has a number of children and each child has a reference back. In the description of the child you mark this description as “isParent: true”. This causes it no longer to be serialized by json and on deserialisation it is set to the new parent (using the parsing context)
- There is an owner. The owner holds a number of values, and this is on (or more) of it. In the description of the references objects you refer to the owner in the “isOwnedIn:”. The value method should return a collection that each value is a member of. On serialisation it writes the jsonKey (you need to implement this method). On deserialisation it looks this value up in the collection. Of this value is not found, the key is retained in the collection. Note that this is processed after the other part of model is restored. I.e. it is possible to first write articles with references to an article group, and then write the article groups the articles can belong to.

You can find some examples in the tests of what you can serialise and how to set up the Magritte descriptions accordingly. If these cases do not cover your needs you can create your own reader / writer that does this. Keep them in pairs, and you should probably inherit from the MJBasicJson-Writer / -Reader, as some of the added functionality may interfere.

If there are any questions I am of course happy to answer them. And if you find bugs, please let me know.

Cheers,
Diego
_______________________________________________
Magritte, Pier and Related Tools ...
https://www.iam.unibe.ch/mailman/listinfo/smallwiki