STON shouldn't care about shared references in JSON mode

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

STON shouldn't care about shared references in JSON mode

Peter Uhnak
Hi,

I don't think this should throw an error when I am producing JSON.

```
d := { 
'a' -> #().
'b' -> #().
} asDictionary.

STON toJsonString: d.
```

dtto this

```
a := {'hm'}.

d := { 
'a' -> a.
'b' -> a.
} asDictionary.

STON toJsonString: d.
```

Maybe I should forgo using STON>>toJson* out of laziness (it is preinstalled) and always go for NeoJSON?

Peter
Reply | Threaded
Open this post in threaded view
|

Re: STON shouldn't care about shared references in JSON mode

Sven Van Caekenberghe-2
Hi Peter,

Both cases will pass using this expression:

String streamContents: [ :out |
        STON jsonWriter
                referencePolicy: #ignore;
                on: out;
                nextPut: d ].

The problem is what to do with cycles.

Sven

> On 27 Mar 2018, at 17:43, Peter Uhnák <[hidden email]> wrote:
>
> Hi,
>
> I don't think this should throw an error when I am producing JSON.
>
> ```
> d := {
> 'a' -> #().
> 'b' -> #().
> } asDictionary.
>
> STON toJsonString: d.
> ```
>
> dtto this
>
> ```
> a := {'hm'}.
>
> d := {
> 'a' -> a.
> 'b' -> a.
> } asDictionary.
>
> STON toJsonString: d.
> ```
>
> Maybe I should forgo using STON>>toJson* out of laziness (it is preinstalled) and always go for NeoJSON?
>
> Peter


Reply | Threaded
Open this post in threaded view
|

Re: STON shouldn't care about shared references in JSON mode

Herby Vojčík


Sven Van Caekenberghe wrote:

> Hi Peter,
>
> Both cases will pass using this expression:
>
> String streamContents: [ :out |
> STON jsonWriter
> referencePolicy: #ignore;
> on: out;
> nextPut: d ].
>
> The problem is what to do with cycles.
Be faithful to what JSON.stringify does in JS. See attachment. :-)

Herby

> Sven
>
>> On 27 Mar 2018, at 17:43, Peter Uhnák<[hidden email]>  wrote:
>>
>> Hi,
>>
>> I don't think this should throw an error when I am producing JSON.
>>
>> ```
>> d := {
>> 'a' ->  #().
>> 'b' ->  #().
>> } asDictionary.
>>
>> STON toJsonString: d.
>> ```
>>
>> dtto this
>>
>> ```
>> a := {'hm'}.
>>
>> d := {
>> 'a' ->  a.
>> 'b' ->  a.
>> } asDictionary.
>>
>> STON toJsonString: d.
>> ```
>>
>> Maybe I should forgo using STON>>toJson* out of laziness (it is preinstalled) and always go for NeoJSON?
>>
>> Peter
>
>

json-circular.png (38K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: STON shouldn't care about shared references in JSON mode

Peter Uhnak
Hi Sven,

shouldn't it be set to #ignore by default for JSON then? Or is there a use case where it makes sense to have something else for JSON?

Herbert: I am not talking about (infinitely) recursive references, but referencing the same object, e.g.


Peter

On Tue, Mar 27, 2018 at 6:08 PM, Herbert Vojčík <[hidden email]> wrote:


Sven Van Caekenberghe wrote:
Hi Peter,

Both cases will pass using this expression:

String streamContents: [ :out |
        STON jsonWriter
                referencePolicy: #ignore;
                on: out;
                nextPut: d ].

The problem is what to do with cycles.

Be faithful to what JSON.stringify does in JS. See attachment. :-)

Herby


Sven

On 27 Mar 2018, at 17:43, Peter Uhnák<[hidden email]>  wrote:

Hi,

I don't think this should throw an error when I am producing JSON.

```
d := {
        'a' ->  #().
        'b' ->  #().
} asDictionary.

STON toJsonString: d.
```

dtto this

```
a := {'hm'}.

d := {
        'a' ->  a.
        'b' ->  a.
} asDictionary.

STON toJsonString: d.
```

Maybe I should forgo using STON>>toJson* out of laziness (it is preinstalled) and always go for NeoJSON?

Peter



Reply | Threaded
Open this post in threaded view
|

Re: STON shouldn't care about shared references in JSON mode

Herby Vojčík


Peter Uhnák wrote:
> Hi Sven,
>
> shouldn't it be set to #ignore by default for JSON then? Or is there a
> use case where it makes sense to have something else for JSON?

To keep STON practices / assumptions while using JSON to hold the data
where STON is not supported yet, maybe? IIRC STON is positioning itself
as JSON superset; but maybe only syntactic superset, while semantically
it can be a bit different (so STONize to more low-level, JSON, syntax
via STON; JSONize faithfully to JSON semantics using NeoJSON; but maybe
I am just making things up).

> Herbert: I am not talking about (infinitely) recursive references, but

I know; Sven explicitly asked what to do with cycles in that case, though.

> referencing the same object, e.g.
>
>
> Peter
>
> On Tue, Mar 27, 2018 at 6:08 PM, Herbert Vojčík <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>
>
>     Sven Van Caekenberghe wrote:
>
>         Hi Peter,
>
>         Both cases will pass using this expression:
>
>         String streamContents: [ :out |
>                  STON jsonWriter
>                          referencePolicy: #ignore;
>                          on: out;
>                          nextPut: d ].
>
>         The problem is what to do with cycles.
>
>
>     Be faithful to what JSON.stringify does in JS. See attachment. :-)
>
>     Herby
>
>
>         Sven
>
>             On 27 Mar 2018, at 17:43, Peter Uhnák<[hidden email]
>             <mailto:[hidden email]>>  wrote:
>
>             Hi,
>
>             I don't think this should throw an error when I am producing
>             JSON.
>
>             ```
>             d := {
>             'a' ->  #().
>             'b' ->  #().
>             } asDictionary.
>
>             STON toJsonString: d.
>             ```
>
>             dtto this
>
>             ```
>             a := {'hm'}.
>
>             d := {
>             'a' ->  a.
>             'b' ->  a.
>             } asDictionary.
>
>             STON toJsonString: d.
>             ```
>
>             Maybe I should forgo using STON>>toJson* out of laziness (it
>             is preinstalled) and always go for NeoJSON?
>
>             Peter
>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: STON shouldn't care about shared references in JSON mode

Peter Uhnak
Ah sorry, I'm blind.

Yes, ideally it should throw up an error in case of recursive.
And I see that NeoJSON runs into infinite cycle, so I guess this should be addressed there too.



On Tue, Mar 27, 2018 at 6:41 PM, Herbert Vojčík <[hidden email]> wrote:


Peter Uhnák wrote:
Hi Sven,

shouldn't it be set to #ignore by default for JSON then? Or is there a
use case where it makes sense to have something else for JSON?

To keep STON practices / assumptions while using JSON to hold the data where STON is not supported yet, maybe? IIRC STON is positioning itself as JSON superset; but maybe only syntactic superset, while semantically it can be a bit different (so STONize to more low-level, JSON, syntax via STON; JSONize faithfully to JSON semantics using NeoJSON; but maybe I am just making things up).

Herbert: I am not talking about (infinitely) recursive references, but

I know; Sven explicitly asked what to do with cycles in that case, though.

referencing the same object, e.g.


Peter

On Tue, Mar 27, 2018 at 6:08 PM, Herbert Vojčík <[hidden email]
<mailto:[hidden email]>> wrote:



    Sven Van Caekenberghe wrote:

        Hi Peter,

        Both cases will pass using this expression:

        String streamContents: [ :out |
                 STON jsonWriter
                         referencePolicy: #ignore;
                         on: out;
                         nextPut: d ].

        The problem is what to do with cycles.


    Be faithful to what JSON.stringify does in JS. See attachment. :-)

    Herby


        Sven

            On 27 Mar 2018, at 17:43, Peter Uhnák<[hidden email]
            <mailto:[hidden email]>>  wrote:

            Hi,

            I don't think this should throw an error when I am producing
            JSON.

            ```
            d := {
            'a' ->  #().
            'b' ->  #().
            } asDictionary.

            STON toJsonString: d.
            ```

            dtto this

            ```
            a := {'hm'}.

            d := {
            'a' ->  a.
            'b' ->  a.
            } asDictionary.

            STON toJsonString: d.
            ```

            Maybe I should forgo using STON>>toJson* out of laziness (it
            is preinstalled) and always go for NeoJSON?

            Peter






Reply | Threaded
Open this post in threaded view
|

Re: STON shouldn't care about shared references in JSON mode

Sven Van Caekenberghe-2
The JSON spec does (of course) not deal with cycles nor structure sharing, STON does (by design).

The class comment clearly states:

...
- referencePolicy <#normal|#ignore|#error> default is #normal
        if #normal, track and count object references and use references to implement sharing and break cycles
        if #error, track object references and signal STONWriterError when a shared reference is encountered
        if #ignore, don't track object references which might loop forever on cycles

...

For JSON production (not a main goes of STON), it was set to #error, but like I showed, it you could prefer #ignore. You have to know what to feed it.

Cycle/shared structure checking is expensive, i.e. slow(er) and uses more memory.

I am not sure it is easy to make a difference between a cycle and a shared structure, I would have to think about that.

And yes, NeoJSON will go happily in cycles.

> On 27 Mar 2018, at 18:52, Peter Uhnák <[hidden email]> wrote:
>
> Ah sorry, I'm blind.
>
> Yes, ideally it should throw up an error in case of recursive.
> And I see that NeoJSON runs into infinite cycle, so I guess this should be addressed there too.
>
>
>
> On Tue, Mar 27, 2018 at 6:41 PM, Herbert Vojčík <[hidden email]> wrote:
>
>
> Peter Uhnák wrote:
> Hi Sven,
>
> shouldn't it be set to #ignore by default for JSON then? Or is there a
> use case where it makes sense to have something else for JSON?
>
> To keep STON practices / assumptions while using JSON to hold the data where STON is not supported yet, maybe? IIRC STON is positioning itself as JSON superset; but maybe only syntactic superset, while semantically it can be a bit different (so STONize to more low-level, JSON, syntax via STON; JSONize faithfully to JSON semantics using NeoJSON; but maybe I am just making things up).
>
> Herbert: I am not talking about (infinitely) recursive references, but
>
> I know; Sven explicitly asked what to do with cycles in that case, though.
>
> referencing the same object, e.g.
>
>
> Peter
>
> On Tue, Mar 27, 2018 at 6:08 PM, Herbert Vojčík <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>
>
>     Sven Van Caekenberghe wrote:
>
>         Hi Peter,
>
>         Both cases will pass using this expression:
>
>         String streamContents: [ :out |
>                  STON jsonWriter
>                          referencePolicy: #ignore;
>                          on: out;
>                          nextPut: d ].
>
>         The problem is what to do with cycles.
>
>
>     Be faithful to what JSON.stringify does in JS. See attachment. :-)
>
>     Herby
>
>
>         Sven
>
>             On 27 Mar 2018, at 17:43, Peter Uhnák<[hidden email]
>             <mailto:[hidden email]>>  wrote:
>
>             Hi,
>
>             I don't think this should throw an error when I am producing
>             JSON.
>
>             ```
>             d := {
>             'a' ->  #().
>             'b' ->  #().
>             } asDictionary.
>
>             STON toJsonString: d.
>             ```
>
>             dtto this
>
>             ```
>             a := {'hm'}.
>
>             d := {
>             'a' ->  a.
>             'b' ->  a.
>             } asDictionary.
>
>             STON toJsonString: d.
>             ```
>
>             Maybe I should forgo using STON>>toJson* out of laziness (it
>             is preinstalled) and always go for NeoJSON?
>
>             Peter
>
>
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: STON shouldn't care about shared references in JSON mode

Herby Vojčík


Sven Van Caekenberghe wrote:
> The JSON spec does (of course) not deal with cycles nor structure sharing, STON does (by design).

That surprises me. I thought cycles are explicitly prohibited in JSON.

> The class comment clearly states:
>
> ...
> - referencePolicy<#normal|#ignore|#error>  default is #normal
> if #normal, track and count object references and use references to implement sharing and break cycles
> if #error, track object references and signal STONWriterError when a shared reference is encountered
> if #ignore, don't track object references which might loop forever on cycles
>
> ...
>
> For JSON production (not a main goes of STON), it was set to #error, but like I showed, it you could prefer #ignore. You have to know what to feed it.
>
> Cycle/shared structure checking is expensive, i.e. slow(er) and uses more memory.

Maybe not. You do not need to _count_ references actually, just
mark/unmark on enter/leave (that is, no (Identity)Dictionary is needed,
just an (Identity)Set). Assuming you do depth-first.

> I am not sure it is easy to make a difference between a cycle and a shared structure, I would have to think about that.
>
> And yes, NeoJSON will go happily in cycles.

See above.

>> On 27 Mar 2018, at 18:52, Peter Uhnák<[hidden email]>  wrote:
>>
>> Ah sorry, I'm blind.
>>
>> Yes, ideally it should throw up an error in case of recursive.
>> And I see that NeoJSON runs into infinite cycle, so I guess this should be addressed there too.
>>
>>
>>
>> On Tue, Mar 27, 2018 at 6:41 PM, Herbert Vojčík<[hidden email]>  wrote:
>>
>>
>> Peter Uhnák wrote:
>> Hi Sven,
>>
>> shouldn't it be set to #ignore by default for JSON then? Or is there a
>> use case where it makes sense to have something else for JSON?
>>
>> To keep STON practices / assumptions while using JSON to hold the data where STON is not supported yet, maybe? IIRC STON is positioning itself as JSON superset; but maybe only syntactic superset, while semantically it can be a bit different (so STONize to more low-level, JSON, syntax via STON; JSONize faithfully to JSON semantics using NeoJSON; but maybe I am just making things up).
>>
>> Herbert: I am not talking about (infinitely) recursive references, but
>>
>> I know; Sven explicitly asked what to do with cycles in that case, though.
>>
>> referencing the same object, e.g.
>>
>>
>> Peter
>>
>> On Tue, Mar 27, 2018 at 6:08 PM, Herbert Vojčík<[hidden email]
>> <mailto:[hidden email]>>  wrote:
>>
>>
>>
>>      Sven Van Caekenberghe wrote:
>>
>>          Hi Peter,
>>
>>          Both cases will pass using this expression:
>>
>>          String streamContents: [ :out |
>>                   STON jsonWriter
>>                           referencePolicy: #ignore;
>>                           on: out;
>>                           nextPut: d ].
>>
>>          The problem is what to do with cycles.
>>
>>
>>      Be faithful to what JSON.stringify does in JS. See attachment. :-)
>>
>>      Herby
>>
>>
>>          Sven
>>
>>              On 27 Mar 2018, at 17:43, Peter Uhnák<[hidden email]
>>              <mailto:[hidden email]>>   wrote:
>>
>>              Hi,
>>
>>              I don't think this should throw an error when I am producing
>>              JSON.
>>
>>              ```
>>              d := {
>>              'a' ->   #().
>>              'b' ->   #().
>>              } asDictionary.
>>
>>              STON toJsonString: d.
>>              ```
>>
>>              dtto this
>>
>>              ```
>>              a := {'hm'}.
>>
>>              d := {
>>              'a' ->   a.
>>              'b' ->   a.
>>              } asDictionary.
>>
>>              STON toJsonString: d.
>>              ```
>>
>>              Maybe I should forgo using STON>>toJson* out of laziness (it
>>              is preinstalled) and always go for NeoJSON?
>>
>>              Peter
>>
>>
>>
>>
>>
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: STON shouldn't care about shared references in JSON mode

Peter Uhnak
In reply to this post by Sven Van Caekenberghe-2
The class comment clearly states:

The docs also say "When using JSON mode the reference policy should be set to #error or #ignore for full JSON compatibility." But demonstrably setting it to #error does not give me full JSON compatibility.
 

If I feed it valid input, then I expect valid output. Now it throws up. This is a bug.

If I feed it infinite input, then I cannot be surprised it will start producing infinite output. Having a cycle detection is a nice feature, but I cannot sacrifice functionality for it.

In the end this is your call, maybe it makes sense for STON's json to be closer to STON's behavior, even if it sometimes breaks down. And I can always switch to NeoJSON if I want to be sure.

Peter

 
...
- referencePolicy <#normal|#ignore|#error> default is #normal
        if #normal, track and count object references and use references to implement sharing and break cycles
        if #error, track object references and signal STONWriterError when a shared reference is encountered
        if #ignore, don't track object references which might loop forever on cycles

...

For JSON production (not a main goes of STON), it was set to #error, but like I showed, it you could prefer #ignore. You have to know what to feed it.

Cycle/shared structure checking is expensive, i.e. slow(er) and uses more memory.

I am not sure it is easy to make a difference between a cycle and a shared structure, I would have to think about that.

And yes, NeoJSON will go happily in cycles.

> On 27 Mar 2018, at 18:52, Peter Uhnák <[hidden email]> wrote:
>
> Ah sorry, I'm blind.
>
> Yes, ideally it should throw up an error in case of recursive.
> And I see that NeoJSON runs into infinite cycle, so I guess this should be addressed there too.
>
>
>
> On Tue, Mar 27, 2018 at 6:41 PM, Herbert Vojčík <[hidden email]> wrote:
>
>
> Peter Uhnák wrote:
> Hi Sven,
>
> shouldn't it be set to #ignore by default for JSON then? Or is there a
> use case where it makes sense to have something else for JSON?
>
> To keep STON practices / assumptions while using JSON to hold the data where STON is not supported yet, maybe? IIRC STON is positioning itself as JSON superset; but maybe only syntactic superset, while semantically it can be a bit different (so STONize to more low-level, JSON, syntax via STON; JSONize faithfully to JSON semantics using NeoJSON; but maybe I am just making things up).
>
> Herbert: I am not talking about (infinitely) recursive references, but
>
> I know; Sven explicitly asked what to do with cycles in that case, though.
>
> referencing the same object, e.g.
>
>
> Peter
>
> On Tue, Mar 27, 2018 at 6:08 PM, Herbert Vojčík <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>
>
>     Sven Van Caekenberghe wrote:
>
>         Hi Peter,
>
>         Both cases will pass using this expression:
>
>         String streamContents: [ :out |
>                  STON jsonWriter
>                          referencePolicy: #ignore;
>                          on: out;
>                          nextPut: d ].
>
>         The problem is what to do with cycles.
>
>
>     Be faithful to what JSON.stringify does in JS. See attachment. :-)
>
>     Herby
>
>
>         Sven
>
>             On 27 Mar 2018, at 17:43, Peter Uhnák<[hidden email]
>             <mailto:[hidden email]>>  wrote:
>
>             Hi,
>
>             I don't think this should throw an error when I am producing
>             JSON.
>
>             ```
>             d := {
>             'a' ->  #().
>             'b' ->  #().
>             } asDictionary.
>
>             STON toJsonString: d.
>             ```
>
>             dtto this
>
>             ```
>             a := {'hm'}.
>
>             d := {
>             'a' ->  a.
>             'b' ->  a.
>             } asDictionary.
>
>             STON toJsonString: d.
>             ```
>
>             Maybe I should forgo using STON>>toJson* out of laziness (it
>             is preinstalled) and always go for NeoJSON?
>
>             Peter
>
>
>
>
>
>