JSON keys not quoted?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

JSON keys not quoted?

Esteban A. Maringolo
If you convert a Dictionary with numbers as keys it will produce not
valid JSON [1], since all object keys must be quoted.

E.g.
NeoJSONWriter toString: (Dictionary with: 1->'foo').
or
(Dictionary with: 1->'foo') asJson

Should the library generate quoted keys?

Seaside's WAJsonCanvas is manual, so it is up to the developer to
render the object properly, but NeoJSON does it automatically.

In Javascript this will throw a syntax error:
> JSON.parse('{1: "foo"}')

And this would generate a quoted key when serialized:
> JSON.stringify({1: "foo"})
> {"1": "foo"}

So my question is: should the library quote whatever you put as key in
order to produce valid JSON syntax? [2]

Regards.

[1] https://jsonlint.com/
[2] https://www.ecma-international.org/publications/standards/Ecma-404.htm

Esteban A. Maringolo

Reply | Threaded
Open this post in threaded view
|

Re: JSON keys not quoted?

Sven Van Caekenberghe-2
Good catch.

Indeed, JSON keys have to be strings.

The question is, should the writer convert them for you, or should you simply not use them ?

I mean, there are many Pharo constructs that cannot be converted to JSON.

Some could be output to JSON, but you won't be able to read them back in.

So, if conversion would happen, the dictionary with integer keys will be read back with strings keys.

In STON, keys can be anything, even complex objects, like in Pharo itself.

What will you do with { (1@2)->100 } asDictionary in JSON ?

I have to think about this.

> On 22 Feb 2019, at 21:21, Esteban Maringolo <[hidden email]> wrote:
>
> If you convert a Dictionary with numbers as keys it will produce not
> valid JSON [1], since all object keys must be quoted.
>
> E.g.
> NeoJSONWriter toString: (Dictionary with: 1->'foo').
> or
> (Dictionary with: 1->'foo') asJson
>
> Should the library generate quoted keys?
>
> Seaside's WAJsonCanvas is manual, so it is up to the developer to
> render the object properly, but NeoJSON does it automatically.
>
> In Javascript this will throw a syntax error:
>> JSON.parse('{1: "foo"}')
>
> And this would generate a quoted key when serialized:
>> JSON.stringify({1: "foo"})
>> {"1": "foo"}
>
> So my question is: should the library quote whatever you put as key in
> order to produce valid JSON syntax? [2]
>
> Regards.
>
> [1] https://jsonlint.com/
> [2] https://www.ecma-international.org/publications/standards/Ecma-404.htm
>
> Esteban A. Maringolo
>


Reply | Threaded
Open this post in threaded view
|

Re: JSON keys not quoted?

Esteban A. Maringolo
El vie., 22 feb. 2019 a las 18:24, Sven Van Caekenberghe
(<[hidden email]>) escribió:

>
> Good catch.
>
> Indeed, JSON keys have to be strings.
>
> The question is, should the writer convert them for you, or should you simply not use them ?
>
> I mean, there are many Pharo constructs that cannot be converted to JSON.
>
> Some could be output to JSON, but you won't be able to read them back in.
> So, if conversion would happen, the dictionary with integer keys will be read back with strings keys.

This is the same as with any object, you can serialize any object to a
JSON object ({"foo": "baz"...}), but you can't read it back UNLESS you
know exactly what to convert it to.

Maybe there should be an "integer" mapping that converts the number in
string representation back to an Integer when deserializing it.


> In STON, keys can be anything, even complex objects, like in Pharo itself.

That's why STON is better. :)


> What will you do with { (1@2)->100 } asDictionary in JSON ?

That is not valid JSON and not even JS. I mean, AFAIU in JS properties
are also strings, even array indices.

Try to do this:
> JSON.stringify({{x: 1, y: 2}: 100})

And it will fail.

E.g.
> point = {x: 1, y: 2};
> object = {point: 100}
> object
< {point: 100}

There the "point" is a literal, not a variable reference.

But if instead you do
> point = {x: 1, y: 2};
> object = {}.
> object[point] = 100;
> object
< {[object Object]: 100}

you get a frankenstein because the key got converted to a string:

> Object.keys(obj)
< ["[object Object]"]

The issue here is that the JSON writer creates invalid JSON syntax.
So it's either JSON (valid) or it's something else.

Not that I'm a fan of this, but I got bitten when submitting a
document to CouchDB which got rejected because of this invalid syntax.

To make things worse:
> object = {1: 'foo'}
> object[1]
< "foo"
> object["1"]
< "foo"

WAT! [1]


> I have to think about this.

Food for thought for the weekend.


Regards,



[1] https://www.destroyallsoftware.com/talks/wat

Esteban A. Maringolo

Reply | Threaded
Open this post in threaded view
|

Re: JSON keys not quoted?

Sven Van Caekenberghe-2


> On 22 Feb 2019, at 22:52, Esteban Maringolo <[hidden email]> wrote:
>
> El vie., 22 feb. 2019 a las 18:24, Sven Van Caekenberghe
> (<[hidden email]>) escribió:
>>
>> Good catch.
>>
>> Indeed, JSON keys have to be strings.
>>
>> The question is, should the writer convert them for you, or should you simply not use them ?
>>
>> I mean, there are many Pharo constructs that cannot be converted to JSON.
>>
>> Some could be output to JSON, but you won't be able to read them back in.
>> So, if conversion would happen, the dictionary with integer keys will be read back with strings keys.
>
> This is the same as with any object, you can serialize any object to a
> JSON object ({"foo": "baz"...}), but you can't read it back UNLESS you
> know exactly what to convert it to.
>
> Maybe there should be an "integer" mapping that converts the number in
> string representation back to an Integer when deserializing it.
>
>
>> In STON, keys can be anything, even complex objects, like in Pharo itself.
>
> That's why STON is better. :)
>
>
>> What will you do with { (1@2)->100 } asDictionary in JSON ?
>
> That is not valid JSON and not even JS. I mean, AFAIU in JS properties
> are also strings, even array indices.
>
> Try to do this:
>> JSON.stringify({{x: 1, y: 2}: 100})
>
> And it will fail.
>
> E.g.
>> point = {x: 1, y: 2};
>> object = {point: 100}
>> object
> < {point: 100}
>
> There the "point" is a literal, not a variable reference.
>
> But if instead you do
>> point = {x: 1, y: 2};
>> object = {}.
>> object[point] = 100;
>> object
> < {[object Object]: 100}
>
> you get a frankenstein because the key got converted to a string:
>
>> Object.keys(obj)
> < ["[object Object]"]
>
> The issue here is that the JSON writer creates invalid JSON syntax.
> So it's either JSON (valid) or it's something else.

OK, that is a very good point.

So you if it gave an error that would be good too ?

> Not that I'm a fan of this, but I got bitten when submitting a
> document to CouchDB which got rejected because of this invalid syntax.
>
> To make things worse:
>> object = {1: 'foo'}
>> object[1]
> < "foo"
>> object["1"]
> < "foo"
>
> WAT! [1]
>
>
>> I have to think about this.
>
> Food for thought for the weekend.
>
>
> Regards,
>
>
>
> [1] https://www.destroyallsoftware.com/talks/wat
>
> Esteban A. Maringolo


Reply | Threaded
Open this post in threaded view
|

Re: JSON keys not quoted?

Ben Coman
In reply to this post by Sven Van Caekenberghe-2
Just curiousity... Is an idempotent round trip required/expected ?

cheers -ben

On Sat, 23 Feb 2019 at 05:24, Sven Van Caekenberghe <[hidden email]> wrote:
Good catch.

Indeed, JSON keys have to be strings.

The question is, should the writer convert them for you, or should you simply not use them ?

I mean, there are many Pharo constructs that cannot be converted to JSON.

Some could be output to JSON, but you won't be able to read them back in.

So, if conversion would happen, the dictionary with integer keys will be read back with strings keys.

In STON, keys can be anything, even complex objects, like in Pharo itself.

What will you do with { (1@2)->100 } asDictionary in JSON ?

I have to think about this.

> On 22 Feb 2019, at 21:21, Esteban Maringolo <[hidden email]> wrote:
>
> If you convert a Dictionary with numbers as keys it will produce not
> valid JSON [1], since all object keys must be quoted.
>
> E.g.
> NeoJSONWriter toString: (Dictionary with: 1->'foo').
> or
> (Dictionary with: 1->'foo') asJson
>
> Should the library generate quoted keys?
>
> Seaside's WAJsonCanvas is manual, so it is up to the developer to
> render the object properly, but NeoJSON does it automatically.
>
> In Javascript this will throw a syntax error:
>> JSON.parse('{1: "foo"}')
>
> And this would generate a quoted key when serialized:
>> JSON.stringify({1: "foo"})
>> {"1": "foo"}
>
> So my question is: should the library quote whatever you put as key in
> order to produce valid JSON syntax? [2]
>
> Regards.
>
> [1] https://jsonlint.com/
> [2] https://www.ecma-international.org/publications/standards/Ecma-404.htm
>
> Esteban A. Maringolo
>


Reply | Threaded
Open this post in threaded view
|

Re: JSON keys not quoted?

Sven Van Caekenberghe-2


> On 23 Feb 2019, at 03:53, Ben Coman <[hidden email]> wrote:
>
> Just curiousity... Is an idempotent round trip required/expected ?

Not necessarily, but symmetry is important and a lack of symmetry is often a possible indication of trouble.

Anyway, since the standard is crystal clear that JSON keys/names in objects must be strings, I am enforcing that now. It is not OK to silently generate illegal JSON.

https://github.com/svenvc/NeoJSON/commit/c368bbf475f60d564cd3fb708e99b92efea8ffd3

(And I made the same change to STON in JSON mode)

> cheers -ben
>
> On Sat, 23 Feb 2019 at 05:24, Sven Van Caekenberghe <[hidden email]> wrote:
> Good catch.
>
> Indeed, JSON keys have to be strings.
>
> The question is, should the writer convert them for you, or should you simply not use them ?
>
> I mean, there are many Pharo constructs that cannot be converted to JSON.
>
> Some could be output to JSON, but you won't be able to read them back in.
>
> So, if conversion would happen, the dictionary with integer keys will be read back with strings keys.
>
> In STON, keys can be anything, even complex objects, like in Pharo itself.
>
> What will you do with { (1@2)->100 } asDictionary in JSON ?
>
> I have to think about this.
>
> > On 22 Feb 2019, at 21:21, Esteban Maringolo <[hidden email]> wrote:
> >
> > If you convert a Dictionary with numbers as keys it will produce not
> > valid JSON [1], since all object keys must be quoted.
> >
> > E.g.
> > NeoJSONWriter toString: (Dictionary with: 1->'foo').
> > or
> > (Dictionary with: 1->'foo') asJson
> >
> > Should the library generate quoted keys?
> >
> > Seaside's WAJsonCanvas is manual, so it is up to the developer to
> > render the object properly, but NeoJSON does it automatically.
> >
> > In Javascript this will throw a syntax error:
> >> JSON.parse('{1: "foo"}')
> >
> > And this would generate a quoted key when serialized:
> >> JSON.stringify({1: "foo"})
> >> {"1": "foo"}
> >
> > So my question is: should the library quote whatever you put as key in
> > order to produce valid JSON syntax? [2]
> >
> > Regards.
> >
> > [1] https://jsonlint.com/
> > [2] https://www.ecma-international.org/publications/standards/Ecma-404.htm
> >
> > Esteban A. Maringolo
> >
>
>


Reply | Threaded
Open this post in threaded view
|

Re: JSON keys not quoted?

Richard O'Keefe
In reply to this post by Esteban A. Maringolo
If numeric keys are silently written as strings when generating
JSON, you can get JSON that is technically legal but practically
ambiguous.  Start with the obvious:
   (Dictionary new: 2) at: 1 put true; at: '1' put false; asJson
=> {"1": true, "1": false}.
More subtly,
   d := Dictionary new: 2.
   d at: 1.0 put: true.
   d at: 1.0 successor put: false.
   FloatPrintPolicy
     value: InexactFloatPrintPolicy new
     during: [d asJson]
=> {"1.0": true, "1.0": false}.

All things considered, *not* accepting numeric keys seems like
the best thing to do.


On Sat, 23 Feb 2019 at 09:23, Esteban Maringolo <[hidden email]> wrote:
If you convert a Dictionary with numbers as keys it will produce not
valid JSON [1], since all object keys must be quoted.

E.g.
NeoJSONWriter toString: (Dictionary with: 1->'foo').
or
(Dictionary with: 1->'foo') asJson

Should the library generate quoted keys?

Seaside's WAJsonCanvas is manual, so it is up to the developer to
render the object properly, but NeoJSON does it automatically.

In Javascript this will throw a syntax error:
> JSON.parse('{1: "foo"}')

And this would generate a quoted key when serialized:
> JSON.stringify({1: "foo"})
> {"1": "foo"}

So my question is: should the library quote whatever you put as key in
order to produce valid JSON syntax? [2]

Regards.

[1] https://jsonlint.com/
[2] https://www.ecma-international.org/publications/standards/Ecma-404.htm

Esteban A. Maringolo