CouchDB was: MySQL question?
Locked
 
|
Igor, I noticed your CouchDB library on SqueakSource, but browsing from the web interface it looked like a fork of the JSON libraries and I couldn't quite tell what was going on. What is the status of this? Is it done and is there a tutorial?
Mike |
Re: CouchDB was: MySQL question?
|
|
2010/1/12 Mike Hales <[hidden email]>:
>> >> >> Or use better DB - CouchDB ;) >> > Igor, I noticed your CouchDB library on SqueakSource, but browsing from the > web interface it looked like a fork of the JSON libraries and I couldn't > quite tell what was going on. What is the status of this? Is it done and is > there a tutorial? I forked the JSON library to not interfere with existing one. My main point of displeasure with original, is that JsonObject using a collection of key/value pairs , held in OrderedCollection, while in my implementation a JsonObject is a subclass of Dictionary, which makes accessing properties much faster. Also, an original CouchDB package (http://www.squeaksource.com/CouchDB) using a curl plugin for speaking with database. While my implementation using a sockets, provided by squeak, which allows me to read the data in streaming mode, so i could start parsing a json document, even if its not fully retrieved yet i.e.: stream := (db streaming documentAt: '12345') readStream. json := Json readFrom: stream. > Mike -- Best regards, Igor Stasenko AKA sig. |
Re: CouchDB was: MySQL question?
|
|
On Tue, 12 Jan 2010, Igor Stasenko wrote:
> 2010/1/12 Mike Hales <[hidden email]>: >>> >>> >>> Or use better DB - CouchDB ;) >>> >> Igor, I noticed your CouchDB library on SqueakSource, but browsing from the >> web interface it looked like a fork of the JSON libraries and I couldn't >> quite tell what was going on. What is the status of this? Is it done and is >> there a tutorial? > > I forked the JSON library to not interfere with existing one. > My main point of displeasure with original, is that JsonObject using a > collection > of key/value pairs , held in OrderedCollection, > while in my implementation a JsonObject is a subclass of Dictionary, > which makes accessing properties much faster. Older versions (up to JSON-ul.15) used a Dictionary. OrderedCollection was a bad idea performance-wise. Levente > > Also, an original CouchDB package (http://www.squeaksource.com/CouchDB) > using a curl plugin for speaking with database. > While my implementation using a sockets, provided by squeak, which allows me > to read the data in streaming mode, so i could start parsing a json > document, even > if its not fully retrieved yet i.e.: > > stream := (db streaming documentAt: '12345') readStream. > json := Json readFrom: stream. > >> Mike > > > -- > Best regards, > Igor Stasenko AKA sig. > > |
Re: CouchDB was: MySQL question?
|
|
In reply to this post by Igor Stasenko
2010/1/12 Igor Stasenko <[hidden email]>:
> 2010/1/12 Mike Hales <[hidden email]>: >>> >>> >>> Or use better DB - CouchDB ;) >>> >> Igor, I noticed your CouchDB library on SqueakSource, but browsing from the >> web interface it looked like a fork of the JSON libraries and I couldn't >> quite tell what was going on. What is the status of this? Is it done and is >> there a tutorial? > > I forked the JSON library to not interfere with existing one. > My main point of displeasure with original, is that JsonObject using a > collection > of key/value pairs , held in OrderedCollection, > while in my implementation a JsonObject is a subclass of Dictionary, > which makes accessing properties much faster. > > Also, an original CouchDB package (http://www.squeaksource.com/CouchDB) > using a curl plugin for speaking with database. > While my implementation using a sockets, provided by squeak, which allows me > to read the data in streaming mode, so i could start parsing a json > document, even > if its not fully retrieved yet i.e.: > > stream := (db streaming documentAt: '12345') readStream. > json := Json readFrom: stream. > it is just: json := db streaming documentAt: '12345'. or: json := db documentAt: '12345'. the difference between these two, that first will start parsing data once it available, while second will wait till all document data will be retrieved. Currently, there are some subtle bugs in JSON parsing code, like using #peek: which prevents it from working with SocketStream. Either SocketStream needs fixing, or JSON parser, to avoid using it. >> Mike > > > -- > Best regards, > Igor Stasenko AKA sig. > -- Best regards, Igor Stasenko AKA sig. |
Re: CouchDB was: MySQL question?
|
|
In reply to this post by Levente Uzonyi-2
2010/1/12 Levente Uzonyi <[hidden email]>:
> On Tue, 12 Jan 2010, Igor Stasenko wrote: > >> 2010/1/12 Mike Hales <[hidden email]>: >>>> >>>> >>>> Or use better DB - CouchDB ;) >>>> >>> Igor, I noticed your CouchDB library on SqueakSource, but browsing from >>> the >>> web interface it looked like a fork of the JSON libraries and I couldn't >>> quite tell what was going on. What is the status of this? Is it done and >>> is >>> there a tutorial? >> >> I forked the JSON library to not interfere with existing one. >> My main point of displeasure with original, is that JsonObject using a >> collection >> of key/value pairs , held in OrderedCollection, >> while in my implementation a JsonObject is a subclass of Dictionary, >> which makes accessing properties much faster. > > Older versions (up to JSON-ul.15) used a Dictionary. OrderedCollection was a > bad idea performance-wise. > subclassing a Dictionary. Because in some places i need to tell the difference between Dictionary and JsonObject because i want to be allow to serialize and deserialize arbitrary smalltalk objects as documents, including collections and dictionaries , of course. So, when deserializing, i need to determine somehow, what instance to create - a Dictionary or another JsonObject. > > Levente > >> >> Also, an original CouchDB package (http://www.squeaksource.com/CouchDB) >> using a curl plugin for speaking with database. >> While my implementation using a sockets, provided by squeak, which allows >> me >> to read the data in streaming mode, so i could start parsing a json >> document, even >> if its not fully retrieved yet i.e.: >> >> stream := (db streaming documentAt: '12345') readStream. >> json := Json readFrom: stream. >> >>> Mike >> >> >> -- >> Best regards, >> Igor Stasenko AKA sig. >> >> > > -- Best regards, Igor Stasenko AKA sig. |
|
|
I made a Json parser in VisualWorks for our product, and I used an OrderedDictionary, because I wanted to be able to parse json to a JsonObject, then emit a json string back that would be equivalent. With a Dictionary, the order would change, thus making the original and emitted json strings different. Maybe that's why an OrderedCollection was used.
Mike Mike Hales Engineering Manager KnowledgeScape www.kscape.com On Mon, Jan 11, 2010 at 5:02 PM, Igor Stasenko <[hidden email]> wrote:
|
Re: CouchDB was: MySQL question?
|
|
2010/1/12 Mike Hales <[hidden email]>:
> I made a Json parser in VisualWorks for our product, and I used an > OrderedDictionary, because I wanted to be able to parse json to a > JsonObject, then emit a json string back that would be equivalent. With a > Dictionary, the order would change, thus making the original and emitted > json strings different. Maybe that's why an OrderedCollection was used. > Mike I see. Can you tell, where it makes difference (except visual)? Is there something what depends on a specific key order? > Mike Hales > Engineering Manager > KnowledgeScape > www.kscape.com > > > On Mon, Jan 11, 2010 at 5:02 PM, Igor Stasenko <[hidden email]> wrote: >> >> 2010/1/12 Levente Uzonyi <[hidden email]>: >> > On Tue, 12 Jan 2010, Igor Stasenko wrote: >> > >> >> 2010/1/12 Mike Hales <[hidden email]>: >> >>>> >> >>>> >> >>>> Or use better DB - CouchDB ;) >> >>>> >> >>> Igor, I noticed your CouchDB library on SqueakSource, but browsing >> >>> from >> >>> the >> >>> web interface it looked like a fork of the JSON libraries and I >> >>> couldn't >> >>> quite tell what was going on. What is the status of this? Is it done >> >>> and >> >>> is >> >>> there a tutorial? >> >> >> >> I forked the JSON library to not interfere with existing one. >> >> My main point of displeasure with original, is that JsonObject using a >> >> collection >> >> of key/value pairs , held in OrderedCollection, >> >> while in my implementation a JsonObject is a subclass of Dictionary, >> >> which makes accessing properties much faster. >> > >> > Older versions (up to JSON-ul.15) used a Dictionary. OrderedCollection >> > was a >> > bad idea performance-wise. >> > >> I'm not sure yet. It maybe better to use a dictionary ivar instead of >> subclassing a Dictionary. >> Because in some places i need to tell the difference between >> Dictionary and JsonObject >> because i want to be allow to serialize and deserialize arbitrary >> smalltalk objects as documents, >> including collections and dictionaries , of course. >> So, when deserializing, i need to determine somehow, what instance to >> create - a Dictionary or >> another JsonObject. >> >> >> > >> > Levente >> > >> >> >> >> Also, an original CouchDB package (http://www.squeaksource.com/CouchDB) >> >> using a curl plugin for speaking with database. >> >> While my implementation using a sockets, provided by squeak, which >> >> allows >> >> me >> >> to read the data in streaming mode, so i could start parsing a json >> >> document, even >> >> if its not fully retrieved yet i.e.: >> >> >> >> stream := (db streaming documentAt: '12345') readStream. >> >> json := Json readFrom: stream. >> >> >> >>> Mike >> >> >> >> >> >> -- >> >> Best regards, >> >> Igor Stasenko AKA sig. >> >> >> >> >> > >> > >> >> >> >> -- >> Best regards, >> Igor Stasenko AKA sig. >> > > > > > -- Best regards, Igor Stasenko AKA sig. |
JSON "object"s are *unordered* (was Re: CouchDB was: MySQL question?)
|
|
Igor Stasenko wrote:
> Is there something what depends on a specific key order? If there is, it's busted. >From json.org: "An object is an unordered set of name/value pairs." >From RFC 4627: "An object is an unordered collection of zero or more name/value pairs, where a name is a string and a value is a string, number, boolean, null, object, or array." Both quotes are referring to the semantics of the represented values (the "infoset" if you like), rather than the transfer syntax. My question is: what motivated the change to using a specific key ordering when serializing to JSON? From examining MC history, it looks like Colin Putney introduced the change: Colin, can you comment? (In particular, since Dictionary>>= operates without regard for order of key insertion, it must have been something to do with external representation rather than manipulation of values from within the image.) Regards, Tony -- [][][] Tony Garnock-Jones | Mob: +44 (0)7905 974 211 [][] LShift Ltd | Tel: +44 (0)20 7729 7060 [] [] http://www.lshift.net/ | Email: [hidden email] |
|
|
In reply to this post by Igor Stasenko
For me, having it be ordered was only about testing. For example I wanted to be able to simply have some test like:
test: aString self assert: ( JsonObject readFrom: aString readStream ) printJson = aString. That dramatically simplified my testing of all my objects that needed to serialize to Json. Having the underlying JsonObject preserve order made that work. Having the internal representation be ordered is not incorrect, but could encourage others to rely on the order, which is incorrect. Semantically, the correct way to preserve order is to use an array element, not key value pairs. So using a Dictionary, rather than an OrderedCollection or OrderedSet should be fine. Mike Mike Hales Engineering Manager KnowledgeScape www.kscape.com On Tue, Jan 12, 2010 at 1:00 AM, Igor Stasenko <[hidden email]> wrote:
|
Re: JSON "object"s are *unordered*
|
|
In reply to this post by Tony Garnock-Jones-2
On 2010-01-12, at 8:35 AM, Tony Garnock-Jones wrote: > Igor Stasenko wrote: >> Is there something what depends on a specific key order? > > If there is, it's busted. > >> From json.org: "An object is an unordered set of name/value pairs." > >> From RFC 4627: "An object is an unordered collection of zero or more > name/value pairs, where a name is a string and a value is a string, > number, boolean, null, object, or array." > > Both quotes are referring to the semantics of the represented values > (the "infoset" if you like), rather than the transfer syntax. > > My question is: what motivated the change to using a specific key > ordering when serializing to JSON? From examining MC history, it looks > like Colin Putney introduced the change: Colin, can you comment? > > (In particular, since Dictionary>>= operates without regard for order of > key insertion, it must have been something to do with external > representation rather than manipulation of values from within the image.) I think it was for testing. I had a bunch of tests in OB-Web that asserted this or that JSON object was being sent over the wire, and didn't want to get into the circularity of parsing them again to do the comparisons. Making the serialization order consistent made the tests easier. That was meant to be a private change though, not sure how it got pushed upstream. It appears to be causing havoc; sorry about that! Colin |
Re: CouchDB was: MySQL question?
|
|
In reply to this post by Mike Hales
2010/1/12 Mike Hales <[hidden email]>:
> For me, having it be ordered was only about testing. For example I wanted to > be able to simply have some test like: > > test: aString > > self assert: ( JsonObject readFrom: aString readStream ) printJson = > aString. > If your intent to test that serialization/deserialization does not breaking the data, you may write: obj := JsonObject new at: x put: y ; at: z put: w ; ... self assert: obj = (JsonObject readFrom: obj printJson readStream) or: obj := JsonObject readFrom: '{ "x" : 5 , "y" : 10 }'. self assert: obj = (JsonObject readFrom: obj printJson readStream) The point here , that you should compare the living objects - json objects, instead of their serialized form. The way how object serializing itself, and what format its using or in what order it puts its contents is completely not interesting for us. > That dramatically simplified my testing of all my objects that needed to > serialize to Json. Having the underlying JsonObject preserve order made that > work. Having the internal representation be ordered is not incorrect, but > could encourage others to rely on the order, which is incorrect. > Semantically, the correct way to preserve order is to use an array element, > not key value pairs. So using a Dictionary, rather than an OrderedCollection > or OrderedSet should be fine. > > Mike > > Mike Hales > Engineering Manager > KnowledgeScape > www.kscape.com > > > On Tue, Jan 12, 2010 at 1:00 AM, Igor Stasenko <[hidden email]> wrote: >> >> 2010/1/12 Mike Hales <[hidden email]>: >> > I made a Json parser in VisualWorks for our product, and I used an >> > OrderedDictionary, because I wanted to be able to parse json to a >> > JsonObject, then emit a json string back that would be equivalent. With >> > a >> > Dictionary, the order would change, thus making the original and emitted >> > json strings different. Maybe that's why an OrderedCollection was used. >> > Mike >> >> I see. Can you tell, where it makes difference (except visual)? >> Is there something what depends on a specific key order? >> >> > Mike Hales >> > Engineering Manager >> > KnowledgeScape >> > www.kscape.com >> > >> > >> > On Mon, Jan 11, 2010 at 5:02 PM, Igor Stasenko <[hidden email]> >> > wrote: >> >> >> >> 2010/1/12 Levente Uzonyi <[hidden email]>: >> >> > On Tue, 12 Jan 2010, Igor Stasenko wrote: >> >> > >> >> >> 2010/1/12 Mike Hales <[hidden email]>: >> >> >>>> >> >> >>>> >> >> >>>> Or use better DB - CouchDB ;) >> >> >>>> >> >> >>> Igor, I noticed your CouchDB library on SqueakSource, but browsing >> >> >>> from >> >> >>> the >> >> >>> web interface it looked like a fork of the JSON libraries and I >> >> >>> couldn't >> >> >>> quite tell what was going on. What is the status of this? Is it >> >> >>> done >> >> >>> and >> >> >>> is >> >> >>> there a tutorial? >> >> >> >> >> >> I forked the JSON library to not interfere with existing one. >> >> >> My main point of displeasure with original, is that JsonObject using >> >> >> a >> >> >> collection >> >> >> of key/value pairs , held in OrderedCollection, >> >> >> while in my implementation a JsonObject is a subclass of Dictionary, >> >> >> which makes accessing properties much faster. >> >> > >> >> > Older versions (up to JSON-ul.15) used a Dictionary. >> >> > OrderedCollection >> >> > was a >> >> > bad idea performance-wise. >> >> > >> >> I'm not sure yet. It maybe better to use a dictionary ivar instead of >> >> subclassing a Dictionary. >> >> Because in some places i need to tell the difference between >> >> Dictionary and JsonObject >> >> because i want to be allow to serialize and deserialize arbitrary >> >> smalltalk objects as documents, >> >> including collections and dictionaries , of course. >> >> So, when deserializing, i need to determine somehow, what instance to >> >> create - a Dictionary or >> >> another JsonObject. >> >> >> >> >> >> > >> >> > Levente >> >> > >> >> >> >> >> >> Also, an original CouchDB package >> >> >> (http://www.squeaksource.com/CouchDB) >> >> >> using a curl plugin for speaking with database. >> >> >> While my implementation using a sockets, provided by squeak, which >> >> >> allows >> >> >> me >> >> >> to read the data in streaming mode, so i could start parsing a json >> >> >> document, even >> >> >> if its not fully retrieved yet i.e.: >> >> >> >> >> >> stream := (db streaming documentAt: '12345') readStream. >> >> >> json := Json readFrom: stream. >> >> >> >> >> >>> Mike >> >> >> >> >> >> >> >> >> -- >> >> >> Best regards, >> >> >> Igor Stasenko AKA sig. >> >> >> >> >> >> >> >> > >> >> > >> >> >> >> >> >> >> >> -- >> >> Best regards, >> >> Igor Stasenko AKA sig. >> >> >> > >> > >> > >> > >> > >> >> >> >> -- >> Best regards, >> Igor Stasenko AKA sig. >> > > > > > -- Best regards, Igor Stasenko AKA sig. |
Re: Re: JSON "object"s are *unordered*
|
|
In reply to this post by Colin Putney
Colin Putney wrote:
> I think it was for testing. Yeah, I've suffered from similar issues trying to test the other JSON libraries I've written over the years. There's also another legit(ish) reason to want a consistent serialisation order: canonicalisation for signing. I don't know of anyone doing that, yet, though :-) Perhaps I'll change it, then, so that it uses a Dictionary internally, and add some way of serializing to JSON with keys in *sorted* order, which then gives a kind of canonical form for the tests... I'll take a look this weekend, if I get time. > That was meant to be a private change though, not sure > how it got pushed upstream. Hmm, it could easily have been something *I* did actually: I can't remember where I found the .avi.24 that I merged with .ul.15. Perhaps it was from some unrelated repository. :-( If so, then sorry, all, for the hassle! Regards, Tony |
«
Return to Squeak - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |