MongoDB: console commands? query embedded indexed fields?
Locked
MongoDB: console commands? query embedded indexed fields?
 
|
Hi,
I have 2 questions concerning use of mongoDB classes: 1) how can I send mongo console commands directly from smalltalk 2) how to query on an embedded collection with an index Model: I have trips and each trip has an embedded collection of N receipts like this: { "tripName" : "trip 1", {"receipts" : [ {"receiptDescription" : "receipt 1" } {"receiptDescription" : "receipt 2" } {"receiptDescription" : "receipt 3" } ], } Index: For quick search, I created an index on the embeded[1] receipts with this command at the mongoDB console in a terminal (NOT in smalltalk) (mongo console starts with mongo in the MongoDB/bin dir): db.Trips.createIndex({"receipts.receiptDescription":1}) with a command like this in the mongoDB console db.Trips.find({"receipts.receiptDescription":"receipt 1"}) I get the resulting receipt. Before, I have to set the database with "use {databasename}" in thr mongodb console My questions: 1) how can I send mongo console commands directly from smalltalk, e.g. use database, set index etc. Is this possible? I did not succeed with the mongo command: method, it always returns "no such cmd". 2) how can I search for a receipt which is indexed as described from smalltalk? I did not succeed with the description [2] I am not sure if it is covered by it. Regards Sabine [1] http://docs.mongodb.org/manual/core/index-multikey/#index-arrays-with-embedded-documents [2] http://smallworks.eu/web/blog/2013-07-18-Voyage-advanced-queries |
Re: MongoDB: console commands? query embedded indexed fields?
|
Sabine,
Creating indexes is not yet supported. I want to do that myself. As always there seems to be to less people implementing stuff like that.
Trip selectMany: { 'receipts.receiptDescription' -> 'Tankquittung' } asDictionary should do. You can use just the nested properties form. The index will be selected automatically. If you are not sure about queries read about explain [1] to optimize that stuff. Norbert
|
Re: MongoDB: console commands? query embedded indexed fields?
|
|
Hi Norbert, I didn't want to use "use database" for queries but for defining the index from within smalltalk. But if creating indexes is not supported, I will set the indexes within the mongo console. The explain option is very helpful I did not know this. Your answer was very helpful for me, thanks a lot! Sabine 2015-08-04 10:42 GMT+02:00 NorbertHartl [via Smalltalk] <[hidden email]>: Sabine, |
Re: MongoDB: console commands? query embedded indexed fields?
|
|
Norbert,
I have to come back to your answer. With the .explain option, I can see, how performance is with and without index on the _mongo console_. As an example, I see: with index only 2 objects scanned, without: 200. Performance is perfect with index. But _from smalltalk_, it seems to iterate over all objects, also with the index set. Slow. You wrote: "The index will be selected automatically." It seems that it does not. My question: MongoQuery has an attribute named "where" which is filled with the mentioned "receipts.receiptDescription". Because of the name of this attribute "where" and the slow performance, I assume that the mongo classes use the $where command: http://docs.mongodb.org/manual/reference/operator/query/where/ Is this right? In this case with the statement Trip selectMany: { 'receipts.receiptDescription' -> 'Tankquittung' } asDictionary the index would not be set automatically. "The $where provides greater flexibility, but requires that the database processes the JavaScript expression or function for each document in the collection. " Then my question is if the mongo classes support database queries with index and if yes, how.
I am very interested in your opinion and I think other users should have the same requirements (querying indexed variables) - how did they solve this? Regards Sabine 2015-08-04 14:47 GMT+02:00 Sabine Manaa <[hidden email]>: > > Hi Norbert, > > I didn't want to use "use database" for queries but for defining the index from within smalltalk. But if creating indexes is not supported, I will set the indexes within the mongo console. > > The explain option is very helpful I did not know this. > > Your answer was very helpful for me, thanks a lot! > Sabine > > > > 2015-08-04 10:42 GMT+02:00 NorbertHartl [via Smalltalk] <[hidden email]>: >> >> Sabine, >> >> Am 04.08.2015 um 07:08 schrieb Sabine Manaa <[hidden email]>: >> >> Hi, >> >> I have 2 questions concerning use of mongoDB classes: >> 1) how can I send mongo console commands directly from smalltalk >> 2) how to query on an embedded collection with an index >> >> Model: >> I have trips and each trip has an embedded collection of N receipts like >> this: >> { "tripName" : "trip 1", >> {"receipts" : [ >> {"receiptDescription" : "receipt 1" } >> {"receiptDescription" : "receipt 2" } >> {"receiptDescription" : "receipt 3" } ], >> } >> >> Index: >> For quick search, I created an index on the embeded[1] receipts with this >> command at the mongoDB console in a terminal (NOT in smalltalk) >> (mongo console starts with mongo in the MongoDB/bin dir): >> >> db.Trips.createIndex({"receipts.receiptDescription":1}) >> >> with a command like this in the mongoDB console >> >> db.Trips.find({"receipts.receiptDescription":"receipt 1"}) >> >> I get the resulting receipt. >> >> Before, I have to set the database with "use {databasename}" in thr mongodb >> console >> >> My questions: >> 1) how can I send mongo console commands directly from smalltalk, e.g. use >> database, set index etc. >> >> >> I don't know why you want to use "use database". For queries you don't need this because you choose the database and collection by using the smalltalk classes. You can look there if something you need exists. >> Creating indexes is not yet supported. I want to do that myself. As always there seems to be to less people implementing stuff like that. >> >> Is this possible? >> I did not succeed with the mongo command: method, it always returns "no such >> cmd". >> >> 2) how can I search for a receipt which is indexed as described from >> smalltalk? >> >> >> A simple >> >> Trip selectMany: { 'receipts.receiptDescription' -> 'Tankquittung' } asDictionary >> >> should do. You can use just the nested properties form. The index will be selected automatically. If you are not sure about queries read about explain [1] to optimize that stuff. >> >> [1] http://docs.mongodb.org/manual/reference/method/cursor.explain/#cursor.explain >> >> Norbert >> >> >> I did not succeed with the description [2] I am not sure if it is covered by >> it. >> >> Regards >> Sabine >> >> >> [1] >> http://docs.mongodb.org/manual/core/index-multikey/#index-arrays-with-embedded-documents >> [2] http://smallworks.eu/web/blog/2013-07-18-Voyage-advanced-queries >> >> >> >> -- >> View this message in context: http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882.html >> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com. >> >> >> >> >> ________________________________ >> If you reply to this email, your message will be added to the discussion below: >> http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4840888.html >> To start a new topic under Pharo Smalltalk Users, email [hidden email] >> To unsubscribe from MongoDB: console commands? query embedded indexed fields?, click here. >> NAML > > > > ________________________________ > View this message in context: Re: MongoDB: console commands? query embedded indexed fields? > > Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com. |
Re: MongoDB: console commands? query embedded indexed fields?
|
|
Hi Sabine, Not related to these indexes per se but I had performance problems associated with the fact that MongoTalk asked for all collections again and again. I cached them (in my own MongoTalk version) and added debugging to the MongoOperations so that I could see what was going on. This alone sped up the thing by a huge factor. My collections have more than 20-30K entries and I am querying them extensively. No perf problem even without the indexes. How many entries are you scanning? Phil On Thu, Aug 6, 2015 at 2:49 PM, Sabine Manaa <[hidden email]> wrote:
|
Re: MongoDB: console commands? query embedded indexed fields?
|
|
In reply to this post by Sabine Manaa
Sabine,
The handling of indexes needs to be automatic. The rule of thumb is that if you use where clause like b = , c = , a= than you need an compound index in the same order being index(b,c,a). This is handled by the database when it analyzes the query it tries to find an appropriate index. Or seeing it from the other side: If you issue a query with index and the index does not fit the query it is a problem anyway. Are you sure you have only this one thing in the where query? I use e.g. COInstalledApp selectOne: { 'device.id' -> aString. 'application.__id' -> anApplication voyageId } asDictionary on a collection that has approx. 1 Mio. entires and it is fast. I didn't check with explain but I'll do if I find time :) If I were you I would step in the debugger down to the Voyage part after serialization. Then you have the exact Dictionary that is used to send to mongo. You can then use this query to do an "explain" on it and that should show you something. Another thing you can try (although I don't think makes any difference) is to use the non-javascript query variant. That would be Trip selectMany: { 'receipts' -> { 'receiptDescription' -> 'Tankquittung' } asDictionary } asDictionary Norbert
|
Re: MongoDB: console commands? query embedded indexed fields?
|
|
Hi Norbert, Hi Phil, thanks for your help! Now I know what the problem is/was. :-))) The query was right, the index was used, all is fine. BUT: I used an inspector to look at the query's result and opening the inspector forced to load all the other objects which belong to the object found. Explanation: I have a person which has N trips with N receipts, all with back pointers. Opening an Inspector with one trip loads its person and all other trips of this person hahaha. The magritte description isLazy but the inspector forced the loading of the objects in the VOMongoRepository at first time after image starting. Time millisecondsToRun: [(RKATrip selectMany: { 'receipts.receiptDescription' -> 'a' } asDictionary) size] 764 . Time millisecondsToRun: [(RKATrip selectMany: { 'receipts.receiptDescription' -> 'a' } asDictionary) inspect] 17234 So, I messed around with myself. Currently I do not have so much data but I hope in future there will be a lot and I want the application prepared for that. BTW: I found this nice document: https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/ws/Voyage/Voyage.pdf I was mentioned there, thanks! Esteban, if you want, you could change my name, I married and changed my name to Manaa. have a nice weekend Sabine 2015-08-06 15:36 GMT+02:00 NorbertHartl [via Smalltalk] <[hidden email]>: Sabine, |
Re: MongoDB: console commands? query embedded indexed fields?
|
|
Well, we call this an inversed Heisenbug, don't we? :) That is a problem we should be aware when using proxy implementations and tools like gt tools. It is very likely that a powerful tool will inquire special information on a proxy that will trigger it. It depends on the tool and your model. Are you storing those backreferences in mongo as well? Norbert
|
Re: MongoDB: console commands? query embedded indexed fields?
|
|
Hi Norbert, lol, I had to google that: https://en.wikipedia.org/wiki/Heisenbug funny but that's it. It is always a good feeling when I know what the reason for a certain behavior is. Now I can proceed with the implementation of the search functionality. no, in Mongo I don't store the back references, they are created when the objects are instantiated in pharo. Wouldn't you do it like this? What is the background for your question? Regards Sabine 2015-08-07 15:06 GMT+02:00 NorbertHartl [via Smalltalk] <[hidden email]>:
|
Re: MongoDB: console commands? query embedded indexed fields?
|
|
The background is that there is a difference between an in-memory graph and a persisted one. So storing forth and backreferences is asking for trouble. Having back references being transient and reestablishing them on materialization is a good way of having the best of both world. Norbert
|
«
Return to Pharo Smalltalk Users
|
1 view|%1 views
| Free forum by Nabble | Edit this page |