Voyage - collecting data from Mongo

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Voyage - collecting data from Mongo

Mark Rizun
Hi,

Is it possible to retrieve data from Mongo collection if it was not created via Voyage?
Meaning that I do not have a class in Pharo that would correspond to said collection (should I implement one?).

Cheers,
Mark
Reply | Threaded
Open this post in threaded view
|

Re: Voyage - collecting data from Mongo

Holger Freyther

> On 28. Apr 2017, at 14:27, Mark Rizun <[hidden email]> wrote:
>
> Hi,

Hi!


> Is it possible to retrieve data from Mongo collection if it was not created
> via Voyage?
> Meaning that I do not have a class in Pharo that would correspond to said
> collection (should I implement one?).

Yes. But you probably need to customize a bit.

1.) Make sure you refer to the right collection...

descriptionContainer
        <mongoContainer>
        ^VOMongoContainer new
                collectionName: 'yourExistingCollectionName'
                yourself


2.) For materialization.. you should at least have
        _id (okay every entry does that)
        VOMongoSerializer fieldVersion (#version on my old Voyage)
        VOMongoSerializer fieldType (#instanceOf)

and then things should work out. You can probably add #version and #instanceOf to your existing objects?



holger
Reply | Threaded
Open this post in threaded view
|

Re: Voyage - collecting data from Mongo

Stephane Ducasse-3
mark 

what is such collection?

Stef

On Fri, Apr 28, 2017 at 11:04 PM, Holger Freyther <[hidden email]> wrote:

> On 28. Apr 2017, at 14:27, Mark Rizun <[hidden email]> wrote:
>
> Hi,

Hi!


> Is it possible to retrieve data from Mongo collection if it was not created
> via Voyage?
> Meaning that I do not have a class in Pharo that would correspond to said
> collection (should I implement one?).

Yes. But you probably need to customize a bit.

1.) Make sure you refer to the right collection...

descriptionContainer
        <mongoContainer>
        ^VOMongoContainer new
                collectionName: 'yourExistingCollectionName'
                yourself


2.) For materialization.. you should at least have
        _id (okay every entry does that)
        VOMongoSerializer fieldVersion (#version on my old Voyage)
        VOMongoSerializer fieldType (#instanceOf)

and then things should work out. You can probably add #version and #instanceOf to your existing objects?



holger

Reply | Threaded
Open this post in threaded view
|

Re: Voyage - collecting data from Mongo

Mark Rizun
Hi,

Holger, thank you, I will try your suggestion. 
However, I use MongoDB 3.4, and I think that Voyage has support for only versions under 3.0. Am I right?

Stef, collection in mongo is a container for documents. For example, here three documents are inserted in collection "inventory":

db.inventory.insertMany([
   { item: "journal", qty: 25, tags: ["blank", "red"], size: { h: 14, w: 21, uom: "cm" } },
   { item: "mat", qty: 85, tags: ["gray"], size: { h: 27.9, w: 35.5, uom: "cm" } },
   { item: "mousepad", qty: 25, tags: ["gel", "blue"], size: { h: 19, w: 22.85, uom: "cm" } }
])

Mark

2017-04-30 10:37 GMT+02:00 Stephane Ducasse <[hidden email]>:
mark 

what is such collection?

Stef

On Fri, Apr 28, 2017 at 11:04 PM, Holger Freyther <[hidden email]> wrote:

> On 28. Apr 2017, at 14:27, Mark Rizun <[hidden email]> wrote:
>
> Hi,

Hi!


> Is it possible to retrieve data from Mongo collection if it was not created
> via Voyage?
> Meaning that I do not have a class in Pharo that would correspond to said
> collection (should I implement one?).

Yes. But you probably need to customize a bit.

1.) Make sure you refer to the right collection...

descriptionContainer
        <mongoContainer>
        ^VOMongoContainer new
                collectionName: 'yourExistingCollectionName'
                yourself


2.) For materialization.. you should at least have
        _id (okay every entry does that)
        VOMongoSerializer fieldVersion (#version on my old Voyage)
        VOMongoSerializer fieldType (#instanceOf)

and then things should work out. You can probably add #version and #instanceOf to your existing objects?



holger


Reply | Threaded
Open this post in threaded view
|

Re: Voyage - collecting data from Mongo

Holger Freyther

> On 30. Apr 2017, at 10:47, Mark Rizun <[hidden email]> wrote:
>
> Hi,

Hi!


> Holger, thank you, I will try your suggestion.
> However, I use MongoDB 3.4, and I think that Voyage has support for only versions under 3.0. Am I right?

oh? Is that written somewhere in the Voyage documentation? The MongoDB wire protocol is upwards compatible, so while one might not use the new features yet, your application will work.

holger
Reply | Threaded
Open this post in threaded view
|

Re: Voyage - collecting data from Mongo

Mark Rizun



oh? Is that written somewhere in the Voyage documentation?

Sorry, I wasn't very accurate in previous email.
Actually, my friend is using MongoTalk/Voyage, and she encountered a problem that it is not possible to work with databases created in 3.0+ versions of MongoDB. 
Probably this issue is related to new storage types (connection part is fine).

 MongoDB wire protocol is upwards compatible, so while one might not use the new features yet, your application will work.

OK
Reply | Threaded
Open this post in threaded view
|

Re: Voyage - collecting data from Mongo

Holger Freyther

> On 30. Apr 2017, at 11:18, Mark Rizun <[hidden email]> wrote:
>
>
>


> Sorry, I wasn't very accurate in previous email.
> Actually, my friend is using MongoTalk/Voyage, and she encountered a problem that it is not possible to work with databases created in 3.0+ versions of MongoDB.
> Probably this issue is related to new storage types (connection part is fine).

I am running a Pharo3.0 image (with an older Voyage) against MongoDB 3.2. I think I am already using "WiredTiger" as database. If you can reproduce it, we can have a look. The only issue I have seen is Sabines authentication failure with more "modern" schemes.

holger
Reply | Threaded
Open this post in threaded view
|

Re: Voyage - collecting data from Mongo

Mark Rizun

I am running a Pharo3.0 image (with an older Voyage) against MongoDB 3.2. I think I am already using "WiredTiger" as database. If you can reproduce it, we can have a look. The only issue I have seen is Sabines authentication failure with more "modern" schemes.

I have Pharo 6 and the newest Voyage + MongoDB 3.4. I successfully connected and was able to add new collections. The only problem was that I couldn't access existing collections created in MongoDB. 
I don't have access to my computer at the moment, but I will try your first suggestion and let you know how it goes.

Reply | Threaded
Open this post in threaded view
|

Re: Voyage - collecting data from Mongo

chrismihaylyk
In reply to this post by Holger Freyther
Hello, Holger!

When I'm using MongoTalk in Pharo 5 and MongoDB 3.4 I can only get a list of existing databases, using command 

Mongo default open databases. 

But when I want to get some database's collection 

(Mongo default open databaseNamed: 'SomeDatabase') collectionAt:'SomeCollection'.

it returns nil to me, although there is this collection, with some data at MongoDB.

But when I'm using MongoTalk in Pharo 5  with MongoDB storage for example 2.6 version everything works fine.
Even at Pharo 5.0, there are built-in Mongo Browser and it gives the same result - show the only list of databases.
What it the problem? Am I doing something wrong?

Thanks, Khrystyna.

2017-04-30 12:27 GMT+03:00 Holger Freyther <[hidden email]>:

> On 30. Apr 2017, at 11:18, Mark Rizun <[hidden email]> wrote:
>
>
>


> Sorry, I wasn't very accurate in previous email.
> Actually, my friend is using MongoTalk/Voyage, and she encountered a problem that it is not possible to work with databases created in 3.0+ versions of MongoDB.
> Probably this issue is related to new storage types (connection part is fine).

I am running a Pharo3.0 image (with an older Voyage) against MongoDB 3.2. I think I am already using "WiredTiger" as database. If you can reproduce it, we can have a look. The only issue I have seen is Sabines authentication failure with more "modern" schemes.

holger

Reply | Threaded
Open this post in threaded view
|

Re: Voyage - collecting data from Mongo

Stephane Ducasse-3
In reply to this post by Stephane Ducasse-3
Mark
once you have a solution could you post it because I would like to add your question to the voyage chapter.
stef

On Sun, Apr 30, 2017 at 10:37 AM, Stephane Ducasse <[hidden email]> wrote:
mark 

what is such collection?

Stef

On Fri, Apr 28, 2017 at 11:04 PM, Holger Freyther <[hidden email]> wrote:

> On 28. Apr 2017, at 14:27, Mark Rizun <[hidden email]> wrote:
>
> Hi,

Hi!


> Is it possible to retrieve data from Mongo collection if it was not created
> via Voyage?
> Meaning that I do not have a class in Pharo that would correspond to said
> collection (should I implement one?).

Yes. But you probably need to customize a bit.

1.) Make sure you refer to the right collection...

descriptionContainer
        <mongoContainer>
        ^VOMongoContainer new
                collectionName: 'yourExistingCollectionName'
                yourself


2.) For materialization.. you should at least have
        _id (okay every entry does that)
        VOMongoSerializer fieldVersion (#version on my old Voyage)
        VOMongoSerializer fieldType (#instanceOf)

and then things should work out. You can probably add #version and #instanceOf to your existing objects?



holger


Reply | Threaded
Open this post in threaded view
|

Re: Voyage - collecting data from Mongo

Mark Rizun

Mark
once you have a solution could you post it because I would like to add your question to the voyage chapter.
stef

Sure thing, Stef. 
Reply | Threaded
Open this post in threaded view
|

Re: Voyage - collecting data from Mongo

Sabine Manaa
Khrystyna ,

I also use a new version of mongo with authentication (but only old version) and wired Tiger.

I am not sure but possibly you can try to overwrite the following method to fix the problem:

MongoDatabase>>collections
| reply names | 
reply := self command: {(#listCollections -> 1)} asDictionary.
names := ((reply at: 'cursor') at: 'firstBatch') collect: [ :each | each at: #name ].
^ names collect: [ :each | MongoCollection database: self name: each ]

2017-04-30 12:24 GMT+02:00 Mark Rizun [via Smalltalk] <[hidden email]>:

Mark
once you have a solution could you post it because I would like to add your question to the voyage chapter.
stef

Sure thing, Stef. 



If you reply to this email, your message will be added to the discussion below:
http://forum.world.st/Voyage-collecting-data-from-Mongo-tp4944784p4944944.html
To start a new topic under Pharo Smalltalk Users, email [hidden email]
To unsubscribe from Pharo Smalltalk Users, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

Re: Voyage - collecting data from Mongo

chrismihaylyk
Sabine, thank you very much! It's very helpful, work fine.

2017-04-30 17:09 GMT+03:00 Sabine Manaa <[hidden email]>:
Khrystyna ,

I also use a new version of mongo with authentication (but only old version) and wired Tiger.

I am not sure but possibly you can try to overwrite the following method to fix the problem:

MongoDatabase>>collections
| reply names | 
reply := self command: {(#listCollections -> 1)} asDictionary.
names := ((reply at: 'cursor') at: 'firstBatch') collect: [ :each | each at: #name ].
^ names collect: [ :each | MongoCollection database: self name: each ]

2017-04-30 12:24 GMT+02:00 Mark Rizun [via Smalltalk] <[hidden email]>:

Mark
once you have a solution could you post it because I would like to add your question to the voyage chapter.
stef

Sure thing, Stef. 



If you reply to this email, your message will be added to the discussion below:
http://forum.world.st/Voyage-collecting-data-from-Mongo-tp4944784p4944944.html
To start a new topic under Pharo Smalltalk Users, email [hidden email]
To unsubscribe from Pharo Smalltalk Users, click here.
NAML



View this message in context: Re: Voyage - collecting data from Mongo

Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Voyage - collecting data from Mongo

Sabine Manaa
Happy to help. I got this information from Udo Schneider in slack several weeks ago.
As far as I know, he works on an update for several things concerning mongo but I don't know when he will release it.


chrismihaylyk [via Smalltalk] <[hidden email]> schrieb am So. 30. Apr. 2017 um 16:45:
Sabine, thank you very much! It's very helpful, work fine.
2017-04-30 17:09 GMT+03:00 Sabine Manaa <[hidden email]>:
Khrystyna ,

I also use a new version of mongo with authentication (but only old version) and wired Tiger.

I am not sure but possibly you can try to overwrite the following method to fix the problem:

MongoDatabase>>collections
| reply names | 
reply := self command: {(#listCollections -> 1)} asDictionary.
names := ((reply at: 'cursor') at: 'firstBatch') collect: [ :each | each at: #name ].
^ names collect: [ :each | MongoCollection database: self name: each ]
2017-04-30 12:24 GMT+02:00 Mark Rizun [via Smalltalk] <[hidden email]>:

Mark
once you have a solution could you post it because I would like to add your question to the voyage chapter.
stef

Sure thing, Stef. 



If you reply to this email, your message will be added to the discussion below:
http://forum.world.st/Voyage-collecting-data-from-Mongo-tp4944784p4944944.html
To start a new topic under Pharo Smalltalk Users, email [hidden email]
To unsubscribe from Pharo Smalltalk Users, click here.
NAML

View this message in context: Re: Voyage - collecting data from Mongo

Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
If you reply to this email, your message will be added to the discussion below:
To start a new topic under Pharo Smalltalk Users, email [hidden email]
To unsubscribe from Pharo Smalltalk Users, click here.
NAML
Reply | Threaded
Open this post in threaded view
|

Re: Voyage - collecting data from Mongo

Mark Rizun
Hi,

the solution to my initial issue is to create a class MyClass with two class side methods

MyClass class >> isVoyageRoot
^ true

MyClass class >> descriptionContainer
    <voyageContainer>
    ^ VOContainer new
        collectionName: 'myCollection';
        yourself

Also, to properly read the data one should add instance variables to depending on what is in the database. For example:

{ "_id" : ObjectId("5900a0175bc65a2b7973b48a"), "item" : "canvas", "qty" : 100, "tags" : [ "cotton" ] }

In this case MyClass should have instanceVariables: 'item qty tags' and accessors.
Plus, on the class side

mongoItem
<mongoDescription>
^ VOToOneDescription new
attributeName: 'item';
kind: String;
yourself

mongoQty
<mongoDescription>
^ VOToOneDescription new
attributeName: 'qty';
kind: Integer;
yourself

mongoTags
<mongoDescription>
^ VOToOneDescription new
attributeName: 'tags';
kind: OrderedCollection;
yourself

After that one can connect to database and get the information.

| repository | 
repository := VOMongoRepository database: 'databaseName'.
repository selectAll: MyClass

Big thanks to Sabine, Esteban and Holger!

Cheers,
Mark

2017-04-30 19:20 GMT+02:00 Sabine Manaa <[hidden email]>:
Happy to help. I got this information from Udo Schneider in slack several weeks ago.
As far as I know, he works on an update for several things concerning mongo but I don't know when he will release it.


chrismihaylyk [via Smalltalk] <[hidden email]> schrieb am So. 30. Apr. 2017 um 16:45:
Sabine, thank you very much! It's very helpful, work fine.
2017-04-30 17:09 GMT+03:00 Sabine Manaa <[hidden email]>:
Khrystyna ,

I also use a new version of mongo with authentication (but only old version) and wired Tiger.

I am not sure but possibly you can try to overwrite the following method to fix the problem:

MongoDatabase>>collections
| reply names | 
reply := self command: {(#listCollections -> 1)} asDictionary.
names := ((reply at: 'cursor') at: 'firstBatch') collect: [ :each | each at: #name ].
^ names collect: [ :each | MongoCollection database: self name: each ]
2017-04-30 12:24 GMT+02:00 Mark Rizun [via Smalltalk] <[hidden email]>:

Mark
once you have a solution could you post it because I would like to add your question to the voyage chapter.
stef

Sure thing, Stef. 



If you reply to this email, your message will be added to the discussion below:
http://forum.world.st/Voyage-collecting-data-from-Mongo-tp4944784p4944944.html
To start a new topic under Pharo Smalltalk Users, email [hidden email]
To unsubscribe from Pharo Smalltalk Users, click here.
NAML

View this message in context: Re: Voyage - collecting data from Mongo

Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
If you reply to this email, your message will be added to the discussion below:
To start a new topic under Pharo Smalltalk Users, email [hidden email]
To unsubscribe from Pharo Smalltalk Users, click here.
NAML


View this message in context: Re: Voyage - collecting data from Mongo
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Voyage - collecting data from Mongo

Stephane Ducasse-3
Tx I will add that to the Voyage booklet I will edit soon :)



On Tue, May 2, 2017 at 3:29 PM, Mark Rizun <[hidden email]> wrote:
Hi,

the solution to my initial issue is to create a class MyClass with two class side methods

MyClass class >> isVoyageRoot
^ true

MyClass class >> descriptionContainer
    <voyageContainer>
    ^ VOContainer new
        collectionName: 'myCollection';
        yourself

Also, to properly read the data one should add instance variables to depending on what is in the database. For example:

{ "_id" : ObjectId("5900a0175bc65a2b7973b48a"), "item" : "canvas", "qty" : 100, "tags" : [ "cotton" ] }

In this case MyClass should have instanceVariables: 'item qty tags' and accessors.
Plus, on the class side

mongoItem
<mongoDescription>
^ VOToOneDescription new
attributeName: 'item';
kind: String;
yourself

mongoQty
<mongoDescription>
^ VOToOneDescription new
attributeName: 'qty';
kind: Integer;
yourself

mongoTags
<mongoDescription>
^ VOToOneDescription new
attributeName: 'tags';
kind: OrderedCollection;
yourself

After that one can connect to database and get the information.

| repository | 
repository := VOMongoRepository database: 'databaseName'.
repository selectAll: MyClass

Big thanks to Sabine, Esteban and Holger!

Cheers,
Mark

2017-04-30 19:20 GMT+02:00 Sabine Manaa <[hidden email]>:
Happy to help. I got this information from Udo Schneider in slack several weeks ago.
As far as I know, he works on an update for several things concerning mongo but I don't know when he will release it.


chrismihaylyk [via Smalltalk] <[hidden email]> schrieb am So. 30. Apr. 2017 um 16:45:
Sabine, thank you very much! It's very helpful, work fine.
2017-04-30 17:09 GMT+03:00 Sabine Manaa <[hidden email]>:
Khrystyna ,

I also use a new version of mongo with authentication (but only old version) and wired Tiger.

I am not sure but possibly you can try to overwrite the following method to fix the problem:

MongoDatabase>>collections
| reply names | 
reply := self command: {(#listCollections -> 1)} asDictionary.
names := ((reply at: 'cursor') at: 'firstBatch') collect: [ :each | each at: #name ].
^ names collect: [ :each | MongoCollection database: self name: each ]
2017-04-30 12:24 GMT+02:00 Mark Rizun [via Smalltalk] <[hidden email]>:

Mark
once you have a solution could you post it because I would like to add your question to the voyage chapter.
stef

Sure thing, Stef. 



If you reply to this email, your message will be added to the discussion below:
http://forum.world.st/Voyage-collecting-data-from-Mongo-tp4944784p4944944.html
To start a new topic under Pharo Smalltalk Users, email [hidden email]
To unsubscribe from Pharo Smalltalk Users, click here.
NAML

View this message in context: Re: Voyage - collecting data from Mongo

Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
If you reply to this email, your message will be added to the discussion below:
To start a new topic under Pharo Smalltalk Users, email [hidden email]
To unsubscribe from Pharo Smalltalk Users, click here.
NAML


View this message in context: Re: Voyage - collecting data from Mongo
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.