[Q] Question on Magma (or other Persistence Engine for Squeak/Seaside)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Following method - under this message is my schema generation class
method for my personal seaside application. As you know I'm using
SQLite3 through FFI. My problem is that the speed is rather slow even
though I'm the only user if the number of record grows especially
finding records with tags.

Ok, now my real question is that can anyone provide me simple example of
Magma or other OODB or persistence engine for squeak/seaside so that I
can convert following SQLite3 based system to suggested one. My only
restriction is that squeak/seaside should not be affected by accidental
quit/close. For example, I've had problems with MinneStore when I close
squeak without saving and move/rename my folder/directory though I did
not specify fixed db path.

Thanks in advance and following is my method.

createSchema
        "self createSchema"
        | db |
        db _ SQLiteConnection fileNamed: self dbPath.

        db executeQuery: 'CREATE TABLE NOTE(NID INTEGER PRIMARY KEY
AUTOINCREMENT, TITLE STRING, DESCR STRING)'.
        db executeQuery: 'CREATE INDEX NOTETITLE ON NOTE (TITLE)'.

        db executeQuery: 'CREATE TABLE SOMEDAY(NID INTEGER PRIMARY KEY
AUTOINCREMENT, TITLE STRING, DESCR STRING)'.
        db executeQuery: 'CREATE INDEX SOMEDAYTITLE ON SOMEDAY (TITLE)'.

        db executeQuery: 'CREATE TABLE ACTION(NID INTEGER PRIMARY KEY
AUTOINCREMENT, TITLE STRING, DESCR STRING, DONE CHARACTER(1))'.
        db executeQuery: 'CREATE INDEX ACTIONTITLE ON ACTION (TITLE)'.
        db executeQuery: 'CREATE INDEX ACTIONDONE ON ACTION (DONE)'.

        db executeQuery: 'CREATE TABLE REMINDER(NID INTEGER PRIMARY KEY
AUTOINCREMENT, TITLE STRING, DESCR STRING, DONE CHARACTER(1), DUEDATE
INTEGER)'.
        db executeQuery: 'CREATE INDEX REMINDERTITLE ON REMINDER(TITLE)'.
        db executeQuery: 'CREATE INDEX REMINDERDONE ON REMINDER(DONE)'.
        db executeQuery: 'CREATE INDEX REMINDERDUEDATE ON REMINDER(DUEDATE)'.

        db executeQuery: 'CREATE TABLE WAITFOR(NID INTEGER PRIMARY KEY
AUTOINCREMENT, TITLE STRING, DESCR STRING, DONE CHARACTER(1), DUEDATE
INTEGER, OWNERS STRING)'.
        db executeQuery: 'CREATE INDEX WAITFORTITLE ON WAITFOR(TITLE)'.
        db executeQuery: 'CREATE INDEX WAITFORDONE ON WAITFOR(DONE)'.
        db executeQuery: 'CREATE INDEX WAITFORDUEDATE ON WAITFOR(DUEDATE)'.

        db executeQuery: 'CREATE TABLE TAG(TID INTEGER PRIMARY KEY
AUTOINCREMENT, NAME STRING)'.
        db executeQuery: 'CREATE INDEX TAGNAME ON TAG(NAME)'.

        db executeQuery: 'CREATE TABLE OBJTAG(RID INTEGER PRIMARY KEY
AUTOINCREMENT, NID INTEGER, TID INTEGER, NCLS STRING)'.
        db executeQuery: 'CREATE INDEX OBJTAGCLS ON OBJTAG(NCLS)'.
        db executeQuery: 'CREATE INDEX OBJTAGNID ON OBJTAG(NID)'.
        db executeQuery: 'CREATE INDEX OBJTAGTID ON OBJTAG(TID)'.
        db executeQuery: 'CREATE INDEX OBJTAGNIDCLS ON OBJTAG(NID, NCLS)'.

        db executeQuery: 'SELECT LAST_INSERT_ROWID()'.
        db close.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGHzcwQqspS1+XJHgRAo4fAJ9K4UYVditWAGHAPmy9uD+xon/t4ACeNwLm
VwyGcu8giklajGPj1WLLn9A=
=Js/x
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hilaire Fernandes wrote:
> I think there is somewere the sushi store example with Magma persistency
> you can look at. But moving from RDB to ODB you may need to adapt a bit
> your model.
>
> Hilaire
>

For me, the most difficult part is - and this is my major reason for
previous mail;

1. how to perform "SELECT something FROM ... WHERE NAME=xxx" kind of
operation in ODB.

2. if I save dictionary class to magma, and I changed or added/removed
element from dictionary, the whole dictionary is saved/restored to/from
db file? or just changed element?

Yes, I have almost no experience on this as you might already know, but
I cannot find proper example for me. I'll look on sushi store example.

Thank you.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGH4iBQqspS1+XJHgRAuuUAKDY0oQmaSiW+NUZiVhcK2XReQ2ZCQCdGdl0
Z+5GIoaT7K4arLSE4xxmOnw=
=1qlo
-----END PGP SIGNATURE-----

Hi,

> For me, the most difficult part is - and this is my major reason for
> previous mail;
>
> 1. how to perform "SELECT something FROM ... WHERE NAME=xxx" kind of operation in ODB.

In Magma:
        people := self getMagmaCollectionForPeople.
        oldies := self people where: [ :p | (p name = 'Smith') & (p age > 60) ].
        oldies do: [ :p | p doSomething ].

See Lava is you reallywant to do SQL with a Magma database.


> 2. if I save dictionary class to magma, and I changed or added/removed
> element from dictionary, the whole dictionary is saved/restored to/from
> db file? or just changed element?

the whole dictionary. If you want posher behaviour ask the Magma list to revive MagmaDictionary.
 
> Yes, I have almost no experience on this as you might already know, but
> I cannot find proper example for me. I'll look on sushi store example.

I wrote the sushi store example, so you can ask me directly.


Brent

Hi!

> For me, the most difficult part is - and this is my major reason for
> previous mail;
>
> 1. how to perform "SELECT something FROM ... WHERE NAME=xxx" kind of
> operation in ODB.

How to handle such operations vary depending on ODB.

In Magma you have very nice support for two things needed:
  - Large Collection classes. A MagmaCollection is a multi purpose "super
large" (segmented) collection.
  - Proper indexing support for fast lookups in MagmaCollections. Magma is
quite flexible in this department.

So whenever you have many objects that you need to do "selects" from you
should hold those objects in a MagmaCollection instead of a regular Squeak
collection. And to get fast lookups you need to create indexes.

See details here: http://wiki.squeak.org/squeak/2639

> 2. if I save dictionary class to magma, and I changed or added/removed
> element from dictionary, the whole dictionary is saved/restored to/from
> db file? or just changed element?

You wrote "changed or added/removed". If you change the element object
*itself* then the Dictionary isn't modified at all. But if you modify the
Dictionary itself (add or remove elements) then - yes - the whole
dictionary will be "saved/restored".

So the conclusion again is that it is perfectly fine to use regular Squeak
collections like Dictionary etc, as long as they don't get "too large".

In Gjallar we use just a handful MagmaCollections for the objects that we
get a LOT of and need to do fast "selects" from.

regards, Göran

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Brent Pinkney wrote:
I don't need SQL - I do not like it :-) - I just want to use dictionary
because I think inserting and retrieving to/from magma collection is not
O(1) like dictionary or SQL DB(I'm not sure). So if above code works
with speed, I have almost no problem on it. I think tag management part
will be next big problem for me.

>> Yes, I have almost no experience on this as you might already know, but
>> I cannot find proper example for me. I'll look on sushi store example.
>
> I wrote the sushi store example, so you can ask me directly.
>

Thanks in advance.

>
> Brent
>

Sungjin Chun
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGID7EQqspS1+XJHgRAg7UAJ0QJpV3jWNM+nxqwk5epW0o/UyolACcChP8
uaofR8jymmHmHn/QJsQGeSQ=
=ucs8
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Brent Pinkney wrote:

>> Yes, I have almost no experience on this as you might already know, but
>> I cannot find proper example for me. I'll look on sushi store example.
>
> I wrote the sushi store example, so you can ask me directly.
>

Hi,

Magma Version of Sushi Store Example (I've downloaded change set from
wiki, seasideSushiStoreWithMagma.3.cs) is rather slow. It takes almost 3
to 4 seconds to display each browse page. It seems that one magma
connection is used for one session(right?), why is this slow? For me,
it's slower than SQLite(FFI version), though I'm not tested directly.
Are there anything I missed here? If this problem is known, what other
ODB for squeak is recommended, OmniBase? SmartFileDictionary?

Thanks in advance.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGJdqSQqspS1+XJHgRAuTnAJ9+7DauBXlTTJiO0FSsSz6PGUnolgCgkTQh
JY4QnAD6rVOyGGZtm9oVlDo=
=WCIj
-----END PGP SIGNATURE-----

Hi Sungjin, 3 to 4 seconds is definitely not characteristic of Magma
performance.  Maybe for the initial connection, but after that you
should be seeing sub-second times.  Here are some performance
benchmarks for Magma:

  http://wiki.squeak.org/squeak/5606

I really wish the Sushi store demo was up to date, but no one has had
time to look at it.  But it should be pretty close..  I've only ever
even installed Seaside once (just to review the Sushi store demo,
Brent did it) so I'm not sure what has changed since then..
Certainly, if you or anyone is willing to freshen it up, I'll be happy
to help with the Magma-side of things..

thanks..


On 4/18/07, Sungjin Chun <[hidden email]> wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Chris Muller wrote:
Your comment is really good for me. My original objective for changing
SQLite to Magam(or other OODBMS) is speed problem. I thought that my
program is slow because of SQLite but after reading your comment, I've
profiled my application and found that the real problem - it takes
almost 3 seconds on my web browser to finish rendering and 900ms from
profiled result. - the slowness is due to seaside. So, I do not need to
change my repository yet (at least now) and I will test my application
with newer version of seaside - 2.7 and 2.8 - if possible.

Thank you for all you guys, thank you.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGJ00VQqspS1+XJHgRArobAJ9Ahehft/7D2YJV5NdttoiEH/f4owCeNqyb
JqIHNaqH84RkKJpCcLBTibg=
=J2eX
-----END PGP SIGNATURE-----