Automatic Object Storage To MySQL

I don't mean to get to far off topic but I figured some people on  
this mailing list might be interested.

I have written some code that allows automatic storage of Objects to  
a MySQL database without the developer having to write any SQL or  
create a single table. It does this by examining the objects i-vars  
creating a table and saving them to a database biased on what it finds.

Only a few member types are supported (hopefully more in the future)
   1. Nil
   2. String
   3. Number
   4. Boolean
   5. Date
   6. DateAndTime
   7. Float
   8. Array (Dictionary)
   9. List (OrderedCollection)
   10. Class (My not apply to languages other than Smalltalk)
   11. Children of IOSPersistence

Objects can not change type. So member var foo can not contain a  
String one minute and a Number the next. But it can contain any child  
of IOPersistence if the field type is IOPersistence.

The code is less than ideal and needs some cleanup and the  
OrderedCollection/Dictionary stuff needs some serious help.

The project can be found under the title "Automatic Object Storage To  
MySQL" on SqueakSource or  http://www.squeaksource.com/IOSPersistent/ .

I would love to get some feedback or see if there is any interest in  
this sort of thing. The implementation in my mind is not important  
but I think the idea of automatic Object to Relational mapping is.


Thanks
Will
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

On Jan 25, 2006, at 12:40 PM, William Harford wrote:

> I don't mean to get to far off topic but I figured some people on  
> this mailing list might be interested.

Thanks, Will.  I don't think it's off topic - persistence is a fairly  
common thing to discuss on this list.

This is great work to have going on.  It reminds me of ReStore for  
Dolphin - have you looked at that at all?
http://www.solutionsoft.co.uk/restore/

It would also be interesting (to me, anyway) to see if there was some  
natural integration possible between this and ROE.  Though I guess  
this is focused on MySQL (for now? How specific is it?), whereas ROE  
is focused on PostgreSQL.

Another model that might be useful inspiration (in that people seem  
to like it) is ActiveRecord from Rails.

Can you show us some code snippets of example use?

Avi
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Hi Will,

> I would love to get some feedback or see if there is any interest in
> this sort of thing. The implementation in my mind is not important
> but I think the idea of automatic Object to Relational mapping is.

sure, I am very much interested into this.

> I have written some code that allows automatic storage of Objects to
> a MySQL database without the developer having to write any SQL or
> create a single table. It does this by examining the objects i-vars
> creating a table and saving them to a database biased on what it finds.

I wonder why you are trying to guess the i-vars? Wouldn't it be much
more powerful to have a meta-model (like Magritte) where you could
specify what and how to store the values of a model? I think the code
could also benefit from a propre description, so that you could avoid
all those long manual type-checks with the nested #ifThen: clauses.

What about the performance of your library?

Cheers,
Lukas

--
Lukas Renggli
http://www.lukas-renggli.ch
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

On 25-Jan-06, at 4:16 PM, Avi Bryant wrote:
>
> This is great work to have going on.  It reminds me of ReStore for  
> Dolphin - have you looked at that at all?
> http://www.solutionsoft.co.uk/restore/
>

I appear to have the same goals as restore.

> It would also be interesting (to me, anyway) to see if there was  
> some natural integration possible between this and ROE.  Though I  
> guess this is focused on MySQL (for now? How specific is it?),  
> whereas ROE is focused on PostgreSQL.

In some ways MySQL can be though of as the lest common denominator  
(this has changed in the past couple years but it's hard to teach an  
old dog new tricks). I don't see any reason (in fact I think it would  
be a good thing) to be able to use any SQL database for storage.

I looked at ROE some time ago and thought it's method for querying  
was very nice. Something ROE like for querying results might make a  
good companion. My only hesitation would be how that might effect  
different implementation done in different languages. I planed on  
having IOSPersistent libraries for multiple languages and would like  
to keep the API similar for all implementations.

That being said...
I do not want to sacrifice usability and simplicity. If other  
languages don't have the power of smalltalk this library should not  
suffer.


>
> Can you show us some code snippets of example use?

Sure ....
A simple example... Create a new class like so

IOSPersistent subclass: #IOSPageInfo
     instanceVariableNames: 'user date fileId page'
     classVariableNames: ''
     poolDictionaries: ''
     category: 'IOS-FLHSPMerge'


IOSPersistent implements automatic getter/setter methods this may or  
may not be a good idea but so far I like it. Because of this you can  
use this class with no further actions (but you normally would) .

pObject := IOSPageInfo new.
pObject user: (AUser newWithName: 'aname'). "AUser is a child of  
IOSPersistent"
pObject date: (Date today).
pObject fileId: 1.
pObject page: 5.

To save the object pass it an instance on a JdmConnection and  call  
#save

pObject pxxDbConnection:db; save.

IOSPersistent will create the table if needed and decide to either  
UPDATE or INSERT.
It will also save any i-vars that contain an IOSPersistent object  
that have never been saved. It needs to to get the objects ID.

IOSPersistent object are resolved using a lookup table.




>
> Avi
> _______________________________________________
> Seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

On Jan 25, 2006, at 12:40 PM, William Harford wrote:

> I don't mean to get to far off topic but I figured some people on  
> this mailing list might be interested.
>
> I have written some code that allows automatic storage of Objects  
> to a MySQL database without the developer having to write any SQL  
> or create a single table. It does this by examining the objects i-
> vars creating a table and saving them to a database biased on what  
> it finds.

This sounds a lot like Og (www.nitrohq.com) for Ruby. You describe  
the instance variables of your objects and their relations, and Og  
generates the tables and does all the ORM magic behind the scenes.

> I would love to get some feedback or see if there is any interest  
> in this sort of thing. The implementation in my mind is not  
> important but I think the idea of automatic Object to Relational  
> mapping is.

I'm very much interested in this sort of thing and would love to lend  
a hand. It seems to fill the empty space for a lightweight ORM.

-Dane

_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

On 25-Jan-06, at 4:23 PM, Lukas Renggli wrote:
>
> I wonder why you are trying to guess the i-vars? Wouldn't it be much
> more powerful to have a meta-model (like Magritte) where you could
> specify what and how to store the values of a model? I think the code
> could also benefit from a propre description, so that you could avoid
> all those long manual type-checks with the nested #ifThen: clauses.

On of the core ideas of the project is to limit the amount of work a  
developer has to do. The deveolper should only have to be minimally  
aware of how the object is being stored. Think of it this way. The  
developer has already defined the class why should she have to define  
the way the class is stored. The library should be able to figure out  
how to store the object with out any input from the developer other  
than the class definition. While the library code would benefit by  
offloading some work onto the developer it's my position that the  
developer would not.

It might be nice if the developer could easily do either or. So  
either have the library determine how to store the i-var or allow the  
developer to direct the library. That might give us the best of both  
worlds (or the worst).

I agree that the current method of determining the type and saving to  
the database is kludgey and should be improved. Suggestion?



>
> What about the performance of your library?

Well it's early but so far the performance has been adequate and the  
one application I ported to this library performed much better than  
GOODS.

I would think at this point large arrays particularly arrays of  
IOSPersistent objects do to the fact that they must all be resolved  
when the object is resolved. Large trees of object would also be a  
problem but this is on the top of my list to resolve.

I would also add that I think it's ok to trade more than some  
computing performance for programmer performance. It's usualy cheeper  
to through hardware at the problem than it is to through programmers  
or programming time at the problem.

You milage may vary.

Thanks Will

_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

On 25-Jan-06, at 5:16 PM, Dane Jensen wrote:
>
> This sounds a lot like Og (www.nitrohq.com) for Ruby. You describe  
> the instance variables of your objects and their relations, and Og  
> generates the tables and does all the ORM magic behind the scenes.

I have not looked at Og but IOSPersistent does not ask the programmer  
to describe anything other declaring the class. I am lazy and like to  
do as little work as I can get away with:-).



>
> I'm very much interested in this sort of thing and would love to  
> lend a hand. It seems to fill the empty space for a lightweight ORM.

Please by all means. I think I configured the project on SqueakSource  
to allow anyone to upload. Have a look at the code (don't run  
screaming) and make any changes, additions, or ideas you would like.  
What ever strikes your fancy.

>
> -Dane
>
> _______________________________________________
> Seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

I would encourage you to keep the MySQL-specific stuff in its own classes that can be swapped to support additional databases. (I specifically avoid mysql in favor or Postgres for a bunch of reasons - licensing among them).

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of William Harford
Sent: Wednesday, January 25, 2006 12:40 PM
To: The Squeak Enterprise Aubergines Server - general discussion.
Subject: [Seaside] Automatic Object Storage To MySQL

I don't mean to get to far off topic but I figured some people on this mailing list might be interested.

I have written some code that allows automatic storage of Objects to a MySQL database without the developer having to write any SQL or create a single table. It does this by examining the objects i-vars creating a table and saving them to a database biased on what it finds.

Only a few member types are supported (hopefully more in the future)
   1. Nil
   2. String
   3. Number
   4. Boolean
   5. Date
   6. DateAndTime
   7. Float
   8. Array (Dictionary)
   9. List (OrderedCollection)
   10. Class (My not apply to languages other than Smalltalk)
   11. Children of IOSPersistence

Objects can not change type. So member var foo can not contain a String one minute and a Number the next. But it can contain any child of IOPersistence if the field type is IOPersistence.

The code is less than ideal and needs some cleanup and the OrderedCollection/Dictionary stuff needs some serious help.

The project can be found under the title "Automatic Object Storage To MySQL" on SqueakSource or  http://www.squeaksource.com/IOSPersistent/ .

I would love to get some feedback or see if there is any interest in this sort of thing. The implementation in my mind is not important but I think the idea of automatic Object to Relational mapping is.


Thanks
Will
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

On 25-Jan-06, at 7:02 PM, Blanchard, Todd wrote:

> I would encourage you to keep the MySQL-specific stuff in its own  
> classes that can be swapped to support additional databases. (I  
> specifically avoid mysql in favor or Postgres for a bunch of  
> reasons - licensing among them).

I would tend to agree. In my organization this is not much of a  
concern and would fall midway on my priority list but I would  
encourage and help anyone with abstracting out the DB stuff.

I don't do anything fancy in fact I think it would work with MySQL 3  
as well (i have not tested it) so I don't think it would be very  
difficult.

Thanks
Will
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

William Harford wrote:
> On of the core ideas of the project is to limit the amount of work a  
> developer has to do. The deveolper should only have to be minimally  
> aware of how the object is being stored. Think of it this way. The  
> developer has already defined the class why should she have to define  
> the way the class is stored. The library should be able to figure out  
> how to store the object with out any input from the developer other  
> than the class definition. While the library code would benefit by  
> offloading some work onto the developer it's my position that the  
> developer would not.

I recently watched the Ruby on Rails movie, and AFAICT
it seems to read the database each time the system runs,
to generate the needed meta-data. So the question arises:
which should be the canonical definition - the database
or the code?

Like you, I've been going from conceptual model, to code,
to database. In this case, if I change the model, I have
to figure out how to migrate the database tables and the
existing data records. In Rails, one seems to manually alter
the database, then figure out how to make the code adapt
(only the parts that aren't automatically fixed need work).

> I agree that the current method of determining the type and saving to  
> the database is kludgey and should be improved. Suggestion?

I agree with Lukas - use meta-data. The question is where to
put it (without altering Class/Behaviour to make space).
Currently, I use a class method, but I'd like something
lighter-weight. So, I've been thinking of doing something
like Monticello, which just puts rules on the category names.
In this case, the instance variable names would have a type suffix.
That feels kind of ugly, though. Maybe the accessors could
have the suffix stripped off.

--
Yanni Chiu

_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

> I agree with Lukas - use meta-data. The question is where to
> put it (without altering Class/Behaviour to make space).
> Currently, I use a class method, but I'd like something
> lighter-weight. So, I've been thinking of doing something
> like Monticello, which just puts rules on the category names.
> In this case, the instance variable names would have a type suffix.
> That feels kind of ugly, though. Maybe the accessors could
> have the suffix stripped off.

MethodAnnotations or Pragmas? Or annotations for instance variables?
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Yanni Chiu wrote:
[Hi all - new user just exploring the options of Seaside<=>Databases]

IMHO the database is canonical or you might as well not use it. The
whole point of a relational database is that it represents the structure
of your data and defines any required constraints. If you're not going
to do that, you might as well not have a RDBMS underneath your app.

To me ROE looks more promising (thanks Ari - you answered my first
question without me asking it). I want something that lets me generate
code from my database, not vice-versa. It looks like ROE can help me
with that.

--
   Richard Huxton
   Archonet Ltd
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

On 1/26/06, Philippe Marschall <[hidden email]> wrote:
> MethodAnnotations or Pragmas? Or annotations for instance variables?

The latter. I would really like to see some experimentation that
removes the last non-object warts from Squeak - the way that classes
and methods are categorized, the definition of
class/instance/classinstance vars, etcetera.
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

I just saw a demo in person and the speaker's mantra was "Favor Convention over Configuration".  IOW, they have meta model files written in xml, but you only have to specify the bits that don't follow the convention - like if you have an oddly named field that needs to be called something else in code. For instance, it assumes every primary key field is called 'id'.  You can change that per table if you want to.  

Everything else is inferred.

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Yanni Chiu

William Harford wrote:
> On of the core ideas of the project is to limit the amount of work a
> developer has to do.
...
I recently watched the Ruby on Rails movie, and AFAICT it seems to read the database each time the system runs, to generate the needed meta-data.
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Will,

This is great! I have an application waiting for this already.

On 1/25/06, William Harford <[hidden email]> wrote:

--
Jason Rogers

"Where there is no vision, the people perish..."
    Proverbs 29:18
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

I suggest you to have a look at Magritte.  You use a simple meta  
description to declare more information
that is used everywhere.

IOSPersistent subclass: #IOSPageInfo
     instanceVariableNames: 'user date fileId page'
     classVariableNames: ''
     poolDictionaries: ''
     category: 'IOS-FLHSPMerge'


IOSPageInfo class>>descriptionDate
        ^ MADateDescription starting: 1900; required; storedAs: SimpleDate

IOSPageInfo class>>fileId
        ^ MASmallIntegerDescription accessors: fieldId;

Note that this is not Magritte but this was to gave you the idea.

How do you handle relationships in your approach?

Stef
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

On 26-Jan-06, at 4:07 PM, stephane ducasse wrote:


>
> I suggest you to have a look at Magritte.  You use a simple meta  
> description to declare more information
> that is used everywhere.

Currently you can not specify field types. The field type is guessed  
at the time of table creation from the contents of the i-var if nil  
nothing is created. Shall we say lazy table creation ?

Specifying or more accurately allowing the support of more types is  
top of the priority list but the object knowing how to store it's  
self with out the programmer telling is the main goal of IOSPersistent.


>
> How do you handle relationships in your approach?

For IOSPersistent objects I use a lookup table. So an i-var that  
contains an IOSPersistent object can at any time store any  
IOSPersistent object or nil.

Other than nil i-vars can not change there types. Once a string  
always a string in every instance unless you create a IOSPersistent  
proxy object (it's easy to do).

>
> Stef
> _______________________________________________
> Seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside