my fear of database design and Seaside

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

my fear of database design and Seaside

Chris Dawson
Hi all,

I'm plugging away and enjoying learning about Seaside.  From my limited reading so far I gather that there are massive benefits to using something like Magma or GemStone/S over traditional ORM into RDBMSs.  If I understand from attending Randal Schwartz's great talk at BarCamp here in Portland a few weeks back that Seaside can take complicated object structures and just stick them into a object database as-is without the overhead of mapping that structure into SQL, and that this is powerful and fast.  My concern as I design my application is, however, that I despite knowing the basics of database design and simple normalization that I will do something stupid and create structures that are not scalable or searchable.  When I use a traditional database I assume that the database designers have thought of a lot of the details of implementation and are forcing me into making choices about how to store the data so that it is at least moderately searchable.  I'm not saying that most of the people on this list could not build the right structures as Smalltalk objects.  I'm saying I question my own ability to do so, and Seaside seems to enable me to do this, which might be a great joy for you all, but which might be for the worst in my case.  If this is true, perhaps I should use GLORP over Postgres, yet I worry I will lose some of the magic pixie dust that made Seaside seem so different.  Or, should I not worry about this, as it will always be faster and more scalable to drop in a cluster of GemStones and do a dictionary lookup in my code than it will be to do a join in a MySQL database?

I do love learning about Seaside, it is stretching my brain is such nice ways.  Thanks in advance for your comments.

Chris

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: my fear of database design and Seaside

Avi Bryant-2
Hi Chris,

My advice is to not worry about it for now.  You will, no doubt, eventually hit bottlenecks in your code because of the way you have set up your data structures, but I predict that
a) you will get surprisingly far before this happens (linear scans of in-memory data scale better than you think), and
b) when you do, it won't be a huge deal to add the indexing you need to make it fast.

One thing you might want to consider in your design which will make life easier for you later, is having some kind of notification to a central manager object that a given piece of data has changed.  Even if you don't do anything with it yet, it will eventually probably be useful to know when you have to update indices, invalidate caches, and so on.

Avi


On Thu, May 15, 2008 at 5:27 PM, Chris Dawson <[hidden email]> wrote:
Hi all,

I'm plugging away and enjoying learning about Seaside.  From my limited reading so far I gather that there are massive benefits to using something like Magma or GemStone/S over traditional ORM into RDBMSs.  If I understand from attending Randal Schwartz's great talk at BarCamp here in Portland a few weeks back that Seaside can take complicated object structures and just stick them into a object database as-is without the overhead of mapping that structure into SQL, and that this is powerful and fast.  My concern as I design my application is, however, that I despite knowing the basics of database design and simple normalization that I will do something stupid and create structures that are not scalable or searchable.  When I use a traditional database I assume that the database designers have thought of a lot of the details of implementation and are forcing me into making choices about how to store the data so that it is at least moderately searchable.  I'm not saying that most of the people on this list could not build the right structures as Smalltalk objects.  I'm saying I question my own ability to do so, and Seaside seems to enable me to do this, which might be a great joy for you all, but which might be for the worst in my case.  If this is true, perhaps I should use GLORP over Postgres, yet I worry I will lose some of the magic pixie dust that made Seaside seem so different.  Or, should I not worry about this, as it will always be faster and more scalable to drop in a cluster of GemStones and do a dictionary lookup in my code than it will be to do a join in a MySQL database?

I do love learning about Seaside, it is stretching my brain is such nice ways.  Thanks in advance for your comments.

Chris

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside



_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: my fear of database design and Seaside

James Robertson-7
You might well be interested in what we (Cincom) are doing with Seaside and databases as well.  We will be pushing out ActiveRecord support on top of the GLORP ORM.

James Robertson
Cincom Smalltalk Product Evangelist
Talk Small and Carry a Big Class Library




On May 15, 2008, at 9:22 PM, Avi Bryant wrote:

Hi Chris,

My advice is to not worry about it for now.  You will, no doubt, eventually hit bottlenecks in your code because of the way you have set up your data structures, but I predict that
a) you will get surprisingly far before this happens (linear scans of in-memory data scale better than you think), and
b) when you do, it won't be a huge deal to add the indexing you need to make it fast.

One thing you might want to consider in your design which will make life easier for you later, is having some kind of notification to a central manager object that a given piece of data has changed.  Even if you don't do anything with it yet, it will eventually probably be useful to know when you have to update indices, invalidate caches, and so on.

Avi


On Thu, May 15, 2008 at 5:27 PM, Chris Dawson <[hidden email]> wrote:
Hi all,

I'm plugging away and enjoying learning about Seaside.  From my limited reading so far I gather that there are massive benefits to using something like Magma or GemStone/S over traditional ORM into RDBMSs.  If I understand from attending Randal Schwartz's great talk at BarCamp here in Portland a few weeks back that Seaside can take complicated object structures and just stick them into a object database as-is without the overhead of mapping that structure into SQL, and that this is powerful and fast.  My concern as I design my application is, however, that I despite knowing the basics of database design and simple normalization that I will do something stupid and create structures that are not scalable or searchable.  When I use a traditional database I assume that the database designers have thought of a lot of the details of implementation and are forcing me into making choices about how to store the data so that it is at least moderately searchable.  I'm not saying that most of the people on this list could not build the right structures as Smalltalk objects.  I'm saying I question my own ability to do so, and Seaside seems to enable me to do this, which might be a great joy for you all, but which might be for the worst in my case.  If this is true, perhaps I should use GLORP over Postgres, yet I worry I will lose some of the magic pixie dust that made Seaside seem so different.  Or, should I not worry about this, as it will always be faster and more scalable to drop in a cluster of GemStones and do a dictionary lookup in my code than it will be to do a join in a MySQL database?

I do love learning about Seaside, it is stretching my brain is such nice ways.  Thanks in advance for your comments.

Chris

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside


_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside


_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: my fear of database design and Seaside

jgfoster
In reply to this post by Chris Dawson
Chris,

While Avi and James have adequately covered the serious responses, I  
couldn't help but chuckle about your assumption that database  
designers have thought a lot about the design of the typical  
application's database. I'm sure that such people exist and that they  
could do this work--I've just never seen it happen on a real project.  
Mostly the developers add features (based on customer demands) until  
it gets too complex to manage. If that is what is going to happen, you  
will get a lot further if you can avoid (OODBMS) or simplify (GLORP))  
one level of complexity (O/R mapping).

James

On May 16, 2008, at 2:27 AM, Chris Dawson wrote:

> Hi all,
>
> I'm plugging away and enjoying learning about Seaside.  From my  
> limited reading so far I gather that there are massive benefits to  
> using something like Magma or GemStone/S over traditional ORM into  
> RDBMSs.  If I understand from attending Randal Schwartz's great talk  
> at BarCamp here in Portland a few weeks back that Seaside can take  
> complicated object structures and just stick them into a object  
> database as-is without the overhead of mapping that structure into  
> SQL, and that this is powerful and fast.  My concern as I design my  
> application is, however, that I despite knowing the basics of  
> database design and simple normalization that I will do something  
> stupid and create structures that are not scalable or searchable.  
> When I use a traditional database I assume that the database  
> designers have thought of a lot of the details of implementation and  
> are forcing me into making choices about how to store the data so  
> that it is at least moderately searchable.  I'm not saying that most  
> of the people on this list could not build the right structures as  
> Smalltalk objects.  I'm saying I question my own ability to do so,  
> and Seaside seems to enable me to do this, which might be a great  
> joy for you all, but which might be for the worst in my case.  If  
> this is true, perhaps I should use GLORP over Postgres, yet I worry  
> I will lose some of the magic pixie dust that made Seaside seem so  
> different.  Or, should I not worry about this, as it will always be  
> faster and more scalable to drop in a cluster of GemStones and do a  
> dictionary lookup in my code than it will be to do a join in a MySQL  
> database?
>
> I do love learning about Seaside, it is stretching my brain is such  
> nice ways.  Thanks in advance for your comments.
>
> Chris
> _______________________________________________
> seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: my fear of database design and Seaside

NorbertHartl
In reply to this post by Chris Dawson
Hi,

that is exactly the question that drove me over a year ago. I started my
project using GOODS. Than I discovered magma and developed the very
first prototype. And then there was doubt. I haven't dealt much with
OODBs until then. Letting the fear win I changed the code to use Glorp
instead. And I must confess that this was the biggest fault I've done
in this project. Don't get me wrong it is not Glorp. Glorp is a wonder-
ful piece of software. It is just ORM that doesn't fit well in a lot
of scenarios you might develop. You can't do arbitrary references
between objects. Inheritance is also not easy to use. ...In the end you
deal a lot with the ORM mapping tool and even SQL. The optimization
isn't done automatically by the ORM tool. You have to know how to get
the ORM tool to work in an optimized way. If your model isn't that
complex and interconnected you have a chance to get it right with Glorp.
In my case I had to spent a lot of work on using Glorp and my stuff is
still incredibly slow. The benefits I get from using portgres is that
you get a fulltext retrieval engine and a geo database for free.
If I would start all over again taking my experience and the changed
situation that there is now gemstone with seaside I would give it a try.
On the other side I would recommend that you start simple. Don't think
about zillions of request you surely will have in a few months :).
Changing the persistence layer isn't that difficult. You could start
developing without a database (the squeak image is one). Then if you
need extra stuff change it. At the time you want to met certain feature,
concurrency or speed requirements I guess you'll know what to use.

Norbert

On Thu, 2008-05-15 at 17:27 -0700, Chris Dawson wrote:

> Hi all,
>
> I'm plugging away and enjoying learning about Seaside.  From my
> limited reading so far I gather that there are massive benefits to
> using something like Magma or GemStone/S over traditional ORM into
> RDBMSs.  If I understand from attending Randal Schwartz's great talk
> at BarCamp here in Portland a few weeks back that Seaside can take
> complicated object structures and just stick them into a object
> database as-is without the overhead of mapping that structure into
> SQL, and that this is powerful and fast.  My concern as I design my
> application is, however, that I despite knowing the basics of database
> design and simple normalization that I will do something stupid and
> create structures that are not scalable or searchable.  When I use a
> traditional database I assume that the database designers have thought
> of a lot of the details of implementation and are forcing me into
> making choices about how to store the data so that it is at least
> moderately searchable.  I'm not saying that most of the people on this
> list could not build the right structures as Smalltalk objects.  I'm
> saying I question my own ability to do so, and Seaside seems to enable
> me to do this, which might be a great joy for you all, but which might
> be for the worst in my case.  If this is true, perhaps I should use
> GLORP over Postgres, yet I worry I will lose some of the magic pixie
> dust that made Seaside seem so different.  Or, should I not worry
> about this, as it will always be faster and more scalable to drop in a
> cluster of GemStones and do a dictionary lookup in my code than it
> will be to do a join in a MySQL database?
>
> I do love learning about Seaside, it is stretching my brain is such
> nice ways.  Thanks in advance for your comments.
>
> Chris
> _______________________________________________
> seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: my fear of database design and Seaside

tblanchard
In reply to this post by Chris Dawson
Actually, Seaside doesn't do anything at all for data persistence.  
Seaside is totally about user interface and application state.  That  
said, there are a lot of options for storing data outside of the  
application - use whatever you like.


On May 15, 2008, at 5:27 PM, Chris Dawson wrote:

> Hi all,
>
> I'm plugging away and enjoying learning about Seaside.  From my  
> limited reading so far I gather that there are massive benefits to  
> using something like Magma or GemStone/S over traditional ORM into  
> RDBMSs.  If I understand from attending Randal Schwartz's great talk  
> at BarCamp here in Portland a few weeks back that Seaside can take  
> complicated object structures and just stick them into a object  
> database as-is without the overhead of mapping that structure into  
> SQL, and that this is powerful and fast.  My concern as I design my  
> application is, however, that I despite knowing the basics of  
> database design and simple normalization that I will do something  
> stupid and create structures that are not scalable or searchable.  
> When I use a traditional database I assume that the database  
> designers have thought of a lot of the details of implementation and  
> are forcing me into making choices about how to store the data so  
> that it is at least moderately searchable.  I'm not saying that most  
> of the people on this list could not build the right structures as  
> Smalltalk objects.  I'm saying I question my own ability to do so,  
> and Seaside seems to enable me to do this, which might be a great  
> joy for you all, but which might be for the worst in my case.  If  
> this is true, perhaps I should use GLORP over Postgres, yet I worry  
> I will lose some of the magic pixie dust that made Seaside seem so  
> different.  Or, should I not worry about this, as it will always be  
> faster and more scalable to drop in a cluster of GemStones and do a  
> dictionary lookup in my code than it will be to do a join in a MySQL  
> database?
>
> I do love learning about Seaside, it is stretching my brain is such  
> nice ways.  Thanks in advance for your comments.
>
> Chris
> _______________________________________________
> seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

RE: my fear of database design and Seaside

Ramon Leon-5
> Actually, Seaside doesn't do anything at all for data persistence.  
> Seaside is totally about user interface and application state.  That  
> said, there are a lot of options for storing data outside of the  
> application - use whatever you like.

I was waiting for someone to mention that.  Rails has everything thinking
that web frameworks are about persistence.

Ramon Leon
http://onsmalltalk.com

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: my fear of database design and Seaside

Randal L. Schwartz
In reply to this post by tblanchard
>>>>> "Todd" == Todd Blanchard <[hidden email]> writes:

Todd> Actually, Seaside doesn't do anything at all for data persistence.   Seaside
Todd> is totally about user interface and application state.  That  said, there are
Todd> a lot of options for storing data outside of the  application - use whatever
Todd> you like.

And in fact, my StS talk is all about using various persistence frameworks
*with* seaside.  Enjoy.

--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<[hidden email]> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: my fear of database design and Seaside

Victor-67
Hi Randal,

Is (will be) your talk available somewhere?

Thanks,
VĂ­ctor

==============================

----- Original Message -----
From: "Randal L. Schwartz" <[hidden email]>
To: "Todd Blanchard" <[hidden email]>
Cc: "Seaside - general discussion" <[hidden email]>
Sent: Friday, May 16, 2008 11:07 PM
Subject: Re: [Seaside] my fear of database design and Seaside


>>>>>> "Todd" == Todd Blanchard <[hidden email]> writes:
>
> Todd> Actually, Seaside doesn't do anything at all for data persistence.
> Seaside
> Todd> is totally about user interface and application state.  That  said,
> there are
> Todd> a lot of options for storing data outside of the  application - use
> whatever
> Todd> you like.
>
> And in fact, my StS talk is all about using various persistence frameworks
> *with* seaside.  Enjoy.
>
> --
> Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777
> 0095
> <[hidden email]> <URL:http://www.stonehenge.com/merlyn/>
> Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
> See http://methodsandmessages.vox.com/ for Smalltalk and Seaside
> discussion
> _______________________________________________
> seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: my fear of database design and Seaside

Randal L. Schwartz
>>>>> "Victor" == Victor  <[hidden email]> writes:

Victor> Hi Randal,
Victor> Is (will be) your talk available somewhere?

When I finish writing it, yes. :)

Every talk I make that isn't a full course will be released under creative
commons, non commercial, share alike.

--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<[hidden email]> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside