How do you handle constraints on persisted collections?

I'm trying to make the leap from RDBMS/ORM thinking to pure objects,
and am running into some problems.  The first is how to handle
constraints on collections, when the objects inside of that collection
are mutable.  Quick example, ensuring that only one user exists with a
certain username.  It's trivial to check that a username is available
before adding the user to a UserRepository.  If a user can update his
own username though, it'll bypass any check on the UserRepository and
invalidate that constraint.  A couple options...

1. Have a ChangeUsername command that reserves that username in the
repo, tells the user object to change its username, and then tells the
repo to free the original username
2. When user username: is called, publish a UsernameChanged event that
the repository subscribes to.  The repo can choose to reject the
change if it would violate the repo's constraints.

The first is just a bit ugly and procedural.  The second requires the
User object to publish events whenever something happens - which is
easy to forget to do, or you're left with determining which method
calls warrant the publishing of events.  Also you'd have to roll back
the change if the event is rejected.

You could handle that stuff by writing some framework code to manage
the tedious stuff...I don't love either approach though.  How do you
guys tackle this sort of problem?

Pat
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Pat Maddox wrote:
> I'm trying to make the leap from RDBMS/ORM thinking to pure objects,
> and am running into some problems.  The first is how to handle
> constraints on collections, when the objects inside of that collection
> are mutable.  Quick example, ensuring that only one user exists with a
> certain username.  It's trivial to check that a username is available
> before adding the user to a UserRepository.  If a user can update his
> own username though, it'll bypass any check on the UserRepository and
> invalidate that constraint.  A couple options...

Maybe there should be fields that the user is not allowed to modify if
it means so much for your app logic/consistency.

>
> 1. Have a ChangeUsername command that reserves that username in the
> repo, tells the user object to change its username, and then tells the
> repo to free the original username
> 2. When user username: is called, publish a UsernameChanged event that
> the repository subscribes to.  The repo can choose to reject the
> change if it would violate the repo's constraints.
>
I use the Memento pattern or some variation of it. The idea is to allow
the user to modify some copy of the object and not the real object. Then
after the proper valitation is done, the old object is discarded and
replaced by the new object or the old object it is applied the same
changes as the copy the user operated on. Something like this:

modifyUser: anUser
| user editor |
editor := UserEditor newFor: anUser copy. "check this method on Object"
editor validate
        ifTrue: [ ^ editor object ] "returns new object, after validation"
^ anUser "user not modified"

The method returns either the same object without modifications or a new
object that passed validation and is now the new user object.
Of course, you can make the validation message to log errors somewhere
to be shown to the user to retry the user modification.

Cheers,
Miguel Cobá



_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Pat,

In my experience this sort of constraint is somewhat rare and I don't  
feel like it needs a generalized solution. That is, it does make sense  
to require all user IDs to be unique, but multiple users with the same  
name might be okay (Sr./Jr.). In particular, you should design things  
so that it is legal for two people to have the same SSN, driver's  
license, passport number, etc.

Most values that need to be unique can be assigned by the system and  
never changed. A user ID is special in that we often want to allow the  
user to propose an ID. For this, I'd probably use a Dictionary as the  
collection and then either forbid changing the user ID or provide a  
special API for changing the user ID. You could even leave the ID off  
the User (as an instance variable) and use #'keyAtValue:' (or is it  
#'keyWithValue:' or #'keyForValue:'?) to get the ID from the  
collection. Then you would be guaranteed that only one user would have  
a particular ID (though you would have to do a bit of work to ensure  
that a user had exactly one ID).

James

On May 18, 2009, at 4:50 PM, Pat Maddox wrote:

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Hi Pat,
if the goal allows it, I'd favor a KISS approach of no more than 20 min
implementation time validating it from the presentation (components) and leave
the model less restricted and move forward.
best
sebastian

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

The short answer is you don't.

The longer answer is you're still thinking relationally; thinking in sets.
OODB's and RDBM's are different paradigms and have vastly different strength
and weaknesses.  RDBM's are much better at guaranteeing data integrity
through things such as constraints and keys rather than collections and
pointers, there's a theoretical foundation for this, but OODB's have no such
mathematical rigor.  This is a weakness of the OODB.  There is no one
provable right way to do it.

On the other hand, RDBM's have to force everything into a single fixed
schema data structure to accomplish this.  The problem is this isn't the
data structure most programs actually use.  RDBM's don't speak objects, the
primary method programs actually use to model their domain, and every
attempt to bridge that layer by automatically mapping objects to tables
leads to very serious constraints on the object model.  It becomes infected
with compromises to performance or compromises to the flexibility of the
model in order to make it fit.  Last but not least you're stuck with a whole
new layer of code to maintain, the mapping layer, that wouldn't even be
necessary with an OODB and prior to Rails was often more code than the
domain model itself.  If you really enjoy programming with objects,
relational databases will never quite fit right, there's just an impendence
mismatch that as far as I know no one has ever solved cleanly.

So for a query optimizer, declarative constraints and indexing, you have to
do a whole lot more programming.  A fair trade to some for the rigor they
require, but for those who choose the OODB, they're willing to trade the
rigor for the pragmatism of doing a ton less work and actually getting
things done that just work good enough.  I can't tell you how many RDBM's
I've run across that in production that didn't have a single foreign key in
place or use constraints at all (often for performance reasons) and chug
along just fine making plenty of money for the gold owner.  You're
experienced with Rails, so you're aware that most Rails apps don't use
foreign keys at all.  Rails treats the db like a big persistent hash table
preferring to keep the logic in the model.  This is a very object way of
doing things.  It's not perfect, but it works well enough in practice.
 
Look at what Rails has to do to try and fake polymorphic collections,
something that is so utterly trivial in an OODBMs that you don't even
realize it's anything special to do.  Toss a bunch of object into a
collection and commit; done.  The flexibility you gain in modeling things
any way you wish using any data structure you want more than makes up for
the things you lose, i.e. simple declarative constraints and a query
optimizer that lets you just slap on indexes after the fact.

With an OODB you have to think more up front and structure you domain in
such a way that your primary use cases are primarily handled by pointer
navigation.  If you're building a blog, you index the posts by the URL in a
dictionary, you put the comments into a collection in the post.  Maybe you
hang the posts off the user who wrote them as well.  When it's all setup,
there's very little if any querying going on.  Now OODB's do support indexed
collections when you need queries but if you're writing a behavioral app
rather than a reporting engine, more often than not you don't want queries
you want tree's of related objects that actually do things.  

If you need to use some special high performance custom written collection
to index your model just the way you want it, an OODB lets you, it's just
another object after all.  If you need collections of objects where every
instance could have wildly different data because you allow the user to add
custom fields, the OODB has no problem with this.  It doesn't care about the
structure of your objects.  The price you pay for that flexibility is you're
on your own, no declarative constraints or query optimizer.  On the other
hand, you're not limited in what you want to do.  A relational db just can't
do user generated fields without converting those fields into rows and
losing its queryability and performance.

Sorry, that turned into a longer post than I'd intended and I'm sure you
already know much of if not all of this, but others who are considering
OODBs might find this useful.  It's a different paradigm and requires a
different style of thinking but pragmatically has a lot of benefits if being
productive is the goal.

Anyway, my approach to your particular issue would be to simply make the
name field immutable after being set the first time.  A simple guard clause
in the accessor that threw an exception if someone tried to set the field if
it already had a value and the new value was different would protect any
unintended aliasing bugs.  After all, an object should encapsulate its own
business rules and if the field were unique I'd have modeled that by hashing
it in a dictionary by that fields value.  It's not the rigorous guarantee
that a constraint on a set is, but objects aren't about sets, they're about
instances and in any real program it's pragmatic enough to get the job done.

Ramon Leon
http://onsmalltalk.com






_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

On Mon, May 18, 2009 at 6:50 PM, Pat Maddox <[hidden email]> wrote:
> I'm trying to make the leap from RDBMS/ORM thinking to pure objects,
> and am running into some problems.  The first is how to handle
> constraints on collections, when the objects inside of that collection
> are mutable.  Quick example, ensuring that only one user exists with a
> certain username.  It's trivial to check that a username is available
> before adding the user to a UserRepository.  If a user can update his
> own username though, it'll bypass any check on the UserRepository and
> invalidate that constraint.  A couple options...

Two aspects.

First, expressing the constraint itself, to which you alluded.  In
loose Relational Model parlance you're talking about a table
constraint, specifically, a key constraint.  From the OO perspective,
I think of these as constraints on the class.  However, I implement
them as instance methods to allow reference of the instance itself
and, thus, use of a single method for both insert and update checks.
For example:

MyClass>>checkUniqueName: aString
    | count |
    count := self repository count: [ :each | each name = aString and
each ~= self ].
    (count = 0) ifFalse: [ self error: 'Constraint violation: ',
aString, ' is not unique.' ]

Second, where to send the constraint check message.  For updates, I
just send it as a guard message in every accessor that mutates the
variable in question.  For inserts, it depends on how intimately you
tie instantiation and insertion. #checkUniqueName above works
correctly whether the instance is already in the repository or not.

Cheers,

Mike
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Although, if two users are updating their usernames in separate transactions (say, in GemStone), this would do nothing to prevent both of them from succeeding, thus, this type of constraint checking really only applies when adding users to a collection which would stop this from happening by virtue of a commit conflict, no?

-Boris

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Michael Forster
Sent: Tuesday, May 19, 2009 2:59 PM
To: Seaside - general discussion
Subject: Re: [Seaside] How do you handle constraints on persisted collections?

On Mon, May 18, 2009 at 6:50 PM, Pat Maddox <[hidden email]> wrote:
> I'm trying to make the leap from RDBMS/ORM thinking to pure objects,
> and am running into some problems.  The first is how to handle
> constraints on collections, when the objects inside of that collection
> are mutable.  Quick example, ensuring that only one user exists with a
> certain username.  It's trivial to check that a username is available
> before adding the user to a UserRepository.  If a user can update his
> own username though, it'll bypass any check on the UserRepository and
> invalidate that constraint.  A couple options...

Two aspects.

First, expressing the constraint itself, to which you alluded.  In
loose Relational Model parlance you're talking about a table
constraint, specifically, a key constraint.  From the OO perspective,
I think of these as constraints on the class.  However, I implement
them as instance methods to allow reference of the instance itself
and, thus, use of a single method for both insert and update checks.
For example:

MyClass>>checkUniqueName: aString
    | count |
    count := self repository count: [ :each | each name = aString and
each ~= self ].
    (count = 0) ifFalse: [ self error: 'Constraint violation: ',
aString, ' is not unique.' ]

Second, where to send the constraint check message.  For updates, I
just send it as a guard message in every accessor that mutates the
variable in question.  For inserts, it depends on how intimately you
tie instantiation and insertion. #checkUniqueName above works
correctly whether the instance is already in the repository or not.

Cheers,

Mike
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

> Although, if two users are updating their usernames in separate
> transactions (say, in GemStone), this would do nothing to prevent both
> of them from succeeding, thus, this type of constraint checking really
> only applies when adding users to a collection which would stop this
> from happening by virtue of a commit conflict, no?
>
> -Boris

Not if you're using a reduced conflict collection to allow concurrent
updates of the collection, but otherwise yea you'd get a commit conflict.

Ramon Leon
http://onsmalltalk.com

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

On Tue, May 19, 2009 at 5:05 PM, Boris Popov <[hidden email]> wrote:
> Although, if two users are updating their usernames in separate transactions (say, in GemStone), this would do nothing to prevent both of them from succeeding, thus, this type of constraint checking really only applies when adding users to a collection which would stop this from happening by virtue of a commit conflict, no?
>
> -Boris

Yes, exactly.  I saw no indication of the UserRepository being
anything but a simple collection, so I deliberately focused on the
implementation of the constraint expression and how it might be
applied to a collection.

And thank you for raising this point. It highlights the numerous and
terribly nasty details that we forget when we dump our SQL DMBSs in a
fit of ORM frustration, start Googling for an OODBMS, and then, in
dismay, entertain brewing up our own home-grown solution. That way
lies madness.  I know.  I have used various DBMSs for twenty years,
including SQL-based ones, OO-based ones, and _real_ RDBMSs (Dataphor
and Rel).  I've also written my own partial implementations to better
understand the problems.  It's one thing to eschew the use of the
Relational Model; it's another to do so oblivious to the implications.

Mike
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Clearly this would complicate the problem even further proving my point
:)

-Boris

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Ramon
Leon
Sent: Tuesday, May 19, 2009 3:22 PM
To: 'Seaside - general discussion'
Subject: RE: [Seaside] How do you handle constraints on persisted
collections?

> Although, if two users are updating their usernames in separate
> transactions (say, in GemStone), this would do nothing to prevent both
> of them from succeeding, thus, this type of constraint checking really
> only applies when adding users to a collection which would stop this
> from happening by virtue of a commit conflict, no?
>
> -Boris

Not if you're using a reduced conflict collection to allow concurrent
updates of the collection, but otherwise yea you'd get a commit
conflict.

Ramon Leon
http://onsmalltalk.com

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

As Ramon mentioned earlier:

  "if the field were unique I'd have modeled that by hashing it in a dictionary by that fields value."

GemStone provides an RcKeyValueDictionary that allows for conflict free concurrent updates when keys do not collide. Commit conflicts occur if an attempt is made by two sessions to concurrently add or remove the same key.

If two users pick the same new name (because there is no existing key in their views), they will both attempt to commit with the new name and the second user to commit will get a commit conflict, in GLASS, the http request is retried on a commit failure and when the rename request is attempted the second time, the key will have already been taken so the slow user should be told that the name is already taken.

Dale
----- "Boris Popov" <[hidden email]> wrote:

| Clearly this would complicate the problem even further proving my
| point
| :)
|
| -Boris
|
| -----Original Message-----
| From: [hidden email]
| [mailto:[hidden email]] On Behalf Of
| Ramon
| Leon
| Sent: Tuesday, May 19, 2009 3:22 PM
| To: 'Seaside - general discussion'
| Subject: RE: [Seaside] How do you handle constraints on persisted
| collections?
|
| > Although, if two users are updating their usernames in separate
| > transactions (say, in GemStone), this would do nothing to prevent
| both
| > of them from succeeding, thus, this type of constraint checking
| really
| > only applies when adding users to a collection which would stop
| this
| > from happening by virtue of a commit conflict, no?
| >
| > -Boris
|
| Not if you're using a reduced conflict collection to allow concurrent
| updates of the collection, but otherwise yea you'd get a commit
| conflict.
|
| Ramon Leon
| http://onsmalltalk.com
|
| _______________________________________________
| seaside mailing list
| [hidden email]
| http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
| _______________________________________________
| seaside mailing list
| [hidden email]
| http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

On Mon, May 18, 2009 at 10:49 PM, Ramon Leon <[hidden email]> wrote:
> Anyway, my approach to your particular issue would be to simply make the
> name field immutable after being set the first time.  A simple guard clause
> in the accessor that threw an exception if someone tried to set the field if
> it already had a value and the new value was different would protect any
> unintended aliasing bugs.  After all, an object should encapsulate its own
> business rules and if the field were unique I'd have modeled that by hashing
> it in a dictionary by that fields value.  It's not the rigorous guarantee
> that a constraint on a set is, but objects aren't about sets, they're about
> instances and in any real program it's pragmatic enough to get the job done.

This got me going in the right direction. Here's what I did:

User>>username: is set-once, for initialization (raises an error if
you call it a second time)
UserRepository changeUsername: fromName to: toName
  | user |
  self ensureUsernameIsAvailable: toName.
  user := users at: fromName.
  users at: toName put: user.
  user replenishUsername: toName.
  users at: fromName put: nil.

This lets me create a user and add it to the repo.  Any changes to the
username have to go through the repo, where it can do its validity
checking.  I put replenishUsername in the repository protocol to make
it clear that it was meant for use by the repository.  At first I had
named the method hydrateUsername but that felt too ORMy to me, I
wanted something more organic sounding since it's pure objects :)

When I needed to add a constraint that a given email address appear in
the list only once, I created a UserProfile object that holds the
username and email.  UserProfile is fully immutable this time,
User>>profile: is write-once and User has a replenishProfile method.
UserRepository now has changeUsername:to: and changeEmail:to: that
find the profile, create a new profile with the given username/email
and existing info, and then updates the user with replenishProfile.

Simple code that I feel is cleanly separated, and will be easy to
change should the requirements get more complex.

Thank you for the feedback everyone, it was very helpful.  I'm curious
to hear any thoughts on my current solution.

Pat
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

On Wed, May 20, 2009 at 2:44 AM, Pat Maddox <[hidden email]> wrote:
[...]

> Simple code that I feel is cleanly separated, and will be easy to
> change should the requirements get more complex.
>
> Thank you for the feedback everyone, it was very helpful.  I'm curious
> to hear any thoughts on my current solution.
>
> Pat

Hi Pat,

Time will tell, and I do not say that in a mean spirit.

My point is that you've taken the first, if small, step toward your
own Greenspun's Tenth applied to DBMS.  Choosing to implement your own
DBMS, you've avoid the ORM (well, Object-SQL Mapping) overhead -- and
that is a reasonable decision in many cases.  However, now, you're
having to implement just one of the many features that a SQL DBMS
provides.  At what point will the amount and complexity of that code
exceed the ORM code?  I'm not say that it will, and I'm not saying
that your approach will have been wrong if it does.  I'm just saying
that you should keep this in mind.  It's a pay-up-front vs.
pay-as-you-go tradeoff, and not the simple "dump relational, go
objects" decision that some would have you believe.

Best of luck,

Mike
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Surely much of this code would need to be written anyway, in order to
unit test, and verify that the system is mapping properly with the
database?

On Wed, May 20, 2009 at 1:50 PM, Michael Forster <[hidden email]> wrote: _______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

On Wed, May 20, 2009 at 7:56 AM, Marcin Tustin <[hidden email]> wrote:
> Surely much of this code would need to be written anyway, in order to
> unit test, and verify that the system is mapping properly with the
> database?

I didn't say that the ORM code is small or trivial. I suggested that a
point may be reached where it is smaller and/or simpler than
implementing constraints (and many other DBMS features) by hand. For
comparison, implementing constraints procedurally via SQL triggers is
far more verbose, time-consuming, and dangerous than a one-line SQL
key or check declaration.  That most SQL DBMSs do not provide the full
range of constraint declarations prescribed by the Relational Model is
a short-coming of the former, not the latter.

In my experience, your conjecture bears out in small systems where the
number and complexity of constraints required is low.  However, I have
seen very few such systems stay that way for long.  Customers throw
all kinds of business rules at you.

Mike
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

I think we're talking slightly at cross-purposes: I'm suggesting that
code to support things like constraints would often have to be written
anyway, as part of creating tests to verify that the code works.

For very simple systems, this will be very simple, and it may be
easier to use a DBMS. For moderately complex systems, this might be
complex enough that you might not save much time (if any) with a
database. For very complex systems, it's often best to have a separate
element that can check the sanity of your data (the database).

Other features may or may not be replicated - querying is comething in
particular that can get painful in large systems.

On Wed, May 20, 2009 at 2:14 PM, Michael Forster <[hidden email]> wrote: _______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

On Wed, May 20, 2009 at 8:56 AM, Marcin Tustin <[hidden email]> wrote:
> I think we're talking slightly at cross-purposes: I'm suggesting that
> code to support things like constraints would often have to be written
> anyway, as part of creating tests to verify that the code works.

Absolutely, if you're writing constraints procedurally.  And if you're
writing procedural constaints in Smalltalk you have a significant
advantage in unit testing over SQL triggers.  However, if you're using
declarative SQL constraints (primary key, foreign key/references, row
check, table check, and etc.) then you do not need to write any tests.
 The point of declarative constraints is that the DBMS has already
been tested with regard to enforcing them.  Better yet, a correctly
implemented SQL DBMS won't allow you to declare a constraint on
existing data that would violate the former.  Try that with procedural
constraints.


> For very simple systems, this will be very simple, and it may be
> easier to use a DBMS. For moderately complex systems, this might be
> complex enough that you might not save much time (if any) with a
> database. For very complex systems, it's often best to have a separate
> element that can check the sanity of your data (the database).

Other way around.  With simple systems (and we should define
"simple"), you might be less likely to need many SQL DBMS features, so
you can save the cost of ORM.  With more complex systems, you are more
likely to need them and the cost of ORM could very well be far less
than the cost of coding some very tricky constraints, optimized
queries, etc.

Mike
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside