Smalltalk › Frameworks & Tools › Seaside › Seaside General

GOODS

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

21 messages Options

Tomas Svarovsky

GOODS

I am quite new to this whole thing (smalltalk seaside and squeak) so
please, remember you have to play a little slower with me.

I was following tutorials from Hasso-Plattner-Institut on data
persistence. I ve compiled goods version 3.06, I ve installed
smalltalk GOODS client using Squeak map (version 80) and started
playing with it.

Everything is fine until I try to commit any change into database.
KKComitFailure is thrown and GOODS server is saying 'CRC check failed
for transaction from client squeakC18'. I ve done some googling, but
found only this
http://www.nabble.com/GOODS-CRC-check-failed-for-transaction-t3549186.html
, which is unfortunately not very helpful, though seems like exactly
my case. I ve tried to search for some newer version of GOODS client,
but seems that version 80 is the last one released. Can anyone be more
specific on solving this problem than already mentioned link?

Thanks a lot in advance TS
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Sebastian Sastre-2

RE: GOODS

Hi Thomas,

do you *have* to use GOODS for persistance or you have choosed to do
it? If it's your choice I can tell you that I'll be very motivated not to
use GOODS at all. Also you can give a chance to Magma instead.

cheers,

Sebastian Sastre

> -----Mensaje original-----
> De: [hidden email]
> [mailto:[hidden email]] En nombre
> de Tomáš Svárovský
> Enviado el: Domingo, 18 de Noviembre de 2007 17:17
> Para: [hidden email]
> Asunto: [Seaside] GOODS
>
> I am quite new to this whole thing (smalltalk seaside and
> squeak) so please, remember you have to play a little slower with me.
>
> I was following tutorials from Hasso-Plattner-Institut on
> data persistence. I ve compiled goods version 3.06, I ve
> installed smalltalk GOODS client using Squeak map (version
> 80) and started playing with it.
>
> Everything is fine until I try to commit any change into database.
> KKComitFailure is thrown and GOODS server is saying 'CRC
> check failed for transaction from client squeakC18'. I ve
> done some googling, but found only this
> http://www.nabble.com/GOODS-CRC-check-failed-for-transaction-t

3549186.html

> , which is unfortunately not very helpful, though seems like
> exactly my case. I ve tried to search for some newer version
> of GOODS client, but seems that version 80 is the last one
> released. Can anyone be more specific on solving this problem
> than already mentioned link?
>
> Thanks a lot in advance TS
> _______________________________________________
> seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Tomas Svarovsky

Re: GOODS

In reply to this post by Tomas Svarovsky

Hi Sebastian, thanks for an answer

Using goods was definitely my choice as I am just playing with seaside
and not using it for anything meaningful right now. As I have said I
am quite new so I welcomed the fact that there was something like a
step by step tutorial for setting up a database (I ve already tried
magma but didnt manage to get it up and running. I will probably have
to try it again).

Can you be more specific about why not using GOODS? For my purposes I
dont care if I use object oriented or relational DB. I just would like
to learn at least one reliable way to store my data in seaside app
(and store it right inside an image in an OrderedCollection or
something like that just doesnt seem right, but correct me if I am
wrong), so if you could propose a solution that seems to be stable and
reliable and potentially usable in production, that would be very
helpful as well.

thanks again
Tomas

> Message: 3
> Date: Sun, 18 Nov 2007 22:22:56 -0200
> From: "Sebastian Sastre" <[hidden email]>
> Subject: RE: [Seaside] GOODS
> To: "'Seaside - general discussion'"
> <[hidden email]>
> Message-ID:
> <!~![hidden email]>
>
> Content-Type: text/plain; charset="iso-8859-2"
>
> Hi Thomas,
>
> do you *have* to use GOODS for persistance or you have choosed to do
> it? If it's your choice I can tell you that I'll be very motivated not to
> use GOODS at all. Also you can give a chance to Magma instead.
>
> cheers,
>
> Sebastian Sastre
>
> > -----Mensaje original-----
> > De: [hidden email]
> > [mailto:[hidden email]] En nombre
> > de Tomá¹ Svárovský
> > Enviado el: Domingo, 18 de Noviembre de 2007 17:17
> > Para: [hidden email]
> > Asunto: [Seaside] GOODS
> >
> > I am quite new to this whole thing (smalltalk seaside and
> > squeak) so please, remember you have to play a little slower with me.
> >
> > I was following tutorials from Hasso-Plattner-Institut on
> > data persistence. I ve compiled goods version 3.06, I ve
> > installed smalltalk GOODS client using Squeak map (version
> > 80) and started playing with it.
> >
> > Everything is fine until I try to commit any change into database.
> > KKComitFailure is thrown and GOODS server is saying 'CRC
> > check failed for transaction from client squeakC18'. I ve
> > done some googling, but found only this
> > http://www.nabble.com/GOODS-CRC-check-failed-for-transaction-t
> 3549186.html
> > , which is unfortunately not very helpful, though seems like
> > exactly my case. I ve tried to search for some newer version
> > of GOODS client, but seems that version 80 is the last one
> > released. Can anyone be more specific on solving this problem
> > than already mentioned link?
> >
> > Thanks a lot in advance TS
> > _______________________________________________
> > seaside mailing list
> > [hidden email]
> > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

cdavidshaffer

Re: GOODS

In reply to this post by Tomas Svarovsky

Tomáš Svárovský wrote:
> Everything is fine until I try to commit any change into database.
> KKComitFailure is thrown and GOODS server is saying 'CRC check failed
> for transaction from client squeakC18'.

I have published an updated GOODS client which performs the needed CRC
check. Please load the latest version from SqueakMap. It should be
considered alpha as I have only tested some simple operations. Please
e-mail me (or post here) if you find any bugs.

David

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

cdavidshaffer

Re: Re: GOODS

In reply to this post by Tomas Svarovsky

Tomáš Svárovský wrote:

> Hi Sebastian, thanks for an answer
>
> Using goods was definitely my choice as I am just playing with seaside
> and not using it for anything meaningful right now. As I have said I
> am quite new so I welcomed the fact that there was something like a
> step by step tutorial for setting up a database (I ve already tried
> magma but didnt manage to get it up and running. I will probably have
> to try it again).
>
> Can you be more specific about why not using GOODS? For my purposes I
> dont care if I use object oriented or relational DB. I just would like
> to learn at least one reliable way to store my data in seaside app
> (and store it right inside an image in an OrderedCollection or
> something like that just doesnt seem right, but correct me if I am
> wrong), so if you could propose a solution that seems to be stable and
> reliable and potentially usable in production, that would be very
> helpful as well.
>

Tomas,

I don't want to interrupt Sebastian as I'm sure he has some very good
reasons for not liking GOODS. But I do want to add that there are
several of us using it in production and it is a great persistence
solution in Squeak. The only problem I really have with it (which is not
a GOODS problem but a Squeak problem) is that commits can be very slow
if you have loaded a lot of objects in a transaction. That is, when you
commit the Squeak client needs to check all of the objects loaded in
your transaction from the database to see if any of them are "dirty"
(have been changed since they were loaded). This process involves a
linear search which can be very slow when you have a lot of objects
loaded. I added the method #cacheSize to KKDatabase in order for me to
be able to monitor this. This problem might be solved by WriteBarrier
but I've never been able to get that to work efficiently. In the mean
time I have several Squeak/Seaside production applications using GOODS
and, with some small tuning here and there, they all chug along nicely.

David

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Sebastian Sastre-2

RE: Re: GOODS

In reply to this post by Tomas Svarovsky

Oh now you mention it I also had a problem the first time I've tried Magma. I remember to make it work by loading it as the first project of a fresh squeak image and only then load the rest of the packages.
Also I remember the need of re reading the magma tutorial several times. It is short but full of details. If you methodically follow those there isn't too much space for errors.

GOODS is not worth IMHO. Is not designed to be any good for a smalltalk system. I think Magma is very promising for what you describe. For experimenting is also very confortable because it's transparency just rocks.

But also have in mind that if you don't need transactions you can persist in the image itself or if you are more conforable in an external file of serialized objects by using ImageSegments (you can google a little you will find it in any modern squeak image).

cheers,

Sebastian Sastre

> -----Mensaje original-----
> De: [hidden email]
> [mailto:[hidden email]] En nombre
> de Tomáš Svárovský
> Enviado el: Lunes, 19 de Noviembre de 2007 12:32
> Para: [hidden email]
> Asunto: [Seaside] Re: GOODS
>
> Hi Sebastian, thanks for an answer
>
> Using goods was definitely my choice as I am just playing
> with seaside and not using it for anything meaningful right
> now. As I have said I am quite new so I welcomed the fact
> that there was something like a step by step tutorial for
> setting up a database (I ve already tried magma but didnt
> manage to get it up and running. I will probably have to try
> it again).
>
> Can you be more specific about why not using GOODS? For my
> purposes I dont care if I use object oriented or relational
> DB. I just would like to learn at least one reliable way to
> store my data in seaside app (and store it right inside an
> image in an OrderedCollection or something like that just
> doesnt seem right, but correct me if I am wrong), so if you
> could propose a solution that seems to be stable and reliable
> and potentially usable in production, that would be very
> helpful as well.
>
> thanks again
> Tomas
>
>
>
> > Message: 3
> > Date: Sun, 18 Nov 2007 22:22:56 -0200
> > From: "Sebastian Sastre" <[hidden email]>
> > Subject: RE: [Seaside] GOODS
> > To: "'Seaside - general discussion'"
> > <[hidden email]>
> > Message-ID:
> >
> >
> <!~!UENERkVCMDkAAQACAAAAAAAAAAAAAAAAABgAAAAAAAAAx5oTyKKcHEiH5jLobrYqEM
> > [hidden email]>
> >
> > Content-Type: text/plain; charset="iso-8859-2"
> >
> > Hi Thomas,
> >
> > do you *have* to use GOODS for persistance or you
> have choosed
> > to do it? If it's your choice I can tell you that I'll be very
> > motivated not to use GOODS at all. Also you can give a
> chance to Magma instead.
> >
> > cheers,
> >
> > Sebastian Sastre
> >
> > > -----Mensaje original-----
> > > De: [hidden email]
> > > [mailto:[hidden email]] En nombre de
> > > Tomá¹ Svárovský Enviado el: Domingo, 18 de Noviembre de 2007 17:17
> > > Para: [hidden email]
> > > Asunto: [Seaside] GOODS
> > >
> > > I am quite new to this whole thing (smalltalk seaside and
> > > squeak) so please, remember you have to play a little
> slower with me.
> > >
> > > I was following tutorials from Hasso-Plattner-Institut on data
> > > persistence. I ve compiled goods version 3.06, I ve installed
> > > smalltalk GOODS client using Squeak map (version
> > > 80) and started playing with it.
> > >
> > > Everything is fine until I try to commit any change into database.
> > > KKComitFailure is thrown and GOODS server is saying 'CRC check
> > > failed for transaction from client squeakC18'. I ve done some
> > > googling, but found only this
> > > http://www.nabble.com/GOODS-CRC-check-failed-for-transaction-t
> > 3549186.html
> > > , which is unfortunately not very helpful, though seems
> like exactly
> > > my case. I ve tried to search for some newer version of GOODS
> > > client, but seems that version 80 is the last one released. Can
> > > anyone be more specific on solving this problem than already
> > > mentioned link?
> > >
> > > Thanks a lot in advance TS
> > > _______________________________________________
> > > seaside mailing list
> > > [hidden email]
> > > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
> _______________________________________________
> seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

keith1y

Re: Re: GOODS and Magma

Hi All,

I have been working on a revised Magma-Seaside interface using Session Helpers, i.e. this allows you to add Magma to any seaside application without needing a specialized subclass of WASession.I also have a tutorial portion for Magma almost ready.

I just need a bit of spare time for testing, so watch this space.

Keith

======

Draft Tutorial follows:

Saving all data in an object-oriented database: Magma

Another option to make your data persistent is to use an object-oriented database like Magma.

Installing the Magma Database

Magma is written entirely in Squeak, so there is no need to install a separate server application. Both Magma and Seaside can run in the same image. To install (into a 3.10 based image) just execute the following. (This may take a while)

Installer universes install: 'Magma seasideHelper'.

Configuring your application to use Magma

In the configuration for the 'todo' application you will need to add WAMagmaConfiguration to the configuration "ancestry". Select it from the drop down menu and click "Add". As soon as you do this, a new group of configuration options will appear. You will need to select the "WAMagma", helper class from the drop down menu.

The WAMagma class provides the basic interface between Magma Sessions and Seaside Sessions. Subclasses provide specialized session management such as shared or pooled sessions.

Using Magma

Magma is very easy to use with seaside. To obtain and use a database session helper, simply send #magma to the current session. For example the code "self session magma" will work within all seaside components.

To use Magma in this application in a similar manner to the way we have connected to databases in other examples we need to create an interfacing class, to which we can add our specialized querying methods. We will use this interfacing class as our root object in the database.

(note at present if a subclass of Dictionary is used then it is not possible to add instVars to this class)
Dictionary subclass: #StMagmaDatabase     instanceVariableNames: ''     classVariableNames: ''     poolDictionaries: ''     category: 'STTutTodoApp' StSession-#initialize     super initialize.     self db: (self magma rootAs: StMagmaDatabase)

The tutorial application accesses the database via the #db accessor. Here we define this using the #rootAs: helper method, which does everything we need, both to initialize the database, and to provide us the interface that we want to it.

Initializing the Database Root

To initialize our users collection we define the initialization on our root object, and we need a method to return the users from the database, to add a new user, and to find a user.

StMagmaDatabase-#initialize     | users |     users := OrderedCollection new.     self at: #users put: users. StMagmaDatabase-#users         ^ self at: #users StMagmaDatabase-#addUser: newUser         ^ self users add: newUser StMagmaDatabase-#findUserByEmail: anEmail     ^ self users         detect: [:each | each email equals: anEmail]         ifNone:[]

Notice how for basic usage we have not needed any database specific code!

If our userbase is going to grow to a significant size it makes sense to re-implement the above using some database features, so let us introduce a MagmaCollection, a collection designed to enable concurrent access to large indexed data sets.
(http://wiki.squeak.org/squeak/2639)

In this instance we use a factory method to create our MagmaCollection. This allows us to switch the Magma helper to seamlessly by using WAMockMagma instead of WAMagma.

initialize     | users |     users := WAMagma factory newMagmaCollection.     users addIndex: (MaSearchStringIndex attribute: #email) beAscii.     self at: #users put: users findUserByEmail: anEmail     ^ (self users where: [ :each | each email equals: anEmail ] ) firstOrNil
to be continued....

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Tomas Svarovsky

Re: GOODS

In reply to this post by cdavidshaffer

Hello David,

that was really quick (wow). I ve tried to set it up again and it
works on image (3.9) from http://damien.cassou.free.fr/squeak-web/.

I ve actually tested it on 3.10 beta from the same site as well and I
didnt manage to get it to work. It throws MessageNotUnderstood:
Undefined object >> connection. From my debugging it seems like some
instance variables of recordCache didnt get initialized, but I ve
encountered really strange behavior during weekend with this image, so
I assume it is because of its beta state.

so thx for a bugfix I am staying with 3.9 and will play with GOODS some more.

TS

On 11/19/07, C. David Shaffer <[hidden email]> wrote:

> Tomáš Svárovský wrote:
> > Everything is fine until I try to commit any change into database.
> > KKComitFailure is thrown and GOODS server is saying 'CRC check failed
> > for transaction from client squeakC18'.
>
> I have published an updated GOODS client which performs the needed CRC
> check. Please load the latest version from SqueakMap. It should be
> considered alpha as I have only tested some simple operations. Please
> e-mail me (or post here) if you find any bugs.
>
> David
>
> _______________________________________________
> seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Tomas Svarovsky

Re: Re: GOODS

In reply to this post by Sebastian Sastre-2

Transparency would definitely be something I would welcome (at least
for learning), so I will definitely try installing magma again soon.

On the other hand from what David said GOODS would work for me very
well too, so as ussually I will have to try both and decide for myself
:-)

About storing stuff right inside image. I am feeling strange about
this. I ve come from PHP/Ruby world so I am yet a little freaked out
by this forever living image (though I have to say it is very
appealing). Is there any penalty for storing stuff right inside an
image? Should I keep it as small as I can or it is not tha critical?

TS

On 11/19/07, Sebastian Sastre <[hidden email]> wrote:

> Oh now you mention it I also had a problem the first time I've tried Magma. I remember to make it work by loading it as the first project of a fresh squeak image and only then load the rest of the packages.
> Also I remember the need of re reading the magma tutorial several times. It is short but full of details. If you methodically follow those there isn't too much space for errors.
>
> GOODS is not worth IMHO. Is not designed to be any good for a smalltalk system. I think Magma is very promising for what you describe. For experimenting is also very confortable because it's transparency just rocks.
>
> But also have in mind that if you don't need transactions you can persist in the image itself or if you are more conforable in an external file of serialized objects by using ImageSegments (you can google a little you will find it in any modern squeak image).
>
> cheers,
>
> Sebastian Sastre
>
> > -----Mensaje original-----
> > De: [hidden email]
> > [mailto:[hidden email]] En nombre
> > de Tomáš Svárovský
> > Enviado el: Lunes, 19 de Noviembre de 2007 12:32
> > Para: [hidden email]
> > Asunto: [Seaside] Re: GOODS
> >
> > Hi Sebastian, thanks for an answer
> >
> > Using goods was definitely my choice as I am just playing
> > with seaside and not using it for anything meaningful right
> > now. As I have said I am quite new so I welcomed the fact
> > that there was something like a step by step tutorial for
> > setting up a database (I ve already tried magma but didnt
> > manage to get it up and running. I will probably have to try
> > it again).
> >
> > Can you be more specific about why not using GOODS? For my
> > purposes I dont care if I use object oriented or relational
> > DB. I just would like to learn at least one reliable way to
> > store my data in seaside app (and store it right inside an
> > image in an OrderedCollection or something like that just
> > doesnt seem right, but correct me if I am wrong), so if you
> > could propose a solution that seems to be stable and reliable
> > and potentially usable in production, that would be very
> > helpful as well.
> >
> > thanks again
> > Tomas
> >
> >
> >
> > > Message: 3
> > > Date: Sun, 18 Nov 2007 22:22:56 -0200
> > > From: "Sebastian Sastre" <[hidden email]>
> > > Subject: RE: [Seaside] GOODS
> > > To: "'Seaside - general discussion'"
> > > <[hidden email]>
> > > Message-ID:
> > >
> > >
> > <!~!UENERkVCMDkAAQACAAAAAAAAAAAAAAAAABgAAAAAAAAAx5oTyKKcHEiH5jLobrYqEM
> > > [hidden email]>
> > >
> > > Content-Type: text/plain; charset="iso-8859-2"
> > >
> > > Hi Thomas,
> > >
> > > do you *have* to use GOODS for persistance or you
> > have choosed
> > > to do it? If it's your choice I can tell you that I'll be very
> > > motivated not to use GOODS at all. Also you can give a
> > chance to Magma instead.
> > >
> > > cheers,
> > >
> > > Sebastian Sastre
> > >
> > > > -----Mensaje original-----
> > > > De: [hidden email]
> > > > [mailto:[hidden email]] En nombre de
> > > > Tomá¹ Svárovský Enviado el: Domingo, 18 de Noviembre de 2007 17:17
> > > > Para: [hidden email]
> > > > Asunto: [Seaside] GOODS
> > > >
> > > > I am quite new to this whole thing (smalltalk seaside and
> > > > squeak) so please, remember you have to play a little
> > slower with me.
> > > >
> > > > I was following tutorials from Hasso-Plattner-Institut on data
> > > > persistence. I ve compiled goods version 3.06, I ve installed
> > > > smalltalk GOODS client using Squeak map (version
> > > > 80) and started playing with it.
> > > >
> > > > Everything is fine until I try to commit any change into database.
> > > > KKComitFailure is thrown and GOODS server is saying 'CRC check
> > > > failed for transaction from client squeakC18'. I ve done some
> > > > googling, but found only this
> > > > http://www.nabble.com/GOODS-CRC-check-failed-for-transaction-t
> > > 3549186.html
> > > > , which is unfortunately not very helpful, though seems
> > like exactly
> > > > my case. I ve tried to search for some newer version of GOODS
> > > > client, but seems that version 80 is the last one released. Can
> > > > anyone be more specific on solving this problem than already
> > > > mentioned link?
> > > >
> > > > Thanks a lot in advance TS
> > > > _______________________________________________
> > > > seaside mailing list
> > > > [hidden email]
> > > > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
> > _______________________________________________
> > seaside mailing list
> > [hidden email]
> > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>
> _______________________________________________
> seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Avi Bryant-2

Re: Re: GOODS

On 11/19/07, Tomas Svarovsky <[hidden email]> wrote:
> Transparency would definitely be something I would welcome (at least
> for learning), so I will definitely try installing magma again soon.

In what way is Magma more transparent than GOODS? AFAIK they both use
very similar strategies to be transparent (proxies for lazy loading,
persistence by reachability, no need to explicitly mark objects dirty,
etc).

Avi
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Avi Bryant-2

Re: Re: GOODS

In reply to this post by cdavidshaffer

On 11/19/07, C. David Shaffer <[hidden email]> wrote:

> I don't want to interrupt Sebastian as I'm sure he has some very good
> reasons for not liking GOODS. But I do want to add that there are
> several of us using it in production and it is a great persistence
> solution in Squeak. The only problem I really have with it (which is not
> a GOODS problem but a Squeak problem) is that commits can be very slow
> if you have loaded a lot of objects in a transaction.

So you're finding that writes are the main bottleneck, and that read
performance is fine? I'd be curious to hear more about this; in my
own limited experiments the read performance was what concerned me
most.

Avi
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

cdavidshaffer

Re: Re: GOODS

Avi Bryant wrote:
> So you're finding that writes are the main bottleneck, and that read
> performance is fine? I'd be curious to hear more about this; in my
> own limited experiments the read performance was what concerned me
> most.
>
I have tuned the read code a bit (at the expense of support for
arbitrary GOODS class name -> Smalltalk class name mappings) and that
has improved the only problems I've had with reads. I think we're
getting close to being limited by network issues. I believe that GOODS
supports talking on a UNIX domain socket so it might be interesting to
see if that would improve things much. It hasn't been an issue for me
though. For applications where you rapidly populate large models you
might run into trouble...I don't have any examples of that. I worked on
an app last year, which never saw the light of day unfortunately, where
we had a fair number of large binary objects (png's and morphs) stored
in GOODS and reads were a bit sluggish. "a bit" here meant not bad
enough compared to non-GOODS related problems to be worth the time to
investigate. I've found tweaking the server.cluster_size parameter can
also help to alleviate read-related performance problems.

Performance problems with GOODS, in my experience, are typically:

1) connection time so I keep a pool of fresh connections around
2) writes when the cache is large or when you have large objects to
compare (arrays, for example) -- this requires some clever flushing
strategies to fix when it occurs

WriteBarrier seemed promising to fix #2 but it causes all kinds of other
problems for me. I had hoped to have funding to pay someone to
implement VM-level immutability support in Squeak...that would solve the
problem without bytecode tricks. But that funding fell through with the
aforementioned project. I think such a thing would substantially
benefit GOODS and Magma.

David

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

cdavidshaffer

Re: Re: GOODS

Oh yea, one more problem I recall (which maybe this is what you were
thinking of) is that the "changed object notification" can be
troublesome. Each transaction gets a list of objects that other
transactions have changed. Re-reading those can take a while.
Especially since each active session will (eventually) probably request
them. This might be worth fixing if it is causing anyone problems...

David

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Avi Bryant-2

Re: Re: GOODS

In reply to this post by cdavidshaffer

On 11/19/07, C. David Shaffer <[hidden email]> wrote:

> Performance problems with GOODS, in my experience, are typically:
>
> 1) connection time so I keep a pool of fresh connections around
> 2) writes when the cache is large or when you have large objects to
> compare (arrays, for example) -- this requires some clever flushing
> strategies to fix when it occurs

1) Do you get a new connection from the pool for each seaside request?
If so, how do you deal with references to objects that stick around
between requests? If not, how does the pooling work?

2) This idea has come up before but I'm curious what your take on it
is these days: would it help to have a single shared connection that
was used most of the time, and had a very large cache but never any
commits, and then switch to a pooled, small-cache transaction whenever
you need to do writes? I'm picturing an interface something like
this:

renderItemsOn: html
connection root allItems do:
[:ea |
...
html anchor callback: [self delete: ea]; text: 'Delete']

delete: anItem
connection writeConnectionFor: anItem do:
[:write :item |
write root allItems remove: item ifAbsent: [].
write commit]

Avi
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

cdavidshaffer

Re: Re: GOODS

Avi Bryant wrote:
>
> 1) Do you get a new connection from the pool for each seaside request?
> If so, how do you deal with references to objects that stick around
> between requests? If not, how does the pooling work?
>
No, this pool is just a small set of "fresh" connections that I keep.
When a new seaside session is created it gets a session from this pool
and, I kick a background process that creates another connection to fill
the empty slot in the pool. It is important that these connections have
_no_ objects loaded on them otherwise they will cause GOODS to queue
object change notifications.

> 2) This idea has come up before but I'm curious what your take on it
> is these days: would it help to have a single shared connection that
> was used most of the time, and had a very large cache but never any
> commits, and then switch to a pooled, small-cache transaction whenever
> you need to do writes? I'm picturing an interface something like
> this:
>
> renderItemsOn: html
> connection root allItems do:
> [:ea |
> ...
> html anchor callback: [self delete: ea]; text: 'Delete']
>
> delete: anItem
> connection writeConnectionFor: anItem do:
> [:write :item |
> write root allItems remove: item ifAbsent: [].
> write commit]
>
>

Yes, I did something very similar. Since I had hidden all mutation
inside command objects each command went through a transition from one
transaction to another before it executed. Each command was responsible
for supplying the list of GOODS objects that needed to be moved. Worked
very well. Oddly enough I didn't use a single read-only
transaction...each session still had its own read transaction. Maybe
the next step would have been to get rid of that obvious redundancy but
that would have required making the transactions thread-safe. As I
recall something had to be done to make sure that the read-only session
saw changes at reasonable points (I think my commands sent #refresh to
the read-only session after the command was complete).

David

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Sebastian Sastre-2

RE: Re: GOODS

In reply to this post by Tomas Svarovsky

> About storing stuff right inside image. I am feeling strange
> about this. I ve come from PHP/Ruby world so I am yet a
> little freaked out by this forever living image (though I
> have to say it is very appealing). Is there any penalty for
> storing stuff right inside an image? Should I keep it as
> small as I can or it is not tha critical?
>
> TS
>
Tomas just remember to remain kind of skeptical about images they can become
corrupt and you can have other potential problems. So if your scope is a
cheap non intrusive prototypical phase of the development it will certainly
can help you.
ImagesSegments are kind of better choice but more or less the same is valid.
They didn't failed me (yet :)
Also have in mind that if you can make later the persistence compromise your
development speed may suffer less.

Cheers,

Sebastian
PS: I'm not confortable persisting in images and in fact I don't use them
for that but it's true that they are an option.

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Sebastian Sastre-2

RE: Re: GOODS

In reply to this post by cdavidshaffer

> I don't want to interrupt Sebastian as I'm sure he has some
> very good reasons for not liking GOODS. But I do want to add
> that there are several of us using it in production and it is
> a great persistence solution in Squeak. The only problem I
> really have with it (which is not a GOODS problem but a
> Squeak problem) is that commits can be very slow if you have
> loaded a lot of objects in a transaction. That is, when you
> commit the Squeak client needs to check all of the objects
> loaded in your transaction from the database to see if any of
> them are "dirty"
> (have been changed since they were loaded). This process
> involves a linear search which can be very slow when you have
> a lot of objects loaded. I added the method #cacheSize to
> KKDatabase in order for me to be able to monitor this. This
> problem might be solved by WriteBarrier but I've never been
> able to get that to work efficiently. In the mean time I have
> several Squeak/Seaside production applications using GOODS
> and, with some small tuning here and there, they all chug
> along nicely.
>
> David
>

Oh Davis please be my guest to add whatever you experienced in this
discussion. I recognise I've tried GOODS (I even made a port of the squeak
client to dolphin smalltalk some time ago) and failed to use it reasonably
for a system of about 40k objects. Even with Btrees and such I was unable to
achieve a half of what I got with any other persistance mechanism. I didn't
work for me.
I see GOODS as a layer of ACID writting of scrutcures in disk not as a more
complete persistence solutions. That gap it's covered by the Avi's Squeak
client of course but with write barrier and all it is still a no go for me.
On the other hand I'm having a good experience with Magma. I moved from
ImageSegments to a magma repository in one day.

Cheers,

Sebastian

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Sebastian Sastre-2

RE: Re: GOODS

In reply to this post by Avi Bryant-2

>
> In what way is Magma more transparent than GOODS? AFAIK they
> both use very similar strategies to be transparent (proxies
> for lazy loading, persistence by reachability, no need to
> explicitly mark objects dirty, etc).
>
> Avi
> _______________________________________________

Hey Avi! How are you? I think it's not necessarily *more* transparent. It
happens to have a lot of transparence. Perhaps the magmasession api has more
features than the goods client at this time? My point is that having Magma I
see no value to choose GOODS in Squeak todays. In fact I believe that
maturing Magma will add value to the whole community at this point. Anyway
the fact is that I've moved from ImageSegments to Magma with minimal code
intrusion.
But I think that every case should be specifically tested proven to be
worth. I recognise that ImageSegments still providing enormous flexibility
and extreme simplicity.
Cheers,
Sebastian

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Avi Bryant-2

Re: Re: GOODS

In reply to this post by Sebastian Sastre-2

On 11/19/07, Sebastian Sastre <[hidden email]> wrote:

> Tomas just remember to remain kind of skeptical about images they can become
> corrupt and you can have other potential problems.

Have you ever seen a corrupt image? Dabble DB has probably created...
oh, I don't know, certainly over a million images, and the single time
I have ever seen one get corrupted was because of a hardware failure.

What I *have* seen are images where the data is difficult to access
because the UI process is dead, say, or there's an infinite recursion
that causes them to run out of memory very shortly after startup. The
data is still there, but I have to admit that I prefer writing stuff
out to ImageSegments every so often to avoid this potential hassle
(among other reasons).

Avi
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

johnmci

Re: Re: GOODS

On Nov 19, 2007, at 7:48 PM, Avi Bryant wrote:

> Have you ever seen a corrupt image? Dabble DB has probably created...
> oh, I don't know, certainly over a million images, and the single time
> I have ever seen one get corrupted was because of a hardware failure.

In 10 years or so of writing squeak plugins, and doing VM changes I'll
note the Squeak GC logic is
*extremely* efficient in finding corrupted memory. Even finding things
like when you forget to provide
magic padding for a stack frame of a certain size on an FFI call and
discovering that Apple's Quicktime programmers
exploited the fact there was four bytes of free space on the stack
frame they could abuse.

--
=
=
=
========================================================================
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com
=
=
=
========================================================================

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside