mapping seaside sessions onto gem sessions?

> Hi all,
>
> We are currently experimenting to port a multi-user Seaside application to GemStone 2.4.4.1. We installed the Web Edition on our Mac OS X boxes and got the installation running quite fine.
>
> However, we cannot see how we can make a seaside session map onto a gem session. Is that possible in any way? We need this because each seaside session needs to have its own view on the object database. Each seaside session manipulates the same data in different ways and therefore we cannot share the same data in each seaside session.
>
> Given the fact that Seaside runs as an application inside a gem session, it seems impossible to do this. Or are we overlooking something?
>
> Hope someone can shed some light on this for us ;-)
>
> best regards
> Johan Brichau

> Johan,
>
> I think the answer is that you really don't need to do anything special, except load up and run Seaside.
>
> The seaside that runs on GemStone is _designed_ to process only one request per gem. An abort is automatically performed at the start of process handling and a commit is performed when the response is ready ... in effect the session owns the gem and all of the image state while a request is being processed.
>
> You can run multiple gems and arrange to round robin the requests from lightppd or apache to the gems.
>
> You will want to have n gems running where n is the number of concurrent requests that you want to process. If more requests come in that you have gems, the requests will queue up inside one of the gems and wait for the process to finish...
>
> Dale
>
> Johan Brichau wrote:
>> Hi all,
>> We are currently experimenting to port a multi-user Seaside application to GemStone 2.4.4.1. We installed the Web Edition on our Mac OS X boxes and got the installation running quite fine.
>> However, we cannot see how we can make a seaside session map onto a gem session. Is that possible in any way? We need this because each seaside session needs to have its own view on the object database. Each seaside session manipulates the same data in different ways and therefore we cannot share the same data in each seaside session.
>> Given the fact that Seaside runs as an application inside a gem session, it seems impossible to do this. Or are we overlooking something?
>> Hope someone can shed some light on this for us ;-)
>> best regards
>> Johan Brichau
>

> Hi Dale,
>
> Thanks for your response. I think I understand how a seaside request
> is mapped onto a gemstone transaction automatically. However, we do
> not want to a db view to be associated to a request. Instead, we want
> a db view to be associated to a seaside session. I can see how to
> deactivate the begin- and commitTransaction operations at the begin
> and end of a request, but that still means that requests for
> different seaside sessions share the same db view on the objects.
>
> In our implementation (currently running in Pharo), we open a db
> connection per seaside session and keep it in the WASession instance.
> That means that each session has its separate view on the db: i.e.
> each user has its own view on the shared data. We perform db
> transactions manually because we need to explicitly handle conflict
> situations.
>
> From what I can read in the Gemstone documentation, it seems
> impossible to do that?
>
> Johan
>
> On 23 Jul 2010, at 02:24, Dale Henrichs wrote:
>
>> Johan,
>>
>> I think the answer is that you really don't need to do anything
>> special, except load up and run Seaside.
>>
>> The seaside that runs on GemStone is _designed_ to process only one
>> request per gem. An abort is automatically performed at the start
>> of process handling and a commit is performed when the response is
>> ready ... in effect the session owns the gem and all of the image
>> state while a request is being processed.
>>
>> You can run multiple gems and arrange to round robin the requests
>> from lightppd or apache to the gems.
>>
>> You will want to have n gems running where n is the number of
>> concurrent requests that you want to process. If more requests come
>> in that you have gems, the requests will queue up inside one of the
>> gems and wait for the process to finish...
>>
>> Dale
>>
>> Johan Brichau wrote:
>>> Hi all, We are currently experimenting to port a multi-user
>>> Seaside application to GemStone 2.4.4.1. We installed the Web
>>> Edition on our Mac OS X boxes and got the installation running
>>> quite fine. However, we cannot see how we can make a seaside
>>> session map onto a gem session. Is that possible in any way? We
>>> need this because each seaside session needs to have its own view
>>> on the object database. Each seaside session manipulates the same
>>> data in different ways and therefore we cannot share the same
>>> data in each seaside session. Given the fact that Seaside runs as
>>> an application inside a gem session, it seems impossible to do
>>> this. Or are we overlooking something? Hope someone can shed some
>>> light on this for us ;-) best regards Johan Brichau
>

> Johan,
>
> I'm not quite sure that I understand what you are trying to do and I'm not quite sure where to begin.
> Here's a picture of GemStone:
>
>  - a gem can only hold one "view" of the shared data in memory at
>    a time
>  - at transaction boundaries the "view" is updated
>    - an abort ignores all changes to persistent objects and updates
>      the gems "view" to the latest.
>    - a commit updates the latest "view" with your modifications to
>      persistent objects. A transaction conflict occurs if another
>      gemstone session made changes to the same object(s) that you
>      changed.
>
> So the scope of a view is the entire gem/vm. It is not possible for a single gem to have multiple views into the db.
>
> With Seaside and GemStone, it _is_ possible to store session-specific data in a WASession instance and the WASession instances are persisted, so each request may have access to the state associated with the session and that state may indeed be different for each session.
>
> With Seaside and GemStone, only one request is processed at a time in a gem so assuming that you have persisted the session-specific information with the session instance, the entire vm will hold the "view" that is associated with that session while processing the request for that session. Think of every GemStone vm as a separate Process instance. So concurrent seaside requests can be processed by having several gems running, each holding the view for the session that is being processed by that gem. Note that the Seaside framework for Gemstone ensures that no two vms are handling requests for the _same_ session at the _same_ time.
>
> So at this point (making some assumptions about your application):
>
>  If your application is written such that each session makes _direct_
>  updates to the shared data over the course of several http requests,
>  then it is true that you'll either need change the way your
>  application works or change the way Seaside/GemStone works.
>
> Is this a correct assumption? If not then I need more information, if so, then I have a couple more questions to ask...before declaring that its "impossible to do that":)
>
> Dale
>
> Johan Brichau wrote:
>> Hi Dale,
>> Thanks for your response. I think I understand how a seaside request
>> is mapped onto a gemstone transaction automatically. However, we do
>> not want to a db view to be associated to a request. Instead, we want
>> a db view to be associated to a seaside session. I can see how to
>> deactivate the begin- and commitTransaction operations at the begin
>> and end of a request, but that still means that requests for
>> different seaside sessions share the same db view on the objects.
>> In our implementation (currently running in Pharo), we open a db
>> connection per seaside session and keep it in the WASession instance.
>> That means that each session has its separate view on the db: i.e.
>> each user has its own view on the shared data. We perform db
>> transactions manually because we need to explicitly handle conflict
>> situations.
>> From what I can read in the Gemstone documentation, it seems
>> impossible to do that?
>> Johan
>> On 23 Jul 2010, at 02:24, Dale Henrichs wrote:
>>> Johan,
>>> I think the answer is that you really don't need to do anything
>>> special, except load up and run Seaside.
>>> The seaside that runs on GemStone is _designed_ to process only one
>>> request per gem. An abort is automatically performed at the start
>>> of process handling and a commit is performed when the response is
>>> ready ... in effect the session owns the gem and all of the image
>>> state while a request is being processed.
>>> You can run multiple gems and arrange to round robin the requests
>>> from lightppd or apache to the gems.
>>> You will want to have n gems running where n is the number of
>>> concurrent requests that you want to process. If more requests come
>>> in that you have gems, the requests will queue up inside one of the
>>> gems and wait for the process to finish...
>>> Dale
>>> Johan Brichau wrote:
>>>> Hi all, We are currently experimenting to port a multi-user
>>>> Seaside application to GemStone 2.4.4.1. We installed the Web
>>>> Edition on our Mac OS X boxes and got the installation running
>>>> quite fine. However, we cannot see how we can make a seaside
>>>> session map onto a gem session. Is that possible in any way? We
>>>> need this because each seaside session needs to have its own view
>>>> on the object database. Each seaside session manipulates the same
>>>> data in different ways and therefore we cannot share the same
>>>> data in each seaside session. Given the fact that Seaside runs as
>>>> an application inside a gem session, it seems impossible to do
>>>> this. Or are we overlooking something? Hope someone can shed some
>>>> light on this for us ;-) best regards Johan Brichau
>

>
> I think your assumption is correct. Each of our seaside sessions
> (WASession instances) contains a reference to the shared data and
> makes updates to that shared data during the handling of a request.
>
> If I understand Gemstone correctly, the following happens: Because
> each WASession instance is held in the same gem, the shared data that
> is held in each such instance will be updated when a commit/abort
> transaction happens in that gem, regardless of the seaside session
> that commit/abort happens in. In essence, the shared data held in all
> WASession instances is updated to the latest view (given that only a
> single gem is running).

>
> The way our application currently works in Pharo, is that each
> Seaside session maintains a reference to a separate GOODS db session.
> Each such Seaside session thus has a separate view on the shared
> data. Because each seaside session (normally) corresponds to a
> different user using the system, each user of the application has its
> own view on the data. That view of the data ultimately corresponds to
> what that user sees on his/her screen. When users/sessions make
> concurrent changes to the data, they are doing that based on their
> proper view on the data. If they make conflicting changes, we need to
> handle those explicitly. Moreover, whenever one user/session makes
> changes to the shared data, we do not want the shared data of the
> other users/sessions to be updated because they still have that view
> on the data in their browser windows. Their view should only be
> updated when their session handles a new request so that we can
> detect conflicting changes with respect to the state they departed
> from (which they see in their browser window).
>
> I hope I was able to describe our application more clearly now. What
> we are thinking right now, is that we should rather use Gemstone in a
> client-server setup, essentially replacing the GOODS db with
> Gemstone. Or do you think there is still a way to make the
> application run completely in Gemstone? I do not see how we could
> change the way our application works if we want to use the db
> transaction mechanism to detect/resolve conflicting changes by
> different users to the shared data :-(

> I am getting a better picture of how your application is structured and at
> this point I'm not quite ready to give up:)
>
> The issue revolves around conflict detection and preserving information
> about the original shared data view across transaction boundaries.
>
> GemStone's conflict detection is based upon detecting 'physical' conflicts
> when two sessions update the same object. IN GLASS, the window for physical
> conflict detection is very small (the time it takes to process a request),
> so that makes relying on 'physical' conflict detection pretty useless.
>
> One _could_ run a single session per gem to get long running transactions
> for detecting 'physical' conflicts, but then you are opening a different can
> of worms.
>
> To do 'conflict detection' in GLASS, you'd need to create a global counter
> or a shared data timestamp that was updated whenever the shared data was
> updated. When the browser screen is populated, you'd record the value of the
> counter or timestamp in your session state. When the update request was
> processed, you'd compare the current value of the counter or timestamp with
> your saved value (before doing any updates) and if the value hadn't changed,
> then you could freely update the shared state. If the value had changed,
> then you'd need to go into 'conflict resolution mode'...
>
> Depending upon your actual application this kind of technique may or may not
> work...
>

>> I am getting a better picture of how your application is structured and at
>> this point I'm not quite ready to give up:)
>>
>> The issue revolves around conflict detection and preserving information
>> about the original shared data view across transaction boundaries.
>>
>> GemStone's conflict detection is based upon detecting 'physical' conflicts
>> when two sessions update the same object. IN GLASS, the window for physical
>> conflict detection is very small (the time it takes to process a request),
>> so that makes relying on 'physical' conflict detection pretty useless.
>>
>> One _could_ run a single session per gem to get long running transactions
>> for detecting 'physical' conflicts, but then you are opening a different can
>> of worms.
>>
>> To do 'conflict detection' in GLASS, you'd need to create a global counter
>> or a shared data timestamp that was updated whenever the shared data was
>> updated. When the browser screen is populated, you'd record the value of the
>> counter or timestamp in your session state. When the update request was
>> processed, you'd compare the current value of the counter or timestamp with
>> your saved value (before doing any updates) and if the value hadn't changed,
>> then you could freely update the shared state. If the value had changed,
>> then you'd need to go into 'conflict resolution mode'...
>>
>> Depending upon your actual application this kind of technique may or may not
>> work...
>>
>
> And idea to add on to this:
>
> Depending on the number of objects/complexity of the changes/conflicts
> you need to detect.
> You could keep a set of changes to object and information about what
> the state of the change was
> when you started. Then compare for conflicts. This is basically the
> same as what Dale is saying.
>
> We used this in an application where we didn't do conflict resolution
> but did need to track changes
> to objects over time. Basically instead of storing the actual object,
> you stored a series of changes.
> Each change had the time of the change and the updated object state.
> If you have the timestamp
> of when the session started, you could find any changes made since
> that point and then manually
> resolve them with the resolution itself being another change instance.
>

> On 26 Jul 2010, at 21:46, Dale Henrichs wrote:
>
>> To do 'conflict detection' in GLASS, you'd need to create a global
>> counter or a shared data timestamp that was updated whenever the
>> shared data was updated. When the browser screen is populated,
>> you'd record the value of the counter or timestamp in your session
>> state. When the update request was processed, you'd compare the
>> current value of the counter or timestamp with your saved value
>> (before doing any updates) and if the value hadn't changed, then
>> you could freely update the shared state. If the value had changed,
>> then you'd need to go into 'conflict resolution mode'...
>>
>> Depending upon your actual application this kind of technique may
>> or may not work...
>
> Unfortunately, the fact that we rely on the database conflict
> detection to detect 'physical' conflicts is because the user can
> change almost anything in a large dataset from almost everywhere in
> the application. Keeping a timestamp for any data that can be changed
> at a certain point in the application (and using it to check
> conflicts) involves quite some work. This is because the dataset is a
> graph of interconnected objects, where operations on each object
> might trigger distant changes to other objects. Those 'distant
> changes' may very well conflict with changes made by another user. To
> detect physical conflicts in such a data model, we need to modify all
> state changing methods such that they do a timestamp check. In
> addition, we also need to keep and update the timestamps for a large
> set of objects at every request, which probably has an impact as
> well.
>
> In our current setup, the database connection essentially holds on to
> that timestamp concept on its own and signals us a conflict when
> committing. That approach has allowed us to abstract away from a fair
> amount of complexity but makes a port to GLASS indeed less trivial.
>
> However, we are not ready to give up either ;-)
>
> The thing is that we already use a manual write barrier for the GOODS
> db (for performance reasons). This write barrier keeps all objects
> that are changed in the transaction. This would allow us to keep a
> changelog and effectively detect if objects were changed by other
> seaside sessions during the processing of a request.  This is
> probably very similar to what Sean proposed.
>
> Thanks guys for the discussion! I hope we will be able to make that
> work!
>
> Johan
>

> Johan Brichau wrote:
>>
>> On 26 Jul 2010, at 21:46, Dale Henrichs wrote:
>>
>>> To do 'conflict detection' in GLASS, you'd need to create a global
>>> counter or a shared data timestamp that was updated whenever the
>>> shared data was updated. When the browser screen is populated,
>>> you'd record the value of the counter or timestamp in your session
>>> state. When the update request was processed, you'd compare the
>>> current value of the counter or timestamp with your saved value
>>> (before doing any updates) and if the value hadn't changed, then
>>> you could freely update the shared state. If the value had changed,
>>> then you'd need to go into 'conflict resolution mode'...
>>>
>>> Depending upon your actual application this kind of technique may
>>> or may not work...
>>
>> Unfortunately, the fact that we rely on the database conflict
>> detection to detect 'physical' conflicts is because the user can
>> change almost anything in a large dataset from almost everywhere in
>> the application. Keeping a timestamp for any data that can be changed
>> at a certain point in the application (and using it to check
>> conflicts) involves quite some work. This is because the dataset is a
>> graph of interconnected objects, where operations on each object
>> might trigger distant changes to other objects. Those 'distant
>> changes' may very well conflict with changes made by another user. To
>> detect physical conflicts in such a data model, we need to modify all
>> state changing methods such that they do a timestamp check. In
>> addition, we also need to keep and update the timestamps for a large
>> set of objects at every request, which probably has an impact as
>> well.
>>
>> In our current setup, the database connection essentially holds on to
>> that timestamp concept on its own and signals us a conflict when
>> committing. That approach has allowed us to abstract away from a fair
>> amount of complexity but makes a port to GLASS indeed less trivial.
>>
>> However, we are not ready to give up either ;-)
>>
>> The thing is that we already use a manual write barrier for the GOODS
>> db (for performance reasons). This write barrier keeps all objects
>> that are changed in the transaction. This would allow us to keep a
>> changelog and effectively detect if objects were changed by other
>> seaside sessions during the processing of a request.  This is
>> probably very similar to what Sean proposed.
>>
>> Thanks guys for the discussion! I hope we will be able to make that
>> work!
>>
>> Johan
>>
>
> Johan,
>
> I'm glad that you are finding a way to move your application into GLASS.  In
> general, I like the idea of the application-level changeLog because it makes
> the process of managing conflict resolution much easier since you have the
> potential to view all of the changes since you obtained your view, not just
> the changes that resulted in physical conflicts, there are other advantages
> in the areas of testing and "auditing" where the application-specific
> changelog is very useful...
>
> Dale
>