Class var and class inst var caches in libraries

With a full gs server (rather than gem-limited glass) it is uncommon that someone attempts forks within a gem rather than doing work in a separate gem. I had developed a framework for easily forking work in separate synchronous or asynchronous gems. That approach was safer and faster (due to custom gem configurations). You might consider licensing more glass gems rather than forging a less traveled path risking transaction view surprises.

What of your code is unavoidably sensitive to a momentary size inconsistency with session temps? Perhaps more than one might start a long lazy-initialization process? That could be handled with another session temp to hold a semaphore or recording an initialization state flag, but that also needs pre-initialization.

Pre-initialization of variables avoids the structure changes risks discussed by lazy-initialization. Similarly, it is a good idea to avoid lazy-initialization in persistent shared variables due to commit conflicts between competing gems. Forks within a gem are not common though, so don't be too surprised if you start encountering problems that few had seen even if using semaphores properly.

Paul Baumann

Paul makes a good point about using separate gems instead of forking green threads ... with green threads you have to be very careful that an abort or commit in one thread does not affect ongoing work in another thread or persist data prematurely.

Take a look at GsExternalSession for code that supports executing code in a separate session ... probably not as advanced as Paul's fraemwork, but it is a good starting point ...

See the ServiceVM project[1] for an example of a framework that dedicates a single vm for performing "background tasks".

Dale

With a full gs server (rather than gem-limited glass) it is uncommon that someone attempts forks within a gem rather than doing work in a separate gem. I had developed a framework for easily forking work in separate synchronous or asynchronous gems. That approach was safer and faster (due to custom gem configurations). You might consider licensing more glass gems rather than forging a less traveled path risking transaction view surprises.

What of your code is unavoidably sensitive to a momentary size inconsistency with session temps? Perhaps more than one might start a long lazy-initialization process? That could be handled with another session temp to hold a semaphore or recording an initialization state flag, but that also needs pre-initialization.

Pre-initialization of variables avoids the structure changes risks discussed by lazy-initialization. Similarly, it is a good idea to avoid lazy-initialization in persistent shared variables due to commit conflicts between competing gems. Forks within a gem are not common though, so don't be too surprised if you start encountering problems that few had seen even if using semaphores properly.

Paul Baumann

On Oct 7, 2016 3:13 PM, "monty via Glass" <[hidden email]> wrote:

My mail client wrongly sent this response to Dale, not to the list, so I'm forwarding it

> Sent: Friday, October 07, 2016 at 3:05 PM
> From: monty <[hidden email]>
> To: [hidden email]
> Subject: Re: [Glass] Class var and class inst var caches in libraries
>
> Thanks, I think this will do, but I have one more question. If a class method lazily initializes a SessionTemp to the same value (using at:ifAbsentPut: for example), but that method is executed at roughly the same time in different processes (due to fork), is there a possibility that "SessionTemps current" itself could get corrupted? For example, that it could report an inaccurate #size, or that its internal state in some other way could get corrupted due to the concurrent update?
>
> > Sent: Tuesday, October 04, 2016 at 3:21 PM
> > From: "Dale Henrichs via Glass" <[hidden email]>
> > To: [hidden email]
> > Subject: Re: [Glass] Class var and class inst var caches in libraries
> >
> > Yes, lazy initialization for shared variables in a multi-gem system is
> > not a good idea.
> >
> > Since it sounds like you do not need the values persisted and shared
> > between gems, you should be using the class SessionTemps. Here's an
> > example from Collection class>>randomForPicking:
> >
> > randomForPicking
> >    | random |
> >    random := SessionTemps current
> >      at: #'COLLECTION_RANDOM_FOR_PICKING'
> >      otherwise: nil.
> >    random == nil
> >      ifTrue: [ random := Random new.
> >        SessionTemps current at: #'COLLECTION_RANDOM_FOR_PICKING' put:
> > random ].
> >    ^ random
> >
> > SessionTemps is a subclass of SymbolDictionary and can be used for
> > storing session-specific globals.
> >
> > If you have the need to sharing persistent values then an explicit
> > initialization in the class initialization method is the right answer.
> > The initialize method gets run during install and at least the
> > initialization is safe from commit conflicts.
> >
> > Dale
> >
> >
> > On 10/04/2016 11:56 AM, monty via Glass wrote:
> > > Lots of libraries use caches stored in class vars and class isnt vars that are either explicitly or lazily initialized. But with multiple Gems accessing the same repo, this can produce write-write conflicts. In fact, simply sending #initialize to the same class from different Gems can cause them. What are the best ways to implement these caches in GS? Write-locks seem like overkill, so I'm interested in simple ways of making them non-persistent.
> > > _______________________________________________
> > > Glass mailing list
> > > [hidden email]
> > > http://lists.gemtalksystems.com/mailman/listinfo/glass
> >
> > _______________________________________________
> > Glass mailing list
> > [hidden email]
> > http://lists.gemtalksystems.com/mailman/listinfo/glass
> >
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Use application design that provides division of responsibility. Avoid having more than one gem updating the same value holder. Gem-specific value holders can be used, referenced in gem-specific dictionary (or object).  Any other gems can query committed assignments of others to find consensus. Somewhat like an rc collection implementation, and similar to chain of responsibility design pattern.

One technique is to add an instruction object to an RcQueue that a single manager processes in a separate dedicated gem. Many gems give instructions, and one manager processes them without conflicts. Gem-to-gem signaling can be part of that design if delay loops are not desired or timely. Nested transactions might come in handy for this now that gs has them. The commits and aborts used for queued work may not be ideal for some applications.

There are many tricks and techniques, the most appropriate depends on your application needs. Review the RC collections to see if any help you. Your application code needs to have a strategy to avoid conflicts though. Locking can be avoided for most code. It seems odd that your code has multiple gems updating the same class variables.

Paul Baumann

Collections that use value holders, like Dictionary having Association instances, only have write conflicts on growth or when a value is updated by two gems. A recursion lock on growth (and perhaps removals) can be enough if more than one session is not assigning the same keys. Pre-allocation (using #new:) may remove the need for locks because there would be no resize. A lock on the association can be useful to avoid duplication of effort between sessions that might precede commit failure. KeyValueCollection is object-allocation efficient (faster for some uses) but would have more conflicts on the structures holding values.