Process Model Documentation

Hi Ken,

The important difference with GS process scheduling is that GS doesn't automatically switch to a scheduled process for as many reasons as other Smalltalk dialects would. A GS forked process won't automatically switch processing to a scheduled process while control returns to VW and the gem is otherwise "idle". Think of the GS scheduler as a queue that you need to explicitly cause switches in. Semaphore waits can force switches and so can a (priority dependent) "Processor yield". I don't recall if a Delay or sleep in GS allows available processes to get attention (like other dialects would). A forked process that ends will cause a switch to an available process.

I've implemented several ways of forcing distributed parallel processing between VW and gem. The fundamentals are:

1) VW initiates a request in GS that forks work and quickly returns. Use a semaphore to 'ratchet' VW into starting the GS activity immediately instead of just having it scheduled for when the activeProcess gets suspended; here is an example that also shows how you can prove the sequence you expect:

fetchRowsInBackground

                "For maximum performance, this starts getting data from GS while VW is busy opening the window. -plb"

                | ratchetSem seq |

                self rows_release.

                self gbsSession_isValid

                                ifFalse: [self gbsSession: (GBSM currentSessionIfNone: [^nil])].

                seq := OrderedCollection new.

                ratchetSem := Semaphore new.

                seq add: 1.

                [

                                seq add: 3.

                                ratchetSem signal.

                                seq add: 4.

                                self avoidRefreshWhile: [self queryGroupRowsForMilestone].

                                seq add: 6.

                ] fork.

                seq add: 2.

                ratchetSem wait.

                seq add: 5.

                "seq inspect."

2) VW forks a VW process that then gets GS to actually start working on the GS forked process.

3) What the original VW process does while the forked work is being done is completely up to the design for your needs, but the original process will wait on GBS semaphores if there is any attempt to communicate with GS while the forked process are active. That would limit parallel activity, so therefore it is good to design so that the original VW process provides a value holder that is to be updated with results (or progress).

4) It can be good to design forked GS processing to return back to VW every so often so that nothing ever feels hung. This demonstrates a milestone based approach:

queryGroupRowsForMilestone

                "msFromMilestone is nil if manager is not running.

                msFromMilestone is 0 if milestone has been attained.

                msFromMilestone is negative number of ms that manager index is overdue.

                msFromMilestone is positive number of ms that manager index is expected to complete in.

                "

                | msFromMilestone msToWait pass |

                msFromMilestone := self queryGroupRows.

                (msFromMilestone isNil or: [msFromMilestone = 0]) ifTrue: [

                                "Either the query manager is dead (nil) or the milestone is within an allowed age."

                                unattainedMilestone := nil.

                                ^msFromMilestone

                ].

                "We're going to have to wait for it..."

                pass := false.

                #(100 200 400 800 1200 1200 1200 1200 1200 1200 1200 1200 1200 1200 1200 1200 1200 1200 1200 1200)

                anySatisfy: [:overdueWaitMs |

                                pass := msFromMilestone isNil or: [msFromMilestone = 0].

                                pass ifFalse: [

                                                msToWait := overdueWaitMs max: (msFromMilestone min: 4000).

                                                (Delay forMilliseconds: msToWait) wait.

                                                msFromMilestone := self queryMsFromMilestone.

                                ].

                                pass

                ].

                "We either found it or else the manager refresh is overdue now; regardless, we just clear the milestone and answer visible results."

                unattainedMilestone := nil.

                ^self queryGroupRows

5) Every time the forked VW process returns from GS it should update some form of result holder. If the result holder has dependents that may be visual then you should do the assignment within a #uiEventFor: block. The #uiEventFor: schedules whatever process is (or becomes) the active UI Process.

6) Your VW windows should be oriented for a request-and-forget style than an back-and-forth incremental replication. Your replication specs should be defined to return little more than what you need but certainly no less. Stub faults should be eliminated.

7) Depending on your needs, you can either have the forked process do incremental replication or you can have the forked process just inform when results are available so that the active UI Process will then replicate at once in the end. Your choice can depend on where time is spent most and the amount of GS object updates (from transaction view changes) that are likely to happen during forked processing.

8) When doing parallel processing you should anticipate that object and collections of objects may have changed (from a transaction view change) since the GS process last had activity. If you are enumerating a large root collection of domain objects then understand that the collection may have changed size since you last saw it. One trick is to have the forked GS process enumerate a copy of the collection. A trick for sequenceable collections that only grow is that they can be enumerated starting from the last position to the first. Results gathered in small interactive chunks may not always reflect state from a single GS transaction view. You could for example collect objects that would no longer be found on a second look. State gathered from objects during chunked enumeration can be different than state gathered from those same objects after all chunks have been enumerated.

9) A final note is that I recall that tricks like these had exposed bugs (as hangs) with the single-round-trip implementation until at least GBS release 7.2.1patch3. You are not likely to experience those problems in the latest release or in releases or older than 7x. I was able to work around the problems but others should use the latest GBS release.

The GS_CollectionViews framework that I released public domain a couple years ago has tricks like these built into it. For many queries there is no faster way to query GS for results and return them to VW. An entirely different approach that can be better for larger jobs that benefit from different memory configurations is to do work in a separate remote gems headed by topaz. I've released a GS_TopazLauncher framework as public domain to make that easy. I haven't had time to post the newer versions of those frameworks nor give good documentation for them.

Paul Baumann

Is any documentation available on the GemStone process model? Maybe I'm overlooking something, but the Programming and SysAdmin Guides don't seem to mention anything about it.