[ANN] Process-Specific Variables

I'd like to announce the first release of Process-Specific Variables,
from

http://www.squeaksource.com/ProcessLocalStorage.html

Process-specific variables are variables that are neither global
nor specific to an object (like instance variables) or to a class.
Instead, they can have a different value in the context of each
Process.

New variables are declared by inheriting from a base class, similar
to the way WADynamicVariable of Seaside works.

There are two kinds of process-specific variables:
- ProcessLocalVariable is the base class for process-specific
   read-write variables. They are similar to thread-local storage
   (TLS) available in various operating systems. Reading and writing
   uses #value and #value:
- DynamicVariable is the base class of dynamically-scoped read-only
   variables. They are similar to the dynamic scoping in LISP.
   They can be read using #value, and set for the duration of a block
   with #value:during:

The implementation uses a per-process dictionary which is keyed by
the variable class.

Comments are welcome,

Martin

Martin,

        I'm sure you are able to think about the aplicability of
Process-Specific Variables. Can you elaborate some examples? I'm pretty sure
I'm not the only one that is not up to date with that and I think it will
help to understand the practical consequences of using them facilitating the
ponderation of it's value.

        thanks,

Sebastian Sastre

I haven't looked at the code so I'm guessing but from the description it
sounds as if you'd be doing something like:

        ProcessLocalVariable subclass: #MyGlobalVar

and then something like:

        MyGlobalVar value. "retrieve its value"
        MyGlobalVar value: aValue. "set the value"

except that the value of the variable is only valid in the scope of the
current process.

The idea of having a "class per variable" is actually intriguing to me -
I had never thought of it that way but it makes for some interesting
implications. And with a trivial modification to the compiler would
allow you to do something like, e.g.,

        MyGlobalVar. "retrieve value"
        MyGlobalVar := aValue. "set the value"

Cheers,
   - Andreas

Sebastian Sastre wrote:

Il giorno mar, 13/03/2007 alle 12.33 -0700, Andreas Raab ha scritto:
Isn't this similar to what DynamicBindings does?

        Giovanni

Sebastian Sastre schrieb:
> I'm sure you are able to think about the aplicability of
> Process-Specific Variables. Can you elaborate some examples? I'm pretty sure
> I'm not the only one that is not up to date with that and I think it will
> help to understand the practical consequences of using them facilitating the
> ponderation of it's value.

It's in the wider field of context-oriented/context-aware programming.
Application code (in particular server applications) often have the same
code operating in various contexts, and methods need to find out what
context they operate in.

To give a few examples:
- KomHttpServer includes HttpRequest>>current, which always gives the
   "current" HTTP request. This is set in a process-specific way when
   the request is read from the wire, then all processing methods can
   access it (in the context of a Process), even though they don't
   get the request object as a parameter. This is implemented using the
   DynamicBindings package.
- Seaside offers WACurrentSession>>value, which always provides you
   with the current Seaside session object (related to, but not identical
   with the HTTP request: the session object will store data that
   wasn't transmitted in the last request, but accumulated throughout
   the session). Again, in Seaside, you "always" need to know what
   the "current session" is, but don't want to pass this as a
   parameter.
   At the same time, this can't be a global variable, since multiple
   processes may simulatenously process requests.
- In a transaction environment, there is often a need to know what
   the "current" transaction is.
- In testing, it may be interesting to know whether code is run
   "in" the tester, i.e. whether it was invoked through the test
   runner or not.

These are all examples for dynamic variables: somebody sets it
for a process, then runs some code that only reads it, and then
it should get unset (revert to the previous value).

Process-local variables differ in that a value "persists" even
after the code setting it completes. For an introduction, see

http://en.wikipedia.org/wiki/Thread-local_storage

In Squeak, the only example I could find that may want to use
it is, again, in Seaside:
- WASpotProfiler>>profile: currently reads

        | time value |
        time := Time millisecondsToRun: [value := aBlock value].
        (readings at: Processor activeProcess ifAbsentPut: [OrderedCollection
new]) add: time.
        ^ value

   With process-local variables that could change to

         time := Time millisecondsToRun: [value := aBlock value].
        WAReadings value add: time.
         ^value

   Here, I use the feature that the process-local variables as
   I defined them allow the definition of a default value,
   which would be OrderedCollection new in the case of WAReadings.
   Of course, one would also need a way to gather totals across
   all processes (I'm uncertain why WASpotProfiler collects
   them per-process).

- In VisualWorks, a similar mechanism is available; they use
   process-local variables to record the "current debugger"
   per Process, and to inject values into a process from the
   outside.

- Also in VW, they have per-process UI managers.

- In the GNU C library, thread-local storage is used to reduce
   contention in memory management. Each thread has its own
   memory arena from which memory is allocated, and global
   synchronization is only performed when the arena is exhausted.

- Also in the C library, the value of errno (the last error
   that occurred) is thread-specific. It can't be a dynamic
   variable, since it is set by the callee, and read by the
   caller.

- A random-number generator may store its state in a thread-local
   object, in order to provide a repeatable sequence of pseudo-
   random number on a per-thread basis (rather than having
   other threads interfere by picking numbers out of that sequence).


In reviewing code from other languages, it seems that they often
use thread-local variables as the foundation for implementing
dynamic variables on top of it. I believe there is a use for
both of them, so I included them both.

HTH,
Martin

Andreas Raab schrieb:
> I haven't looked at the code so I'm guessing but from the description it
> sounds as if you'd be doing something like:
>
>     ProcessLocalVariable subclass: #MyGlobalVar
>
> and then something like:
>
>     MyGlobalVar value. "retrieve its value"
>     MyGlobalVar value: aValue. "set the value"

Exactly so.

> except that the value of the variable is only valid in the scope of the
> current process.

Right. If no value was set, the variable will get a #default message,
to which it can either provide a value, or raise an error. The default
for default is nil.

> The idea of having a "class per variable" is actually intriguing to me -
> I had never thought of it that way but it makes for some interesting
> implications. And with a trivial modification to the compiler would
> allow you to do something like, e.g.,
>
>     MyGlobalVar. "retrieve value"
>     MyGlobalVar := aValue. "set the value"

The tricky part is to still allow access to the class, e.g. for
further subclassing, browsing, etc, if desired.

Anyway, if there is interest, I'm happy to contribute it to the Squeak
project, in some form.

Regards,
Martin

Giovanni Corriga wrote:
> Il giorno mar, 13/03/2007 alle 12.33 -0700, Andreas Raab ha scritto:
>> MyGlobalVar value. "retrieve its value"
>> MyGlobalVar value: aValue. "set the value"
>
> Isn't this similar to what DynamicBindings does?

Good question, I don't know (anyone knowing DynamicBindings care to
compare the approaches?) I've looked at the code in the meantime and it
does exactly what I thought it would ;-) though there was one thing that
surprised me: ProcessSpecificVariable caches the computed hash value,
which I assume it does for performance reasons. The reason this
surprises me is that I would not have expected this to have a
significant performance impact, maybe that's just a left-over?

But anyway, the idea is pretty cool, and I'm sure I can find uses for
it. If someone could compare/contrast with DynamicBindings this would be
greatly appreciated, too.

Cheers,
   - Andreas

Giovanni Corriga schrieb:
>> except that the value of the variable is only valid in the scope of the
>> current process.
>
> Isn't this similar to what DynamicBindings does?

Indeed. I see two important differences, though:
- different (better?) implementation strategy: DynamicBinding is based
   on exceptions to find the innermost binding; this requires a stack
   walk for a matching exception handler. It then uses a linked list of
   dictionaries for lookup. So lookup time is typically linear with the
   stack depth, and then linear with the depth of the binding stack.
   My implementation offers constant-time lookup (if there are no
   dictionary collisions), as it has the current value always in the
   current process (getting the current process, getting a slot of it,
   fetching the reference to the global class, and passing that as
   a dictionary key are all constant-time).
- separation of process-local and dynamic variables: in DynamicBindings,
   you can rebind a variable at any point in time, and it uses the
   innermost binding (whereever that is). In my solution, dynamic
   variables can be only set, then read, but not written (i.e. rebound).
   Experiences with dynamic scoping in LISP show that they might be
   difficult to understand; I expect that making them read-only gives
   more readable code, as you don't need to worry that a function
   you call may change the value behind you.
   OTOH, process-local variables are read-write, but they don't have
   any nesting to them. This is, AFAICT, not directly supported in
   DynamicBinding, as you need to create at least one scope
   explicitly to establish per-process values. In process-local
   variables (called TLS elsewhere), every Process will have it own
   value for it right from the start, using a default mechanism.

Regards,
Martin

Andreas Raab schrieb:
> Good question, I don't know (anyone knowing DynamicBindings care to
> compare the approaches?) I've looked at the code in the meantime and it
> does exactly what I thought it would ;-) though there was one thing that
> surprised me: ProcessSpecificVariable caches the computed hash value,
> which I assume it does for performance reasons. The reason this
> surprises me is that I would not have expected this to have a
> significant performance impact, maybe that's just a left-over?

It is indeed for performance reasons. Computing the hash alone,

1000000 timesRepeat:[C hash]

for some class C, on my machine, uses roughly 900ms to run
using the standard class hash, and uses 180ms to run using
the cached hash. A plain C yourself loop uses 120ms. So for
the hashing alone, the speedup is fairly dramatic (factor of
30, roughly). I haven't timed the impact on complete lookup
time for a variable (but could do so if there is interest).

In any case, that's a minor detail of the implementation;
taking it out has no effect other than slowing it a bit
and freeing 4 bytes per subclass.

> But anyway, the idea is pretty cool, and I'm sure I can find uses for
> it. If someone could compare/contrast with DynamicBindings this would be
> greatly appreciated, too.

See the message I just sent. In summary: DynamicBindings uses a stack
walk to find the innermost binding (using notifications). I couldn't
completely understand the implementation, as it has code like
'tmp _ thisContext. "..." tmp sender sender sender swapSender:
something' in it, but I still think that lookup should be fairly
expensive for a deep stack. I also don't like it that it mixes
nested scopes with allowing write access from any point in the
control flow.

Regards,
Martin