[ANN] A Unifying Mechanism for the Implementation of Concurrency Abstractions

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[ANN] A Unifying Mechanism for the Implementation of Concurrency Abstractions

Stefan Marr-2
Hi, beware the shameless plug:

Our paper 

                   Identifying A Unifying Mechanism for the
                 Implementation of Concurrency Abstractions on
                       Multi-Language Virtual Machines [1]

introduces an ownership-based Meta-object Protocol (MOP) to enable the
enforcement of the various concurrency semantics required by the different
programming models discussed in literature.

The prototypical implementation is done on top of Pharo 1.3 and available at
SqueakSource3 [2] under the MIT license.

You might be interested in it to experiment with new concurrent programming
models. Typically, green threads are enough to be exposed to race conditions
and other correctness problems. Thus, a CogVM is sufficient for experiments.
RoarVM support is coming later, enabling experiments with the parallel aspects
of these programming models as well.

For the evaluation, we implemented a couple of concurrency models on top of
it. Most notably, you can use Clojure-like agents, AmbientTalk-like actors,
a basic CSP, Active Objects, and an STM that is a port of Lukas Renggli's
work. These concurrency models work out of the box with the Metacello
configuration given below (to the degree they were tested and benchmarked).

You might also be interested in the AmbientTalkST implementation that was done
as comparison as well as the direct port of Lukas' STM to Pharo 1.3 [3].

Please see the paper [1] for an introduction to the MOP and an example how to
use it to make objects immutable and implement Clojure's agents guaranteeing
all required semantics.

The paper's abstract:

    Supporting all known abstractions for concurrent and parallel programming
    in a virtual machines (VM) is a futile undertaking, but it is required to
    give programmers appropriate tools and performance. Instead of supporting
    all abstractions directly, VMs need a unifying mechanism similar to
    INVOKEDYNAMIC for JVMs.

    Our survey of parallel and concurrent programming concepts identifies
    concurrency abstractions as the ones benefiting most from support in a VM.
    Currently, their semantics is often weakened, reducing their engineering
    benefits. They require a mechanism to define flexible language guarantees.

    Based on this survey, we define an ownership-based meta-object protocol as
    candidate for VM support. We demonstrate its expressiveness by
    implementing actor semantics, software transactional memory, agents, CSP,
    and active objects. While the performance of our prototype confirms the
    need for VM support, it also shows that the chosen mechanism is
    appropriate to express a wide range of concurrency abstractions in a
    unified way.

The code can be loaded into a fresh Pharo 1.3 image [4] with the following
Gofer expression. However, be aware this is experimental software.
Furthermore, note that you will need to acknowledge changes to the Process
class:


      Gofer new
          url: 'http://ss3.gemstone.com/ss/Omni';
          package: 'ConfigurationOfOmni';
          load.
      (Smalltalk at: #ConfigurationOfOmni) loadTOOLS2012.

As mentioned before, the paper gives some examples. The definition of 
different concurrency semantics can be found in subclasses of OstDomain.
Examples of how to use any of the existing concurrency models are either the
unit tests, or the included benchmarks.

Note that the MOP based implementation of AmbientTalkST does use the same
class names. Thus, to use the ad-hoc implementation, you will need to load it
into a different image:

      Gofer new
          url: 'http://ss3.gemstone.com/ss/Omni';
          package: 'ConfigurationOfAmbientTalkST';
          load.
      (Smalltalk at: #ConfigurationOfAmbientTalkST) loadTOOLS2012.

Similarly, the port of Lukas Renggli's STM to Pharo needs to be loaded 
into a separate image, too. This version is not using our MOP:

      Gofer new
          url: 'http://ss3.gemstone.com/ss/LRSTM';
          package: 'ConfigurationOfLRSTM';
          load.
      (Smalltalk at: #ConfigurationOfLRSTM) loadTOOLS2012.


Acknowledgements:

Thanks to Lukas for his STM implementation I used here. It was also a strong
inspiration for my bytecode transformation approach.

Best regards
Stefan



[1] http://soft.vub.ac.be/~smarr/2012/03/identifying-a-unifying-mechanism-for-the-implementation-of-concurrency-abstractions-on-multi-language-virtual-machines/
[2] http://ss3.gemstone.com/ss/Omni.html
[3] http://ss3.gemstone.com/ss/LRSTM.html
[4] http://gforge.inria.fr/frs/download.php/29273/Pharo-1.3-13315.zip



-- 
Stefan Marr
Software Languages Lab
Vrije Universiteit Brussel
Pleinlaan 2 / B-1050 Brussels / Belgium
http://soft.vub.ac.be/~smarr
Phone: +32 2 629 2974
Fax:   +32 2 629 3525

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] A Unifying Mechanism for the Implementation of Concurrency Abstractions

Angel Java Lopez
Great! Very interesting, informative. A good reading for this weekend and beyond.

I want STM for Smalltalk, and Actor model. And remote objects (message passing? Erlang-like?)
My toy experiment for STM
http://ajlopez.wordpress.com/2010/11/15/transactional-objects-in-ajtalk-a-quick-intro/ 
For actor model (I used the term "agent" but I think "actor" is more appropiate)
For remote objects

My current idea: all these features can be implemented by decorators, over normal objects, like:

mytransobj := anObject asTransactional
myactor := anObject asActor
myobjexposedtoremoteserver := anObject asMarshallByRef "or something alike"

instead of dedicated special classes or objects

On Fri, Mar 9, 2012 at 11:38 AM, Stefan Marr <[hidden email]> wrote:
Hi, beware the shameless plug:

Our paper 

                   Identifying A Unifying Mechanism for the
                 Implementation of Concurrency Abstractions on
                       Multi-Language Virtual Machines [1]

introduces an ownership-based Meta-object Protocol (MOP) to enable the
enforcement of the various concurrency semantics required by the different
programming models discussed in literature.

The prototypical implementation is done on top of Pharo 1.3 and available at
SqueakSource3 [2] under the MIT license.

You might be interested in it to experiment with new concurrent programming
models. Typically, green threads are enough to be exposed to race conditions
and other correctness problems. Thus, a CogVM is sufficient for experiments.
RoarVM support is coming later, enabling experiments with the parallel aspects
of these programming models as well.

For the evaluation, we implemented a couple of concurrency models on top of
it. Most notably, you can use Clojure-like agents, AmbientTalk-like actors,
a basic CSP, Active Objects, and an STM that is a port of Lukas Renggli's
work. These concurrency models work out of the box with the Metacello
configuration given below (to the degree they were tested and benchmarked).

You might also be interested in the AmbientTalkST implementation that was done
as comparison as well as the direct port of Lukas' STM to Pharo 1.3 [3].

Please see the paper [1] for an introduction to the MOP and an example how to
use it to make objects immutable and implement Clojure's agents guaranteeing
all required semantics.

The paper's abstract:

    Supporting all known abstractions for concurrent and parallel programming
    in a virtual machines (VM) is a futile undertaking, but it is required to
    give programmers appropriate tools and performance. Instead of supporting
    all abstractions directly, VMs need a unifying mechanism similar to
    INVOKEDYNAMIC for JVMs.

    Our survey of parallel and concurrent programming concepts identifies
    concurrency abstractions as the ones benefiting most from support in a VM.
    Currently, their semantics is often weakened, reducing their engineering
    benefits. They require a mechanism to define flexible language guarantees.

    Based on this survey, we define an ownership-based meta-object protocol as
    candidate for VM support. We demonstrate its expressiveness by
    implementing actor semantics, software transactional memory, agents, CSP,
    and active objects. While the performance of our prototype confirms the
    need for VM support, it also shows that the chosen mechanism is
    appropriate to express a wide range of concurrency abstractions in a
    unified way.

The code can be loaded into a fresh Pharo 1.3 image [4] with the following
Gofer expression. However, be aware this is experimental software.
Furthermore, note that you will need to acknowledge changes to the Process
class:


      Gofer new
          url: 'http://ss3.gemstone.com/ss/Omni';
          package: 'ConfigurationOfOmni';
          load.
      (Smalltalk at: #ConfigurationOfOmni) loadTOOLS2012.

As mentioned before, the paper gives some examples. The definition of 
different concurrency semantics can be found in subclasses of OstDomain.
Examples of how to use any of the existing concurrency models are either the
unit tests, or the included benchmarks.

Note that the MOP based implementation of AmbientTalkST does use the same
class names. Thus, to use the ad-hoc implementation, you will need to load it
into a different image:

      Gofer new
          url: 'http://ss3.gemstone.com/ss/Omni';
          package: 'ConfigurationOfAmbientTalkST';
          load.
      (Smalltalk at: #ConfigurationOfAmbientTalkST) loadTOOLS2012.

Similarly, the port of Lukas Renggli's STM to Pharo needs to be loaded 
into a separate image, too. This version is not using our MOP:

      Gofer new
          url: 'http://ss3.gemstone.com/ss/LRSTM';
          package: 'ConfigurationOfLRSTM';
          load.
      (Smalltalk at: #ConfigurationOfLRSTM) loadTOOLS2012.


Acknowledgements:

Thanks to Lukas for his STM implementation I used here. It was also a strong
inspiration for my bytecode transformation approach.

Best regards
Stefan






-- 
Stefan Marr
Software Languages Lab
Vrije Universiteit Brussel
Pleinlaan 2 / B-1050 Brussels / Belgium
http://soft.vub.ac.be/~smarr
Phone: +32 2 629 2974
Fax:   +32 2 629 3525


Reply | Threaded
Open this post in threaded view
|

Re: [ANN] A Unifying Mechanism for the Implementation of Concurrency Abstractions

Stefan Marr-2
Hi:

On Friday, March 9, 2012 3:59:48 PM UTC+1, ajlopez wrote:
My current idea: all these features can be implemented by decorators, over normal objects, like:

mytransobj := anObject asTransactional
myactor := anObject asActor
myobjexposedtoremoteserver := anObject asMarshallByRef "or something alike"

instead of dedicated special classes or objects

The important difference between my approach and 'just decorating' is that I use bytecode transformation to achieve transparency and enforceability. Sure, inheritance is not a desirable technique to enforce for instance actor guarantees.
However, the design aspect is not the main reason to use something different. Enforceability is the main reason. Neither an inheritance scheme nor some kind of decorating/wrapping will provide you with the full engineering benefits of 'language level guarantee enforcement'.

Typical concurrency guarantees like encapsulation, immutability, or asynchronous invokation are hard or impossible to enforce in a library. That's where my ownership-based meta-object protocol comes in.

Best regards
Stefan
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] A Unifying Mechanism for the Implementation of Concurrency Abstractions

Eliot Miranda-2


On Fri, Mar 9, 2012 at 3:08 PM, Stefan Marr <[hidden email]> wrote:
Hi:


On Friday, March 9, 2012 3:59:48 PM UTC+1, ajlopez wrote:
My current idea: all these features can be implemented by decorators, over normal objects, like:

mytransobj := anObject asTransactional
myactor := anObject asActor
myobjexposedtoremoteserver := anObject asMarshallByRef "or something alike"

instead of dedicated special classes or objects

The important difference between my approach and 'just decorating' is that I use bytecode transformation to achieve transparency and enforceability. Sure, inheritance is not a desirable technique to enforce for instance actor guarantees.
However, the design aspect is not the main reason to use something different. Enforceability is the main reason. Neither an inheritance scheme nor some kind of decorating/wrapping will provide you with the full engineering benefits of 'language level guarantee enforcement'.

Typical concurrency guarantees like encapsulation, immutability, or asynchronous invokation are hard or impossible to enforce in a library. That's where my ownership-based meta-object protocol comes in.

But an interesting question is what is the base semantics that mean you can implement without redefining bytecode?  For example, if all your rewrites do is redefine inst var access then replacing direct inst var access by message send (e.g. a la Newspeak) is all one needs.  So Stefan, could you have a go at answering the question?


Best regards
Stefan



--
best,
Eliot

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] A Unifying Mechanism for the Implementation of Concurrency Abstractions

Stefan Marr-2
Hi Eliot:

On Saturday, March 10, 2012 12:36:04 AM UTC+1, Eliot wrote:
Typical concurrency guarantees like encapsulation, immutability, or asynchronous invokation are hard or impossible to enforce in a library. That's where my ownership-based meta-object protocol comes in.

But an interesting question is what is the base semantics that mean you can implement without redefining bytecode?  For example, if all your rewrites do is redefine inst var access then replacing direct inst var access by message send (e.g. a la Newspeak) is all one needs.  So Stefan, could you have a go at answering the question?

(Is there something important missing in these sentences? I might misunderstand the question.)
Sure, you can always model inst var accesses with message sends, but that isn't enough.
Beside that, you still want a hook that allows you to define you own concurrency related semantics.

Classically, these kind of things are done via a metaclasses/class hierarchy, enabling you to define exactly what happens when a message is received, and how to process it.
However, metaclasses are not orthogonal to your classes, instead, they classify your classes.
With an ownership-base MOP, we can define these semantics orthogonal to any class hierarchy.
And, we can even change the owner of an object, and that way, we can change which semantics are enforced.

So, for Smalltalk we need to take care of field access, message sends, primitives, and reflective operations. These are delegated to a so-called domain, which in my case defines the semantics of a concurrency model, and for instance enforces immutability or that only one thread at a time is executing in an actor's domain, etc.

Does that answer what you were asking for?

Best regards
Stefan
 
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] A Unifying Mechanism for the Implementation of Concurrency Abstractions

Gilad Bracha-2


On Fri, Mar 9, 2012 at 4:50 PM, Stefan Marr <[hidden email]> wrote:

So, for Smalltalk we need to take care of field access, message sends, primitives, and reflective operations. These are delegated to a so-called domain, which in my case defines the semantics of a concurrency model, and for instance enforces immutability or that only one thread at a time is executing in an actor's domain, etc.

Does that answer what you were asking for?

I can't speak for Eliot, but I would hope so. It sounds like you are using intercession to introduce various concurrency models, and implementing intercession via byte code rewriting. need to reinterpret the semantics of the language's basic operations. 

The fewer of these there are, the easier that is. I'm not sure why reflective operations are any different than ordinary sends in Smalltalk. In Newspeak, you'd want to intercede on message sends, since, logically, that's all there is.  Of course, depending where intercession is implemented and what optimizations are performed by what component, things may be more complicated.

I hope to find the time to read your work.

--
Cheers, Gilad