Threads safety in Pharo

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Threads safety in Pharo

Pharo Smalltalk Users mailing list
Hi,

Somebody once evoked the problem of threads safety in Pharo. With a friend of mine who is expert in formal methods and process scheduling, we would like to have a look on it.
Does anyone knows a good document describing the problem of Pharo with threads safety or at least any document that we can start with?

Thanks in advance,
Abdelghani


Reply | Threaded
Open this post in threaded view
|

Re: Threads safety in Pharo

Guillermo Polito
I believe there is no such a document. It would be however interesting to investigate it a bit deeper. In general, the problem we talk about when we talk about thread safety is the following: Can we run a workspace in a separate thread than a browser and provide correct results? Can we run two browsers concurrently and change code from both of them?

The thread safety problem has several levels I'd say:

 - the kernel of the language (classes, methods, compilation, and so on) cannot safely be modified while other threads are running. Thus, if we compile a new version of a method in a thread, that could break a separate thread that was using that method.

 - there are several libraries using global state. We should investigate if they work correctly when using them concurrently.
    E.g., source code management is not thread safe. This could cause a source code corruption if several methods are modified concurrently.
    Like that, we should investigate all libraries and see what should be adapted and if there is a need at all.

 - in a third level, general core libraries (collections, networking, files, etc) are not designed to be thread safe and that is natural. For example, most of the time you don't want that your collection is accessed concurrently. For such cases, some libraries could provide some extensions/wrappers/whatsoever to provide some synchronization mechanism. Otherwise it is the responsibility of the user to synchronize usages with semaphores/mutexes.
 

On Mon, Jul 31, 2017 at 1:38 AM, Alidra Abdelghani via Pharo-users <[hidden email]> wrote:


---------- Forwarded message ----------
From: Alidra Abdelghani <[hidden email]>
To: [hidden email]
Cc: "Stéphane Ducasse" <[hidden email]>, farid arfi <[hidden email]>
Bcc: 
Date: Mon, 31 Jul 2017 01:38:58 +0200
Subject: Threads safety in Pharo
Hi,

Somebody once evoked the problem of threads safety in Pharo. With a friend of mine who is expert in formal methods and process scheduling, we would like to have a look on it.
Does anyone knows a good document describing the problem of Pharo with threads safety or at least any document that we can start with?

Thanks in advance,
Abdelghani






--

   

Guille Polito


Research Engineer

French National Center for Scientific Research - http://www.cnrs.fr



Web: http://guillep.github.io

Phone: +33 06 52 70 66 13

Reply | Threaded
Open this post in threaded view
|

Re: Threads safety in Pharo

Ben Coman
In reply to this post by Pharo Smalltalk Users mailing list
Not sure I'll have what you're looking for, but to start, do you mean Pharo's green threads or vm native threads?
cheers -ben

On Mon, Jul 31, 2017 at 7:38 AM, Alidra Abdelghani via Pharo-users <[hidden email]> wrote:


---------- Forwarded message ----------
From: Alidra Abdelghani <[hidden email]>
To: [hidden email]
Cc: "Stéphane Ducasse" <[hidden email]>, farid arfi <[hidden email]>
Bcc: 
Date: Mon, 31 Jul 2017 01:38:58 +0200
Subject: Threads safety in Pharo
Hi,

Somebody once evoked the problem of threads safety in Pharo. With a friend of mine who is expert in formal methods and process scheduling, we would like to have a look on it.
Does anyone knows a good document describing the problem of Pharo with threads safety or at least any document that we can start with?

Thanks in advance,
Abdelghani




Reply | Threaded
Open this post in threaded view
|

Re: Threads safety in Pharo

Stephane Ducasse-3
I would love to have an analysis of assumptions made in some code.
Because my impression is that the concurrent code is sometimes defined
knowing the underlying logic of scheduler and this is not good.
As I said to abdel privately in french it would be great to start from
my french squeak book (Yes I wrote one long time ago) chapter on
concurrent programming and turn it into a pharo chapter.

Stef

On Tue, Aug 1, 2017 at 1:31 PM, Ben Coman <[hidden email]> wrote:

> Not sure I'll have what you're looking for, but to start, do you mean
> Pharo's green threads or vm native threads?
> cheers -ben
>
> On Mon, Jul 31, 2017 at 7:38 AM, Alidra Abdelghani via Pharo-users
> <[hidden email]> wrote:
>>
>>
>>
>> ---------- Forwarded message ----------
>> From: Alidra Abdelghani <[hidden email]>
>> To: [hidden email]
>> Cc: "Stéphane Ducasse" <[hidden email]>, farid arfi
>> <[hidden email]>
>> Bcc:
>> Date: Mon, 31 Jul 2017 01:38:58 +0200
>> Subject: Threads safety in Pharo
>> Hi,
>>
>> Somebody once evoked the problem of threads safety in Pharo. With a friend
>> of mine who is expert in formal methods and process scheduling, we would
>> like to have a look on it.
>> Does anyone knows a good document describing the problem of Pharo with
>> threads safety or at least any document that we can start with?
>>
>> Thanks in advance,
>> Abdelghani
>>
>>
>>
>

Squeak_09.pdf (772K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Threads safety in Pharo

Pharo Smalltalk Users mailing list
In reply to this post by Ben Coman
Hi Ben,

On 01 Aug 2017, at 13:31, Ben Coman <[hidden email]> wrote:

Not sure I'll have what you're looking for, but to start, do you mean Pharo's green threads or vm native threads?
I believe this is about Pharo threads. 
I am not quite sure of what the vm native threads are or if there is a safety problem with them. But still, it could be interesting to investigate :)

Thanks
Abdelghani

cheers -ben

On Mon, Jul 31, 2017 at 7:38 AM, Alidra Abdelghani via Pharo-users <[hidden email]> wrote:


---------- Forwarded message ----------
From: Alidra Abdelghani <[hidden email]>
To: [hidden email]
Cc: "Stéphane Ducasse" <[hidden email]>, farid arfi <[hidden email]>
Bcc: 
Date: Mon, 31 Jul 2017 01:38:58 +0200
Subject: Threads safety in Pharo
Hi,

Somebody once evoked the problem of threads safety in Pharo. With a friend of mine who is expert in formal methods and process scheduling, we would like to have a look on it.
Does anyone knows a good document describing the problem of Pharo with threads safety or at least any document that we can start with?

Thanks in advance,
Abdelghani





Reply | Threaded
Open this post in threaded view
|

Re: Threads safety in Pharo

Pharo Smalltalk Users mailing list
In reply to this post by Guillermo Polito

On 01 Aug 2017, at 09:36, Stéphane Ducasse <[hidden email]> wrote:

To complement.

What would be fun is to investigate the implementation of ProcessScheduler, Process and check the implementation. 

Abdel there is a nice presentation of the processor and process in the Squeak french book. 
Great, I taught some of theses concepts to my students some times ago and we did some fun lab sessions with the Linux scheduler

This was on my todo to turn it into a book chapter for Pharo. 

So may be we could work together to write a little booklet on this aspect to get started.

I would love to :D


Stef

Reply | Threaded
Open this post in threaded view
|

Re: Threads safety in Pharo

Pharo Smalltalk Users mailing list
In reply to this post by Guillermo Polito
Thanks Guille for your answer,
From what I understand, it is mainly a synchronisation problem. Why cant' existing synchronisation mechanisms (semaphores and mutex) be used?
Abdelghani


On 31 Jul 2017, at 18:44, Guillermo Polito <[hidden email]> wrote:

I believe there is no such a document. It would be however interesting to investigate it a bit deeper. In general, the problem we talk about when we talk about thread safety is the following: Can we run a workspace in a separate thread than a browser and provide correct results? Can we run two browsers concurrently and change code from both of them?

The thread safety problem has several levels I'd say:

 - the kernel of the language (classes, methods, compilation, and so on) cannot safely be modified while other threads are running. Thus, if we compile a new version of a method in a thread, that could break a separate thread that was using that method.

 - there are several libraries using global state. We should investigate if they work correctly when using them concurrently.
    E.g., source code management is not thread safe. This could cause a source code corruption if several methods are modified concurrently.
    Like that, we sh n ould investigate all libraries and see what should be adapted and if there is a need at all.

 - in a third level, general core libraries (collections, networking, files, etc) are not designed to be thread safe and that is natural. For example, most of the time you don't want that your collection is accessed concurrently. For such cases, some libraries could provide some extensions/wrappers/whatsoever to provide some synchronization mechanism. Otherwise it is the responsibility of the user to synchronize usages with semaphores/mutexes.
 

On Mon, Jul 31, 2017 at 1:38 AM, Alidra Abdelghani via Pharo-users <[hidden email]> wrote:


---------- Forwarded message ----------
From: Alidra Abdelghani <[hidden email]>
To: [hidden email]
Cc: "Stéphane Ducasse" <[hidden email]>, farid arfi <[hidden email]>
Bcc: 
Date: Mon, 31 Jul 2017 01:38:58 +0200
Subject: Threads safety in Pharo
Hi,

Somebody once evoked the problem of threads safety in Pharo. With a friend of mine who is expert in formal methods and process scheduling, we would like to have a look on it.
Does anyone knows a good document describing the problem of Pharo with threads safety or at least any document that we can start with?

Thanks in advance,
Abdelghani






--
   
Guille Polito

Research Engineer
French National Center for Scientific Research - http://www.cnrs.fr


Phone: +33 06 52 70 66 13

Reply | Threaded
Open this post in threaded view
|

Re: Threads safety in Pharo

Stephane Ducasse-3
In reply to this post by Pharo Smalltalk Users mailing list
>
> Great, I taught some of theses concepts to my students some times ago and we did some fun lab sessions with the Linux scheduler
>
> This was on my todo to turn it into a book chapter for Pharo.
>
> So may be we could work together to write a little booklet on this aspect to get started.
>
>
> I would love to :D

Be my guest.
I can create a booklet in no time.
I once started to collect notes on PharoLimbo/ConcurrencyBasics on github.
Start there!

Stef

>
>
> Stef
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Threads safety in Pharo

Stephane Ducasse-3
In reply to this post by Pharo Smalltalk Users mailing list


On Fri, Aug 4, 2017 at 7:06 PM, Alidra Abdelghani via Pharo-users <[hidden email]> wrote:


---------- Forwarded message ----------
From: Alidra Abdelghani <[hidden email]>
To: Guillermo Polito <[hidden email]>
Cc: Any question about pharo is welcome <[hidden email]>, "Stéphane Ducasse" <[hidden email]>, farid arfi <[hidden email]>
Bcc: 
Date: Fri, 4 Aug 2017 19:06:00 +0200
Subject: Re: [Pharo-users] Threads safety in Pharo
Thanks Guille for your answer,
From what I understand, it is mainly a synchronisation problem. Why cant' existing synchronisation mechanisms (semaphores and mutex) be used?

They can but it means to analyse existing code and produce thread safe one.



 
Abdelghani


On 31 Jul 2017, at 18:44, Guillermo Polito <[hidden email]> wrote:

I believe there is no such a document. It would be however interesting to investigate it a bit deeper. In general, the problem we talk about when we talk about thread safety is the following: Can we run a workspace in a separate thread than a browser and provide correct results? Can we run two browsers concurrently and change code from both of them?

The thread safety problem has several levels I'd say:

 - the kernel of the language (classes, methods, compilation, and so on) cannot safely be modified while other threads are running. Thus, if we compile a new version of a method in a thread, that could break a separate thread that was using that method.

 - there are several libraries using global state. We should investigate if they work correctly when using them concurrently.
    E.g., source code management is not thread safe. This could cause a source code corruption if several methods are modified concurrently.
    Like that, we sh n ould investigate all libraries and see what should be adapted and if there is a need at all.

 - in a third level, general core libraries (collections, networking, files, etc) are not designed to be thread safe and that is natural. For example, most of the time you don't want that your collection is accessed concurrently. For such cases, some libraries could provide some extensions/wrappers/whatsoever to provide some synchronization mechanism. Otherwise it is the responsibility of the user to synchronize usages with semaphores/mutexes.
 

On Mon, Jul 31, 2017 at 1:38 AM, Alidra Abdelghani via Pharo-users <[hidden email]> wrote:


---------- Forwarded message ----------
From: Alidra Abdelghani <[hidden email]>
To: [hidden email]
Cc: "Stéphane Ducasse" <[hidden email]>, farid arfi <[hidden email]>
Bcc: 
Date: Mon, 31 Jul 2017 01:38:58 +0200
Subject: Threads safety in Pharo
Hi,

Somebody once evoked the problem of threads safety in Pharo. With a friend of mine who is expert in formal methods and process scheduling, we would like to have a look on it.
Does anyone knows a good document describing the problem of Pharo with threads safety or at least any document that we can start with?

Thanks in advance,
Abdelghani






--
   
Guille Polito

Research Engineer
French National Center for Scientific Research - http://www.cnrs.fr


Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13



Reply | Threaded
Open this post in threaded view
|

Thread-safe initialization of class state (was Re: Threads safety in Pharo)

monty-3
In reply to this post by Stephane Ducasse-3
Here's a hypothetical broken class method that does lazy initialization of a class inst var:

fileTypes
        fileTypes ifNil: [
                fileTypes := Dictionary new.
                fileTypes
                        at: 'txt' put: 'Text File';
                        at: 'html' put: 'Web Page';
                        at: 'pdf' put: 'Portable Document Format File';
                        at: 'doc' put: 'Microsoft Word Document'].
        ^ fileTypes.

Because the assignment is done first and the initialization is done after with a cascade of interruptable sends of #at:put:, there's a window after the assignment where 'fileTypes' is not nil but also not fully initialized--a race condition.

The fix is simple. Do the initialization before the atomic assignment takes place, so the var is only ever bound to nil or a fully initialized object:

fileTypes
        fileTypes ifNil: [
                fileTypes :=
                        Dictionary new
                                at: 'txt' put: 'Text File';
                                at: 'html' put: 'Web Page';
                                at: 'pdf' put: 'Portable Document Format File';
                                at: 'doc' put: 'Microsoft Word Document';
                                yourself].
        ^ fileTypes.

The fixed code is still vulnerable to duplicate initialization, because the initialization sequence is interruptable and 'fileTypes' is nil during it, but as long as the initialization is cheap enough, has no side effects that restrict how often it can be done, and it's enough that the initialized objects are equal (but not identical), that's OK.

If it's too complex for a single statement, you can use a temp vars or put it in a separate factory method:

fileTypes
        fileTypes ifNil: [
                fileTypes := self newFileTypes].
        ^ fileTypes.

Similar precautions (given how easy) might as well be taken with explicit initialization of class state too. Of course if the object is mutated later (in other methods), then Mutexes or other constructs are needed to guard access. But for immutable class state, ensuring initialization is done before assignment should be enough.

> Sent: Tuesday, August 01, 2017 at 7:36 AM
> From: "Stephane Ducasse" <[hidden email]>
> To: "Any question about pharo is welcome" <[hidden email]>
> Subject: Re: [Pharo-users] Threads safety in Pharo
>
> I would love to have an analysis of assumptions made in some code.
> Because my impression is that the concurrent code is sometimes defined
> knowing the underlying logic of scheduler and this is not good.
> As I said to abdel privately in french it would be great to start from
> my french squeak book (Yes I wrote one long time ago) chapter on
> concurrent programming and turn it into a pharo chapter.
>
> Stef
>
> On Tue, Aug 1, 2017 at 1:31 PM, Ben Coman <[hidden email]> wrote:
> > Not sure I'll have what you're looking for, but to start, do you mean
> > Pharo's green threads or vm native threads?
> > cheers -ben
> >
> > On Mon, Jul 31, 2017 at 7:38 AM, Alidra Abdelghani via Pharo-users
> > <[hidden email]> wrote:
> >>
> >>
> >>
> >> ---------- Forwarded message ----------
> >> From: Alidra Abdelghani <[hidden email]>
> >> To: [hidden email]
> >> Cc: "Stéphane Ducasse" <[hidden email]>, farid arfi
> >> <[hidden email]>
> >> Bcc:
> >> Date: Mon, 31 Jul 2017 01:38:58 +0200
> >> Subject: Threads safety in Pharo
> >> Hi,
> >>
> >> Somebody once evoked the problem of threads safety in Pharo. With a friend
> >> of mine who is expert in formal methods and process scheduling, we would
> >> like to have a look on it.
> >> Does anyone knows a good document describing the problem of Pharo with
> >> threads safety or at least any document that we can start with?
> >>
> >> Thanks in advance,
> >> Abdelghani
> >>
> >>
> >>
> >
>

Reply | Threaded
Open this post in threaded view
|

Re: Thread-safe initialization of class state (was Re: Threads safety in Pharo)

Tim Mackinnon
Interesting, your example was subtle enough that I had to read it a few times to understand the issue. (I was caught up on the ifNil and not the contents).

Actually, thinking about it - isn't the real issue the #ifNil - you want an atomic version.

It strikes me you could wrap the whole concept into something like an AtomicLazyVariable? E.g.

initialise
   fileTypes := AtomicLazyVariable using: [
       Dictionary new
           at: 'txt' put: 'Text File';
           at: 'html' put: 'Web Page';
           yourself ].

Then have something

fileTypes 
     ^fileTypes value

Where you have a critical section in #value?

Tim

Sent from my iPhone

On 11 Aug 2017, at 07:53, monty <[hidden email]> wrote:

Here's a hypothetical broken class method that does lazy initialization of a class inst var:

fileTypes
   fileTypes ifNil: [
       fileTypes := Dictionary new.
       fileTypes
           at: 'txt' put: 'Text File';
           at: 'html' put: 'Web Page';
           at: 'pdf' put: 'Portable Document Format File';
           at: 'doc' put: 'Microsoft Word Document'].
   ^ fileTypes.

Because the assignment is done first and the initialization is done after with a cascade of interruptable sends of #at:put:, there's a window after the assignment where 'fileTypes' is not nil but also not fully initialized--a race condition.

The fix is simple. Do the initialization before the atomic assignment takes place, so the var is only ever bound to nil or a fully initialized object:

fileTypes
   fileTypes ifNil: [
       fileTypes :=
           Dictionary new
               at: 'txt' put: 'Text File';
               at: 'html' put: 'Web Page';
               at: 'pdf' put: 'Portable Document Format File';
               at: 'doc' put: 'Microsoft Word Document';
               yourself].
   ^ fileTypes.

The fixed code is still vulnerable to duplicate initialization, because the initialization sequence is interruptable and 'fileTypes' is nil during it, but as long as the initialization is cheap enough, has no side effects that restrict how often it can be done, and it's enough that the initialized objects are equal (but not identical), that's OK.

If it's too complex for a single statement, you can use a temp vars or put it in a separate factory method:

fileTypes
   fileTypes ifNil: [
       fileTypes := self newFileTypes].
   ^ fileTypes.

Similar precautions (given how easy) might as well be taken with explicit initialization of class state too. Of course if the object is mutated later (in other methods), then Mutexes or other constructs are needed to guard access. But for immutable class state, ensuring initialization is done before assignment should be enough.

Sent: Tuesday, August 01, 2017 at 7:36 AM
From: "Stephane Ducasse" <[hidden email]>
To: "Any question about pharo is welcome" <[hidden email]>
Subject: Re: [Pharo-users] Threads safety in Pharo

I would love to have an analysis of assumptions made in some code.
Because my impression is that the concurrent code is sometimes defined
knowing the underlying logic of scheduler and this is not good.
As I said to abdel privately in french it would be great to start from
my french squeak book (Yes I wrote one long time ago) chapter on
concurrent programming and turn it into a pharo chapter.

Stef

On Tue, Aug 1, 2017 at 1:31 PM, Ben Coman <[hidden email]> wrote:
Not sure I'll have what you're looking for, but to start, do you mean
Pharo's green threads or vm native threads?
cheers -ben

On Mon, Jul 31, 2017 at 7:38 AM, Alidra Abdelghani via Pharo-users
<[hidden email]> wrote:



---------- Forwarded message ----------
From: Alidra Abdelghani <[hidden email]>
To: [hidden email]
Cc: "Stéphane Ducasse" <[hidden email]>, farid arfi
<[hidden email]>
Bcc:
Date: Mon, 31 Jul 2017 01:38:58 +0200
Subject: Threads safety in Pharo
Hi,

Somebody once evoked the problem of threads safety in Pharo. With a friend
of mine who is expert in formal methods and process scheduling, we would
like to have a look on it.
Does anyone knows a good document describing the problem of Pharo with
threads safety or at least any document that we can start with?

Thanks in advance,
Abdelghani






Reply | Threaded
Open this post in threaded view
|

Re: Thread-safe initialization of class state (was Re: Threads safety in Pharo)

Denis Kudriashov
In reply to this post by monty-3
What package you explore? I not find fileTypes method in Pharo 7. 

2017-08-11 8:53 GMT+02:00 monty <[hidden email]>:
Here's a hypothetical broken class method that does lazy initialization of a class inst var:

fileTypes
        fileTypes ifNil: [
                fileTypes := Dictionary new.
                fileTypes
                        at: 'txt' put: 'Text File';
                        at: 'html' put: 'Web Page';
                        at: 'pdf' put: 'Portable Document Format File';
                        at: 'doc' put: 'Microsoft Word Document'].
        ^ fileTypes.

Because the assignment is done first and the initialization is done after with a cascade of interruptable sends of #at:put:, there's a window after the assignment where 'fileTypes' is not nil but also not fully initialized--a race condition.

The fix is simple. Do the initialization before the atomic assignment takes place, so the var is only ever bound to nil or a fully initialized object:

fileTypes
        fileTypes ifNil: [
                fileTypes :=
                        Dictionary new
                                at: 'txt' put: 'Text File';
                                at: 'html' put: 'Web Page';
                                at: 'pdf' put: 'Portable Document Format File';
                                at: 'doc' put: 'Microsoft Word Document';
                                yourself].
        ^ fileTypes.

The fixed code is still vulnerable to duplicate initialization, because the initialization sequence is interruptable and 'fileTypes' is nil during it, but as long as the initialization is cheap enough, has no side effects that restrict how often it can be done, and it's enough that the initialized objects are equal (but not identical), that's OK.

If it's too complex for a single statement, you can use a temp vars or put it in a separate factory method:

fileTypes
        fileTypes ifNil: [
                fileTypes := self newFileTypes].
        ^ fileTypes.

Similar precautions (given how easy) might as well be taken with explicit initialization of class state too. Of course if the object is mutated later (in other methods), then Mutexes or other constructs are needed to guard access. But for immutable class state, ensuring initialization is done before assignment should be enough.

> Sent: Tuesday, August 01, 2017 at 7:36 AM
> From: "Stephane Ducasse" <[hidden email]>
> To: "Any question about pharo is welcome" <[hidden email]>
> Subject: Re: [Pharo-users] Threads safety in Pharo
>
> I would love to have an analysis of assumptions made in some code.
> Because my impression is that the concurrent code is sometimes defined
> knowing the underlying logic of scheduler and this is not good.
> As I said to abdel privately in french it would be great to start from
> my french squeak book (Yes I wrote one long time ago) chapter on
> concurrent programming and turn it into a pharo chapter.
>
> Stef
>
> On Tue, Aug 1, 2017 at 1:31 PM, Ben Coman <[hidden email]> wrote:
> > Not sure I'll have what you're looking for, but to start, do you mean
> > Pharo's green threads or vm native threads?
> > cheers -ben
> >
> > On Mon, Jul 31, 2017 at 7:38 AM, Alidra Abdelghani via Pharo-users
> > <[hidden email]> wrote:
> >>
> >>
> >>
> >> ---------- Forwarded message ----------
> >> From: Alidra Abdelghani <[hidden email]>
> >> To: [hidden email]
> >> Cc: "Stéphane Ducasse" <[hidden email]>, farid arfi
> >> <[hidden email]>
> >> Bcc:
> >> Date: Mon, 31 Jul 2017 01:38:58 +0200
> >> Subject: Threads safety in Pharo
> >> Hi,
> >>
> >> Somebody once evoked the problem of threads safety in Pharo. With a friend
> >> of mine who is expert in formal methods and process scheduling, we would
> >> like to have a look on it.
> >> Does anyone knows a good document describing the problem of Pharo with
> >> threads safety or at least any document that we can start with?
> >>
> >> Thanks in advance,
> >> Abdelghani
> >>
> >>
> >>
> >
>


Reply | Threaded
Open this post in threaded view
|

Re: Thread-safe initialization of class state (was Re: Threads safety in Pharo)

monty-3
In reply to this post by Tim Mackinnon
> Sent: Friday, August 11, 2017 at 5:51 AM
> From: "Tim Mackinnon" <[hidden email]>
> To: "Any question about pharo is welcome" <[hidden email]>
> Subject: Re: [Pharo-users] Thread-safe initialization of class state (was Re: Threads safety in Pharo)
>
> Interesting, your example was subtle enough that I had to read it a few times to understand the issue. (I was caught up on the ifNil and not the contents).

That's the problem. They're close enough that someone could mistakenly refactor the correct code into the incorrect code--and that code would be perfectly fine for lazy initialization of instance state in a non-concurrent class. That's why I always insert a small comment before explaining why it was written that way.
 

> Actually, thinking about it - isn't the real issue the #ifNil - you want an atomic version.
>  
> It strikes me you could wrap the whole concept into something like an AtomicLazyVariable? E.g.
>  
> initialise
>    fileTypes := AtomicLazyVariable using: [
>
>        Dictionary new
>            at: 'txt' put: 'Text File';
>            at: 'html' put: 'Web Page';
>            yourself ].
>  
> Then have something
>  
> fileTypes
>      ^fileTypes value
>  
> Where you have a critical section in #value?
>  
> Tim
>  
> Sent from my iPhone
> On 11 Aug 2017, at 07:53, monty <[hidden email][mailto:[hidden email]]> wrote:
>  
> Here's a hypothetical broken class method that does lazy initialization of a class inst var:
>
> fileTypes
>    fileTypes ifNil: [
>        fileTypes := Dictionary new.
>        fileTypes
>            at: 'txt' put: 'Text File';
>            at: 'html' put: 'Web Page';
>            at: 'pdf' put: 'Portable Document Format File';
>            at: 'doc' put: 'Microsoft Word Document'].
>    ^ fileTypes.
>
> Because the assignment is done first and the initialization is done after with a cascade of interruptable sends of #at:put:, there's a window after the assignment where 'fileTypes' is not nil but also not fully initialized--a race condition.
>
> The fix is simple. Do the initialization before the atomic assignment takes place, so the var is only ever bound to nil or a fully initialized object:
>
> fileTypes
>    fileTypes ifNil: [
>        fileTypes :=
>            Dictionary new
>                at: 'txt' put: 'Text File';
>                at: 'html' put: 'Web Page';
>                at: 'pdf' put: 'Portable Document Format File';
>                at: 'doc' put: 'Microsoft Word Document';
>                yourself].
>    ^ fileTypes.
>
> The fixed code is still vulnerable to duplicate initialization, because the initialization sequence is interruptable and 'fileTypes' is nil during it, but as long as the initialization is cheap enough, has no side effects that restrict how often it can be done, and it's enough that the initialized objects are equal (but not identical), that's OK.
>
> If it's too complex for a single statement, you can use a temp vars or put it in a separate factory method:
>
> fileTypes
>    fileTypes ifNil: [
>        fileTypes := self newFileTypes].
>    ^ fileTypes.
>
> Similar precautions (given how easy) might as well be taken with explicit initialization of class state too. Of course if the object is mutated later (in other methods), then Mutexes or other constructs are needed to guard access. But for immutable class state, ensuring initialization is done before assignment should be enough.
>  Sent: Tuesday, August 01, 2017 at 7:36 AMFrom: "Stephane Ducasse" <[hidden email][mailto:[hidden email]]>To: "Any question about pharo is welcome" <[hidden email][mailto:[hidden email]]>Subject: Re: [Pharo-users] Threads safety in Pharo I would love to have an analysis of assumptions made in some code.Because my impression is that the concurrent code is sometimes definedknowing the underlying logic of scheduler and this is not good.As I said to abdel privately in french it would be great to start frommy french squeak book (Yes I wrote one long time ago) chapter onconcurrent programming and turn it into a pharo chapter. Stef On Tue, Aug 1, 2017 at 1:31 PM, Ben Coman <[hidden email][mailto:[hidden email]]> wrote:Not sure I'll have what you're looking for, but to start, do you meanPharo's green threads or vm native threads?cheers -ben On Mon, Jul 31, 2017 at 7:38 AM, Alidra Abdelghani via Pharo-users<[hidden email][mailto:[hidden email]]> wrote:   ---------- Forwarded message ----------From: Alidra Abdelghani <[hidden email][mailto:[hidden email]]>To: [hidden email][mailto:[hidden email]]Cc: "Stéphane Ducasse" <[hidden email][mailto:[hidden email]]>, farid arfi<[hidden email][mailto:[hidden email]]>Bcc:Date: Mon, 31 Jul 2017 01:38:58 +0200Subject: Threads safety in PharoHi, Somebody once evoked the problem of threads safety in Pharo. With a friendof mine who is expert in formal methods and process scheduling, we wouldlike to have a look on it.Does anyone knows a good document describing the problem of Pharo withthreads safety or at least any document that we can start with? Thanks in advance,Abdelghani

Reply | Threaded
Open this post in threaded view
|

Re: Thread-safe initialization of class state (was Re: Threads safety in Pharo)

monty-3
In reply to this post by Denis Kudriashov
> Sent: Friday, August 11, 2017 at 6:36 AM
> From: "Denis Kudriashov" <[hidden email]>
> To: "Any question about pharo is welcome" <[hidden email]>
> Subject: Re: [Pharo-users] Thread-safe initialization of class state (was Re: Threads safety in Pharo)
>
> What package you explore? I not find fileTypes method in Pharo 7.

Like I said, it's a hypothetical example. But I'm sure you could find similar examples of unsafe class state initialization in the image and in popular libraries.

> 2017-08-11 8:53 GMT+02:00 monty <[hidden email][mailto:[hidden email]]>:Here's a hypothetical broken class method that does lazy initialization of a class inst var:
>
> fileTypes
>         fileTypes ifNil: [
>                 fileTypes := Dictionary new.
>                 fileTypes
>                         at: 'txt' put: 'Text File';
>                         at: 'html' put: 'Web Page';
>                         at: 'pdf' put: 'Portable Document Format File';
>                         at: 'doc' put: 'Microsoft Word Document'].
>         ^ fileTypes.
>
> Because the assignment is done first and the initialization is done after with a cascade of interruptable sends of #at:put:, there's a window after the assignment where 'fileTypes' is not nil but also not fully initialized--a race condition.
>
> The fix is simple. Do the initialization before the atomic assignment takes place, so the var is only ever bound to nil or a fully initialized object:
>
> fileTypes
>         fileTypes ifNil: [
>                 fileTypes :=
>                         Dictionary new
>                                 at: 'txt' put: 'Text File';
>                                 at: 'html' put: 'Web Page';
>                                 at: 'pdf' put: 'Portable Document Format File';
>                                 at: 'doc' put: 'Microsoft Word Document';
>                                 yourself].
>         ^ fileTypes.
>
> The fixed code is still vulnerable to duplicate initialization, because the initialization sequence is interruptable and 'fileTypes' is nil during it, but as long as the initialization is cheap enough, has no side effects that restrict how often it can be done, and it's enough that the initialized objects are equal (but not identical), that's OK.
>
> If it's too complex for a single statement, you can use a temp vars or put it in a separate factory method:
>
> fileTypes
>         fileTypes ifNil: [
>                 fileTypes := self newFileTypes].
>         ^ fileTypes.
>
> Similar precautions (given how easy) might as well be taken with explicit initialization of class state too. Of course if the object is mutated later (in other methods), then Mutexes or other constructs are needed to guard access. But for immutable class state, ensuring initialization is done before assignment should be enough.
>
> > Sent: Tuesday, August 01, 2017 at 7:36 AM
> > From: "Stephane Ducasse" <[hidden email][mailto:[hidden email]]>
> > To: "Any question about pharo is welcome" <[hidden email][mailto:[hidden email]]>
> > Subject: Re: [Pharo-users] Threads safety in Pharo
> >
> > I would love to have an analysis of assumptions made in some code.
> > Because my impression is that the concurrent code is sometimes defined
> > knowing the underlying logic of scheduler and this is not good.
> > As I said to abdel privately in french it would be great to start from
> > my french squeak book (Yes I wrote one long time ago) chapter on
> > concurrent programming and turn it into a pharo chapter.
> >
> > Stef
> >
> > On Tue, Aug 1, 2017 at 1:31 PM, Ben Coman <[hidden email][mailto:[hidden email]]> wrote:
> > > Not sure I'll have what you're looking for, but to start, do you mean
> > > Pharo's green threads or vm native threads?
> > > cheers -ben
> > >
> > > On Mon, Jul 31, 2017 at 7:38 AM, Alidra Abdelghani via Pharo-users
> > > <[hidden email][mailto:[hidden email]]> wrote:
> > >>
> > >>
> > >>
> > >> ---------- Forwarded message ----------
> > >> From: Alidra Abdelghani <[hidden email][mailto:[hidden email]]>
> > >> To: [hidden email][mailto:[hidden email]]
> > >> Cc: "Stéphane Ducasse" <[hidden email][mailto:[hidden email]]>, farid arfi
> > >> <[hidden email][mailto:[hidden email]]>
> > >> Bcc:
> > >> Date: Mon, 31 Jul 2017 01:38:58 +0200
> > >> Subject: Threads safety in Pharo
> > >> Hi,
> > >>
> > >> Somebody once evoked the problem of threads safety in Pharo. With a friend
> > >> of mine who is expert in formal methods and process scheduling, we would
> > >> like to have a look on it.
> > >> Does anyone knows a good document describing the problem of Pharo with
> > >> threads safety or at least any document that we can start with?
> > >>
> > >> Thanks in advance,
> > >> Abdelghani
> > >>
> > >>
> > >>
> > >
> >
>

Reply | Threaded
Open this post in threaded view
|

Re: Thread-safe initialization of class state (was Re: Threads safety in Pharo)

gcotelli
Suerly we can add a Critics Rule trying to detect the invalid pattern.

On Fri, Aug 11, 2017 at 10:39 AM, monty <[hidden email]> wrote:
> Sent: Friday, August 11, 2017 at 6:36 AM
> From: "Denis Kudriashov" <[hidden email]>
> To: "Any question about pharo is welcome" <[hidden email]>
> Subject: Re: [Pharo-users] Thread-safe initialization of class state (was Re: Threads safety in Pharo)
>
> What package you explore? I not find fileTypes method in Pharo 7.

Like I said, it's a hypothetical example. But I'm sure you could find similar examples of unsafe class state initialization in the image and in popular libraries.

> 2017-08-11 8:53 GMT+02:00 monty <[hidden email][mailto:[hidden email]]>:Here's a hypothetical broken class method that does lazy initialization of a class inst var:
>
> fileTypes
>         fileTypes ifNil: [
>                 fileTypes := Dictionary new.
>                 fileTypes
>                         at: 'txt' put: 'Text File';
>                         at: 'html' put: 'Web Page';
>                         at: 'pdf' put: 'Portable Document Format File';
>                         at: 'doc' put: 'Microsoft Word Document'].
>         ^ fileTypes.
>
> Because the assignment is done first and the initialization is done after with a cascade of interruptable sends of #at:put:, there's a window after the assignment where 'fileTypes' is not nil but also not fully initialized--a race condition.
>
> The fix is simple. Do the initialization before the atomic assignment takes place, so the var is only ever bound to nil or a fully initialized object:
>
> fileTypes
>         fileTypes ifNil: [
>                 fileTypes :=
>                         Dictionary new
>                                 at: 'txt' put: 'Text File';
>                                 at: 'html' put: 'Web Page';
>                                 at: 'pdf' put: 'Portable Document Format File';
>                                 at: 'doc' put: 'Microsoft Word Document';
>                                 yourself].
>         ^ fileTypes.
>
> The fixed code is still vulnerable to duplicate initialization, because the initialization sequence is interruptable and 'fileTypes' is nil during it, but as long as the initialization is cheap enough, has no side effects that restrict how often it can be done, and it's enough that the initialized objects are equal (but not identical), that's OK.
>
> If it's too complex for a single statement, you can use a temp vars or put it in a separate factory method:
>
> fileTypes
>         fileTypes ifNil: [
>                 fileTypes := self newFileTypes].
>         ^ fileTypes.
>
> Similar precautions (given how easy) might as well be taken with explicit initialization of class state too. Of course if the object is mutated later (in other methods), then Mutexes or other constructs are needed to guard access. But for immutable class state, ensuring initialization is done before assignment should be enough.
>
> > Sent: Tuesday, August 01, 2017 at 7:36 AM
> > From: "Stephane Ducasse" <[hidden email][mailto:[hidden email]]>
> > To: "Any question about pharo is welcome" <[hidden email][mailto:[hidden email]]>
> > Subject: Re: [Pharo-users] Threads safety in Pharo
> >
> > I would love to have an analysis of assumptions made in some code.
> > Because my impression is that the concurrent code is sometimes defined
> > knowing the underlying logic of scheduler and this is not good.
> > As I said to abdel privately in french it would be great to start from
> > my french squeak book (Yes I wrote one long time ago) chapter on
> > concurrent programming and turn it into a pharo chapter.
> >
> > Stef
> >
> > On Tue, Aug 1, 2017 at 1:31 PM, Ben Coman <[hidden email][mailto:[hidden email]]> wrote:
> > > Not sure I'll have what you're looking for, but to start, do you mean
> > > Pharo's green threads or vm native threads?
> > > cheers -ben
> > >
> > > On Mon, Jul 31, 2017 at 7:38 AM, Alidra Abdelghani via Pharo-users
> > > <[hidden email][mailto:[hidden email]]> wrote:
> > >>
> > >>
> > >>
> > >> ---------- Forwarded message ----------
> > >> From: Alidra Abdelghani <[hidden email][mailto:[hidden email]]>
> > >> To: [hidden email][mailto:[hidden email]]
> > >> Cc: "Stéphane Ducasse" <[hidden email][mailto:[hidden email]]>, farid arfi
> > >> <[hidden email][mailto:[hidden email]]>
> > >> Bcc:
> > >> Date: Mon, 31 Jul 2017 01:38:58 +0200
> > >> Subject: Threads safety in Pharo
> > >> Hi,
> > >>
> > >> Somebody once evoked the problem of threads safety in Pharo. With a friend
> > >> of mine who is expert in formal methods and process scheduling, we would
> > >> like to have a look on it.
> > >> Does anyone knows a good document describing the problem of Pharo with
> > >> threads safety or at least any document that we can start with?
> > >>
> > >> Thanks in advance,
> > >> Abdelghani
> > >>
> > >>
> > >>
> > >
> >
>


Reply | Threaded
Open this post in threaded view
|

Re: Thread-safe initialization of class state (was Re: Threads safety in Pharo)

Tim Mackinnon
I think that thread safety comes with a penalty - so the overarching question is are we trying to create thread safe browsers (or does a thread safe penalty even impact browsing as maybe it's negligible?)

Monty - maybe you could measure the impact if you are interested in this? If it's not much and we just need to encapsulate it with some mechanism (potentially having thread safety critics we can enable) then it's possibly something Calypso can embrace.

This said, multi user editing is not something that has really taken off. Jason at VW did lots with Wolfpack, and it wasn't an idea that gained lots of traction a few years ago (sadly).

It's tricky as we have to spend our engineering resources wisely.

Tim

Sent from my iPhone

On 11 Aug 2017, at 19:41, Gabriel Cotelli <[hidden email]> wrote:

Suerly we can add a Critics Rule trying to detect the invalid pattern.

On Fri, Aug 11, 2017 at 10:39 AM, monty <[hidden email]> wrote:
> Sent: Friday, August 11, 2017 at 6:36 AM
> From: "Denis Kudriashov" <[hidden email]>
> To: "Any question about pharo is welcome" <[hidden email]>
> Subject: Re: [Pharo-users] Thread-safe initialization of class state (was Re: Threads safety in Pharo)
>
> What package you explore? I not find fileTypes method in Pharo 7.

Like I said, it's a hypothetical example. But I'm sure you could find similar examples of unsafe class state initialization in the image and in popular libraries.

> 2017-08-11 8:53 GMT+02:00 monty <[hidden email][mailto:[hidden email]]>:Here's a hypothetical broken class method that does lazy initialization of a class inst var:
>
> fileTypes
>         fileTypes ifNil: [
>                 fileTypes := Dictionary new.
>                 fileTypes
>                         at: 'txt' put: 'Text File';
>                         at: 'html' put: 'Web Page';
>                         at: 'pdf' put: 'Portable Document Format File';
>                         at: 'doc' put: 'Microsoft Word Document'].
>         ^ fileTypes.
>
> Because the assignment is done first and the initialization is done after with a cascade of interruptable sends of #at:put:, there's a window after the assignment where 'fileTypes' is not nil but also not fully initialized--a race condition.
>
> The fix is simple. Do the initialization before the atomic assignment takes place, so the var is only ever bound to nil or a fully initialized object:
>
> fileTypes
>         fileTypes ifNil: [
>                 fileTypes :=
>                         Dictionary new
>                                 at: 'txt' put: 'Text File';
>                                 at: 'html' put: 'Web Page';
>                                 at: 'pdf' put: 'Portable Document Format File';
>                                 at: 'doc' put: 'Microsoft Word Document';
>                                 yourself].
>         ^ fileTypes.
>
> The fixed code is still vulnerable to duplicate initialization, because the initialization sequence is interruptable and 'fileTypes' is nil during it, but as long as the initialization is cheap enough, has no side effects that restrict how often it can be done, and it's enough that the initialized objects are equal (but not identical), that's OK.
>
> If it's too complex for a single statement, you can use a temp vars or put it in a separate factory method:
>
> fileTypes
>         fileTypes ifNil: [
>                 fileTypes := self newFileTypes].
>         ^ fileTypes.
>
> Similar precautions (given how easy) might as well be taken with explicit initialization of class state too. Of course if the object is mutated later (in other methods), then Mutexes or other constructs are needed to guard access. But for immutable class state, ensuring initialization is done before assignment should be enough.
>
> > Sent: Tuesday, August 01, 2017 at 7:36 AM
> > From: "Stephane Ducasse" <[hidden email][mailto:[hidden email]]>
> > To: "Any question about pharo is welcome" <[hidden email][mailto:[hidden email]]>
> > Subject: Re: [Pharo-users] Threads safety in Pharo
> >
> > I would love to have an analysis of assumptions made in some code.
> > Because my impression is that the concurrent code is sometimes defined
> > knowing the underlying logic of scheduler and this is not good.
> > As I said to abdel privately in french it would be great to start from
> > my french squeak book (Yes I wrote one long time ago) chapter on
> > concurrent programming and turn it into a pharo chapter.
> >
> > Stef
> >
> > On Tue, Aug 1, 2017 at 1:31 PM, Ben Coman <[hidden email][mailto:[hidden email]]> wrote:
> > > Not sure I'll have what you're looking for, but to start, do you mean
> > > Pharo's green threads or vm native threads?
> > > cheers -ben
> > >
> > > On Mon, Jul 31, 2017 at 7:38 AM, Alidra Abdelghani via Pharo-users
> > > <[hidden email][mailto:[hidden email]]> wrote:
> > >>
> > >>
> > >>
> > >> ---------- Forwarded message ----------
> > >> From: Alidra Abdelghani <[hidden email][mailto:[hidden email]]>
> > >> To: [hidden email][mailto:[hidden email]]
> > >> Cc: "Stéphane Ducasse" <[hidden email][mailto:[hidden email]]>, farid arfi
> > >> <[hidden email][mailto:[hidden email]]>
> > >> Bcc:
> > >> Date: Mon, 31 Jul 2017 01:38:58 +0200
> > >> Subject: Threads safety in Pharo
> > >> Hi,
> > >>
> > >> Somebody once evoked the problem of threads safety in Pharo. With a friend
> > >> of mine who is expert in formal methods and process scheduling, we would
> > >> like to have a look on it.
> > >> Does anyone knows a good document describing the problem of Pharo with
> > >> threads safety or at least any document that we can start with?
> > >>
> > >> Thanks in advance,
> > >> Abdelghani
> > >>
> > >>
> > >>
> > >
> >
>