Smalltalk › Pharo › Pharo Smalltalk Users

Threads safety in Pharo

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

16 messages Options

Pharo Smalltalk Users mailing list

Threads safety in Pharo

Hi,

Somebody once evoked the problem of threads safety in Pharo. With a friend of mine who is expert in formal methods and process scheduling, we would like to have a look on it.
Does anyone knows a good document describing the problem of Pharo with threads safety or at least any document that we can start with?

Thanks in advance,
Abdelghani

Guillermo Polito

Re: Threads safety in Pharo

I believe there is no such a document. It would be however interesting to investigate it a bit deeper. In general, the problem we talk about when we talk about thread safety is the following: Can we run a workspace in a separate thread than a browser and provide correct results? Can we run two browsers concurrently and change code from both of them?

The thread safety problem has several levels I'd say:

- the kernel of the language (classes, methods, compilation, and so on) cannot safely be modified while other threads are running. Thus, if we compile a new version of a method in a thread, that could break a separate thread that was using that method.

- there are several libraries using global state. We should investigate if they work correctly when using them concurrently.

E.g., source code management is not thread safe. This could cause a source code corruption if several methods are modified concurrently.

Like that, we should investigate all libraries and see what should be adapted and if there is a need at all.

- in a third level, general core libraries (collections, networking, files, etc) are not designed to be thread safe and that is natural. For example, most of the time you don't want that your collection is accessed concurrently. For such cases, some libraries could provide some extensions/wrappers/whatsoever to provide some synchronization mechanism. Otherwise it is the responsibility of the user to synchronize usages with semaphores/mutexes.

On Mon, Jul 31, 2017 at 1:38 AM, Alidra Abdelghani via Pharo-users <[hidden email]> wrote:

---------- Forwarded message ----------
From: Alidra Abdelghani <[hidden email]>
To: [hidden email]
Cc: "Stéphane Ducasse" <[hidden email]>, farid arfi <[hidden email]>
Bcc:
Date: Mon, 31 Jul 2017 01:38:58 +0200
Subject: Threads safety in Pharo
Hi,

Somebody once evoked the problem of threads safety in Pharo. With a friend of mine who is expert in formal methods and process scheduling, we would like to have a look on it.
Does anyone knows a good document describing the problem of Pharo with threads safety or at least any document that we can start with?

Thanks in advance,
Abdelghani

Guille Polito

Research Engineer

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: +33 06 52 70 66 13

Ben Coman

Re: Threads safety in Pharo

In reply to this post by Pharo Smalltalk Users mailing list

Not sure I'll have what you're looking for, but to start, do you mean Pharo's green threads or vm native threads?

cheers -ben

On Mon, Jul 31, 2017 at 7:38 AM, Alidra Abdelghani via Pharo-users <[hidden email]> wrote:

---------- Forwarded message ----------
From: Alidra Abdelghani <[hidden email]>
To: [hidden email]
Cc: "Stéphane Ducasse" <[hidden email]>, farid arfi <[hidden email]>
Bcc:
Date: Mon, 31 Jul 2017 01:38:58 +0200
Subject: Threads safety in Pharo
Hi,

Somebody once evoked the problem of threads safety in Pharo. With a friend of mine who is expert in formal methods and process scheduling, we would like to have a look on it.
Does anyone knows a good document describing the problem of Pharo with threads safety or at least any document that we can start with?

Thanks in advance,
Abdelghani

Stephane Ducasse-3

Re: Threads safety in Pharo

I would love to have an analysis of assumptions made in some code.
Because my impression is that the concurrent code is sometimes defined
knowing the underlying logic of scheduler and this is not good.
As I said to abdel privately in french it would be great to start from
my french squeak book (Yes I wrote one long time ago) chapter on
concurrent programming and turn it into a pharo chapter.

Stef

On Tue, Aug 1, 2017 at 1:31 PM, Ben Coman <[hidden email]> wrote:

> Not sure I'll have what you're looking for, but to start, do you mean
> Pharo's green threads or vm native threads?
> cheers -ben
>
> On Mon, Jul 31, 2017 at 7:38 AM, Alidra Abdelghani via Pharo-users
> <[hidden email]> wrote:
>>
>>
>>
>> ---------- Forwarded message ----------
>> From: Alidra Abdelghani <[hidden email]>
>> To: [hidden email]
>> Cc: "Stéphane Ducasse" <[hidden email]>, farid arfi
>> <[hidden email]>
>> Bcc:
>> Date: Mon, 31 Jul 2017 01:38:58 +0200
>> Subject: Threads safety in Pharo
>> Hi,
>>
>> Somebody once evoked the problem of threads safety in Pharo. With a friend
>> of mine who is expert in formal methods and process scheduling, we would
>> like to have a look on it.
>> Does anyone knows a good document describing the problem of Pharo with
>> threads safety or at least any document that we can start with?
>>
>> Thanks in advance,
>> Abdelghani
>>
>>
>>
>

Squeak_09.pdf (772K) Download Attachment

Pharo Smalltalk Users mailing list

Re: Threads safety in Pharo

In reply to this post by Ben Coman

Hi Ben,

On 01 Aug 2017, at 13:31, Ben Coman <[hidden email]> wrote:

Not sure I'll have what you're looking for, but to start, do you mean Pharo's green threads or vm native threads?

I believe this is about Pharo threads.

I am not quite sure of what the vm native threads are or if there is a safety problem with them. But still, it could be interesting to investigate :)

Thanks

Abdelghani

cheers -ben

On Mon, Jul 31, 2017 at 7:38 AM, Alidra Abdelghani via Pharo-users <[hidden email]> wrote:

---------- Forwarded message ----------
From: Alidra Abdelghani <[hidden email]>
To: [hidden email]
Cc: "Stéphane Ducasse" <[hidden email]>, farid arfi <[hidden email]>
Bcc:
Date: Mon, 31 Jul 2017 01:38:58 +0200
Subject: Threads safety in Pharo
Hi,

Somebody once evoked the problem of threads safety in Pharo. With a friend of mine who is expert in formal methods and process scheduling, we would like to have a look on it.
Does anyone knows a good document describing the problem of Pharo with threads safety or at least any document that we can start with?

Thanks in advance,
Abdelghani

Pharo Smalltalk Users mailing list

Re: Threads safety in Pharo

In reply to this post by Guillermo Polito

On 01 Aug 2017, at 09:36, Stéphane Ducasse <[hidden email]> wrote:

To complement.

What would be fun is to investigate the implementation of ProcessScheduler, Process and check the implementation.

Abdel there is a nice presentation of the processor and process in the Squeak french book.

Great, I taught some of theses concepts to my students some times ago and we did some fun lab sessions with the Linux scheduler

This was on my todo to turn it into a book chapter for Pharo.

So may be we could work together to write a little booklet on this aspect to get started.

I would love to :D

Stef

Pharo Smalltalk Users mailing list

Re: Threads safety in Pharo

In reply to this post by Guillermo Polito

Thanks Guille for your answer,

From what I understand, it is mainly a synchronisation problem. Why cant' existing synchronisation mechanisms (semaphores and mutex) be used?

Abdelghani

On 31 Jul 2017, at 18:44, Guillermo Polito <[hidden email]> wrote:

I believe there is no such a document. It would be however interesting to investigate it a bit deeper. In general, the problem we talk about when we talk about thread safety is the following: Can we run a workspace in a separate thread than a browser and provide correct results? Can we run two browsers concurrently and change code from both of them?

The thread safety problem has several levels I'd say:

- the kernel of the language (classes, methods, compilation, and so on) cannot safely be modified while other threads are running. Thus, if we compile a new version of a method in a thread, that could break a separate thread that was using that method.

- there are several libraries using global state. We should investigate if they work correctly when using them concurrently.
E.g., source code management is not thread safe. This could cause a source code corruption if several methods are modified concurrently.
Like that, we sh n ould investigate all libraries and see what should be adapted and if there is a need at all.

- in a third level, general core libraries (collections, networking, files, etc) are not designed to be thread safe and that is natural. For example, most of the time you don't want that your collection is accessed concurrently. For such cases, some libraries could provide some extensions/wrappers/whatsoever to provide some synchronization mechanism. Otherwise it is the responsibility of the user to synchronize usages with semaphores/mutexes.

On Mon, Jul 31, 2017 at 1:38 AM, Alidra Abdelghani via Pharo-users <[hidden email]> wrote:

---------- Forwarded message ----------
From: Alidra Abdelghani <[hidden email]>
To: [hidden email]
Cc: "Stéphane Ducasse" <[hidden email]>, farid arfi <[hidden email]>
Bcc:
Date: Mon, 31 Jul 2017 01:38:58 +0200
Subject: Threads safety in Pharo
Hi,

Somebody once evoked the problem of threads safety in Pharo. With a friend of mine who is expert in formal methods and process scheduling, we would like to have a look on it.
Does anyone knows a good document describing the problem of Pharo with threads safety or at least any document that we can start with?

Thanks in advance,
Abdelghani

--

Guille Polito

Research Engineer
French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io
Phone: +33 06 52 70 66 13

Stephane Ducasse-3

Re: Threads safety in Pharo

In reply to this post by Pharo Smalltalk Users mailing list

>
> Great, I taught some of theses concepts to my students some times ago and we did some fun lab sessions with the Linux scheduler
>
> This was on my todo to turn it into a book chapter for Pharo.
>
> So may be we could work together to write a little booklet on this aspect to get started.
>
>
> I would love to :D

Be my guest.
I can create a booklet in no time.
I once started to collect notes on PharoLimbo/ConcurrencyBasics on github.
Start there!

Stef

>
>
> Stef
>
>
>

Stephane Ducasse-3

Re: Threads safety in Pharo

In reply to this post by Pharo Smalltalk Users mailing list

On Fri, Aug 4, 2017 at 7:06 PM, Alidra Abdelghani via Pharo-users <[hidden email]> wrote:

---------- Forwarded message ----------
From: Alidra Abdelghani <[hidden email]>
To: Guillermo Polito <[hidden email]>
Cc: Any question about pharo is welcome <[hidden email]>, "Stéphane Ducasse" <[hidden email]>, farid arfi <[hidden email]>
Bcc:
Date: Fri, 4 Aug 2017 19:06:00 +0200
Subject: Re: [Pharo-users] Threads safety in Pharo
Thanks Guille for your answer,
From what I understand, it is mainly a synchronisation problem. Why cant' existing synchronisation mechanisms (semaphores and mutex) be used?

They can but it means to analyse existing code and produce thread safe one.

Abdelghani

On 31 Jul 2017, at 18:44, Guillermo Polito <[hidden email]> wrote:

I believe there is no such a document. It would be however interesting to investigate it a bit deeper. In general, the problem we talk about when we talk about thread safety is the following: Can we run a workspace in a separate thread than a browser and provide correct results? Can we run two browsers concurrently and change code from both of them?

The thread safety problem has several levels I'd say:

- the kernel of the language (classes, methods, compilation, and so on) cannot safely be modified while other threads are running. Thus, if we compile a new version of a method in a thread, that could break a separate thread that was using that method.

- there are several libraries using global state. We should investigate if they work correctly when using them concurrently.
E.g., source code management is not thread safe. This could cause a source code corruption if several methods are modified concurrently.
Like that, we sh n ould investigate all libraries and see what should be adapted and if there is a need at all.

- in a third level, general core libraries (collections, networking, files, etc) are not designed to be thread safe and that is natural. For example, most of the time you don't want that your collection is accessed concurrently. For such cases, some libraries could provide some extensions/wrappers/whatsoever to provide some synchronization mechanism. Otherwise it is the responsibility of the user to synchronize usages with semaphores/mutexes.

On Mon, Jul 31, 2017 at 1:38 AM, Alidra Abdelghani via Pharo-users <[hidden email]> wrote:

---------- Forwarded message ----------
From: Alidra Abdelghani <[hidden email]>
To: [hidden email]
Cc: "Stéphane Ducasse" <[hidden email]>, farid arfi <[hidden email]>
Bcc:
Date: Mon, 31 Jul 2017 01:38:58 +0200
Subject: Threads safety in Pharo
Hi,

Somebody once evoked the problem of threads safety in Pharo. With a friend of mine who is expert in formal methods and process scheduling, we would like to have a look on it.
Does anyone knows a good document describing the problem of Pharo with threads safety or at least any document that we can start with?

Thanks in advance,
Abdelghani

--

Guille Polito

Research Engineer
French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io
Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13

monty-3

Thread-safe initialization of class state (was Re: Threads safety in Pharo)

In reply to this post by Stephane Ducasse-3

Here's a hypothetical broken class method that does lazy initialization of a class inst var:

fileTypes
fileTypes ifNil: [
fileTypes := Dictionary new.
fileTypes
at: 'txt' put: 'Text File';
at: 'html' put: 'Web Page';
at: 'pdf' put: 'Portable Document Format File';
at: 'doc' put: 'Microsoft Word Document'].
^ fileTypes.

Because the assignment is done first and the initialization is done after with a cascade of interruptable sends of #at:put:, there's a window after the assignment where 'fileTypes' is not nil but also not fully initialized--a race condition.

The fix is simple. Do the initialization before the atomic assignment takes place, so the var is only ever bound to nil or a fully initialized object:

fileTypes
fileTypes ifNil: [
fileTypes :=
Dictionary new
at: 'txt' put: 'Text File';
at: 'html' put: 'Web Page';
at: 'pdf' put: 'Portable Document Format File';
at: 'doc' put: 'Microsoft Word Document';
yourself].
^ fileTypes.

The fixed code is still vulnerable to duplicate initialization, because the initialization sequence is interruptable and 'fileTypes' is nil during it, but as long as the initialization is cheap enough, has no side effects that restrict how often it can be done, and it's enough that the initialized objects are equal (but not identical), that's OK.

If it's too complex for a single statement, you can use a temp vars or put it in a separate factory method:

fileTypes
fileTypes ifNil: [
fileTypes := self newFileTypes].
^ fileTypes.

Similar precautions (given how easy) might as well be taken with explicit initialization of class state too. Of course if the object is mutated later (in other methods), then Mutexes or other constructs are needed to guard access. But for immutable class state, ensuring initialization is done before assignment should be enough.

> Sent: Tuesday, August 01, 2017 at 7:36 AM
> From: "Stephane Ducasse" <[hidden email]>
> To: "Any question about pharo is welcome" <[hidden email]>
> Subject: Re: [Pharo-users] Threads safety in Pharo
>
> I would love to have an analysis of assumptions made in some code.
> Because my impression is that the concurrent code is sometimes defined
> knowing the underlying logic of scheduler and this is not good.
> As I said to abdel privately in french it would be great to start from
> my french squeak book (Yes I wrote one long time ago) chapter on
> concurrent programming and turn it into a pharo chapter.
>
> Stef
>
> On Tue, Aug 1, 2017 at 1:31 PM, Ben Coman <[hidden email]> wrote:
> > Not sure I'll have what you're looking for, but to start, do you mean
> > Pharo's green threads or vm native threads?
> > cheers -ben
> >
> > On Mon, Jul 31, 2017 at 7:38 AM, Alidra Abdelghani via Pharo-users
> > <[hidden email]> wrote:
> >>
> >>
> >>
> >> ---------- Forwarded message ----------
> >> From: Alidra Abdelghani <[hidden email]>
> >> To: [hidden email]
> >> Cc: "Stéphane Ducasse" <[hidden email]>, farid arfi
> >> <[hidden email]>
> >> Bcc:
> >> Date: Mon, 31 Jul 2017 01:38:58 +0200
> >> Subject: Threads safety in Pharo
> >> Hi,
> >>
> >> Somebody once evoked the problem of threads safety in Pharo. With a friend
> >> of mine who is expert in formal methods and process scheduling, we would
> >> like to have a look on it.
> >> Does anyone knows a good document describing the problem of Pharo with
> >> threads safety or at least any document that we can start with?
> >>
> >> Thanks in advance,
> >> Abdelghani
> >>
> >>
> >>
> >
>

Tim Mackinnon

Re: Thread-safe initialization of class state (was Re: Threads safety in Pharo)

Interesting, your example was subtle enough that I had to read it a few times to understand the issue. (I was caught up on the ifNil and not the contents).

Actually, thinking about it - isn't the real issue the #ifNil - you want an atomic version.

It strikes me you could wrap the whole concept into something like an AtomicLazyVariable? E.g.

initialise

fileTypes := AtomicLazyVariable using: [

Dictionary new
at: 'txt' put: 'Text File';
at: 'html' put: 'Web Page';
yourself ].

Then have something

fileTypes

^fileTypes value

Where you have a critical section in #value?

Tim

Sent from my iPhone

On 11 Aug 2017, at 07:53, monty <[hidden email]> wrote:

Here's a hypothetical broken class method that does lazy initialization of a class inst var:

fileTypes
fileTypes ifNil: [
fileTypes := Dictionary new.
fileTypes
at: 'txt' put: 'Text File';
at: 'html' put: 'Web Page';
at: 'pdf' put: 'Portable Document Format File';
at: 'doc' put: 'Microsoft Word Document'].
^ fileTypes.

Because the assignment is done first and the initialization is done after with a cascade of interruptable sends of #at:put:, there's a window after the assignment where 'fileTypes' is not nil but also not fully initialized--a race condition.

The fix is simple. Do the initialization before the atomic assignment takes place, so the var is only ever bound to nil or a fully initialized object:

fileTypes
fileTypes ifNil: [
fileTypes :=
Dictionary new
at: 'txt' put: 'Text File';
at: 'html' put: 'Web Page';
at: 'pdf' put: 'Portable Document Format File';
at: 'doc' put: 'Microsoft Word Document';
yourself].
^ fileTypes.

The fixed code is still vulnerable to duplicate initialization, because the initialization sequence is interruptable and 'fileTypes' is nil during it, but as long as the initialization is cheap enough, has no side effects that restrict how often it can be done, and it's enough that the initialized objects are equal (but not identical), that's OK.

If it's too complex for a single statement, you can use a temp vars or put it in a separate factory method:

fileTypes
fileTypes ifNil: [
fileTypes := self newFileTypes].
^ fileTypes.

Similar precautions (given how easy) might as well be taken with explicit initialization of class state too. Of course if the object is mutated later (in other methods), then Mutexes or other constructs are needed to guard access. But for immutable class state, ensuring initialization is done before assignment should be enough.

Sent: Tuesday, August 01, 2017 at 7:36 AM
From: "Stephane Ducasse" <[hidden email]>
To: "Any question about pharo is welcome" <[hidden email]>
Subject: Re: [Pharo-users] Threads safety in Pharo

I would love to have an analysis of assumptions made in some code.
Because my impression is that the concurrent code is sometimes defined
knowing the underlying logic of scheduler and this is not good.
As I said to abdel privately in french it would be great to start from
my french squeak book (Yes I wrote one long time ago) chapter on
concurrent programming and turn it into a pharo chapter.

Stef

On Tue, Aug 1, 2017 at 1:31 PM, Ben Coman <[hidden email]> wrote:
Not sure I'll have what you're looking for, but to start, do you mean
Pharo's green threads or vm native threads?
cheers -ben

On Mon, Jul 31, 2017 at 7:38 AM, Alidra Abdelghani via Pharo-users
<[hidden email]> wrote:

---------- Forwarded message ----------
From: Alidra Abdelghani <[hidden email]>
To: [hidden email]
Cc: "Stéphane Ducasse" <[hidden email]>, farid arfi
<[hidden email]>
Bcc:
Date: Mon, 31 Jul 2017 01:38:58 +0200
Subject: Threads safety in Pharo
Hi,

Somebody once evoked the problem of threads safety in Pharo. With a friend
of mine who is expert in formal methods and process scheduling, we would
like to have a look on it.
Does anyone knows a good document describing the problem of Pharo with
threads safety or at least any document that we can start with?

Thanks in advance,
Abdelghani

Denis Kudriashov

Re: Thread-safe initialization of class state (was Re: Threads safety in Pharo)

In reply to this post by monty-3

What package you explore? I not find fileTypes method in Pharo 7.

2017-08-11 8:53 GMT+02:00 monty <[hidden email]>:

Here's a hypothetical broken class method that does lazy initialization of a class inst var:

fileTypes
fileTypes ifNil: [
fileTypes := Dictionary new.
fileTypes
at: 'txt' put: 'Text File';
at: 'html' put: 'Web Page';
at: 'pdf' put: 'Portable Document Format File';
at: 'doc' put: 'Microsoft Word Document'].
^ fileTypes.

Because the assignment is done first and the initialization is done after with a cascade of interruptable sends of #at:put:, there's a window after the assignment where 'fileTypes' is not nil but also not fully initialized--a race condition.

The fix is simple. Do the initialization before the atomic assignment takes place, so the var is only ever bound to nil or a fully initialized object:

fileTypes
fileTypes ifNil: [
fileTypes :=
Dictionary new
at: 'txt' put: 'Text File';
at: 'html' put: 'Web Page';
at: 'pdf' put: 'Portable Document Format File';
at: 'doc' put: 'Microsoft Word Document';
yourself].
^ fileTypes.

The fixed code is still vulnerable to duplicate initialization, because the initialization sequence is interruptable and 'fileTypes' is nil during it, but as long as the initialization is cheap enough, has no side effects that restrict how often it can be done, and it's enough that the initialized objects are equal (but not identical), that's OK.

If it's too complex for a single statement, you can use a temp vars or put it in a separate factory method:

fileTypes
fileTypes ifNil: [
fileTypes := self newFileTypes].
^ fileTypes.

Similar precautions (given how easy) might as well be taken with explicit initialization of class state too. Of course if the object is mutated later (in other methods), then Mutexes or other constructs are needed to guard access. But for immutable class state, ensuring initialization is done before assignment should be enough.

> Sent: Tuesday, August 01, 2017 at 7:36 AM
> From: "Stephane Ducasse" <[hidden email]>
> To: "Any question about pharo is welcome" <[hidden email]>
> Subject: Re: [Pharo-users] Threads safety in Pharo
>
> I would love to have an analysis of assumptions made in some code.
> Because my impression is that the concurrent code is sometimes defined
> knowing the underlying logic of scheduler and this is not good.
> As I said to abdel privately in french it would be great to start from
> my french squeak book (Yes I wrote one long time ago) chapter on
> concurrent programming and turn it into a pharo chapter.
>
> Stef
>
> On Tue, Aug 1, 2017 at 1:31 PM, Ben Coman <[hidden email]> wrote:
> > Not sure I'll have what you're looking for, but to start, do you mean
> > Pharo's green threads or vm native threads?
> > cheers -ben
> >
> > On Mon, Jul 31, 2017 at 7:38 AM, Alidra Abdelghani via Pharo-users
> > <[hidden email]> wrote:
> >>
> >>
> >>
> >> ---------- Forwarded message ----------
> >> From: Alidra Abdelghani <[hidden email]>
> >> To: [hidden email]
> >> Cc: "Stéphane Ducasse" <[hidden email]>, farid arfi
> >> <[hidden email]>
> >> Bcc:
> >> Date: Mon, 31 Jul 2017 01:38:58 +0200
> >> Subject: Threads safety in Pharo
> >> Hi,
> >>
> >> Somebody once evoked the problem of threads safety in Pharo. With a friend
> >> of mine who is expert in formal methods and process scheduling, we would
> >> like to have a look on it.
> >> Does anyone knows a good document describing the problem of Pharo with
> >> threads safety or at least any document that we can start with?
> >>
> >> Thanks in advance,
> >> Abdelghani
> >>
> >>
> >>
> >
>

monty-3

Re: Thread-safe initialization of class state (was Re: Threads safety in Pharo)

In reply to this post by Tim Mackinnon

> Sent: Friday, August 11, 2017 at 5:51 AM
> From: "Tim Mackinnon" <[hidden email]>
> To: "Any question about pharo is welcome" <[hidden email]>
> Subject: Re: [Pharo-users] Thread-safe initialization of class state (was Re: Threads safety in Pharo)
>
> Interesting, your example was subtle enough that I had to read it a few times to understand the issue. (I was caught up on the ifNil and not the contents).

That's the problem. They're close enough that someone could mistakenly refactor the correct code into the incorrect code--and that code would be perfectly fine for lazy initialization of instance state in a non-concurrent class. That's why I always insert a small comment before explaining why it was written that way.

> Actually, thinking about it - isn't the real issue the #ifNil - you want an atomic version.
>
> It strikes me you could wrap the whole concept into something like an AtomicLazyVariable? E.g.
>
> initialise
> fileTypes := AtomicLazyVariable using: [
>
> Dictionary new
> at: 'txt' put: 'Text File';
> at: 'html' put: 'Web Page';
> yourself ].
>
> Then have something
>
> fileTypes
> ^fileTypes value
>
> Where you have a critical section in #value?
>
> Tim
>
> Sent from my iPhone
> On 11 Aug 2017, at 07:53, monty <[hidden email][mailto:[hidden email]]> wrote:
>
> Here's a hypothetical broken class method that does lazy initialization of a class inst var:
>
> fileTypes
> fileTypes ifNil: [
> fileTypes := Dictionary new.
> fileTypes
> at: 'txt' put: 'Text File';
> at: 'html' put: 'Web Page';
> at: 'pdf' put: 'Portable Document Format File';
> at: 'doc' put: 'Microsoft Word Document'].
> ^ fileTypes.
>
> Because the assignment is done first and the initialization is done after with a cascade of interruptable sends of #at:put:, there's a window after the assignment where 'fileTypes' is not nil but also not fully initialized--a race condition.
>
> The fix is simple. Do the initialization before the atomic assignment takes place, so the var is only ever bound to nil or a fully initialized object:
>
> fileTypes
> fileTypes ifNil: [
> fileTypes :=
> Dictionary new
> at: 'txt' put: 'Text File';
> at: 'html' put: 'Web Page';
> at: 'pdf' put: 'Portable Document Format File';
> at: 'doc' put: 'Microsoft Word Document';
> yourself].
> ^ fileTypes.
>
> The fixed code is still vulnerable to duplicate initialization, because the initialization sequence is interruptable and 'fileTypes' is nil during it, but as long as the initialization is cheap enough, has no side effects that restrict how often it can be done, and it's enough that the initialized objects are equal (but not identical), that's OK.
>
> If it's too complex for a single statement, you can use a temp vars or put it in a separate factory method:
>
> fileTypes
> fileTypes ifNil: [
> fileTypes := self newFileTypes].
> ^ fileTypes.
>
> Similar precautions (given how easy) might as well be taken with explicit initialization of class state too. Of course if the object is mutated later (in other methods), then Mutexes or other constructs are needed to guard access. But for immutable class state, ensuring initialization is done before assignment should be enough.
> Sent: Tuesday, August 01, 2017 at 7:36 AMFrom: "Stephane Ducasse" <[hidden email][mailto:[hidden email]]>To: "Any question about pharo is welcome" <[hidden email][mailto:[hidden email]]>Subject: Re: [Pharo-users] Threads safety in Pharo I would love to have an analysis of assumptions made in some code.Because my impression is that the concurrent code is sometimes definedknowing the underlying logic of scheduler and this is not good.As I said to abdel privately in french it would be great to start frommy french squeak book (Yes I wrote one long time ago) chapter onconcurrent programming and turn it into a pharo chapter. Stef On Tue, Aug 1, 2017 at 1:31 PM, Ben Coman <[hidden email][mailto:[hidden email]]> wrote:Not sure I'll have what you're looking for, but to start, do you meanPharo's green threads or vm native threads?cheers -ben On Mon, Jul 31, 2017 at 7:38 AM, Alidra Abdelghani via Pharo-users<[hidden email][mailto:[hidden email]]> wrote: ---------- Forwarded message ----------From: Alidra Abdelghani <[hidden email][mailto:[hidden email]]>To: [hidden email][mailto:[hidden email]]Cc: "Stéphane Ducasse" <[hidden email][mailto:[hidden email]]>, farid arfi<[hidden email][mailto:[hidden email]]>Bcc:Date: Mon, 31 Jul 2017 01:38:58 +0200Subject: Threads safety in PharoHi, Somebody once evoked the problem of threads safety in Pharo. With a friendof mine who is expert in formal methods and process scheduling, we wouldlike to have a look on it.Does anyone knows a good document describing the problem of Pharo withthreads safety or at least any document that we can start with? Thanks in advance,Abdelghani

monty-3

Re: Thread-safe initialization of class state (was Re: Threads safety in Pharo)

In reply to this post by Denis Kudriashov

> Sent: Friday, August 11, 2017 at 6:36 AM
> From: "Denis Kudriashov" <[hidden email]>
> To: "Any question about pharo is welcome" <[hidden email]>
> Subject: Re: [Pharo-users] Thread-safe initialization of class state (was Re: Threads safety in Pharo)
>
> What package you explore? I not find fileTypes method in Pharo 7.

Like I said, it's a hypothetical example. But I'm sure you could find similar examples of unsafe class state initialization in the image and in popular libraries.

> 2017-08-11 8:53 GMT+02:00 monty <[hidden email][mailto:[hidden email]]>:Here's a hypothetical broken class method that does lazy initialization of a class inst var:
>
> fileTypes
> fileTypes ifNil: [
> fileTypes := Dictionary new.
> fileTypes
> at: 'txt' put: 'Text File';
> at: 'html' put: 'Web Page';
> at: 'pdf' put: 'Portable Document Format File';
> at: 'doc' put: 'Microsoft Word Document'].
> ^ fileTypes.
>
> Because the assignment is done first and the initialization is done after with a cascade of interruptable sends of #at:put:, there's a window after the assignment where 'fileTypes' is not nil but also not fully initialized--a race condition.
>
> The fix is simple. Do the initialization before the atomic assignment takes place, so the var is only ever bound to nil or a fully initialized object:
>
> fileTypes
> fileTypes ifNil: [
> fileTypes :=
> Dictionary new
> at: 'txt' put: 'Text File';
> at: 'html' put: 'Web Page';
> at: 'pdf' put: 'Portable Document Format File';
> at: 'doc' put: 'Microsoft Word Document';
> yourself].
> ^ fileTypes.
>
> The fixed code is still vulnerable to duplicate initialization, because the initialization sequence is interruptable and 'fileTypes' is nil during it, but as long as the initialization is cheap enough, has no side effects that restrict how often it can be done, and it's enough that the initialized objects are equal (but not identical), that's OK.
>
> If it's too complex for a single statement, you can use a temp vars or put it in a separate factory method:
>
> fileTypes
> fileTypes ifNil: [
> fileTypes := self newFileTypes].
> ^ fileTypes.
>
> Similar precautions (given how easy) might as well be taken with explicit initialization of class state too. Of course if the object is mutated later (in other methods), then Mutexes or other constructs are needed to guard access. But for immutable class state, ensuring initialization is done before assignment should be enough.
>
> > Sent: Tuesday, August 01, 2017 at 7:36 AM
> > From: "Stephane Ducasse" <[hidden email][mailto:[hidden email]]>
> > To: "Any question about pharo is welcome" <[hidden email][mailto:[hidden email]]>
> > Subject: Re: [Pharo-users] Threads safety in Pharo
> >
> > I would love to have an analysis of assumptions made in some code.
> > Because my impression is that the concurrent code is sometimes defined
> > knowing the underlying logic of scheduler and this is not good.
> > As I said to abdel privately in french it would be great to start from
> > my french squeak book (Yes I wrote one long time ago) chapter on
> > concurrent programming and turn it into a pharo chapter.
> >
> > Stef
> >
> > On Tue, Aug 1, 2017 at 1:31 PM, Ben Coman <[hidden email][mailto:[hidden email]]> wrote:
> > > Not sure I'll have what you're looking for, but to start, do you mean
> > > Pharo's green threads or vm native threads?
> > > cheers -ben
> > >
> > > On Mon, Jul 31, 2017 at 7:38 AM, Alidra Abdelghani via Pharo-users
> > > <[hidden email][mailto:[hidden email]]> wrote:
> > >>
> > >>
> > >>
> > >> ---------- Forwarded message ----------
> > >> From: Alidra Abdelghani <[hidden email][mailto:[hidden email]]>
> > >> To: [hidden email][mailto:[hidden email]]
> > >> Cc: "Stéphane Ducasse" <[hidden email][mailto:[hidden email]]>, farid arfi
> > >> <[hidden email][mailto:[hidden email]]>
> > >> Bcc:
> > >> Date: Mon, 31 Jul 2017 01:38:58 +0200
> > >> Subject: Threads safety in Pharo
> > >> Hi,
> > >>
> > >> Somebody once evoked the problem of threads safety in Pharo. With a friend
> > >> of mine who is expert in formal methods and process scheduling, we would
> > >> like to have a look on it.
> > >> Does anyone knows a good document describing the problem of Pharo with
> > >> threads safety or at least any document that we can start with?
> > >>
> > >> Thanks in advance,
> > >> Abdelghani
> > >>
> > >>
> > >>
> > >
> >
>

gcotelli

Re: Thread-safe initialization of class state (was Re: Threads safety in Pharo)

Suerly we can add a Critics Rule trying to detect the invalid pattern.

On Fri, Aug 11, 2017 at 10:39 AM, monty <[hidden email]> wrote:

> Sent: Friday, August 11, 2017 at 6:36 AM
> From: "Denis Kudriashov" <[hidden email]>
> To: "Any question about pharo is welcome" <[hidden email]>
> Subject: Re: [Pharo-users] Thread-safe initialization of class state (was Re: Threads safety in Pharo)
>
> What package you explore? I not find fileTypes method in Pharo 7.

Like I said, it's a hypothetical example. But I'm sure you could find similar examples of unsafe class state initialization in the image and in popular libraries.

> 2017-08-11 8:53 GMT+02:00 monty <[hidden email][mailto:[hidden email]]>:Here's a hypothetical broken class method that does lazy initialization of a class inst var:

>
> fileTypes
> fileTypes ifNil: [
> fileTypes := Dictionary new.
> fileTypes
> at: 'txt' put: 'Text File';
> at: 'html' put: 'Web Page';
> at: 'pdf' put: 'Portable Document Format File';
> at: 'doc' put: 'Microsoft Word Document'].
> ^ fileTypes.
>
> Because the assignment is done first and the initialization is done after with a cascade of interruptable sends of #at:put:, there's a window after the assignment where 'fileTypes' is not nil but also not fully initialized--a race condition.
>
> The fix is simple. Do the initialization before the atomic assignment takes place, so the var is only ever bound to nil or a fully initialized object:
>
> fileTypes
> fileTypes ifNil: [
> fileTypes :=
> Dictionary new
> at: 'txt' put: 'Text File';
> at: 'html' put: 'Web Page';
> at: 'pdf' put: 'Portable Document Format File';
> at: 'doc' put: 'Microsoft Word Document';
> yourself].
> ^ fileTypes.
>
> The fixed code is still vulnerable to duplicate initialization, because the initialization sequence is interruptable and 'fileTypes' is nil during it, but as long as the initialization is cheap enough, has no side effects that restrict how often it can be done, and it's enough that the initialized objects are equal (but not identical), that's OK.
>
> If it's too complex for a single statement, you can use a temp vars or put it in a separate factory method:
>
> fileTypes
> fileTypes ifNil: [
> fileTypes := self newFileTypes].
> ^ fileTypes.
>
> Similar precautions (given how easy) might as well be taken with explicit initialization of class state too. Of course if the object is mutated later (in other methods), then Mutexes or other constructs are needed to guard access. But for immutable class state, ensuring initialization is done before assignment should be enough.
>
> > Sent: Tuesday, August 01, 2017 at 7:36 AM

> > From: "Stephane Ducasse" <[hidden email][mailto:[hidden email]]>
> > To: "Any question about pharo is welcome" <[hidden email][mailto:[hidden email]]>
> > Subject: Re: [Pharo-users] Threads safety in Pharo
> >
> > I would love to have an analysis of assumptions made in some code.
> > Because my impression is that the concurrent code is sometimes defined
> > knowing the underlying logic of scheduler and this is not good.
> > As I said to abdel privately in french it would be great to start from
> > my french squeak book (Yes I wrote one long time ago) chapter on
> > concurrent programming and turn it into a pharo chapter.
> >
> > Stef
> >
> > On Tue, Aug 1, 2017 at 1:31 PM, Ben Coman <[hidden email][mailto:[hidden email]]> wrote:
> > > Not sure I'll have what you're looking for, but to start, do you mean
> > > Pharo's green threads or vm native threads?
> > > cheers -ben
> > >
> > > On Mon, Jul 31, 2017 at 7:38 AM, Alidra Abdelghani via Pharo-users
> > > <[hidden email][mailto:[hidden email]]> wrote:
> > >>
> > >>
> > >>
> > >> ---------- Forwarded message ----------
> > >> From: Alidra Abdelghani <[hidden email][mailto:[hidden email]]>

> > >> To: [hidden email][mailto:[hidden email]]
> > >> Cc: "Stéphane Ducasse" <[hidden email][mailto:[hidden email]]>, farid arfi
> > >> <[hidden email][mailto:[hidden email]]>
> > >> Bcc:
> > >> Date: Mon, 31 Jul 2017 01:38:58 +0200
> > >> Subject: Threads safety in Pharo
> > >> Hi,
> > >>
> > >> Somebody once evoked the problem of threads safety in Pharo. With a friend
> > >> of mine who is expert in formal methods and process scheduling, we would
> > >> like to have a look on it.
> > >> Does anyone knows a good document describing the problem of Pharo with
> > >> threads safety or at least any document that we can start with?
> > >>
> > >> Thanks in advance,
> > >> Abdelghani
> > >>
> > >>
> > >>
> > >
> >
>

Tim Mackinnon

Re: Thread-safe initialization of class state (was Re: Threads safety in Pharo)

I think that thread safety comes with a penalty - so the overarching question is are we trying to create thread safe browsers (or does a thread safe penalty even impact browsing as maybe it's negligible?)

Monty - maybe you could measure the impact if you are interested in this? If it's not much and we just need to encapsulate it with some mechanism (potentially having thread safety critics we can enable) then it's possibly something Calypso can embrace.

This said, multi user editing is not something that has really taken off. Jason at VW did lots with Wolfpack, and it wasn't an idea that gained lots of traction a few years ago (sadly).

It's tricky as we have to spend our engineering resources wisely.

Tim

Sent from my iPhone

On 11 Aug 2017, at 19:41, Gabriel Cotelli <[hidden email]> wrote:

Suerly we can add a Critics Rule trying to detect the invalid pattern.

On Fri, Aug 11, 2017 at 10:39 AM, monty <[hidden email]> wrote:
> Sent: Friday, August 11, 2017 at 6:36 AM
> From: "Denis Kudriashov" <[hidden email]>
> To: "Any question about pharo is welcome" <[hidden email]>
> Subject: Re: [Pharo-users] Thread-safe initialization of class state (was Re: Threads safety in Pharo)
>
> What package you explore? I not find fileTypes method in Pharo 7.

Like I said, it's a hypothetical example. But I'm sure you could find similar examples of unsafe class state initialization in the image and in popular libraries.

> 2017-08-11 8:53 GMT+02:00 monty <[hidden email][mailto:[hidden email]]>:Here's a hypothetical broken class method that does lazy initialization of a class inst var:

>
> fileTypes
> fileTypes ifNil: [
> fileTypes := Dictionary new.
> fileTypes
> at: 'txt' put: 'Text File';
> at: 'html' put: 'Web Page';
> at: 'pdf' put: 'Portable Document Format File';
> at: 'doc' put: 'Microsoft Word Document'].
> ^ fileTypes.
>
> Because the assignment is done first and the initialization is done after with a cascade of interruptable sends of #at:put:, there's a window after the assignment where 'fileTypes' is not nil but also not fully initialized--a race condition.
>
> The fix is simple. Do the initialization before the atomic assignment takes place, so the var is only ever bound to nil or a fully initialized object:
>
> fileTypes
> fileTypes ifNil: [
> fileTypes :=
> Dictionary new
> at: 'txt' put: 'Text File';
> at: 'html' put: 'Web Page';
> at: 'pdf' put: 'Portable Document Format File';
> at: 'doc' put: 'Microsoft Word Document';
> yourself].
> ^ fileTypes.
>
> The fixed code is still vulnerable to duplicate initialization, because the initialization sequence is interruptable and 'fileTypes' is nil during it, but as long as the initialization is cheap enough, has no side effects that restrict how often it can be done, and it's enough that the initialized objects are equal (but not identical), that's OK.
>
> If it's too complex for a single statement, you can use a temp vars or put it in a separate factory method:
>
> fileTypes
> fileTypes ifNil: [
> fileTypes := self newFileTypes].
> ^ fileTypes.
>
> Similar precautions (given how easy) might as well be taken with explicit initialization of class state too. Of course if the object is mutated later (in other methods), then Mutexes or other constructs are needed to guard access. But for immutable class state, ensuring initialization is done before assignment should be enough.
>
> > Sent: Tuesday, August 01, 2017 at 7:36 AM

> > From: "Stephane Ducasse" <[hidden email][mailto:[hidden email]]>
> > To: "Any question about pharo is welcome" <[hidden email][mailto:[hidden email]]>
> > Subject: Re: [Pharo-users] Threads safety in Pharo
> >
> > I would love to have an analysis of assumptions made in some code.
> > Because my impression is that the concurrent code is sometimes defined
> > knowing the underlying logic of scheduler and this is not good.
> > As I said to abdel privately in french it would be great to start from
> > my french squeak book (Yes I wrote one long time ago) chapter on
> > concurrent programming and turn it into a pharo chapter.
> >
> > Stef
> >
> > On Tue, Aug 1, 2017 at 1:31 PM, Ben Coman <[hidden email][mailto:[hidden email]]> wrote:
> > > Not sure I'll have what you're looking for, but to start, do you mean
> > > Pharo's green threads or vm native threads?
> > > cheers -ben
> > >
> > > On Mon, Jul 31, 2017 at 7:38 AM, Alidra Abdelghani via Pharo-users
> > > <[hidden email][mailto:[hidden email]]> wrote:
> > >>
> > >>
> > >>
> > >> ---------- Forwarded message ----------
> > >> From: Alidra Abdelghani <[hidden email][mailto:[hidden email]]>

> > >> To: [hidden email][mailto:[hidden email]]
> > >> Cc: "Stéphane Ducasse" <[hidden email][mailto:[hidden email]]>, farid arfi
> > >> <[hidden email][mailto:[hidden email]]>
> > >> Bcc:
> > >> Date: Mon, 31 Jul 2017 01:38:58 +0200
> > >> Subject: Threads safety in Pharo
> > >> Hi,
> > >>
> > >> Somebody once evoked the problem of threads safety in Pharo. With a friend
> > >> of mine who is expert in formal methods and process scheduling, we would
> > >> like to have a look on it.
> > >> Does anyone knows a good document describing the problem of Pharo with
> > >> threads safety or at least any document that we can start with?
> > >>
> > >> Thanks in advance,
> > >> Abdelghani
> > >>
> > >>
> > >>
> > >
> >
>