Smalltalk › Squeak › Squeak - Dev

Process-specific state broken and uncomplete

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

25 messages Options

Igor Stasenko

Process-specific state broken and uncomplete

Hello,

i just found that Squeak images Process class contains unused ivars:
island env

in Pharo image, env ivar is used for holding a process-specific state,
which is convenient, so one could use:
Processor activeProcess environmentAt: put:...

but in Squeak image there is even no such methods.

There are also some caveats with such thing:
during process termination, Processor activeProcess could point to
different process,
and so, accessing to process environment may lead to error(s).

To avoid issues like these, the process termination action (stack
unwinding) should be always performed only for active process.
One of a trick i think, is to replace the suspendedContext with own
context, and then activate given process in order to unwind its stack
etc.

Another way is to remember environment somewhere else during process
termination, and use different method to access it i.e. instead of:
Processor activeProcess environmentAt:
use
self processEnvironmentAt:

where #processEnvironmentAt: is implemented in Object class, and can
handle termination gracefully.

But i think replacing suspendedContext is more generic.
What you think?

--
Best regards,
Igor Stasenko AKA sig.

Andreas.Raab

Re: Process-specific state broken and uncomplete

On 11/7/2010 9:39 PM, Igor Stasenko wrote:
> What you think?

Nag Eliot to push our changes for process-faithful debugging. This is
the Right Way to deal with this stuff.

Cheers,
- Andreas

Eliot Miranda-2

Re: Process-specific state broken and uncomplete

On Mon, Nov 8, 2010 at 12:44 AM, Andreas Raab <[hidden email]> wrote:

On 11/7/2010 9:39 PM, Igor Stasenko wrote:

What you think?

Nag Eliot to push our changes for process-faithful debugging. This is the Right Way to deal with this stuff.

Agreed. I'll do this soon (after my FAST presentations?)

Cheers,
- Andreas

CdAB63

Problems while installing TweakCore

While loading TweakCore there's a problem:

    [done] whileFalse: [
        entryArray := self primLookupEntryIn: path index: index.
        #badDirectoryPath = entryArray ifTrue: [
            ^(InvalidDirectoryError pathName: pathName) signal].
        entryArray == nil
            ifTrue: [done := true]
            ifFalse: [aBlock value: (DirectoryEntry fromArray: entryArray)].
        index := index + 1].

Definition of DirectoryEntry does not understand the message "fromArray:" but "fromArray:directory:"

DirectoryEntry>>fromArray: array directory: aFileDirectoryOrServerDirectory
    | entryType |
    entryType := (array at: 4)
        ifTrue: [ DirectoryEntryDirectory ]
        ifFalse: [ DirectoryEntryFile ].
    ^ entryType
        directory: aFileDirectoryOrServerDirectory
        name: (array at: 1)
        creationTime: (array at: 2)
        modificationTime: (array at: 3)
        fileSize: (array at: 5)

This problem happens at several points of TweakCore.

Best regards

CdAB

Hannes Hirzel

Re: Problems while installing TweakCore

Casimiro,

Have you seen this email
http://lists.squeakfoundation.org/pipermail/squeak-dev/2010-May/151048.html

where Nikolay Suslov managed to load Tweak into Squeak 4.1?

--Hannes

On 11/8/10, Casimiro de Almeida Barreto <[hidden email]> wrote:

> While loading TweakCore there's a problem:
>
> [done] whileFalse: [
> entryArray := self primLookupEntryIn: path index: index.
> #badDirectoryPath = entryArray ifTrue: [
> ^(InvalidDirectoryError pathName: pathName) signal].
> entryArray == nil
> ifTrue: [done := true]
> ifFalse: [aBlock value: (*DirectoryEntry fromArray:
> entryArray*)].
> index := index + 1].
>
> Definition of DirectoryEntry does not understand the message
> "fromArray:" but "fromArray:directory:"
>
> DirectoryEntry>>fromArray: array directory:
> aFileDirectoryOrServerDirectory
> | entryType |
> entryType := (array at: 4)
> ifTrue: [ DirectoryEntryDirectory ]
> ifFalse: [ DirectoryEntryFile ].
> ^ entryType
> directory: aFileDirectoryOrServerDirectory
> name: (array at: 1)
> creationTime: (array at: 2)
> modificationTime: (array at: 3)
> fileSize: (array at: 5)
>
>
> This problem happens at several points of TweakCore.
>
> Best regards
>
> CdAB
>

Andreas.Raab

Re: Problems while installing TweakCore

In reply to this post by CdAB63

Try it now. The method directoryContentsFor:do: was added to 4.2 and
created a conflict when loading TweakCore. I've removed it from Tweak so
hopefully it will work now.

Cheers,
- Andreas

On 11/8/2010 11:08 AM, Casimiro de Almeida Barreto wrote:

> While loading TweakCore there's a problem:
>
> [done] whileFalse: [
> entryArray := self primLookupEntryIn: path index: index.
> #badDirectoryPath = entryArray ifTrue: [
> ^(InvalidDirectoryError pathName: pathName) signal].
> entryArray == nil
> ifTrue: [done := true]
> ifFalse: [aBlock value: (*DirectoryEntry fromArray: entryArray*)].
> index := index + 1].
>
> Definition of DirectoryEntry does not understand the message
> "fromArray:" but "fromArray:directory:"
>
> DirectoryEntry>>fromArray: array directory:
> aFileDirectoryOrServerDirectory
> | entryType |
> entryType := (array at: 4)
> ifTrue: [ DirectoryEntryDirectory ]
> ifFalse: [ DirectoryEntryFile ].
> ^ entryType
> directory: aFileDirectoryOrServerDirectory
> name: (array at: 1)
> creationTime: (array at: 2)
> modificationTime: (array at: 3)
> fileSize: (array at: 5)
>
>
> This problem happens at several points of TweakCore.
>
> Best regards
>
> CdAB
>
>
>
>

Igor Stasenko

Re: Process-specific state broken and uncomplete

In reply to this post by Andreas.Raab

On 8 November 2010 10:44, Andreas Raab <[hidden email]> wrote:
> On 11/7/2010 9:39 PM, Igor Stasenko wrote:
>>
>> What you think?
>
> Nag Eliot to push our changes for process-faithful debugging. This is the
> Right Way to deal with this stuff.
>

i remember that discussion. Still, i meant a different functionality.
Consider a following code:

[
" init session here "

[ ... do something ... ]

ensure: [ self session close]
] fork

where session implemented as:

session
^ Processor activeProcess environmentAt: #mySession

I could write a test case for this example.
Currently such example will work only in Pharo, and only if ensure
block evaluated within same active process (which is not, if you
terminating this process from other process).

> Cheers,
> - Andreas
>
>

--
Best regards,
Igor Stasenko AKA sig.

Andreas.Raab

Re: Process-specific state broken and uncomplete

On 11/8/2010 1:22 PM, Igor Stasenko wrote:
> Currently such example will work only in Pharo, and only if ensure
> block evaluated within same active process (which is not, if you
> terminating this process from other process).

Well, then don't write code like that :-) Seriously, your code example
seems broken. First, in the case of:

[
" init session here "
[ ... do something ... ] ensure: [ self session close]
] fork

it will function correctly since your session is initialized *within*
the fork not outside of it, thus Processor activeProcess is the child
process, not the parent process. But even if we leave that out, I'd
expect your code to read, e.g.,

" init session here "
forkedSession := self session.
[
self session: forkedSession.
[ ... do something ... ] ensure: [ self session close]
] fork

So I'd say that the fault is solely in your code.

FWIW, the only realistic alternative that I could see is if the TLS
either implements some sort of inheritance for the environment lookup
(i.e., look up locally and if locally undefined traverse through parent
processes) or copies all the variables into a newly forked process. Both
approaches raise some very nasty concurrency issues which is why I
prefer the simple (non-)solution we have today. A bug like in the code
above is easy to find and to fix as it is completely reproducible.
Contrary to which concurrent access to the environment dicitionary is
not, and even if it is serialized you still need to make sure the access
to the contents is serialized as well.

Cheers,
- Andreas

Igor Stasenko

Re: Process-specific state broken and uncomplete

On 9 November 2010 00:05, Andreas Raab <[hidden email]> wrote:

> On 11/8/2010 1:22 PM, Igor Stasenko wrote:
>>
>> Currently such example will work only in Pharo, and only if ensure
>> block evaluated within same active process (which is not, if you
>> terminating this process from other process).
>
> Well, then don't write code like that :-) Seriously, your code example seems
> broken. First, in the case of:
>
> [
> " init session here "
> [ ... do something ... ] ensure: [ self session close]
> ] fork
>
> it will function correctly since your session is initialized *within* the
> fork not outside of it, thus Processor activeProcess is the child process,
> not the parent process. But even if we leave that out, I'd expect your code
> to read, e.g.,
>
> " init session here "
> forkedSession := self session.
> [
> self session: forkedSession.
> [ ... do something ... ] ensure: [ self session close]
> ] fork
>

hmm.. i don't think that session initialization should always take
place within same process where it is used. I could pass already
initialized session, and then " init session here " is only to put it
into process environment var.
I want to associate a certain state with process (which is a session).
So, then any domain object could access it by using 'self session',
without need of using exception mechanism (like seaside does with
WADynamicVariable), or even worse, keep a direct reference to it.

For instance, when you creating a new process on a server for each
client socket connection, this is
logical to associate that process with a session.

> So I'd say that the fault is solely in your code.
>
> FWIW, the only realistic alternative that I could see is if the TLS either
> implements some sort of inheritance for the environment lookup (i.e., look
> up locally and if locally undefined traverse through parent processes) or
> copies all the variables into a newly forked process. Both approaches raise
> some very nasty concurrency issues which is why I prefer the simple
> (non-)solution we have today. A bug like in the code above is easy to find
> and to fix as it is completely reproducible. Contrary to which concurrent
> access to the environment dicitionary is not, and even if it is serialized
> you still need to make sure the access to the contents is serialized as
> well.
>

i din't considered about parent/child relationship between processes
(like unix style processes).
It opens some interesting perspectives, but i don't think we need such
relationship so much,
because its easy to share state between processes without such
'unixish' ad-hoc composition.

> Cheers,
> - Andreas
>
>

--
Best regards,
Igor Stasenko AKA sig.

Andreas.Raab

Re: Process-specific state broken and uncomplete

On 11/8/2010 2:33 PM, Igor Stasenko wrote:

> On 9 November 2010 00:05, Andreas Raab<[hidden email]> wrote:
>> " init session here "
>> forkedSession := self session.
>> [
>> self session: forkedSession.
>> [ ... do something ... ] ensure: [ self session close]
>> ] fork
>>
>
> hmm.. i don't think that session initialization should always take
> place within same process where it is used. I could pass already
> initialized session, and then " init session here " is only to put it
> into process environment var.

Sure. But then it's initialized within the correct scope so your ensure
block will work correctly. (fwiw, at this point I'm a bit confused about
what you're trying to do and how it wouldn't work currently; can you
perhaps write a simple test that illustrates the problem?)

> I want to associate a certain state with process (which is a session).
> So, then any domain object could access it by using 'self session',
> without need of using exception mechanism (like seaside does with
> WADynamicVariable), or even worse, keep a direct reference to it.
>
> For instance, when you creating a new process on a server for each
> client socket connection, this is
> logical to associate that process with a session.

I'm not so sure about that :-) But yes, it's something that could be
done. Though, again I would expect that the initialization of the
session global happens within the fork, thus all of the above should
work fine.

> i din't considered about parent/child relationship between processes
> (like unix style processes).
> It opens some interesting perspectives, but i don't think we need such
> relationship so much,
> because its easy to share state between processes without such
> 'unixish' ad-hoc composition.

Agreed.

Cheers,
- Andreas

Levente Uzonyi-2

Re: [Pharo-project] Process-specific state broken and uncomplete

In reply to this post by Igor Stasenko

On Mon, 8 Nov 2010, Igor Stasenko wrote:

> Hello,
>
> i just found that Squeak images Process class contains unused ivars:
> island env

IIRC those were added by Matthew Fulmer to support ProcessSpecific _and_
Cobalt. The idea is that the two can be used together this way, because
the definition of Process doesn't have to be changed.

>
> in Pharo image, env ivar is used for holding a process-specific state,
> which is convenient, so one could use:
> Processor activeProcess environmentAt: put:...
>
> but in Squeak image there is even no such methods.

So they integrated the full ProcessSpecific package. Is it a problem that
you have to load it into Squeak?

Levente

>
> There are also some caveats with such thing:
> during process termination, Processor activeProcess could point to
> different process,
> and so, accessing to process environment may lead to error(s).
>
> To avoid issues like these, the process termination action (stack
> unwinding) should be always performed only for active process.
> One of a trick i think, is to replace the suspendedContext with own
> context, and then activate given process in order to unwind its stack
> etc.
>
> Another way is to remember environment somewhere else during process
> termination, and use different method to access it i.e. instead of:
> Processor activeProcess environmentAt:
> use
> self processEnvironmentAt:
>
> where #processEnvironmentAt: is implemented in Object class, and can
> handle termination gracefully.
>
> But i think replacing suspendedContext is more generic.
> What you think?
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
>

Igor Stasenko

Re: Process-specific state broken and uncomplete

In reply to this post by Andreas.Raab

On 9 November 2010 02:02, Andreas Raab <[hidden email]> wrote:

> On 11/8/2010 2:33 PM, Igor Stasenko wrote:
>>
>> On 9 November 2010 00:05, Andreas Raab<[hidden email]> wrote:
>>>
>>> " init session here "
>>> forkedSession := self session.
>>> [
>>> self session: forkedSession.
>>> [ ... do something ... ] ensure: [ self session close]
>>> ] fork
>>>
>>
>> hmm.. i don't think that session initialization should always take
>> place within same process where it is used. I could pass already
>> initialized session, and then " init session here " is only to put it
>> into process environment var.
>
> Sure. But then it's initialized within the correct scope so your ensure
> block will work correctly. (fwiw, at this point I'm a bit confused about
> what you're trying to do and how it wouldn't work currently; can you perhaps
> write a simple test that illustrates the problem?)
>

Suppose you have a domain object which provides following ways to
access the session:

session: forkedSession
Processor activeProcess environmentAt: #session put: forkedSession

session
^ Processor activeProcess environmentAt: #session

now, here what i'd like to write:

| session |

session := MySession new .
session initialize blabla.

"good fork"
good := [
self session: session.
[ self process stuff ] ensure: [ session close ]
] newProcess.

good resume.
1 minute asDelay wait.
good terminate.
self assert: (session isClosed).

In the above case, everything is ok: session object is passed in 'session' temp
and visible inside ensure block directly, so, no matter what may
happen (process termination),
it is guaranteed that session will be closed at the end.

"bad fork"

session := MySession new .
session initialize blabla.

bad := [
self session: session.
[ self process stuff ] ensure: [ self session close ]
] newProcess.

bad resume.
1 minute asDelay wait.
bad terminate.
self assert: (session isClosed).

In this case, a session is accessed indirectly, through process specific state.
And test might fail or not, depending if process was already
terminated before sending #terminate,
or its still running.

The 'bad fork' illustrates that one should not attempt to access a
process specific state in ensure blocks,
because during process termination, stack unwinding is performed by
other process, which could have different session object,
or don't have it at all.

This is simple, of course, to implement session finalization without
using process spefic state. But now consider that session has not only
#close method, but allows doing something more userful, like:

self session backupStuff.
[ .. do something.. ] ensure: [ self session restoreStuff ].

And again, same problem: unless you have a direct reference to session
object, you can't do it.
Not with using process-specific state, because it is flawed.

In other words, anything which may want to access to a
process-specific state, like:

[ ... ... ] ensure: [ (Processor activeProcess environmentAt: #foo)
doSomething ]

have no guarantees to work correctly in all cases.

This defeats a purpose of process-specific state, because you can't
rely on it for building a reliable stuff.

>> I want to associate a certain state with process (which is a session).
>> So, then any domain object could access it by using 'self session',
>> without need of using exception mechanism (like seaside does with
>> WADynamicVariable), or even worse, keep a direct reference to it.
>>
>> For instance, when you creating a new process on a server for each
>> client socket connection, this is
>> logical to associate that process with a session.
>
> I'm not so sure about that :-) But yes, it's something that could be done.
> Though, again I would expect that the initialization of the session global
> happens within the fork, thus all of the above should work fine.
>

i don't see how initialization related to problem, that
process-specific state are not accessible for all cases
and all stages of process life, as seen by developer, when he using
#ensure: blocks.

>> i din't considered about parent/child relationship between processes
>> (like unix style processes).
>> It opens some interesting perspectives, but i don't think we need such
>> relationship so much,
>> because its easy to share state between processes without such
>> 'unixish' ad-hoc composition.
>
> Agreed.
>
> Cheers,
> - Andreas
>
>

--
Best regards,
Igor Stasenko AKA sig.

Igor Stasenko

Re: [Pharo-project] Process-specific state broken and uncomplete

In reply to this post by Levente Uzonyi-2

On 9 November 2010 03:31, Levente Uzonyi <[hidden email]> wrote:

> On Mon, 8 Nov 2010, Igor Stasenko wrote:
>
>> Hello,
>>
>> i just found that Squeak images Process class contains unused ivars:
>> island env
>
> IIRC those were added by Matthew Fulmer to support ProcessSpecific _and_
> Cobalt. The idea is that the two can be used together this way, because the
> definition of Process doesn't have to be changed.
>

Sure.

>>
>> in Pharo image, env ivar is used for holding a process-specific state,
>> which is convenient, so one could use:
>> Processor activeProcess environmentAt: put:...
>>
>> but in Squeak image there is even no such methods.
>
> So they integrated the full ProcessSpecific package. Is it a problem that
> you have to load it into Squeak?
>
>

So, its a separate package, except that ivars already there..
Strange choice. Maybe it worth mentioning then in Process comment that
these vars are there for purpose?
Otherwise someone may attempt to remove them :)

> Levente
>
>>
>> There are also some caveats with such thing:
>> during process termination, Processor activeProcess could point to
>> different process,
>> and so, accessing to process environment may lead to error(s).
>>
>> To avoid issues like these, the process termination action (stack
>> unwinding) should be always performed only for active process.
>> One of a trick i think, is to replace the suspendedContext with own
>> context, and then activate given process in order to unwind its stack
>> etc.
>>
>> Another way is to remember environment somewhere else during process
>> termination, and use different method to access it i.e. instead of:
>> Processor activeProcess environmentAt:
>> use
>> self processEnvironmentAt:
>>
>> where #processEnvironmentAt: is implemented in Object class, and can
>> handle termination gracefully.
>>
>> But i think replacing suspendedContext is more generic.
>> What you think?
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>>
>
>

--
Best regards,
Igor Stasenko AKA sig.

Levente Uzonyi-2

Re: Process-specific state broken and uncomplete

In reply to this post by Igor Stasenko

On Tue, 9 Nov 2010, Igor Stasenko wrote:

snip

> Suppose you have a domain object which provides following ways to
> access the session:
>
> session: forkedSession
> Processor activeProcess environmentAt: #session put: forkedSession
>
> session
> ^ Processor activeProcess environmentAt: #session
>
>
> now, here what i'd like to write:
>
> | session |
>
> session := MySession new .
> session initialize blabla.
>
> "good fork"
> good := [
> self session: session.
> [ self process stuff ] ensure: [ session close ]
> ] newProcess.
>
> good resume.
> 1 minute asDelay wait.
> good terminate.
> self assert: (session isClosed).
>
> In the above case, everything is ok: session object is passed in 'session' temp
> and visible inside ensure block directly, so, no matter what may
> happen (process termination),
> it is guaranteed that session will be closed at the end.
>
> "bad fork"
>
> session := MySession new .
> session initialize blabla.
>
> bad := [
> self session: session.
> [ self process stuff ] ensure: [ self session close ]
> ] newProcess.
>
> bad resume.
> 1 minute asDelay wait.
> bad terminate.
> self assert: (session isClosed).
>
>
> In this case, a session is accessed indirectly, through process specific state.
> And test might fail or not, depending if process was already
> terminated before sending #terminate,
> or its still running.
>
> The 'bad fork' illustrates that one should not attempt to access a
> process specific state in ensure blocks,
> because during process termination, stack unwinding is performed by
> other process, which could have different session object,
> or don't have it at all.
>
> This is simple, of course, to implement session finalization without
> using process spefic state. But now consider that session has not only
> #close method, but allows doing something more userful, like:
>
> self session backupStuff.
> [ .. do something.. ] ensure: [ self session restoreStuff ].
>
> And again, same problem: unless you have a direct reference to session
> object, you can't do it.
> Not with using process-specific state, because it is flawed.
>
> In other words, anything which may want to access to a
> process-specific state, like:
>
> [ ... ... ] ensure: [ (Processor activeProcess environmentAt: #foo)
> doSomething ]
>
> have no guarantees to work correctly in all cases.
>
> This defeats a purpose of process-specific state, because you can't
> rely on it for building a reliable stuff.

Not really. You can store the sessions in a session store which is
responsible for retrieving the session:

[
sessionStore useSession: sessionKey during: [ :session |
Processor activeProcess environmentAt: #session put: session.
self doSomething.
Processor activeProcess environmentAt: #session put: nil ] ] fork.

SessionStore >> useSession: sessionKey during: aBlock

| session |
[
session := self sessionFor: sessionKey.
aBlock value: session ]
ensure: [
session ifNotNil: [
self releaseSession: session ] ]

In #doSomething you can access the session via the process. We're using
this pattern in PostgresV3's connection pool. It works pretty well.

Levente

snip

Levente Uzonyi-2

Re: [Pharo-project] [squeak-dev] Re: Process-specific state broken and uncomplete

In reply to this post by Igor Stasenko

On Tue, 9 Nov 2010, Igor Stasenko wrote:

> On 9 November 2010 03:31, Levente Uzonyi <[hidden email]> wrote:

snip

>>
>> So they integrated the full ProcessSpecific package. Is it a problem that
>> you have to load it into Squeak?
>>
>>
> So, its a separate package, except that ivars already there..

Yes, it's on SqueakSource.

> Strange choice. Maybe it worth mentioning then in Process comment that
> these vars are there for purpose?
> Otherwise someone may attempt to remove them :)

Good idea.

Levente

snip

Igor Stasenko

Re: Process-specific state broken and uncomplete

In reply to this post by Levente Uzonyi-2

On 9 November 2010 05:59, Levente Uzonyi <[hidden email]> wrote:

> On Tue, 9 Nov 2010, Igor Stasenko wrote:
>
> snip
>
>> Suppose you have a domain object which provides following ways to
>> access the session:
>>
>> session: forkedSession
>> Processor activeProcess environmentAt: #session put: forkedSession
>>
>> session
>> ^ Processor activeProcess environmentAt: #session
>>
>>
>> now, here what i'd like to write:
>>
>> | session |
>>
>> session := MySession new .
>> session initialize blabla.
>>
>> "good fork"
>> good := [
>> self session: session.
>> [ self process stuff ] ensure: [ session close ]
>> ] newProcess.
>>
>> good resume.
>> 1 minute asDelay wait.
>> good terminate.
>> self assert: (session isClosed).
>>
>> In the above case, everything is ok: session object is passed in 'session'
>> temp
>> and visible inside ensure block directly, so, no matter what may
>> happen (process termination),
>> it is guaranteed that session will be closed at the end.
>>
>> "bad fork"
>>
>> session := MySession new .
>> session initialize blabla.
>>
>> bad := [
>> self session: session.
>> [ self process stuff ] ensure: [ self session close ]
>> ] newProcess.
>>
>> bad resume.
>> 1 minute asDelay wait.
>> bad terminate.
>> self assert: (session isClosed).
>>
>>
>> In this case, a session is accessed indirectly, through process specific
>> state.
>> And test might fail or not, depending if process was already
>> terminated before sending #terminate,
>> or its still running.
>>
>> The 'bad fork' illustrates that one should not attempt to access a
>> process specific state in ensure blocks,
>> because during process termination, stack unwinding is performed by
>> other process, which could have different session object,
>> or don't have it at all.
>>
>> This is simple, of course, to implement session finalization without
>> using process spefic state. But now consider that session has not only
>> #close method, but allows doing something more userful, like:
>>
>> self session backupStuff.
>> [ .. do something.. ] ensure: [ self session restoreStuff ].
>>
>> And again, same problem: unless you have a direct reference to session
>> object, you can't do it.
>> Not with using process-specific state, because it is flawed.
>>
>> In other words, anything which may want to access to a
>> process-specific state, like:
>>
>> [ ... ... ] ensure: [ (Processor activeProcess environmentAt: #foo)
>> doSomething ]
>>
>> have no guarantees to work correctly in all cases.
>>
>> This defeats a purpose of process-specific state, because you can't
>> rely on it for building a reliable stuff.
>
> Not really. You can store the sessions in a session store which is
> responsible for retrieving the session:
>
> [
> sessionStore useSession: sessionKey during: [ :session |
> Processor activeProcess environmentAt: #session put: session.
> self doSomething.
> Processor activeProcess environmentAt: #session put: nil ] ]
> fork.
>
> SessionStore >> useSession: sessionKey during: aBlock
>
> | session |
> [
> session := self sessionFor: sessionKey.
> aBlock value: session ]
> ensure: [
> session ifNotNil: [
> self releaseSession: session ] ]
>
> In #doSomething you can access the session via the process. We're using this
> pattern in PostgresV3's connection pool. It works pretty well.
>

No, you can't. I feel sorry for you :)

A #useSession:during: using ensure, and of course it will work
correctly, because you not accessing session via
process-specific storage. This is illustrated by my "good fork" example.

now replace #doSomething with this:

[ 1 year asDelay wait ] ensure: [
self assert: (Processor activeProcess environmentAt: #session) notNil ]

and then terminate a process which running your fork.

In attachment you'll find a test, which shows the problem (i using
tests from Pharo image).

>
> Levente
>
>
> snip
>
>

--
Best regards,
Igor Stasenko AKA sig.

ProcessSpecificTest-testLocalVariableInEnsureBlock.st (692 bytes) Download Attachment

Levente Uzonyi-2

Re: [Pharo-project] [squeak-dev] Re: Process-specific state broken and uncomplete

On Tue, 9 Nov 2010, Igor Stasenko wrote:

snip

>
> No, you can't. I feel sorry for you :)
>
> A #useSession:during: using ensure, and of course it will work
> correctly, because you not accessing session via
> process-specific storage. This is illustrated by my "good fork" example.

Erm, no. In both of your examples the session wasn't aquired by the forked
process. It was available via a variable. If you aquire the session _in_
the forked process _from_ the session store, then the #ensure: block in
the session store will be able to retrieve it.

>
> now replace #doSomething with this:
>
> [ 1 year asDelay wait ] ensure: [
> self assert: (Processor activeProcess environmentAt: #session) notNil ]
>
> and then terminate a process which running your fork.

Replaced (in theory) and terminated. What will happen? This assertion in
this #ensure: block will fail, but the session will be retrieved properly
by the session store's unwind block.

So you can use process local variables, but you shouldn't access them
during unwinding (#ensure:, #ifCurtailed:, whatever). This is possible as
I showed it in my example.

>
> In attachment you'll find a test, which shows the problem (i using
> tests from Pharo image).

The test fails, because you're accessing a process local variable in an
unwind block, which you shouldn't do.

Levente

>
>
>>
>> Levente
>>
>>
>> snip
>>
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>

Igor Stasenko

Re: [Pharo-project] [squeak-dev] Re: Process-specific state broken and uncomplete

On 9 November 2010 06:42, Levente Uzonyi <[hidden email]> wrote:

> On Tue, 9 Nov 2010, Igor Stasenko wrote:
>
> snip
>
>>
>> No, you can't. I feel sorry for you :)
>>
>> A #useSession:during: using ensure, and of course it will work
>> correctly, because you not accessing session via
>> process-specific storage. This is illustrated by my "good fork" example.
>
> Erm, no. In both of your examples the session wasn't aquired by the forked
> process. It was available via a variable. If you aquire the session _in_ the
> forked process _from_ the session store, then the #ensure: block in the
> session store will be able to retrieve it.
>

this doesn't changes anything. The point is that once you accessing
_any_ process-specific state in ensure
block, you're in trouble. And of course, if you capturing such state
before entering ensure block, you are safe.
But this not always an option.

>>
>> now replace #doSomething with this:
>>
>> [ 1 year asDelay wait ] ensure: [
>> self assert: (Processor activeProcess environmentAt: #session) notNil ]
>>
>> and then terminate a process which running your fork.
>
> Replaced (in theory) and terminated. What will happen? This assertion in
> this #ensure: block will fail, but the session will be retrieved properly by
> the session store's unwind block.
>
> So you can use process local variables, but you shouldn't access them during
> unwinding (#ensure:, #ifCurtailed:, whatever). This is possible as I showed
> it in my example.
>

i didn't said this is not possible.

>>
>> In attachment you'll find a test, which shows the problem (i using
>> tests from Pharo image).
>
> The test fails, because you're accessing a process local variable in an
> unwind block, which you shouldn't do.
>

Why i shouldn't?
Give me the reasons why. ("because it works that way" is not a good answer ;)
It sounds like you feel completely happy with current status quo, and
don't want to address that problem.

I don't know how else to convince you that this is wrong.
Suppose you wrote a library, which using a process-local state, buried
somewhere deeply.
Users of your library might even be not aware of it, because they will
be using a higher-level api.
A developer who writes own code, sure thing, thinks that he is free to
use #ensure: #ifCurtailed:
whereever he needs to, isnt?
But with stuff like that, you putting him in the position, that he
should be aware of such subtle nuances, and be very careful about
using your api inside ensure blocks.

This is really beyond my understanding. Why we should not provide a
consistent API instead?

>
> Levente
>

--
Best regards,
Igor Stasenko AKA sig.

Igor Stasenko

Re: [Pharo-project] Process-specific state broken and uncomplete

In reply to this post by Igor Stasenko

On 9 November 2010 08:06, Philippe Marschall <[hidden email]> wrote:

> On 08.11.2010 06:39, Igor Stasenko wrote:
>> Hello,
>>
>> i just found that Squeak images Process class contains unused ivars:
>> island env
>>
>> in Pharo image, env ivar is used for holding a process-specific state,
>> which is convenient, so one could use:
>> Processor activeProcess environmentAt: put:...
>>
>> but in Squeak image there is even no such methods.
>>
>> There are also some caveats with such thing:
>> during process termination, Processor activeProcess could point to
>> different process,
>> and so, accessing to process environment may lead to error(s).
>>
>> To avoid issues like these, the process termination action (stack
>> unwinding) should be always performed only for active process.
>> One of a trick i think, is to replace the suspendedContext with own
>> context, and then activate given process in order to unwind its stack
>> etc.
>>
>> Another way is to remember environment somewhere else during process
>> termination, and use different method to access it i.e. instead of:
>> Processor activeProcess environmentAt:
>> use
>> self processEnvironmentAt:
>>
>> where #processEnvironmentAt: is implemented in Object class, and can
>> handle termination gracefully.
>>
>> But i think replacing suspendedContext is more generic.
>> What you think?
>
> Not so convinced. Fast thread locals are important and a Symbol lookup
> in an IdentityDictionary is pretty fast. And polluting Object with stuff
> that doesn't belong there isn't a very convincing alternative to me.
>

does that means that you prefer to replace suspendedContext?

> Cheers
> PHilippe
>
>
>

--
Best regards,
Igor Stasenko AKA sig.

CdAB63

Re: Problems while installing TweakCore

In reply to this post by Andreas.Raab

Em 08-11-2010 17:23, Andreas Raab escreveu:

Try it now. The method directoryContentsFor:do: was added to 4.2 and created a conflict when loading TweakCore. I've removed it from Tweak so hopefully it will work now.

Cheers,
- Andreas

There are problems. It seems that UnixFileDirectory(FileDirectory)>>directoryContentsFor:do: still makes use of old form as seem in:

UnixFileDirectory>>directoryContentsFor: fullPath do: aBlock
    "Return a collection of directory entries for the files and directories in the directory with the given path. See primLookupEntryIn:index: for further details."
    "FileDirectory default directoryContentsFor: ''"

    | index done entryArray path |
    index := 1.
    done := false.
    path :=     fullPath asVmPathName.
    [done] whileFalse: [
        entryArray := self primLookupEntryIn: path index: index.
        #badDirectoryPath = entryArray ifTrue: [
            ^(InvalidDirectoryError pathName: pathName) signal].
        entryArray == nil
            ifTrue: [done := true]
            ifFalse: [aBlock value: (DirectoryEntry fromArray: entryArray)].
        index := index + 1].

signature.asc (269 bytes) Download Attachment