Smalltalk › Squeak › Squeak - Dev

Ensuring canvas safety (using canvas by multiple processes)

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

14 messages Options

Igor Stasenko

Ensuring canvas safety (using canvas by multiple processes)

The problem with OpenGL is, that GL context state is not a simple
thing, which can be switched quickly.
The drawing pipeline could be very complex, and if you allow multiple
processes to issue drawing commands, it almost guaranteed that you
will break things.

Let me illustrate a problem.
Suppose you have a Device, which provides a canvas which can be used
to draw on it.

canvas := Device getCanvas. " device at 'safe' state here "
myVisuals drawThingsUsing: canvas. "device is not safe during drawing "
" we finished drawing, now we are safe "

.. the main problem, is that when you received a canvas instance you
can free to do something like:

canvas := Device getCanvas.
1 to: 10 do: [:i | [ (self at: i) drawOn: canvas ] fork ].

and at this point you are not safe anymore. You can easily break
Device state, if multiple processes will try issue different drawing
commands using canvas.

I really don't like putting semaphores everywhere. It will be a
performance killer.

One way to isolate things, is to provide protocols like:

Device drawExclusively: [:canvas | .. drawing code here .. ].

but again, this is not guarantees that, developer will not use
received canvas reference to do nasty things..
Do an active process check in all methods of canvas?
Any ideas?

--
Best regards,
Igor Stasenko AKA sig.

Michael van der Gulik-2

Re: Ensuring canvas safety (using canvas by multiple processes)

On Feb 12, 2008 9:06 PM, Igor Stasenko <[hidden email]> wrote:

The problem with OpenGL is, that GL context state is not a simple
thing, which can be switched quickly.
The drawing pipeline could be very complex, and if you allow multiple
processes to issue drawing commands, it almost guaranteed that you
will break things.

Let me illustrate a problem.
Suppose you have a Device, which provides a canvas which can be used
to draw on it.

canvas := Device getCanvas. " device at 'safe' state here "
myVisuals drawThingsUsing: canvas. "device is not safe during drawing "
" we finished drawing, now we are safe "

.. the main problem, is that when you received a canvas instance you
can free to do something like:

canvas := Device getCanvas.
1 to: 10 do: [:i | [ (self at: i) drawOn: canvas ] fork ].

and at this point you are not safe anymore. You can easily break
Device state, if multiple processes will try issue different drawing
commands using canvas.

I really don't like putting semaphores everywhere. It will be a
performance killer.

One way to isolate things, is to provide protocols like:

Device drawExclusively: [:canvas | .. drawing code here .. ].

but again, this is not guarantees that, developer will not use
received canvas reference to do nasty things..
Do an active process check in all methods of canvas?
Any ideas?

Lots, but it depends on what the problem actually is. Could you describe it in more detail?

One option is to modify Canvas (or a subclass) to have a getLock method which returns a Mutex (aka Semaphore) unique to that Canvas. Your code can then do "mutex critical: [...]" blocks to assure atomicity.

However, it would seem to me that the problem is with the user of the Canvas. With any canvas, you need to issue the drawing instructions in the right order to preserve the z-index of the elements added. It's the user who must make sure that it does not have two threads drawing in the same Rectangle concurrently.

Don't be sparing with the use of Semaphores. Correct code is better than fast code.

Gulik.

--
http://people.squeakfoundation.org/person/mikevdg
http://gulik.pbwiki.com/

Igor Stasenko

Re: Ensuring canvas safety (using canvas by multiple processes)

On 12/02/2008, Michael van der Gulik <[hidden email]> wrote:
>
>
> Lots, but it depends on what the problem actually is. Could you describe it
> in more detail?
>
well, when you issuing a command like:

canvas translateBy: offset during: [ ... ].

canvas does following:

gl pushMatrix.
gl translateBy: offset.
aBlock value.
gl popMatrix.

operations with matrix affecting global state, if you try to draw
anything in parallel process, while in current process you evaluating
a block, you will be screwed up.

Another issue is with using glBegin/glEnd pair. These commands can't
be nested, also a number of valid GL operations inside glBegin/glEnd
are limited.

In general, any code, that doing like:

gl changeSomeState.
..some code..
gl revertToPreviousState.

is potentially leading to nirvana, if you can't guarantee a proper
order of commands, issued to OpenGL.

> One option is to modify Canvas (or a subclass) to have a getLock method
> which returns a Mutex (aka Semaphore) unique to that Canvas. Your code can
> then do "mutex critical: [...]" blocks to assure atomicity.
>
> However, it would seem to me that the problem is with the user of the
> Canvas. With any canvas, you need to issue the drawing instructions in the
> right order to preserve the z-index of the elements added. It's the user who
> must make sure that it does not have two threads drawing in the same
> Rectangle concurrently.
>
> Don't be sparing with the use of Semaphores. Correct code is better than
> fast code.

That's what i fear most. Adding semaphores will kill performance :)
In C, i can simply put a canvas var in thread-local storage, so it's
value will be unique for each thread of execution.
Interesting, is something like this can be done for squeak?

So, i can write something like:

object := Processor threadedVariable. "should it be a Smalltalk's method?"
object value: (Array new:5).

object value "should return array with 5 elements"
[ object value ] fork. "object value should return nil for new
process, since it's not initialized to anything"

if properly implemented, a #value method can be very fast (w/o using
dictionaries or sets).
For instance, by adding a single variable to process , where i can
hold references to these threaded-vars, and threadedVariable then will
hold a slot index. Then #value can be:

value
^ Processor currentProcess slotAt: slotNum

Process slotAt: num
^ slots at: num ifAbsent: [nil].

--
Best regards,
Igor Stasenko AKA sig.

Igor Stasenko

Re: Ensuring canvas safety (using canvas by multiple processes)

Btw, this problem concerns not only OpenGL canvas implementation.
Even with bitblt, some operations are not thread-safe.
And in general, what mechanisms you planning to add to SecureSqueak to
guarantee that some code will get exclusive access to functions of
some device?

For instance, try:
10 timesRepeat: [
[ Smalltalk logChange: 'say goodbye to' , 1 seconds asDelay wait
asString, ' your .changes file' ] fork.
]

--
Best regards,
Igor Stasenko AKA sig.

Klaus D. Witzel

Re: Ensuring canvas safety (using canvas by multiple processes)

In reply to this post by Igor Stasenko

Hi Igor,

how long do you want your per-process variable to live? What do you want
to do when it gets corrupted (incomplete operations due to DNU etc)?

You might want to check (as yet not used it)

- http://www.squeaksource.com/ProcessLocalStorage.html

And Seaside's (self session) variable is also an interesting solution,
AFAIK cross-Smalltalk-dialect :)

/Klaus

On Tue, 12 Feb 2008 10:23:36 +0100, Igor wrote:

> On 12/02/2008, Michael van der Gulik wrote:
>>
>>
>> Lots, but it depends on what the problem actually is. Could you
>> describe it
>> in more detail?
>>
> well, when you issuing a command like:
>
> canvas translateBy: offset during: [ ... ].
>
> canvas does following:
>
> gl pushMatrix.
> gl translateBy: offset.
> aBlock value.
> gl popMatrix.
>
> operations with matrix affecting global state, if you try to draw
> anything in parallel process, while in current process you evaluating
> a block, you will be screwed up.
>
> Another issue is with using glBegin/glEnd pair. These commands can't
> be nested, also a number of valid GL operations inside glBegin/glEnd
> are limited.
>
> In general, any code, that doing like:
>
> gl changeSomeState.
> ..some code..
> gl revertToPreviousState.
>
> is potentially leading to nirvana, if you can't guarantee a proper
> order of commands, issued to OpenGL.
>
>> One option is to modify Canvas (or a subclass) to have a getLock method
>> which returns a Mutex (aka Semaphore) unique to that Canvas. Your code
>> can
>> then do "mutex critical: [...]" blocks to assure atomicity.
>>
>> However, it would seem to me that the problem is with the user of the
>> Canvas. With any canvas, you need to issue the drawing instructions in
>> the
>> right order to preserve the z-index of the elements added. It's the
>> user who
>> must make sure that it does not have two threads drawing in the same
>> Rectangle concurrently.
>>
>> Don't be sparing with the use of Semaphores. Correct code is better than
>> fast code.
>
> That's what i fear most. Adding semaphores will kill performance :)
> In C, i can simply put a canvas var in thread-local storage, so it's
> value will be unique for each thread of execution.
> Interesting, is something like this can be done for squeak?
>
> So, i can write something like:
>
> object := Processor threadedVariable. "should it be a Smalltalk's
> method?"
> object value: (Array new:5).
>
> object value "should return array with 5 elements"
> [ object value ] fork. "object value should return nil for new
> process, since it's not initialized to anything"
>
> if properly implemented, a #value method can be very fast (w/o using
> dictionaries or sets).
> For instance, by adding a single variable to process , where i can
> hold references to these threaded-vars, and threadedVariable then will
> hold a slot index. Then #value can be:
>
> value
> ^ Processor currentProcess slotAt: slotNum
>
> Process slotAt: num
> ^ slots at: num ifAbsent: [nil].
>

Igor Stasenko

Re: Ensuring canvas safety (using canvas by multiple processes)

In reply to this post by Igor Stasenko

On 12/02/2008, Igor Stasenko <[hidden email]> wrote:

> Btw, this problem concerns not only OpenGL canvas implementation.
> Even with bitblt, some operations are not thread-safe.
> And in general, what mechanisms you planning to add to SecureSqueak to
> guarantee that some code will get exclusive access to functions of
> some device?
>
> For instance, try:
> 10 timesRepeat: [
> [ Smalltalk logChange: 'say goodbye to' , 1 seconds asDelay wait
> asString, ' your .changes file' ] fork.
> ]
>

Oh, it seems not breaking things, but i think you understand what i'm
talking about :)

--
Best regards,
Igor Stasenko AKA sig.

Igor Stasenko

Re: Ensuring canvas safety (using canvas by multiple processes)

In reply to this post by Klaus D. Witzel

On 12/02/2008, Klaus D. Witzel <[hidden email]> wrote:
> Hi Igor,
>
> how long do you want your per-process variable to live? What do you want
> to do when it gets corrupted (incomplete operations due to DNU etc)?
>
well, a per-process variable can simply check it's availability, and
free slot (in #finalize), if it's not longer in use.

> You might want to check (as yet not used it)
>
> - http://www.squeaksource.com/ProcessLocalStorage.html
>
> And Seaside's (self session) variable is also an interesting solution,
> AFAIK cross-Smalltalk-dialect :)
>
Tried to load this package , with errors :)
My MC don't see 'env' variable in Process. Ah, it's simply can't track
such changes as extension.

> /Klaus
>
> On Tue, 12 Feb 2008 10:23:36 +0100, Igor wrote:
>
> > On 12/02/2008, Michael van der Gulik wrote:
> >>
> >>
> >> Lots, but it depends on what the problem actually is. Could you
> >> describe it
> >> in more detail?
> >>
> > well, when you issuing a command like:
> >
> > canvas translateBy: offset during: [ ... ].
> >
> > canvas does following:
> >
> > gl pushMatrix.
> > gl translateBy: offset.
> > aBlock value.
> > gl popMatrix.
> >
> > operations with matrix affecting global state, if you try to draw
> > anything in parallel process, while in current process you evaluating
> > a block, you will be screwed up.
> >
> > Another issue is with using glBegin/glEnd pair. These commands can't
> > be nested, also a number of valid GL operations inside glBegin/glEnd
> > are limited.
> >
> > In general, any code, that doing like:
> >
> > gl changeSomeState.
> > ..some code..
> > gl revertToPreviousState.
> >
> > is potentially leading to nirvana, if you can't guarantee a proper
> > order of commands, issued to OpenGL.
> >
> >> One option is to modify Canvas (or a subclass) to have a getLock method
> >> which returns a Mutex (aka Semaphore) unique to that Canvas. Your code
> >> can
> >> then do "mutex critical: [...]" blocks to assure atomicity.
> >>
> >> However, it would seem to me that the problem is with the user of the
> >> Canvas. With any canvas, you need to issue the drawing instructions in
> >> the
> >> right order to preserve the z-index of the elements added. It's the
> >> user who
> >> must make sure that it does not have two threads drawing in the same
> >> Rectangle concurrently.
> >>
> >> Don't be sparing with the use of Semaphores. Correct code is better than
> >> fast code.
> >
> > That's what i fear most. Adding semaphores will kill performance :)
> > In C, i can simply put a canvas var in thread-local storage, so it's
> > value will be unique for each thread of execution.
> > Interesting, is something like this can be done for squeak?
> >
> > So, i can write something like:
> >
> > object := Processor threadedVariable. "should it be a Smalltalk's
> > method?"
> > object value: (Array new:5).
> >
> > object value "should return array with 5 elements"
> > [ object value ] fork. "object value should return nil for new
> > process, since it's not initialized to anything"
> >
> > if properly implemented, a #value method can be very fast (w/o using
> > dictionaries or sets).
> > For instance, by adding a single variable to process , where i can
> > hold references to these threaded-vars, and threadedVariable then will
> > hold a slot index. Then #value can be:
> >
> > value
> > ^ Processor currentProcess slotAt: slotNum
> >
> > Process slotAt: num
> > ^ slots at: num ifAbsent: [nil].
> >
>
>
>
>

--
Best regards,
Igor Stasenko AKA sig.

Michael van der Gulik-2

Re: Ensuring canvas safety (using canvas by multiple processes)

In reply to this post by Igor Stasenko

On Feb 12, 2008 10:23 PM, Igor Stasenko <[hidden email]> wrote:

On 12/02/2008, Michael van der Gulik <[hidden email]> wrote:
>
>
> Lots, but it depends on what the problem actually is. Could you describe it
> in more detail?
>
well, when you issuing a command like:

canvas translateBy: offset during: [ ... ].

canvas does following:

gl pushMatrix.
gl translateBy: offset.
aBlock value.
gl popMatrix.

operations with matrix affecting global state, if you try to draw
anything in parallel process, while in current process you evaluating
a block, you will be screwed up.

Another issue is with using glBegin/glEnd pair. These commands can't
be nested, also a number of valid GL operations inside glBegin/glEnd
are limited.

In general, any code, that doing like:

gl changeSomeState.
..some code..
gl revertToPreviousState.

is potentially leading to nirvana, if you can't guarantee a proper
order of commands, issued to OpenGL.

I thought being in nirvana was meant to be a good thing?

Anyway, I understand what you mean now. This looks like an implementation issue of your canvas, so the thread protection should be encapsulated inside it.

I.e.

myMutex critical: [
    gl changeSomeState.
    ..some code..
    gl revertToPreviousState.
].

Just make sure that you keep the critical sections as small as possible, and don't call any outside code (including blocks).

I don't think there is a large speed issue here by using Semaphores. If you are worried about speed, you could have two canvases: a reentrant one and a non-reentrant one. Somehow they could reuse code, but I'll leave this as an exercise for you.

Gulik.

--
http://people.squeakfoundation.org/person/mikevdg
http://gulik.pbwiki.com/

Igor Stasenko

Re: Ensuring canvas safety (using canvas by multiple processes)

On 12/02/2008, Michael van der Gulik <[hidden email]> wrote:

>
>
> On Feb 12, 2008 10:23 PM, Igor Stasenko <[hidden email]> wrote:
> >
> > On 12/02/2008, Michael van der Gulik <[hidden email]> wrote:
> > >
> > >
> > > Lots, but it depends on what the problem actually is. Could you describe
> it
> > > in more detail?
> > >
> > well, when you issuing a command like:
> >
> > canvas translateBy: offset during: [ ... ].
> >
> > canvas does following:
> >
> > gl pushMatrix.
> > gl translateBy: offset.
> > aBlock value.
> > gl popMatrix.
> >
> > operations with matrix affecting global state, if you try to draw
> > anything in parallel process, while in current process you evaluating
> > a block, you will be screwed up.
> >
> > Another issue is with using glBegin/glEnd pair. These commands can't
> > be nested, also a number of valid GL operations inside glBegin/glEnd
> > are limited.
> >
> > In general, any code, that doing like:
> >
> > gl changeSomeState.
> > ..some code..
> > gl revertToPreviousState.
> >
> > is potentially leading to nirvana, if you can't guarantee a proper
> > order of commands, issued to OpenGL.
> >
> >
>
>
> I thought being in nirvana was meant to be a good thing?
>

There are many gradations of this state :)

> Anyway, I understand what you mean now. This looks like an implementation
> issue of your canvas, so the thread protection should be encapsulated inside
> it.
>
> I.e.
>
> myMutex critical: [
> gl changeSomeState.
> ..some code..
> gl revertToPreviousState.
> ].
>

It's still unsafe. Tell you why:

myMutex critical: [
gl changeSomeState.
aBlock value.
gl revertToPreviousState.
].

while block still can contain evil forks...

> Just make sure that you keep the critical sections as small as possible, and
> don't call any outside code (including blocks).
>
> I don't think there is a large speed issue here by using Semaphores. If you
> are worried about speed, you could have two canvases: a reentrant one and a
> non-reentrant one. Somehow they could reuse code, but I'll leave this as an
> exercise for you.
>

Well, i'm looking for a golden balance between safety and speed. I
don't need a bullet-proof system, just a dumb-proof one :)

>
> Gulik.
>
> --
> http://people.squeakfoundation.org/person/mikevdg
> http://gulik.pbwiki.com/
>

--
Best regards,
Igor Stasenko AKA sig.

Michael van der Gulik-2

Re: Ensuring canvas safety (using canvas by multiple processes)

In reply to this post by Igor Stasenko

On Feb 12, 2008 10:38 PM, Igor Stasenko <[hidden email]> wrote:

Btw, this problem concerns not only OpenGL canvas implementation.
Even with bitblt, some operations are not thread-safe.
And in general, what mechanisms you planning to add to SecureSqueak to
guarantee that some code will get exclusive access to functions of
some device?

For instance, try:
10 timesRepeat: [
[ Smalltalk logChange: 'say goodbye to' , 1 seconds asDelay wait
asString, ' your .changes file' ] fork.
]

The above is much more effectively achieved by starting up an image twice and saving code in each :-).

I'll have to handle each situation as I find them. Generally, I'd have to make sure all APIs are thread safe. Also, remember that with Namespaces, an object only has access to a very limited set of other objects (in theory at least).

In the specific case of Canvas, I'll be using something which I still have to give a good name to -- maybe Gate, Proxy, Interface, or Valve or something. It will be an object that implements the public sub-set of Canvas's methods and forwards messages on for a particular clipping Rectangle on that Canvas. When permission to that Canvas is no longer required, the connection is broken and that Valve/Gate/Interface becomes useless. I'm sure E-lang has a good name for these; its a pattern from programming with capabilities.

Gulik.

--
http://people.squeakfoundation.org/person/mikevdg
http://gulik.pbwiki.com/

Michael van der Gulik-2

Re: Ensuring canvas safety (using canvas by multiple processes)

In reply to this post by Igor Stasenko

On Feb 12, 2008 11:01 PM, Igor Stasenko <[hidden email]> wrote:

> Anyway, I understand what you mean now. This looks like an implementation
> issue of your canvas, so the thread protection should be encapsulated inside
> it.
>
> I.e.
>
> myMutex critical: [
> gl changeSomeState.
> ..some code..
> gl revertToPreviousState.
> ].
>

It's still unsafe. Tell you why:

myMutex critical: [
gl changeSomeState.
aBlock value.
gl revertToPreviousState.
].

while block still can contain evil forks...

Yea, well, don't evaluate untrusted blocks in critical regions :-).

And forks aren't evil. They're quite nice, really; try doing the same in Java or C!

Gulik.

--
http://people.squeakfoundation.org/person/mikevdg
http://gulik.pbwiki.com/

Igor Stasenko

Re: Ensuring canvas safety (using canvas by multiple processes)

On 12/02/2008, Michael van der Gulik <[hidden email]> wrote:

>
>
> On Feb 12, 2008 11:01 PM, Igor Stasenko <[hidden email]> wrote:
> >
> >
> > > Anyway, I understand what you mean now. This looks like an
> implementation
> > > issue of your canvas, so the thread protection should be encapsulated
> inside
> > > it.
> > >
> > > I.e.
> > >
> > > myMutex critical: [
> > > gl changeSomeState.
> > > ..some code..
> > > gl revertToPreviousState.
> > > ].
> > >
> >
> > It's still unsafe. Tell you why:
> >
> > myMutex critical: [
> > gl changeSomeState.
> > aBlock value.
> > gl revertToPreviousState.
> > ].
> >
> > while block still can contain evil forks...
> >
> >
> >
> >
>
>
> Yea, well, don't evaluate untrusted blocks in critical regions :-).
>
> And forks aren't evil. They're quite nice, really; try doing the same in
> Java or C!
>

Sure.

I think, best fit for such needs, i think, is using a proxy, which
value depends on active process.
It's really a most safer approach (since you can't bypass proxy when
interacting with canvas).
The bad, is that proxies are slooow :(

>
> Gulik.
>

--
Best regards,
Igor Stasenko AKA sig.

keith1y

Re: Ensuring canvas safety (using canvas by multiple processes)

In reply to this post by Klaus D. Witzel

Klaus D. Witzel wrote:
> Hi Igor,
>
> how long do you want your per-process variable to live? What do you
> want to do when it gets corrupted (incomplete operations due to DNU etc)?
>
> You might want to check (as yet not used it)
>
> - http://www.squeaksource.com/ProcessLocalStorage.html
>
Hi,

there is a version in http://www.squeaksource.com/Logging with some
additional features. I particularly liked the ability to swap
DateAndTime for a different one for a single thread, so as to have a
clock that runs in slowmotion or backwards. I wondered whether doing the
same thing for "Smalltalk" would give us a "poor-mans-namespaces" solution.

Keith

Jason Johnson-5

[squeak-dev] Re: Ensuring canvas safety (using canvas by multiple processes)

In reply to this post by Michael van der Gulik-2

On Tue, Feb 12, 2008 at 9:37 AM, Michael van der Gulik
<[hidden email]> wrote:
>
> Don't be sparing with the use of Semaphores. Correct code is better than
> fast code.

Well, the screen is a resource and IMO it's best to protect a resource
by a process, not a bunch of messy Mutex logic. Especially something
like the screen that has to be centrally managed anyway (i.e. a window
manager).

I see that Igor made a comment below about proxies being "slow", but
the thing is, if you make a "proxy", "gate", "driver" or whatever you
want to call it, you make the DSL for how to interact with it so it
doesn't have to be slow. If you make the driver just take OpenGL
codes and apply them then it probably will be slow, but for example X
apparently saw that Font handling was slow so they make applications
define fonts ahead of time and thereafter all font references just
pass the number/name received for this initial font setup (and I think
the font setups are read-only and can be shared behind the scenes
between unrelated applications).