Smalltalk › Squeak › Squeak VM

Fix on management of events in OSX

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

8 messages Options

Guillermo Polito

Fix on management of events in OSX

Hi all,

While playing with SDL on OSX we have noticed that the main window in the VM is randomly eating events that belong to other windows. This happens because we are consuming events regardless the window they come from.

The corresponding code is in the method pumpRunLoopEventSendAndSignal: in

https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/iOS/vm/OSX/sqSqueakOSXApplication%2Bevents.m

I've playing around with making the VM consume only events that come from its own window or from no window (window=null).

This seems to work correctly, and I've had run into no issues in the last couple of hours.

I've also prepared a PR with the associated changes in here: https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/295.

I've put a guard so my code is only valid for PHAROVM builds, but I'd gladly remove the guards if people think this is a good move for all the community.

Thanks,

Guille

Eliot Miranda-2

Re: Fix on management of events in OSX

Hi Guille,

On Oct 22, 2018, at 3:31 AM, Guillermo Polito <[hidden email]> wrote:

Hi all,

While playing with SDL on OSX we have noticed that the main window in the VM is randomly eating events that belong to other windows. This happens because we are consuming events regardless the window they come from.

The corresponding code is in the method pumpRunLoopEventSendAndSignal: in
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/iOS/vm/OSX/sqSqueakOSXApplication%2Bevents.m

I've playing around with making the VM consume only events that come from its own window or from no window (window=null).
This seems to work correctly, and I've had run into no issues in the last couple of hours.

I've also prepared a PR with the associated changes in here: https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/295.
I've put a guard so my code is only valid for PHAROVM builds, but I'd gladly remove the guards if people think this is a good move for all the community.

Yes please!

Thanks,
Guille

Guillermo Polito

Re: Fix on management of events in OSX

Hi,

I see my first PR got integrated too fast. I've made a new one this morning that removes the guard and adds some comments in the code.

Also, answering the questions that arised in the PR, I believe this should have no effect on systems using a single window.

However, with multiple windowing systems, this (my patch) makes the consumption of events a cooperative task between the windowing systems.

For example, if we manage to have a SDL window open but SDL does not consume its events (because we don't call the event processing routine), then those events stay in the queue for ever.

I'm still not happy with the solution but I can deal with it right now, and it should not affect anybody else...

To make it more robust I've been trying to

- Use a separate queue to store "alien" events and then repost them in the queue afterwards (https://developer.apple.com/documentation/appkit/nsapplication/1428710-postevent?language=objc).

But I could not manage to make it work yet, and Cocoa's support/documentation does really NOT help here... :)

- Maybe we can just discard "very old" events?

But on the other side this solution is maybe simple enough to do the job.

Comments are welcome ^^

Cheers,

Guille

On Mon, Oct 22, 2018 at 3:16 PM Eliot Miranda <[hidden email]> wrote:

Hi Guille,

On Oct 22, 2018, at 3:31 AM, Guillermo Polito <[hidden email]> wrote:

Hi all,

While playing with SDL on OSX we have noticed that the main window in the VM is randomly eating events that belong to other windows. This happens because we are consuming events regardless the window they come from.

The corresponding code is in the method pumpRunLoopEventSendAndSignal: in
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/iOS/vm/OSX/sqSqueakOSXApplication%2Bevents.m

I've playing around with making the VM consume only events that come from its own window or from no window (window=null).
This seems to work correctly, and I've had run into no issues in the last couple of hours.

I've also prepared a PR with the associated changes in here: https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/295.
I've put a guard so my code is only valid for PHAROVM builds, but I'd gladly remove the guards if people think this is a good move for all the community.

Yes please!

Thanks,
Guille

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: +33 06 52 70 66 13

Eliot Miranda-2

Lock Ups on macOS 10.13 [was Fix on management of events in OSX]

Hi Guille,

I'm noticing some lock ups, primarily after resolution changes, or long periods of not in interacting. I wonder whether there are a class of window events that the loop does not handle and if this produces a lock up, or rather a failure to notice that new events are incoming. I *think* I have a reproducible case on a MacBookPro whose display resolution is set to "More Space" (see the Displays setting tab).

Have a VM running on the laptop screen while the laptop is connected to an Apple Cinema display (in this configuration the laptop's retina display reverts to its default resolution).

Then disconnect the cinema display, at which point the MBP's retina display's resolution changes back to :"More Space" and, at least when I tried this morning, the VM locked up.

On Tue, Oct 23, 2018 at 2:08 AM Guillermo Polito <[hidden email]> wrote:

Hi,

I see my first PR got integrated too fast. I've made a new one this morning that removes the guard and adds some comments in the code.

Also, answering the questions that arised in the PR, I believe this should have no effect on systems using a single window.
However, with multiple windowing systems, this (my patch) makes the consumption of events a cooperative task between the windowing systems.
For example, if we manage to have a SDL window open but SDL does not consume its events (because we don't call the event processing routine), then those events stay in the queue for ever.

I'm still not happy with the solution but I can deal with it right now, and it should not affect anybody else...

To make it more robust I've been trying to
- Use a separate queue to store "alien" events and then repost them in the queue afterwards (https://developer.apple.com/documentation/appkit/nsapplication/1428710-postevent?language=objc).
But I could not manage to make it work yet, and Cocoa's support/documentation does really NOT help here... :)
- Maybe we can just discard "very old" events?

But on the other side this solution is maybe simple enough to do the job.

Comments are welcome ^^
Cheers,
Guille

On Mon, Oct 22, 2018 at 3:16 PM Eliot Miranda <[hidden email]> wrote:

Hi Guille,

On Oct 22, 2018, at 3:31 AM, Guillermo Polito <[hidden email]> wrote:

Hi all,

While playing with SDL on OSX we have noticed that the main window in the VM is randomly eating events that belong to other windows. This happens because we are consuming events regardless the window they come from.

The corresponding code is in the method pumpRunLoopEventSendAndSignal: in
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/iOS/vm/OSX/sqSqueakOSXApplication%2Bevents.m

I've playing around with making the VM consume only events that come from its own window or from no window (window=null).
This seems to work correctly, and I've had run into no issues in the last couple of hours.

I've also prepared a PR with the associated changes in here: https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/295.
I've put a guard so my code is only valid for PHAROVM builds, but I'd gladly remove the guards if people think this is a good move for all the community.

Yes please!

Thanks,
Guille

--

Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: +33 06 52 70 66 13

_,,,^..^,,,_

best, Eliot

Eliot Miranda-2

Re: Lock Ups on macOS 10.13 [was Fix on management of events in OSX]

Hi Guille, Hi John, Hi All,

On Tue, Oct 30, 2018 at 2:10 PM Eliot Miranda <[hidden email]> wrote:

Hi Guille,

I'm noticing some lock ups, primarily after resolution changes, or long periods of not in interacting. I wonder whether there are a class of window events that the loop does not handle and if this produces a lock up, or rather a failure to notice that new events are incoming. I *think* I have a reproducible case on a MacBookPro whose display resolution is set to "More Space" (see the Displays setting tab).
Have a VM running on the laptop screen while the laptop is connected to an Apple Cinema display (in this configuration the laptop's retina display reverts to its default resolution).
Then disconnect the cinema display, at which point the MBP's retina display's resolution changes back to :"More Space" and, at least when I tried this morning, the VM locked up.

Guille, I think you're off the hook. I edited the event loop to include:

if (!(event.window == 0 || event.window == gDelegateApp.window)){

printf("breaking with event.window = %lx (gDA.window = %lx)\n", event.window, gDelegateApp.window);

break;

}

but I see no print out when the lock up occurs. John, here's how to reproduce a lock up on macOS 10.13, MacBookPro with retina display:

1. set the display resolution to Default

2. launch an image (Pharo or Squeak)

3. set the display resolution to "More Space"

the image now no longer receives events, and the main window is not resizable. Restoring the display to Default does not unlock the system.

And the same lock up occurs in the opposite direction; set the resolution to "More Space", launch and change to "Default" and again the system, locks up.

On Tue, Oct 23, 2018 at 2:08 AM Guillermo Polito <[hidden email]> wrote:

Hi,

I see my first PR got integrated too fast. I've made a new one this morning that removes the guard and adds some comments in the code.

Also, answering the questions that arised in the PR, I believe this should have no effect on systems using a single window.
However, with multiple windowing systems, this (my patch) makes the consumption of events a cooperative task between the windowing systems.
For example, if we manage to have a SDL window open but SDL does not consume its events (because we don't call the event processing routine), then those events stay in the queue for ever.

I'm still not happy with the solution but I can deal with it right now, and it should not affect anybody else...

To make it more robust I've been trying to
- Use a separate queue to store "alien" events and then repost them in the queue afterwards (https://developer.apple.com/documentation/appkit/nsapplication/1428710-postevent?language=objc).
But I could not manage to make it work yet, and Cocoa's support/documentation does really NOT help here... :)
- Maybe we can just discard "very old" events?

But on the other side this solution is maybe simple enough to do the job.

Comments are welcome ^^
Cheers,
Guille

On Mon, Oct 22, 2018 at 3:16 PM Eliot Miranda <[hidden email]> wrote:

Hi Guille,

On Oct 22, 2018, at 3:31 AM, Guillermo Polito <[hidden email]> wrote:

Hi all,

While playing with SDL on OSX we have noticed that the main window in the VM is randomly eating events that belong to other windows. This happens because we are consuming events regardless the window they come from.

The corresponding code is in the method pumpRunLoopEventSendAndSignal: in
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/iOS/vm/OSX/sqSqueakOSXApplication%2Bevents.m

I've playing around with making the VM consume only events that come from its own window or from no window (window=null).
This seems to work correctly, and I've had run into no issues in the last couple of hours.

I've also prepared a PR with the associated changes in here: https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/295.
I've put a guard so my code is only valid for PHAROVM builds, but I'd gladly remove the guards if people think this is a good move for all the community.

Yes please!

Thanks,
Guille

--

Guille Polito
Research Engineer
Centre de Recherche en Informatique, Signal et Automatique de Lille
CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: +33 06 52 70 66 13

--
_,,,^..^,,,_
best, Eliot

_,,,^..^,,,_

best, Eliot

Ben Coman

Re: Lock Ups on macOS 10.13 [was Fix on management of events in OSX]

On Wed, 31 Oct 2018 at 05:48, Eliot Miranda <[hidden email]> wrote:

Hi Guille, Hi John, Hi All,
On Tue, Oct 30, 2018 at 2:10 PM Eliot Miranda <[hidden email]> wrote:
Hi Guille,

I'm noticing some lock ups, primarily after resolution changes, or long periods of not in interacting. I wonder whether there are a class of window events that the loop does not handle and if this produces a lock up, or rather a failure to notice that new events are incoming. I *think* I have a reproducible case on a MacBookPro whose display resolution is set to "More Space" (see the Displays setting tab).
Have a VM running on the laptop screen while the laptop is connected to an Apple Cinema display (in this configuration the laptop's retina display reverts to its default resolution).
Then disconnect the cinema display, at which point the MBP's retina display's resolution changes back to :"More Space" and, at least when I tried this morning, the VM locked up.

Guille, I think you're off the hook. I edited the event loop to include:

if (!(event.window == 0 || event.window == gDelegateApp.window)){
printf("breaking with event.window = %lx (gDA.window = %lx)\n", event.window, gDelegateApp.window);
break;
}

Random thought...

Perhaps when the screen resizes the window bounds are not resized, so for example while the mouse looks like its inside the window, the event coordinates end up outside the window, so the native event dispatching doesn't send the event to the window. It would be interesting to display the bounds-of-the-window together with the event-coordinates and observe how they interact when the screen resolution changes.

cheers -ben

Ben Coman

Re: Lock Ups on macOS 10.13 [was Fix on management of events in OSX]

On Thu, 1 Nov 2018 at 21:54, Ben Coman <[hidden email]> wrote:

On Wed, 31 Oct 2018 at 05:48, Eliot Miranda <[hidden email]> wrote:

Hi Guille, Hi John, Hi All,
On Tue, Oct 30, 2018 at 2:10 PM Eliot Miranda <[hidden email]> wrote:
Hi Guille,

I'm noticing some lock ups, primarily after resolution changes, or long periods of not in interacting. I wonder whether there are a class of window events that the loop does not handle and if this produces a lock up, or rather a failure to notice that new events are incoming. I *think* I have a reproducible case on a MacBookPro whose display resolution is set to "More Space" (see the Displays setting tab).
Have a VM running on the laptop screen while the laptop is connected to an Apple Cinema display (in this configuration the laptop's retina display reverts to its default resolution).
Then disconnect the cinema display, at which point the MBP's retina display's resolution changes back to :"More Space" and, at least when I tried this morning, the VM locked up.

Guille, I think you're off the hook. I edited the event loop to include:

if (!(event.window == 0 || event.window == gDelegateApp.window)){
printf("breaking with event.window = %lx (gDA.window = %lx)\n", event.window, gDelegateApp.window);
break;
}

Random thought...
Perhaps when the screen resizes the window bounds are not resized, so for example while the mouse looks like its inside the window, the event coordinates end up outside the window, so the native event dispatching doesn't send the event to the window. It would be interesting to display the bounds-of-the-window together with the event-coordinates and observe how they interact when the screen resolution changes.

Just some blind trawling of the net...

Returns the "backing scale factor" of the view's window

https://searchfox.org/mozilla-central/rev/37efe4d0e6d74152023e5e142f670e6a2cec478c/widget/cocoa/nsChildView.h#338-352

Get the backing scale factor from an object that supports this selector (NSView/Window/Screen, on 10.7 or later)

https://dxr.mozilla.org/mozilla-central/rev/681eb7dfa324dd50403c382888929ea8b8b11b00/widget/cocoa/nsCocoaUtils.h#120-130

viewDidChangeBackingProperties

https://searchfox.org/mozilla-central/rev/37efe4d0e6d74152023e5e142f670e6a2cec478c/widget/cocoa/nsChildView.h#113

Coding for high resolution on OS X? Read this.

http://supermegaultragroovy.com/2012/10/24/coding-for-high-resolution-on-os-x-read-this/

For windows with zero width or height, the backingScaleFactor method is broken

https://fossies.org/linux/firefox/widget/cocoa/nsCocoaWindow.mm

Apple's resizing code not working...

https://stackoverflow.com/questions/9204132/same-command-different-results-setframe-nsview

NSEvent & NSDeviceIndependentModifierFlagsMask

http://mirror.informatimago.com/next/developer.apple.com/releasenotes/Cocoa/AppKit.html

Handle Dynamic Changes in Window Resolution Only When You Must

https://developer.apple.com/library/archive/documentation/GraphicsAnimation/Conceptual/HighResolutionOSX/CapturingScreenContents/CapturingScreenContents.html#//apple_ref/doc/uid/TP40012302-CH10-SW20

cheers -ben

P.S. maybe not directly related, but just curious...

For events, only y-coordinates are 1-based.

https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/CocoaDrawingGuide/Transforms/Transforms.html

"NSEvent -mouseLocation returns invalid coordinate for Retina backing store"

http://www.openradar.me/11905408

Dealing with multi screens programming

http://www.thinkandbuild.it/deal-with-multiple-screens-programming/

Tobias Pape

Re: Lock Ups on macOS 10.13 [was Fix on management of events in OSX]

> On 02.11.2018, at 04:56, Ben Coman <[hidden email]> wrote:
>
>
>
> On Thu, 1 Nov 2018 at 21:54, Ben Coman <[hidden email]> wrote:
>
>
> On Wed, 31 Oct 2018 at 05:48, Eliot Miranda <[hidden email]> wrote:
>
> Hi Guille, Hi John, Hi All,
> On Tue, Oct 30, 2018 at 2:10 PM Eliot Miranda <[hidden email]> wrote:
> Hi Guille,
>
> I'm noticing some lock ups, primarily after resolution changes, or long periods of not in interacting. I wonder whether there are a class of window events that the loop does not handle and if this produces a lock up, or rather a failure to notice that new events are incoming. I *think* I have a reproducible case on a MacBookPro whose display resolution is set to "More Space" (see the Displays setting tab).
> Have a VM running on the laptop screen while the laptop is connected to an Apple Cinema display (in this configuration the laptop's retina display reverts to its default resolution).
> Then disconnect the cinema display, at which point the MBP's retina display's resolution changes back to :"More Space" and, at least when I tried this morning, the VM locked up.
>
> Guille, I think you're off the hook. I edited the event loop to include:
>
> if (!(event.window == 0 || event.window == gDelegateApp.window)){
> printf("breaking with event.window = %lx (gDA.window = %lx)\n", event.window, gDelegateApp.window);
> break;
> }
>
> Random thought...
> Perhaps when the screen resizes the window bounds are not resized, so for example while the mouse looks like its inside the window, the event coordinates end up outside the window, so the native event dispatching doesn't send the event to the window. It would be interesting to display the bounds-of-the-window together with the event-coordinates and observe how they interact when the screen resolution changes.
>
> Just some blind trawling of the net...
>
> Returns the "backing scale factor" of the view's window
> https://searchfox.org/mozilla-central/rev/37efe4d0e6d74152023e5e142f670e6a2cec478c/widget/cocoa/nsChildView.h#338-352
>
> Get the backing scale factor from an object that supports this selector (NSView/Window/Screen, on 10.7 or later)
> https://dxr.mozilla.org/mozilla-central/rev/681eb7dfa324dd50403c382888929ea8b8b11b00/widget/cocoa/nsCocoaUtils.h#120-130
>
> viewDidChangeBackingProperties
> https://searchfox.org/mozilla-central/rev/37efe4d0e6d74152023e5e142f670e6a2cec478c/widget/cocoa/nsChildView.h#113
>
> Coding for high resolution on OS X? Read this.
> http://supermegaultragroovy.com/2012/10/24/coding-for-high-resolution-on-os-x-read-this/
>
> For windows with zero width or height, the backingScaleFactor method is broken
> https://fossies.org/linux/firefox/widget/cocoa/nsCocoaWindow.mm
>
> Apple's resizing code not working...
> https://stackoverflow.com/questions/9204132/same-command-different-results-setframe-nsview
>
> NSEvent & NSDeviceIndependentModifierFlagsMask
> http://mirror.informatimago.com/next/developer.apple.com/releasenotes/Cocoa/AppKit.html
>
> Handle Dynamic Changes in Window Resolution Only When You Must
> https://developer.apple.com/library/archive/documentation/GraphicsAnimation/Conceptual/HighResolutionOSX/CapturingScreenContents/CapturingScreenContents.html#//apple_ref/doc/uid/TP40012302-CH10-SW20
>

https://github.com/OpenSmalltalk/opensmalltalk-vm/tree/krono/highdpi-v2

https://github.com/OpenSmalltalk/opensmalltalk-vm/compare/krono/highdpi-v2#diff-f10ade4595eece5eefc09ddd29fc0efaR87

https://github.com/OpenSmalltalk/opensmalltalk-vm/compare/krono/highdpi-v2#diff-b1d8ae032cbb765b78f0e5a700f5aae8R122

Best
-Tobias

> cheers -ben
>
>
> P.S. maybe not directly related, but just curious...
>
> For events, only y-coordinates are 1-based.
> https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/CocoaDrawingGuide/Transforms/Transforms.html
>
> "NSEvent -mouseLocation returns invalid coordinate for Retina backing store"
> http://www.openradar.me/11905408
>
> Dealing with multi screens programming
> http://www.thinkandbuild.it/deal-with-multiple-screens-programming/