Event filters for user input events (i.e. mouse clicks etc.)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Event filters for user input events (i.e. mouse clicks etc.)

marcel.taeumel
Hi, there.

Here is a change set that adds event filters to Squeak:
event-filters.cs

It is implemented like event listeners but with a spearation between global event filters in the HandMorph and local event filters in any other Morph.

Both event filters and event listeners do only work on event copies to not interfere with regular event dispatching. Such bugs would be quite challenging to hunt down.

Here is a usage example:

m := ActiveWorld submorphs third. "some system window"
filter := PluggableEventFilter new on: [:evt :target |
        Transcript showln: evt printString. false].

m addEventFilter: filter.

m removeEventFilter: filter.
m addKeyboardFilter: filter.

Just like event listeners, there are generic event filters, mouse event filters, and keyboard event filters. All filter objects are weakly referenced - just like event filters.

Where to use event filters?
- Processing global keyboard shortcuts and avoiding some other Morph to handle the same key
- Temporarily disable specific events for a morph from the outside

The general idea is based on the Qt framework:
http://doc.qt.io/qt-5/eventsandfilters.html#event-filters

Do we want that in the Trunk?

Best,
Marcel
Reply | Threaded
Open this post in threaded view
|

Re: Event filters for user input events (i.e. mouse clicks etc.)

marcel.taeumel
Hi, there.

I just noticed that event filters include the whole event listener functionality. If filters always return false, they behave like listeners. Plus, any morph can have its own set of filters attached.

Anyway, old projects might depend on traditional event listeners with the #handleListenEvent: callback.

Best,
Marcel
Reply | Threaded
Open this post in threaded view
|

Re: Event filters for user input events (i.e. mouse clicks etc.)

timrowledge

> On 17-05-2016, at 1:32 PM, marcel.taeumel <[hidden email]> wrote:
>
> Hi, there.
>
> I just noticed that event filters include the whole event listener
> functionality. If filters always return false, they behave like listeners.
> Plus, any morph can have its own set of filters attached.

I was about to mention that. It would make reasonable sense to replace the listeners with filters, assuming a tolerable legacy api can be supported.

I’m not sure about having the filters work only on copies of events though. Seems to me that a significant part of the potential usefulness would be to have filters transform events either in the geometric sense of altering coordinates etc or in changing the other details (convert a Leap Motion input event coming in via socket into a mouse event, stuff like that) or even just dropping it entirely because it is a MouseEventForLockedDemo. Another use would be a fake event filter that reads lists of x/y/button and transforms into ‘real’ events as a way to handle recorded event stream for playback.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Useful random insult:- His page was intentionally left blank.



Reply | Threaded
Open this post in threaded view
|

Re: Event filters for user input events (i.e. mouse clicks etc.)

marcel.taeumel
Hi, Tim.

The idea of using event filters to actually transform events is interesting.

How would a fake event filter be triggered if not by real events?

Best,
Marcel
Reply | Threaded
Open this post in threaded view
|

Re: Event filters for user input events (i.e. mouse clicks etc.)

Tony Garnock-Jones-3
In reply to this post by timrowledge
On 05/18/2016 12:12 PM, tim Rowledge wrote:
> Seems to me that a significant part of the potential
> usefulness would be to have filters transform events [...]

One interesting approach would be to take a leaf from functional
programming.

Instead of returning a boolean from a filter, you'd have it expect a
continuation block.

  filter := PluggableEventFilter new on: [:e :target :k |
     Transcript showln: e printString.
     k value: e ].

It would invoke the block with some event, or not, according to the
filtering decision.

This gives it an opportunity to pass on either its argument; or a copy
of its argument; or its argument, modified; or something unrelated.

Tony

Reply | Threaded
Open this post in threaded view
|

Re: Event filters for user input events (i.e. mouse clicks etc.)

marcel.taeumel
Tony Garnock-Jones-3 wrote
On 05/18/2016 12:12 PM, tim Rowledge wrote:
> Seems to me that a significant part of the potential
> usefulness would be to have filters transform events [...]

One interesting approach would be to take a leaf from functional
programming.

Instead of returning a boolean from a filter, you'd have it expect a
continuation block.

  filter := PluggableEventFilter new on: [:e :target :k |
     Transcript showln: e printString.
     k value: e ].

It would invoke the block with some event, or not, according to the
filtering decision.

This gives it an opportunity to pass on either its argument; or a copy
of its argument; or its argument, modified; or something unrelated.

Tony

Hi Tony,

I like the idea of filters transforming events. Here, the filter could just return the transformed event instead of a boolean value. (Note that the last statement in a block determines its return value.)

Here is a new version of event filter that does that: event-filters.cs

Don't forget to revert the previous change set because I cleaned up this one. Use a fresh trunk image at best.


wnd := ActiveWorld submorphs third. "some window?"
filter := PluggableEventFilter on: [:evt :target |
        evt isMouseOver ifFalse: [
        Transcript showln: evt printString, ' ', target printString].
        evt wasIgnored: true.
        evt].
wnd addMouseFilter: filter.
wnd removeMouseFilter: filter.


Best,
Marcel
Reply | Threaded
Open this post in threaded view
|

Re: Event filters for user input events (i.e. mouse clicks etc.)

timrowledge
In reply to this post by marcel.taeumel

> On 18-05-2016, at 9:25 AM, marcel.taeumel <[hidden email]> wrote:

> The idea of using event filters to actually transform events is interesting.

Filters can often be useful. We have, for example, the various canvases and scroll-wrapping morphs which filter how input and output is seen by the wrapped bitmaps etc. An obvious event filter (and maybe a stupid one in the real world) would be one that centralises the mouse button swapping; if button swapping is turned on it, err, swaps the button value around.  Right now we have a centralised place in EventSensor>processEvent: for something that global (and seriously - for a mouse type event we check the platform name every time?) but having a type of filter involved at lower levels would allow some morphs to *not* get swapped buttons. I dunno, maybe useful for testing how the UI behaves on a different GUI?

If you pass your events *through* a filter instead of just having them on the side then you can do much more.

>
> How would a fake event filter be triggered if not by real events?

If some morph/app has a suitable event filter then clearly we can transform any incoming event into any other kind of event (as long as it is one that the morph will not raise an error for) as they are sent by the whole Hand/PasteUp complex. To trigger a faking-filter I guess the best thing would be to throw a FakeEvent at the event handling code at whatever level you need (anywhere from the local world PasteUpMorph on down to your local morph of interest) and let you filter convert that to whatever it wants. Why not simply make the ‘actual’ event type and pass that in? You might have a variety of faking-filters doing different things and want to test them, maybe you want the FakeEvents to be easily assembled by parsing a socket stream or file, I don’t know. It just needs you to bypass the eventsensor and push an event.

I honestly don’t know if it’s an actually useful idea.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Asking whether machines can think is like asking whether submarines can swim.



Reply | Threaded
Open this post in threaded view
|

Re: Event filters for user input events (i.e. mouse clicks etc.)

marcel.taeumel
tim Rowledge wrote
> On 18-05-2016, at 9:25 AM, marcel.taeumel <[hidden email]> wrote:

> The idea of using event filters to actually transform events is interesting.

Filters can often be useful. We have, for example, the various canvases and scroll-wrapping morphs which filter how input and output is seen by the wrapped bitmaps etc. An obvious event filter (and maybe a stupid one in the real world) would be one that centralises the mouse button swapping; if button swapping is turned on it, err, swaps the button value around.  Right now we have a centralised place in EventSensor>processEvent: for something that global (and seriously - for a mouse type event we check the platform name every time?) but having a type of filter involved at lower levels would allow some morphs to *not* get swapped buttons. I dunno, maybe useful for testing how the UI behaves on a different GUI?

If you pass your events *through* a filter instead of just having them on the side then you can do much more.

>
> How would a fake event filter be triggered if not by real events?

If some morph/app has a suitable event filter then clearly we can transform any incoming event into any other kind of event (as long as it is one that the morph will not raise an error for) as they are sent by the whole Hand/PasteUp complex. To trigger a faking-filter I guess the best thing would be to throw a FakeEvent at the event handling code at whatever level you need (anywhere from the local world PasteUpMorph on down to your local morph of interest) and let you filter convert that to whatever it wants. Why not simply make the ‘actual’ event type and pass that in? You might have a variety of faking-filters doing different things and want to test them, maybe you want the FakeEvents to be easily assembled by parsing a socket stream or file, I don’t know. It just needs you to bypass the eventsensor and push an event.

I honestly don’t know if it’s an actually useful idea.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Asking whether machines can think is like asking whether submarines can swim.
Hi, there!

Here is another update: event-filters.cs

The filters should work now as expected. The change set includes a refactoring of MorphicEventDispatcher and HandMorph >> #handleEvent:. Be sure to update the trunk to the latest version before filing-in the change set.

Then you can do things like "if you type a y, make a z out of it and vice versa":

filter := PluggableEventFilter on: [:evt :target |
        (evt isKeyboard and: [evt keyCharacter = $z])
                ifTrue: [evt instVarNamed: #keyValue put: $y asInteger]
                ifFalse: [(evt isKeyboard and: [evt keyCharacter = $y])
                        ifTrue: [evt instVarNamed: #keyValue put: $z asInteger]].
        evt].

ActiveHand addEventFilter: filter.
ActiveHand removeEventFilter: filter.

Best,
Marcel
Reply | Threaded
Open this post in threaded view
|

Re: Event filters for user input events (i.e. mouse clicks etc.)

Tony Garnock-Jones-3
On 2016-05-20 9:19 AM, marcel.taeumel wrote:
> Then you can do things like "if you type a y, make a z out of it and vice
> versa": [...]

Would it be possible to have "if you type y, pretend you typed abcde"?

That is, instead of having the filter yield zero or one events (for
refiltering??), could it yield zero or more events?

Tony

Reply | Threaded
Open this post in threaded view
|

Re: Event filters for user input events (i.e. mouse clicks etc.)

marcel.taeumel
Tony Garnock-Jones-3 wrote
On 2016-05-20 9:19 AM, marcel.taeumel wrote:
> Then you can do things like "if you type a y, make a z out of it and vice
> versa": [...]

Would it be possible to have "if you type y, pretend you typed abcde"?

That is, instead of having the filter yield zero or one events (for
refiltering??), could it yield zero or more events?

Tony
Hi Tony,

yes, I think so. You event create code is kind of awkward but I would work like this:

...
[:event :target |
   ...
   event hand
      handleEvent: ((KeyboardEvent newFrom: event) stroke: $a);
      handleEvent: ((KeyboardEvent newFrom: event) stroke: $b);
      handleEvent: ((KeyboardEvent newFrom: event) stroke: $c);
      handleEvent: ((KeyboardEvent newFrom: event) stroke: $d);
      handleEvent: ((KeyboardEvent newFrom: event) stroke: $e).
   event wasIgnored: true.
   event].

Note that this API "(KeyboardEvent newFrom: event) stroke: $a" does not exist at the moment. It is not that convenient to generate new events but "event hand handleEvent: ..." should work. I have not tried this yet, however, the event dispatching and handling code should be reentrant.

I think that I should separate upstream and downstream event filters. When calling "ActiveHand addEventFilter: ..." it is only downstream because this is where event processing starts. When calling "someMorphInTheWorld addEventFilter: ..." this is rather upstream because it starts at Morph >> #handleEvent:. Hmm... downstream event filters for any morph would be possible in Morph >> #processEvent:using:.

Still ... I am struggling with the concept of keyboard focus and mouse focus. There, you directly call Morph >> #handleFocusEvent:, which calls Morph >> #handleEvent:. No bubbling up, no bubbling down. If focused morphs want to bubble down, they have to call Morph >> #processEvent: and watch for the #rejected pattern. See UserDialogBoxMorph >> #handleFocusEvent: for an example. I now that this is a performance thing. If you type in a text field, you do not want to bubble all the way down from the world. Again and again. And again.

Best,
Marcel
Reply | Threaded
Open this post in threaded view
|

Re: Event filters for user input events (i.e. mouse clicks etc.)

timrowledge

> On 20-05-2016, at 12:33 PM, marcel.taeumel <[hidden email]> wrote:
> yes, I think so. You event create code is kind of awkward but I would work
> like this:
>
> ...
> [:event :target |
>   ...
>   event hand
>      handleEvent: ((KeyboardEvent newFrom: event) stroke: $a);
>      handleEvent: ((KeyboardEvent newFrom: event) stroke: $b);
>      handleEvent: ((KeyboardEvent newFrom: event) stroke: $c);
>      handleEvent: ((KeyboardEvent newFrom: event) stroke: $d);
>      handleEvent: ((KeyboardEvent newFrom: event) stroke: $e).
>   event wasIgnored: true.
>   event].

You’d need to be *very* careful here; catching a keyboard event and then sending more keyboard events via the same route…

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Abdicate (v.), to give up all hope of ever having a flat stomach.







Reply | Threaded
Open this post in threaded view
|

Re: Event filters for user input events (i.e. mouse clicks etc.)

marcel.taeumel
tim Rowledge wrote
> On 20-05-2016, at 12:33 PM, marcel.taeumel <[hidden email]> wrote:
> yes, I think so. You event create code is kind of awkward but I would work
> like this:
>
> ...
> [:event :target |
>   ...
>   event hand
>      handleEvent: ((KeyboardEvent newFrom: event) stroke: $a);
>      handleEvent: ((KeyboardEvent newFrom: event) stroke: $b);
>      handleEvent: ((KeyboardEvent newFrom: event) stroke: $c);
>      handleEvent: ((KeyboardEvent newFrom: event) stroke: $d);
>      handleEvent: ((KeyboardEvent newFrom: event) stroke: $e).
>   event wasIgnored: true.
>   event].

You’d need to be *very* careful here; catching a keyboard event and then sending more keyboard events via the same route…

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Abdicate (v.), to give up all hope of ever having a flat stomach.
Hi Tim,

it wouldn't be the same route. Calling HandMorph >> #processEvent: means a fresh start for processing the event argument. With all its glory. :-) Shared state in HandMorph is an issue though:

- keyboard focus
- mouse focus
- mouse click state
- mouse over handler
- event listeners
(- event filters)
(- event stats)

However, you actually can send multiple events along the "same route" if you mean the same target:

[:event :target |
  ...
   target "<--- do NOT start event processing over again but use same target"
     handleEvent: ((KeyboardEvent newFrom: event) stroke: $a);
     handleEvent: ((KeyboardEvent newFrom: event) stroke: $b);
     handleEvent: ((KeyboardEvent newFrom: event) stroke: $c);
     handleEvent: ((KeyboardEvent newFrom: event) stroke: $d);
     handleEvent: ((KeyboardEvent newFrom: event) stroke: $e).
  event wasIgnored: true.
  event].

Given that there is a double-dispatch pattern in UserInputEvent >> #sentTo: and a event type discrimination in MorphicEventDispatcher >> #dispatchEvent:with:, we are fine, even with changing event types and not just multiplicity in a filter.

The return value in an event filter, however, should always be a single event. Ignored if not needed.

Best,
Marcel
Reply | Threaded
Open this post in threaded view
|

Re: Event filters for user input events (i.e. mouse clicks etc.)

marcel.taeumel
Hi, there.

Here is a new version of event filters with support for separating capturing filters from bubbling filters.
https://www.w3.org/TR/DOM-Level-2-Events/events.html#Events-flow-capture
https://www.w3.org/TR/DOM-Level-2-Events/events.html#Events-flow-bubbling

Here is the change set: event-filters.cs

Note that you should either use a fresh trunk image with the latest updates OR unload all the changes from my previous change set because I cleaned up this one. So, there will be some "method removed" calls missing.

Anyway, with support for capturing filters, you can also filter, for example, mouse events for system windows that are always active (see preference "Windows Active Only On Top" disabled).

wnd := ActiveWorld submorphs fourth.
mouseFilter := PluggableEventFilter on: [:evt :target | evt ignore].
wnd addMouseCaptureFilter: mouseFilter.
wnd removeMouseCaptureFilter: mouseFilter.

See how the behavior is different when you use the filter for the bubbling phase:

wnd addMouseBubbleFilter: mouseFilter.
wnd removeMouseBubbleFilter: mouseFilter.

Note that event bubbling is implemented in MorphicEventDispatcher. If there would be no bubbling, those filters would be meaningless. However, I am not aware of an alternative event dispatcher for Squeak's Morphic.

What other dispatch patterns are out there? :-)

Best,
Marcel
Reply | Threaded
Open this post in threaded view
|

Re: Event filters for user input events (i.e. mouse clicks etc.)

marcel.taeumel
marcel.taeumel wrote
Hi, there.

Here is a new version of event filters with support for separating capturing filters from bubbling filters.
https://www.w3.org/TR/DOM-Level-2-Events/events.html#Events-flow-capture
https://www.w3.org/TR/DOM-Level-2-Events/events.html#Events-flow-bubbling

Here is the change set: event-filters.cs

Note that you should either use a fresh trunk image with the latest updates OR unload all the changes from my previous change set because I cleaned up this one. So, there will be some "method removed" calls missing.

Anyway, with support for capturing filters, you can also filter, for example, mouse events for system windows that are always active (see preference "Windows Active Only On Top" disabled).

wnd := ActiveWorld submorphs fourth.
mouseFilter := PluggableEventFilter on: [:evt :target | evt ignore].
wnd addMouseCaptureFilter: mouseFilter.
wnd removeMouseCaptureFilter: mouseFilter.

See how the behavior is different when you use the filter for the bubbling phase:

wnd addMouseBubbleFilter: mouseFilter.
wnd removeMouseBubbleFilter: mouseFilter.

Note that event bubbling is implemented in MorphicEventDispatcher. If there would be no bubbling, those filters would be meaningless. However, I am not aware of an alternative event dispatcher for Squeak's Morphic.

What other dispatch patterns are out there? :-)

Best,
Marcel
Hi, there.

I benched the event filters mechanism without having any active filters to reveal the impact of my changes. Here is the benchmark:

0) Close all windows.

1) Open a system browser and position it in the upper-left corner right below the world main docking bar.

2) Open a workspace, not covering the system browser.

3) Execute some code:

event := MouseButtonEvent new
                setType: #mouseDown
                position: 100@100
                which: 2r000 "no change"
                buttons: 2r100 "red/left pressed"
                hand: ActiveHand
                stamp: 0. "not needed"
                       
[
        event resetHandlerFields.
        event hand sendMouseEvent: event.
] bench.

On my Windows 10 machine (MS Surface Pro 3, Core i7 mobile) with Cog/Spur #3663 and Squeak #15987 I got:

Before: 3.73 microseconds per run
After: 4.5 microseconds per run

This tests a mouse-down event because it involves full dispatching (resp. capturing) and handling (resp. bubbling) without taking the shortcut via mouse focus (or keyboard focus).

This benchmark combined with MessageTally class >> #spyOn: provided interesting insights about expensive calls.

About 5.9 percent are spend for sending generic and mouse bubble filters. Note that 4.1 percent are spend for accessing the Morph's properties to find out that there are no filters at the moment. This could be optimized by adding instance variables to MorphExtension.

(Interestingly, about 11 percent (before: 12.7 percent) are spent in Morph >> #handleMouseDown: to call HandMorph >> #removePendingBalloonFor:.)

(On a side note, the current way to provide global keyboard shortcuts eats up 47.1 percent (before: 48.4 precent) for this mouse down handling because of installing/removing event listeners in PasteUpMorph >> #becomeActiveDuring:. But this will be optimized on a few days.)

The following questions remain:
- Can I put this in trunk and use it to improve the implementation of global keyboard shortcuts?
- Should I add instance variables in MorphExtension to speed-up the look-up? Note that I already did add instance variables to HandMorph to speed-up the look-up of global event capturing filters.
- Is it conceptually good to re-evaluate the filters if one of them changes the kind of an event? This affects only mouse event filters and keyboard event filters that convert between mouse and keyboard. Note that an endless loop is possible if such filters do not converge.

Best,
Marcel
Reply | Threaded
Open this post in threaded view
|

Re: Event filters for user input events (i.e. mouse clicks etc.)

Chris Muller-3
Hi Marcel, global keyboard shortcuts are important, a 1-microsecond
slowdown per event does not sound significant.  But, yes, for
performance critical code like event handling, its probably worth
adding an instvar than a dictionary lookup.

I don't know under what case an event-type would ever change.  Events
are events generated by the input devices.  They should basically be
created and then immutable...

On Fri, May 27, 2016 at 8:40 AM, marcel.taeumel <[hidden email]> wrote:

> marcel.taeumel wrote
>> Hi, there.
>>
>> Here is a new version of event filters with support for separating
>> capturing filters from bubbling filters.
>> https://www.w3.org/TR/DOM-Level-2-Events/events.html#Events-flow-capture
>> https://www.w3.org/TR/DOM-Level-2-Events/events.html#Events-flow-bubbling
>>
>> Here is the change set:
>> event-filters.cs <http://forum.world.st/file/n4896486/event-filters.cs>
>>
>> Note that you should either use a fresh trunk image with the latest
>> updates OR unload all the changes from my previous change set because I
>> cleaned up this one. So, there will be some "method removed" calls
>> missing.
>>
>> Anyway, with support for capturing filters, you can also filter, for
>> example, mouse events for system windows that are always active (see
>> preference "Windows Active Only On Top" disabled).
>>
>> wnd := ActiveWorld submorphs fourth.
>> mouseFilter := PluggableEventFilter on: [:evt :target | evt ignore].
>> wnd addMouseCaptureFilter: mouseFilter.
>> wnd removeMouseCaptureFilter: mouseFilter.
>>
>> See how the behavior is different when you use the filter for the bubbling
>> phase:
>>
>> wnd addMouseBubbleFilter: mouseFilter.
>> wnd removeMouseBubbleFilter: mouseFilter.
>>
>> Note that event bubbling is implemented in MorphicEventDispatcher. If
>> there would be no bubbling, those filters would be meaningless. However, I
>> am not aware of an alternative event dispatcher for Squeak's Morphic.
>>
>> What other dispatch patterns are out there? :-)
>>
>> Best,
>> Marcel
>
> Hi, there.
>
> I benched the event filters mechanism without having any active filters to
> reveal the impact of my changes. Here is the benchmark:
>
> 0) Close all windows.
>
> 1) Open a system browser and position it in the upper-left corner right
> below the world main docking bar.
>
> 2) Open a workspace, not covering the system browser.
>
> 3) Execute some code:
>
> event := MouseButtonEvent new
>                 setType: #mouseDown
>                 position: 100@100
>                 which: 2r000 "no change"
>                 buttons: 2r100 "red/left pressed"
>                 hand: ActiveHand
>                 stamp: 0. "not needed"
>
> [
>         event resetHandlerFields.
>         event hand sendMouseEvent: event.
> ] bench.
>
> On my Windows 10 machine (MS Surface Pro 3, Core i7 mobile) with Cog/Spur
> #3663 and Squeak #15987 I got:
>
> Before: 3.73 microseconds per run
> After: 4.5 microseconds per run
>
> This tests a mouse-down event because it involves full dispatching (resp.
> capturing) and handling (resp. bubbling) without taking the shortcut via
> mouse focus (or keyboard focus).
>
> This benchmark combined with MessageTally class >> #spyOn: provided
> interesting insights about expensive calls.
>
> About 5.9 percent are spend for sending generic and mouse bubble filters.
> Note that 4.1 percent are spend for accessing the Morph's properties to find
> out that there are no filters at the moment. This could be optimized by
> adding instance variables to MorphExtension.
>
> (Interestingly, about 11 percent (before: 12.7 percent) are spent in Morph
>>> #handleMouseDown: to call HandMorph >> #removePendingBalloonFor:.)
>
> (On a side note, the current way to provide global keyboard shortcuts eats
> up 47.1 percent (before: 48.4 precent) for this mouse down handling because
> of installing/removing event listeners in PasteUpMorph >>
> #becomeActiveDuring:. But this will be optimized on a few days.)
>
> The following questions remain:
> - Can I put this in trunk and use it to improve the implementation of global
> keyboard shortcuts?
> - Should I add instance variables in MorphExtension to speed-up the look-up?
> Note that I already did add instance variables to HandMorph to speed-up the
> look-up of global event capturing filters.
> - Is it conceptually good to re-evaluate the filters if one of them changes
> the kind of an event? This affects only mouse event filters and keyboard
> event filters that convert between mouse and keyboard. Note that an endless
> loop is possible if such filters do not converge.
>
> Best,
> Marcel
>
>
>
> --
> View this message in context: http://forum.world.st/Event-filters-for-user-input-events-i-e-mouse-clicks-etc-tp4895521p4897830.html
> Sent from the Squeak - Dev mailing list archive at Nabble.com.
>

Reply | Threaded
Open this post in threaded view
|

Re: Event filters for user input events (i.e. mouse clicks etc.)

Tobias Pape

On 27.05.2016, at 20:36, Chris Muller <[hidden email]> wrote:

> Hi Marcel, global keyboard shortcuts are important, a 1-microsecond
> slowdown per event does not sound significant.  But, yes, for
> performance critical code like event handling, its probably worth
> adding an instvar than a dictionary lookup.
>
> I don't know under what case an event-type would ever change.  Events
> are events generated by the input devices.  They should basically be
> created and then immutable...
>

Unless you want a filter that turns a mouse click into a keypress, like for automation?

Best
        -Tobias

> On Fri, May 27, 2016 at 8:40 AM, marcel.taeumel <[hidden email]> wrote:
>> marcel.taeumel wrote
>>> Hi, there.
>>>
>>> Here is a new version of event filters with support for separating
>>> capturing filters from bubbling filters.
>>> https://www.w3.org/TR/DOM-Level-2-Events/events.html#Events-flow-capture
>>> https://www.w3.org/TR/DOM-Level-2-Events/events.html#Events-flow-bubbling
>>>
>>> Here is the change set:
>>> event-filters.cs <http://forum.world.st/file/n4896486/event-filters.cs>
>>>
>>> Note that you should either use a fresh trunk image with the latest
>>> updates OR unload all the changes from my previous change set because I
>>> cleaned up this one. So, there will be some "method removed" calls
>>> missing.
>>>
>>> Anyway, with support for capturing filters, you can also filter, for
>>> example, mouse events for system windows that are always active (see
>>> preference "Windows Active Only On Top" disabled).
>>>
>>> wnd := ActiveWorld submorphs fourth.
>>> mouseFilter := PluggableEventFilter on: [:evt :target | evt ignore].
>>> wnd addMouseCaptureFilter: mouseFilter.
>>> wnd removeMouseCaptureFilter: mouseFilter.
>>>
>>> See how the behavior is different when you use the filter for the bubbling
>>> phase:
>>>
>>> wnd addMouseBubbleFilter: mouseFilter.
>>> wnd removeMouseBubbleFilter: mouseFilter.
>>>
>>> Note that event bubbling is implemented in MorphicEventDispatcher. If
>>> there would be no bubbling, those filters would be meaningless. However, I
>>> am not aware of an alternative event dispatcher for Squeak's Morphic.
>>>
>>> What other dispatch patterns are out there? :-)
>>>
>>> Best,
>>> Marcel
>>
>> Hi, there.
>>
>> I benched the event filters mechanism without having any active filters to
>> reveal the impact of my changes. Here is the benchmark:
>>
>> 0) Close all windows.
>>
>> 1) Open a system browser and position it in the upper-left corner right
>> below the world main docking bar.
>>
>> 2) Open a workspace, not covering the system browser.
>>
>> 3) Execute some code:
>>
>> event := MouseButtonEvent new
>>                setType: #mouseDown
>>                position: 100@100
>>                which: 2r000 "no change"
>>                buttons: 2r100 "red/left pressed"
>>                hand: ActiveHand
>>                stamp: 0. "not needed"
>>
>> [
>>        event resetHandlerFields.
>>        event hand sendMouseEvent: event.
>> ] bench.
>>
>> On my Windows 10 machine (MS Surface Pro 3, Core i7 mobile) with Cog/Spur
>> #3663 and Squeak #15987 I got:
>>
>> Before: 3.73 microseconds per run
>> After: 4.5 microseconds per run
>>
>> This tests a mouse-down event because it involves full dispatching (resp.
>> capturing) and handling (resp. bubbling) without taking the shortcut via
>> mouse focus (or keyboard focus).
>>
>> This benchmark combined with MessageTally class >> #spyOn: provided
>> interesting insights about expensive calls.
>>
>> About 5.9 percent are spend for sending generic and mouse bubble filters.
>> Note that 4.1 percent are spend for accessing the Morph's properties to find
>> out that there are no filters at the moment. This could be optimized by
>> adding instance variables to MorphExtension.
>>
>> (Interestingly, about 11 percent (before: 12.7 percent) are spent in Morph
>>>> #handleMouseDown: to call HandMorph >> #removePendingBalloonFor:.)
>>
>> (On a side note, the current way to provide global keyboard shortcuts eats
>> up 47.1 percent (before: 48.4 precent) for this mouse down handling because
>> of installing/removing event listeners in PasteUpMorph >>
>> #becomeActiveDuring:. But this will be optimized on a few days.)
>>
>> The following questions remain:
>> - Can I put this in trunk and use it to improve the implementation of global
>> keyboard shortcuts?
>> - Should I add instance variables in MorphExtension to speed-up the look-up?
>> Note that I already did add instance variables to HandMorph to speed-up the
>> look-up of global event capturing filters.
>> - Is it conceptually good to re-evaluate the filters if one of them changes
>> the kind of an event? This affects only mouse event filters and keyboard
>> event filters that convert between mouse and keyboard. Note that an endless
>> loop is possible if such filters do not converge.
>>
>> Best,
>> Marcel
>>
>>
>>
>> --
>> View this message in context: http://forum.world.st/Event-filters-for-user-input-events-i-e-mouse-clicks-etc-tp4895521p4897830.html
>> Sent from the Squeak - Dev mailing list archive at Nabble.com.



Reply | Threaded
Open this post in threaded view
|

Re: Event filters for user input events (i.e. mouse clicks etc.)

Chris Muller-4
On Fri, May 27, 2016 at 2:40 PM, Tobias Pape <[hidden email]> wrote:

>
> On 27.05.2016, at 20:36, Chris Muller <[hidden email]> wrote:
>
>> Hi Marcel, global keyboard shortcuts are important, a 1-microsecond
>> slowdown per event does not sound significant.  But, yes, for
>> performance critical code like event handling, its probably worth
>> adding an instvar than a dictionary lookup.
>>
>> I don't know under what case an event-type would ever change.  Events
>> are events generated by the input devices.  They should basically be
>> created and then immutable...
>>
>
> Unless you want a filter that turns a mouse click into a keypress, like for automation?

Instead of converting an event, shouldn't the automater-thingy just
push a new event?

The responsibility of Events is to interface the UI hardware
(keyboard, mouse, etc.) to some UI framework (MVC, Morphic).  They
represent the low-level, immutable "data" from the hardware,
interpreted by the UI framework.  IMHO, this "signal" between VM and
Morphic should remain a thin, clean, simple and raw encapsulation of
the data (i.e., getters and setters).  I don't think
application-specific behaviors belong way down there..  You mentioned,
automation?

Reply | Threaded
Open this post in threaded view
|

Re: Event filters for user input events (i.e. mouse clicks etc.)

marcel.taeumel
Chris Muller-4 wrote
On Fri, May 27, 2016 at 2:40 PM, Tobias Pape <[hidden email]> wrote:
>
> On 27.05.2016, at 20:36, Chris Muller <[hidden email]> wrote:
>
>> Hi Marcel, global keyboard shortcuts are important, a 1-microsecond
>> slowdown per event does not sound significant.  But, yes, for
>> performance critical code like event handling, its probably worth
>> adding an instvar than a dictionary lookup.
>>
>> I don't know under what case an event-type would ever change.  Events
>> are events generated by the input devices.  They should basically be
>> created and then immutable...
>>
>
> Unless you want a filter that turns a mouse click into a keypress, like for automation?

Instead of converting an event, shouldn't the automater-thingy just
push a new event?

The responsibility of Events is to interface the UI hardware
(keyboard, mouse, etc.) to some UI framework (MVC, Morphic).  They
represent the low-level, immutable "data" from the hardware,
interpreted by the UI framework.  IMHO, this "signal" between VM and
Morphic should remain a thin, clean, simple and raw encapsulation of
the data (i.e., getters and setters).  I don't think
application-specific behaviors belong way down there..  You mentioned,
automation?
If you want to convert a mouse click into a keypress for automation, you would have two options to write a global/hand capture filter:

1st option) Write a mouse filter that ignores the given click event, generates a keyboard event, and processes that keyboard event via "givenMouseEvent hand processEvent: newKeyboardEvent.". Eventually, return that original (but ignored) mouse event.

2nd option) Write a mouse filter that just generates a keyboard event and returns that.

I want to ensure that both options yield the same results to mitigate programming errors. The 1st option would trigger all active filters again, and hence also keyboard filters, because event processing starts again. The 2nd option continues the current event processing and hence should also trigger global keyboard filters (again). Just make option 1 behave like option 2.

@Chris: Yes, such event filters are specific to Morphic. In MVC, such filters do still not exist. The code that I added resides in Morph, HandMorph, and MorphEventDispatcher. However, we should consider moving the UserInputEvent tree out of Morphic into the base system for other frameworks to work with user input events in an object-oriented fashion. :-) One could also implement generic event filters in EventSensor >> #nextEvent but this is not what I had in mind here. Our global keyboard shortcuts are for Morphic only.

Best,
Marcel