Unique Mouse Wheel Events

For such a change, the VMs need to me adapted.

I propose to pass the *correct* operating system events from the VM into Smalltalk and then to create a MouseWheelEvent from that data there.

I further argue that the VM should not shadow *any* kind of user input such as CTRL+LEFTCLICK and ALT+LEFTCLICK to emulate other mouse buttons. Any of this kind of magic should be introduced in Smalltalk-land not VM-land.

Best,
Marcel

I looked up some older discussions about this:

http://forum.world.st/squeak-dev-Mouse-mapping-preference-in-win32-squeak-VM-td77688.html
http://forum.world.st/Mouse-buttons-was-Browser-pane-width-adjustment-td3088397.html
http://forum.world.st/Squeak-4-5-Windows-right-click-debacle-td4775136.html
http://forum.world.st/Changing-keyboard-binding-td70507.html
http://forum.world.st/Note-Squeak-4-1-Unix-preset-preferences-for-right-mouse-button-td1951759.html
http://forum.world.st/How-difficult-is-it-to-change-focus-on-scroll-td4772542.html

Best,
Marcel

Let me elaborate this a little bit more.

I claim that tools (in-image) should provide feasible interaction techniques for all kinds of common user input methods. For example, a calculator application should be usable with both mouse and keyboard input. Some users may have no mouse attached or just prefer hitting keys. It's the tools' job to think about that -- not the VM's.

Now, we want to be cross-platform compatible and thus we need a mapping from operating system-specific user input events to Smalltalk image-land. This is already done for some events. At the moment, the image gets some arrays with magic numbers and decodes it to real objects such as KeyboardEvent and MouseEvent. However, this mapping needs to be *injective* (math speak) and must not mask any user input information (like CTRL+LEFTCLICK does).

(https://en.wikipedia.org/wiki/Injective_function)

What kinds of events do operating systems provide nowadays?

* keyboard events (w/ modifiers, deadkeys, etc.)
* mouse click events (w/ modifiers)
* mouse wheel events
* touch events
* ...

We should support those events at VM-level and pass them using an apropriate format to Smalltalk-level.

To give you an impression on this kind of cross-platform abstraction, take a short look at the Qt API and its QInputEvent inheritance tree: http://doc.qt.io/qt-5/qinputevent.html

On backwards compatibility: A simple change set could be loaded into older images to perform that old-style event mapping in the image (HandMorph >> #processEvents).

Best,
Marcel

On Tue, Mar 24, 2015 at 01:09:42AM -0700, Marcel Taeumel wrote:
Marcel,

Thanks for raising this topic, and for providing the references.

Providing a good abstract representation of input events for both the VM
and the image would be a very worthwhile project. It is also becoming more
important as we consider devices with touch and swipe inputs.

The SqueakJS VM makes it amazingly easy to run Squeak on my cell phone,
but we do not yet have a good model for the user input interactions.

Dave

Marcel Taeumel wrote

For such a change, the VMs need to me adapted.

Yes, the changes have already been merged into the latest Pharo VM for Mac OS X.

Marcel Taeumel wrote

I propose to pass the *correct* operating system events from the VM into Smalltalk and then to create a MouseWheelEvent from that data there.

I also agree! This change is a temporary solution so that horizontal scrolling works while we are inventing the future :)

I'm not quite sure I understand.

It shouldn't matter if ctrl-arrow OS keyboard events have some system-wide binding. E.g. on a Mac, this is the default to switch between desktops.

The mouse wheel does *not* generate an OS keyboard event. We only synthesize a VM keyboard event for the image to consume. So how could that possibly affect anything outside the image?

I'm not against changing how the VM handles scroll events. However, I would like to understand what the actual problem is in the first place.

- Bert -

> It shouldn't matter if ctrl-arrow OS keyboard events have some system-wide binding
Sorry, I didn't explain it well. ctrl-arrow is apparently a standard Linux key combo, which Squeak/Pharo text editors therefore use to jump among words. The conflict is inside the image.

> It shouldn't matter if ctrl-arrow OS keyboard events have some system-wide binding
Sorry, I didn't explain it well. ctrl-arrow is apparently a standard Linux key combo, which Squeak/Pharo text editors therefore use to jump among words. The conflict is inside the image.

Cheers,
Sean

---

View this message in context: Re: Unique Mouse Wheel Events
Sent from the Squeak - Dev mailing list archive at Nabble.com.

Le 24/03/2015 14:35, Bert Freudenberg a écrit :

> I'm not against changing how the VM handles scroll events. However, I would like to understand what the actual problem is in the first place.
>
> - Bert -
>
Hi Bert,

(a bit long, sorry).

I don't know is Squeak vm is impacted, it was originally about Pharo,
the source was the same but may be the repos are not the same (?),
but may be Squeak could be interested in that?

The problem is about the current handling for horizontal mouse wheel
that maps mouse wheel events to Ctrl-Arrow  keyboard events.
Currently the image side detect Ctrl-ArrowUp and Ctrl-ArrowDown and
maps them as Vertical Mouse wheel event (this mapping done at the vm
level). It works fine.

An attempt was made in Pharo image to detect and handle horizontal mouse
wheel event the same way (Ctrl-Left Ctrl-Right) and to map
them as MouseWheelEvent in HandMorph class (generateKeyboarEvent:) but
we lost the ability to use Ctrl-left / Ctrl-right
for editing (jump from word to word).
You could use Alt-left Alt-Right but it gets hard when you want to
cut/paste along the way
(the switch between Alt and Ctrl keys was prone to editing errors)
It was a pain.

So the question was about the ability to add an extra modifier at the VM
level
for the generated event, precisely to add ALT and CTRL extra modifiers
to the generated event
(and handling them correctly in the image of course - in HandMorph class
first).
This should be a non breaking change for the vm part (correct me if I'm
wrong).


I had a look at the vm code, I think it could be done for Unix in
sqUnixX11.c at line 3533:
            int modifiers= modifierState ^ CtrlKeyBit;
=> Add ALT + CTRL modifiers (xored)

For Windows (not really needed but eventually) quite the same in
sqWin32Window.c at 266
       evt->modifiers = CtrlKeyBit;
(to check xored or not?)
As stated by MS, at https://msdn.microsoft.com/en-us/library/ms997498.aspx
the horizontal mouse wheel seems to be not really handled:
"...
Horizontal scrolling will not work, or is likely not to work, in the
following scenarios:

     The application uses custom horizontal scroll bars.
     The application has no horizontal scroll bars.
     The horizontal scroll bar is physically placed in a nonstandard way
relative to the window it is associated with.
"
As Pharo (squeak too) do not have native widgets , my understanding is
that it should not work.
=>No problem

For Mac  .... well I don't know mac
but had a look at the source anyway
sqMacUIEvents around l 1912 ?

     if (wheelMouseDelta != 0) {
...
        evt->modifiers = modifierMap[(controlKey >> 8)];
my guess-> the change goes around here ?

May be an interested Mac developer could have a look and try?
(nb I can do the mod build and test for unix and windows not for mac)

(Sean ?)


--
Regards,

Alain

Le 25/03/2015 00:02, Alain Rastoul a écrit :

> for the generated event, precisely to add ALT and CTRL extra modifiers
> to the generated event

tipo : to add ALT and *SHIFT* modifier
CTRL is already here
--
Regards,

Alain

> I had a look at the vm code, I think it could be done for Unix in
> For Windows (not really needed but eventually) quite the same in
Cool! I couldn't find those. I'll get on it!

> May be an interested Mac developer could have a look and try?
Already integrated in PharoVM :)

btw the shortcut is now "ctrl + alt + shift + cmd + arrowKey", and as mentioned the VM change should be backward compatible with existing Squeak and Pharo

> I had a look at the vm code, I think it could be done for Unix in
> sqUnixX11.c at line 3533:
> ...
> For Windows (not really needed but eventually) quite the same in
> sqWin32Window.c at 266
Okay, I took a first pass...
https://github.com/pharo-project/pharo-vm/pull/73
I have no idea if any of this works because: a) I can't compile it because I'm on a Mac and b) I haven't programmed C in almost 20 years. But it's a start!