Administrator
|
Cheers,
Sean |
For such a change, the VMs need to me adapted.
I propose to pass the *correct* operating system events from the VM into Smalltalk and then to create a MouseWheelEvent from that data there. I further argue that the VM should not shadow *any* kind of user input such as CTRL+LEFTCLICK and ALT+LEFTCLICK to emulate other mouse buttons. Any of this kind of magic should be introduced in Smalltalk-land not VM-land. Best, Marcel |
In reply to this post by marcel.taeumel (old)
Let me elaborate this a little bit more.
I claim that tools (in-image) should provide feasible interaction techniques for all kinds of common user input methods. For example, a calculator application should be usable with both mouse and keyboard input. Some users may have no mouse attached or just prefer hitting keys. It's the tools' job to think about that -- not the VM's. Now, we want to be cross-platform compatible and thus we need a mapping from operating system-specific user input events to Smalltalk image-land. This is already done for some events. At the moment, the image gets some arrays with magic numbers and decodes it to real objects such as KeyboardEvent and MouseEvent. However, this mapping needs to be *injective* (math speak) and must not mask any user input information (like CTRL+LEFTCLICK does). (https://en.wikipedia.org/wiki/Injective_function) What kinds of events do operating systems provide nowadays? * keyboard events (w/ modifiers, deadkeys, etc.) * mouse click events (w/ modifiers) * mouse wheel events * touch events * ... We should support those events at VM-level and pass them using an apropriate format to Smalltalk-level. To give you an impression on this kind of cross-platform abstraction, take a short look at the Qt API and its QInputEvent inheritance tree: http://doc.qt.io/qt-5/qinputevent.html On backwards compatibility: A simple change set could be loaded into older images to perform that old-style event mapping in the image (HandMorph >> #processEvents). Best, Marcel |
On Tue, Mar 24, 2015 at 01:09:42AM -0700, Marcel Taeumel wrote:
> Let me elaborate this a little bit more. > > I claim that tools (in-image) should provide feasible interaction techniques > for all kinds of common user input methods. For example, a calculator > application should be usable with both mouse and keyboard input. Some users > may have no mouse attached or just prefer hitting keys. It's the tools' job > to think about that -- not the VM's. > > Now, we want to be cross-platform compatible and thus we need a mapping from > operating system-specific user input events to Smalltalk image-land. This is > already done for some events. At the moment, the image gets some arrays with > magic numbers and decodes it to real objects such as KeyboardEvent and > MouseEvent. However, this mapping needs to be *injective* (math speak) and > must not mask any user input information (like CTRL+LEFTCLICK does). > > (https://en.wikipedia.org/wiki/Injective_function) > > What kinds of events do operating systems provide nowadays? > > * keyboard events (w/ modifiers, deadkeys, etc.) > * mouse click events (w/ modifiers) > * mouse wheel events > * touch events > * ... > > We should support those events at VM-level and pass them using an apropriate > format to Smalltalk-level. > > To give you an impression on this kind of cross-platform abstraction, take a > short look at the Qt API and its QInputEvent inheritance tree: > http://doc.qt.io/qt-5/qinputevent.html > > On backwards compatibility: A simple change set could be loaded into older > images to perform that old-style event mapping in the image (HandMorph >> > #processEvents). > Marcel, Thanks for raising this topic, and for providing the references. Providing a good abstract representation of input events for both the VM and the image would be a very worthwhile project. It is also becoming more important as we consider devices with touch and swipe inputs. The SqueakJS VM makes it amazingly easy to run Squeak on my cell phone, but we do not yet have a good model for the user input interactions. Dave |
Administrator
|
In reply to this post by marcel.taeumel (old)
Yes, the changes have already been merged into the latest Pharo VM for Mac OS X. I also agree! This change is a temporary solution so that horizontal scrolling works while we are inventing the future :)
Cheers,
Sean |
In reply to this post by Sean P. DeNigris
> On 23.03.2015, at 19:34, DeNigris Sean <[hidden email]> wrote: > > Cross-posting to Pharo dev, Squeak dev, and Squeak VM lists... > > Recently, we discovered on the Pharo dev list that the key combination used to fake wheel events - ctrl + arrowKey - may be a bit too simple. Apparently ctrl+horizontalArrowKey is a standard in-use combo in Linux and Windows, so adding horizontal wheel events broke some users' workflows. So I propose "ctrl + alt + shift + cmd + arrowKey". > The new combo is: > > • very unlikely to be typed > • backward-compatible with existing Pharo and Squeak images, which only checks that either cmd or ctrl is pressed > • frees up ctrl + arrow for our Linux/Windows friends It shouldn't matter if ctrl-arrow OS keyboard events have some system-wide binding. E.g. on a Mac, this is the default to switch between desktops. The mouse wheel does *not* generate an OS keyboard event. We only synthesize a VM keyboard event for the image to consume. So how could that possibly affect anything outside the image? I'm not against changing how the VM handles scroll events. However, I would like to understand what the actual problem is in the first place. - Bert - smime.p7s (5K) Download Attachment |
Administrator
|
> It shouldn't matter if ctrl-arrow OS keyboard events have some system-wide binding
Sorry, I didn't explain it well. ctrl-arrow is apparently a standard Linux key combo, which Squeak/Pharo text editors therefore use to jump among words. The conflict is inside the image.
Cheers,
Sean |
Ah, that makes much more sense :) - Bert -
smime.p7s (8K) Download Attachment |
In reply to this post by Bert Freudenberg
Le 24/03/2015 14:35, Bert Freudenberg a écrit :
> I'm not against changing how the VM handles scroll events. However, I would like to understand what the actual problem is in the first place. > > - Bert - > Hi Bert, (a bit long, sorry). I don't know is Squeak vm is impacted, it was originally about Pharo, the source was the same but may be the repos are not the same (?), but may be Squeak could be interested in that? The problem is about the current handling for horizontal mouse wheel that maps mouse wheel events to Ctrl-Arrow keyboard events. Currently the image side detect Ctrl-ArrowUp and Ctrl-ArrowDown and maps them as Vertical Mouse wheel event (this mapping done at the vm level). It works fine. An attempt was made in Pharo image to detect and handle horizontal mouse wheel event the same way (Ctrl-Left Ctrl-Right) and to map them as MouseWheelEvent in HandMorph class (generateKeyboarEvent:) but we lost the ability to use Ctrl-left / Ctrl-right for editing (jump from word to word). You could use Alt-left Alt-Right but it gets hard when you want to cut/paste along the way (the switch between Alt and Ctrl keys was prone to editing errors) It was a pain. So the question was about the ability to add an extra modifier at the VM level for the generated event, precisely to add ALT and CTRL extra modifiers to the generated event (and handling them correctly in the image of course - in HandMorph class first). This should be a non breaking change for the vm part (correct me if I'm wrong). I had a look at the vm code, I think it could be done for Unix in sqUnixX11.c at line 3533: int modifiers= modifierState ^ CtrlKeyBit; => Add ALT + CTRL modifiers (xored) For Windows (not really needed but eventually) quite the same in sqWin32Window.c at 266 evt->modifiers = CtrlKeyBit; (to check xored or not?) As stated by MS, at https://msdn.microsoft.com/en-us/library/ms997498.aspx the horizontal mouse wheel seems to be not really handled: "... Horizontal scrolling will not work, or is likely not to work, in the following scenarios: The application uses custom horizontal scroll bars. The application has no horizontal scroll bars. The horizontal scroll bar is physically placed in a nonstandard way relative to the window it is associated with. " As Pharo (squeak too) do not have native widgets , my understanding is that it should not work. =>No problem For Mac .... well I don't know mac but had a look at the source anyway sqMacUIEvents around l 1912 ? if (wheelMouseDelta != 0) { ... evt->modifiers = modifierMap[(controlKey >> 8)]; my guess-> the change goes around here ? May be an interested Mac developer could have a look and try? (nb I can do the mod build and test for unix and windows not for mac) (Sean ?) -- Regards, Alain |
Le 25/03/2015 00:02, Alain Rastoul a écrit :
> for the generated event, precisely to add ALT and CTRL extra modifiers > to the generated event tipo : to add ALT and *SHIFT* modifier CTRL is already here -- Regards, Alain |
Administrator
|
In reply to this post by Alain Rastoul-2
> I had a look at the vm code, I think it could be done for Unix in
> For Windows (not really needed but eventually) quite the same in Cool! I couldn't find those. I'll get on it! > May be an interested Mac developer could have a look and try? Already integrated in PharoVM :) btw the shortcut is now "ctrl + alt + shift + cmd + arrowKey", and as mentioned the VM change should be backward compatible with existing Squeak and Pharo
Cheers,
Sean |
Administrator
|
In reply to this post by Alain Rastoul-2
> I had a look at the vm code, I think it could be done for Unix in
> sqUnixX11.c at line 3533: > ... > For Windows (not really needed but eventually) quite the same in > sqWin32Window.c at 266 Okay, I took a first pass... https://github.com/pharo-project/pharo-vm/pull/73 I have no idea if any of this works because: a) I can't compile it because I'm on a Mac and b) I haven't programmed C in almost 20 years. But it's a start!
Cheers,
Sean |
Free forum by Nabble | Edit this page |
Cross-posting to Pharo dev, Squeak dev, and Squeak VM lists...
Recently, we discovered on the Pharo dev list that the key combination used to fake wheel events - ctrl + arrowKey - may be a bit too simple. Apparently ctrl+horizontalArrowKey is a standard in-use combo in Linux and Windows, so adding horizontal wheel events broke some users' workflows. So I propose "ctrl + alt + shift + cmd + arrowKey".
The new combo is:
For Pharo, there is:
NB: I changed the combo in the VM for Mac OS and OSX. Are these the only two platforms that fake wheel events? I searched a bit but didn't see similar logic for other platforms...