[OpenSmalltalk/opensmalltalk-vm] Cross-platform differences for CTRL keys (#456)

See #396, #454, and #42. Maybe also #43.

I investigated the differences in the low-level event array sent by the VM to the image for relevant CTRL-key combinations.

For key-character events, that array is structured as follows:

{
   2 "type for keyboard event"
   293847 "timestamp"
   ... "character code ASCII"
   0 "key character, not key down or key up"
   2 "ctrl key as modifier pressed only"
   ... "character code as Unicode"
   0 "reserved"
   1 "host window number 1"
}

So, please focus on those character codes.

Windows 10 (Version 1909)
cog.spur 201911282316
Squeak 5.3beta 19276
Image format 6521 (32 bit)

mouse wheel up
#(2 91706953 30 0 2 0 0 1)
mouse wheel down
#(2 91731109 31 0 2 0 0 1)

ctrl + up
#(2 91746671 30 0 2 30 0 1)
ctrl + down
#(2 91778531 31 0 2 31 0 1)

ctrl + c
#(2 91802453 3 0 2 3 0 1)

Result: Our mouse-wheel hack does not set the Unicode character. Only the ASCII character.

Debian 10 (Gnome, X11)
cog.spur 201911282316
Squeak 5.3beta 19276
Image format 68021 (64 bit)

mouse wheel up
#(2 109872 30 0 15 30 0 1)
mouse wheel down
#(2 109923 31 0 15 31 0 1)

ctrl + up
#(2 212175 30 0 2 30 0 1)
ctrl + down
#(2 249663 31 0 2 31 0 1)

ctrl + c
#(2 184765 99 0 2 99 0 1)

Result: Our mouse-wheel hack raises all modifier keys as pressed. Not just the CTRL key. Also, the ASCII control characters are not considered. See https://en.wikipedia.org/wiki/Control_character.

macOS Sierra (10.12.6)
cog.spur 201911282316
Squeak 5.3beta 19276
Image format 68021 (64 bit)

mouse wheel up
#(2 57855 30 0 15 30 0 1)
mouse wheel down
#(2 84690 31 0 15 31 0 1)

ctrl + up
#(2 290999 30 0 2 30 0 1)
ctrl + down
#(2 322197 31 0 2 31 0 1)

ctrl + c
#(2 337191 3 0 2 3 0 1)

Result: Our mouse-wheel hack raises all modifier keys as pressed. Not just the CTRL key.

Implications for the image side: We cannot consistently (or easily or without magic) duplicate or swap or unpack character codes via KeyDecodeTable in EventSensor >> #processEvent: because:

• The Linux platform does not transport control characters to the image.
• The Windows platform omits to set Unicode for mouse-wheel events.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Some addition to Debian 10 (Gnome, X11) from above:

ctrl + [
#(2 729109 27 0 2 91 0 1)
ctrl + ]
#(2 773893 29 0 2 93 0 1)

ctrl + {
#(2 564471 27 0 3 123 0 1)
ctrl + }
#(2 665358 29 0 3 125 0 1)

ctrl + C
#(2 808422 67 0 3 67 0 1)

So, it seems that only for ctrl + Aa-Zz those ASCII and Unicode codes got duplicated. That's very strange.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Note: linux behavior is not strange, it is deliberate, see this:
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/a4a08f98c17b23ad54b5e361ffc0280e2ea8e0c3/platforms/unix/vm-display-X11/sqUnixX11.c#L2065

It was introduced with 1b837f9

If we want to fix it, we can just enclose this piece of code with #ifdef PharoVM, as it is already the case for similar code on windows:
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/a4a08f98c17b23ad54b5e361ffc0280e2ea8e0c3/platforms/win32/vm/sqWin32Window.c#L1308

Though, on windows, only the utf32 field (event at: 6) is changed, while on linux charCode field (event at: 3) is also changed.

I do not see equivalent code on OSX...

However, I suggest deeper change, as in this branch:
https://github.com/OpenSmalltalk/opensmalltalk-vm/tree/Nuke-MacRoman

We should have charCode transformed into a platform independent keyCode, hence not being an ASCII control character, but rather a letter in case of Ctrl+a. (X11 KeySym and windows Virtual Key codes already match ASCII for keys representing ASCII characters).

Then the utf32 can eventually be the ASCII control character or not, whatever we want, we have to decide...

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

I have fixed ctrl+a to ctrl+z on linux and mousewheels on windows (all meta bits are now set on all OS). Please check and report if we can close.

Note 1: If we want, we can also generate mousewheels events on windows (instead of arrow key+meta bits events), I opened a branch for that. But trackpads may generate way too many small deltas that overwhelm the image side event loop, so not sure it is ready.

Note 2: for exotic ctrl+key, I'm afraid that it will be difficult to obtain consistent behavior cross-OS cross-keyboard...

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Hi Nicolas,

I'm following the discussion on the background. I think you're right on using the keycode and not an ascii code. Using ascii to identify keys is a vestige from old "silly terminals". However, there are no keys in ascii to identify for example keys like ctrl, cmd or alt by themselves (and not as modifiers).

Regarding keyboard events (dunno about mousewheel in particular), my take here is that existing windowing/ui frameworks (cocoa/carbon, x11, windows forms..) send two different kind of events:

• key up / key down => identify wether a key is pressed or released with key code, no text is available. Here we can catch single ctrl/alt/cmd/F[X]/etc presses.
• a text event (which we call key press ourselves, maybe misleading) which sends text in some encoding as soon as it is available + associated modifiers

Now, the complication here to me is that historically morphic (in squeak and pharo, probably cuis still too?) has been using the text event to drive everything, even shortcuts. So the code managing keyup/downs is not used at all in my experience, and the code managing keypresses in morphs is overly complicated. And changing this while changing the VM at the same time can lead easily to a state where we cannot type :).

Now, as a single small step in what I believe is the right direction, I've implemented in Pharo mappings of keyboard virtual tables => the VM sends the raw virtual keycode to the image side (which depends on the platform) and on the image side this platform dependent virtual keycode is mapped to a common representation for all of them. Like that a single virtual key code table is used in client code. When I did that a couple of years ago, I've chosen to use "in image" X11's virtual key codes because it looked the broader and it seemed to contain the ones from osx and windows. Check https://github.com/pharo-project/pharo/blob/Pharo8.0/src/System-Platforms/KeyboardKey.class.st.

Then the idea is that morphic events are extended to get the correct key object with a double dispatch:

KeyboardEvent >> key
	^Smalltalk os keyForValue: keyValue

UnixPlatform >> keyForValue: aKeyValue
	^KeyboardKey valueForUnixPlatform: aKeyValue.

Still missing, some accessors for common keys like KeyboardKey f11 KeyboardKey ctrl, which would allow for code like the following to open/close a morph while a key is being pressed (without an active wait ;)).

MyMorph>>keyDown: anEvent
	anEvent key = KeyboardKey alt
             ifTrue: [ self openWindowSelector ]

MyMorph>>keyUp: anEvent
	anEvent key = KeyboardKey alt
             ifTrue: [ self closeWindowSelector ]

A similar approach could be implemented vm side to map events. I like however the image one because it can be changed easily :)

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Moreover, as soon as we are outside the "occidental" keyboard mappings, the one key -> one letter mapping becomes false :). For example, the character 腕 is a single character and has a single unicode code point. Thus, it comes as a single keypress/keychar event from the vm point of view.
However, I had to make several key presses: u + d + e + space + enter. So using text events to drive shortcuts and recognise wether a key was pressed or not falls short very quickly ^^. So recognizing wether I was pressing ctrl or alt during that character is far from obvious...

(then there is also integration with the system's IME, but that's another story...)

Sorry if I'm a bit offtopic

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Hi Guile,
I think that we can find an agreement. X11 KeySym looks like the best choice to me, and I would like to use that in (event at: 3). Apart the arrow, prior, next, home, end, BS, delete, insert, Escape, Tab that we already map to some ASCII control character in a platform independent manner (breaking compatibility on those might be a pain).

Note that the squeak is using keystroke in lower event level, and OSVM is using EventKeyChar, when a character is produced (not just a key pressed).
Shortcuts should probably be handled at EventKeyDown like it usually is on various OSes (that's the windows recommended way). We can as well connect Morphic keyPressed: to event keydown if we want to, not too difficult.

Note that we also have unused slots (reserved1) in keyboard events, and I thought that we could use that to transfer OS dependent keyCode. This way, we could experiment both image side handling, and VM side handling of more complex events (composition etc...). Also, we would preserve a smooth transition without complete locked in syndrome.

Note that even for occidental languages, I'm not satisfied with linux handling of dead keys (^ ¨ `) at VM side, I get decomposed unicode while the image rather expect precomposed (we can change the image for simple thing like that, but it's lot more work for more complex cases like you have shown).

So IMO, OSVM should provide:

• a platform independent keycode in (event at: 3)
• utf32 (ucs4) character in (event at: 6) at least for EventKeyChar
• a platform dependent keycode (reserved1 - to be renamed) for image side experiments
  There can be keydown events without corresponding keychar event.
  There can be several consecutive keychar events following a single keydown event.
  There can be keydown/keychar events without keyup event for repeated keys, and the OSVM should take repeat count of keys into account! (pass that in some unused event field too...).

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.