INFO REQUEST: key & mouse state machine / API

 
Greetings,

I have a side project to get the vm-display-fbdev up on Linux using
libevdev.

I am at the point where I have hacked the evtest program to capture
events bu need more info to "fill out the forms" (i.e. register events).

At this point I am confused as to the current state of "Cross Platform
Input Event" handling.

A state machine and usage API would be most helpful.


Current Status:

Events come in at each (debounced) keypress, with repeats
[sample-events.txt].

KeyModifiers (and MouseBUttons) are tracked as expected [misc-code.txt]

I need to fill in the event structures [e.g. record-key.txt].

I have been looking through the "Porting Squeak" and source code.


As the "Cross-platform differences for CTRL keys" topic has been raised
here, a state machine recognizer would be helpful to understand how
mouse/key/trackpad/gamepad/.. events are recognized across platforms  --
along with scancode>keycode translation mappings.

Perhaps a good project to undertake?


Any hints appreciated.  I really am confused as to what mappings are
required.

Thanks much,
-KenD

 
OK. What I am gleaning is that I should map from

/usr/include/linux/input-event-codes.h
#define KEY_L                   38
#define KEY_PAGEDOWN           109

to:

/usr/include/X11/keysymdef
#define XK_L               0x004c  /* U+004C LATIN CAPITAL LETTER L */
#define XK_l               0x006c  /* U+006C LATIN SMALL LETTER L */
#define XK_Page_Down       0xff56

UNICODE
  L -> 0x004C
  l -> 0x006C
  page down -> 0x21DF

given
   recordKeyboardEvent(int keyCode, int pressCode, int modifiers, int
ucs4)

When I see Key: (38 = x26)  'l'
-> recordKeyboardEvent( 0x006c, EventKeyChar, 0, 0x006c)

when I see Key: (38 = x26) + Shift  'L'
-> recordKeyboardEvent( 0x004c, EventKeyChar, ShiftKeyBit, 0x004c)

When I see Key: (109=0x6d) KEY_PAGEDOWN -> XK_Page_Down = 0xFF56
-> recordKeyboardEvent( 0xFF56, EventKeyUp, 0, 0x21DF)

Is this close to right?

Thanks much,
-KenD

 

 
Nicolas,

Thanks much for the reply.

I can deal with Work In Progress.  I appreciate the warning.

As usual, answers lead to more questions.

> Currently, we have to generate:
> - an EventKeyDown at each keydown, and at each repeat
> - an EventKeyStroke if ever the key is producing a character (or
> several
> EventKeyStroke if we produce several characters)
> - an EventKeyUp when the key is released

At present, if I hold down a modifier key (e.g. ctrl or shift) and a
key-char, I get a modifier key-down then a huge number of modifier key
repeats mixed in with a key-down and key-up of the character key.

I can see wanting the key-down and key-up with the character key, plus
the modifiers.  But as with many rackpad micro-moves, if the VM really
wants the modifier-key repeats, perhaps repeat count would save the VM
from having to drop unprocessed events.  I really don't see how modifier
key-doen/repeat/key-ups being useful in the absence of a char-key
down/repeat/up.  Really?  That is unexpected.  Easy to generate this.  I
can certainly try it out.

> Those events should have a platform independent key code in charCode
> field
> (event at: 3)
> And a 32 bits unicode character in utf32 field (event at: 6) - that is
> also
> known as ucs4

The keyboard libevdev event will give, e.g. ('l' + ShiftBit) rather than
'L' or ('2' + ShiftBit) rather than '@'.

Is the more 'raw' report desired, or should I use the upcased char-key?
[I presume the latter]

> When Ctrl+letter is pressed, it delivers an ASCII control character
> (code 1
> to 26) instead of the keycode of the letter

When I see Key: (38 = x26)  'l' + Ctrl + keyDown
-> recordKeyboardEvent( 0x0021, EventKeyStroke, CtrlKeyBit, 0x004c)

No problem.  I can map ctrl+l -> '^l' just as shift+l -> 'L'

> But PharoVM wants the uppercase character code in both charCode and
> utf32
> fields...

NOt quite sure how to encode both ^l and L in same charCode ;^)

 From the description, it sounds like:
   left-ctrl down
   left-ctrl repeat*
   char-l down
   left-ctrl repeat*
   char-l up
   left-ctel repeat*
   left-ctrl up

Where I was thinking of just reporting
   left-l down +Ctrl
   left-l up +Ctrl

...

sqUnixX11.c is a large and daunting file.  I will have to grovel over
the translateCode() to decode what is happening.  Not sure how
char-key+Shift+Ctrl+Alt+Meta is properly reported.

In particular, why not just track modifier key state (seet attached).  
The code seems to "erase" the meta-key state when reporting event rather
than when the key-up's are received.


> Having a single keystroke with precomposed unicode would be nice, but
> it
> can eventually be 2 keystroke with decomposed unicode...

At some point one must decide where characters are composed.  One can
give "cooked" chars to the VM, which allows for various input methods
for non-latin scripts.

vm-display-fbdev is much simpler than vm-device-X11.

I will try some experiments as things progress and see where I get into
trouble.

Thanks again for answering.  We all have busy lives..
-KenD

 

 

 



Hi Ken,

there is work in progress wrt keyboard events.

Currently, we have to generate:

- an EventKeyDown at each keydown, and at each repeat

- an EventKeyStroke if ever the key is producing a character (or several EventKeyStroke if we produce several characters)

- an EventKeyUp when the key is released

Those events should have a platform independent key code in charCode field (event at: 3)

And a 32 bits unicode character in utf32 field (event at: 6) - that is also known as ucs4

Currently, the key code is almost universal, but not realy, this is WIP as I said above.

The code is not explicit.

It matches iso latin1 encoding for printable characters.

Good news, it's also the choice of X11 KeySym and windows virtual key codes!

When Ctrl+letter is pressed, it delivers an ASCII control character (code 1 to 26) instead of the keycode of the letter

But PharoVM wants the uppercase character code in both charCode and utf32 fields...

For some well known "symbolic" keys (tab escape backspace del insert pageUp pageDown arrows...) the encoding chosen by OSVM is not that of X11 KeySym.

It's rather that of curses or ncurses (If I remember, it's too long ago, and too late to start searching the net), see function translateCode

https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/f70cf5f34041e0e197542651e90505a0b86325f6/platforms/unix/vm-display-X11/sqUnixX11.c#L4558

We also deliver a EventKeyChar (keystroke) for these events with the same encoding for utf32 field.

You get the essentials... You will have to do something with dead keys too (^ ¨ ` etc...).

Having a single keystroke with precomposed unicode would be nice, but it can eventually be 2 keystroke with decomposed unicode...

I was suggesting to further use thre reserved1 field to pass the device dependent keycode to the image, if ever we want to experiment leaner VM (and fater image!). But this will necessitate passing ore state, like caps lock, num lock, etc...

See https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/456

In other issues, I tried to reverse engineer all those layers.

Some are dusty, we do not change VM everyday...

And when we do it, we always add, never remove code, compatibility oblige!!!

I always tell that I pay removed lines twice than added, but only figurativley, otherwise my colleagues would immediately start committing many useless lines to pocket thrice ;)

Indeed.  One idea would be to split the file into three.  One would be a small “root” file that includes either sqUnixX11Legacy.c or sqUnixX11Spur.c. The existing, or last known good, or 4.5 release version would be made into sqUnixX11Legacy.c, and the current version into sqUnixX11Spur.c.  We can then remove any backward compatibility prior to 4.6, which IIRC is the first Spur release, or 5.0 if we don’t care about 4.x with Spur, since 4.6 was really a prerelease.  We leave sqUnixX11Legacy.c to rot, only ensuring it can compile, and attempt to cleanup sqUnixX11Spur.c as best we can.

 

> Am Fr., 27. Dez. 2019 um 23:29 Uhr schrieb Nicolas Cellier <
> [hidden email]>:

> Currently, we have to generate:
> - an EventKeyDown at each keydown, and at each repeat

BTW, I built current (Linux/Arm64) with DEBUG_MOUSE_EVENT and
DEBUG_KEYBOARD_EVENT.

Most unexpected results.

E.g. for every single up-or-down 'click' on the mouse wheel, I see three
(count 'em 3) events (DOWN/Key/UP).

I was expecting to see a single event.

Is there a deep reason against a single composite report per wheel
'click' ?

I understand about history, but ..


Thanks much,
-KenD