sq.h changes, grabbed reserved field in sqKeyboardEvent

In our work with Sophie we became aware of a shortcoming in how  
keyboard events are created and what data they contain
and what data is needed to transform keyboard command on various  
obscure European keyboards into meaningful interactions
within Tweak and other important applications like Sophie.

In the past on the macintosh and we believe on Windows the data  
coming up as a result of
a keystroke was

key down
charCode:   a magic number mapping to a key on the keyboard,  
different between mac/windows
pressCode: 0

key char
charCode the macroman value of the key pressed
pressCode the utf-32 value of the key pressed

key up
charCode:   a magic number mapping to a key on the keyboard,  
different between mac/windows
pressCode: 0

Now the results will be.

key down
charCode:   a magic number mapping to a key on the keyboard,  
different between mac/windows
pressCode the utf-32 value of the key pressed
additionalData the macroman value of the key pressed

key char
charCode the macroman value of the key pressed
pressCode the utf-32 value of the key pressed
additionalData a magic number mapping to a key on the keyboard,  
different between mac/windows


key up
charCode:   a magic number mapping to a key on the keyboard,  
different between mac/windows
pressCode the utf-32 value of the key pressed
additionalData the macroman value of the key pressed


This preserves the existing behaviour, but adds the ability to better  
support non-usa keyboards and character sets.
I will make this change btw weekend baring a more elegant solution.

/* keyboard input event */
typedef struct sqKeyboardEvent
{
   int type; /* EventTypeKeyboard */
   unsigned int timeStamp; /* time stamp */
   int charCode; /* character code in Mac Roman encoding */
   int pressCode; /* press code; any of EventKeyXXX */
   int modifiers; /* combination of xxxKeyBit */
   int utf32Code; /* UTF-32 unicode value */
   int additionCharacterData; /* Additional data about charcter for  
platform specific tweak dispatching */
   int windowIndex; /* host window structure */
} sqKeyboardEvent;


--
========================================================================
===
John M. McIntosh <[hidden email]> 1-800-477-2659
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

John -

John M McIntosh wrote:
> In our work with Sophie we became aware of a shortcoming in how keyboard
> events are created and what data they contain and what data is needed
> to transform keyboard command on various obscure European keyboards
> into meaningful interactions within Tweak and other important
> applications like Sophie.

What exactly is the shortcoming that you are referring to?

> In the past on the macintosh and we believe on Windows the data coming
> up as a result of a keystroke was
>
> key down
> charCode:      a magic number mapping to a key on the keyboard,
> different between mac/windows
> pressCode    the utf-32 value of the key pressed
> additionalData    the macroman value of the key pressed

This won't work at all on Windows. There is no way for me to generate
either a utf32 or a macroman value for "the key pressed". All that's
reported is a virtual key code which is arbitrarily assigned by Windows,
e.g., one of the VK_XXX constants.

> key char
> charCode    the macroman value of the key pressed
> pressCode    the utf-32 value of the key pressed
> additionalData    a magic number mapping to a key on the keyboard,
> different between mac/windows

And this won't work either on Windows, in particular not for
international keyboards. When keys are composed we get sequences like
"accent key down, accent key up, character key down, (composed)
character input, character key up". This happens on the Mac as well, for
example try pressing option-E first (you get keydown/keyup) and then e
(which gives you accent-e). So there is not necessarily a correspondence
between a key and its ascii value.

> This preserves the existing behaviour, but adds the ability to better
> support non-usa keyboards and character sets.

Could you please first state the problem before proposing a solution ;-)

Cheers,
   - Andreas

Andreas Raab wrote:

> Could you please first state the problem before proposing a solution ;-)

Michael Rueger wrote:
>> I'm not quite sure about the consequences for some of the weirder
>> keyboard layouts though:
>>
>> From a user's point of view (what the user thinks he pressed)
>> ctrl-shift-VK_1 on "normal" keyboards would mean ctrl-!,
>> on French ones it would mean ctrl-1 .

Ah, keyboard layouts. What a pain. Can we please (please, please,
please!) not require VMs to deal with those? It's a pain and it's a mess
and I'd rather have some image code deal with it. So, instead of
extending keyboard events, how about simply reporting a "keyboard layout
identifier"[*] that the image can use to map keys reported by the VM
into whatever it wants to?

[*] On Windows, that would be an identifier from GetKeyboardLayoutName
(http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winui/winui/windowsuserinterface/userinput/keyboardinput/keyboardinputreference/keyboardinputfunctions/getkeyboardlayoutname.asp)

Cheers,
   - Andreas

So why can't you use ToUnicode() to translate the virtual key code or  
the scan key code into the unicode character?

Sure I can return 42 the magic number from LMGetKbdType() meaning you  
are using as Estonia keyboard, or maybe
78 for Devanagari keyboard in the Indian Language Kit, however it's  
unlikely I or any other developer will want to code the
keyboard to ascii/macroman/unicode whatever translation tables in the  
image.


On 29-Mar-06, at 4:16 PM, Andreas Raab wrote:

--
========================================================================
===
John M. McIntosh <[hidden email]> 1-800-477-2659
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

Hi John -

> So why can't you use ToUnicode() to translate the virtual key code or
> the scan key code into the unicode character?

I wish it were that simple but it wouldn't work - not for the problem
statement that Mike posted (that's why I was asking for it).

The problem statement itself (which is perfectly valid from a user
perspective) is actually remarkably ill-defined from a system
perspective: It asks the key mapper to take three keys as input
(Ctrl-Shift-1) and generate 2 keys output (Ctrl-! or somesuch). The
reason this is ill-defined is that that's not how key mappers work (at
least the one's I know about) - all they care about is the combination
of all keys. Which means that ToUnicode will happily respond to
Ctrl-Shift-1 saying "oh, sorry, there is no Unicode character for that
combination".

And, IIRC (but I need to verify this) ToUnicode is the precise manual
equivalent of what Windows does anyway when it translates WM_KEYDOWN
into WM_CHAR. Which means that all you need is to look at what
characters get generated already to find out how ToUnicode would
respond. Which in the case of our Ctrl-Shift-1 example is not at all (I
tested this with various keyboard layouts which supports the theory).

So for the plain Unicode mapping I can use ToUnicode() - in fact I am
using it already (via WM_KEYDOWN -> WM_CHAR translation) and report it
in the utf32 field for the key stroke event. But I *can't* use
ToUnicode() to solve the problem we were discussing - to map the input
while making arbitrary distinctions between Ctrl and Shift in the
Windows key mapper (not unless we start masking out ctrl state which
immediately leads to other issues - that is exactly what I mean by "it's
a pain").

If you want such a distinction (and that's why the problem statement is
so important) you need to do it manually to begin with. And I'd rather
have that code in the image.

BTW, just out of interest: Does the Mac make a distinction between
certain modifiers when it maps keyboard input? I'd be quite surprised if
you'd get "Ctrl-!" out of mapping Ctrl-Shift-1 as input (but what do I
know ;-)

 > Sure I can return 42 the magic number from LMGetKbdType() meaning you
 > are using as Estonia keyboard, or maybe
 > 78 for Devanagari keyboard in the Indian Language Kit, however it's
 > unlikely I or any other developer will want to code the
 > keyboard to ascii/macroman/unicode whatever translation tables in the
 > image.

That's one good thing about Open Source: It only takes one programmer to
fix that problem and if he's out there the Internet can find him.

Cheers,
   - Andreas

  John,

> Sure I can return 42 the magic number from LMGetKbdType() meaning you  
> are using as Estonia keyboard, or maybe
> 78 for Devanagari keyboard in the Indian Language Kit, however it's  
> unlikely I or any other developer will want to code the
> keyboard to ascii/macroman/unicode whatever translation tables in the  
> image.

  And, why is that?  That is how it works right now in Squeak 3.8.

  If the VM is too clever but not take the Asian language input method
into account, I'm afraid that there will be no easy way to get the
functionality back.  On current Windows and some older versions of Mac
VM, people can input Japanese, Korean, Chinese, etc. into Squeakland
(Squeak 3.8 based).  But the last patched version of Unix VM (3.6-3 or
such), we needed to disable all the conversions implemented in the VM
after all.

-- Yoshiki

Here is another run at the fence:

John M McIntosh wrote:
> Sure I can return 42 the magic number from LMGetKbdType() meaning you
> are using as Estonia keyboard, or maybe
> 78 for Devanagari keyboard in the Indian Language Kit, however it's
> unlikely I or any other developer will want to code the
> keyboard to ascii/macroman/unicode whatever translation tables in the
> image.

What we could do for trying to solve the problem Mike described, is to
expose the functionality of ToUnicode or equivalent. E.g., instead of
having the VM doing all the mapping, allow the image request information
about "what would the result of mapping this key combination be" and
basically use the information we get out of the keyboard events (meta
state and key code). That way, if you wanted to map Ctrl-Shift-1 to
Ctrl-! you'd filter out the Ctrl key, map Shift-1 to "!" and are done.

The issue here is that I've got no idea how hard it is to support this
on other platforms...

Cheers,
   - Andreas

k, so on the key up/down can we pass up the unicode value? At least  
on the mac at that point I know the unicode values in relationship to
what the keyboard interaction was.
On 31-Mar-06, at 12:15 PM, Andreas Raab wrote:

--
========================================================================
===
John M. McIntosh <[hidden email]> 1-800-477-2659
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

John M McIntosh wrote:
> k, so on the key up/down can we pass up the unicode value? At least on
> the mac at that point I know the unicode values in relationship to
> what the keyboard interaction was.

Well ... to be honest, I'd rather not do that unless we have a good
reason to do it. Mostly, because people might get used to the idea to
grab a utf char out of a key down event where in reality that mapping is
inappropriate (unless you know precisely what you're doing and where). I
agree that it can be helpful to know the utf value (if one exist) but
I'm not even sure we can do this easily cross-platform (you still
haven't answered my question about Ctrl-Shift-1 on Macs from an earlier
email).

So if you could explain to me just once more which problem you're trying
to solve with sticking the utf char into these event's I'd greatly
appreciate it. I might appear daft here but I really don't want us to go
down a slippery path that'll lead to a system which works in 98% of all
cases but where the important ones are in the remaining 2%.

Cheers,
   - Andreas

Before discussing, let alone debating, mechanism perhaps we could  
agree on policy and expected outcome?

i.e. What should be the end result of whatever vm and image code is  
involved?

Once that is decided we can make sensible progress on the rest. If it  
takes each vm doing something different and one or more platform  
specific classes handling key code translation in some case and not  
in others, who cares? The end result for the user is all that matters  
to anyone but us chickens. *We* just need to worry if we can  
understand the mechanism in a year or twos time when some changes are  
needed.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Semolina - a  system of signalling with pudding.

Hi Tim -

> Before discussing, let alone debating, mechanism perhaps we could agree
> on policy and expected outcome?
>
> i.e. What should be the end result of whatever vm and image code is
> involved?

I think you're asking the same question that I asked, namely for the
problem statement.

Cheers,
   - Andreas

This was the original problem statement



On 31-Mar-06, at 2:42 PM, Andreas Raab wrote:

--
========================================================================
===
John M. McIntosh <[hidden email]> 1-800-477-2659
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

So what is the problem with the current version of the Mac VM not  
allowing people
to input Japanese, Korean, Chinese?


On 30-Mar-06, at 4:22 AM, Yoshiki Ohshima wrote:

--
========================================================================
===
John M. McIntosh <[hidden email]> 1-800-477-2659
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

On 31-Mar-06, at 2:42 PM, Andreas Raab wrote:
>
> I think you're asking the same question that I asked, namely for  
> the problem statement.
Not really; what I would like is a clear statement of what is wanted  
at the end. Well, maybe. Some people define a problem statement as a  
complaint of what isn't right, I suppose "It does this wrong and I  
want it to do that instead" is a reasonable stretch.

After whatever processing the vm and some optional image code does is  
complete, what information should be passed up to the portable side  
of the event system?  Under what conditions? What parts are nice but  
optional and what are utterly crucial?

Pressing 'm' pretty much works now. I think. Exactly what events  
*must* go to the portable event system for this?

What if you're on a japanese keyboard and have to do ctl-shift-alt-
meta-RMS- and then type 247 before releasing any?

What if your machine has a totally raw keyboard and tells you "key 7  
was pushed. key 42 was pushed. key 42 was released. key 17 was  
pushed. key 7 was released. key 17 was released"

What about an OS that tells you "character 'M' is here", ie you would  
have to infer the shift key being pressed from the character being a  
capital M.

Those are all plausible inputs to the best of my knowledge. What I'd  
like to see is an agreement about what the eventual outputs should be.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Klingon Code Warrior:- 10) "This code is a piece of crap!  You have  
no honor!"

  John,

> So what is the problem with the current version of the Mac VM not  
> allowing people
> to input Japanese, Korean, Chinese?

  It turned out that Squeak 3.8.11beta5U allows it (great!).  The
problem was the Unix VM...

-- Yoshiki

Oh, ya, I was following the unix vm for clues about how to implement  
the key down/up/char logic and noticed
it was broken.

I'll note that Tetsuya Hayashi [hidden email] supplied patches  
and testing to confirm the mac
carbon vm worked correctly for Japanese input back in april of 2004.

Also note all this work was re-factored on the conversion to unix  
file names and I have some concerns
that I might have made a mistake or two since I'm unable to  
understand how to fully test to confirm everything
works as expected, beyond careful reading.


On 1-Apr-06, at 6:24 PM, Yoshiki Ohshima wrote:

--
========================================================================
===
John M. McIntosh <[hidden email]> 1-800-477-2659
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===