SqueakVM help for OLPC

> Folks,
>
> Since I'm not sure all of you have followed the discussion about  
> scroll events (which then went into keyboard events) here's my  
> proposal again: I would like to add a utf32Code field to  
> sqKeyboardEvent which serves the purpose to provide a UTF-32 value  
> if the VM provides extended input services (the Mac VM already  
> does; others will surely follow). The utf32Code field only needs to  
> be filled in for EventKeyChar events (e.g., neither for  
> EventKeyDown or EventKeyUp since those are used for reporting keys  
> not characters).
>
> John is already playing with this but I want to make sure there is  
> general agreement on this issue. Ian, Tim? Any extra comments?
>
> Cheers,
>   - Andreas
>

> In our work with Sophie we became aware of a shortcoming in how  
> keyboard events are created and what data they contain
> and what data is needed to transform keyboard command on various  
> obscure European keyboards into meaningful interactions
> within Tweak and other important applications like Sophie.
>
> In the past on the macintosh and we believe on Windows the data  
> coming up as a result of
> a keystroke was
>
> key down
> charCode:   a magic number mapping to a key on the keyboard,  
> different between mac/windows
> pressCode: 0
>
> key char
> charCode the macroman value of the key pressed
> pressCode the utf-32 value of the key pressed
>
> key up
> charCode:   a magic number mapping to a key on the keyboard,  
> different between mac/windows
> pressCode: 0
>

> Now the results will be.
>
> key down
> charCode:   a magic number mapping to a key on the keyboard,  
> different between mac/windows
> pressCode the utf-32 value of the key pressed
> additionalData the macroman value of the key pressed
>
> key char
> charCode the macroman value of the key pressed
> pressCode the utf-32 value of the key pressed
> additionalData a magic number mapping to a key on the keyboard,  
> different between mac/windows
>
>
> key up
> charCode:   a magic number mapping to a key on the keyboard,  
> different between mac/windows
> pressCode the utf-32 value of the key pressed
> additionalData the macroman value of the key pressed
>
>
> This preserves the existing behaviour, but adds the ability to  
> better support non-usa keyboards and character sets.
> I will make this change btw weekend baring a more elegant solution.
>
> /* keyboard input event */
> typedef struct sqKeyboardEvent
> {
>   int type; /* EventTypeKeyboard */
>   unsigned int timeStamp; /* time stamp */
>   int charCode; /* character code in Mac Roman encoding */
>   int pressCode; /* press code; any of EventKeyXXX */
>   int modifiers; /* combination of xxxKeyBit */
>   int utf32Code; /* UTF-32 unicode value */
>   int additionCharacterData; /* Additional data about charcter for  
> platform specific tweak dispatching */
>   int windowIndex; /* host window structure */
> } sqKeyboardEvent;

> On May 15, 2007, at 20:51 , Chris Petsos wrote:
>
>> We are trying to make Squeak work with Unicode...Yoshiki Ohshima has
>> been very helpful till now and his suggestion was to let all the data
>> that the platform (e.g. Windows, Unix etc...) is sending,  enter the
>> Squeak VM as is and deal with them there (in the image side) with
>> InputInterpreter classes.
>
> Yeah, I know, for some strange historical reason the Japanese folks  
> like this approach ;)
>
> I much prefer the very simple solution that John put into the Mac  
> VM - add a char32 field to each KeyStroke event containing the  
> unicode value. Nice and simple.
>
>> Another clue is that Squeak in Windows uses ANSI versions of Windows
>> functions which support only 8-bit characters
>
> That's highly irrelevant to the Unix VM. And for filenames we'd  
> most likely should use UTF-8 which is 8-bit and actually supported  
> by the Unix VM right now.
>
> - Bert -
>
>

> > We are trying to make Squeak work with Unicode...Yoshiki Ohshima has
> > been very helpful till now and his suggestion was to let all the data
> > that the platform (e.g. Windows, Unix etc...) is sending,  enter the
> > Squeak VM as is and deal with them there (in the image side) with
> > InputInterpreter classes.
>
> Yeah, I know, for some strange historical reason the Japanese folks  
> like this approach ;)
>
> I much prefer the very simple solution that John put into the Mac VM  
> - add a char32 field to each KeyStroke event containing the unicode  
> value. Nice and simple.

> Just for the record this is what we have done till now for Unicode  
> support in the Windows VM...
> We've made the application send Unicode chars to the image window  
> and forward this unicode value to the utf32Code data member with  
> the following...
>
> --- sqWin32Window.c 2007/05/18 06:04:50 1.1
> +++ sqWin32Window.c 2007/05/18 10:26:22
> @@ -799,6 +799,9 @@
>    CreatePrefsMenu();
>    SetWindowTitle();
>    SetForegroundWindow(stWindow);
> +/* Force Unicode WM_CHAR */
> +SetWindowLongW(stWindow,GWL_WNDPROC,GetWindowLong
> (stWindow,GWL_WNDPROC));
> +SetWindowLongW(consoleWindow,GWL_WNDPROC,GetWindowLong
> (consoleWindow,GWL_WNDPROC));
>  #ifndef NO_DROP
>    /* drag and drop needs to be set up on per-window basis */
>    SetupDragAndDrop();
> @@ -1069,7 +1072,7 @@
>    ctrl = GetKeyState(VK_CONTROL) & 0x8000;
>    /* now the key code */
>    virtCode = mapVirtualKey(msg->wParam);
> -  keyCode = keymap[msg->wParam];
> +  keyCode = msg->wParam;
>    /* press code must differentiate */
>    switch(msg->message) {
>      case WM_KEYDOWN:
> @@ -1110,14 +1113,14 @@
>    evt = (sqKeyboardEvent*) sqNextEventPut();
>    evt->type = EventTypeKeyboard;
>    evt->timeStamp = msg->time;
> -  evt->charCode = keyCode;
> +  evt->charCode = keymap[keyCode & 0xff];
>    evt->pressCode = pressCode;
>    evt->modifiers = 0;
>    evt->modifiers |= alt ? CommandKeyBit : 0;
>    evt->modifiers |= shift ? ShiftKeyBit : 0;
>    evt->modifiers |= ctrl ? CtrlKeyBit : 0;
>    evt->windowIndex = msg->hwnd == stWindow ? 0 : (int) msg->hwnd;
> -  evt->utf32Code = 0;
> +  evt->utf32Code = keyCode;
>    /* clean up reserved */
>    evt->reserved1 = 0;
>    /* note: several keys are not reported as character events;
>
> In the image side we've created a UnicodeInputInterpreter which is  
> dependent on this sixth field (the utf32Code in the evtBuf Array)  
> for acquiring the unicode value with the following method...
>
> ----------------------------------------------------------------------
> ----------------------
> nextCharFrom: sensor firstEvt: evtBuf
>
> | keyValue |
> " Unicode charCode expected as the sixth data member ! "
> keyValue := evtBuf sixth.
> Transcript show: evtBuf asString; cr.
> ^ keyValue asCharacter.
>
> ----------------------------------------------------------------------
> ----------------------
>
> So this is it...we have unicode charcodes inside the image...
>
>>  If we can replace all VMs, that would do.  Of course, the image can
>> check the VM version so that they can install proper "interpreter"
>> (including non-interpreting one for later VMs.)
>>
>>  It should be better to use pre-composed chars for these chars, but
>> I'm not entirely sure if it makes sense for all different input
>> methods.
>
> If you have any suggestions-modifications that you feel that will  
> help us avoid problems in the future, they are most welcome!
>
> Right now we trying to load a Unicode font. We've tried your font  
> Yoshiki (greek.out) but it has some problems. We are having a '?'  
> character along with every "carriage return"...
> The same effect happens when we try to load TrueType fonts... we  
> have a small "TrueType Box" in the place of  '?' :-) .
> Any help on how we can properly load a unicode font?

>> Squeak's TTF importer discards code points > 255. Here is a hack  
>> around this:
>>
>> http://lists.squeakfoundation.org/pipermail/squeak-dev/2006-June/ 
>> 105184.html
>
> When i load the package i get this Syntax Error...
> "Postscript:"
> self inform: 'If keyboard input stops working now, please disable  
> the 'unicodeInput'Nothing more expected -> preference'

>
> On Wednesday 16 May 2007 12:39 am, Bert Freudenberg wrote:
> > On May 15, 2007, at 20:51 , Chris Petsos wrote:
> > > We are trying to make Squeak work with Unicode...Yoshiki Ohshima has
> > > been very helpful till now and his suggestion was to let all the data
> > > that the platform (e.g. Windows, Unix etc...) is sending,  enter the
> > > Squeak VM as is and deal with them there (in the image side) with
> > > InputInterpreter classes.
> >
> > Yeah, I know, for some strange historical reason the Japanese folks
> > like this approach ;)
>
> This woud be a preferred approach for Indic languages too. The ability to
> switch layouts through hotkeys is crucial because of multiplicity of
> languages (over 23!). I could start a paragraph in Kannada layout, switch

> Have you tried handling WM_UNICHAR instead of tricking the VM into
> Unicode mode? If you do what you did above some other messages come
> in
> with Unicode args too and while this may be desirable at some point
> I'm
> not sure if we want this right away. For the time being I'd feel
> safer
> if we would handle WM_UNICHAR explicitly. Also, WM_UNICHAR uses
> (according to the docs) UTF-32 instead of UTF-16 as in your code.
> This
> might come in handy at some point.