[squeak-dev] The state of Squeak I18n support: let's improve it

> There has been a lot of excellent work done on Unicode in Squeak.
> Unfortunately, not all of it works 'out of the box'.
>
> I've started working on trying to figure out how to enable the basic
> operations (input, display, and copy/paste) for several languages.
> With the kind help of several Russian squeakers, I've managed to
> create relatively simple instructions on how to do so for Russian,
> which I've posted here: http://wiki.squeak.org/squeak/5773 ; I've
> personally tested the 3.10 instructions under Linux.  However, getting
> to this point involved days of experimentation, including quite a lot
> of image hangs.  The reason I created the documentation was that,
> despite reading everything on squeak.org and the mailing list archives
> about Russian and Cyrillic support, I still couldn't get it working -
> it took combining the instructions of two people who had had similar
> experiences, and the patches of two others, to have everything work,
> although there are some problems with garbage characters with
> copy/paste sometimes.
>
> A side effect of these instructions is that various accented European
> characters (such as à, å, ò, ö, ĵ, ñ, ç, č, ć, š, đ, etc -  
> everything
> I've tried) work.  This cursory testing suggests that
> Italian/French/German/Spanish/Swedish/Esperanto/Slovenian/etc
> operations also work after following the Russian instructions; they do
> not work by default.  Perhaps more interestingly, Thai and Greek also
> work; Arabic nearly does, but only displays in its 'general Unicode'
> form (rather than correctly changing what appears based on the
> position of each letter in the word: see
> http://en.wikipedia.org/wiki/Arabic_alphabet for a better
> description).
>
> I'm now going through the same process for Japanese.  Installing
> 'Japanese Environment Installer' (JEI) from SqueakMap only gave me a
> working display of Japanese characters, without working input or
> copy/paste.  Possibly interestingly, direct input while using a
> Japanese keymap (with no input method) to type kana works with the
> Russian patches, but not with the JEI patches.
>
> Nothing using extra input method software works anywhere in Squeak in
> any configuration I've tried.  Deadkeys also don't work.
>
> I would like to request help from all Squeakers who can input,
> display, and copy/paste text in languages which have characters or
> writing systems that don't appear in the languages mentioned above,
> and especially of those who have everything working with Japanese.
> Any instructions on how you did it would be greatly appreciated.  I
> intend to keep documenting the simplest ways to currently set things
> up, which will hopefully have the side effect of making a baseline set
> of patches to have things Just Work become clear.  Any reports of
> errata in the documentation, platform differences, or bugs are also
> extremely welcome.
>
> Thank you;
> Katerina Barone-Adesi

>> We fixed many of these shortcomings in the OLPC Linux VM and image.  
>> Keyboard
>> input works with dead keys, compose keys, and XIM. Clipboard works  
>> with
>> unicode, formatted/rich text and images (this builds on the Sophie  
>> guys'
>> work). There is rendering of various text scripts via Pango,  
>> including e.g.
>> Devanagari with correct glyph shaping (Nepal is one of the pilot  
>> countries).
>
>
>
> I've tried the OLPC Squeak image, using the instructions at
> http://wiki.laptop.org/go/Etoys#Method_1_.28Easiest.29_-_Use_Squeakland_installation_and_the_OLPC_image
> and the link to what it says is the newest image:
> http://etoys.laptop.org/src/etoys-image-and-pr.zip .  The results were
> mixed.
>
> Successes:
> - Accented Latin characters work immediately.
> - Japanese support partially works: switching to Japanese, fonts are
> successfully loaded, and Japanese is displayed.
>
> Problems:
> - Deadkeys don't work.
> - Anthy + SKIM didn't work.
> - typing kana by using a Japanese keymap shows up as '?', even when
> etoys is showing up properly in Japanese.
>
> - Missing fonts and broken 'choose language' support:
> --typing Arabic only shows '?', and selecting Arabic as a language
> doesn't change that; nothing seems to happen.
> --selecting 'Persian' translates the user interface into a series of
> '?', with a tiny bit of English for the strings that aren't
> translated; the necessary font is missing, and there is no prompt to
> add it.  This also applies to Singhalese and Urdu.  Turkish has a much
> more minor version of this problem - most letters are there, but a few
> are missing.
> --typing Greek only shows '?'.  Selecting Greek as a language says
> "This language needs additional fonts.  Do you want to install the
> fonts? Yes / No".  Selecting 'yes' brings up a debugger, because the
> .sar file for Greek fonts is missing.  This also applies to Russian,
> Chinese, and Romanian.
> -- Selecting Korean as a language brings up a debugger: "Error: My
> subclass should have overridden #leadingChar".
>
> I didn't evaluate Marathi, Mongolian, Nepali, Pushto, or Telegu, since
> I don't have my system set up to input in them and none of the strings
> that I happened to have on my screen were translated when I switched
> to them.  I also didn't evaluate copy/paste for anything.
>
> So: the situation is slightly better than what
> http://www.nabble.com/3.10---Mac-OS-X-Leopard---accent-chars-and-keyboard-input-td15332280.html
> led me to believe.  Accented Latin characters, when typed directly
> (ie, without deadkeys/input methods) work out of the box in the OLPC
> image, which I hope makes it into the basic Squeak image soon.
> However, there are definitely still problems left as well.
>
> If any of this is user error on my part, I'd be very glad. Assuming at
> least some of it is not, while the OLPC image is a step in the right
> direction - and probably would make a better baseline than the 'basic'
> Squeak image - I'm still left with several of the problems my previous
> email mentioned, and still very glad to hear about any successes
> people have had with them.  Some of this stuff has clearly worked for
> some people at some point, which is part of what makes the current
> situation and its regressions so unexpected.
>
> Regards;
> Katerina Barone-Adesi

> --selecting 'Persian' translates the user interface into a series of
> '?', with a tiny bit of English for the strings that aren't
> translated; the necessary font is missing, and there is no prompt to
> add it.  This also applies to Singhalese and Urdu.  Turkish has a much
> more minor version of this problem - most letters are there, but a few
> are missing.
> --typing Greek only shows '?'.  Selecting Greek as a language says
> "This language needs additional fonts.  Do you want to install the
> fonts? Yes / No".  Selecting 'yes' brings up a debugger, because the
> .sar file for Greek fonts is missing.  This also applies to Russian,
> Chinese, and Romanian.

> I've tried using the OLPC VM [1] and
> http://etoys.laptop.org/src/etoys-dev.zip (with all 239 updates).  The
> results are improved, but still mixed:
>
> What works:
> - dead keys (as long as I run squeak -vm-display-X11 -compositioninput
> etoys-dev.image).
>
> What half-works:
> - Japanese input.  I can type with Anthy+Skim and see hiragana
> correctly, but as soon as I press space or enter, everything but the
> first character turns into gibberish (not Kanji - accented Latin
> characters and unicode missing-character boxes).  I don't think it's a
> font problem, as it looks ok while I'm inputting it, and when I switch
> to Japanese, all of the menus show up properly in Japanese.
>
> What doesn't work:
> - Russian input.  It mainly shows unicode boxes as well,

> [1]: http://etoys.laptop.org/rpms/squeak-vm-3.10-3olpc10.i386.rpm and
> squeak -version:
> 3.10-3 #1 Fri Aug 29 19:30:22 CEST 2008 gcc 4.1.2
> Squeak3.10beta of 22 July 2007 [latest update: #7159]
> Linux fedora7 2.6.23.17-88.fc7 #1 SMP Thu May 15 00:35:10 EDT 2008
> i686 i686 i386 GNU/Linux
> default plugin location: /usr/lib/squeak/3.10-3/*.so
>
> Regards;
> Katerina Barone-Adesi
> <bad-input-cropped.png>

>>> What doesn't work:
>>> - Russian input.  It mainly shows unicode boxes as well,
>>
>> Were the (few) translated strings correctly displayed in Cyrillic?  
>> E.g., in
>> the Supplies flap the book morph should be labeled "книга".
>
> No, because Russian wasn't one of the language options, so I didn't
> have any translated strings.  However, copying text from wikipedia in
> Russian into the workspace showed it correctly.   Japanese was an
> option, and the translated strings displayed correctly.
>
>>> Also, oddly, there are many less language options: only German,
>>> English, Spanish, French, Japanese and Portuguese.  The rest have
>>> disappeared.
>>
>> That is indeed odd. Apparently it does not find the .mo files in  
>> the locale
>> directory next to etoys.image.
>
> There are no .mo files (and there is no locale directory) in
> etoys-dev.zip; it contains   QuickGuides/index.pr,
> QuickGuides/preload-index.sexp.data.gz, Gallery.020.pr,
> EtoysActivity.004.pr, EtoysV3.sources, Tutorials.009.pr,
> etoys-dev.image, etoys-dev.changes, and po/etoys/, which contains 29
> po files.