Keycaps??

How do I discover how to type typographic quotes on a Mac? I'm discovering
some of the combinations just by trial and error -- e.g. for double-quotes
option-shift-l option-shift-h seems to work. For the single quote I have
option-shift-j for the opener and ??? -- I can't find the closing single
quote. (In the MorphicFontEditor it shows me this as character D5.)

Where did this keyboard mapping come from??

More generally, how do I get the whole keyboard mapping? MacOS classic has
a program called keycaps that shows all the characters from various
shift-ish combinations.

How do I "discover" the keyboard mappings? This is really frustrating.

Is there a reason these key mappings don't follow "standard Mac"?

-Thanks, Jim

You should indicate which macintosh VM and version you are using.

Under os-x you go to system preferences, the international icon,  
select the Input Menu tab and ensure the "Show input menu in menu  
bar"  is checked
that should show an icon in  your menu bar which you can select, then  
from that menu select "Show keyboard viewer"

Under os-x the keystrokes are given as key down, key char and key up  
events to Squeak. These values are given in MacRoman, Unicode, and  
keyboard keycodes with depending on how they are entered, either from  
the keyboard or via an keyboard input palette. Usually the MacRoman  
values are given after os-x translation services had decide what to  
do with the multiple keystrokes involved.

The three key board events are then mangled by Smalltalk code into  
what you actually get in Morphic, and of course differently in Tweak.

You need to give a bit more information about the keyboard  you have  
set in os-x preferences, which keystrokes you are using, what you  
expect that keystroke to give you
say in TextEdit, and what happens in Squeak.




On 10-Jun-06, at 11:44 AM, Jim Rosenberg wrote:

--
========================================================================
===
John M. McIntosh <[hidden email]> 1-800-477-2659
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

> You should indicate which macintosh VM and version you are using.

VM is 3.8.6Beta6.app.

Squeak is 3.8 update #6665.

> You need to give a bit more information about the keyboard  you have  set
> in os-x preferences, which keystrokes you are using, what you  expect
> that keystroke to give you say in TextEdit, and what happens in Squeak.

I'm showing US keyboard script Roman.

The "normal" keys for typographic quotes I expect are:

Double-quotes: option-[, shift-option-[

Single-quotes: option-], shift-option-]

They work fine in every app except Squeak. (They work fine in TextEdit.)

When I bring down the character palette, highlight the right single quote
and click insert, Squeak shows it as simply a question-mark. When I type
shift-option-], instead of getting the right single quote, I'm getting the
character at position 92 in the MorphicFontEditor.

It sounds like my keyboard mapping has gotten confused somehow -- if you
can straighten me out I'd be much obliged!

-Thanks, Jim

> When I bring down the character palette, highlight the right single quote
> and click insert, Squeak shows it as simply a question-mark. When I type
> shift-option-], instead of getting the right single quote, I'm getting
> the character at position 92 in the MorphicFontEditor.

Oops, correction. I had the wrong character in the character palette. When
I insert from the character palette into Squeak the right-single quote also
shows as character 92. I'm wanting D5.

-Thanks, Jim

Works fine in a 3.5-5180 image, likely an artifact of:

a) switch from original apple fonts to accufonts
b) multiple language support.

likely someone else on the list can explain which of these two items  
is the culprit.

I'll note if you type option-] in a workspace  you see nothing, but  
if you copy/paste that nothing to TextEdit why you get the expected  
characters.

On 10-Jun-06, at 4:01 PM, Jim Rosenberg wrote:

--
========================================================================
===
John M. McIntosh <[hidden email]> 1-800-477-2659
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

Jim,

Squeak has switched from MacRoman encoding to Unicode in 3.8. The 8-
bit subset of unicode, iso-8859-1 (a.k.a. Latin1) is what can be  
reached by old-style keyboard mappings and fonts. While MacRoman  
*did* have english typographic quotes, Latin1 only has french («...»):

        http://en.wikipedia.org/wiki/MacRoman

        http://en.wikipedia.org/wiki/Latin1

Indeed, I can type french quotes fine in a 3.8 Squeak.

Note that Windows puts english typographic quotes into the reserved  
slots of Latin1, which makes people use them, which makes browser  
developers show them as if they were indeed valid latin1 chars, which  
leads to severe confusion:

        http://en.wikipedia.org/wiki/Windows-1252

While we could employ a similar hack, the *proper* way for us would  
be to use unicode text, which Squeak should be able to do since 3.8  
thanks to the fine Japanese folkss:

        WideString from: #(16r2018 16r2019 16r201C 16r201D)

This is displayed as '????', because we lack unicode fonts containing  
the right glyphs.

The Mac VM already provides Unicode keyboard input, as 6th element of  
the event buffer. It's unused sofar. You can use it by inserting the  
following line into MacUnicodeInputInterpreter>>nextCharFrom:firstEvt:

        evtBuf fourth = EventKeyChar ifTrue: [^Unicode value: evtBuf sixth].

Leave the rest of that method in place, even though it is incorrect  
(#macToSqueak should produce Unicode chars nowadays, but for  
historical reasons still produces an 8-bit mapping).

Also, to make Morphic use that interpreter temporarily:

        LanguageEnvironment classPool at: #InputInterpreterClass put:  
MacUnicodeInputInterpreter.
        ActiveHand clearKeyboardInterpreter.

With that, Option-[ actually inserts the character valued 16r201C,  
though it still is displayed as "?". You can verify that using

        $? asUnicode hex

(where the "?" is typed as "Option-[") which gives '16r201C'.

There are free bitmapped (BDF) and vector (TTF) fonts, but I am not  
sure if anyone has tried to make them available in Squeak, yet:

        http://freeunifont.sourceforge.net/
        http://www.nongnu.org/freefont/

Also, I'm unsure about the unicode support of VMs other than John's.

- Bert -

Am 11.06.2006 um 07:34 schrieb John M McIntosh:

I just hacked a bit of Unicode support into the truetype fonts. See  
attachment.

- Bert -




Am 12.06.2006 um 15:35 schrieb Bert Freudenberg:

Here is another one:
http://map.squeak.org/account/package/2c1a81e1-4e86-40c8-90b5-824adc4263c5
(using it russian language support:
http://minnow.cc.gatech.edu/squeak/5773 )

I'm wondering for a quite a long time - are we 'unicodized' or not yet?
Is the current state of m17n kind of final and I just don't get how to
use it or something else should be done? What do other non-latin folks
think about it? Does the fact that nobody but me complains mean that all
is ok?
A lot of questions and it is not a sarcasm.

As for me, stock image and VMs definitely are not enabled for Russian.

Danil

sorry, this link:
http://map.squeak.org/package/2c1a81e1-4e86-40c8-90b5-824adc4263c5
> Here is another one:
> http://map.squeak.org/account/package/2c1a81e1-4e86-40c8-90b5-824adc4263c5 
>
> (using it russian language support:
> http://minnow.cc.gatech.edu/squeak/5773 )

On Tue, 13 Jun 2006 19:09:30 +0200, Bert Freudenberg wrote:
> I just hacked a bit of Unicode support into the truetype fonts. See
> attachment.

Hey, this picture looks great! Now the ISO entity set groups are waiting  
for being rendered in Squeak :)

- http://www.w3.org/TR/1998/PR-math-19980224/chapter6.html

Thank you Bert and all others who made unicodization happening in Squeak.

/Klaus

> - Bert -
>
>

Well since we are working with macroman, unicode and latin-1 let me  
remind folks of an outstanding issue with VM File names/paths.

On the mac the vm is shipped with VM path names set to be macroman,  
and you can change that choice to latin-1 or utf-8.
People that changed the VM to latin-1, then discovered that there are  
characters like (option-f) ƒ which are valid
characters to put in folder names/file names that then will NOT  
convert from utf-8 to latin-1, the conversion fails and you're toast.

Ok, fine leave as macRoman, then add in that you are taking the  
macroman path name and pretending it's latin-1, this means of course  
what you see in the file browser isn't reality.
Fortunately the character code isn't tampered with it's just  
displayed wrong, so using the path does the right thing.

Likely one might consider using utf-8, and then ensuring display  
usages of pathnames do the right thing....


On 13-Jun-06, at 10:09 AM, Bert Freudenberg wrote:

> I just hacked a bit of Unicode support into the truetype fonts. See  
> attachment.
>
> - Bert -
>
========================================================================
===
John M. McIntosh <[hidden email]> 1-800-477-2659
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

We are "unicodized", sort-of ;-)

My cs is rather trivial, it works without a VM modification on the  
Mac, which provides Unicode keycodes in addition to the old MacRoman  
ones. Someone should try it on a Windows VM, I think it does the  
same. I hadn't seen your work, before, but my changes to TTFontReader  
and TTCFont are quite minimal. I changed the glyph cache to use a 64K  
WeakArray instead of 256 glyphs, which is way too inefficient I  
guess. Clipboard does not work, it's not Unicode-aware.

I have no idea how the Japanese folks do their input, I think they  
use specific code-tables, not Unicode. Their m17n stuff supports  
other encodings than Unicode, too. Also, I don't know which fonts  
they use, I have no idea how to use TTCFontSets and the TTFontReader  
subclasses etc.

The advantage of using Unicode is that it's nicely generic for many  
languages - I can type japanese, cyrillic, and greek text  
immediately, as the screenshot demonstrates. Maybe I should've posted  
the changeset header before:

"Change Set: unicodeHack-bf
Date: 13 June 2006
Author: Bert Freudenberg

Enables unicode input by VMs supporting the utf32 field in stroke  
events. Patches TTFontReader to read unicode glyph map. and retain  
all chars. Patches TTCFont to allow more than 256 characters.

You need to install Truetype fonts after loading this Changeset. This  
one is free with many glyphs: http://www.nongnu.org/freefont/"

- Bert -

Am 13.06.2006 um 20:47 schrieb danil osipchuk:

  Danil,

> I'm wondering for a quite a long time - are we 'unicodized' or not yet?
> Is the current state of m17n kind of final and I just don't get how to
> use it or something else should be done? What do other non-latin folks
> think about it? Does the fact that nobody but me complains mean that all
> is ok?
> A lot of questions and it is not a sarcasm.
>
> As for me, stock image and VMs definitely are not enabled for Russian.

  Please keep it in mind that "Unicodizing" and "enable language YYY"
are different issues (not only in Squeak, but in any systems that
try to deal with them.).  Current Squeak is Unicodized, but many
languages are wanting to be implemented.

  In Squeak, each language requires a few methods get implemented, and
basically the native speakers need to yell what fonts they want to use
for their language.

  (By definition of Unicode, there is no single font that can make
everybody happy for Unicode.  Not only in Squeak, but in any systems
that try to deal with them.)

  For example, what font do you want to use Russian.  For performance
reason, it would be nice that there is a set of bitmap fonts in
different size that matches the Accuny fonts, and also a TT font for
some other purposes.

-- Yoshiki

Yoshiki-san,

First let me express gratitude for all work you have done for m17n.  
Certainly it is a great work.

>>
>> As for me, stock image and VMs definitely are not enabled for Russian.
>
>   Please keep it in mind that "Unicodizing" and "enable language YYY"
> are different issues (not only in Squeak, but in any systems that
> try to deal with them.).  Current Squeak is Unicodized, but many
> languages are wanting to be implemented.

Yes, I understand this completely and in fact my complain originates from  
an attempt to implement such support for Russian. I succeeded with it but  
I'm not happy with things I had to do for it and how I've done them. It  
was painful all the way down. I think most of the problems are VM-related.

What I mean when asking if we are fully 'unicodized'? I expected that we  
will have a unified approach for text handling in Squeak - all of the text  
inside of squeak is, well, in 'squeak' format (as in various converters  
idioms). But it seems that we have a hybrid of an old charset-aware and  
new unicode implementation and one must be very careful when dealing with  
it. I'm not talking about access to external resource, it is obviously  
right thing to have multiple charset representations for them.
But how about this one:

CP1251ClipboardInterpreter>>fromSystemClipboard: aString

        | result converter |
       
        result := WriteStream on: (String new: aString size).
        converter := CP1251TextConverter new.
        aString do: [:each |
                result nextPut: (converter toSqueak: each macToSqueak) asCharacter.
        ].
       
        ^ result contents.


Note the #macToSqueak in above. One could argue that clipboard is an  
'external' resource which happen to be in mac encoding and therefore  
deserves special care, but why we actually should ever do things like this  
if we are fully unicode compliant? This trick have been copied from  
someone's else language-environment, I'd probably never come up with this  
on my own. Not all converters for some reason use this macToSqueak  
conversion - this is another interesting point to consider.

About Unix VMs. Please correct me if I'm wrong but is not everyone who did  
implementation for his language modified VM in one way or another? I've  
found that stock VM doesn't work for me (I've tried FreeBSD and Kubuntu) -  
when language is Russian, key chars don't find their way into the image  
(and I tried all reasonable combinations of command-line switches and  
environment variables).
When I'm changing x2sqKey (it is x2sqKeyPlain)  to x2sqKeyInput it starts  
working:

///in sqUnixX11.c
typedef int (*x2sqKey_t)(XKeyEvent *xevt);

static int x2sqKeyPlain(XKeyEvent *xevt);
static int x2sqKeyInput(XKeyEvent *xevt);

static x2sqKey_t x2sqKey= x2sqKeyInput;

I didn't manage to make copy/paste between Unix-VM and outer world work  
(just run out of steam when experimenting with it).

On windows VM I had to modify sqWin32Window.c. When user copies Russian  
text in squeak into clipboard text is being corrupted *unless* current  
keyboard layout is Russian. This happens because windows doesn't know  
anything about locale of the data being copied. So modification in  
sqWin32Window.c:

hLocale = GlobalAlloc(GMEM_MOVEABLE | GMEM_DDESHARE, sizeof(DWORD));
pLocale = (DWORD *) GlobalLock(hLocale);
*pLocale = GetUserDefaultLCID();
GlobalUnlock(hLocale);
SetClipboardData(CF_LOCALE, hLocale);

I'm not sure that everyone will be happy with it, but at least it works  
for Russian.


>   In Squeak, each language requires a few methods get implemented, and
> basically the native speakers need to yell what fonts they want to use
> for their language.
>
>   (By definition of Unicode, there is no single font that can make
> everybody happy for Unicode.  Not only in Squeak, but in any systems
> that try to deal with them.)

This is an interesting point, because I certainly used to think about  
unicode fonts as about something what suites all of the users at once.
I guess that this is where leadingChar comes from (because I still don't  
know what leadingChar is needed for and how to use it correctly)?

>   For example, what font do you want to use Russian.  For performance
> reason, it would be nice that there is a set of bitmap fonts in
> different size that matches the Accuny fonts, and also a TT font for
> some other purposes.

Most of the time I've spent was in the battle with fonts. I did see  
TTCFontReader and it was obviously used by you and others, so it must be  
useful.  It seems that Bert somehow managed to 'hack' it (btw, it was the  
word 'hack' in the heading of Bert's original message what triggered my  
response, because it is the most hacking activity I was ever been involved  
:)). But I didn't manage to do anything with it, so I had to almost  
completely dissect TTFontReader  and to reassemble it so I could read  
ttf-fonts (hence  
http://map.squeak.org/accountbyid/2bf29ca7-cb92-4c16-ae18-6b271117a660/package/2c1a81e1-4e86-40c8-90b5-824adc4263c5).  
TTC sub-hierarchies has gone as a result, so my changes again are not  
compatible with main distribution. I've seen for at least two times people  
asking on the list 'how do I read my Indian or whatever ttf font into the  
image' and nobody answered, so I just did it myself.

The net effect of all above is that I've managed to add support for  
Russian but I've ended with system which doesn't seem to be compatible  
with Japan (for instance) environment both on vm and image levels. It is  
certainly may indicate that I misunderstood concepts, but the fact is:  
adding support for another language is not just a matter of adding of a  
couple of the LanguageEnvoronment derived methods and classes into the  
system.

>
> -- Yoshiki
>
>


        Danil