[squeak-dev] Cuis update

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
51 messages Options
123
Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Re: Pasting unicode (was Re: Pharo:: Re: Cuis update)

Andreas.Raab
Michael Rueger wrote:
> Bert Freudenberg wrote:
>
>> One of the problems, yes. But note your example only works because the
>> Windows VM does paste unicode. On the Mac I get this:
>
> And the plugin then return non-precomposed UTF8.

http://developer.apple.com/qa/qa2001/qa1235.html

> There a couple of "interesting" things that need fixing for full unicode
> support, too many to quickly explain. Best would be to look at the
> differences in the Pharo packages.

Can you be a bit more specific?

Thanks,
   - Andreas

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Pasting unicode (was Re: Pharo:: Re: Cuis update)

Michael Rueger-6
On Fri, Apr 17, 2009 at 5:13 PM, Andreas Raab <[hidden email]> wrote:


>> There a couple of "interesting" things that need fixing for full unicode
>> support, too many to quickly explain. Best would be to look at the
>> differences in the Pharo packages.
>
> Can you be a bit more specific?

handling composing keyboards on Linux, command shortcuts on non latin
keyboards, assumptions about MacRoman all over the place...

probably more that I have conveniently forgotten to regain some sanity ;-)

michael

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Pasting unicode (was Re: Pharo:: Re: Cuis update)

Juan Vuletich-4
In reply to this post by Andreas.Raab
Andreas Raab wrote:
> Michael Rueger wrote:
>> Bert Freudenberg wrote:
>>
>>> One of the problems, yes. But note your example only works because
>>> the Windows VM does paste unicode. On the Mac I get this:
Hi Andreas,

Is there some Windows VM parameter to get 8 bit clipboard data? For non
unicode-aware images like Cuis, or older Squeak releases, it would be
great to get ISO 8859-1 or MacRoman codes for chars above 128. For
example, if I enter รก (a with acute accent) via keyboard, I get char
code 135 both on Win and Mac. If I paste the same from clipboard on Mac,
I get the same.

But on Windows I get 2 bytes, 195 and 161, and these non-unicode images
get confused.

I guess I could use an older VM, but I'd like to use the latest if possible.

Thanks,
Juan Vuletich

Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Re: Pasting unicode (was Re: Pharo:: Re: Cuis update)

Andreas.Raab
Juan Vuletich wrote:
> Is there some Windows VM parameter to get 8 bit clipboard data?

Starting with the 3.10 series the Windows VMs use UTF-8 exclusively and
consistently in its entire interface. All strings passed into or out of
the Windows VM use UTF-8. This is true for clipboard as well as files,
directories  etc.

> But on Windows I get 2 bytes, 195 and 161, and these non-unicode images
> get confused.

If you want to use non-ascii characters you will need an encoder and a
decoder for the range of characters your image supports. You can start
from here:

   http://bugs.squeak.org/view.php?id=7314

This has a *very* fast conversion to and from UTF-8 encoded
representations for strings and the only thing you need to change is to
restrict the incoming unicode value in utf8ToSqueak to whatever range of
characters you support.

Cheers,
   - Andreas

Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Re: Pasting unicode (was Re: Pharo:: Re: Cuis update)

Andreas.Raab
In reply to this post by Bert Freudenberg
Bert Freudenberg wrote:
> Below that the paste from ru.wikipedia. These are actual question marks
> (ASCII 63), the Mac VM does not paste Unicode even if setting
> SqueakEncodingType to UTF-8 (only the extended clipboard plugin does this).

Fortunately, at least that part is very easy to fix. I asked our CEO if
we could release our clipboard support code back to the community and he
agreed, so find attched an sqMacUIClipBoard.c version which does support
Unicode copy/paste properly via UTF-8 (this should work as a drop-in
replacement to the existing sqMacUIClipBoard.c). Brought to you by the
friendly folks at Qwaq.

Cheers,
   - Andreas



LICENSE (1K) Download Attachment
sqMacUIClipBoard.c (8K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Re: [Vm-dev] Re: Pasting unicode (was Re: Pharo:: Re: Cuis update)

Juan Vuletich-4
Andreas Raab wrote:

>  
>
> Bert Freudenberg wrote:
>> Below that the paste from ru.wikipedia. These are actual question
>> marks (ASCII 63), the Mac VM does not paste Unicode even if setting
>> SqueakEncodingType to UTF-8 (only the extended clipboard plugin does
>> this).
>
> Fortunately, at least that part is very easy to fix. I asked our CEO
> if we could release our clipboard support code back to the community
> and he agreed, so find attched an sqMacUIClipBoard.c version which
> does support Unicode copy/paste properly via UTF-8 (this should work
> as a drop-in replacement to the existing sqMacUIClipBoard.c). Brought
> to you by the friendly folks at Qwaq.
>
> Cheers,
>   - Andreas

Thanks Andreas! This will make a lot of people happy!

BTW, the image needs to deal with different VMs using various encoding
for clipboard data. Currently, a check is made, to assume UTF-8 if on
Windows, and MacRoman elsewhere. It seems Mac will support UTF-8 soon.

I think we need a new primitive, perhaps #clipboardEncoding, or
#vmTextEncoding, that could answer UTF-8 / MacRoman / ASCII and may be
others. That would ease supporting various VMs with the same image.

WRT keyboard events, it seems not to be necessary. VMs store Unicode
keystrokes in different slots in the event buffer, than the one used for
MacRoman. So, the image can tell, right?

What do you think?

Cheers,
Juan Vuletich

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: Pasting unicode (was Re: Pharo:: Re: Cuis update)

Juan Vuletich-4
In reply to this post by Andreas.Raab
Andreas Raab wrote:

> Juan Vuletich wrote:
>> Is there some Windows VM parameter to get 8 bit clipboard data?
>
> Starting with the 3.10 series the Windows VMs use UTF-8 exclusively
> and consistently in its entire interface. All strings passed into or
> out of the Windows VM use UTF-8. This is true for clipboard as well as
> files, directories  etc.
>
>> But on Windows I get 2 bytes, 195 and 161, and these non-unicode
>> images get confused.
>
> If you want to use non-ascii characters you will need an encoder and a
> decoder for the range of characters your image supports. You can start
> from here:
>
>   http://bugs.squeak.org/view.php?id=7314
>
> This has a *very* fast conversion to and from UTF-8 encoded
> representations for strings and the only thing you need to change is
> to restrict the incoming unicode value in utf8ToSqueak to whatever
> range of characters you support.
>
> Cheers,
>   - Andreas
Thank you Andreas! This is exactly what I needed.

I have ready the changeset for supporting Latin-1 (ISO 8859-1). I'll
publish a new release this weekend.

Cheers,
Juan Vuletich

Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Re: [Vm-dev] Re: Pasting unicode (was Re: Pharo:: Re: Cuis update)

Andreas.Raab
In reply to this post by Juan Vuletich-4
Juan Vuletich wrote:

> BTW, the image needs to deal with different VMs using various encoding
> for clipboard data. Currently, a check is made, to assume UTF-8 if on
> Windows, and MacRoman elsewhere. It seems Mac will support UTF-8 soon.
>
> I think we need a new primitive, perhaps #clipboardEncoding, or
> #vmTextEncoding, that could answer UTF-8 / MacRoman / ASCII and may be
> others. That would ease supporting various VMs with the same image.
>
> WRT keyboard events, it seems not to be necessary. VMs store Unicode
> keystrokes in different slots in the event buffer, than the one used for
> MacRoman. So, the image can tell, right?
>
> What do you think?

Although you are technically correct, I think the days of Latin1 only
support are simply over and I would prefer to move forward and leave
them completely behind us. The best way to achieve this is when all VMs
use UTF-8 everywhere and all VMs produce Unicode input characters. The
changes to support this aren't difficult (just a bit tedious) and the
image side is trivial for those images that don't need (or want) to
support Unicode. I really don't see any value in fiddling with different
encodings since UTF-8 is both canonical and well supported.

Cheers,
   - Andreas

Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Re: [Vm-dev] Re: Pasting unicode (was Re: Pharo:: Re: Cuis update)

johnmci
In reply to this post by Andreas.Raab
Although this allows you to deal with UTF8 characters I was hoping  
that we could migrate to the extended clipboard logic that was  
proposed by

http://lists.squeakfoundation.org/pipermail/vm-dev/2006-April/000584.html

We successfully used that in Sophie to pull utf-8, utf-16, macroman,  
and various forms of picture data, plus Finder aliases when you do cmd-
copy on a file in the Finder,
at best this is a patch towards that goal, and if it requires a new VM  
and image changes, why not provide all the existing functionality so  
people can copy/paste images?

Also some applications just didn't provide kUTTypeUTF8PlainText so the  
logic would review the available types on the clipboard and choose the  
one with the best options, although Microsoft office was a hassle  
because it would place both text in 50 formats AND a TIFF of the PAGE  
on the clipboard so we had to somewhat guess does the
client want a TIFF from Word, or text from Word, because if you copied  
an image in Word, it would also place a text description of the  
graphic object on the clipboard.


On 17-Apr-09, at 1:35 PM, Andreas Raab wrote:

> Bert Freudenberg wrote:
>> Below that the paste from ru.wikipedia. These are actual question  
>> marks (ASCII 63), the Mac VM does not paste Unicode even if setting  
>> SqueakEncodingType to UTF-8 (only the extended clipboard plugin  
>> does this).
>
> Fortunately, at least that part is very easy to fix. I asked our CEO  
> if we could release our clipboard support code back to the community  
> and he agreed, so find attched an sqMacUIClipBoard.c version which  
> does support Unicode copy/paste properly via UTF-8 (this should work  
> as a drop-in replacement to the existing sqMacUIClipBoard.c).  
> Brought to you by the friendly folks at Qwaq.
>
> Cheers,
>  - Andreas
> <LICENSE.txt><sqMacUIClipBoard.c>

--
=
=
=
========================================================================
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
=
=
=
========================================================================




Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Re: [Vm-dev] Re: Pasting unicode (was Re: Pharo:: Re: Cuis update)

Juan Vuletich-4
In reply to this post by Andreas.Raab
Andreas Raab wrote:

>
> Juan Vuletich wrote:
>> BTW, the image needs to deal with different VMs using various
>> encoding for clipboard data. Currently, a check is made, to assume
>> UTF-8 if on Windows, and MacRoman elsewhere. It seems Mac will
>> support UTF-8 soon.
>>
>> I think we need a new primitive, perhaps #clipboardEncoding, or
>> #vmTextEncoding, that could answer UTF-8 / MacRoman / ASCII and may
>> be others. That would ease supporting various VMs with the same image.
>>
>> WRT keyboard events, it seems not to be necessary. VMs store Unicode
>> keystrokes in different slots in the event buffer, than the one used
>> for MacRoman. So, the image can tell, right?
>>
>> What do you think?
>
> Although you are technically correct, I think the days of Latin1 only
> support are simply over and I would prefer to move forward and leave
> them completely behind us. The best way to achieve this is when all
> VMs use UTF-8 everywhere and all VMs produce Unicode input characters.
> The changes to support this aren't difficult (just a bit tedious) and
> the image side is trivial for those images that don't need (or want)
> to support Unicode. I really don't see any value in fiddling with
> different encodings since UTF-8 is both canonical and well supported.
>
> Cheers,
>   - Andreas

You are right. The only reason is that it might take time until VMs are
available for all platforms. My proposal allows for a smoother
transition, and it would also allow people to continue using older VMs
without much trouble. As you know, the image side changes to allow this
are easy too.

Cheers,
Juan Vuletich

Reply | Threaded
Open this post in threaded view
|

[squeak-dev] Re: [Vm-dev] Re: Pasting unicode (was Re: Pharo:: Re: Cuis update)

johnmci
In reply to this post by Andreas.Raab
Ok, as far as I can tell this requires a 10.4 SDK to build and link.  
In the past I built the ppc part using the 10.3.x SDK. So I'll try  
with the 10.4.x ppc SDK, and see what happens.

If there are any os-x 10.2 or 10.3 users out there they may want to  
try copy/paste with the pending 4.1.0b1 macintosh carbon vm


On 17-Apr-09, at 1:35 PM, Andreas Raab wrote:

> Bert Freudenberg wrote:
>> Below that the paste from ru.wikipedia. These are actual question  
>> marks (ASCII 63), the Mac VM does not paste Unicode even if setting  
>> SqueakEncodingType to UTF-8 (only the extended clipboard plugin  
>> does this).
>
> Fortunately, at least that part is very easy to fix. I asked our CEO  
> if we could release our clipboard support code back to the community  
> and he agreed, so find attched an sqMacUIClipBoard.c version which  
> does support Unicode copy/paste properly via UTF-8 (this should work  
> as a drop-in replacement to the existing sqMacUIClipBoard.c).  
> Brought to you by the friendly folks at Qwaq.
>
> Cheers,
>  - Andreas
> <LICENSE.txt><sqMacUIClipBoard.c>

--
=
=
=
========================================================================
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
=
=
=
========================================================================




123