Win32 VM 3.10.6 problem with special characters

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Win32 VM 3.10.6 problem with special characters

Jens Mönig
dear squeakers,

I have run into two problems after switching from a 3.9.2 vm to the current 3.10.6 in a deployed application using the current stable 3.9 image. The main reason for switching was the new splash feature, which does help a lot on slower machines. However there seems to be an issue with special characters (German Umlaute, sharp s's and paragraph characters).

The first problem is, files containing such characters (Umlaute) are shown garbled in StandardFileMenu.

Interestingly these 'garbled' file(name)s load perfectly ok from StandardFileMenu as well as using Smalltalk getSystemAttribute: 3, or FileStream requestDropStream: n. But it just doesn't look nice in a deployed application, and I don't really want to restrict users' creativity in composing inventive fileNames ;-).

The second problem is, these special characters simply get lost in the clipboard.

This is somewhat worse, because my application builds on user defined strings as Dictionary keys. Whenever someone  pastes such a string from another place in or outside the image using the clipboard it first has to be manually corrected, otherwise things don't work out.

As I'm mostly just 'lurking' in here I'm not sure if this problem has been reported before or - indeed - if it's a problem for others as well or just a side effect of the unicode effort aparently going on. For the moment, I have switched back to the 'vintage' 3.9.2 vm (which works just fine). But I wouldn't want to be 'left bedind', so any help is appreciated.

Thanks
-Jens

Reply | Threaded
Open this post in threaded view
|

Re: Win32 VM 3.10.6 problem with special characters

Andreas.Raab
Hi Jens -

The problem you see comes from the VM now universally taking UTF-8 and
your (3.9) image being unaware of this situation. You will have to apply
the changes available here:

http://bugs.squeak.org/view.php?id=6523
http://bugs.squeak.org/view.php?id=6525
http://bugs.squeak.org/view.php?id=6526

Cheers,
  - Andreas

Jens Moenig wrote:

> dear squeakers,
>
> I have run into two problems after switching from a 3.9.2 vm to the current
> 3.10.6 in a deployed application using the current stable 3.9 image. The
> main reason for switching was the new splash feature, which does help a lot
> on slower machines. However there seems to be an issue with special
> characters (German Umlaute, sharp s's and paragraph characters).
>
> The first problem is, files containing such characters (Umlaute) are shown
> garbled in StandardFileMenu.
>
> Interestingly these 'garbled' file(name)s load perfectly ok from
> StandardFileMenu as well as using Smalltalk getSystemAttribute: 3, or
> FileStream requestDropStream: n. But it just doesn't look nice in a deployed
> application, and I don't really want to restrict users' creativity in
> composing inventive fileNames ;-).
>
> The second problem is, these special characters simply get lost in the
> clipboard.
>
> This is somewhat worse, because my application builds on user defined
> strings as Dictionary keys. Whenever someone  pastes such a string from
> another place in or outside the image using the clipboard it first has to be
> manually corrected, otherwise things don't work out.
>
> As I'm mostly just 'lurking' in here I'm not sure if this problem has been
> reported before or - indeed - if it's a problem for others as well or just a
> side effect of the unicode effort aparently going on. For the moment, I have
> switched back to the 'vintage' 3.9.2 vm (which works just fine). But I
> wouldn't want to be 'left bedind', so any help is appreciated.
>
> Thanks
> -Jens
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Win32 VM 3.10.6 problem with special characters

Jens Mönig
Hi Andreas,

Thanks a lot for this info. I'll try it, or I might just wait till the 3.10 image reaches some stability, provided your changesets will be incorporated there, and port my whole app to 3.10 for further development.

Jens


Andreas Raab wrote
Hi Jens -

The problem you see comes from the VM now universally taking UTF-8 and
your (3.9) image being unaware of this situation. You will have to apply
the changes available here:

http://bugs.squeak.org/view.php?id=6523
http://bugs.squeak.org/view.php?id=6525
http://bugs.squeak.org/view.php?id=6526

Cheers,
  - Andreas

Jens Moenig wrote:
> dear squeakers,
>
> I have run into two problems after switching from a 3.9.2 vm to the current
> 3.10.6 in a deployed application using the current stable 3.9 image. The
> main reason for switching was the new splash feature, which does help a lot
> on slower machines. However there seems to be an issue with special
> characters (German Umlaute, sharp s's and paragraph characters).
>
> The first problem is, files containing such characters (Umlaute) are shown
> garbled in StandardFileMenu.
>
> Interestingly these 'garbled' file(name)s load perfectly ok from
> StandardFileMenu as well as using Smalltalk getSystemAttribute: 3, or
> FileStream requestDropStream: n. But it just doesn't look nice in a deployed
> application, and I don't really want to restrict users' creativity in
> composing inventive fileNames ;-).
>
> The second problem is, these special characters simply get lost in the
> clipboard.
>
> This is somewhat worse, because my application builds on user defined
> strings as Dictionary keys. Whenever someone  pastes such a string from
> another place in or outside the image using the clipboard it first has to be
> manually corrected, otherwise things don't work out.
>
> As I'm mostly just 'lurking' in here I'm not sure if this problem has been
> reported before or - indeed - if it's a problem for others as well or just a
> side effect of the unicode effort aparently going on. For the moment, I have
> switched back to the 'vintage' 3.9.2 vm (which works just fine). But I
> wouldn't want to be 'left bedind', so any help is appreciated.
>
> Thanks
> -Jens
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Win32 VM 3.10.6 problem with special characters

Yoshiki Ohshima-2
In reply to this post by Andreas.Raab
  Hello,

  What is the best way to tell if the VM supports Unicode from the
image?  On Windows VM 3.10.6 on Windows XP, I get the following (and
some more over 10000 and under 0):

---------------
SmalltalkImage current getSystemAttribute: 1001 'Win32'
SmalltalkImage current getSystemAttribute: 1002 'NT'
SmalltalkImage current getSystemAttribute: 1003 'IX86'
SmalltalkImage current getSystemAttribute: 1004 'Squeak3.9 of 7 November 2006 [latest update: #7067]'
SmalltalkImage current getSystemAttribute: 1005 'Win32'
SmalltalkImage current getSystemAttribute: 1006 'Win32 built on Aug 30 2007 21:39:35 Compiler: 2.95.2 19991024 (release)'
---------------

  I would like to get a String that contains "3.10.6", but none of
above does.

  The same goes for the Unix VM.  I cannot seem to get a string with
"3.9-12" or such.  How about if we define system attribute 1007 for a
"real VM version" (equivalent of VM_VERSION macro for Windows)?

-- Yoshiki

Reply | Threaded
Open this post in threaded view
|

Re: Win32 VM 3.10.6 problem with special characters

Andreas.Raab
 >   The same goes for the Unix VM.  I cannot seem to get a string with
 > "3.9-12" or such.  How about if we define system attribute 1007 for a
 > "real VM version" (equivalent of VM_VERSION macro for Windows)?

Sounds fine to me.

Cheers,
   - Andreas

Yoshiki Ohshima wrote:

>   Hello,
>
>   What is the best way to tell if the VM supports Unicode from the
> image?  On Windows VM 3.10.6 on Windows XP, I get the following (and
> some more over 10000 and under 0):
>
> ---------------
> SmalltalkImage current getSystemAttribute: 1001 'Win32'
> SmalltalkImage current getSystemAttribute: 1002 'NT'
> SmalltalkImage current getSystemAttribute: 1003 'IX86'
> SmalltalkImage current getSystemAttribute: 1004 'Squeak3.9 of 7 November 2006 [latest update: #7067]'
> SmalltalkImage current getSystemAttribute: 1005 'Win32'
> SmalltalkImage current getSystemAttribute: 1006 'Win32 built on Aug 30 2007 21:39:35 Compiler: 2.95.2 19991024 (release)'
> ---------------
>
>   I would like to get a String that contains "3.10.6", but none of
> above does.
>
>   The same goes for the Unix VM.  I cannot seem to get a string with
> "3.9-12" or such.  How about if we define system attribute 1007 for a
> "real VM version" (equivalent of VM_VERSION macro for Windows)?
>
> -- Yoshiki
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Win32 VM 3.10.6 problem with special characters

Bert Freudenberg
In reply to this post by Yoshiki Ohshima-2

On Oct 16, 2007, at 23:01 , Yoshiki Ohshima wrote:

>   Hello,
>
>   What is the best way to tell if the VM supports Unicode from the
> image?  On Windows VM 3.10.6 on Windows XP, I get the following (and
> some more over 10000 and under 0):
>
> ---------------
> SmalltalkImage current getSystemAttribute: 1001 'Win32'
> SmalltalkImage current getSystemAttribute: 1002 'NT'
> SmalltalkImage current getSystemAttribute: 1003 'IX86'
> SmalltalkImage current getSystemAttribute: 1004 'Squeak3.9 of 7  
> November 2006 [latest update: #7067]'
> SmalltalkImage current getSystemAttribute: 1005 'Win32'
> SmalltalkImage current getSystemAttribute: 1006 'Win32 built on Aug  
> 30 2007 21:39:35 Compiler: 2.95.2 19991024 (release)'
> ---------------
>
>   I would like to get a String that contains "3.10.6", but none of
> above does.
>
>   The same goes for the Unix VM.  I cannot seem to get a string with
> "3.9-12" or such.  How about if we define system attribute 1007 for a
> "real VM version" (equivalent of VM_VERSION macro for Windows)?

Why not let the VM return the encoding string from a system attribute?

- Bert -



Reply | Threaded
Open this post in threaded view
|

Re: Win32 VM 3.10.6 problem with special characters

Yoshiki Ohshima-2
> >   The same goes for the Unix VM.  I cannot seem to get a string with
> > "3.9-12" or such.  How about if we define system attribute 1007 for a
> > "real VM version" (equivalent of VM_VERSION macro for Windows)?
>
> Why not let the VM return the encoding string from a system attribute?

  Oh yeah, I now remember that.  We can define four different
primitives (keyboard, clip board, file path name, and preferred file
contents).  Having the VM version is sometimes useful, though.

-- Yoshiki

Reply | Threaded
Open this post in threaded view
|

Re: Win32 VM 3.10.6 problem with special characters

Andreas.Raab
Yoshiki Ohshima wrote:
>>>   The same goes for the Unix VM.  I cannot seem to get a string with
>>> "3.9-12" or such.  How about if we define system attribute 1007 for a
>>> "real VM version" (equivalent of VM_VERSION macro for Windows)?
>> Why not let the VM return the encoding string from a system attribute?
>
>   Oh yeah, I now remember that.  We can define four different
> primitives (keyboard, clip board, file path name, and preferred file
> contents).  Having the VM version is sometimes useful, though.

To which the answers would be: UTF-8, UTF-8, UTF-8 and, err, UTF-8 ;-)
(Seriously, I fail to see the need for four different primitives given
that UTF-8 is so prevalent today and has easy conversions on every known
platform under the sun)

Cheers,
   - Andreas

Reply | Threaded
Open this post in threaded view
|

Re: Win32 VM 3.10.6 problem with special characters

Bert Freudenberg

On Oct 17, 2007, at 6:05 , Andreas Raab wrote:

> Yoshiki Ohshima wrote:
>>>>   The same goes for the Unix VM.  I cannot seem to get a string  
>>>> with
>>>> "3.9-12" or such.  How about if we define system attribute 1007  
>>>> for a
>>>> "real VM version" (equivalent of VM_VERSION macro for Windows)?
>>> Why not let the VM return the encoding string from a system  
>>> attribute?
>>   Oh yeah, I now remember that.  We can define four different
>> primitives (keyboard, clip board, file path name, and preferred file
>> contents).  Having the VM version is sometimes useful, though.
>
> To which the answers would be: UTF-8, UTF-8, UTF-8 and, err,  
> UTF-8 ;-) (Seriously, I fail to see the need for four different  
> primitives given that UTF-8 is so prevalent today and has easy  
> conversions on every known platform under the sun)

I don't need more than one for OLPC. The keyboard thing is solved by  
the utf32 charcode field in keyboard events. Clipboard has a new  
plugin anyway. File path encoding is the only one remaining, because  
"preferred file contents" is of questionable utility.

But, the Asian folks used to violently disagree, and adding multiple  
sys attribs is cheap. Yoshiki, can you explain again why we need  
everything separately?

- Bert -



Reply | Threaded
Open this post in threaded view
|

Re: Win32 VM 3.10.6 problem with special characters

Yoshiki Ohshima-2
  Bert,

> > To which the answers would be: UTF-8, UTF-8, UTF-8 and, err,  
> > UTF-8 ;-) (Seriously, I fail to see the need for four different  
> > primitives given that UTF-8 is so prevalent today and has easy  
> > conversions on every known platform under the sun)
>
> I don't need more than one for OLPC. The keyboard thing is solved by  
> the utf32 charcode field in keyboard events. Clipboard has a new  
> plugin anyway. File path encoding is the only one remaining, because  
> "preferred file contents" is of questionable utility.
>
> But, the Asian folks used to violently disagree, and adding multiple  
> sys attribs is cheap. Yoshiki, can you explain again why we need  
> everything separately?

  Well, I didn't know if the disagreement was "violent".

  For the newer VM, it is fine to just assume:

   (UTF-32, UTF-8, UTF-8)

so we even didn't need three attributes.  If we are to add a system
attribute, the older VMs returns nil for that and image figures out in
old fashion.

  We need the system encoding and preferred text encoding and the
latter is the job of LocalePlugin.  Apart from that, having another
system attribute that returns the VM_VERSION is good, too.

-- Yoshiki