Encoding CJK URL?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Encoding CJK URL?

Wowerat Dirk
My code to download some UTF8 text with japanese kanji from an url:

url := 'http://www.hatinosu.net/onsen/import.kml' asZnUrl.
url_string := url retrieveContents.
url_string class. ByteString

Now the japanese parts of url_string are unreadable.
What I need is WidString and not ByteString.

On linux I "solved" the problem with this workaround.

download := 'wget -O /tmp/file.tmp "http://www.hatinosu.net/onsen/import.kml"'.
OSProcess thisOSProcess waitForCommand: download.
fs := FileStream fileNamed: '/tmp/file.tmp'.
string := fs contents.
fs close.
string class. WideString

How do I do this without extern programms?

Dirk.
Reply | Threaded
Open this post in threaded view
|

Re: Encoding CJK URL?

Sven Van Caekenberghe-2
Hi Dirk,

Please try:

ZnDefaultCharacterEncoder
  value: ZnUTF8Encoder new
  during: [
    ZnClient new
      get: 'http://www.hatinosu.net/onsen/import.kml'; 
      yourself ].
               
ZnDefaultCharacterEncoder
  value: ZnUTF8Encoder new
  during: [ 'http://www.hatinosu.net/onsen/import.kml'asZnUrl retrieveContents ].

The mime-type/content-type of the response, application/vnd.google-earth.kml+xml, does not (directly/automatically) specify the encoding to use (as in charset=utf-8), which result in a ZnNullEncoder being used.

HTH,

Sven

PS: I also assume you are using a Unicode font with the necessary glyphs.

On 04 Sep 2014, at 22:28, Wowerat Dirk <[hidden email]> wrote:

> My code to download some UTF8 text with japanese kanji from an url:
>
> url := 'http://www.hatinosu.net/onsen/import.kml' asZnUrl.
> url_string := url retrieveContents.
> url_string class. ByteString
>
> Now the japanese parts of url_string are unreadable.
> What I need is WidString and not ByteString.
>
> On linux I "solved" the problem with this workaround.
>
> download := 'wget -O /tmp/file.tmp "http://www.hatinosu.net/onsen/import.kml"'.
> OSProcess thisOSProcess waitForCommand: download.
> fs := FileStream fileNamed: '/tmp/file.tmp'.
> string := fs contents.
> fs close.
> string class. WideString
>
> How do I do this without extern programms?
>
> Dirk.