Smalltalk › Pharo › Pharo Smalltalk Users

Sqlite and UTF8 Problem

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

4 messages Options

Roger Gilliar

Sqlite and UTF8 Problem

Hi,

I would like to access a sqlite database from pharo. So far that seems to work. The only problem is that my sqlite database contains UTF-8 data and that for example Ä is displayed as ¦Ñ.

Is there a workaround for this problem ?

BTW: I'm using Pharo 1.4 and installed sqlite support like this:

Gofer new

squeaksource: 'MetacelloRepository';

package: 'ConfigurationOfSQLite3';

load.

((Smalltalk at: #ConfigurationOfSQLite3) project version: '1.2') load: 'Tests'.

Regards

Roger

Igor Stasenko

Re: Sqlite and UTF8 Problem

You can use text converters to convert data back and forth between
different encodings.
See UTF8TextConverter

On 23 April 2012 20:14, Roger Gilliar <[hidden email]> wrote:

> Hi,
>
> I would like to access a sqlite database from pharo. So far that seems to
> work. The only problem is that my sqlite database contains UTF-8 data and
> that for example Ä is displayed as ¦Ñ.
>
> Is there a workaround for this problem ?
>
> BTW: I'm using Pharo 1.4 and installed sqlite support like this:
>
> Gofer new
> squeaksource: 'MetacelloRepository';
> package: 'ConfigurationOfSQLite3';
> load.
>
> ((Smalltalk at: #ConfigurationOfSQLite3) project version: '1.2') load:
> 'Tests'.
>
>
> Regards
> Roger

--
Best regards,
Igor Stasenko.

Roger Gilliar

Re: Sqlite and UTF8 Problem

In reply to this post by Roger Gilliar

See UTF8TextConverter
---------------------------------
Thanks for the tip. But shouldn't that be done by the system. The database uses UTF-8. Pharo uses UTF-8 so an Ö in the database should give me an Ö in the transcript window.

BTW: I tried the UTF8TextConverter and it just produces more garbage.

Regards
Roger

NorbertHartl

Re: Sqlite and UTF8 Problem

Roger,

Am 24.04.2012 um 06:41 schrieb Roger Gilliar:

> See UTF8TextConverter
> ---------------------------------
> Thanks for the tip. But shouldn't that be done by the system. The database uses UTF-8. Pharo uses UTF-8 so an Ö in the database should give me an Ö in the transcript window.
>
> BTW: I tried the UTF8TextConverter and it just produces more garbage.
>

pharo does not use UTF-8. It uses multibyte Characters. UTF-8 is a storage format. If you really get UTF-8 from sqlite than the TextConverter is the right thing to do. So let's take a lowercase ö. If you see something like Ã¶ where the ö should be than you can be sure it is utf-8 encoded. Then you can do

UTF8TextConverter new nextFromStream: 'Ã¶' asString readStream

to test it and get $ö. If you get something different then the encoding you get from sqlite is not UTF-8. You should ask yourself who put that string into sqlite. It might be a storage failure as well. Sqlite would probably just convert everything into UTF-8 regardless what you give to it.

Norbert