Smalltalk › Frameworks & Tools › Magritte, Pier and Related Tools

encodings in pharo

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

7 messages Options

Dennis Schetinin

encodings in pharo

Got totallly confused with encodings. I try to process strings with russian characters entered through Pier using caracters entered in Pharo itself (using RegEx, for example).
But those strings are different. For example, code for russian symbol "№" by Pier string is (SmallInteger) 8470, but by Pharo it's 1069555990.
I tried different combinations of #convert(...) methods in String and similar in WAKomEncoding but without any success so far...

How can I convert from Pharo's encoding to Pier's one?

(...sorry, but I'm unable to follow all those long discussions about encodings in Pharo)

--
Dennis Schetinin

_______________________________________________
Magritte, Pier and Related Tools ...
https://www.iam.unibe.ch/mailman/listinfo/smallwiki

Lukas Renggli

Re: encodings in pharo

> How can I convert from Pharo's encoding to Pier's one?

Pier has no encoding on its own. Encoding depends on what Seaside
server adaptor you are using.

WAKom: If you use this server adapter no conversion is happening, that
means the strings are in UTF-8 and look messed up when being looked at
from within the image. Common operation like #size,
#includesSubstring:, #matches: and #, might return unexpected results.
As long as the strings are sent back verbatim that works well and is
very efficient.

WAKomEncoded: If you use this server adapter incoming strings are
converted from UTF-8 to Squeak, outgoing ones from Squeak to UTF-8.
These strings look ok in Squeak, as long as the font you are using
supports to display these characters. Conerting from and to UTF-8 is
slow, but your regular expressions should work assuming that your
regular expression engine works on wide strings.

Cheers,
Lukas

--
Lukas Renggli
http://www.lukas-renggli.ch
_______________________________________________
Magritte, Pier and Related Tools ...
https://www.iam.unibe.ch/mailman/listinfo/smallwiki

Dennis Schetinin

Re: encodings in pharo

2009/5/26 Lukas Renggli <[hidden email]>

> How can I convert from Pharo's encoding to Pier's one?

Pier has no encoding on its own. Encoding depends on what Seaside
server adaptor you are using.

WAKomEncoded:

I'm using this one. And the strings coming from Pier looks ok. And so the strings I enter manually (e.g. in Workspace) in Pharo do. But they (Pier's and Pharo's ones) are in different encodings actually and can not be compared. I think I have to convert Pharo's strings to Pier's encoding (I know the terms are not correct enough but I hope you'll get the idea)... For example, if I have Doctor instances entered using Pier, I want to find one with a particular name:

Doctor allInstances select: [:each | each title = 'some russian here' ] --- this never will work as actual titles are encoded differently than 'some russian here'... So I guess I have to use ('some russian here' convertToEncoding: 'the correct encoding') or something like that... But I don't know what exactly. I found decision in some earlier versions of Pharo/Pier combination... but it doesn't work anymore...

--
Dennis Schetinin

_______________________________________________
Magritte, Pier and Related Tools ...
https://www.iam.unibe.ch/mailman/listinfo/smallwiki

Lukas Renggli

Re: encodings in pharo

> I'm using this one. And the strings coming from Pier looks ok. And so the
> strings I enter manually (e.g. in Workspace) in Pharo do. But they (Pier's
> and Pharo's ones) are in different encodings actually and can not be
> compared. I think I have to convert Pharo's strings to Pier's encoding (I
> know the terms are not correct enough but I hope you'll get the idea)... For
> example, if I have Doctor instances entered using Pier, I want to find one
> with a particular name:

There were some recent changes related to encodings in Pharo, but I
don't know if this affects Seaside.

Can you reproduce the problem in Squeak 3.9.1 for example? Maybe you
should report this to the Pharo list instead, I don't know a solution.

Lukas

--
Lukas Renggli
http://www.lukas-renggli.ch
_______________________________________________
Magritte, Pier and Related Tools ...
https://www.iam.unibe.ch/mailman/listinfo/smallwiki

stéphane ducasse-2

Re: encodings in pharo

Yes please!!!
UTF-8 was changed by mike and we need to shake it.

Stef

On May 26, 2009, at 5:41 PM, Lukas Renggli wrote:

>> I'm using this one. And the strings coming from Pier looks ok. And
>> so the
>> strings I enter manually (e.g. in Workspace) in Pharo do. But they
>> (Pier's
>> and Pharo's ones) are in different encodings actually and can not be
>> compared. I think I have to convert Pharo's strings to Pier's
>> encoding (I
>> know the terms are not correct enough but I hope you'll get the
>> idea)... For
>> example, if I have Doctor instances entered using Pier, I want to
>> find one
>> with a particular name:
>
> There were some recent changes related to encodings in Pharo, but I
> don't know if this affects Seaside.
>
> Can you reproduce the problem in Squeak 3.9.1 for example? Maybe you
> should report this to the Pharo list instead, I don't know a solution.
>
> Lukas
>
> --
> Lukas Renggli
> http://www.lukas-renggli.ch
> _______________________________________________
> Magritte, Pier and Related Tools ...
> https://www.iam.unibe.ch/mailman/listinfo/smallwiki

_______________________________________________
Magritte, Pier and Related Tools ...
https://www.iam.unibe.ch/mailman/listinfo/smallwiki

Dennis Schetinin

Re: encodings in pharo

I'll try to arrange some time for the problem anon... Thank you for the replies...

2009/5/26 stéphane ducasse <[hidden email]>

Yes please!!!
UTF-8 was changed by mike and we need to shake it.

Stef

On May 26, 2009, at 5:41 PM, Lukas Renggli wrote:

I'm using this one. And the strings coming from Pier looks ok. And so the
strings I enter manually (e.g. in Workspace) in Pharo do. But they (Pier's
and Pharo's ones) are in different encodings actually and can not be
compared. I think I have to convert Pharo's strings to Pier's encoding (I
know the terms are not correct enough but I hope you'll get the idea)... For
example, if I have Doctor instances entered using Pier, I want to find one
with a particular name:

There were some recent changes related to encodings in Pharo, but I
don't know if this affects Seaside.

Can you reproduce the problem in Squeak 3.9.1 for example? Maybe you
should report this to the Pharo list instead, I don't know a solution.

Lukas

--
Lukas Renggli
http://www.lukas-renggli.ch

--
Dennis Schetinin

_______________________________________________
Magritte, Pier and Related Tools ...
https://www.iam.unibe.ch/mailman/listinfo/smallwiki

Philippe Marschall

Re: encodings in pharo

In reply to this post by Lukas Renggli

2009/5/26 Lukas Renggli <[hidden email]>:

>> I'm using this one. And the strings coming from Pier looks ok. And so the
>> strings I enter manually (e.g. in Workspace) in Pharo do. But they (Pier's
>> and Pharo's ones) are in different encodings actually and can not be
>> compared. I think I have to convert Pharo's strings to Pier's encoding (I
>> know the terms are not correct enough but I hope you'll get the idea)... For
>> example, if I have Doctor instances entered using Pier, I want to find one
>> with a particular name:
>
> There were some recent changes related to encodings in Pharo, but I
> don't know if this affects Seaside.

If I understand the changes correctly then it shouldn't.

Cheers
Philippe
_______________________________________________
Magritte, Pier and Related Tools ...
https://www.iam.unibe.ch/mailman/listinfo/smallwiki