German umlauts: symbols converted to Slovene?

Hi,

in 5.6 i could do:

self observee hasPositionSelected
     ifFalse:
             [self redirectToView: #profilePositionNichtGewählt.
             ^self].

In case your mailer doesn't display correctly, the last word of the
symbol has a German "a" umlaut.

While I tend to avoid German umlauts in message names this one slipped
through and in 6.9beta2 results in:

ERROR: view named #'profilePositionNichtGewÀhlt' does not exist

in the browser.


Cheers,

Herbert                          mailto:[hidden email]

_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida

On Fri, 8 May 2009 08:34:42 +0200
Herbert König <[hidden email]> wrote:


From where through what to where did you move your code
from 5.6 to the new beta?

I'd wager you were using an image using ISO-8859-1 encoding
for internal string storage, and then somehow managed to save
the file using UTF-8 outside the image. The new image would
also be using ISO-8859-1 and interpret the bytes as shown.

tilde-A is usually a dead giveaway for this kind of problem,
as it's the ISO-8859-1 representation of the UTF-8 prefix byte
used for the relevant character range.

s.
_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida

Hi Herbert,

I don't think this is related to Slovenian specific support in Aida. In
any case, it is better to avoid any non ASCII text in your code. For
such text we have already the translation support in new Aida. See our demo:

        http://demo.aidaweb.si/demo.html?view=calendar

This should be shown automatically in your language if it is sl, en, es,
fr. If not, then you can switch session language:

        http://demo.aidaweb.si/demo.html?view=calendar&setLanguage=sl
        http://demo.aidaweb.si/demo.html?view=calendar&setLanguage=es
        http://demo.aidaweb.si/demo.html?view=calendar&setLanguage=fr
        http://demo.aidaweb.si/demo.html?view=calendar&setLanguage=en

Look at WebDemoApp>>viewCalendar and class methods, in newest beta to
see how this is imlpemented.

Just for a taste about what is coming in 6.0 :)

Janko

       


Herbert König pravi:
--
Janko Mivšek
Svetovalec za informatiko
Eranova d.o.o.
Ljubljana, Slovenija
www.eranova.si
tel:  01 514 22 55
faks: 01 514 22 56
gsm: 031 674 565
_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida

Hi Janko,

JM> I don't think this is related to Slovenian specific support in Aida. In
JM> any case, it is better to avoid any non ASCII text in your code.
while I agree with that I'd like to find out what happens (see my
reply to Stefan).

Herbert                            mailto:[hidden email]

_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida

Hi Stefan,

SS> From where through what to where did you move your code
SS> from 5.6 to the new beta?
nice sentence :-))

Well all in the same 8.2.1 image, removed 5.6 via monticello, cleaned
up junk, installed the beta mcz and fixed the single WebApplication
subclass I had in that image. Then applied my changes to Aida via a
copy of that image still having 5.6 using ^C and ^V.


SS> I'd wager you were using an image using ISO-8859-1 encoding
SS> for internal string storage, and then somehow managed to save
SS> the file using UTF-8 outside the image. The new image would
SS> also be using ISO-8859-1 and interpret the bytes as shown.

TextConverter default returns an UTF8TextConverter and the the message
with that name as well as the code containing the symbol with the
message name never left that image.

SS> tilde-A is usually a dead giveaway for this kind of problem,
SS> as it's the ISO-8859-1 representation of the UTF-8 prefix byte
SS> used for the relevant character range.

If I didn't get you wrong then I have to find out a compiledMethod's
literals encoding and the encoding of that message name?

Copying the message contents to a name without umlaut and changing the
symbol in the view method makes it work. Then taking a new name with a
different umlaut ...gewühlt :-) and changing the symbol to that new
name reproduces the problem.

Do you think this rules out the in image coding of strings?

While I agree, I should not use umlauts in message names I'd like to
understand what happens.



Cheers,

Herbert                            mailto:[hidden email]

_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida

On Fri, 8 May 2009 16:36:29 +0200
Herbert König <[hidden email]> wrote:

> If I didn't get you wrong then I have to find out a compiledMethod's
> literals encoding and the encoding of that message name?

the encoding should be the one your image is using internally
to store strings. so I'd expect it to be a "hidden" information
not easily discovered. Please prove me wrong :-)

> Copying the message contents to a name without umlaut and changing the
> symbol in the view method makes it work. Then taking a new name with a
> different umlaut ...gewühlt :-) and changing the symbol to that new
> name reproduces the problem.

at least it's a problem that most probably does not originate with
any environmental dependencies like the one I guessed.

> Do you think this rules out the in image coding of strings?

I've been out of touch with AIDA long enough to not know what
I'm talking about, but *assuming* that the view name appears
somewhere, e.g. in the URL, it would be a sensible precaution
to convert the string to something that is usable in URLs.
UTF-8 is a candidate here.

I guess that somewhere within AIDA a method converting a string
to UTF-8 is applied before emitting a string and it is missing
its counterpart conversion when AIDA is digesting the external
input.

If the view name actually becomes part of the URL, the conversion
from ISO to UTF-8 might also happen in the browser. I remember
such a thing happening with IE6 a while back. As I recall it,
IE6 accepted an ISO-8859-1 encoded URL but sent the request
to the server using UTF-8 encoding.

So if AIDA (or Swazoo?) has changed in the way that incoming requests
are parsed, it might be a starting point for investigating further.

s.
_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida

Stefan,


SS> the encoding should be the one your image is using internally
SS> to store strings. so I'd expect it to be a "hidden" information
SS> not easily discovered. Please prove me wrong :-)

I'd be happy if I could as I would know much more about Squeak then.

Anyway inspecting the methodDict of the class in both images, the "ä"
is coded as Character value 228 in both images. As well in the message
name (methodDict values..) as in the literals of the compiled view method.
But then I don't know how much the tools try to help instead of
presenting the raw data.

I'll follow your other clues, the browser is Firefox 3.0.10 in both
cases.

Thanks,

Herbert                            mailto:[hidden email]

_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida

On Fri, 8 May 2009 17:26:23 +0200
Herbert König <[hidden email]> wrote:

use a low-level tool to get a page from your app with a link
to your umlaut-view and inspect the file with some low-level
or encoding-aware editor.

If the umlaut-link is UTF-8 encoded, it's probably somewhere
in your squeak image. If not, then it has to be the browser
trying to be smart/standards compliant.

Firebug could also be helpful, but I'm a bit suspicious about its
helpful automatics when dealing with such issues.

s.
_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida

Hi Stefan, Herbert,

Stefan Schmiedl pravi:
...
> I guess that somewhere within AIDA a method converting a string
> to UTF-8 is applied before emitting a string and it is missing
> its counterpart conversion when AIDA is digesting the external
> input.

UTF8 conversion in Aida is done only where needed, that is only for text
when you adding it with addText: and similar methods.

In this particular case the conversion was not done, except for error
message, which was then wrongly encoded.

That is at least my initial diagnosis but le me try reproducing it on my
image...



--
Janko Mivšek
AIDA/Web
Smalltalk Web Application Server
http://www.aidaweb.si
_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida

Hi Herbert,

I tried to reproduce your error but failed even to add ä in the code
from my keyboard on both Squeak 3.9 and VW image.

Because I think this case is not so important and because of portability
issues we should rather avoid using any non ASCII text in the code, so
I'd rather close this case, but have it in mind, just in case for the
future.

Best regards
Janko

Herbert König pravi:
--
Janko Mivšek
Svetovalec za informatiko
Eranova d.o.o.
Ljubljana, Slovenija
www.eranova.si
tel:  01 514 22 55
faks: 01 514 22 56
gsm: 031 674 565
_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida

Hi Janko,


JM> I tried to reproduce your error but failed even to add ä in the code
JM> from my keyboard on both Squeak 3.9 and VW image.

JM> Because I think this case is not so important and because of portability
JM> issues we should rather avoid using any non ASCII text in the code, so
JM> I'd rather close this case, but have it in mind, just in case for the
JM> future.

you're right and I hope we didn't misunderstand each other here.

Me wanting to know did not mean you should investigate!

I'm very new to everything called Web programming so I will try
Stefan's tips (wget the url from Aida), search my image for what I
find there as I guess a lot of the HTML Aida produces is a literal
somewhere in the image.


Cheers,

Herbert                            mailto:[hidden email]

_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida

Hi Herbert,

Herbert König pravi:

Well, how can I answer you at all without investigating at least a bit;)

You know, I also don't know everything about Aida from the head. More
and more I rely on the comments I put in the code. And by investigating
from the scratch.

> I'm very new to everything called Web programming so I will try
> Stefan's tips (wget the url from Aida), search my image for what I
> find there as I guess a lot of the HTML Aida produces is a literal
> somewhere in the image.

I approached to the investigation simply by trying to rename some view
to include umlauts, then rename the view method also, then I wanted to
follow the matching algorithm in WebApplication class>>viewMethofForView: .

But I think the problem starts because Aida don't utf8 encode view names
in Urls at all. This is probably the reason the view name is not correct
when come to the #viewMethodForView: .

That was just few hints, which could help...

Janko


--
Janko Mivšek
AIDA/Web
Smalltalk Web Application Server
http://www.aidaweb.si
_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida

Hi Janko,

JM>> I tried to reproduce your error but failed even to add ä in the code
JM>> from my keyboard on both Squeak 3.9 and VW image.

the problem is more serious than I thought.

One way to reproduce the error is to go to the demo site's viewAjax.
In life input or delayed input try to enter € (Euro sign) or ²
(superscript 2 or 3) or whatever you can reach with the right Alt key
on your keyboard.

Seems like you can not input any non ASCII character into Aida's
inputs any more.




Cheers,

Herbert                            mailto:[hidden email]

_______________________________________________
Aida mailing list
[hidden email]
http://lists.aidaweb.si/mailman/listinfo/aida