WAUrl class>>#decodePercent:

> Thanks for entering a bug.
>
> In the meantime, I added a filter for the text to submit in the ajax request (using replace()), so the bug doesn`t hurt in my specific case any more.
>
> It is, btw, an interesting question what decoding of a uri that ends with one or multiple % should result in... I can't test right now, but i also thing decoding `abc%6' also fails because the methods expects two digits... (I am far away from an image at the moment...)
>
> Joachim
>
> Stephan Eggermont <[hidden email]> schrieb:
>
>> Nice find
>>
>> http://code.google.com/p/seaside/issues/detail?id=762
>>
>> Stephan
>> _______________________________________________
>> seaside mailing list
>> [hidden email]
>> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>>
> _______________________________________________
> seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

> I'm guessing the decoding should throw an error anyway since the string that does not adhere to the encoded format.
>
> On 22 Aug 2013, at 14:17, Joachim Tuchel <[hidden email]> wrote:
>
>> Thanks for entering a bug.
>>
>> In the meantime, I added a filter for the text to submit in the ajax request (using replace()), so the bug doesn`t hurt in my specific case any more.
>>
>> It is, btw, an interesting question what decoding of a uri that ends with one or multiple % should result in... I can't test right now, but i also thing decoding `abc%6' also fails because the methods expects two digits... (I am far away from an image at the moment...)
>>
>> Joachim
>>
>> Stephan Eggermont <[hidden email]> schrieb:
>>
>>> Nice find
>>>
>>> http://code.google.com/p/seaside/issues/detail?id=762
>>>
>>> Stephan
>>> _______________________________________________
>>> seaside mailing list
>>> [hidden email]
>>> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>>>
>> _______________________________________________
>> seaside mailing list
>> [hidden email]
>> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
> _______________________________________________
> seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>

> Okay, so the bug is almost obsolete ;-)
>
> Just change the exception to something better than the result of next
> being undefined, like "Illegal URI", and all is good ;-)
>
> In my case it was an ajax callback that got a parameter like
> '16=test%', because I had entered 'test%' into a text field that sends
> its contents in an ajax request. You may argue that characters like %,
> *, _ shouldn't be allowed for such cases anyways for all kinds of
> reasons (SQL injection, anyone?).
>
> What I should do, obviously, is to use encodeURI() to convert the
> input field's contents before I use them as parameter of an ajax call,
> right? Or does this lead to double encoding?
> I would expect $.ajax to encode its parameters correctly. Am I wrong?
>
> Joachim
>
>
> Am 22.08.13 15:01, schrieb Johan Brichau:
>> I'm guessing the decoding should throw an error anyway since the
>> string that does not adhere to the encoded format.
>>
>> On 22 Aug 2013, at 14:17, Joachim Tuchel <[hidden email]>
>> wrote:
>>
>>> Thanks for entering a bug.
>>>
>>> In the meantime, I added a filter for the text to submit in the ajax
>>> request (using replace()), so the bug doesn`t hurt in my specific
>>> case any more.
>>>
>>> It is, btw, an interesting question what decoding of a uri that ends
>>> with one or multiple % should result in... I can't test right now,
>>> but i also thing decoding `abc%6' also fails because the methods
>>> expects two digits... (I am far away from an image at the moment...)
>>>
>>> Joachim
>>>
>>> Stephan Eggermont <[hidden email]> schrieb:
>>>
>>>> Nice find
>>>>
>>>> http://code.google.com/p/seaside/issues/detail?id=762
>>>>
>>>> Stephan
>>>> _______________________________________________
>>>> seaside mailing list
>>>> [hidden email]
>>>> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>>>>
>>> _______________________________________________
>>> seaside mailing list
>>> [hidden email]
>>> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>> _______________________________________________
>> seaside mailing list
>> [hidden email]
>> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>>
>
>

> Sorry for answering to my own question.
>
> I added an encodeURI() before sending the ajax request and all is well now. The uris to be parsed can now contain as many percent signs as they want, the decodePercent: method doesn't fail any more.
> So this was my fault for sure.
>
> Still I don't think that decodePercent: should fail because it tries to read past the end of the incoming string/stream. It should throw an exception that states a uri seems to be ill-formed...
>
> Joachim
>
> Am 22.08.13 15:13, schrieb [hidden email]:
>> Okay, so the bug is almost obsolete ;-)
>>
>> Just change the exception to something better than the result of next being undefined, like "Illegal URI", and all is good ;-)
>>
>> In my case it was an ajax callback that got a parameter like '16=test%', because I had entered 'test%' into a text field that sends its contents in an ajax request. You may argue that characters like %, *, _ shouldn't be allowed for such cases anyways for all kinds of reasons (SQL injection, anyone?).
>>
>> What I should do, obviously, is to use encodeURI() to convert the input field's contents before I use them as parameter of an ajax call, right? Or does this lead to double encoding?
>> I would expect $.ajax to encode its parameters correctly. Am I wrong?
>>
>> Joachim
>>
>>
>> Am 22.08.13 15:01, schrieb Johan Brichau:
>>> I'm guessing the decoding should throw an error anyway since the string that does not adhere to the encoded format.
>>>
>>> On 22 Aug 2013, at 14:17, Joachim Tuchel <[hidden email]> wrote:
>>>
>>>> Thanks for entering a bug.
>>>>
>>>> In the meantime, I added a filter for the text to submit in the ajax request (using replace()), so the bug doesn`t hurt in my specific case any more.
>>>>
>>>> It is, btw, an interesting question what decoding of a uri that ends with one or multiple % should result in... I can't test right now, but i also thing decoding `abc%6' also fails because the methods expects two digits... (I am far away from an image at the moment...)
>>>>
>>>> Joachim
>>>>
>>>> Stephan Eggermont <[hidden email]> schrieb:
>>>>
>>>>> Nice find
>>>>>
>>>>> http://code.google.com/p/seaside/issues/detail?id=762
>>>>>
>>>>> Stephan
>>>>> _______________________________________________
>>>>> seaside mailing list
>>>>> [hidden email]
>>>>> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>>>>>
>>>> _______________________________________________
>>>> seaside mailing list
>>>> [hidden email]
>>>> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>>> _______________________________________________
>>> seaside mailing list
>>> [hidden email]
>>> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>>>
>>
>>
>
>
> --
> --
> ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel          mailto:[hidden email] Fliederweg 1                         http://www.objektfabrik.de
> D-71640 Ludwigsburg     http://joachimtuchel.wordpress.com
> Telefon: +49 7141 56 10 86 0         Fax: +49 7141 56 10 86 1
>
> _______________________________________________
> seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

> Hi,
>
> I just stumbled across a problem when processing an Ajax callback for a
> specialized autocompleter. I get a server side "Index out of range" if I
> enter a String ending with $% into the textInput (which I have to supress
> anyways, but that is another story).
>
> The problem is in WAUrl class>>#decodePercent: . the method doesn't check if
> the percent character is the last in the url string.
>
> You can recreate the problem by evaluating
>
> WAUrl decodePercent: 'abc%'
>
> Please note that in Pharo 2.0 I get another exception (receiver of
> asUpppercase is nil), but still this doesn't work. So I guess my code ( I am
> using VA Smalltalk and its older version of Seaside) would also fail on the
> current stable Seaside version.

>> encodeURI() would solve it as well, but that brings up a new problem,
>> because the String received at the server side then is encoded and has to be
>> decoded before it can be processed. AFAIK, VAST cannot do that yet - which
>> is another story...
> Decoding the URL is exactly what the code throwing the exception tries
> to do. Are you sure VAST can't do this? If this is true that should
> show up in other places as well like having % in a GET form.
>
> Can you to to /tests/functional/WAInputGetFunctionalTest and enter
> 'test%' (without '') into "Text Input" and see if that works?

> Hi Philippe,
>
>
>
> Am 29.08.13 17:18, schrieb Philippe Marschall:
>> Ok, so the decoding of this URL fails
>> /kontolino?_s=t0JuC1crW9NO2Bw5&_k=8qoE8Ph8YOUaO2a-&19=test% If this
>> indeed the raw URL and not already decoded then it is invalid and can
>> therefore not be decoded.
>
> Agreed. It is invalid. I know. And therefor, decodePercent: cannot
> handle it. So you can either forget my initial bug report or simply
> throw a better exception (stating the URL is invalid instead of Stream
> access fail).
>
> My Problem is that I am unsure whether I am expecting more from
> Seaside that it is intended to do.
>
> Because if I encodeURI() the input text before I send the Ajax
> request, the callback on the server will get '19=test%25' (the encoded
> url fragment) instead of 'test%' (a decoded url fragment). Do I really
> have to decode the String in an Ajax callback block by hand? Or is
> this failing on VAST only and there needs to be added something.
>
>>> encodeURI() would solve it as well, but that brings up a new problem,
>>> because the String received at the server side then is encoded and
>>> has to be
>>> decoded before it can be processed. AFAIK, VAST cannot do that yet -
>>> which
>>> is another story...
>> Decoding the URL is exactly what the code throwing the exception tries
>> to do. Are you sure VAST can't do this? If this is true that should
>> show up in other places as well like having % in a GET form.
>>
>> Can you to to /tests/functional/WAInputGetFunctionalTest and enter
>> 'test%' (without '') into "Text Input" and see if that works?
> Well, that works, of course. But I think it doesn't prove much. The
> text input on form submit travels as contents of the POST request and
> not as an URL fragment, like an Ajax request does. At least that is
> how I interpret my Firebug output. The text input is transported as
> 'text%' and is not encoded at all (Or Firebug is doing something on
> the fly to display POST arguments).
>
> So what all this seems to boil down to is more a question than a bug
> report:
>
> 1.) I think it is clear now that I should encodeURI() the contents of
> my text input field before handing it to the server as a parameter in
> an AJAX request.
>
> 2.) I haven't yet found a place in Seaside where the URL or its
> parameters of an AJAX request are treated differently than "normal"
> POST requests. So maybe I already found the answer to my question
> which then is "I need to decode the String by hand". But maybe you
> have a tip for me where I could look for such a place.
>
> 3.) How do others who wrote their jQuery plugins in plain JavaScript
> handle the case? Do they encodeURI() on the client and do they need to
> decode by hand on the server? I ask mainly to find out if I am seeing
> a VAST specific problem or a general one. And to find out if I am
> expecting too much from Seaside (or if I am probably breeding a new
> feature request for Seaside).
>
> 4.) Maybe the right question to the wrong list: how can I decodeURI:
> in VAST?
>
> Joachim
>
>
>

> Hi Philippe,
>
>
>
> Am 29.08.13 17:18, schrieb Philippe Marschall:
>
>> Ok, so the decoding of this URL fails
>> /kontolino?_s=t0JuC1crW9NO2Bw5&_k=8qoE8Ph8YOUaO2a-&19=test% If this indeed
>> the raw URL and not already decoded then it is invalid and can therefore not
>> be decoded.
>
>
> Agreed. It is invalid. I know. And therefor, decodePercent: cannot handle
> it. So you can either forget my initial bug report or simply throw a better
> exception (stating the URL is invalid instead of Stream access fail).
>
> My Problem is that I am unsure whether I am expecting more from Seaside that
> it is intended to do.

>>> encodeURI() would solve it as well, but that brings up a new problem,
>>> because the String received at the server side then is encoded and has to
>>> be
>>> decoded before it can be processed. AFAIK, VAST cannot do that yet -
>>> which
>>> is another story...
>>
>> Decoding the URL is exactly what the code throwing the exception tries
>> to do. Are you sure VAST can't do this? If this is true that should
>> show up in other places as well like having % in a GET form.
>>
>> Can you to to /tests/functional/WAInputGetFunctionalTest and enter
>> 'test%' (without '') into "Text Input" and see if that works?
>
> Well, that works, of course. But I think it doesn't prove much. The text
> input on form submit travels as contents of the POST request and not as an
> URL fragment, like an Ajax request does. At least that is how I interpret my
> Firebug output. The text input is transported as 'text%' and is not encoded
> at all (Or Firebug is doing something on the fly to display POST arguments).

> On Fri, Aug 30, 2013 at 10:43 AM, [hidden email]
> <[hidden email]> wrote:
>> Hi Philippe,
>>
>>
>>
>> Am 29.08.13 17:18, schrieb Philippe Marschall:
>>
>>> Ok, so the decoding of this URL fails
>>> /kontolino?_s=t0JuC1crW9NO2Bw5&_k=8qoE8Ph8YOUaO2a-&19=test% If this indeed
>>> the raw URL and not already decoded then it is invalid and can therefore not
>>> be decoded.
>>
>>
>> Agreed. It is invalid. I know. And therefor, decodePercent: cannot handle
>> it. So you can either forget my initial bug report or simply throw a better
>> exception (stating the URL is invalid instead of Stream access fail).
>>
>> My Problem is that I am unsure whether I am expecting more from Seaside that
>> it is intended to do.
>
> That's intentional. The contract is more or less:
> You send valid percent encoded URLs and Seaside does the decoding for
> you. Everything Seaside presents to you is already decoded. You can
> view the encoded content but have to go through some hoops.
>
>> Because if I encodeURI() the input text before I send the Ajax request, the
>> callback on the server will get '19=test%25' (the encoded url fragment)
>> instead of 'test%' (a decoded url fragment). Do I really have to decode the
>> String in an Ajax callback block by hand?
>
> No, Seaside should do this for you. In fact the code throwing the
> exception tries to do exactly this.
>
>> Or is this failing on VAST only
>> and there needs to be added something.
>
> Without further information it's hard to say.
>
>>>> encodeURI() would solve it as well, but that brings up a new problem,
>>>> because the String received at the server side then is encoded and has to
>>>> be
>>>> decoded before it can be processed. AFAIK, VAST cannot do that yet -
>>>> which
>>>> is another story...
>>>
>>> Decoding the URL is exactly what the code throwing the exception tries
>>> to do. Are you sure VAST can't do this? If this is true that should
>>> show up in other places as well like having % in a GET form.
>>>
>>> Can you to to /tests/functional/WAInputGetFunctionalTest and enter
>>> 'test%' (without '') into "Text Input" and see if that works?
>>
>> Well, that works, of course. But I think it doesn't prove much. The text
>> input on form submit travels as contents of the POST request and not as an
>> URL fragment, like an Ajax request does. At least that is how I interpret my
>> Firebug output. The text input is transported as 'text%' and is not encoded
>> at all (Or Firebug is doing something on the fly to display POST arguments).
>
> Yes it does. The form method is GET and not POST so the parameters get
> added to the request URL and not the method body. And Seaside/VASt
> does the decoding. This suggest that there is not a bug in VASt.
>
> From the stack trace you sent it looks as if you're doing a GET and not a POST.
>
>> So what all this seems to boil down to is more a question than a bug report:
>>
>> 1.) I think it is clear now that I should encodeURI() the contents of my
>> text input field before handing it to the server as a parameter in an AJAX
>> request.
>
> Yes
>
>> 2.) I haven't yet found a place in Seaside where the URL or its parameters
>> of an AJAX request are treated differently than "normal" POST requests. So
>> maybe I already found the answer to my question which then is "I need to
>> decode the String by hand". But maybe you have a tip for me where I could
>> look for such a place.
>
> Look for senders of #isXmlHttpRequest but you shouldn't have to do any decoding.
>
>> 3.) How do others who wrote their jQuery plugins in plain JavaScript handle
>> the case? Do they encodeURI() on the client and do they need to decode by
>> hand on the server? I ask mainly to find out if I am seeing a VAST specific
>> problem or a general one. And to find out if I am expecting too much from
>> Seaside (or if I am probably breeding a new feature request for Seaside).
>
> encodeURI() on the client is the way to go. Seaside expects valid HTTP requests.

> On Fri, Aug 30, 2013 at 12:45 PM, Philippe Marschall
> <[hidden email]> wrote:
>> On Fri, Aug 30, 2013 at 10:43 AM, [hidden email]
>> <[hidden email]> wrote:
>>> Hi Philippe,
>>>
>>>
>>>
>>> Am 29.08.13 17:18, schrieb Philippe Marschall:
>>>
>>>> Ok, so the decoding of this URL fails
>>>> /kontolino?_s=t0JuC1crW9NO2Bw5&_k=8qoE8Ph8YOUaO2a-&19=test% If this indeed
>>>> the raw URL and not already decoded then it is invalid and can therefore not
>>>> be decoded.
>>>
>>> Agreed. It is invalid. I know. And therefor, decodePercent: cannot handle
>>> it. So you can either forget my initial bug report or simply throw a better
>>> exception (stating the URL is invalid instead of Stream access fail).
>>>
>>> My Problem is that I am unsure whether I am expecting more from Seaside that
>>> it is intended to do.
>> That's intentional. The contract is more or less:
>> You send valid percent encoded URLs and Seaside does the decoding for
>> you. Everything Seaside presents to you is already decoded. You can
>> view the encoded content but have to go through some hoops.
>>
>>> Because if I encodeURI() the input text before I send the Ajax request, the
>>> callback on the server will get '19=test%25' (the encoded url fragment)
>>> instead of 'test%' (a decoded url fragment). Do I really have to decode the
>>> String in an Ajax callback block by hand?
>> No, Seaside should do this for you. In fact the code throwing the
>> exception tries to do exactly this.
>>
>>> Or is this failing on VAST only
>>> and there needs to be added something.
>> Without further information it's hard to say.
>>
>>>>> encodeURI() would solve it as well, but that brings up a new problem,
>>>>> because the String received at the server side then is encoded and has to
>>>>> be
>>>>> decoded before it can be processed. AFAIK, VAST cannot do that yet -
>>>>> which
>>>>> is another story...
>>>> Decoding the URL is exactly what the code throwing the exception tries
>>>> to do. Are you sure VAST can't do this? If this is true that should
>>>> show up in other places as well like having % in a GET form.
>>>>
>>>> Can you to to /tests/functional/WAInputGetFunctionalTest and enter
>>>> 'test%' (without '') into "Text Input" and see if that works?
>>> Well, that works, of course. But I think it doesn't prove much. The text
>>> input on form submit travels as contents of the POST request and not as an
>>> URL fragment, like an Ajax request does. At least that is how I interpret my
>>> Firebug output. The text input is transported as 'text%' and is not encoded
>>> at all (Or Firebug is doing something on the fly to display POST arguments).
>> Yes it does. The form method is GET and not POST so the parameters get
>> added to the request URL and not the method body. And Seaside/VASt
>> does the decoding. This suggest that there is not a bug in VASt.
>>
>>  From the stack trace you sent it looks as if you're doing a GET and not a POST.
>>
>>> So what all this seems to boil down to is more a question than a bug report:
>>>
>>> 1.) I think it is clear now that I should encodeURI() the contents of my
>>> text input field before handing it to the server as a parameter in an AJAX
>>> request.
>> Yes
>>
>>> 2.) I haven't yet found a place in Seaside where the URL or its parameters
>>> of an AJAX request are treated differently than "normal" POST requests. So
>>> maybe I already found the answer to my question which then is "I need to
>>> decode the String by hand". But maybe you have a tip for me where I could
>>> look for such a place.
>> Look for senders of #isXmlHttpRequest but you shouldn't have to do any decoding.
>>
>>> 3.) How do others who wrote their jQuery plugins in plain JavaScript handle
>>> the case? Do they encodeURI() on the client and do they need to decode by
>>> hand on the server? I ask mainly to find out if I am seeing a VAST specific
>>> problem or a general one. And to find out if I am expecting too much from
>>> Seaside (or if I am probably breeding a new feature request for Seaside).
>> encodeURI() on the client is the way to go. Seaside expects valid HTTP requests.
> or encodeURIComponent() depending on your use case.
>
> Cheers
> Philippe
> _______________________________________________
> seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>

> encodeURI and encodeURIComponent not only escape characters, but also
> convert special characters into UTF-8. In my case this were German umlauts.
> So I fell hostage to a side effect of encodeURI and the fact that VA ST
> doesn't yet support Unicode and somehow didn't understand this.
>
> So what I did was not wrong per se, I just ignored the whole UTF-8 thing. I
> should have started my search in that area, because it is not the first time
> AJax and its UTF-8 nature bit me.
> For Pharo/Squeak/Gemstone users, this UTF-8 stuff is a non-issue, and
> therefor readers of my posts had absolutely no chance to see the forest
> between all the trees.

> On Fri, Aug 30, 2013 at 9:27 PM, [hidden email]
> <[hidden email]> wrote:
>> Okay, this message costs me some courage now ;-)
>>
>> Philippe, of course Seaside is decoding. Otherwise I would never have gotten
>> an Exception from decodePercent: in the first place. So I am expecting the
>> right thing from Seaside and am getting it.
>>
>> The real problem in my case is that I am using ISO-8859-15 in my
>> application. So the result of getting the text field's contents using val()
>> is an ISO-8859-15 encoded String.
> AFAIK that should be UTF-16 for JavaScript.

>> encodeURI and encodeURIComponent not only escape characters, but also
>> convert special characters into UTF-8. In my case this were German umlauts.
>> So I fell hostage to a side effect of encodeURI and the fact that VA ST
>> doesn't yet support Unicode and somehow didn't understand this.
>>
>> So what I did was not wrong per se, I just ignored the whole UTF-8 thing. I
>> should have started my search in that area, because it is not the first time
>> AJax and its UTF-8 nature bit me.
>> For Pharo/Squeak/Gemstone users, this UTF-8 stuff is a non-issue, and
>> therefor readers of my posts had absolutely no chance to see the forest
>> between all the trees.
> Just for completeness' sake you could try to fake it, accept UTF-8 and
> translate it to ISO-8859-15

> Hi Philippe,
>
> Am 31.08.13 13:48, schrieb Philippe Marschall:
>
>> On Fri, Aug 30, 2013 at 9:27 PM, [hidden email]
>> <[hidden email]> wrote:
>>>
>>> Okay, this message costs me some courage now ;-)
>>>
>>> Philippe, of course Seaside is decoding. Otherwise I would never have
>>> gotten
>>> an Exception from decodePercent: in the first place. So I am expecting
>>> the
>>> right thing from Seaside and am getting it.
>>>
>>> The real problem in my case is that I am using ISO-8859-15 in my
>>> application. So the result of getting the text field's contents using
>>> val()
>>> is an ISO-8859-15 encoded String.
>>
>> AFAIK that should be UTF-16 for JavaScript.
>
> Well, it seems it is exactly what the html page's charset setting says. In
> my case it is ISO-8859-15.

>>> encodeURI and encodeURIComponent not only escape characters, but also
>>> convert special characters into UTF-8. In my case this were German
>>> umlauts.
>>> So I fell hostage to a side effect of encodeURI and the fact that VA ST
>>> doesn't yet support Unicode and somehow didn't understand this.
>>>
>>> So what I did was not wrong per se, I just ignored the whole UTF-8 thing.
>>> I
>>> should have started my search in that area, because it is not the first
>>> time
>>> AJax and its UTF-8 nature bit me.
>>> For Pharo/Squeak/Gemstone users, this UTF-8 stuff is a non-issue, and
>>> therefor readers of my posts had absolutely no chance to see the forest
>>> between all the trees.
>>
>> Just for completeness' sake you could try to fake it, accept UTF-8 and
>> translate it to ISO-8859-15
>
> Yes, I thought about this possibility, but then I decided against it not
> only for the reasons you mention, but also for performance reasons. In an
> autocompleter that reacts to every single keystroke and builds up a
> hierarchical reperesentation of business objects that are retrieved using
> Glorp and renders them in nested <ul> tags, every en/decoding step makes the
> thing slower, and this is a very central place in the app that makes part of
> its strengths.

>> but then the question is what you do with
>> everything of Unicode that doesn't fit into ISO-8859-15. Also we have
>> to option of running UTF-8 but not decoding it. This makes it possible
>> to run UTF-8 on non-Unicode-capable systems but you have to be very
>> careful especially with the backend.
>
> I invested quite some effort in making the whole application from database
> to web page UTF-free, because VAST doesn't support it very well, other than
> converting back and forth whenever a String enters or leaves VAST. Special
> fun is involved with DB field lengths etc. So as strange as it may sound,
> unicode is not necessarily your friend if there is at least one part of the
> chain that doesn't support it.