Smalltalk › Pharo › Pharo Smalltalk Developers

Percent-encoding problem

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

4 messages Options

bpi

Percent-encoding problem

I have a web service where I have to use a URL with percent-encoded (URL encoded) special characters – German Umlauts – but not using UTF8 but ISO-8859-1. I have been given the following table for the „correct“ encodings: http://www.degraeve.com/reference/urlencoding.php

I found out that I can use the following to get the encoding I need (%FC):
ZnPercentEncoder new
characterEncoder: (ZnCharacterEncoder newForEncoding: 'iso-8859-1');
encode: 'ü'.

However, when I try to use the encoded URL with ZnClient
ZnClient new
url: 'http://example.com/%FC';
get.

I get a ZnUTF8Encoder>>errorIllegalLeadingByte (Illegal leading byte for utf-8 encoding).

Any help is greatly appreciated.

Cheers,
Bernhard

Sven Van Caekenberghe-2

Re: Percent-encoding problem

Bernard,

I understand the issue, but this is not so easy to fix.

I must start by saying that encoding a (part of) a URL with anything but UTF-8 is wrong (how would the receiver know, unless you have a mutual external agreement). So I guess you must be using some old web service.

The solution is to make your own ZnUrl subclass and overwrite #encodePath:on:

Then you just instanciate your subclass from elementary parts (no encoding) and pass it to ZnClient>>#url: as an object, not a string.

Mind you, I haven't tried this yet.

Sven

> On 28 Jun 2016, at 20:26, Bernhard Pieber <[hidden email]> wrote:
>
> I have a web service where I have to use a URL with percent-encoded (URL encoded) special characters – German Umlauts – but not using UTF8 but ISO-8859-1. I have been given the following table for the „correct“ encodings: http://www.degraeve.com/reference/urlencoding.php
>
> I found out that I can use the following to get the encoding I need (%FC):
> ZnPercentEncoder new
> characterEncoder: (ZnCharacterEncoder newForEncoding: 'iso-8859-1');
> encode: 'ü'.
>
> However, when I try to use the encoded URL with ZnClient
> ZnClient new
> url: 'http://example.com/%FC';
> get.
>
> I get a ZnUTF8Encoder>>errorIllegalLeadingByte (Illegal leading byte for utf-8 encoding).
>
> Any help is greatly appreciated.
>
> Cheers,
> Bernhard
>
>
>
>

bpi

Re: Percent-encoding problem

Hi Sven,

Thanks for the answer. I needed to overwrite encodeQuery:on: and printQueryOn: for my use case. See attachment. I think being able to optionally configure the characterEncoder would be useful for ZnUrl in general. ZnUrl would then need to pass the characterEncoder to all calls to ZnResourceMetaUtils, not only to some like now. That would also be more consistent. Having ZnResourceMetaUtils use ZnUTF8Encoder by default seems a bit dangerous also. What do you think?

Thanks for Zinc!

Cheers,
Bernhard

> Am 28.06.2016 um 22:16 schrieb Sven Van Caekenberghe <[hidden email]>:
>
> Bernard,
>
> I understand the issue, but this is not so easy to fix.
>
> I must start by saying that encoding a (part of) a URL with anything but UTF-8 is wrong (how would the receiver know, unless you have a mutual external agreement). So I guess you must be using some old web service.
>
> The solution is to make your own ZnUrl subclass and overwrite #encodePath:on:
>
> Then you just instanciate your subclass from elementary parts (no encoding) and pass it to ZnClient>>#url: as an object, not a string.
>
> Mind you, I haven't tried this yet.
>
> Sven
>
>> On 28 Jun 2016, at 20:26, Bernhard Pieber <[hidden email]> wrote:
>>
>> I have a web service where I have to use a URL with percent-encoded (URL encoded) special characters – German Umlauts – but not using UTF8 but ISO-8859-1. I have been given the following table for the „correct“ encodings: http://www.degraeve.com/reference/urlencoding.php
>>
>> I found out that I can use the following to get the encoding I need (%FC):
>> ZnPercentEncoder new
>> characterEncoder: (ZnCharacterEncoder newForEncoding: 'iso-8859-1');
>> encode: 'ü'.
>>
>> However, when I try to use the encoded URL with ZnClient
>> ZnClient new
>> url: 'http://example.com/%FC';
>> get.
>>
>> I get a ZnUTF8Encoder>>errorIllegalLeadingByte (Illegal leading byte for utf-8 encoding).
>>
>> Any help is greatly appreciated.
>>
>> Cheers,
>> Bernhard
>>
>>
>>
>>
>
>

ZnEncodingUrl.st (1K) Download Attachment

Sven Van Caekenberghe-2

Re: Percent-encoding problem

Bernhard,

> On 30 Jun 2016, at 21:12, Bernhard Pieber <[hidden email]> wrote:
>
> Hi Sven,
>
> Thanks for the answer. I needed to overwrite encodeQuery:on: and printQueryOn: for my use case. See attachment. I think being able to optionally configure the characterEncoder would be useful for ZnUrl in general. ZnUrl would then need to pass the characterEncoder to all calls to ZnResourceMetaUtils, not only to some like now. That would also be more consistent. Having ZnResourceMetaUtils use ZnUTF8Encoder by default seems a bit dangerous also. What do you think?

Like I said, IMHO there is no need for this flexibility, UTF-8 is the standard/default, it makes no sense to use another one, doing so is an error.

https://en.wikipedia.org/wiki/Percent-encoding#Current_standard
https://tools.ietf.org/html/rfc3986 (which obsoletes 3 older ones)

Unless you can point me to some still relevant RFC that says otherwise, I am afraid things will stay as they are.

But it is good that you were able to deal with your requirement - your code might be useful for others.

Sven

> Thanks for Zinc!
>
> Cheers,
> Bernhard
>
> <ZnEncodingUrl.st>
>> Am 28.06.2016 um 22:16 schrieb Sven Van Caekenberghe <[hidden email]>:
>>
>> Bernard,
>>
>> I understand the issue, but this is not so easy to fix.
>>
>> I must start by saying that encoding a (part of) a URL with anything but UTF-8 is wrong (how would the receiver know, unless you have a mutual external agreement). So I guess you must be using some old web service.
>>
>> The solution is to make your own ZnUrl subclass and overwrite #encodePath:on:
>>
>> Then you just instanciate your subclass from elementary parts (no encoding) and pass it to ZnClient>>#url: as an object, not a string.
>>
>> Mind you, I haven't tried this yet.
>>
>> Sven
>>
>>> On 28 Jun 2016, at 20:26, Bernhard Pieber <[hidden email]> wrote:
>>>
>>> I have a web service where I have to use a URL with percent-encoded (URL encoded) special characters – German Umlauts – but not using UTF8 but ISO-8859-1. I have been given the following table for the „correct“ encodings: http://www.degraeve.com/reference/urlencoding.php
>>>
>>> I found out that I can use the following to get the encoding I need (%FC):
>>> ZnPercentEncoder new
>>> characterEncoder: (ZnCharacterEncoder newForEncoding: 'iso-8859-1');
>>> encode: 'ü'.
>>>
>>> However, when I try to use the encoded URL with ZnClient
>>> ZnClient new
>>> url: 'http://example.com/%FC';
>>> get.
>>>
>>> I get a ZnUTF8Encoder>>errorIllegalLeadingByte (Illegal leading byte for utf-8 encoding).
>>>
>>> Any help is greatly appreciated.
>>>
>>> Cheers,
>>> Bernhard
>>>
>>>
>>>
>>>
>>
>>
>