Smalltalk › Pharo › Pharo Smalltalk Developers

Zinc redirect bug?

‹ Previous Topic Next Topic ›
 
classic Classic list List threaded Threaded
♦
♦
 4 messages Options
Aliaksei Syrel
Reply | Threaded
Open this post in threaded view
♦
♦
|

Zinc redirect bug?

Aliaksei Syrel
Hello,

I am trying to download a file from bintray with the help of ZnClient.
I'm doing the following:

ZnClient new
   systemPolicy;
   get;
   yourself

which results in Error 403.

At the same time wget works as expected:


Trying to open this url in a browser works too.

I tried to find out what may cause the error and it looks like it is something related to redirects because request and response headers look good...

Thanks,
Alex
Tim Mackinnon
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Zinc redirect bug?

Tim Mackinnon
I’m not sure if this is related - but the Pharo6 image is missing a ZnConstant initialise, and I recall redirect was one of the error codes missing. Possibly re-initialising ZnConstants might fix your problem?

Tim

On 14 Jul 2017, at 12:02, Aliaksei Syrel <[hidden email]> wrote:

Hello,

I am trying to download a file from bintray with the help of ZnClient.
I'm doing the following:

ZnClient new
   systemPolicy;
   get;
   yourself

which results in Error 403.

At the same time wget works as expected:


Trying to open this url in a browser works too.

I tried to find out what may cause the error and it looks like it is something related to redirects because request and response headers look good...

Thanks,
Alex
Sven Van Caekenberghe-2
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Zinc redirect bug?

Sven Van Caekenberghe-2
In reply to this post by Aliaksei Syrel
Hi Aliaksei,

This is an interesting problem, thanks for reporting it. I can reproduce the problem.

If you inspect 'ZnLogEvent announcer' before doing the request, you can see that ZnClient does seem to do the same as 'curl -vOL https://dl.bintray.com/syrel/Moz2D/osx/development/i386/libMoz2D.dylib'. It follows one redirect, to the correct host and URL.

One difference that I see that might lead to the 403 Forbidden is that Zn has a slight difference in the redirect URL being followed.

Curl seems to follow

/86/866a31722e3a9ea19c2bc4eeed2e1e69f7339ddd460500f2a33b4c9d041ef9b4?__gda__=exp=1500058371~hmac=820d22695ef0a821abc7e6eb7be6749ad43e8e0f74ba4b103476aa1b3663b381&response-content-disposition=attachment%3Bfilename%3D%22libMoz2D.dylib%22&response-content-type=application%2Fx-www-form-urlencoded&requestInfo=U2FsdGVkX19XiffCdVJdoADmObVHKjU4Wla5_WKFKHoNAvXQw8ponQIqWBsRUfNNDaULVXlXwB22y7xy0ls3DGkl700_9RxT-HtU92OHYdgKzSq9a6A7p5NIoMi4avCjPk_UCnVbt5U4_nxvkmKtuA

While Zn turns this (parses the URL and then renders it again) into

/86/866a31722e3a9ea19c2bc4eeed2e1e69f7339ddd460500f2a33b4c9d041ef9b4?response-content-type=application/x-www-form-urlencoded&response-content-disposition=attachment;filename%3D%22libMoz2D.dylib%22&requestInfo=U2FsdGVkX19XiffCdVJdoADmObVHKjU4Wla5_WKFKHoNAvXQw8ponQIqWBsRUfNNDaULVXlXwB22y7xy0ls3DGkl700_9RxT-HtU92OHYdgKzSq9a6A7p5NIoMi4avCjPk_UCnVbt5U4_nxvkmKtuA&__gda__=exp%3D1500058371~hmac%3D820d22695ef0a821abc7e6eb7be6749ad43e8e0f74ba4b103476aa1b3663b381

The difference is in the encoding of the __gda__ query parameter value. That contains = characters. These get escaped by Zn (the %3D) but are left untouched by curl. Maybe the server does not like that. I have to look deeper into the spec to figure out if this is a bug or not. Since query parameters look like key=value it seems odd to allow = in the value part.

Another possible issue might be the order of query parameters that got changed.

I'll come back to you if I find out more.

Sven

> On 14 Jul 2017, at 13:02, Aliaksei Syrel <[hidden email]> wrote:
>
> Hello,
>
> I am trying to download a file from bintray with the help of ZnClient.
> I'm doing the following:
>
> ZnClient new
>    systemPolicy;
>    url: 'https://dl.bintray.com/syrel/Moz2D/osx/development/i386/libMoz2D.dylib';
>    get;
>    yourself
>
> which results in Error 403.
>
> At the same time wget works as expected:
>
> wget https://dl.bintray.com/syrel/Moz2D/osx/development/i386/libMoz2D.dylib
>
> Trying to open this url in a browser works too.
>
> I tried to find out what may cause the error and it looks like it is something related to redirects because request and response headers look good...
>
> Thanks,
> Alex


Sven Van Caekenberghe-2
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Zinc redirect bug?

Sven Van Caekenberghe-2

> On 14 Jul 2017, at 21:12, Sven Van Caekenberghe <[hidden email]> wrote:
>
> Hi Aliaksei,
>
> This is an interesting problem, thanks for reporting it. I can reproduce the problem.
>
> If you inspect 'ZnLogEvent announcer' before doing the request, you can see that ZnClient does seem to do the same as 'curl -vOL https://dl.bintray.com/syrel/Moz2D/osx/development/i386/libMoz2D.dylib'. It follows one redirect, to the correct host and URL.
>
> One difference that I see that might lead to the 403 Forbidden is that Zn has a slight difference in the redirect URL being followed.
>
> Curl seems to follow
>
> /86/866a31722e3a9ea19c2bc4eeed2e1e69f7339ddd460500f2a33b4c9d041ef9b4?__gda__=exp=1500058371~hmac=820d22695ef0a821abc7e6eb7be6749ad43e8e0f74ba4b103476aa1b3663b381&response-content-disposition=attachment%3Bfilename%3D%22libMoz2D.dylib%22&response-content-type=application%2Fx-www-form-urlencoded&requestInfo=U2FsdGVkX19XiffCdVJdoADmObVHKjU4Wla5_WKFKHoNAvXQw8ponQIqWBsRUfNNDaULVXlXwB22y7xy0ls3DGkl700_9RxT-HtU92OHYdgKzSq9a6A7p5NIoMi4avCjPk_UCnVbt5U4_nxvkmKtuA
>
> While Zn turns this (parses the URL and then renders it again) into
>
> /86/866a31722e3a9ea19c2bc4eeed2e1e69f7339ddd460500f2a33b4c9d041ef9b4?response-content-type=application/x-www-form-urlencoded&response-content-disposition=attachment;filename%3D%22libMoz2D.dylib%22&requestInfo=U2FsdGVkX19XiffCdVJdoADmObVHKjU4Wla5_WKFKHoNAvXQw8ponQIqWBsRUfNNDaULVXlXwB22y7xy0ls3DGkl700_9RxT-HtU92OHYdgKzSq9a6A7p5NIoMi4avCjPk_UCnVbt5U4_nxvkmKtuA&__gda__=exp%3D1500058371~hmac%3D820d22695ef0a821abc7e6eb7be6749ad43e8e0f74ba4b103476aa1b3663b381
>
> The difference is in the encoding of the __gda__ query parameter value. That contains = characters. These get escaped by Zn (the %3D) but are left untouched by curl. Maybe the server does not like that. I have to look deeper into the spec to figure out if this is a bug or not. Since query parameters look like key=value it seems odd to allow = in the value part.
>
> Another possible issue might be the order of query parameters that got changed.
>
> I'll come back to you if I find out more.
>
> Sven

By (temporarily) changing ZnResourceMetaUtils>>#writeQueryFields:with:on: to use #querySafeSet instead of #queryKeyValueSafeSet (so that $= no longer gets encoded), the request passes, i.e. the redirect is followed and returns 200 OK.

The next problem is the interpretation of the result ;-)

The returned resource is specified to have a content-type of application/x-www-form-urlencoded and Zn interprets it as such. However, that does not seem to be the case. Rather it seems to be binary data. Now, returning application/x-www-form-urlencoded data as response to a request is very weird, I have never seen that.

It seems as if browsers and curl just ignore the actual type and just dump the data in a file (content-disposition: attachment;filename="libMoz2D.dylib"). Where is such a behaviour described ?

Aliaksei, could you maybe upload something like an small JPG or PNG image using the same mechanism ? Then it is easier to test the correctness of the result.

Could it be that you uploaded libMoz2D.dylib incorrectly ? Because application/x-www-form-urlencoded is normally only used in the context of HTTP forms that POSTed.

>> On 14 Jul 2017, at 13:02, Aliaksei Syrel <[hidden email]> wrote:
>>
>> Hello,
>>
>> I am trying to download a file from bintray with the help of ZnClient.
>> I'm doing the following:
>>
>> ZnClient new
>>   systemPolicy;
>>   url: 'https://dl.bintray.com/syrel/Moz2D/osx/development/i386/libMoz2D.dylib';
>>   get;
>>   yourself
>>
>> which results in Error 403.
>>
>> At the same time wget works as expected:
>>
>> wget https://dl.bintray.com/syrel/Moz2D/osx/development/i386/libMoz2D.dylib
>>
>> Trying to open this url in a browser works too.
>>
>> I tried to find out what may cause the error and it looks like it is something related to redirects because request and response headers look good...
>>
>> Thanks,
>> Alex
>


Free forum by Nabble Edit this page