Zinc redirect bug?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Zinc redirect bug?

Aliaksei Syrel
Hello,

I am trying to download a file from bintray with the help of ZnClient.
I'm doing the following:

ZnClient new
   systemPolicy;
   get;
   yourself

which results in Error 403.

At the same time wget works as expected:


Trying to open this url in a browser works too.

I tried to find out what may cause the error and it looks like it is something related to redirects because request and response headers look good...

Thanks,
Alex
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Zinc redirect bug?

Tim Mackinnon
I’m not sure if this is related - but the Pharo6 image is missing a ZnConstant initialise, and I recall redirect was one of the error codes missing. Possibly re-initialising ZnConstants might fix your problem?

Tim

On 14 Jul 2017, at 12:02, Aliaksei Syrel <[hidden email]> wrote:

Hello,

I am trying to download a file from bintray with the help of ZnClient.
I'm doing the following:

ZnClient new
   systemPolicy;
   get;
   yourself

which results in Error 403.

At the same time wget works as expected:


Trying to open this url in a browser works too.

I tried to find out what may cause the error and it looks like it is something related to redirects because request and response headers look good...

Thanks,
Alex

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Zinc redirect bug?

Sven Van Caekenberghe-2
In reply to this post by Aliaksei Syrel
Hi Aliaksei,

This is an interesting problem, thanks for reporting it. I can reproduce the problem.

If you inspect 'ZnLogEvent announcer' before doing the request, you can see that ZnClient does seem to do the same as 'curl -vOL https://dl.bintray.com/syrel/Moz2D/osx/development/i386/libMoz2D.dylib'. It follows one redirect, to the correct host and URL.

One difference that I see that might lead to the 403 Forbidden is that Zn has a slight difference in the redirect URL being followed.

Curl seems to follow

/86/866a31722e3a9ea19c2bc4eeed2e1e69f7339ddd460500f2a33b4c9d041ef9b4?__gda__=exp=1500058371~hmac=820d22695ef0a821abc7e6eb7be6749ad43e8e0f74ba4b103476aa1b3663b381&response-content-disposition=attachment%3Bfilename%3D%22libMoz2D.dylib%22&response-content-type=application%2Fx-www-form-urlencoded&requestInfo=U2FsdGVkX19XiffCdVJdoADmObVHKjU4Wla5_WKFKHoNAvXQw8ponQIqWBsRUfNNDaULVXlXwB22y7xy0ls3DGkl700_9RxT-HtU92OHYdgKzSq9a6A7p5NIoMi4avCjPk_UCnVbt5U4_nxvkmKtuA

While Zn turns this (parses the URL and then renders it again) into

/86/866a31722e3a9ea19c2bc4eeed2e1e69f7339ddd460500f2a33b4c9d041ef9b4?response-content-type=application/x-www-form-urlencoded&response-content-disposition=attachment;filename%3D%22libMoz2D.dylib%22&requestInfo=U2FsdGVkX19XiffCdVJdoADmObVHKjU4Wla5_WKFKHoNAvXQw8ponQIqWBsRUfNNDaULVXlXwB22y7xy0ls3DGkl700_9RxT-HtU92OHYdgKzSq9a6A7p5NIoMi4avCjPk_UCnVbt5U4_nxvkmKtuA&__gda__=exp%3D1500058371~hmac%3D820d22695ef0a821abc7e6eb7be6749ad43e8e0f74ba4b103476aa1b3663b381

The difference is in the encoding of the __gda__ query parameter value. That contains = characters. These get escaped by Zn (the %3D) but are left untouched by curl. Maybe the server does not like that. I have to look deeper into the spec to figure out if this is a bug or not. Since query parameters look like key=value it seems odd to allow = in the value part.

Another possible issue might be the order of query parameters that got changed.

I'll come back to you if I find out more.

Sven

> On 14 Jul 2017, at 13:02, Aliaksei Syrel <[hidden email]> wrote:
>
> Hello,
>
> I am trying to download a file from bintray with the help of ZnClient.
> I'm doing the following:
>
> ZnClient new
>    systemPolicy;
>    url: 'https://dl.bintray.com/syrel/Moz2D/osx/development/i386/libMoz2D.dylib';
>    get;
>    yourself
>
> which results in Error 403.
>
> At the same time wget works as expected:
>
> wget https://dl.bintray.com/syrel/Moz2D/osx/development/i386/libMoz2D.dylib
>
> Trying to open this url in a browser works too.
>
> I tried to find out what may cause the error and it looks like it is something related to redirects because request and response headers look good...
>
> Thanks,
> Alex


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Zinc redirect bug?

Sven Van Caekenberghe-2

> On 14 Jul 2017, at 21:12, Sven Van Caekenberghe <[hidden email]> wrote:
>
> Hi Aliaksei,
>
> This is an interesting problem, thanks for reporting it. I can reproduce the problem.
>
> If you inspect 'ZnLogEvent announcer' before doing the request, you can see that ZnClient does seem to do the same as 'curl -vOL https://dl.bintray.com/syrel/Moz2D/osx/development/i386/libMoz2D.dylib'. It follows one redirect, to the correct host and URL.
>
> One difference that I see that might lead to the 403 Forbidden is that Zn has a slight difference in the redirect URL being followed.
>
> Curl seems to follow
>
> /86/866a31722e3a9ea19c2bc4eeed2e1e69f7339ddd460500f2a33b4c9d041ef9b4?__gda__=exp=1500058371~hmac=820d22695ef0a821abc7e6eb7be6749ad43e8e0f74ba4b103476aa1b3663b381&response-content-disposition=attachment%3Bfilename%3D%22libMoz2D.dylib%22&response-content-type=application%2Fx-www-form-urlencoded&requestInfo=U2FsdGVkX19XiffCdVJdoADmObVHKjU4Wla5_WKFKHoNAvXQw8ponQIqWBsRUfNNDaULVXlXwB22y7xy0ls3DGkl700_9RxT-HtU92OHYdgKzSq9a6A7p5NIoMi4avCjPk_UCnVbt5U4_nxvkmKtuA
>
> While Zn turns this (parses the URL and then renders it again) into
>
> /86/866a31722e3a9ea19c2bc4eeed2e1e69f7339ddd460500f2a33b4c9d041ef9b4?response-content-type=application/x-www-form-urlencoded&response-content-disposition=attachment;filename%3D%22libMoz2D.dylib%22&requestInfo=U2FsdGVkX19XiffCdVJdoADmObVHKjU4Wla5_WKFKHoNAvXQw8ponQIqWBsRUfNNDaULVXlXwB22y7xy0ls3DGkl700_9RxT-HtU92OHYdgKzSq9a6A7p5NIoMi4avCjPk_UCnVbt5U4_nxvkmKtuA&__gda__=exp%3D1500058371~hmac%3D820d22695ef0a821abc7e6eb7be6749ad43e8e0f74ba4b103476aa1b3663b381
>
> The difference is in the encoding of the __gda__ query parameter value. That contains = characters. These get escaped by Zn (the %3D) but are left untouched by curl. Maybe the server does not like that. I have to look deeper into the spec to figure out if this is a bug or not. Since query parameters look like key=value it seems odd to allow = in the value part.
>
> Another possible issue might be the order of query parameters that got changed.
>
> I'll come back to you if I find out more.
>
> Sven

By (temporarily) changing ZnResourceMetaUtils>>#writeQueryFields:with:on: to use #querySafeSet instead of #queryKeyValueSafeSet (so that $= no longer gets encoded), the request passes, i.e. the redirect is followed and returns 200 OK.

The next problem is the interpretation of the result ;-)

The returned resource is specified to have a content-type of application/x-www-form-urlencoded and Zn interprets it as such. However, that does not seem to be the case. Rather it seems to be binary data. Now, returning application/x-www-form-urlencoded data as response to a request is very weird, I have never seen that.

It seems as if browsers and curl just ignore the actual type and just dump the data in a file (content-disposition: attachment;filename="libMoz2D.dylib"). Where is such a behaviour described ?

Aliaksei, could you maybe upload something like an small JPG or PNG image using the same mechanism ? Then it is easier to test the correctness of the result.

Could it be that you uploaded libMoz2D.dylib incorrectly ? Because application/x-www-form-urlencoded is normally only used in the context of HTTP forms that POSTed.

>> On 14 Jul 2017, at 13:02, Aliaksei Syrel <[hidden email]> wrote:
>>
>> Hello,
>>
>> I am trying to download a file from bintray with the help of ZnClient.
>> I'm doing the following:
>>
>> ZnClient new
>>   systemPolicy;
>>   url: 'https://dl.bintray.com/syrel/Moz2D/osx/development/i386/libMoz2D.dylib';
>>   get;
>>   yourself
>>
>> which results in Error 403.
>>
>> At the same time wget works as expected:
>>
>> wget https://dl.bintray.com/syrel/Moz2D/osx/development/i386/libMoz2D.dylib
>>
>> Trying to open this url in a browser works too.
>>
>> I tried to find out what may cause the error and it looks like it is something related to redirects because request and response headers look good...
>>
>> Thanks,
>> Alex
>


Loading...