ZnClient 403 but 200 in Firefox

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

ZnClient 403 but 200 in Firefox

hernanmd
When I visit this site http://ifconfig.me/ip using the web browser I
get the expected text output. However if I try to do it from Pharo 6.1
(Windows 8.1) I get a 403 forbidden response using this code:

ZnClient new
    systemPolicy;
    beOneShot;
    accept: ZnMimeType textPlain;
    timeout: 6000;
    url: 'http://ifconfig.me/ip';
    get.

2018-09-16 03:06:18 011 Retrying ConnectionTimedOut: Cannot connect to
153.121.72.212:80
2018-09-16 03:06:19 012 Connection Established ifconfig.me:80
153.121.72.212 288ms
2018-09-16 03:06:19 013 Request Written a ZnRequest(GET /all.xml) 1ms
2018-09-16 03:06:19 014 Response Read a ZnResponse(403 Forbidden
text/html;charset=iso-8859-1 209B) 318ms
2018-09-16 03:06:19 015 GET /all.xml 403 209B 319ms
2018-09-16 03:06:19 016 Connection Closed 153.121.72.212:80

In Firefox this is the output

Request Headers Firefox:

Host: ifconfig.me
User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:62.0)
Gecko/20100101 Firefox/62.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US
Accept-Encoding: gzip, deflate
DNT: 1
Connection: keep-alive
Upgrade-Insecure-Requests: 1

Response Headers Firefox:

HTTP/1.1 200 OK
Date: Sun, 16 Sep 2018 16:47:02 GMT
Server: Apache
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 32
Connection: close
Content-Type: text/plain


Any idea?

Cheers,

Hernán

Reply | Threaded
Open this post in threaded view
|

Re: ZnClient 403 but 200 in Firefox

Sven Van Caekenberghe-2
Because they don't like Pharo or Zinc ;-)

ZnClient new
   systemPolicy;
   beOneShot;
   url: 'http://ifconfig.me/ip';
   accept: ZnMimeType textPlain;
   headerAt: 'User-Agent' put: 'curl/7.54.0';
   get.

Changing the User-Agent (effectively lying) works.

> On 10 Oct 2018, at 22:02, Hernán Morales Durand <[hidden email]> wrote:
>
> When I visit this site http://ifconfig.me/ip using the web browser I
> get the expected text output. However if I try to do it from Pharo 6.1
> (Windows 8.1) I get a 403 forbidden response using this code:
>
> ZnClient new
>    systemPolicy;
>    beOneShot;
>    accept: ZnMimeType textPlain;
>    timeout: 6000;
>    url: 'http://ifconfig.me/ip';
>    get.
>
> 2018-09-16 03:06:18 011 Retrying ConnectionTimedOut: Cannot connect to
> 153.121.72.212:80
> 2018-09-16 03:06:19 012 Connection Established ifconfig.me:80
> 153.121.72.212 288ms
> 2018-09-16 03:06:19 013 Request Written a ZnRequest(GET /all.xml) 1ms
> 2018-09-16 03:06:19 014 Response Read a ZnResponse(403 Forbidden
> text/html;charset=iso-8859-1 209B) 318ms
> 2018-09-16 03:06:19 015 GET /all.xml 403 209B 319ms
> 2018-09-16 03:06:19 016 Connection Closed 153.121.72.212:80
>
> In Firefox this is the output
>
> Request Headers Firefox:
>
> Host: ifconfig.me
> User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:62.0)
> Gecko/20100101 Firefox/62.0
> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> Accept-Language: en-US
> Accept-Encoding: gzip, deflate
> DNT: 1
> Connection: keep-alive
> Upgrade-Insecure-Requests: 1
>
> Response Headers Firefox:
>
> HTTP/1.1 200 OK
> Date: Sun, 16 Sep 2018 16:47:02 GMT
> Server: Apache
> Vary: Accept-Encoding
> Content-Encoding: gzip
> Content-Length: 32
> Connection: close
> Content-Type: text/plain
>
>
> Any idea?
>
> Cheers,
>
> Hernán
>


Reply | Threaded
Open this post in threaded view
|

Re: ZnClient 403 but 200 in Firefox

philippeback
In reply to this post by hernanmd
Works for me in a 5.0

Did this

c :=ZnClient new
    systemPolicy;
    beOneShot;
    accept: ZnMimeType textPlain;
    timeout: 6000;
    url: 'http://ifconfig.me/ip';
yourself.

c request headers at: 'User-Agent' put: 'Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:62.0)
Gecko/20100101 Firefox/62.0'.

c get.

Response: 95.182.154.13

Phil



On Wed, Oct 10, 2018 at 10:02 PM Hernán Morales Durand <[hidden email]> wrote:
When I visit this site http://ifconfig.me/ip using the web browser I
get the expected text output. However if I try to do it from Pharo 6.1
(Windows 8.1) I get a 403 forbidden response using this code:

ZnClient new
    systemPolicy;
    beOneShot;
    accept: ZnMimeType textPlain;
    timeout: 6000;
    url: 'http://ifconfig.me/ip';
    get.

2018-09-16 03:06:18 011 Retrying ConnectionTimedOut: Cannot connect to
153.121.72.212:80
2018-09-16 03:06:19 012 Connection Established ifconfig.me:80
153.121.72.212 288ms
2018-09-16 03:06:19 013 Request Written a ZnRequest(GET /all.xml) 1ms
2018-09-16 03:06:19 014 Response Read a ZnResponse(403 Forbidden
text/html;charset=iso-8859-1 209B) 318ms
2018-09-16 03:06:19 015 GET /all.xml 403 209B 319ms
2018-09-16 03:06:19 016 Connection Closed 153.121.72.212:80

In Firefox this is the output

Request Headers Firefox:

Host: ifconfig.me
User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:62.0)
Gecko/20100101 Firefox/62.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US
Accept-Encoding: gzip, deflate
DNT: 1
Connection: keep-alive
Upgrade-Insecure-Requests: 1

Response Headers Firefox:

HTTP/1.1 200 OK
Date: Sun, 16 Sep 2018 16:47:02 GMT
Server: Apache
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 32
Connection: close
Content-Type: text/plain


Any idea?

Cheers,

Hernán


Reply | Threaded
Open this post in threaded view
|

Re: ZnClient 403 but 200 in Firefox

hernanmd
In reply to this post by Sven Van Caekenberghe-2
Wow, didn't knew that someone actually could reject/allow specific UAs.

Thank you Sven,

Cheers,

Hernán
El mié., 10 oct. 2018 a las 17:33, Sven Van Caekenberghe
(<[hidden email]>) escribió:

>
> Because they don't like Pharo or Zinc ;-)
>
> ZnClient new
>    systemPolicy;
>    beOneShot;
>    url: 'http://ifconfig.me/ip';
>    accept: ZnMimeType textPlain;
>    headerAt: 'User-Agent' put: 'curl/7.54.0';
>    get.
>
> Changing the User-Agent (effectively lying) works.
>
> > On 10 Oct 2018, at 22:02, Hernán Morales Durand <[hidden email]> wrote:
> >
> > When I visit this site http://ifconfig.me/ip using the web browser I
> > get the expected text output. However if I try to do it from Pharo 6.1
> > (Windows 8.1) I get a 403 forbidden response using this code:
> >
> > ZnClient new
> >    systemPolicy;
> >    beOneShot;
> >    accept: ZnMimeType textPlain;
> >    timeout: 6000;
> >    url: 'http://ifconfig.me/ip';
> >    get.
> >
> > 2018-09-16 03:06:18 011 Retrying ConnectionTimedOut: Cannot connect to
> > 153.121.72.212:80
> > 2018-09-16 03:06:19 012 Connection Established ifconfig.me:80
> > 153.121.72.212 288ms
> > 2018-09-16 03:06:19 013 Request Written a ZnRequest(GET /all.xml) 1ms
> > 2018-09-16 03:06:19 014 Response Read a ZnResponse(403 Forbidden
> > text/html;charset=iso-8859-1 209B) 318ms
> > 2018-09-16 03:06:19 015 GET /all.xml 403 209B 319ms
> > 2018-09-16 03:06:19 016 Connection Closed 153.121.72.212:80
> >
> > In Firefox this is the output
> >
> > Request Headers Firefox:
> >
> > Host: ifconfig.me
> > User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:62.0)
> > Gecko/20100101 Firefox/62.0
> > Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> > Accept-Language: en-US
> > Accept-Encoding: gzip, deflate
> > DNT: 1
> > Connection: keep-alive
> > Upgrade-Insecure-Requests: 1
> >
> > Response Headers Firefox:
> >
> > HTTP/1.1 200 OK
> > Date: Sun, 16 Sep 2018 16:47:02 GMT
> > Server: Apache
> > Vary: Accept-Encoding
> > Content-Encoding: gzip
> > Content-Length: 32
> > Connection: close
> > Content-Type: text/plain
> >
> >
> > Any idea?
> >
> > Cheers,
> >
> > Hernán
> >
>
>

Reply | Threaded
Open this post in threaded view
|

Re: ZnClient 403 but 200 in Firefox

Jerry Kott-3
A lot of thse types of services do that. Not sure why because it’s so easy to spoof anything in an HTTP(s) request...

Jerry Kott
This message has been digitally signed. 
PGP Fingerprint:
A9181736DD2F1B6CC7CF9E51AC8514F48C0979A5



On 10-10-2018, at 3:32 PM, Hernán Morales Durand <[hidden email]> wrote:

Wow, didn't knew that someone actually could reject/allow specific UAs.

Thank you Sven,

Cheers,

Hernán
El mié., 10 oct. 2018 a las 17:33, Sven Van Caekenberghe
(<[hidden email]>) escribió:

Because they don't like Pharo or Zinc ;-)

ZnClient new
  systemPolicy;
  beOneShot;
  url: 'http://ifconfig.me/ip';
  accept: ZnMimeType textPlain;
  headerAt: 'User-Agent' put: 'curl/7.54.0';
  get.

Changing the User-Agent (effectively lying) works.

On 10 Oct 2018, at 22:02, Hernán Morales Durand <[hidden email]> wrote:

When I visit this site http://ifconfig.me/ip using the web browser I
get the expected text output. However if I try to do it from Pharo 6.1
(Windows 8.1) I get a 403 forbidden response using this code:

ZnClient new
  systemPolicy;
  beOneShot;
  accept: ZnMimeType textPlain;
  timeout: 6000;
  url: 'http://ifconfig.me/ip';
  get.

2018-09-16 03:06:18 011 Retrying ConnectionTimedOut: Cannot connect to
153.121.72.212:80
2018-09-16 03:06:19 012 Connection Established ifconfig.me:80
153.121.72.212 288ms
2018-09-16 03:06:19 013 Request Written a ZnRequest(GET /all.xml) 1ms
2018-09-16 03:06:19 014 Response Read a ZnResponse(403 Forbidden
text/html;charset=iso-8859-1 209B) 318ms
2018-09-16 03:06:19 015 GET /all.xml 403 209B 319ms
2018-09-16 03:06:19 016 Connection Closed 153.121.72.212:80

In Firefox this is the output

Request Headers Firefox:

Host: ifconfig.me
User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:62.0)
Gecko/20100101 Firefox/62.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US
Accept-Encoding: gzip, deflate
DNT: 1
Connection: keep-alive
Upgrade-Insecure-Requests: 1

Response Headers Firefox:

HTTP/1.1 200 OK
Date: Sun, 16 Sep 2018 16:47:02 GMT
Server: Apache
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 32
Connection: close
Content-Type: text/plain


Any idea?

Cheers,

Hernán







signature.asc (891 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: ZnClient 403 but 200 in Firefox

NorbertHartl
In reply to this post by hernanmd
It is done very often. In the past you could solve glitches between browser engines. Nowadays it is done e.g. to discover mobile clients. In one of my services the easiest to keep foreigners out was rejecting all requests which are not zinc 😉

Norbert

> Am 11.10.2018 um 00:32 schrieb Hernán Morales Durand <[hidden email]>:
>
> Wow, didn't knew that someone actually could reject/allow specific UAs.
>
> Thank you Sven,
>
> Cheers,
>
> Hernán
> El mié., 10 oct. 2018 a las 17:33, Sven Van Caekenberghe
> (<[hidden email]>) escribió:
>>
>> Because they don't like Pharo or Zinc ;-)
>>
>> ZnClient new
>>   systemPolicy;
>>   beOneShot;
>>   url: 'http://ifconfig.me/ip';
>>   accept: ZnMimeType textPlain;
>>   headerAt: 'User-Agent' put: 'curl/7.54.0';
>>   get.
>>
>> Changing the User-Agent (effectively lying) works.
>>
>>> On 10 Oct 2018, at 22:02, Hernán Morales Durand <[hidden email]> wrote:
>>>
>>> When I visit this site http://ifconfig.me/ip using the web browser I
>>> get the expected text output. However if I try to do it from Pharo 6.1
>>> (Windows 8.1) I get a 403 forbidden response using this code:
>>>
>>> ZnClient new
>>>   systemPolicy;
>>>   beOneShot;
>>>   accept: ZnMimeType textPlain;
>>>   timeout: 6000;
>>>   url: 'http://ifconfig.me/ip';
>>>   get.
>>>
>>> 2018-09-16 03:06:18 011 Retrying ConnectionTimedOut: Cannot connect to
>>> 153.121.72.212:80
>>> 2018-09-16 03:06:19 012 Connection Established ifconfig.me:80
>>> 153.121.72.212 288ms
>>> 2018-09-16 03:06:19 013 Request Written a ZnRequest(GET /all.xml) 1ms
>>> 2018-09-16 03:06:19 014 Response Read a ZnResponse(403 Forbidden
>>> text/html;charset=iso-8859-1 209B) 318ms
>>> 2018-09-16 03:06:19 015 GET /all.xml 403 209B 319ms
>>> 2018-09-16 03:06:19 016 Connection Closed 153.121.72.212:80
>>>
>>> In Firefox this is the output
>>>
>>> Request Headers Firefox:
>>>
>>> Host: ifconfig.me
>>> User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:62.0)
>>> Gecko/20100101 Firefox/62.0
>>> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
>>> Accept-Language: en-US
>>> Accept-Encoding: gzip, deflate
>>> DNT: 1
>>> Connection: keep-alive
>>> Upgrade-Insecure-Requests: 1
>>>
>>> Response Headers Firefox:
>>>
>>> HTTP/1.1 200 OK
>>> Date: Sun, 16 Sep 2018 16:47:02 GMT
>>> Server: Apache
>>> Vary: Accept-Encoding
>>> Content-Encoding: gzip
>>> Content-Length: 32
>>> Connection: close
>>> Content-Type: text/plain
>>>
>>>
>>> Any idea?
>>>
>>> Cheers,
>>>
>>> Hernán
>>>
>>
>>
>


Reply | Threaded
Open this post in threaded view
|

Re: ZnClient 403 but 200 in Firefox

NorbertHartl
In reply to this post by Jerry Kott-3


Am 11.10.2018 um 02:02 schrieb Jerry Kott <[hidden email]>:

A lot of thse types of services do that. Not sure why because it’s so easy to spoof anything in an HTTP(s) request...

Technically this is right. Practically you still have to know which content to provide. I use it to get rid of requests to my service. I don‘t have a problem with those that take the effort to figure out. These are way below 1% of all requests. 

Norbert

Jerry Kott
This message has been digitally signed. 
PGP Fingerprint:
A9181736DD2F1B6CC7CF9E51AC8514F48C0979A5



On 10-10-2018, at 3:32 PM, Hernán Morales Durand <[hidden email]> wrote:

Wow, didn't knew that someone actually could reject/allow specific UAs.

Thank you Sven,

Cheers,

Hernán
El mié., 10 oct. 2018 a las 17:33, Sven Van Caekenberghe
(<[hidden email]>) escribió:

Because they don't like Pharo or Zinc ;-)

ZnClient new
  systemPolicy;
  beOneShot;
  url: 'http://ifconfig.me/ip';
  accept: ZnMimeType textPlain;
  headerAt: 'User-Agent' put: 'curl/7.54.0';
  get.

Changing the User-Agent (effectively lying) works.

On 10 Oct 2018, at 22:02, Hernán Morales Durand <[hidden email]> wrote:

When I visit this site http://ifconfig.me/ip using the web browser I
get the expected text output. However if I try to do it from Pharo 6.1
(Windows 8.1) I get a 403 forbidden response using this code:

ZnClient new
  systemPolicy;
  beOneShot;
  accept: ZnMimeType textPlain;
  timeout: 6000;
  url: 'http://ifconfig.me/ip';
  get.

2018-09-16 03:06:18 011 Retrying ConnectionTimedOut: Cannot connect to
153.121.72.212:80
2018-09-16 03:06:19 012 Connection Established ifconfig.me:80
153.121.72.212 288ms
2018-09-16 03:06:19 013 Request Written a ZnRequest(GET /all.xml) 1ms
2018-09-16 03:06:19 014 Response Read a ZnResponse(403 Forbidden
text/html;charset=iso-8859-1 209B) 318ms
2018-09-16 03:06:19 015 GET /all.xml 403 209B 319ms
2018-09-16 03:06:19 016 Connection Closed 153.121.72.212:80

In Firefox this is the output

Request Headers Firefox:

Host: ifconfig.me
User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:62.0)
Gecko/20100101 Firefox/62.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US
Accept-Encoding: gzip, deflate
DNT: 1
Connection: keep-alive
Upgrade-Insecure-Requests: 1

Response Headers Firefox:

HTTP/1.1 200 OK
Date: Sun, 16 Sep 2018 16:47:02 GMT
Server: Apache
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 32
Connection: close
Content-Type: text/plain


Any idea?

Cheers,

Hernán






Reply | Threaded
Open this post in threaded view
|

Re: ZnClient 403 but 200 in Firefox

tesonep@gmail.com
In reply to this post by hernanmd
Hi Hernan, quite interesting.

I have tried with curl and it works, but with wget it doesn't work. Both using the brew versions in my OSX machine.

To me, it is a problem of the headers.

Cheers.


On Wed, Oct 10, 2018 at 10:07 PM Hernán Morales Durand <[hidden email]> wrote:
When I visit this site http://ifconfig.me/ip using the web browser I
get the expected text output. However if I try to do it from Pharo 6.1
(Windows 8.1) I get a 403 forbidden response using this code:

ZnClient new
    systemPolicy;
    beOneShot;
    accept: ZnMimeType textPlain;
    timeout: 6000;
    url: 'http://ifconfig.me/ip';
    get.

2018-09-16 03:06:18 011 Retrying ConnectionTimedOut: Cannot connect to
153.121.72.212:80
2018-09-16 03:06:19 012 Connection Established ifconfig.me:80
153.121.72.212 288ms
2018-09-16 03:06:19 013 Request Written a ZnRequest(GET /all.xml) 1ms
2018-09-16 03:06:19 014 Response Read a ZnResponse(403 Forbidden
text/html;charset=iso-8859-1 209B) 318ms
2018-09-16 03:06:19 015 GET /all.xml 403 209B 319ms
2018-09-16 03:06:19 016 Connection Closed 153.121.72.212:80

In Firefox this is the output

Request Headers Firefox:

Host: ifconfig.me
User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:62.0)
Gecko/20100101 Firefox/62.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US
Accept-Encoding: gzip, deflate
DNT: 1
Connection: keep-alive
Upgrade-Insecure-Requests: 1

Response Headers Firefox:

HTTP/1.1 200 OK
Date: Sun, 16 Sep 2018 16:47:02 GMT
Server: Apache
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 32
Connection: close
Content-Type: text/plain


Any idea?

Cheers,

Hernán



--
Pablo Tesone.
[hidden email]