Re: Issue 6986 in pharo: Zn issues with Transparent Proxy, Gzip compression and Redirect on Squeaksource

Updates:
        Summary: Zn issues with Transparent Proxy, Gzip compression and Redirect  
on Squeaksource

Comment #8 on issue 6986 by [hidden email]: Zn issues with  
Transparent Proxy, Gzip compression and Redirect on Squeaksource
http://code.google.com/p/pharo/issues/detail?id=6986

(No comment was entered for this change.)


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker

Comment #9 on issue 6986 by [hidden email]: Zn issues with  
Transparent Proxy, Gzip compression and Redirect on Squeaksource
http://code.google.com/p/pharo/issues/detail?id=6986

Camillo,

Explore the following two:

ZnClient new
        setAcceptEncodingGzip;
        url: 'http://pharo.gforge.inria.fr/ci/image/20/20424.zip';
        head;
        yourself.
       
ZnClient new
        setAcceptEncodingGzip;
        url: 'http://stfx.eu/small.html';
        head;
        yourself.

You'll notice that pharo.gforce.inria.fr does not have a Content-Encoding:  
gzip in its response - that is a misconfiguration IMO. Since your  
stacktrace show Zinc using a GZipReadStream in  
ZnEntityReader>>readEntityFromStream I am guessing that the proxy added the  
Content-Encoding: gzip and some mixup resulted.

Can the second example be read from behind the proxy ?

Sven


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker

Comment #10 on issue 6986 by [hidden email]: Zn issues with  
Transparent Proxy, Gzip compression and Redirect on Squeaksource
http://code.google.com/p/pharo/issues/detail?id=6986

Sorry, I tried a random repository on squeaksource to expose the error.  
Other sides work perfectly (like smalltalk hub which is also gzip encoded).  
The link is just the image I used, so we can properly restore the stack  
trace with Fuel :).

And yes gforge, does not always add a gzip header, no way we can change  
this globally though ;) that's why only the text files are properly  
compressed.

Both of your examples work.



_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker

Comment #11 on issue 6986 by [hidden email]: Zn issues with  
Transparent Proxy, Gzip compression and Redirect on Squeaksource
http://code.google.com/p/pharo/issues/detail?id=6986

They both work ? Remember, they were HEAD requests.

Do the GET variants also work ?

ZnClient new
        setAcceptEncodingGzip;
        url: 'http://pharo.gforge.inria.fr/ci/image/20/20424.zip';
        get.
       
ZnClient new
        setAcceptEncodingGzip;
        url: 'http://stfx.eu/small.html';
        get.


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker

Comment #12 on issue 6986 by [hidden email]: Zn issues with  
Transparent Proxy, Gzip compression and Redirect on Squeaksource
http://code.google.com/p/pharo/issues/detail?id=6986

yes all 4 requests work (head AND get).

Believe me, it is something very strange with the redirect on  
squeaksource :P
If I decode the partial response, I get the zipped version  
of "WARedirectXYZ", which IMO is quite strange, no?


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker

Comment #13 on issue 6986 by [hidden email]: Zn issues with  
Transparent Proxy, Gzip compression and Redirect on Squeaksource
http://code.google.com/p/pharo/issues/detail?id=6986

If they all 4 work, there is nothing fundamentally wrong with Zinc I guess.

We probably have to concentrate on the redirect.

Can you give me a concrete URL that shows the problem ?

Have you tried getting the same URL with curl ?  Don't forget to add the  
accept header !


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker

Comment #14 on issue 6986 by [hidden email]: Zn issues with  
Transparent Proxy, Gzip compression and Redirect on Squeaksource
http://code.google.com/p/pharo/issues/detail?id=6986

the url used to get the debug trace above was:  
http://www.squeaksource.com/PharoTaskForces

curl  http://www.squeaksource.com/PharoTaskForces
a WARedirectResponse

curl --header 'accept-encoding: gzip'  
http://www.squeaksource.com/PharoTaskForces
Kw
   JM?,JM. J-.??+N???2

curl --header 'accept-encoding: gzip'  
http://www.squeaksource.com/PharoTaskForces | gunzip
a WARedirectResponse

If Zinc would not trie to fully extract the body form a redirect response  
it would work :) (but I guess that will break other things ;P)


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker

Comment #15 on issue 6986 by [hidden email]: Zn issues with  
Transparent Proxy, Gzip compression and Redirect on Squeaksource
http://code.google.com/p/pharo/issues/detail?id=6986

Aahh, I think you are getting very close !

But the question remains: how can Zinc then do the right thing when not  
behind a proxy ?

I mean, even in that case, Zn will try to extract the the body, right ?

ZnClient new
        setAcceptEncodingGzip;
        url: 'http://www.squeaksource.com/PharoTaskForces';
        get;
        yourself.

ZnClient new
        dontFollowRedirects;
        setAcceptEncodingGzip;
        url: 'http://www.squeaksource.com/PharoTaskForces';
        get;
        yourself.

You can also send #logToTranscript to the ZnClient instance.

BTW: I just discovered that when following the redirect, Zn drops the  
accept-encoding:gzip ...


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker

Comment #16 on issue 6986 by [hidden email]: Zn issues with  
Transparent Proxy, Gzip compression and Redirect on Squeaksource
http://code.google.com/p/pharo/issues/detail?id=6986

so yes it must extract it in both cases...
now there must be a nice side-effect from the proxy that exposes this.  
Maybe it closes the stream earlier?

Now I note a very strange effect. Opening the image, and just running the  
first request from above works, but only the very first time! The second  
one always fails...


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker

Comment #17 on issue 6986 by [hidden email]: Zn issues with  
Transparent Proxy, Gzip compression and Redirect on Squeaksource
http://code.google.com/p/pharo/issues/detail?id=6986

so here's a working and the following non-working log trace:



2012-12-03 13:27:21 591514 I Wrote a ZnRequest(GET /PharoTaskForces)
2012-12-03 13:27:21 591514 D Sent headers
Accept: */*
User-Agent: Zinc HTTP Components 1.0
Accept-Encoding: gzip
Host: www.squeaksource.com

2012-12-03 13:27:22 591514 I Read a ZnResponse(302 Moved Temporarily  
text/html )
2012-12-03 13:27:22 591514 D Received headers
Content-Encoding: gzip
Content-Length: 40
Vary: Accept-Encoding
Connection: close
X-Cache-Lookup: MISS from proxy.uba.ar:8080
X-Cache-Lookup: MISS from proxy.fcen.uba.ar:8080
Content-Type: text/html
X-Cache: MISS from proxy.uba.ar
X-Cache: MISS from proxy.fcen.uba.ar
Date: Mon, 03 Dec 2012 17:27:21 GMT
Location: /PharoTaskForces/
X-Code-Repository: SqueakSource
Via: 1.1 proxy4.gestion.uba.ar:8080 (squid/2.7.STABLE3), 1.0  
proxy.fcen.uba.ar:8080 (squid/2.7.STABLE9)
Server: Comanche/6.2 (unix)

2012-12-03 13:27:22 591514 D Redirecting
2012-12-03 13:27:22 591514 I Wrote a ZnRequest(GET /PharoTaskForces/)
2012-12-03 13:27:22 591514 D Sent headers
Accept: */*
User-Agent: Zinc HTTP Components 1.0
Host: www.squeaksource.com

2012-12-03 13:27:24 591514 I Read a ZnResponse(200 OK text/html 138655B)
2012-12-03 13:27:24 591514 D Received headers
Vary: Accept-Encoding
Content-Length: 138655
Connection: close
Content-Type: text/html
X-Cache-Lookup: MISS from proxy.uba.ar:8080
X-Cache-Lookup: MISS from proxy.fcen.uba.ar:8080
X-Cache: MISS from proxy.uba.ar
X-Cache: MISS from proxy.fcen.uba.ar
Date: Mon, 03 Dec 2012 17:27:22 GMT
X-Code-Repository: SqueakSource
Via: 1.1 proxy2.gestion.uba.ar:8080 (squid/2.7.STABLE3), 1.0  
proxy.fcen.uba.ar:8080 (squid/2.7.STABLE9)
Server: Comanche/6.2 (unix)

2012-12-03 13:27:24 591514 T GET /PharoTaskForces/ 200 138655B 2817ms
2012-12-03 13:27:29 591514 I Wrote a ZnRequest(GET /PharoTaskForces)
2012-12-03 13:27:29 591514 D Sent headers
Accept: */*
User-Agent: Zinc HTTP Components 1.0
Accept-Encoding: gzip
Host: www.squeaksource.com



_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker

Comment #18 on issue 6986 by [hidden email]: Zn issues with  
Transparent Proxy, Gzip compression and Redirect on Squeaksource
http://code.google.com/p/pharo/issues/detail?id=6986

in my case the culprit is

GZipReadStream>>getFirstBuffer
        "Get the first source buffer after initialization has been done"
        sourceStream == nil ifTrue:[^self].
        source := sourceStream next: 1 << 16. "This is more than enough..."
        sourceLimit := source size.

fetching trying to fetch 65536 bytes... how can that work in the normal  
case? ;)


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker

Comment #19 on issue 6986 by [hidden email]: Zn issues with  
Transparent Proxy, Gzip compression and Redirect on Squeaksource
http://code.google.com/p/pharo/issues/detail?id=6986

Ah, the Pharo underground: never go there ;-)

Actually, it looks silly, but it should work: #next: will return what it  
has read when #atEnd, so it behaves like a limited #upToEnd.

When you are stuck here it means that GZipReadStream just can get its data  
just to get started.

I am going home right now, talk to you later this evening...


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker

Updates:
        Labels: -Milestone-2.0

Comment #20 on issue 6986 by [hidden email]: Zn issues with  
Transparent Proxy, Gzip compression and Redirect on Squeaksource
http://code.google.com/p/pharo/issues/detail?id=6986

As this error does not happen for everyone every time, I recathegorze this  
issue as a non-show stopper for 2.0


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker