WebClient httpGet with pre-encoded url's

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

WebClient httpGet with pre-encoded url's

Jan van de Sandt
Hello,

I am busy updating the CloudforkAWS code to use WebClient as a the HTTP client. CloudforkAWS provides a Smalltalk API to the Amazon AWS REST services.

WebClient works really well, the not so common HTTP methods like HEAD, DELETE and PUT that the Simple Storage Service (S3) require work without any problems :-)

There is only one small problem. The WebClient>>httpGet:do: method always encodes the urlString. In my situation the url is already encoded and encoding it a second time leads to errors. In my code I encode the individual query parameters names and values before I compose the complete url. If you have a parameters like this I think it is the only correct way:

param1=you&me
param2=coming for dinner?

So I added the following methods to WebClient:

httpGet: urlString encoded: isEncoded do: aBlock
"GET the response from the given url"
| request |
self initializeFromUrl: urlString.
request := self requestWithUrl: urlString encoded: isEncoded.
request method: 'GET'.
userAgent ifNotNil:[request headerAt: 'User-Agent' put: userAgent].
aBlock value: request.
^self sendRequest: request

and:

requestWithUrl: urlString encoded: isEncoded
"Create an return a new WebRequest initialized with the given url.
Subclasses can use this method to override the default request class."

^WebRequest new initializeFromUrl: urlString encoded: isEncoded.

It would be great if these methods are incorporated in the base package.

Jan.



Reply | Threaded
Open this post in threaded view
|

Re: WebClient httpGet with pre-encoded url's

Andreas.Raab
Hi Jan -

This should be fixed in the latest version. All requests can now
optionally take encoded URL strings. Give it a shot and let me know if
it works for you.

Hope this helps,
   - Andreas

On 8/9/2010 7:32 AM, Jan van de Sandt wrote:

> Hello,
>
> I am busy updating the CloudforkAWS code to use WebClient as a the HTTP
> client. CloudforkAWS provides a Smalltalk API to the Amazon AWS REST
> services.
>
> WebClient works really well, the not so common HTTP methods like HEAD,
> DELETE and PUT that the Simple Storage Service (S3) require work without
> any problems :-)
>
> There is only one small problem. The WebClient>>httpGet:do: method
> always encodes the urlString. In my situation the url is already encoded
> and encoding it a second time leads to errors. In my code I encode the
> individual query parameters names and values before I compose the
> complete url. If you have a parameters like this I think it is the only
> correct way:
>
> param1=you&me
> param2=coming for dinner?
>
> So I added the following methods to WebClient:
>
> httpGet: urlString encoded: isEncoded do: aBlock
> "GET the response from the given url"
> | request |
> self initializeFromUrl: urlString.
> request := self requestWithUrl: urlString encoded: isEncoded.
> request method: 'GET'.
> userAgent ifNotNil:[request headerAt: 'User-Agent' put: userAgent].
> aBlock value: request.
> ^self sendRequest: request
>
> and:
>
> requestWithUrl: urlString encoded: isEncoded
> "Create an return a new WebRequest initialized with the given url.
> Subclasses can use this method to override the default request class."
>
> ^WebRequest new initializeFromUrl: urlString encoded: isEncoded.
>
> It would be great if these methods are incorporated in the base package.
>
> Jan.
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: WebClient httpGet with pre-encoded url's

Jan van de Sandt
Hello Andreas,

Yes, it works fine. Thanks.

But now a request for another feature :-) AWS sometimes sends responses back with 'HTTP/1.1 307 Temporary Redirect'. I see that 301 and 302 redirects are handled automatically. Can you add support for a 307 redirect as well?

Jan.



On Tue, Aug 10, 2010 at 7:17 AM, Andreas Raab <[hidden email]> wrote:
Hi Jan -

This should be fixed in the latest version. All requests can now optionally take encoded URL strings. Give it a shot and let me know if it works for you.

Hope this helps,
 - Andreas


On 8/9/2010 7:32 AM, Jan van de Sandt wrote:
Hello,

I am busy updating the CloudforkAWS code to use WebClient as a the HTTP
client. CloudforkAWS provides a Smalltalk API to the Amazon AWS REST
services.

WebClient works really well, the not so common HTTP methods like HEAD,
DELETE and PUT that the Simple Storage Service (S3) require work without
any problems :-)

There is only one small problem. The WebClient>>httpGet:do: method
always encodes the urlString. In my situation the url is already encoded
and encoding it a second time leads to errors. In my code I encode the
individual query parameters names and values before I compose the
complete url. If you have a parameters like this I think it is the only
correct way:

param1=you&me
param2=coming for dinner?

So I added the following methods to WebClient:

httpGet: urlString encoded: isEncoded do: aBlock
"GET the response from the given url"
| request |
self initializeFromUrl: urlString.
request := self requestWithUrl: urlString encoded: isEncoded.
request method: 'GET'.
userAgent ifNotNil:[request headerAt: 'User-Agent' put: userAgent].
aBlock value: request.
^self sendRequest: request

and:

requestWithUrl: urlString encoded: isEncoded
"Create an return a new WebRequest initialized with the given url.
Subclasses can use this method to override the default request class."

^WebRequest new initializeFromUrl: urlString encoded: isEncoded.

It would be great if these methods are incorporated in the base package.

Jan.









Reply | Threaded
Open this post in threaded view
|

Re: WebClient httpGet with pre-encoded url's

Frank Shearar
Hm, I fell afoul of the 100k limit in posting a fix for this.

I'd changed a single line in WebClient>>sendRequest:content:size:

   "Handle redirect if needed"
   (self allowRedirect and: [#(301 302 307) includes: response code])
ifTrue:[

frank

On 2010/08/10 09:26, Jan van de Sandt wrote:

> Hello Andreas,
>
> Yes, it works fine. Thanks.
>
> But now a request for another feature :-) AWS sometimes sends responses
> back with 'HTTP/1.1 307 Temporary Redirect'. I see that 301 and 302
> redirects are handled automatically. Can you add support for a 307
> redirect as well?
>
> Jan.
>
>
>
> On Tue, Aug 10, 2010 at 7:17 AM, Andreas Raab <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi Jan -
>
>     This should be fixed in the latest version. All requests can now
>     optionally take encoded URL strings. Give it a shot and let me know
>     if it works for you.
>
>     Hope this helps,
>       - Andreas
>
>
>     On 8/9/2010 7:32 AM, Jan van de Sandt wrote:
>
>         Hello,
>
>         I am busy updating the CloudforkAWS code to use WebClient as a
>         the HTTP
>         client. CloudforkAWS provides a Smalltalk API to the Amazon AWS REST
>         services.
>
>         WebClient works really well, the not so common HTTP methods like
>         HEAD,
>         DELETE and PUT that the Simple Storage Service (S3) require work
>         without
>         any problems :-)
>
>         There is only one small problem. The WebClient>>httpGet:do: method
>         always encodes the urlString. In my situation the url is already
>         encoded
>         and encoding it a second time leads to errors. In my code I
>         encode the
>         individual query parameters names and values before I compose the
>         complete url. If you have a parameters like this I think it is
>         the only
>         correct way:
>
>         param1=you&me
>         param2=coming for dinner?
>
>         So I added the following methods to WebClient:
>
>         httpGet: urlString encoded: isEncoded do: aBlock
>         "GET the response from the given url"
>         | request |
>         self initializeFromUrl: urlString.
>         request := self requestWithUrl: urlString encoded: isEncoded.
>         request method: 'GET'.
>         userAgent ifNotNil:[request headerAt: 'User-Agent' put: userAgent].
>         aBlock value: request.
>         ^self sendRequest: request
>
>         and:
>
>         requestWithUrl: urlString encoded: isEncoded
>         "Create an return a new WebRequest initialized with the given url.
>         Subclasses can use this method to override the default request
>         class."
>
>         ^WebRequest new initializeFromUrl: urlString encoded: isEncoded.
>
>         It would be great if these methods are incorporated in the base
>         package.
>
>         Jan.
>
>
>
>
>
>
>
>
>
>