Smalltalk › Squeak › Squeak - Dev

[squeak-dev] Curl

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

7 messages Options

Steven W Riggins

[squeak-dev] Curl

Is anyone using the curl plugin from:

http://wiki.squeak.org/squeak/5865

I am trying to do a post to a site that redirects, but I see two
anomalies:

1) the contents remain in the contents buffer, so that tells me I am
doing posts incorrectly
2) I just get the http page back, not the results of the post

I tried this with another form and it posted ok, but I did get the
post in the contents buffer as well.

The following command to curl works as desired:

curl -d "q=title:help&fmt=xml&rows=50&fl=identifier&xmlsearch=Search" http://www.archive.org/advancedsearch.php/searchresults.php

but:

c := Curl new.
c url: 'http://www.archive.org/advancedsearch.php/searchresults.php'
c clearContents; contents:
'q=title:dodgers&fmt=xml&rows=50&fl=identifier&indent=&xmlsearch=Search'
c post

returns:

'q=title
%3Adodgers&fmt=xml&rows=50&fl=identifier&indent=&xmlsearch=Search'

ie, the post failed.

Steve

Danil Osipchuk-2

Re: [squeak-dev] Curl

Steve hello,

Although I've written the curl plugin I'm no expert in http protocol (redirects, posts etc)..
My shot: you may try to ask curl to follow redirects as in the example below (not onFollowLocation switch):

c := Curl new.
c onFollowLocation.
c url: 'http://www.archive.org/advancedsearch.php/searchresults.php'.
c clearContents; contents: 'q=title:dodgers&fmt=xml&rows=50&fl=identifier&indent=&xmlsearch=Search'.
c post.
c contents.

I'm getting something like this. If it is not what you need, could you please explain correct behaviour so I may able to fix it?

q=title:dodgers&fmt=xml&rows=50&fl=identifier&indent=&xmlsearch=Search<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">99</int><lst name="params"><str name="wt">xml</str><str name="rows">50</str><str name="indent"/><str name="qin">title:dodgers</str><str name="q">title:dodgers</str><str name="fl"/></lst></lst><result name="response" numFound="8" start="0" maxScore="6.4772797"><doc><float name="score">6.4772797</float><float name="avg_rating">0.0</float>...........

Regards,
Danil

2008/8/16 Steven W Riggins <[hidden email]>

Is anyone using the curl plugin from:

http://wiki.squeak.org/squeak/5865

I am trying to do a post to a site that redirects, but I see two anomalies:

1) the contents remain in the contents buffer, so that tells me I am doing posts incorrectly
2) I just get the http page back, not the results of the post

I tried this with another form and it posted ok, but I did get the post in the contents buffer as well.

The following command to curl works as desired:

curl -d "q=title:help&fmt=xml&rows=50&fl=identifier&xmlsearch=Search" http://www.archive.org/advancedsearch.php/searchresults.php

but:

c := Curl new.
c url: 'http://www.archive.org/advancedsearch.php/searchresults.php'
c clearContents; contents: 'q=title:dodgers&fmt=xml&rows=50&fl=identifier&indent=&xmlsearch=Search'
c post

returns:

'q=title%3Adodgers&fmt=xml&rows=50&fl=identifier&indent=&xmlsearch=Search'

ie, the post failed.

Steve

Danil Osipchuk-2

Re: [squeak-dev] Curl

Argh sorry, - just noticed the example with cmd curl invocation - as usual after the submitting of meaningless post...

Danil

Steven W Riggins

Re: [squeak-dev] Curl

I'm sorry?

the onFollowLocation helped a ton! I have been blind this last week.

Which example did you notice?

As you noted, contents after the post has:

q
=title:dodgers&fmt=xml&rows=50&fl=identifier&indent=&xmlsearch=Search<?
xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int
name="QTime">99</int><lst name="params"><str name="wt">xml</str><str
name="rows">50</str><str name="indent"/><str name="qin">title:dodgers</
str><str name="q">title:dodgers</str><str name="fl"/></lst></
lst><result name="response" numFound="8" start="0"
maxScore="6.4772797"><doc><float name="score">6.4772797</float><float
name="avg_rating">0.0</float>...........

but contents still has what *I* put in contents, as well as the
response from the host. Should it? Seems to me that the
"q
=title:dodgers&fmt=xml&rows=50&fl=identifier&indent=&xmlsearch=Search"
bit should not be in contents after the post

Thanks!!!

Steve

On Aug 18, 2008, at 1:13 AM, danil osipchuk wrote:

> Argh sorry, - just noticed the example with cmd curl invocation - as
> usual after the submitting of meaningless post...
>
> Danil
>

Danil Osipchuk-2

Re: [squeak-dev] Curl

Well, I was talking about the example with command line and after another look I've noticed that enabling of redirect did make a difference :)

but contents still has what *I* put in contents, as well as the response from the host. Should it? Seems to me that the "q=title:dodgers&fmt=xml&rows=50&fl=identifier&indent=&xmlsearch=Search" bit should not be in contents after the post

From the user perspective the appending of results to post query is ugly, but this is what libcurl does and it is the first time when I performed post with curl plugin. :)
My first rush would be to rewrite Curl>>#post, so that it caches query length and then cuts it off from the result, but this may be a just a bug in libcurl which has been fixed already. Need to check it with recent libcurl versions.

Thanks!!!

Steve

On Aug 18, 2008, at 1:13 AM, danil osipchuk wrote:

Argh sorry, - just noticed the example with cmd curl invocation - as usual after the submitting of meaningless post...

Danil

Steven W Riggins

Re: [squeak-dev] Curl

On Aug 18, 2008, at 1:45 AM, danil osipchuk wrote:

> From the user perspective the appending of results to post query is
> ugly, but this is what libcurl does and it is the first time when I
> performed post with curl plugin. :)
> My first rush would be to rewrite Curl>>#post, so that it caches
> query length and then cuts it off from the result, but this may be a
> just a bug in libcurl which has been fixed already. Need to check it
> with recent libcurl versions.

Yes i was concerned about hacking out the query as well.

If you can point me to how to build the plugin on the Mac with the
latest libcurl, I'd test it out.

I guess john could help me. I've built plugins before, but everytime
I go near vmmaker I get the hives.

Danil Osipchuk-2

Re: [squeak-dev] Curl

Yes i was concerned about hacking out the query as well.

If you can point me to how to build the plugin on the Mac with the latest libcurl, I'd test it out.

I guess john could help me. I've built plugins before, but everytime I go near vmmaker I get the hives.

I think the feature added in 7.17.1 is to be used:

 CURLOPT_COPYPOSTFIELDS
              Pass a char * as parameter, which should be  the  full  data  to
              post  in an HTTP POST operation. It behaves as the CURLOPT_POST-

              FIELDS option, but the original data are copied by the  library,
              allowing  the  application  to overwrite the original data after
              setting this option.

But to do so one also has to rewrite CurlPlugin side and this plugin is not easiest to build because of it's dependencies. I will update code on squeaksource (last version there compiles against 7.17 and the current version of libcurl is 7.18), meanwhile I suggest to hack the Curl>post to get desired behaviour.