Hi,
I have several Urls, each giving me a piece of information.
I want to write some code that gathers these pieces of information and make a display of the whole situation.
The principle is: I send a request to one URL then parse the result and make a new request to another URL using the parsed result.
I need some code that somehow mimic a human behaviour: it should
1) go to a URL and get the content (easy part)
2) go to a URL and retrieve some session ID then make a more elaborated request (POST, GET, or even trigger some Javascript)
Stephan
_______________________________________________ Beginners mailing list [hidden email] http://lists.squeakfoundation.org/mailman/listinfo/beginners |
Start here:
Good intro to Squeak Networking is the Networking Squeak chapter from the Squeak NuBlue book: http://stephane.ducasse.free.fr/FreeBooks/CollectiveNBlueBook/parsia%202.pdf And then I would check out Albatross, which is a Squeak framework for testing websites (including exercising Javascript). Here is a PDF intro: scg.unibe.ch/archive/projects/Brue06a.pdf Google Squeak Albatross for more info. --David Mitchell blog: http://www.withaguide.com On Tue, Dec 22, 2009 at 8:04 AM, DUFOUR, Stephan (ext.) <[hidden email]> wrote: > Hi, > > I have several Urls, each giving me a piece of information. > I want to write some code that gathers these pieces of information and make > a display of the whole situation. > > The principle is: I send a request to one URL then parse the result and make > a new request to another URL using the parsed result. > > I need some code that somehow mimic a human behaviour: it should > 1) go to a URL and get the content (easy part) > 2) go to a URL and retrieve some session ID then make a more > elaborated request (POST, GET, or even trigger some Javascript) > > Stephan > > > > _______________________________________________ > Beginners mailing list > [hidden email] > http://lists.squeakfoundation.org/mailman/listinfo/beginners > > Beginners mailing list [hidden email] http://lists.squeakfoundation.org/mailman/listinfo/beginners |
Hi,
you can also have a look at squeak soup [1] which is a tolerant HTML/XML parser (Squeak port of Beautiful Soup). The code repository is on squeaksource [2]. hth [1] http://n4.nabble.com/squeak-dev-ANN-Soup-0-1-tt80191.html#a80191 [2] http://www.squeaksource.com/Soup.html 2009/12/22 David Mitchell <[hidden email]> > > Start here: > > Good intro to Squeak Networking is the Networking Squeak chapter from > the Squeak NuBlue > book: > http://stephane.ducasse.free.fr/FreeBooks/CollectiveNBlueBook/parsia%202.pdf > > And then I would check out Albatross, which is a Squeak framework for > testing websites (including exercising Javascript). Here is a PDF > intro: > scg.unibe.ch/archive/projects/Brue06a.pdf > > Google Squeak Albatross for more info. > > --David Mitchell > blog: http://www.withaguide.com > > On Tue, Dec 22, 2009 at 8:04 AM, DUFOUR, Stephan (ext.) > <[hidden email]> wrote: > > Hi, > > > > I have several Urls, each giving me a piece of information. > > I want to write some code that gathers these pieces of information and make > > a display of the whole situation. > > > > The principle is: I send a request to one URL then parse the result and make > > a new request to another URL using the parsed result. > > > > I need some code that somehow mimic a human behaviour: it should > > 1) go to a URL and get the content (easy part) > > 2) go to a URL and retrieve some session ID then make a more > > elaborated request (POST, GET, or even trigger some Javascript) > > > > Stephan > > > > > > > > _______________________________________________ > > Beginners mailing list > > [hidden email] > > http://lists.squeakfoundation.org/mailman/listinfo/beginners > > > > > _______________________________________________ > Beginners mailing list > [hidden email] > http://lists.squeakfoundation.org/mailman/listinfo/beginners -- Cédrick _______________________________________________ Beginners mailing list [hidden email] http://lists.squeakfoundation.org/mailman/listinfo/beginners |
Hi, I had little time to look
at the stuff but I have a global idea of what does what. Albatross is running via
SUnit on localhost. SUnit creates a component,
this component is registered as a What I want is simpler
and different, I want to call external URLs. I wonder if I can replace
SUnit by a simple object and run ABrower with external URLs. Some functions seems to
require The two other tools: -
Selenium / Squelenium / WebTester: The firefox module allows you to
easily write some tests but it cannot export code to Smalltalk. It also seems
to be limited at finding predefined text, maybe the API allows you to search
for variable content or you have to use something else. i.e. for example you
have a customer account, you enter login/password, enter the customer account,
it can check that you have NAME and SURNAME in the resultpage for exemple, but
it won’t give you what the name and the surname are (maybe I simply
didn’t dig enough). -
Beautiful Soup can parse the result of a page and be a good
complement for Selenium. At the moment, I’d
like to focus on Albatross because if I can use all the functions, it’ll
be easy to code. Stephan De : Cédrick Béler
[mailto:[hidden email]] Hi, _______________________________________________ Beginners mailing list [hidden email] http://lists.squeakfoundation.org/mailman/listinfo/beginners |
In reply to this post by DUFOUR, Stephan (ext.)
not sure I understand correctly but to me you don't need albatross.
With Soup:
soupTag := Soup fromUrl: 'yourUrl.html'.
otherTag := (soup findTagByClass: 'whateverClassNameWhereTheSessionIdBelong'). otherUrlString := (messageSoupTag childAt: 3) attributeAt: 'href'. "if the url is in the 3rd child of otherTag"
secondSoupTag := Soup fromUrl: otherUrl. "or convert as a true URL object that you can change" url := otherUrlString asUrl. etc... hth, Cédrick
_______________________________________________ Beginners mailing list [hidden email] http://lists.squeakfoundation.org/mailman/listinfo/beginners |
In reply to this post by DUFOUR, Stephan (ext.)
Hi!
DUFOUR, Stephan (ext.) wrote: > Hi, > > I have several Urls, each giving me a piece of information. > I want to write some code that gathers these pieces of information and > make a display of the whole situation. > > The principle is: I send a request to one URL then parse the result and > make a new request to another URL using the parsed result. > > I need some code that somehow mimic a human behaviour: it should > 1) go to a URL and get the content (easy part) > 2) go to a URL and retrieve some session ID then make a more > elaborated request (POST, GET, or even trigger some Javascript) I have written such a "robot" for a customer, and I need to update it in the coming days, I will see if I can publish the reusable part of it. regards, Göran _______________________________________________ Beginners mailing list [hidden email] http://lists.squeakfoundation.org/mailman/listinfo/beginners |
In reply to this post by DUFOUR, Stephan (ext.)
At 4:00 AM -0800 1/7/10, [hidden email] apparently wrote:
>Date: Thu, 07 Jan 2010 11:13:48 +0100 >From: G?ran Krampe <[hidden email]> >Subject: Re: [Newbies] Making requests to an HTML server >To: "A friendly place to get answers to even the most basic questions > about Squeak." <[hidden email]> >Message-ID: <[hidden email]> >Content-Type: text/plain; charset=ISO-8859-1; format=flowed > >Hi! > >DUFOUR, Stephan (ext.) wrote: >> Hi, >> >> I have several Urls, each giving me a piece of information. >> I want to write some code that gathers these pieces of information and >> make a display of the whole situation. >> >> The principle is: I send a request to one URL then parse the result and >> make a new request to another URL using the parsed result. >> >> I need some code that somehow mimic a human behaviour: it should >> 1) go to a URL and get the content (easy part) >> 2) go to a URL and retrieve some session ID then make a more >> elaborated request (POST, GET, or even trigger some Javascript) > >I have written such a "robot" for a customer, and I need to update it in >the coming days, I will see if I can publish the reusable part of it. > >regards, Gran I am very interested in this. Go Gran! In my case I wish to start with some numbers parsed from the first url, use each of them to create a search at a second pre-known url, then operate on the results of the search to perhaps figure out if there is a pdf download link available. Thx, Ken G. Brown _______________________________________________ Beginners mailing list [hidden email] http://lists.squeakfoundation.org/mailman/listinfo/beginners |
Free forum by Nabble | Edit this page |