Posted by Offray on Apr 07, 2015; 6:05pm
URL: https://forum.world.st/ZnClient-getting-more-that-19-tweet-for-data-scrapping-tp4818162.html

Hi,

Recently Paul DeBruicker taught me how to refine my code for getting
tweets properly. Consider this:

=[1]====================================
| source anUrl tweet |

anUrl := 'https://twitter.com/offrayLC'.
source := Soup fromString: (ZnEasy get: anUrl ) contents asString.
tweets := (source findAllTagsByClass: 'ProfileTweet-text') collect:[:ea
| ea text].
========================================

Is working fine, but I would like to get more that 19 tweets, that is
what you get by default. There is any way to tell ZnEasy and friends to
get more tweets, something similar to what you do when you scroll down
into a twitter page?

And by the way, I would like to make more sense of the Soup I got in the
last line. ea text gives me the tweet contents, but how can I interpret
the metadata in the soup? (is a retweet, date of publishing and so on).
I could make this for most part of the twitter profile page, but the
tweet is kind of elusive, for example how to know that "text" is the
proper message for getting the tweet content? Any pointer to how to make
sense of it by myself is greatly appreciated.

Cheers,

Offray