Re: [Pharo-business] [ANN] DBPedia: Query Wikipedia from Pharo

A followup from the previous post. Fetching country population and charting them using GraphET:

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
| query data diagram |

query := DBPediaSearch new 
setJsonFormat;
setDebugOn;
timeout: 3000;
query: 'SELECT DISTINCT ?name ?population 
WHERE {
?country a dbpedia-owl:Country . 
?country rdfs:label ?name . 
FILTER 
(langMatches(lang(?name), "en")) 
values ?hasPopulation { dbpprop:populationEstimatedbpprop:populationCensus } 
OPTIONAL { ?country ?hasPopulation ?population } 
FILTER (isNumeric(?population)) 
FILTER NOT EXISTS { ?country dbpedia-owl:dissolutionYear ?yearEnd } { ?country dbpprop:iso3166code ?code . } 
UNION { ?country dbpprop:iso31661Alpha ?code . } 
UNION { ?country dbpprop:countryCode ?code . } 
UNION { ?country a yago:MemberStatesOfTheUnitedNations . }}';
execute.

data := (((NeoJSONReader fromString: query) at:#results) at: #bindings) collect: [ :entry | Array with: ( (entry at: #name) at: #value ) with: ( (entry at: #population) at: #value ) asInteger ].

"Use GraphET to render all this"
diagram := GETDiagramBuilder new.
diagram verticalBarDiagram
models: (data reverseSortedAs: #second);
y: #second;
regularAxisAsInteger;
titleLabel: 'Size of countries';
yAxisLabel: 'Population'.
diagram interaction popupText.

diagram open
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Cheers,

Alexandre

On Mar 2, 2014, at 9:22 PM, Alexandre Bergel <[hidden email]> wrote:

I’ve just tried and it works pretty well! Impressive!

Below I describe a small example that fetches some data about the US Universities from DBPedia and visualize them using Roassal2. 

Pick a fresh 3.0 image.

First, you need to load Hernán work, Sven’s NeoJSON, and Roassal 2 (If you are using a Moose Image, there is no need to load Roassal2 since it is already in):
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Gofer it
  smalltalkhubUser: 'SvenVanCaekenberghe' project:  'Neo';
  package:  'ConfigurationOfNeoJSON';
  load.
((Smalltalk at: #ConfigurationOfNeoJSON) load).

Gofer it
  smalltalkhubUser: 'hernan' project: 'DBPedia';
  package: 'DBPedia';
  load.

Gofer it
  smalltalkhubUser: 'ObjectProfile' project:  'Roassal2';
  package:  'ConfigurationOfRoassal2';
  load.
((Smalltalk at: #ConfigurationOfRoassal2) loadBleedingEdge).

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Using Roassal2, I was able to render some data extracted from dbpedia:

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
| map locations rawData rawData2 rawData3 |
map := RTMapBuilder new.

map countries: #('UnitedStates' 'Canada' 'Mexico').
map color: Color veryVeryLightGray.

rawData := DBPediaSearch universitiesInUS.
rawData2 := ((NeoJSONReader fromString: rawData) at: #results) at: #bindings.
rawData3 := rawData2 select: [ :d | d keys includesAll: #('label' 'long' 'lat') ] thenCollect: [ :d | { (Float readFrom: ((d at: 'long') at: 'value')) . (Float readFrom: ((d at: 'lat') at: 'value')) . (d at: 'label' ) at: 'value' } ].


locations := rawData3.
locations do: [ :array |
map cities addCityNamed: array third location: array second @ array first ].
map cities shape size: 8; color: (Color blue alpha: 0.03).
map cities: (locations collect: #third).

map scale: 2.

map render.
map view openInWindowSized: 1000 @ 500.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

This is what you get:

<Screen Shot 2014-03-02 at 9.09.57 PM.png>

This is a small example. Naturally, adding popup for locations is trivial to add.

I have described this on our Facebook page:
https://www.facebook.com/ObjectProfile/photos/a.341189379300999.82969.340543479365589/596623173757617/?type=1&theater

Hernán, since SPARQL is a bit obscure, it would be great if you could add some more example, and also, how to parametrize the examples. For example, now we can get data for the US, how to modify your example to get them for France or Chile? 

Cheers,
Alexandre


On Mar 2, 2014, at 3:43 PM, Hernán Morales Durand <[hidden email]> wrote:

I have uploaded a new configuration so you can query the english Wikipedia dataset from Pharo 3 using SPARQL. Some examples follow:

1) Retrieve in JSON movies from the beautiful Julianne Moore:

| jsonResults |
jsonResults := DBPediaSearch new 
        setJsonFormat;
        timeout: 5000;
        query: 'SELECT DISTINCT ?filmName WHERE {
  ?film foaf:name ?filmName .
  ?film dbpedia-owl:starring ?actress .
  ?actress foaf:name ?name.
  FILTER(contains(?name, "Julianne"))
  FILTER(contains(?name, "Moore"))
}';
        execute

To actually get only the titles using NeoJSON:

((((NeoJSONReader fromString: jsonResults) at: #results) at: #bindings) 
    collect: [ : entry | entry at: #filmName ]) collect: [ : movie | movie at: #value ]


2) Retrieve in XML which genre plays those crazy Dream Theater guys  :

DBPediaSearch new 
        setXmlFormat;
        setDebugOn;
        timeout: 5000;
        query: 'SELECT DISTINCT ?genreLabel
WHERE {
    ?resource dbpprop:genre ?genre.
    ?resource rdfs:label "Dream Theater"@en.
    ?genre rdfs:label ?genreLabel
    FILTER (lang(?genreLabel)="en")
}
 LIMIT 100';
        execute

More examples are available in DBPediaSearch class side. You can install it from the Configuration Browser.
If you want to contribute, just ask me and you will be added as contributor.
Best regards,

Hernán

_______________________________________________
Pharo-business mailing list
[hidden email]
http://lists.pharo.org/mailman/listinfo/pharo-business_lists.pharo.org

-- 
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel  http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

-- 
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel  http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

Excellent Hernán!

There are more Universities and Colleges in France than in Spain or Italy.

-=-=-=-=-=-=-=-=-=-=
| map locations rawData rawData2 rawData3 |
map := RTMapBuilder new.

map countries: #('France' 'Spain' 'Italy').
map color: Color veryVeryLightGray.

rawData := DBPediaSearch universitiesIn: 'France'.

rawData2 := ((NeoJSONReader fromString: rawData) at: #results) at: #bindings.
rawData3 := rawData2 select: [ :d | d keys includesAll: #('label' 'long' 'lat') ] thenCollect: [ :d | { (Float readFrom: ((d at: 'long') at: 'value')) . (Float readFrom: ((d at: 'lat') at: 'value')) . (d at: 'label' ) at: 'value' } ].
locations := rawData3.
locations do: [ :array |
map cities addCityNamed: array third location: array second @ array first ].
map cities shape size: 8; color: (Color blue alpha: 0.03).
map cities: (locations collect: #third).

rawData := DBPediaSearch universitiesIn: 'Italy'.

rawData2 := ((NeoJSONReader fromString: rawData) at: #results) at: #bindings.
rawData3 := rawData2 select: [ :d | d keys includesAll: #('label' 'long' 'lat') ] thenCollect: [ :d | { (Float readFrom: ((d at: 'long') at: 'value')) . (Float readFrom: ((d at: 'lat') at: 'value')) . (d at: 'label' ) at: 'value' } ].
locations := rawData3.
locations do: [ :array |
map cities addCityNamed: array third location: array second @ array first ].
map cities shape size: 8; color: (Color blue alpha: 0.03).
map cities: (locations collect: #third).

rawData := DBPediaSearch universitiesIn: 'Spain'.

rawData2 := ((NeoJSONReader fromString: rawData) at: #results) at: #bindings.
rawData3 := rawData2 select: [ :d | d keys includesAll: #('label' 'long' 'lat') ] thenCollect: [ :d | { (Float readFrom: ((d at: 'long') at: 'value')) . (Float readFrom: ((d at: 'lat') at: 'value')) . (d at: 'label' ) at: 'value' } ].
locations := rawData3.
locations do: [ :array |
map cities addCityNamed: array third location: array second @ array first ].
map cities shape size: 8; color: (Color blue alpha: 0.03).
map cities: (locations collect: #third).


map scale: 5.

map render.
map view openInWindowSized: 1000 @ 500.
-=-=-=-=-=-=-=-=-=-=

On Mar 3, 2014, at 2:43 AM, Hernán Morales Durand <[hidden email]> wrote:

2014-03-02 21:22 GMT-03:00 Alexandre Bergel <[hidden email]>:
I’ve just tried and it works pretty well! Impressive!

Below I describe a small example that fetches some data about the US Universities from DBPedia and visualize them using Roassal2. 

Pick a fresh 3.0 image.

First, you need to load Hernán work, Sven’s NeoJSON, and Roassal 2 (If you are using a Moose Image, there is no need to load Roassal2 since it is already in):
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Gofer it
  smalltalkhubUser: 'SvenVanCaekenberghe' project:  'Neo';
  package:  'ConfigurationOfNeoJSON';
  load.
((Smalltalk at: #ConfigurationOfNeoJSON) load).

Gofer it
  smalltalkhubUser: 'hernan' project: 'DBPedia';
  package: 'DBPedia';
  load.

Gofer it
  smalltalkhubUser: 'ObjectProfile' project:  'Roassal2';
  package:  'ConfigurationOfRoassal2';
  load.
((Smalltalk at: #ConfigurationOfRoassal2) loadBleedingEdge).

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Using Roassal2, I was able to render some data extracted from dbpedia:

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
| map locations rawData rawData2 rawData3 |
map := RTMapBuilder new.

map countries: #('UnitedStates' 'Canada' 'Mexico').
map color: Color veryVeryLightGray.

rawData := DBPediaSearch universitiesInUS.
rawData2 := ((NeoJSONReader fromString: rawData) at: #results) at: #bindings.
rawData3 := rawData2 select: [ :d | d keys includesAll: #('label' 'long' 'lat') ] thenCollect: [ :d | { (Float readFrom: ((d at: 'long') at: 'value')) . (Float readFrom: ((d at: 'lat') at: 'value')) . (d at: 'label' ) at: 'value' } ].


locations := rawData3.
locations do: [ :array |
map cities addCityNamed: array third location: array second @ array first ].
map cities shape size: 8; color: (Color blue alpha: 0.03).
map cities: (locations collect: #third).

map scale: 2.

map render.
map view openInWindowSized: 1000 @ 500.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

This is what you get:

<Screen Shot 2014-03-02 at 9.09.57 PM.png>

This is a small example. Naturally, adding popup for locations is trivial to add.

I have described this on our Facebook page:
https://www.facebook.com/ObjectProfile/photos/a.341189379300999.82969.340543479365589/596623173757617/?type=1&theater


Super cool!! Thanks for sharing the nice mapping.

Hernán, since SPARQL is a bit obscure,

Absolutely, SPARQL is like the Assembler of the web.
 
it would be great if you could add some more example, and also, how to parametrize the examples. For example, now we can get data for the US, how to modify your example to get them for France or Chile? 


Ok, uploaded an updated version. I have parametrized the query triplets as #universitiesIn: englishCountryName, 

DBPediaSearch universitiesIn: 'France'.
DBPediaSearch universitiesIn: 'Chile'.

I have to admit I am still learning SPARQL, but the more queries I execute, the more I discover linked data structure, so I will add convenience methods for easy parsing results.

Cheers,

Hernán

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

-- 
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel  http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.