Report about the D4D datathon challenge

Dear all,

this is a small report about the RESILIENCE team using Pharo&Roassal
for the D4D datathon challenge: http://www.d4d.orange.com/en/home

The idea was to use mobile network data from Senegal provided by
Orange company in order to solve some development problems.

The amount of data provided by Orange was really huge, something like
66 Gb: we have the number and the durations of calls between each pairs of GSM
antennas in Senegal (1666 different antennas) and also some
information about the mobility of people. You can find more
information about the data here:
http://arxiv.org/abs/1407.4885

We try to extract some data from August 2013, because we know that
some floods occurs at that time in Dakar. Our main objective was to to
be able to visualise mobility and communications patterns with ROASSAL
to see if there is any differences before and after the floods.

We had a lot of problems to solve in a short period of time :-)

At the beginning, we couldn't open huge files with Pharo. Only open a
file that is around 3.4Go is impossible. Pharo says that the file does
not exist.
Thierry tells us that us that one must compile the VM with
-D_FILE_OFFSET_BITS=64. (the Cog VMs are also built with
-D_GNU_SOURCE).
Why this is not done by default ?

After that we decide to split the files in many smaller files and we
try to use a mySQL database and various other tools but still have
many troubles because of the size of the data ... At the end, we have
done some visualisations but only on a tiny portion of the data on one
day (one day is around 100Mb of data). You could find some
visualisations done by our team here:
https://twitter.com/sergestinckwich/status/586178094215606273

Managing huge amount in Pharo and also ROASSAL is still a challenge
and we definitively need more tools in this area.

We had some problems with ROASSAL: bugs with RTEdgesBuilder (Alvaro
can you send a report to Alex about that ?), to much time to process
some viz, ...

At the end, our team receive the honorary price:
https://twitter.com/sergestinckwich/status/585866625745887232

This is a brief description of our project.

=========================================================
Project name: RESILIENCE
Team members: Clémence Douard (ENSCI), Onil Goubier (CIRELA), Alvaro
Peralta (LabU / Chile University), Aurélie Thouron (ENSCI), Serge Stinckwich
(IRD / UMMISCO)

The main concern of this project is to improve resilience after an
environmental crisis in a southern city. During the preparedness phase
before crisis, records of inhabitants habits (like patterns of
mobility or communications) are done in order to characterize the
situation before the crisis. Assessments of the improvements will then
be done by providing suitable visualizations of  the resilience
dynamic. The goal is to help citizens to represent these improvements
and to take advantage of them in their daily lives.

We took as an example the city of Dakar during the flooding that
occurred in August 2013.

The data used are those of the dataset 1 (communication between
antennas) and 2 (user mobility) provided Orange. We use this
information, coupled to OSM, and other data related to floods of
August in Dakar to construct maps and indicators to show resilience.
Visualizations are made with agile visualization platform Roassal:
http://agilevisualization.com/
=========================================================

I would like to thank all the team members and also the people who
help us remotely: Alexandre Bergel (ObjectProfile/Chile University)
and Thierry Goubier (CEA)

I would like to thank also the SIMPLON team and ORANGE for organizing
such a nice event.

Regards,
--
Serge Stinckwich
UCBN & UMI UMMISCO 209 (IRD/UPMC)
Every DSL ends up being Smalltalk
http://www.doesnotunderstand.org/

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

Thanks Serge for sharing with us. I think that we need more projects
about "visualization for understanding" complex phenomena, like this
one. I hope to share something about political discourse on social
networks (but I promise not to ask about going beyond defaults of
Twitters ToS on this list again :-P).

Talking about visualization for understanding, I think that this
video/talk gets to the point "Media for Thinking the Unthinkable":

https://vimeo.com/67076984

For my, Pharo/Moose/Roassal are becoming my media for thinking the
unthinkable.

Cheers,

Offray

El 09/04/15 a las 09:52, Serge Stinckwich escribió:

_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev