Hi Pharoers,
For a project I’m starting to prototype, I need collections where objects are inserted according to an index that is a spatiotemporal object. The idea is that each time a person add an « information" in this collection, he does it a a given place and time and I want to be able to query the collection easily according to place or time (or both). I’m hacking it right now but I wonder if people have ideas on that subject. I’m also interested in collection that that are indexed by the timestamp of the insertion. For the implementation of spatiotemporal classe, I choose to use a ZTimestamp and a Geolocation object. Geolocation is quite simple and is responsible among other to get the timezone, but of courses coordinates (or eventually areas). For the collection, I’m still wandering… Something like an OrderedCollection with another internal array to store associated spatiotemporal(s)… Will hack something but I’d love to hear others ideas. Cheers, Cédrik nb: I really think a spatiotemporal class would be interesting as a general data structure today ! |
A dictionary with keys being the timestamp or the spatiotemporal object would probably do it...
I think I’ll do that but I wonder if there are better solutions out there ^^. Cheers, Cédrik |
There was talk about an ordered dictionary implementation IIRC a couple of months ago on this list.
I’ve attached an implementation I found in a thread from 2010 (may not work out of the box). Cheers, Max > On 23 Apr 2016, at 13:46, Cédrick Béler <[hidden email]> wrote: > > A dictionary with keys being the timestamp or the spatiotemporal object would probably do it... > I think I’ll do that but I wonder if there are better solutions out there ^^. > > Cheers, > > Cédrik OrderedDictionary.st (11K) Download Attachment |
In reply to this post by cedreek
I don't have a data structures book handy, but how about 3 SortedCollections? (Assuming 3 dimensions: latitude, longitude & time). It would support a reasonably efficient binary search, and allow for areas and time ranges. Regardless of which solution you choose I agree that it would be a useful general class, so please package it separately. Cheers, On 23/04/2016 1:47 PM, "Cédrick Béler" <[hidden email]> wrote:
A dictionary with keys being the timestamp or the spatiotemporal object would probably do it... |
In reply to this post by Max Leske
Thanks Max,
I’ve seen in Pharo 5 OrderedDictionary, OrderedPreservingDictionary and some others. I guess they have the same intent. I was actually trying them right now :) Cheers, Cédrik > Le 23 avr. 2016 à 13:57, Max Leske <[hidden email]> a écrit : > > There was talk about an ordered dictionary implementation IIRC a couple of months ago on this list. > > I’ve attached an implementation I found in a thread from 2010 (may not work out of the box). > > Cheers, > Max > > > <OrderedDictionary.st> > >> On 23 Apr 2016, at 13:46, Cédrick Béler <[hidden email]> wrote: >> >> A dictionary with keys being the timestamp or the spatiotemporal object would probably do it... >> I think I’ll do that but I wonder if there are better solutions out there ^^. >> >> Cheers, >> >> Cédrik > |
In reply to this post by alistairgrant
Hi Alistair, Thanks, this sounds like a nice suggestion. I have to think about it. Adding altitude would be also interesting in some cases but not as a start (quite limited usage). When you say to package it separately, you’re talking about Geolocation, Spatiotemporal classes ? or also the collection ? I can try something but I think this will deserve more « computational thinking » power than I have ^^ Cheers, Cédrik
|
In reply to this post by cedreek
There are a lot of spatial access data structures: https://en.wikipedia.org/wiki/Spatial_database#Spatial_index But for a prototype I don't think you care; a simple linear search should be ok for testing until you have thousands and thousands of records. It's probably more important to really explore what you need and how your constraints will influence the API (e.g. since we haven't invented teleportation yet, events for one person that are close in time are necessarily close in space) On 23 April 2016 at 14:08, Cédrick Béler <[hidden email]> wrote: Thanks Max, |
In reply to this post by cedreek
On Sat, Apr 23, 2016 at 6:39 PM, Cédrick Béler <[hidden email]> wrote:
> Hi Pharoers, Hi Cedrick, welcome back in the community. > For a project I’m starting to prototype, I need collections where objects are inserted according to an index that is a spatiotemporal object. > > The idea is that each time a person add an « information" in this collection, he does it a a given place and time and I want to be able to query the collection easily according to place or time (or both). > > I’m hacking it right now but I wonder if people have ideas on that subject. I’m also interested in collection that that are indexed by the timestamp of the insertion. > > For the implementation of spatiotemporal classe, I choose to use a ZTimestamp and a Geolocation object. Geolocation is quite simple and is responsible among other to get the timezone, but of courses coordinates (or eventually areas). > > For the collection, I’m still wandering… Something like an OrderedCollection with another internal array to store associated spatiotemporal(s)… Will hack something but I’d love to hear others ideas. You mean something like a timeseries ? I start to work on a basic TimeSeries class quite similar to the one you find in Panda: http://pandas.pydata.org/pandas-docs/stable/timeseries.html -- Serge Stinckwich UCBN & UMI UMMISCO 209 (IRD/UPMC) Every DSL ends up being Smalltalk http://www.doesnotunderstand.org/ |
In reply to this post by Damien Pollet
Yes sure but I keep that in mind for later « exploration ».
Potentially, on a person lifetime, adding hundreds information a day (and probably higher order of magnitude), I think we can easily reach millions of data quite easily :) 100 per day, in 10 years => 365 000 1000 per day in 100 years => 36 500 000 What I want is keeping track of all positions a person can have in his life (what can easily reach millions) and then rely on these « points » to index information added in the system.
but I agree and I’ll work on that first.
yes… for now but in 50 years who knows :) This remark is nevertheless interesting as a premise that can be used to check the validity of data (geospatial resolver are not that precise). But yeah for now, an OrderedDic or another classic collection will do the job and will let me explore the API (and we might discuss it ^^). I start this discussion as I thing this is one central modeling concept of what I want to do. Cheers, Cédrik
|
In reply to this post by SergeStinckwich
Thanks, I was not very far though ^^
I don’t really know, more a geospatial timeserie but It might be interesting. We’ll exchange on that subject I guess. Cheers, Cédrik
|
In reply to this post by cedreek
On 23 April 2016 at 16:04, Cédrick Béler <[hidden email]> wrote:
Sure, but 1) that's not a lot 2) do you even know if you need interactive stuff or if batch processing is ok? |
interactive I would say… but hard to be certain right now as it depends on design choices… do we cache ? do we aggregate past data ? extra indexes ? I’ll see later. I want something simple, keep everything… and query according to time interval, areas (contexts)… But yes for now this is pointless to look for something optimal and I’ll use a dictionary as it will be small at start. Cheers, Cédrik |
Maybe instead of having a Dictionary, you can use my History package [1].
It allows you to add elements to "the history", and the History class will provide certain "lookup" strategies depending on how you want to query it. It is a trimmed down version of an utility class I used for years in commercial products where the History contained thousands of elements. This version uses linear search, my previous implementation used binary search which was significantly faster with lots of elements in the collection. Of course it doesn't solve your spatial queries needs, I don't know if there is something in Smalltalk for that other than the distance between two coordinates[2], PostgreSQL 9.2 has some support for "temporal" data/tables which you could combine with PostGIS if you need to combine a spatial query with a temporal one [3]. Regards, [1] http://www.smalltalkhub.com/#!/~emaringolo/History [2] http://pastebin.com/ThKDXCKK [3] http://clarkdave.net/2015/02/historical-records-with-postgresql-and-temporal-tables-and-sql-2011/ Esteban A. Maringolo 2016-04-23 12:13 GMT-03:00 Cédrick Béler <[hidden email]>: > > > Sure, but 1) that's not a lot > > 2) do you even know if you need interactive stuff or if batch processing is > ok? > > > interactive I would say… but hard to be certain right now as it depends on > design choices… do we cache ? do we aggregate past data ? extra indexes ? > I’ll see later. > I want something simple, keep everything… and query according to time > interval, areas (contexts)… > > But yes for now this is pointless to look for something optimal and I’ll use > a dictionary as it will be small at start. > > Cheers, > > Cédrik > > > > > -- > Damien Pollet > type less, do more [ | ] http://people.untyped.org/damien.pollet > > |
Hi Esteban,
Thanks a lot, I think I’ll try your History package, it seems to be a very good starting point (instead of hacking it myself). I’ll try it soon. Thanks for all the information ;) Cheers, Cédrik > Le 23 avr. 2016 à 18:28, Esteban A. Maringolo <[hidden email]> a écrit : > > Maybe instead of having a Dictionary, you can use my History package [1]. > It allows you to add elements to "the history", and the History class > will provide certain "lookup" strategies depending on how you want to > query it. > > It is a trimmed down version of an utility class I used for years in > commercial products where the History contained thousands of elements. > This version uses linear search, my previous implementation used > binary search which was significantly faster with lots of elements in > the collection. > > Of course it doesn't solve your spatial queries needs, I don't know if > there is something in Smalltalk for that other than the distance > between two coordinates[2], PostgreSQL 9.2 has some support for > "temporal" data/tables which you could combine with PostGIS if you > need to combine a spatial query with a temporal one [3]. > > Regards, > > [1] http://www.smalltalkhub.com/#!/~emaringolo/History > [2] http://pastebin.com/ThKDXCKK > [3] http://clarkdave.net/2015/02/historical-records-with-postgresql-and-temporal-tables-and-sql-2011/ > Esteban A. Maringolo > > > 2016-04-23 12:13 GMT-03:00 Cédrick Béler <[hidden email]>: >> >> >> Sure, but 1) that's not a lot >> >> 2) do you even know if you need interactive stuff or if batch processing is >> ok? >> >> >> interactive I would say… but hard to be certain right now as it depends on >> design choices… do we cache ? do we aggregate past data ? extra indexes ? >> I’ll see later. >> I want something simple, keep everything… and query according to time >> interval, areas (contexts)… >> >> But yes for now this is pointless to look for something optimal and I’ll use >> a dictionary as it will be small at start. >> >> Cheers, >> >> Cédrik >> >> >> >> >> -- >> Damien Pollet >> type less, do more [ | ] http://people.untyped.org/damien.pollet >> >> > |
In reply to this post by cedreek
On Sat, Apr 23, 2016 at 02:13:22PM +0200, Cédrick Béler wrote:
> Hi Alistair, > > Thanks, this sounds like a nice suggestion. > > I have to think about it. > Adding altitude would be also interesting in some cases but not as a start > (quite limited usage). > > When you say to package it separately, you’re talking about Geolocation, > Spatiotemporal classes ? or also the collection ? My impression was that the Geolocation & Spatiotemperal classes already exist (and presumably are already packaged), so I was thinking of the collection class (which would have the geolocation and spatiotemperal as dependencies). > I can try something but I think this will deserve more « computational > thinking » power than I have ^^ Me too, but having something is still better than nothing, and maybe someone will provide ideas for improvements. Cheers, Alistair |
In reply to this post by cedreek
Hi Cédrik,
Have you considered seeing if there are any Voronoi diagram packages in one of the repositories? I believe there is a fair bit of research into algorithms. You search use of spaciotemporal collated data sounds very interesting and there is a segment on computation over segments in this slide deck, that might be something, I don't know: http://graphics.stanford.edu/courses/cs368-06-spring/handouts/Delaunay_2.pdf. Certainly there is utility on the graphics side to use a 4D quadratic model (with GPU accel) and may well be modeled already in squeak/Pharo. Would't that be something! Regards, Robert On 04/23/2016 01:29 PM, Cédrick Béler wrote: > Hi Esteban, > > Thanks a lot, I think I’ll try your History package, it seems to be a very good starting point (instead of hacking it myself). > I’ll try it soon. > > Thanks for all the information ;) > > Cheers, > > Cédrik > > >> Le 23 avr. 2016 à 18:28, Esteban A. Maringolo <[hidden email]> a écrit : >> >> Maybe instead of having a Dictionary, you can use my History package [1]. >> It allows you to add elements to "the history", and the History class >> will provide certain "lookup" strategies depending on how you want to >> query it. >> >> It is a trimmed down version of an utility class I used for years in >> commercial products where the History contained thousands of elements. >> This version uses linear search, my previous implementation used >> binary search which was significantly faster with lots of elements in >> the collection. >> >> Of course it doesn't solve your spatial queries needs, I don't know if >> there is something in Smalltalk for that other than the distance >> between two coordinates[2], PostgreSQL 9.2 has some support for >> "temporal" data/tables which you could combine with PostGIS if you >> need to combine a spatial query with a temporal one [3]. >> >> Regards, >> >> [1] http://www.smalltalkhub.com/#!/~emaringolo/History >> [2] http://pastebin.com/ThKDXCKK >> [3] http://clarkdave.net/2015/02/historical-records-with-postgresql-and-temporal-tables-and-sql-2011/ >> Esteban A. Maringolo >> >> >> 2016-04-23 12:13 GMT-03:00 Cédrick Béler <[hidden email]>: >>> >>> Sure, but 1) that's not a lot >>> >>> 2) do you even know if you need interactive stuff or if batch processing is >>> ok? >>> >>> >>> interactive I would say… but hard to be certain right now as it depends on >>> design choices… do we cache ? do we aggregate past data ? extra indexes ? >>> I’ll see later. >>> I want something simple, keep everything… and query according to time >>> interval, areas (contexts)… >>> >>> But yes for now this is pointless to look for something optimal and I’ll use >>> a dictionary as it will be small at start. >>> >>> Cheers, >>> >>> Cédrik >>> >>> >>> >>> >>> -- >>> Damien Pollet >>> type less, do more [ | ] http://people.untyped.org/damien.pollet >>> >>> > -- Robert . .. ... ^,^ |
In reply to this post by alistairgrant
>
> My impression was that the Geolocation & Spatiotemperal classes already > exist (and presumably are already packaged), I don’t think so but if you know some packages, please tell me :) > so I was thinking of the > collection class (which would have the geolocation and spatiotemperal as > dependencies). > >> I can try something but I think this will deserve more « computational >> thinking » power than I have ^^ > > Me too, but having something is still better than nothing, and maybe > someone will provide ideas for improvements. > I’ll see what I’ll come up with and I’ll publish them. Cheers, Cédrik > > Cheers, > Alistair > > |
In reply to this post by Robert Withers
Hi Robert,
> > Have you considered seeing if there are any Voronoi diagram packages in one of the repositories? I believe there is a fair bit of research into algorithms. You search use of spaciotemporal collated data sounds very interesting and there is a segment on computation over segments in this slide deck, that might be something, I don't know: http://graphics.stanford.edu/courses/cs368-06-spring/handouts/Delaunay_2.pdf. Certainly there is utility on the graphics side to use a 4D quadratic model (with GPU accel) and may well be modeled already in squeak/Pharo. Would't that be something! Thanks for the links, very interesting read. I’ll have a closer look if I need to develop proper algorithms. Cheers, Cédrik > > > Regards, > Robert > > > On 04/23/2016 01:29 PM, Cédrick Béler wrote: >> Hi Esteban, >> >> Thanks a lot, I think I’ll try your History package, it seems to be a very good starting point (instead of hacking it myself). >> I’ll try it soon. >> >> Thanks for all the information ;) >> >> Cheers, >> >> Cédrik >> >> >>> Le 23 avr. 2016 à 18:28, Esteban A. Maringolo <[hidden email]> a écrit : >>> >>> Maybe instead of having a Dictionary, you can use my History package [1]. >>> It allows you to add elements to "the history", and the History class >>> will provide certain "lookup" strategies depending on how you want to >>> query it. >>> >>> It is a trimmed down version of an utility class I used for years in >>> commercial products where the History contained thousands of elements. >>> This version uses linear search, my previous implementation used >>> binary search which was significantly faster with lots of elements in >>> the collection. >>> >>> Of course it doesn't solve your spatial queries needs, I don't know if >>> there is something in Smalltalk for that other than the distance >>> between two coordinates[2], PostgreSQL 9.2 has some support for >>> "temporal" data/tables which you could combine with PostGIS if you >>> need to combine a spatial query with a temporal one [3]. >>> >>> Regards, >>> >>> [1] http://www.smalltalkhub.com/#!/~emaringolo/History >>> [2] http://pastebin.com/ThKDXCKK >>> [3] http://clarkdave.net/2015/02/historical-records-with-postgresql-and-temporal-tables-and-sql-2011/ >>> Esteban A. Maringolo >>> >>> >>> 2016-04-23 12:13 GMT-03:00 Cédrick Béler <[hidden email]>: >>>> >>>> Sure, but 1) that's not a lot >>>> >>>> 2) do you even know if you need interactive stuff or if batch processing is >>>> ok? >>>> >>>> >>>> interactive I would say… but hard to be certain right now as it depends on >>>> design choices… do we cache ? do we aggregate past data ? extra indexes ? >>>> I’ll see later. >>>> I want something simple, keep everything… and query according to time >>>> interval, areas (contexts)… >>>> >>>> But yes for now this is pointless to look for something optimal and I’ll use >>>> a dictionary as it will be small at start. >>>> >>>> Cheers, >>>> >>>> Cédrik >>>> >>>> >>>> >>>> >>>> -- >>>> Damien Pollet >>>> type less, do more [ | ] http://people.untyped.org/damien.pollet >>>> >>>> >> > > -- > Robert > . .. ... ^,^ > > |
In reply to this post by cedreek
2016-04-23 14:29 GMT-03:00 Cédrick Béler <[hidden email]>:
> Hi Esteban, > > Thanks a lot, I think I’ll try your History package, it seems to be a very good starting point (instead of hacking it myself). > I’ll try it soon. > > Thanks for all the information ;) Feel free to modify it if you want, as long as you preserve it's original output, of course :) Regards! Esteban A. Maringolo |
In reply to this post by cedreek
I agree with Alistair; a pattern like EventSourcing works well for building efficient query structures for a set of data along different as-yet-to-be-determined dimensions.
In Pharo terms, you can implement this using a EventStream that announces new entries, then install announcement handlers each updating a dimension-specific data structure. Also makes it easy to add new dimensions if you find you need them; simply add a new handler and replay the EventStream (with the other handlers temporarily disabled). Cheers, Henry
signature.asc (859 bytes) Download Attachment |
Free forum by Nabble | Edit this page |