How to do data merging
Locked
 
|
I'm using Magma in a project where there are users who have their own
offline Magma database for data collection. When these users have an internet connection, their data will be uploaded and "merged" with the data in the Magma database on the server. Both the server and the client are running Pharo. The user will pretty much be the master of their own subset of data, however the data could be changed on the server. My question is if there is already any functionality within Magma to perform some sort of "data merge"? I think I'll have to make the algorithm myself. Right now I'm thinking of making some sort of a "changeset" on each individual object in the database that tracks changed to specific ivars, and the merge would compare read/write timestamps and then fall back on examining the individual changes, and finally throw some error and allow the user to manually choose the winning data. I've seen, but never used, the MVCC and change conflicts that CouchDB does on a per-document basis and I'm kind of thinking in those terms. But I'd just like to know if there is already something out there that I may not know about that I can use. Any feedback would be appreciated! Thanks! -- Dave Woodward ................................................... www.promedmedical.net [hidden email] _______________________________________________ Magma mailing list [hidden email] http://lists.squeakfoundation.org/mailman/listinfo/magma |
|
|
Magma itself does not have such a data-merge function. Such a
function would require quite a bit of code and complexity for Magma, and invariably would not be exactly what different applications would want. Applications are therefore responsible for handling this. However, there are a couple of options to think about: - Magma supports long transactions. Instead of having separate databases, users could connect to the one master db. Objects that were cached in the image can be used and changed while "offline". Just be sure to save the image before exiting it. Then, when network-connection to Magma is back available, restart the image and commit. The problem is, you don't have access to all the objects while offline, just the cached ones. However, application meta-data is often not very big, and so a read-only copy of that repository, accessed via FowardingProxy, could be deployed locally to help mitigate this. These meta-data repositories could be easily kept up to date by the application simply applying log files from the read-write meta-data repository. - Check out Gjaller. As far as I know, Gjaller solved the exact problem you are talking about via a rich Command-pattern implementation. Cheers. On Wed, Feb 3, 2010 at 12:36 PM, Dave Woodward <[hidden email]> wrote: > I'm using Magma in a project where there are users who have their own > offline Magma database for data collection. > > When these users have an internet connection, their data will be uploaded > and "merged" with the data in the Magma database on the server. Both the > server and the client are running Pharo. > > The user will pretty much be the master of their own subset of data, however > the data could be changed on the server. > > My question is if there is already any functionality within Magma to perform > some sort of "data merge"? I think I'll have to make the algorithm myself. > Right now I'm thinking of making some sort of a "changeset" on each > individual object in the database that tracks changed to specific ivars, and > the merge would compare read/write timestamps and then fall back on > examining the individual changes, and finally throw some error and allow the > user to manually choose the winning data. > > I've seen, but never used, the MVCC and change conflicts that CouchDB does > on a per-document basis and I'm kind of thinking in those terms. > > But I'd just like to know if there is already something out there that I may > not know about that I can use. > > Any feedback would be appreciated! Thanks! > > -- > Dave Woodward > ................................................... > www.promedmedical.net > [hidden email] > > _______________________________________________ > Magma mailing list > [hidden email] > http://lists.squeakfoundation.org/mailman/listinfo/magma > Magma mailing list [hidden email] http://lists.squeakfoundation.org/mailman/listinfo/magma |
«
Return to Magma
|
1 view|%1 views
| Free forum by Nabble | Edit this page |