Exporting data from an end of life GS64 system to a relational DB?
Locked
Exporting data from an end of life GS64 system to a relational DB?
 
|
The GemStone system I have been working on the past 11 years has sadly been shelved after a takeover, and I have been tasked with warehousing the data in it to a MS SQL database.
It seems clear to me that this task is not a trivial task, and I hope some members of this list have pointers or advice. In essence, I need something like an ORM mapper in reverse. Not that I really expect anything like this to exist as an off-the-shelf or Open Source solution - how many people could really want it? From looking at the docs, GemConnect does not seem like a good fit. It seems more concerned with reflecting changes made to a relational database into GemStone than the other way around. It does provide a Smalltalk SQL client that could be useful, but beyond that all the mapping facilities it provides are about instantiating new objects from rows in tables, rather than adding rows to tables based on pre-existing objects. I considered whether something like GLORP could be jury-rigged into serving this need. But I can't find a GemStone port of GLORP, and I'm not surprised, because who'd want to do the effort to port something that aims to create "the GemStone illusion" when they already have the GemStone reality? Further, I'm not at all sure whether the data model GLORP supports is generic enough to feasibly tackle an application that has evolved with 11 years without being constrained to follow GLORP's data model. And then there's the minor issue that GLORP seems to want to write dirty objects on its' transaction boundaries, so I'd need to trick it into thinking that the past 11 years of data are one huge transaction to write to SQL all at once. I'm beginning to suspect that this is something I'll need to code myself, the hard way. Or is there any relevant prior art out there? ___________________________
Pieter Nagel Lead Developer PSG Asset Management Tel: 011 772 2300 Fax: 011 388 3223 1st Floor 28 Fricker Road Illovo, 2196 South Africa [hidden email] E-Mail disclaimer can be viewed at PSG Disclaimer • Should you have any problems viewing the disclaimer please e-mail emaildisclaimer@.... _______________________________________________ GemStone-Smalltalk mailing list Archive: http://forum.world.st/Gemstone-Customers-f1461796.html |
Re: Exporting data from an end of life GS64 system to a relational DB?
|
|
> From looking at the docs, GemConnect does not seem like a good fit. It
seems more concerned with reflecting changes made to a > relational database into GemStone than the other way around. Not true. There are capabilities in GemConnect that allow you to "write" a GS Object to an SQL table, once you've established a mapping between the object's instvars and the fields in the table. The tricky part is handling more complex mapping requirements like 1-to-many and many-to-many mappings. At one time we had developed a tool that did that for taking relational data and porting it into GemStone, but we never completed the task for going in the other direction... -- ------------------------------------------------------------------------ Bill Erickson VMware/SpringSource/GemStone/Smalltalk Engineering Software Engineer 15220 NW Greenbrier Parkway #150 [hidden email] Beaverton OR 97006 ------------------------------------------------------------------------ _______________________________________________ GemStone-Smalltalk mailing list Archive: http://forum.world.st/Gemstone-Customers-f1461796.html |
Re: Exporting data from an end of life GS64 system to a relational DB?
|
|
Of course, the fact that GemConnect doesn't support MS SQL is probably a bigger show-stopper in using it to populate your MS SQL DB ;-) From: "William Erickson" <[hidden email]> -- ------------------------------------------------------------------------ Bill Erickson VMware/SpringSource/GemStone/Smalltalk Engineering Software Engineer 15220 NW Greenbrier Parkway #150 Ph: (503)533-3439 [hidden email] Beaverton OR 97006 Fx: (503)533-3220 ------------------------------------------------------------------------ _______________________________________________ GemStone-Smalltalk mailing list Archive: http://forum.world.st/Gemstone-Customers-f1461796.html |
Re: Exporting data from an end of life GS64 system to a relational DB?
|
|
In reply to this post by Bill Erickson
I think if I had to do this, I would dump my major business objects
to some form of ascii-loadable
files and go from there. The advantage is that you can create whatever you want, likely fairly quickly AND that you can always edit the result if need-be. Of course a lot depends on how you business objects are structured and how well that can map to a relational model -- but if its a "business" there must be a reasonable mapping at some level I would think. We have used sql-ascii dump format and .csv to get data out to other people with those funny database things. On 2012-03-15 12:55 PM, William Erickson wrote:
-- Dennis Smith +1 416.798.7948 Cherniak Software Development Corporation Fax: +1 416.798.0948 509-2001 Sheppard Avenue East [hidden email] Toronto, ON M2J 4Z8 [hidden email] Canada http://www.CherniakSoftware.com Entrance off Yorkland Blvd south of Sheppard Ave east of the DVP _______________________________________________ GemStone-Smalltalk mailing list Archive: http://forum.world.st/Gemstone-Customers-f1461796.html |
Re: Exporting data from an end of life GS64 system to a relational DB?
|
|
In reply to this post by Bill Erickson
Thanks for the heads up. I will definitely give GemConnect a second look. The fact that GemConnect does not support MS SQL is not such a big issue - if it helps enough in other
ways, I could always first export to one relational database and then convert that to MS SQL. I consider this a much more trivial problem than mapping our object model to tables.
A bigger problem is the fact that GemConnect is a paid-for product, and the business would be loathe to buy extra software for a system which they have already killed. I see the Sybase version in open-source though - but also seems to date from 2000. So it won't work on GS64 - unless the datestamp is wrong? ___________________________
Pieter Nagel Lead Developer PSG Asset Management Tel: 011 772 2300 Fax: 011 388 3223 1st Floor 28 Fricker Road Illovo, 2196 South Africa [hidden email] E-Mail disclaimer can be viewed at PSG Disclaimer • Should you have any problems viewing the disclaimer please e-mail emaildisclaimer@.... _______________________________________________ GemStone-Smalltalk mailing list Archive: http://forum.world.st/Gemstone-Customers-f1461796.html |
Re: Exporting data from an end of life GS64 system to a relational DB?
|
|
Pieter, ... but the PostgreSQL drivers I mentioned to you are free to use :-) Al the best, On Mar 16, 2012 11:51 AM, "Pieter Nagel" <[hidden email]> wrote:
_______________________________________________ GemStone-Smalltalk mailing list Archive: http://forum.world.st/Gemstone-Customers-f1461796.html |
Re: Exporting data from an end of life GS64 system to a relational DB?
|
|
In reply to this post by Dennis smith-4
Dennis,
I don't think dumping the objects to text file is a real option. The business wants the resulting archive of the system objects to be queryable, specifically by DBAs that are used to MS SQL. But I am not sure how much using text files would simplify the problem, if at all. To my mind, hand-crafting code so each an every object knows how to write itself to file is roughly the same order of complexity as hand-crafting code so each and every object knows to INSERT itself into a table or two. The real challenge for me lies in how to exploit the regularities in the problem space in order to avoid hand-crafting code for each class, and this implies some sort of framework that supports the major kinds of mappings that we will need to do. And ideally, this framework would be off-the-shelf, a reverse ORM mapper so to speak. From what I read So far GemConnect, seems to fit the bill best. ___________________________
Pieter Nagel Lead Developer PSG Asset Management Tel: 011 772 2300 Fax: 011 388 3223 1st Floor 28 Fricker Road Illovo, 2196 South Africa [hidden email] E-Mail disclaimer can be viewed at PSG Disclaimer • Should you have any problems viewing the disclaimer please e-mail emaildisclaimer@.... _______________________________________________ GemStone-Smalltalk mailing list Archive: http://forum.world.st/Gemstone-Customers-f1461796.html |
Re: Exporting data from an end of life GS64 system to a relational DB?
|
|
When I suggested dumping to some for of ascii, I was intending a
form that can be then uploaded into an RDBMS.
Most support a number of forms for uploading. On 2012-03-16 08:18, Pieter Nagel wrote:
-- Dennis Smith +1 416.798.7948 Cherniak Software Development Corporation Fax: +1 416.798.0948 509-2001 Sheppard Avenue East [hidden email] Toronto, ON M2J 4Z8 [hidden email] Canada http://www.CherniakSoftware.com Entrance off Yorkland Blvd south of Sheppard Ave east of the DVP _______________________________________________ GemStone-Smalltalk mailing list Archive: http://forum.world.st/Gemstone-Customers-f1461796.html |
Re: Exporting data from an end of life GS64 system to a relational DB?
|
|
In reply to this post by Pieter Nagel-2
Hi Pieter,
We converted GC Sybase to 64 bits a couple of years ago. You can find it here: http://sourceforge.net/projects/gemconnsyb1ora2/ Regards, Dag Willy Findal-Fossmo Computas AS, Norway On 03/16/2012 12:50 PM, Pieter Nagel wrote:
_______________________________________________ GemStone-Smalltalk mailing list Archive: http://forum.world.st/Gemstone-Customers-f1461796.html |
Re: Exporting data from an end of life GS64 system to a relational DB?
|
|
Dag,
Thanks for the link. According to the FreeTDS Open-source client for connecting to MS SQL, MS SQL and Sybase have some similarity in their network protocol. So perhaps adapting the sybase code to use FreeTDS will not be too difficult, thereby allowing me to accessing MS SQL from GemConnect in GS/64. ___________________________
Pieter Nagel Lead Developer PSG Asset Management Tel: 011 772 2300 Fax: 011 388 3223 1st Floor 28 Fricker Road Illovo, 2196 South Africa [hidden email] E-Mail disclaimer can be viewed at PSG Disclaimer • Should you have any problems viewing the disclaimer please e-mail emaildisclaimer@.... _______________________________________________ GemStone-Smalltalk mailing list Archive: http://forum.world.st/Gemstone-Customers-f1461796.html |
Re: Exporting data from an end of life GS64 system to a relational DB?
|
|
In reply to this post by Dennis smith-4
Pieter,
A few years ago, I did the conversion on a Gemstone 4.x repository that was so large that there were too many objects for GS 5.x. We dumped to text files and used Oracle's SQL*Loader to put the data into an Oracle RDBMS. MS SQL Server has similar buld load capabilities. The migration was complicated by the fact that the Gemstone schema had never had objects migrated when the schema was modified, so that there were instances as many as 4 different versions of some objects. I also have used GemConnect to dump archival data from Gemstone to Oracle (for a system that was originally written for Oracle and then migrated to Gemstone). GemConnect will still likely require that you hand code stuff for each class. Writing all of the data to text files and then importing it into the relational DB is likely to be a better option. I don't think GemConnect is less work. YMMV. On 3/16/12 7:00 AM, Dennis Smith wrote: When I suggested dumping to some for of ascii, I was intending a form that can be then uploaded into an RDBMS. _______________________________________________ GemStone-Smalltalk mailing list Archive: http://forum.world.st/Gemstone-Customers-f1461796.html |
«
Return to Gemstone/S
|
1 view|%1 views
| Free forum by Nabble | Edit this page |