GlorpDBX PostgreSQL Patch
Locked
GlorpDBX PostgreSQL Patch
 
|
Hi,
I have a patch for Glorp(DBX) PostgreSQL to support the newer bytea hex format (see file:///usr/local/pgsql/share/doc/html/datatype-binary.html#AEN5037). I can't write to the repository, so here it is: PostgreSQLPlatform>>#convertSQLStringToByteArray: aString for: aType aString isNil ifTrue: [ ^ nil ]. ^ (aString beginsWith: '\x') ifTrue: [ self convertHexSQLStringToByteArray: aString ] ifFalse: [ self convertEscapeSQLStringToByteArray: aString ] PostgreSQLPlatform>>#convertEscapeSQLStringToByteArray: aString | aStream str | aStream := (ByteArray new: aString size // 4) writeStream. str := aString readStream. [str atEnd] whileFalse: [ |nextChar| nextChar := str next. aStream nextPut: (nextChar = $\ ifTrue: [str peek = $\ ifTrue: [str next asInteger] ifFalse: [Number readFrom: (str next: 3) base: 8]] ifFalse: [nextChar asInteger]) ]. aStream close. ^aStream contents PostgreSQLPlatform>>#convertHexSQLStringToByteArray: aString ^ ByteArray new: (aString size // 2) - 1 streamContents: [ :out | | in | (in := aString readStream) skip: 2. [ in atEnd ] whileFalse: [ out nextPut: (Number readFrom: (in next: 2) base: 16) ] ] #convertEscapeSQLStringToByteArray: is the old code, unmodified; #convertHexSQLStringToByteArray: is new. This might be of use upstream as well, I don't really know. If it would help, I could save this as an MC package somewhere. Regards, Sven |
Re: GlorpDBX PostgreSQL Patch
|
|
Thanks Sven. We let the repo as global read because we usually receive unintentional commits.
Do you think it is worth it to have a kind of DBXTalkInbox ? Alan, do you think this is useful also for Glorp trunk ? Thanks! On Tue, Nov 8, 2011 at 5:59 AM, Sven Van Caekenberghe <[hidden email]> wrote: Hi, -- Mariano http://marianopeck.wordpress.com |
|
|
My inclination would be to
think that this is something that the database driver ought to be
handling rather than having it in Glorp at all. But, given that it's in
there, I suppose it's not unreasonable.
|
Cluster analysis of Pharo source available?
|
|
I like such things, like visualizing class dependencies with girvan newman algorithm:
http://igraph.sourceforge.net/screenshots2.html or here: http://www.little-idiot.de/teambuilding/apache-structure.pdf Should be nice to see, how Pharo has changed over time. Are there such "reflective tools" available in Pharo? (Should be pretty easy to handover a list of Pharo class dependencies to igraph ...) Tnx, Guido Stepken |
Re: Cluster analysis of Pharo source available?
|
|
In reply to this post by Alan Knight-2
On Nov 9, 2011, at 6:15 PM, Guido Stepken wrote: > I like such things, like visualizing class dependencies with girvan newman algorithm: > > http://igraph.sourceforge.net/screenshots2.html > > or here: > > http://www.little-idiot.de/teambuilding/apache-structure.pdf > > Should be nice to see, how Pharo has changed over time. > > Are there such "reflective tools" available in Pharo? (Should be pretty easy to handover a list of Pharo class dependencies to igraph ...) > http://moosetechnology.org -- Marcus Denker -- http://marcusdenker.de |
Re: GlorpDBX PostgreSQL Patch
|
|
In reply to this post by Alan Knight-2
Alan,
On 09 Nov 2011, at 21:20, Alan Knight wrote: > My inclination would be to think that this is something that the database driver ought to be handling rather than having it in Glorp at all. But, given that it's in there, I suppose it's not unreasonable. Well, I just extended the previous conversion that was already located there. I did not consider whether or not that kind of platform specific code was in its right place, architecture wise. So, will it be integrated somewhere ? If it is only integrated in the Pharo/Squeak port that is fine for me too, but it would be frustrating if it got lost on the next upstream integration. I would just love to be able to load the standard release and have it working for me (i.e. loading blobs from a recent PostgreSQL). Thx for the reply, Sven |
Re: GlorpDBX PostgreSQL Patch
|
|
On 10.11.2011 14:29, Sven Van Caekenberghe wrote:
> Alan, > > On 09 Nov 2011, at 21:20, Alan Knight wrote: > >> My inclination would be to think that this is something that the database driver ought to be handling rather than having it in Glorp at all. But, given that it's in there, I suppose it's not unreasonable. > Well, I just extended the previous conversion that was already located there. I did not consider whether or not that kind of platform specific code was in its right place, architecture wise. > > So, will it be integrated somewhere ? If it is only integrated in the Pharo/Squeak port that is fine for me too, but it would be frustrating if it got lost on the next upstream integration. > > I would just love to be able to load the standard release and have it working for me (i.e. loading blobs from a recent PostgreSQL). > > Thx for the reply, > > Sven > OpenDBX-driver. Glorp can't be the only user who wants to read blobs, can it? ;) Cheers, Henry |
Re: GlorpDBX PostgreSQL Patch
|
|
On 10 Nov 2011, at 15:15, Henrik Sperre Johansen wrote: > I agree with Alan, it should be in the Postgres-specific code of OpenDBX-driver. > Glorp can't be the only user who wants to read blobs, can it? ;) I kind of agree as well, but these are two separate things: (1) - I submit a patch to existing code, extending a conversion that was already there (2) - a discussion about architecture and a possible refactoring / improvement It feels like a no brainer to accept (1) and move (2) to the todo/issues list. Sven |
|
|
In reply to this post by Guido Stepken
As the saying says: "Every person to its taste!", I will not delve in the liking or not of the graphs, but I think the question must be posed:
What practical measures can be extracted of such representation and how can they be used to [help] fulfill the Pharo vision? -- Cesar Rabak Em 09/11/2011 19:15, Guido Stepken < [hidden email] > escreveu: I like such things, like visualizing class dependencies with girvan newman algorithm: http://igraph.sourceforge.net/screenshots2.html or here: http://www.little-idiot.de/teambuilding/apache-structure.pdf Should be nice to see, how Pharo has changed over time. Are there such "reflective tools" available in Pharo? (Should be pretty easy to handover a list of Pharo class dependencies to igraph ...) Tnx, Guido Stepken |
Re: Cluster analysis of Pharo source available?
|
|
Project managers directly can *see*, how much brainpower each programmer is investing and in how far he really cleans up code, defines clear and usable (and used by other coprogrammers) interfaces, or he is messing up the code, producing even more work and costs for others.
You know - "lines not written" -> but they directly show up in the graphs! Should be standard everywhere, IMHO. Have fun! Guido Stepken Am 10.11.2011 um 21:25 schrieb [hidden email]: > As the saying says: "Every person to its taste!", I will not delve in the liking or not of the graphs, but I think the question must be posed: > > What practical measures can be extracted of such representation and how can they be used to [help] fulfill the Pharo vision? > > -- > Cesar Rabak > > Em 09/11/2011 19:15, Guido Stepken < [hidden email] > escreveu: > I like such things, like visualizing class dependencies with girvan newman algorithm: > > http://igraph.sourceforge.net/screenshots2.html > > or here: > > http://www.little-idiot.de/teambuilding/apache-structure.pdf > > Should be nice to see, how Pharo has changed over time. > > Are there such "reflective tools" available in Pharo? (Should be pretty easy to handover a list of Pharo class dependencies to igraph ...) > > Tnx, Guido Stepken > > |
Re: GlorpDBX PostgreSQL Patch
|
|
In reply to this post by Henrik Sperre Johansen
Hi guys. I am trying to understand how to integrate this in the OpenDBX driver, but I am a little bit lost.
First, I don't understand when this #byteArrayToSQLStringConverter is used. What is the bytearray and what is the string? one is the database type and the other one is the inst var of one you your mapping class? Second, I don't know WHERE to put such conversion in the OpenDBX driver. Why it should be there if #convertSQLStringToByteArray: aString for: aType will continue to exist in Glorp. So it means I should also modify Glorp? How does the EXDI driver of VW work here? Thanks! On Thu, Nov 10, 2011 at 11:15 AM, Henrik Sperre Johansen <[hidden email]> wrote:
-- Mariano http://marianopeck.wordpress.com |
Re: GlorpDBX PostgreSQL Patch
|
|
On 11.11.2011 12:16, Mariano Martinez Peck wrote:
> Hi guys. I am trying to understand how to integrate this in the > OpenDBX driver, but I am a little bit lost. > First, I don't understand when this #byteArrayToSQLStringConverter is > used. What is the bytearray and what is the string? one is the > database type and the other one is the inst var of one you your > mapping class? > Yes. Blobs in Postgres-SQL is basically byteStrings, where a single byte can be encoded in one of two ways: - Raw with some escaped - As a hex string So the method should be used by the driver when asked to store a (smalltalk) byteArray into a (postgres) blob column. > Second, I don't know WHERE to put such conversion in the OpenDBX > driver. Why it should be there if #convertSQLStringToByteArray: > aString for: aType will continue to exist in Glorp. So it means I > should also modify Glorp? Yes, it should be removed from Glorp. If you want to store bytearrays to blobs in different backend databases (or same backend, but different drivers), converting from smalltalk object to what the database stores has to be the job of the driver. Otherwise, you might end up double-encoding in some cases, if both Glorp and Driver does the conversion, and if not in driver, duplicate the conversion in any users of the driver if they are to work correctly. > > How does the EXDI driver of VW work here? Driver takes care of conversion. Does not currently support hex-string format though. Cheers, Henry |
Re: GlorpDBX PostgreSQL Patch
|
|
In reply to this post by Mariano Martinez Peck
On 11.11.2011 13:00, Henrik Sperre Johansen wrote:
> On 11.11.2011 12:16, Mariano Martinez Peck wrote: >> Hi guys. I am trying to understand how to integrate this in the >> OpenDBX driver, but I am a little bit lost. >> First, I don't understand when this #byteArrayToSQLStringConverter is >> used. What is the bytearray and what is the string? one is the >> database type and the other one is the inst var of one you your >> mapping class? >> > Yes. > Blobs in Postgres-SQL is basically byteStrings, where a single byte > can be encoded in one of two ways: > - Raw with some escaped > - As a hex string > > So the method should be used by the driver when asked to store a > (smalltalk) byteArray into a (postgres) blob column. > >> Second, I don't know WHERE to put such conversion in the OpenDBX >> driver. Why it should be there if #convertSQLStringToByteArray: >> aString for: aType will continue to exist in Glorp. So it means I >> should also modify Glorp? > Yes, it should be removed from Glorp. > If you want to store bytearrays to blobs in different backend > databases (or same backend, but different drivers), converting from > smalltalk object to what the database stores has to be the job of the > driver. > Otherwise, you might end up double-encoding in some cases, if both > Glorp and Driver does the conversion, > and if not in driver, duplicate the conversion in any users of the > driver if they are to work correctly. > >> >> How does the EXDI driver of VW work here? > > Driver takes care of conversion. > Does not currently support hex-string format though. > > Cheers, > Henry > > database, not just one of the cases :) Cheers, Henry |
Re: GlorpDBX PostgreSQL Patch
|
|
In reply to this post by Mariano Martinez Peck
On Fri, Nov 11, 2011 at 8:16 AM, Mariano Martinez Peck <[hidden email]> wrote: Hi guys. I am trying to understand how to integrate this in the OpenDBX driver, but I am a little bit lost. This is because the conversion is being done right now in glorp... We are already performing conversions in OpenDBX, but they are not platform specific conversions, since we handle only ANSI conversions ;). Maybe we can build a PostgreSQLOpenDBXDriver (?)
|
|
|
There are always two possible mechanisms for combining the data type with the SQL statement. We can bind it, in which case the driver is responsible for any conversion, and quite possibly it's being converted into some sort of binary representation. Or we can write it inline in the query as a string. So the VisualWorks drivers would be responsible for doing any conversion if we do binding, but it's true that we need to be responsible if we want to do it as a string.
In Postgresql, at least in VisualWorks, it's a bit confusing, because the Postgresql driver basically doesn't support binding - or at least what it does for binding is insert the literal into the string. That's more expensive, and just ends up the same, so normally I wouldn't use binding on Postgresql.
But the bottom line is that yes, we do need Glorp to be able to do this at its level, and the OpenDBX driver would only be involved if someone was using binding. On Fri, Nov 11, 2011 at 7:33 AM, Guillermo Polito <[hidden email]> wrote:
|
Re: Cluster analysis of Pharo source available?
|
|
In reply to this post by Guido Stepken
On Nov 10, 2011, at 9:44 PM, Guido Stepken wrote: > Project managers directly can *see*, how much brainpower each programmer is investing and in how far he really cleans up code, defines clear and usable (and used by other coprogrammers) interfaces, or he is messing up the code, producing even more work and costs for others. > > You know - "lines not written" -> but they directly show up in the graphs! do you have an example? A web site because I'm interested for Moose. Stef > > Should be standard everywhere, IMHO. > > Have fun! > > Guido Stepken > > Am 10.11.2011 um 21:25 schrieb [hidden email]: > >> As the saying says: "Every person to its taste!", I will not delve in the liking or not of the graphs, but I think the question must be posed: >> >> What practical measures can be extracted of such representation and how can they be used to [help] fulfill the Pharo vision? >> >> -- >> Cesar Rabak >> >> Em 09/11/2011 19:15, Guido Stepken < [hidden email] > escreveu: >> I like such things, like visualizing class dependencies with girvan newman algorithm: >> >> http://igraph.sourceforge.net/screenshots2.html >> >> or here: >> >> http://www.little-idiot.de/teambuilding/apache-structure.pdf >> >> Should be nice to see, how Pharo has changed over time. >> >> Are there such "reflective tools" available in Pharo? (Should be pretty easy to handover a list of Pharo class dependencies to igraph ...) >> >> Tnx, Guido Stepken >> >> > |
|
|
In reply to this post by Guido Stepken
Guido,
Notwithstanding how useful would be to track these attributes you just mentioned, I cannot see how the graphs of the clusters shown in the earlier post can be either a representation of them nor even a summary of what you brought now in this comment. . . Regards, -- Cesar Rabak Em 10/11/2011 18:44, Guido Stepken < [hidden email] > escreveu: Project managers directly can *see*, how much brainpower each programmer is investing and in how far he really cleans up code, defines clear and usable (and used by other coprogrammers) interfaces, or he is messing up the code, producing even more work and costs for others. You know - "lines not written" -> but they directly show up in the graphs! Should be standard everywhere, IMHO. Have fun! Guido Stepken Am 10.11.2011 um 21:25 schrieb [hidden email]: > As the saying says: "Every person to its taste!", I will not delve in the liking or not of the graphs, but I think the question must be posed: > > What practical measures can be extracted of such representation and how can they be used to [help] fulfill the Pharo vision? > > -- > Cesar Rabak > > Em 09/11/2011 19:15, Guido Stepken < [hidden email] > escreveu: > I like such things, like visualizing class dependencies with girvan newman algorithm: > > http://igraph.sourceforge.net/screenshots2.html > > or here: > > http://www.little-idiot.de/teambuilding/apache-structure.pdf > > Should be nice to see, how Pharo has changed over time. > > Are there such "reflective tools" available in Pharo? (Should be pretty easy to handover a list of Pharo class dependencies to igraph ...) > > Tnx, Guido Stepken > > |
|
|
In reply to this post by Guillermo Polito
Hi,
There seems to be a problem with #isPrime in Pharo 1.4. The following example runs apparently forever: 100000000000000000000000000039 isPrime. Unfortunately I used #isPrime to demonstrate multiprocessing with worker images with RemoteTask. RemoteTask seems to work fine on Pharo, but the examples run "forever" and thus appear to fail due to the #isPrime problem. RemoteTask is described here <http://lists.squeakfoundation.org/pipermail/squeak-dev/2011-November/162167.html> Dave |
|
|
Thanks dave
http://code.google.com/p/pharo/issues/detail?id=4997 Stef On Nov 13, 2011, at 4:58 PM, David T. Lewis wrote: > Hi, > > There seems to be a problem with #isPrime in Pharo 1.4. The following example > runs apparently forever: > > 100000000000000000000000000039 isPrime. > > Unfortunately I used #isPrime to demonstrate multiprocessing with worker > images with RemoteTask. RemoteTask seems to work fine on Pharo, but the > examples run "forever" and thus appear to fail due to the #isPrime problem. > > RemoteTask is described here > <http://lists.squeakfoundation.org/pipermail/squeak-dev/2011-November/162167.html> > > > Dave |
|
|
I think this may be just an incomplete update from Squeak trunk. I added
a comment to the issue to explain. Dave On Sun, Nov 13, 2011 at 06:21:50PM +0100, St?phane Ducasse wrote: > Thanks dave > > http://code.google.com/p/pharo/issues/detail?id=4997 > > Stef > > On Nov 13, 2011, at 4:58 PM, David T. Lewis wrote: > > > Hi, > > > > There seems to be a problem with #isPrime in Pharo 1.4. The following example > > runs apparently forever: > > > > 100000000000000000000000000039 isPrime. > > > > Unfortunately I used #isPrime to demonstrate multiprocessing with worker > > images with RemoteTask. RemoteTask seems to work fine on Pharo, but the > > examples run "forever" and thus appear to fail due to the #isPrime problem. > > > > RemoteTask is described here > > <http://lists.squeakfoundation.org/pipermail/squeak-dev/2011-November/162167.html> > > > > > > Dave > |
«
Return to Pharo Smalltalk Developers
|
1 view|%1 views
| Free forum by Nabble | Edit this page |