Databases
Locked
 
|
Bear with me for a moment... First and foremost, this post is *not intended in any way* to criticize the work of anyone who could have worked on those packages/projects/frameworks. Now, I'll tell you a story to make my point. Often times, in public or in private, I help newcomers on Quora, IRC, the mailing list(s) or elsewhere with various Smalltalk environments. Often times, I try to be helpful getting them started on Squeak or Pharo. Often times, they are just amazed by how powerful Smalltalk is... Now, the sad part. Everything goes well until they just want to "code something" to experiment and they ask me about databases... -How can I connect to a MySQL (or insert your favorite database here) with Squeak/Pharo ? As far as I can remember, the database connectivity with Squeak/Pharo has always been an area where, as a Squeak/Pharo community, we suck big time. I don't recall the last time I loaded a package, ran the tests and everything was fine. There is almost ALWAYS something broken with the databases packages. Understandibly, Squeak and Pharo were for a long time moving targets. Whether it was VM changes (FFI, NativeBoost, uFFI, 32 vs 64 bit, OS differences/problems, differences between Squeak and Pharo, etc), code refactoring (cryptography class name changes), the DateAndTime/Timestamp/TimeStamp/Date(Magnitude)/Date(Timespan) were used from one version to the other in the driver, code that was removed by some cleanup, compiler changes/problems/incompatibilities, etc. In other words, database connectivity needs some love. Unfortunately, it looks like there doesn't seem to be a consensus or a shared desire to fix things *together*. There is duplicate effort all over the place. Just as an example, I listed the various implementations of MySQL packages that exists (I did not list the 8-9 of them that had no code but a project listed on one of those sites). So, let's say you are a newcomer and you want to connect to a MySQL database. Here are the choices you are offered (and remember, as a newcomer, you have NO clue which one to pick) : Databases supported : Project name 1) SqueakSource MySQL : Automatic Object Storage To MySQL OpenDBX SqueakDBX 2) SmalltalkHub DBX/Talk / DBXDatabaseModel GlorpDriverMySQL PharoExtras / ODBC DBXTalk / Garage UDBC UDBC2 3) GitHub.io dbxtalk 4) GitHub pharo-rdbms/garage Now, let's say you picked one package. In some cases, if you are lucky, the code will load without any problem or warning. And then, you run the tests. That is where it almost *always* breaks! Today, I tried to load *each and everyone of them* in a Pharo 6.1 (Windows) image : none of them worked right out of the box! In other words, that newcomer I told you about just goes from "WOW!!!" to "WHAT?!?!" in 10 seconds. People are usually not so impressed with a TestRunner that fails a gazillion unit tests... So, now the nice part... Database connectivity is, in my humble opinion, a MUST for Pharo and Squeak. I think it's about time we do *something* about the current state of this mess. There are enough brilliant and talented people in the Squeak and Pharo communities to fix this situation. At least, we could start *talking* about it if we want to address the problems. Ideally, we would come up with a will to change things and establish a plan. In a perfect world, we'd have something like JDBC (or some kind of Smalltalk universal driver) in Smalltalk for native drivers. Plus ODBC connectivity. Plus it would have to work right-out-of-the-box. Plus documentation that has at least an example with Connect/SELECT/UPDATE/DELETE/CREATE/DROP/disconnect simple examples to get newbies started. Plus a solid suite of tests to beat the sh*t out of that code. Plus a compatibility layer for Squeak/Pharo differences. To that regard, Glorp is an example to follow! IMHO, it's about time we share stuff to maximize our efforts! As I told someone recently, it's about time we do something about databases (RDBMS). And as I told that person, I don't mind writing Smalltalk code and SQL scripts and running tests. Even do performance testing. My MySQL Server currently has close to 1 billion records to torture the MySQL driver if needed. And I can also work on SQLite. And Oracle. And DB/2. And MySQL Server. And I don't mind installing other RDBMS on my machine and test other driver ports. Now, it's just a matter of knowing if we all want to work together. As for OpenDBX/DBX, I'm not sure. Having another dependency (and C code) that we more or less control doesn't inspire me. I have yet to see how DBX/OpenDBX gives me any advantage over, say, ODBC... As I said, it's time to discuss! Anyone interested? ----------------- Benoît St-Jean Yahoo! Messenger: bstjean Twitter: @BenLeChialeux Pinterest: benoitstjean Instagram: Chef_Benito IRC: lamneth Blogue: endormitoire.wordpress.com "A standpoint is an intellectual horizon of radius zero". (A. Einstein) |
|
|
Benoît,
Yes, of course databases are important and I know many users ask for them, but on the other hand, the low development activity can only mean that there is not enough interest (or some existing solutions work well enough). There are many SQL and non-SQL databases out there, as well as many, incompatible Smalltalk implementations. I do not believe that there can be one magic solution to fix all problems everywhere. Remember that setting up a database is often a challenge by itself. I do not think the current situation is bad, it is just confusing. Making a good meta overview (i.e. documentation writing) and testing solutions would already help many people. To add to your list, I recently wrote a new PostgreSQL client, P3, that is using only TCP networking (no native library mess). It supports most recent Pharo distributions. https://github.com/svenvc/P3 P3 can be used under Glorp. Glorp in still being maintained and in active use, but could use some love in the promotion department. Here is some documentation http://files.pharo.org/books-pdfs/booklet-Glorp/2017-05-02-Glorp-Spiral.pdf https://medium.com/concerning-pharo/reddit-st-in-10-cool-pharo-classes-1b5327ca0740 https://github.com/SquareBracketAssociates/Booklet-Reddit (updated version of the former) https://bintray.com/squarebracketassociates/wip/download_file?file_path=reddit-wip.pdf Actually, few languages have as nice an Object Relational Mapper as Glorp. There is also Voyage that seems quite popular. Sven > On 15 Jan 2018, at 10:13, Benoit St-Jean via Pharo-users <[hidden email]> wrote: > > > From: Benoit St-Jean <[hidden email]> > Subject: Databases > Date: 15 January 2018 at 10:13:10 GMT+1 > To: "[hidden email]" <[hidden email]>, "[hidden email]" <[hidden email]> > Reply-To: Benoit St-Jean <[hidden email]> > > > Bear with me for a moment... > > First and foremost, this post is *not intended in any way* to criticize the work of anyone who could have worked on those packages/projects/frameworks. > > Now, I'll tell you a story to make my point. > > Often times, in public or in private, I help newcomers on Quora, IRC, the mailing list(s) or elsewhere with various Smalltalk environments. Often times, I try to be helpful getting them started on Squeak or Pharo. Often times, they are just amazed by how powerful Smalltalk is... > > Now, the sad part. Everything goes well until they just want to "code something" to experiment and they ask me about databases... > > -How can I connect to a MySQL (or insert your favorite database here) with Squeak/Pharo ? > > > As far as I can remember, the database connectivity with Squeak/Pharo has always been an area where, as a Squeak/Pharo community, we suck big time. I don't recall the last time I loaded a package, ran the tests and everything was fine. There is almost ALWAYS something broken with the databases packages. Understandibly, Squeak and Pharo were for a long time moving targets. Whether it was VM changes (FFI, NativeBoost, uFFI, 32 vs 64 bit, OS differences/problems, differences between Squeak and Pharo, etc), code refactoring (cryptography class name changes), the DateAndTime/Timestamp/TimeStamp/Date(Magnitude)/Date(Timespan) were used from one version to the other in the driver, code that was removed by some cleanup, compiler changes/problems/incompatibilities, etc. In other words, database connectivity needs some love. > > Unfortunately, it looks like there doesn't seem to be a consensus or a shared desire to fix things *together*. There is duplicate effort all over the place. Just as an example, I listed the various implementations of MySQL packages that exists (I did not list the 8-9 of them that had no code but a project listed on one of those sites). So, let's say you are a newcomer and you want to connect to a MySQL database. Here are the choices you are offered (and remember, as a newcomer, you have NO clue which one to pick) : > > > Databases supported : Project name > > 1) SqueakSource > > MySQL : Automatic Object Storage To MySQL > OpenDBX > SqueakDBX > > > 2) SmalltalkHub > > DBX/Talk / DBXDatabaseModel > GlorpDriverMySQL > PharoExtras / ODBC > DBXTalk / Garage > UDBC > UDBC2 > > 3) GitHub.io > > dbxtalk > > 4) GitHub > > pharo-rdbms/garage > > Now, let's say you picked one package. In some cases, if you are lucky, the code will load without any problem or warning. And then, you run the tests. That is where it almost *always* breaks! > > Today, I tried to load *each and everyone of them* in a Pharo 6.1 (Windows) image : none of them worked right out of the box! > > In other words, that newcomer I told you about just goes from "WOW!!!" to "WHAT?!?!" in 10 seconds. People are usually not so impressed with a TestRunner that fails a gazillion unit tests... > > So, now the nice part... > > Database connectivity is, in my humble opinion, a MUST for Pharo and Squeak. I think it's about time we do *something* about the current state of this mess. There are enough brilliant and talented people in the Squeak and Pharo communities to fix this situation. At least, we could start *talking* about it if we want to address the problems. Ideally, we would come up with a will to change things and establish a plan. > > In a perfect world, we'd have something like JDBC (or some kind of Smalltalk universal driver) in Smalltalk for native drivers. Plus ODBC connectivity. Plus it would have to work right-out-of-the-box. Plus documentation that has at least an example with Connect/SELECT/UPDATE/DELETE/CREATE/DROP/disconnect simple examples to get newbies started. Plus a solid suite of tests to beat the sh*t out of that code. Plus a compatibility layer for Squeak/Pharo differences. To that regard, Glorp is an example to follow! IMHO, it's about time we share stuff to maximize our efforts! > > As I told someone recently, it's about time we do something about databases (RDBMS). And as I told that person, I don't mind writing Smalltalk code and SQL scripts and running tests. Even do performance testing. My MySQL Server currently has close to 1 billion records to torture the MySQL driver if needed. And I can also work on SQLite. And Oracle. And DB/2. And MySQL Server. And I don't mind installing other RDBMS on my machine and test other driver ports. Now, it's just a matter of knowing if we all want to work together. > > As for OpenDBX/DBX, I'm not sure. Having another dependency (and C code) that we more or less control doesn't inspire me. I have yet to see how DBX/OpenDBX gives me any advantage over, say, ODBC... > > As I said, it's time to discuss! > > Anyone interested? > > ----------------- > Benoît St-Jean > Yahoo! Messenger: bstjean > Twitter: @BenLeChialeux > Pinterest: benoitstjean > Instagram: Chef_Benito > IRC: lamneth > Blogue: endormitoire.wordpress.com > "A standpoint is an intellectual horizon of radius zero". (A. Einstein) > > |
|
|
> On 15 Jan 2018, at 10:55, Sven Van Caekenberghe <[hidden email]> wrote: > > Benoît, > > Yes, of course databases are important and I know many users ask for them, but on the other hand, the low development activity can only mean that there is not enough interest (or some existing solutions work well enough). > > There are many SQL and non-SQL databases out there, as well as many, incompatible Smalltalk implementations. I do not believe that there can be one magic solution to fix all problems everywhere. > > Remember that setting up a database is often a challenge by itself. > > I do not think the current situation is bad, it is just confusing. Making a good meta overview (i.e. documentation writing) and testing solutions would already help many people. > > To add to your list, I recently wrote a new PostgreSQL client, P3, that is using only TCP networking (no native library mess). Another important advantage of a TCP/IP based database client is that outstanding/long-running calls do not block your image. > It supports most recent Pharo distributions. > > https://github.com/svenvc/P3 > > P3 can be used under Glorp. Glorp in still being maintained and in active use, but could use some love in the promotion department. > > Here is some documentation > > http://files.pharo.org/books-pdfs/booklet-Glorp/2017-05-02-Glorp-Spiral.pdf > https://medium.com/concerning-pharo/reddit-st-in-10-cool-pharo-classes-1b5327ca0740 > https://github.com/SquareBracketAssociates/Booklet-Reddit (updated version of the former) > https://bintray.com/squarebracketassociates/wip/download_file?file_path=reddit-wip.pdf > > Actually, few languages have as nice an Object Relational Mapper as Glorp. > > There is also Voyage that seems quite popular. > > Sven > >> On 15 Jan 2018, at 10:13, Benoit St-Jean via Pharo-users <[hidden email]> wrote: >> >> >> From: Benoit St-Jean <[hidden email]> >> Subject: Databases >> Date: 15 January 2018 at 10:13:10 GMT+1 >> To: "[hidden email]" <[hidden email]>, "[hidden email]" <[hidden email]> >> Reply-To: Benoit St-Jean <[hidden email]> >> >> >> Bear with me for a moment... >> >> First and foremost, this post is *not intended in any way* to criticize the work of anyone who could have worked on those packages/projects/frameworks. >> >> Now, I'll tell you a story to make my point. >> >> Often times, in public or in private, I help newcomers on Quora, IRC, the mailing list(s) or elsewhere with various Smalltalk environments. Often times, I try to be helpful getting them started on Squeak or Pharo. Often times, they are just amazed by how powerful Smalltalk is... >> >> Now, the sad part. Everything goes well until they just want to "code something" to experiment and they ask me about databases... >> >> -How can I connect to a MySQL (or insert your favorite database here) with Squeak/Pharo ? >> >> >> As far as I can remember, the database connectivity with Squeak/Pharo has always been an area where, as a Squeak/Pharo community, we suck big time. I don't recall the last time I loaded a package, ran the tests and everything was fine. There is almost ALWAYS something broken with the databases packages. Understandibly, Squeak and Pharo were for a long time moving targets. Whether it was VM changes (FFI, NativeBoost, uFFI, 32 vs 64 bit, OS differences/problems, differences between Squeak and Pharo, etc), code refactoring (cryptography class name changes), the DateAndTime/Timestamp/TimeStamp/Date(Magnitude)/Date(Timespan) were used from one version to the other in the driver, code that was removed by some cleanup, compiler changes/problems/incompatibilities, etc. In other words, database connectivity needs some love. >> >> Unfortunately, it looks like there doesn't seem to be a consensus or a shared desire to fix things *together*. There is duplicate effort all over the place. Just as an example, I listed the various implementations of MySQL packages that exists (I did not list the 8-9 of them that had no code but a project listed on one of those sites). So, let's say you are a newcomer and you want to connect to a MySQL database. Here are the choices you are offered (and remember, as a newcomer, you have NO clue which one to pick) : >> >> >> Databases supported : Project name >> >> 1) SqueakSource >> >> MySQL : Automatic Object Storage To MySQL >> OpenDBX >> SqueakDBX >> >> >> 2) SmalltalkHub >> >> DBX/Talk / DBXDatabaseModel >> GlorpDriverMySQL >> PharoExtras / ODBC >> DBXTalk / Garage >> UDBC >> UDBC2 >> >> 3) GitHub.io >> >> dbxtalk >> >> 4) GitHub >> >> pharo-rdbms/garage >> >> Now, let's say you picked one package. In some cases, if you are lucky, the code will load without any problem or warning. And then, you run the tests. That is where it almost *always* breaks! >> >> Today, I tried to load *each and everyone of them* in a Pharo 6.1 (Windows) image : none of them worked right out of the box! >> >> In other words, that newcomer I told you about just goes from "WOW!!!" to "WHAT?!?!" in 10 seconds. People are usually not so impressed with a TestRunner that fails a gazillion unit tests... >> >> So, now the nice part... >> >> Database connectivity is, in my humble opinion, a MUST for Pharo and Squeak. I think it's about time we do *something* about the current state of this mess. There are enough brilliant and talented people in the Squeak and Pharo communities to fix this situation. At least, we could start *talking* about it if we want to address the problems. Ideally, we would come up with a will to change things and establish a plan. >> >> In a perfect world, we'd have something like JDBC (or some kind of Smalltalk universal driver) in Smalltalk for native drivers. Plus ODBC connectivity. Plus it would have to work right-out-of-the-box. Plus documentation that has at least an example with Connect/SELECT/UPDATE/DELETE/CREATE/DROP/disconnect simple examples to get newbies started. Plus a solid suite of tests to beat the sh*t out of that code. Plus a compatibility layer for Squeak/Pharo differences. To that regard, Glorp is an example to follow! IMHO, it's about time we share stuff to maximize our efforts! >> >> As I told someone recently, it's about time we do something about databases (RDBMS). And as I told that person, I don't mind writing Smalltalk code and SQL scripts and running tests. Even do performance testing. My MySQL Server currently has close to 1 billion records to torture the MySQL driver if needed. And I can also work on SQLite. And Oracle. And DB/2. And MySQL Server. And I don't mind installing other RDBMS on my machine and test other driver ports. Now, it's just a matter of knowing if we all want to work together. >> >> As for OpenDBX/DBX, I'm not sure. Having another dependency (and C code) that we more or less control doesn't inspire me. I have yet to see how DBX/OpenDBX gives me any advantage over, say, ODBC... >> >> As I said, it's time to discuss! >> >> Anyone interested? >> >> ----------------- >> Benoît St-Jean >> Yahoo! Messenger: bstjean >> Twitter: @BenLeChialeux >> Pinterest: benoitstjean >> Instagram: Chef_Benito >> IRC: lamneth >> Blogue: endormitoire.wordpress.com >> "A standpoint is an intellectual horizon of radius zero". (A. Einstein) >> >> > |
|
|
2018-01-15 12:25 GMT-03:00 Sven Van Caekenberghe <[hidden email]>:
>> On 15 Jan 2018, at 10:55, Sven Van Caekenberghe <[hidden email]> wrote: >> To add to your list, I recently wrote a new PostgreSQL client, P3, that is using only TCP networking (no native library mess). > > Another important advantage of a TCP/IP based database client is that outstanding/long-running calls do not block your image. I think thas has more to do with the way FFI calls work in Pharo rather than TCP/IP. I.e. Dolphin Smalltalk had the option of non-blocking FFI, used extensively for DB access where there is more wait and transfer state, than execution. Esteban A. Maringolo |
|
|
> On 15 Jan 2018, at 16:34, Esteban A. Maringolo <[hidden email]> wrote: > > 2018-01-15 12:25 GMT-03:00 Sven Van Caekenberghe <[hidden email]>: > >>> On 15 Jan 2018, at 10:55, Sven Van Caekenberghe <[hidden email]> wrote: > >>> To add to your list, I recently wrote a new PostgreSQL client, P3, that is using only TCP networking (no native library mess). >> >> Another important advantage of a TCP/IP based database client is that outstanding/long-running calls do not block your image. > > I think thas has more to do with the way FFI calls work in Pharo > rather than TCP/IP. > I.e. Dolphin Smalltalk had the option of non-blocking FFI, used > extensively for DB access where there is more wait and transfer state, > than execution. Yes, of course, it is not impossible, just much harder to get right. With networking you get this basically for free. |
Re: Databases
|
|
In reply to this post by Pharo Smalltalk Users mailing list
Hi, In my casoe UDBC2 and SQLite have worked pretty well, but I have
a problem in the transition from Pharo 4.0 to 5.0 and UDBC that
was quickly fixed once I asked for community support (if I
remember was related with the location of the Sqlite binary). I
could help with testing such scenario. But yes, SQLite is kind of
the out of the box solution for databases in all other programming
languages, so we need to be sure this experience is working for
our users also (I remember one of our workshop participants
complaining about how this default was even alien for Pharo). It's difficult to transmit some sense of stability in a quickly evolving software like this, but I think that is important to have some defaults working out of the box. Cheers, Offray On 15/01/18 04:13, Benoit St-Jean via
Pharo-users wrote:
|
|
|
In reply to this post by Pharo Smalltalk Users mailing list
Thanks benoit.
I would really like to get a list of working solutions for Pharo and potential problems. So Offray said that SqLite and UDBC2 are working. Do you have some precise and actionable reports? Then someone should fix the problems. Stef On Mon, Jan 15, 2018 at 10:13 AM, Benoit St-Jean via Pharo-users <[hidden email]> wrote: > > > ---------- Forwarded message ---------- > From: Benoit St-Jean <[hidden email]> > To: "[hidden email]" <[hidden email]>, "[hidden email]" <[hidden email]> > Cc: > Bcc: > Date: Mon, 15 Jan 2018 09:13:10 +0000 (UTC) > Subject: Databases > Bear with me for a moment... > > First and foremost, this post is *not intended in any way* to criticize the work of anyone who could have worked on those packages/projects/frameworks. > > Now, I'll tell you a story to make my point. > > Often times, in public or in private, I help newcomers on Quora, IRC, the mailing list(s) or elsewhere with various Smalltalk environments. Often times, I try to be helpful getting them started on Squeak or Pharo. Often times, they are just amazed by how powerful Smalltalk is... > > Now, the sad part. Everything goes well until they just want to "code something" to experiment and they ask me about databases... > > -How can I connect to a MySQL (or insert your favorite database here) with Squeak/Pharo ? > > > As far as I can remember, the database connectivity with Squeak/Pharo has always been an area where, as a Squeak/Pharo community, we suck big time. I don't recall the last time I loaded a package, ran the tests and everything was fine. There is almost ALWAYS something broken with the databases packages. Understandibly, Squeak and Pharo were for a long time moving targets. Whether it was VM changes (FFI, NativeBoost, uFFI, 32 vs 64 bit, OS differences/problems, differences between Squeak and Pharo, etc), code refactoring (cryptography class name changes), the DateAndTime/Timestamp/TimeStamp/Date(Magnitude)/Date(Timespan) were used from one version to the other in the driver, code that was removed by some cleanup, compiler changes/problems/incompatibilities, etc. In other words, database connectivity needs some love. > > Unfortunately, it looks like there doesn't seem to be a consensus or a shared desire to fix things *together*. There is duplicate effort all over the place. Just as an example, I listed the various implementations of MySQL packages that exists (I did not list the 8-9 of them that had no code but a project listed on one of those sites). So, let's say you are a newcomer and you want to connect to a MySQL database. Here are the choices you are offered (and remember, as a newcomer, you have NO clue which one to pick) : > > > Databases supported : Project name > > 1) SqueakSource > > MySQL : Automatic Object Storage To MySQL > OpenDBX > SqueakDBX > > > 2) SmalltalkHub > > DBX/Talk / DBXDatabaseModel > GlorpDriverMySQL > PharoExtras / ODBC > DBXTalk / Garage > UDBC > UDBC2 > > 3) GitHub.io > > dbxtalk > > 4) GitHub > > pharo-rdbms/garage > > Now, let's say you picked one package. In some cases, if you are lucky, the code will load without any problem or warning. And then, you run the tests. That is where it almost *always* breaks! > > Today, I tried to load *each and everyone of them* in a Pharo 6.1 (Windows) image : none of them worked right out of the box! > > In other words, that newcomer I told you about just goes from "WOW!!!" to "WHAT?!?!" in 10 seconds. People are usually not so impressed with a TestRunner that fails a gazillion unit tests... > > So, now the nice part... > > Database connectivity is, in my humble opinion, a MUST for Pharo and Squeak. I think it's about time we do *something* about the current state of this mess. There are enough brilliant and talented people in the Squeak and Pharo communities to fix this situation. At least, we could start *talking* about it if we want to address the problems. Ideally, we would come up with a will to change things and establish a plan. > > In a perfect world, we'd have something like JDBC (or some kind of Smalltalk universal driver) in Smalltalk for native drivers. Plus ODBC connectivity. Plus it would have to work right-out-of-the-box. Plus documentation that has at least an example with Connect/SELECT/UPDATE/DELETE/CREATE/DROP/disconnect simple examples to get newbies started. Plus a solid suite of tests to beat the sh*t out of that code. Plus a compatibility layer for Squeak/Pharo differences. To that regard, Glorp is an example to follow! IMHO, it's about time we share stuff to maximize our efforts! > > As I told someone recently, it's about time we do something about databases (RDBMS). And as I told that person, I don't mind writing Smalltalk code and SQL scripts and running tests. Even do performance testing. My MySQL Server currently has close to 1 billion records to torture the MySQL driver if needed. And I can also work on SQLite. And Oracle. And DB/2. And MySQL Server. And I don't mind installing other RDBMS on my machine and test other driver ports. Now, it's just a matter of knowing if we all want to work together. > > As for OpenDBX/DBX, I'm not sure. Having another dependency (and C code) that we more or less control doesn't inspire me. I have yet to see how DBX/OpenDBX gives me any advantage over, say, ODBC... > > As I said, it's time to discuss! > > Anyone interested? > > ----------------- > Benoît St-Jean > Yahoo! Messenger: bstjean > Twitter: @BenLeChialeux > Pinterest: benoitstjean > Instagram: Chef_Benito > IRC: lamneth > Blogue: endormitoire.wordpress.com > "A standpoint is an intellectual horizon of radius zero". (A. Einstein) > |
«
Return to Pharo Smalltalk Users
|
1 view|%1 views
| Free forum by Nabble | Edit this page |