Databases

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Databases

Squeak - Dev mailing list
Bear with me for a moment...

First and foremost, this post is *not intended in any way* to criticize the work of anyone who could have worked on those packages/projects/frameworks.

Now, I'll tell you a story to make my point.

Often times, in public or in private, I help newcomers on Quora, IRC, the mailing list(s) or elsewhere with various Smalltalk environments.  Often times, I try to be helpful getting them started on Squeak or Pharo.  Often times, they are just amazed by how powerful Smalltalk is... 

Now, the sad part.  Everything goes well until they just want to "code something" to experiment and they ask me about databases...

-How can I connect to a MySQL (or insert your favorite database here) with Squeak/Pharo ?


As far as I can remember, the database connectivity with Squeak/Pharo has always been an area where, as a Squeak/Pharo community, we suck big time.  I don't recall the last time I loaded a package, ran the tests and everything was fine.  There is almost ALWAYS something broken with the databases packages.  Understandibly, Squeak and Pharo were for a long time moving targets.  Whether it was VM changes (FFI, NativeBoost, uFFI, 32 vs 64 bit, OS differences/problems, differences between Squeak and Pharo, etc), code refactoring (cryptography class name changes), the DateAndTime/Timestamp/TimeStamp/Date(Magnitude)/Date(Timespan) were used from one version to the other in the driver, code that was removed by some cleanup, compiler changes/problems/incompatibilities, etc.  In other words, database connectivity needs some love.

Unfortunately, it looks like there doesn't seem to be a consensus or a shared desire to fix things *together*.  There is duplicate effort all over the place.  Just as an example, I listed the various implementations of MySQL packages that exists (I did not list the 8-9 of them that had no code but a project listed on one of those sites).  So, let's say you are a newcomer and you want to connect to a MySQL database.  Here are the choices you are offered (and remember, as a newcomer, you have NO clue which one to pick) :


Databases supported : Project name

1) SqueakSource

MySQL : Automatic Object Storage To MySQL
OpenDBX
SqueakDBX


2) SmalltalkHub

DBX/Talk / DBXDatabaseModel
GlorpDriverMySQL
PharoExtras / ODBC
DBXTalk / Garage
UDBC
UDBC2

3) GitHub.io

dbxtalk

4) GitHub

pharo-rdbms/garage

Now, let's say you picked one package.  In some cases, if you are lucky, the code will load without any problem or warning.  And then, you run the tests.  That is where it almost *always* breaks!

Today, I tried to load *each and everyone of them* in a Pharo 6.1 (Windows) image : none of them worked right out of the box!

In other words, that newcomer I told you about just goes from "WOW!!!" to "WHAT?!?!" in 10 seconds.  People are usually not so impressed with a TestRunner that fails a gazillion unit tests...

So, now the nice part...

Database connectivity is, in my humble opinion, a MUST for Pharo and Squeak. I think it's about time we do *something* about the current state of this mess.  There are enough brilliant and talented people in the Squeak and Pharo communities to fix this situation.  At least, we could start *talking* about it if we want to address the problems.  Ideally, we would come up with a will to change things and establish a plan.

In a perfect world, we'd have something like JDBC (or some kind of Smalltalk universal driver) in Smalltalk for native drivers.  Plus ODBC connectivity. Plus it would have to work right-out-of-the-box.  Plus documentation that has at least an example with Connect/SELECT/UPDATE/DELETE/CREATE/DROP/disconnect simple examples to get newbies started.  Plus a solid suite of tests to beat the sh*t out of that code. Plus a compatibility layer for Squeak/Pharo differences.  To that regard, Glorp is an example to follow!  IMHO, it's about time we share stuff to maximize our efforts!

As I told someone recently, it's about time we do something about databases (RDBMS).  And as I told that person, I don't mind writing Smalltalk code and SQL scripts and running tests.  Even do performance testing.  My MySQL Server currently has close to 1 billion records to torture the MySQL driver if needed.  And I can also work on SQLite.  And Oracle.  And DB/2.  And MySQL Server.  And I don't mind installing other RDBMS on my machine and test other driver ports.  Now, it's just a matter of knowing if we all want to work together.

As for OpenDBX/DBX, I'm not sure.  Having another dependency (and C code) that we more or less control doesn't inspire me.  I have yet to see how DBX/OpenDBX gives me any advantage over, say, ODBC...

As I said, it's time to discuss!

Anyone interested?
 
-----------------
Benoît St-Jean
Yahoo! Messenger: bstjean
Twitter: @BenLeChialeux
Pinterest: benoitstjean
Instagram: Chef_Benito
IRC: lamneth
Blogue: endormitoire.wordpress.com
"A standpoint is an intellectual horizon of radius zero".  (A. Einstein)


Reply | Threaded
Open this post in threaded view
|

Re: Databases

Eliot Miranda-2
Hi Benoit,

On Jan 15, 2018, at 1:13 AM, Benoit St-Jean via Squeak-dev <[hidden email]> wrote:

Bear with me for a moment...

First and foremost, this post is *not intended in any way* to criticize the work of anyone who could have worked on those packages/projects/frameworks.

Now, I'll tell you a story to make my point.

Often times, in public or in private, I help newcomers on Quora, IRC, the mailing list(s) or elsewhere with various Smalltalk environments.  Often times, I try to be helpful getting them started on Squeak or Pharo.  Often times, they are just amazed by how powerful Smalltalk is... 

Now, the sad part.  Everything goes well until they just want to "code something" to experiment and they ask me about databases...

-How can I connect to a MySQL (or insert your favorite database here) with Squeak/Pharo ?


As far as I can remember, the database connectivity with Squeak/Pharo has always been an area where, as a Squeak/Pharo community, we suck big time.  I don't recall the last time I loaded a package, ran the tests and everything was fine.  There is almost ALWAYS something broken with the databases packages.  Understandibly, Squeak and Pharo were for a long time moving targets.  Whether it was VM changes (FFI, NativeBoost, uFFI, 32 vs 64 bit, OS differences/problems, differences between Squeak and Pharo, etc), code refactoring (cryptography class name changes), the DateAndTime/Timestamp/TimeStamp/Date(Magnitude)/Date(Timespan) were used from one version to the other in the driver, code that was removed by some cleanup, compiler changes/problems/incompatibilities, etc.  In other words, database connectivity needs some love.

Unfortunately, it looks like there doesn't seem to be a consensus or a shared desire to fix things *together*.  There is duplicate effort all over the place.  Just as an example, I listed the various implementations of MySQL packages that exists (I did not list the 8-9 of them that had no code but a project listed on one of those sites).  So, let's say you are a newcomer and you want to connect to a MySQL database.  Here are the choices you are offered (and remember, as a newcomer, you have NO clue which one to pick) :


Databases supported : Project name

1) SqueakSource

MySQL : Automatic Object Storage To MySQL
OpenDBX
SqueakDBX


2) SmalltalkHub

DBX/Talk / DBXDatabaseModel
GlorpDriverMySQL
PharoExtras / ODBC
DBXTalk / Garage
UDBC
UDBC2

3) GitHub.io

dbxtalk

4) GitHub

pharo-rdbms/garage

Now, let's say you picked one package.  In some cases, if you are lucky, the code will load without any problem or warning.  And then, you run the tests.  That is where it almost *always* breaks!

Today, I tried to load *each and everyone of them* in a Pharo 6.1 (Windows) image : none of them worked right out of the box!

In other words, that newcomer I told you about just goes from "WOW!!!" to "WHAT?!?!" in 10 seconds.  People are usually not so impressed with a TestRunner that fails a gazillion unit tests...

So, now the nice part...

Database connectivity is, in my humble opinion, a MUST for Pharo and Squeak.

Certainly.

I think it's about time we do *something* about the current state of this mess.  There are enough brilliant and talented people in the Squeak and Pharo communities to fix this situation.  At least, we could start *talking* about it if we want to address the problems.  Ideally, we would come up with a will to change things and establish a plan.

In a perfect world, we'd have something like JDBC (or some kind of Smalltalk universal driver) in Smalltalk for native drivers.  Plus ODBC connectivity. Plus it would have to work right-out-of-the-box.  Plus documentation that has at least an example with Connect/SELECT/UPDATE/DELETE/CREATE/DROP/disconnect simple examples to get newbies started.  Plus a solid suite of tests to beat the sh*t out of that code. Plus a compatibility layer for Squeak/Pharo differences.  To that regard, Glorp is an example to follow!  IMHO, it's about time we share stuff to maximize our efforts!

As I told someone recently, it's about time we do something about databases (RDBMS).  And as I told that person, I don't mind writing Smalltalk code and SQL scripts and running tests.  Even do performance testing.  My MySQL Server currently has close to 1 billion records to torture the MySQL driver if needed.  And I can also work on SQLite.  And Oracle.  And DB/2.  And MySQL Server.  And I don't mind installing other RDBMS on my machine and test other driver ports.  Now, it's just a matter of knowing if we all want to work together.

As for OpenDBX/DBX, I'm not sure.  Having another dependency (and C code) that we more or less control doesn't inspire me.  I have yet to see how DBX/OpenDBX gives me any advantage over, say, ODBC...

As I said, it's time to discuss!

Anyone interested?

I'm interested but probably can't contribute other than by implementing the threaded FFI.  Database connection city is one area where non-blocking access is really important and indeed at Qwaq the requirement for the threaded FFI comes from database access in the server and the need to keep the server responsive while it is otherwise waiting for a query to complete.

So if and when, as I hope to later this year, I can focus on the threaded FFI, database connected city will be one area I'll be using gif testing and I can get involved.

 
-----------------
Benoît St-Jean
Yahoo! Messenger: bstjean
Twitter: @BenLeChialeux
Pinterest: benoitstjean
Instagram: Chef_Benito
IRC: lamneth
Blogue: endormitoire.wordpress.com
"A standpoint is an intellectual horizon of radius zero".  (A. Einstein)