Pharo performance

I think my package is based on the one from the DBX suite. It has a few edits, some of which I'm confident should go in and others just for my needs which probably aren't fit to.

Disappointly, I got the psyco test wrong and it is only taking 30m/s :( I guess this is the difference when something is implemented on top of C and libpq (I presume)

---

From: Mariano Martinez Peck <[hidden email]>
To: Any question about pharo is welcome <[hidden email]>
Sent: Wednesday, 31 July 2013, 22:40
Subject: Re: [Pharo-users] Pharo performance

On Wed, Jul 31, 2013 at 6:32 PM, Chris <[hidden email]> wrote:

It definitely seems to be because most things in the V2 driver are coming in as strings and then being converted. Running the same query on the three systems gives the following for a 65000 double array

PGV2 770m/s
PGV3 200m/s
Psycopg2 130m/s

Now I just need to find out how to get PGV3 as the backend for Glorp!

PLease please please use the Glorp version you get from the DBX suite.

You need to create a subclass of DatabaseDriver. You can take a look to NativePostgresDriver and create a similar one for V3.

I would also like to have Glorp + PostgresV3 running. 

Also...we should somehow use NativePostgresPoolingDriver and NativePostgresConnectionPool for V3... we need some refactor here...

We should join forces!! 

 

Cheers
Chris

Hi Yanni,

On 30 Jul 2013, at 05:17, Yanni Chiu <[hidden email]> wrote:

On 29/07/13 7:08 PM, Sven Van Caekenberghe wrote:

The explanation for the slowdown must be in the PgV2 driver.

The PgV2 protocol is described at:
  http://www.postgresql.org/docs/7.1/static/protocol-message-formats.html

Have a glance at the "AsciiRow" and "BinaryRow" message formats. The driver reads the data off the socket, parsing the the data, as described as described by the message format. With the V2 protocol design, you have to read the result row, one field at a time.

IIUC, in the newer V3? protocol, the AsciiRow/BinaryRow message is replaced by a DataRow message. The DataRow message has the message size included, which could allow the driver to read the entire set of fields for one data row, using a single socket read (or a few buffer sized reads).

I recall seeing an experimental V3 protocol implementation, a few years back - sorry, no links handy. It would be nice to see some benchmarks.

Hope that helps.

Thanks for the response.

I believe the V3 project is here http://www.squeaksource.com/PostgresV3.html.

Now, I probably spoke too fast and should have taken Mariano's advice to never speculate and first measure. Here is my quick test that, for me, shows that PostgresV2 seems more than fast enough (something that I had experienced before without doing benchmarks).

[ self execute: 'select longitude,latitude from log_data limit 10000;' ] timeToRun.

=> 76 ms

[ self execute: 'select longitude,latitude from log_data limit 100000;' ] timeToRun.

=> 765 ms

This is querying for 2 floats from a huge table, over the network. Pretty fast ;-)

So, back to Chris: what exactly are you doing that is (so) slow ?

Anyway, thanks Yanni for all your work on the existing driver !

Sven

--
Sven Van Caekenberghe
Proudly supporting Pharo
http://pharo.org
http://association.pharo.org
http://consortium.pharo.org

--
Mariano
http://marianopeck.wordpress.com