Pharo performance

First thing, printing a Float correctly is a difficult task.

Let's remind the requirements for a decimal printString:

1) every two different Float shall have a different printString representation

2) the printString should be re-interpreted as the same Float

3) the printString should be the shortest decimal that would be re-interpreted as the same Float

Requirement 2) is stronger than 1), and 3) is stronger than 2).

For a REPL, we at least want 2)

If we restrict to requirement 2) then we can simply use printf with 17 decimal digits to accelerate printString.

But then we must expect 0.1 to be printed '0.10000000000000001' or something like that, an awfull thing for a REPL

Requirement 3) is stronger than 2) and it nicely gives 0.1 printString -> '0.1'

This is what Python Java Scheme and maybe some other languages do.

Squeak/Pharo did implement 3) for long by using the algorithm of Scheme, absPrintExactlyOn:base:
It is a relatively easy naive implementation using LargeInteger arithmetic.
But it did not use it by default, which made Squeak/Pharo not even respecting requirement 1)

So all I did was to make this algorithm the default choice (and maybe correct some edge cases, can't remember).

But LargeInteger arithmetic is relatively slow (compared to SmallInteger / Float arithmetic).

So I have done plenty of things to try and (marginally) accelerate printString.
But allmost all should already be integrated in Pharo.

My last experiments consisted in rewriting some LargeInteger primitives to use 32 bits digits instead of 8 bits digits.
This work is mostly ready. I personnally exclusively work with such modified VM.
Only ImageSegment support on BigEndian VM is lacking, but this should hardly be a problem for Pharo.
It has not been integrated, but again the acceleration of Float printString is marginal.

Maybe it could be accelerated with a primitive, but this will be a tough task.

In presence of identified bottleneck, the questions are:

a) can we use a binary format ?

b) can we use another exact representation (like hexadecimal printString of float bits for example, or C printf %a format...)

c) can we use a degraded solution 2) (printf with 17 decimals)

d) can we afford an inexact representation not satisfying 1)-2)-3)

To me, it depends mostly on usage. a) and b) are not for human end-user for example, but is a human going to grok Giga-Floats ?
a) and b) might be solutions for serialization, but it mostly depend on the other end.
It must be an application specific decision

Maybe a support for solution b) and c) could help.

Nicolas

2013/7/29 Mariano Martinez Peck <[hidden email]>

I remember Nicolas Cieller did something about improving performance of Float printing.

Not sure what is the state yet....

On Mon, Jul 29, 2013 at 4:50 PM, Chris <[hidden email]> wrote:

I often use the profiler :)

The float printString is all fairly evenly split which is why I mention about whether another implementation may be required. I'll always raise anything much more obvious that I see!

Hi Chris,

My recommendation in this case is always do not spend a single second trying to figure out what is the bottleneck by yourself. First thing ever to do, is to run a profiler. You can do that very easily in Pharo.

TimeProfiler spyOn: [ Transcript show: 'do something for real here to benchmark'.

Object allSubclasses.   ]  

replace the closure for what you want to benchmark.

Then, share with us if you have interesting finds....

Cheers,

On Mon, Jul 29, 2013 at 3:52 PM, Chris <[hidden email]> wrote:

I've been getting a little concerned with certain aspects of performance recently. Just a couple of examples off the top of my head were trying to do a printString on 200000 floats which takes over 3 seconds. If I do the same in Python it is only 0.25 seconds. Similarly reading 65000 points from a database with the PostgresV2 driver was about 800m/s and only 40 with psycopg. I'd have to try it again but am pretty sure going native was faster than OpenDBX as well. I appreciate Pharo is never going to be able to compete with the static-typed heavyweight languages but would hope we can get performance at least comparable to other dynamic languages :) Is it just that some method implementations are in need of some TLC; more things moved on top of C libraries and primitives and so forth rather than anything with the VM itself?

Cheers
Chris

--
Mariano
http://marianopeck.wordpress.com

--
Mariano
http://marianopeck.wordpress.com

It definitely seems to be because most things in the V2 driver are coming in as strings and then being converted. Running the same query on the three systems gives the following for a 65000 double array

PGV2 770m/s
PGV3 200m/s
Psycopg2 130m/s

Now I just need to find out how to get PGV3 as the backend for Glorp!

Cheers
Chris

Hi Yanni,

On 30 Jul 2013, at 05:17, Yanni Chiu <[hidden email]> wrote:

On 29/07/13 7:08 PM, Sven Van Caekenberghe wrote:

The explanation for the slowdown must be in the PgV2 driver.

The PgV2 protocol is described at:
  http://www.postgresql.org/docs/7.1/static/protocol-message-formats.html

Have a glance at the "AsciiRow" and "BinaryRow" message formats. The driver reads the data off the socket, parsing the the data, as described as described by the message format. With the V2 protocol design, you have to read the result row, one field at a time.

IIUC, in the newer V3? protocol, the AsciiRow/BinaryRow message is replaced by a DataRow message. The DataRow message has the message size included, which could allow the driver to read the entire set of fields for one data row, using a single socket read (or a few buffer sized reads).

I recall seeing an experimental V3 protocol implementation, a few years back - sorry, no links handy. It would be nice to see some benchmarks.

Hope that helps.

Thanks for the response.

I believe the V3 project is here http://www.squeaksource.com/PostgresV3.html.

Now, I probably spoke too fast and should have taken Mariano's advice to never speculate and first measure. Here is my quick test that, for me, shows that PostgresV2 seems more than fast enough (something that I had experienced before without doing benchmarks).

[ self execute: 'select longitude,latitude from log_data limit 10000;' ] timeToRun.

=> 76 ms

[ self execute: 'select longitude,latitude from log_data limit 100000;' ] timeToRun.

=> 765 ms

This is querying for 2 floats from a huge table, over the network. Pretty fast ;-)

So, back to Chris: what exactly are you doing that is (so) slow ?

Anyway, thanks Yanni for all your work on the existing driver !

Sven

--
Sven Van Caekenberghe
Proudly supporting Pharo
http://pharo.org
http://association.pharo.org
http://consortium.pharo.org