Squeak vs Python "smack down"

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Squeak vs Python "smack down"

Göran Krampe
Hi folks!

Since we are on the verge of 4.2, and we have a brand new Cog VM to play
with I felt like dusting off my old Pystone port to Squeak - Sqystone,
which I wrote back in 2004.

At that time Squeak was around 5 times faster than CPython. How do we
stack up today? Yeah, I know - hardly a good benchmark, they all lie etc
etc. :)

I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.


Cog!
====
Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download):
Pystone(1.1) time for 50000 passes = 0.06
This machine benchmarks at 833333.3 pystones/second

NOTE: AFAICT running with more passes does not improve it. Also, not
sure if I could get more out of this if I built from source on my box.


Regular Squeak
==============
Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357
(built from src):
Pystone(1.1) time for 50000 passes = 0.503
This machine benchmarks at 99403.6 pystones/second


Regular CPython
===============
CPython 3.1.2 (newest in Ubuntu Meerkat, minimal):
gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py
Pystone(1.1) time for 50000 passes = 0.57
This machine benchmarks at 87719.3 pystones/second

NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.


Pypy 1.4
========
wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2
gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy
./lib-python/2.5.2/test/pystone.py
Pystone(1.1) time for 50000 passes = 0.15
This machine benchmarks at 333333 pystones/second
gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy
./lib-python/2.5.2/test/pystone.py 5000000
Pystone(1.1) time for 5000000 passes = 4.8
This machine benchmarks at 1.04167e+06 pystones/second

NOTE: Also not built from source. Here we run pystone a second time with
100x more loops and get a substantially better number.


Shedskin 0.7
============
sudo apt-get install g++ libpcre3-dev libgc-dev python-dev
sudo dpkg -i shedskin_0.7_all.deb
wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz
shedskin pystone.py
gokr@quigon:~/python/shedskin-examples-0.7$ make
g++  -O2 -march=native -fomit-frame-pointer -Wno-deprecated  -I.
-I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp
pystone.cpp /usr/share/shedskin/lib/time.cpp
/usr/share/shedskin/lib/re.cpp -lgc -lpcre  -o pystone
gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la
pystone*
-rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone
-rw-r--r-- 1 gokr gokr   9193 2011-02-07 23:00 pystone.cpp
-rw-r--r-- 1 gokr gokr   1893 2011-02-07 23:00 pystone.hpp
-rw-r--r-- 1 gokr gokr   5774 2010-12-11 11:40 pystone.py
gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
This machine benchmarks at 2500000.000000 pystones/second

NOTE: I am wondering a bit about this. It tells the same whatever loops
I give it... But ok, perhaps it is all fine.


Summary
=======

- The regular Squeak VM has not been overrun by CPython in these 6 years
time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the
post anymore). Now they are equal more or less, Squeak still a teeny bit
faster.

- Cog is brutally fast on this one. Compared to CPython and regular
Squeak almost 10x faster.

- Pypy is about 20% faster than Cog if given enough time to actually
start jitting. Cool for the Pypy project! And cool that they aren't that
much faster than Cog. :)

- Shedskin is the "state of the art" of statically compiling Python via
C++ using type inferencing etc etc - so I hear. It is said to be faster
than Cython and Psyco. It ends up beating Cog, but "only" by a factor of
3x. I say "only" because that seems pretty good to me given that Cog is
a JIT and still pretty young and that Shedskin can only run a subset of
Python.


regards, Göran

Reply | Threaded
Open this post in threaded view
|

Re: Squeak vs Python "smack down"

Igor Stasenko
Nice overview  :)

Now add here the syntax, and you know the winner.

2011/2/8 Göran Krampe <[hidden email]>:

> Hi folks!
>
> Since we are on the verge of 4.2, and we have a brand new Cog VM to play
> with I felt like dusting off my old Pystone port to Squeak - Sqystone, which
> I wrote back in 2004.
>
> At that time Squeak was around 5 times faster than CPython. How do we stack
> up today? Yeah, I know - hardly a good benchmark, they all lie etc etc. :)
>
> I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.
>
>
> Cog!
> ====
> Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download):
> Pystone(1.1) time for 50000 passes = 0.06
> This machine benchmarks at 833333.3 pystones/second
>
> NOTE: AFAICT running with more passes does not improve it. Also, not sure if
> I could get more out of this if I built from source on my box.
>
>
> Regular Squeak
> ==============
> Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357 (built
> from src):
> Pystone(1.1) time for 50000 passes = 0.503
> This machine benchmarks at 99403.6 pystones/second
>
>
> Regular CPython
> ===============
> CPython 3.1.2 (newest in Ubuntu Meerkat, minimal):
> gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py
> Pystone(1.1) time for 50000 passes = 0.57
> This machine benchmarks at 87719.3 pystones/second
>
> NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
>
>
> Pypy 1.4
> ========
> wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2
> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy
> ./lib-python/2.5.2/test/pystone.py
> Pystone(1.1) time for 50000 passes = 0.15
> This machine benchmarks at 333333 pystones/second
> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy
> ./lib-python/2.5.2/test/pystone.py 5000000
> Pystone(1.1) time for 5000000 passes = 4.8
> This machine benchmarks at 1.04167e+06 pystones/second
>
> NOTE: Also not built from source. Here we run pystone a second time with
> 100x more loops and get a substantially better number.
>
>
> Shedskin 0.7
> ============
> sudo apt-get install g++ libpcre3-dev libgc-dev python-dev
> sudo dpkg -i shedskin_0.7_all.deb
> wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz
> shedskin pystone.py
> gokr@quigon:~/python/shedskin-examples-0.7$ make
> g++  -O2 -march=native -fomit-frame-pointer -Wno-deprecated  -I.
> -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp
> /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre
>  -o pystone
> gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la
> pystone*
> -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone
> -rw-r--r-- 1 gokr gokr   9193 2011-02-07 23:00 pystone.cpp
> -rw-r--r-- 1 gokr gokr   1893 2011-02-07 23:00 pystone.hpp
> -rw-r--r-- 1 gokr gokr   5774 2010-12-11 11:40 pystone.py
> gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
> This machine benchmarks at 2500000.000000 pystones/second
>
> NOTE: I am wondering a bit about this. It tells the same whatever loops I
> give it... But ok, perhaps it is all fine.
>
>
> Summary
> =======
>
> - The regular Squeak VM has not been overrun by CPython in these 6 years
> time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post
> anymore). Now they are equal more or less, Squeak still a teeny bit faster.
>
> - Cog is brutally fast on this one. Compared to CPython and regular Squeak
> almost 10x faster.
>
> - Pypy is about 20% faster than Cog if given enough time to actually start
> jitting. Cool for the Pypy project! And cool that they aren't that much
> faster than Cog. :)
>
> - Shedskin is the "state of the art" of statically compiling Python via C++
> using type inferencing etc etc - so I hear. It is said to be faster than
> Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I
> say "only" because that seems pretty good to me given that Cog is a JIT and
> still pretty young and that Shedskin can only run a subset of Python.
>
>
> regards, Göran
>
>



--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: Squeak vs Python "smack down"

Eliot Miranda-2
In reply to this post by Göran Krampe
Hi Göran,

    is it possible to repeat the runs with a much higher iteration count so that we get less temporal quantization?  The numbers below are extremely round simply because the number of iterations are so low (on modern hardware).  Also, is the source available?  URL?  Ta!

thanks; this has brightened my day!!

best
Eliot

2011/2/7 Göran Krampe <[hidden email]>
Hi folks!

Since we are on the verge of 4.2, and we have a brand new Cog VM to play with I felt like dusting off my old Pystone port to Squeak - Sqystone, which I wrote back in 2004.

At that time Squeak was around 5 times faster than CPython. How do we stack up today? Yeah, I know - hardly a good benchmark, they all lie etc etc. :)

I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.


Cog!
====
Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download):
Pystone(1.1) time for 50000 passes = 0.06
This machine benchmarks at 833333.3 pystones/second

NOTE: AFAICT running with more passes does not improve it. Also, not sure if I could get more out of this if I built from source on my box.


Regular Squeak
==============
Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357 (built from src):
Pystone(1.1) time for 50000 passes = 0.503
This machine benchmarks at 99403.6 pystones/second


Regular CPython
===============
CPython 3.1.2 (newest in Ubuntu Meerkat, minimal):
gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py
Pystone(1.1) time for 50000 passes = 0.57
This machine benchmarks at 87719.3 pystones/second

NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.


Pypy 1.4
========
wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2
gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py
Pystone(1.1) time for 50000 passes = 0.15
This machine benchmarks at 333333 pystones/second
gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py 5000000
Pystone(1.1) time for 5000000 passes = 4.8
This machine benchmarks at 1.04167e+06 pystones/second

NOTE: Also not built from source. Here we run pystone a second time with 100x more loops and get a substantially better number.


Shedskin 0.7
============
sudo apt-get install g++ libpcre3-dev libgc-dev python-dev
sudo dpkg -i shedskin_0.7_all.deb
wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz
shedskin pystone.py
gokr@quigon:~/python/shedskin-examples-0.7$ make
g++  -O2 -march=native -fomit-frame-pointer -Wno-deprecated  -I. -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre  -o pystone
gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la pystone*
-rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone
-rw-r--r-- 1 gokr gokr   9193 2011-02-07 23:00 pystone.cpp
-rw-r--r-- 1 gokr gokr   1893 2011-02-07 23:00 pystone.hpp
-rw-r--r-- 1 gokr gokr   5774 2010-12-11 11:40 pystone.py
gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
This machine benchmarks at 2500000.000000 pystones/second

NOTE: I am wondering a bit about this. It tells the same whatever loops I give it... But ok, perhaps it is all fine.


Summary
=======

- The regular Squeak VM has not been overrun by CPython in these 6 years time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post anymore). Now they are equal more or less, Squeak still a teeny bit faster.

- Cog is brutally fast on this one. Compared to CPython and regular Squeak almost 10x faster.

- Pypy is about 20% faster than Cog if given enough time to actually start jitting. Cool for the Pypy project! And cool that they aren't that much faster than Cog. :)

- Shedskin is the "state of the art" of statically compiling Python via C++ using type inferencing etc etc - so I hear. It is said to be faster than Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I say "only" because that seems pretty good to me given that Cog is a JIT and still pretty young and that Shedskin can only run a subset of Python.


regards, Göran




Reply | Threaded
Open this post in threaded view
|

Re: Squeak vs Python "smack down"

Levente Uzonyi-2
On Mon, 7 Feb 2011, Eliot Miranda wrote:

> Hi Göran,
>
>    is it possible to repeat the runs with a much higher iteration count so
> that we get less temporal quantization?  The numbers below are extremely
> round simply because the number of iterations are so low (on modern
> hardware).  Also, is the source available?  URL?  Ta!

The code is available on SqueakMap:
http://map.squeak.org/package/81a1617a-228e-4fcf-80fe-fada0ccde940/autoversion/1


Levente

>
> thanks; this has brightened my day!!
>
> best
> Eliot
>
> 2011/2/7 Göran Krampe <[hidden email]>
>
>> Hi folks!
>>
>> Since we are on the verge of 4.2, and we have a brand new Cog VM to play
>> with I felt like dusting off my old Pystone port to Squeak - Sqystone, which
>> I wrote back in 2004.
>>
>> At that time Squeak was around 5 times faster than CPython. How do we stack
>> up today? Yeah, I know - hardly a good benchmark, they all lie etc etc. :)
>>
>> I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.
>>
>>
>> Cog!
>> ====
>> Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary
>> download):
>> Pystone(1.1) time for 50000 passes = 0.06
>> This machine benchmarks at 833333.3 pystones/second
>>
>> NOTE: AFAICT running with more passes does not improve it. Also, not sure
>> if I could get more out of this if I built from source on my box.
>>
>>
>> Regular Squeak
>> ==============
>> Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357
>> (built from src):
>> Pystone(1.1) time for 50000 passes = 0.503
>> This machine benchmarks at 99403.6 pystones/second
>>
>>
>> Regular CPython
>> ===============
>> CPython 3.1.2 (newest in Ubuntu Meerkat, minimal):
>> gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py
>> Pystone(1.1) time for 50000 passes = 0.57
>> This machine benchmarks at 87719.3 pystones/second
>>
>> NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
>>
>>
>> Pypy 1.4
>> ========
>> wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2
>> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy
>> ./lib-python/2.5.2/test/pystone.py
>> Pystone(1.1) time for 50000 passes = 0.15
>> This machine benchmarks at 333333 pystones/second
>> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy
>> ./lib-python/2.5.2/test/pystone.py 5000000
>> Pystone(1.1) time for 5000000 passes = 4.8
>> This machine benchmarks at 1.04167e+06 pystones/second
>>
>> NOTE: Also not built from source. Here we run pystone a second time with
>> 100x more loops and get a substantially better number.
>>
>>
>> Shedskin 0.7
>> ============
>> sudo apt-get install g++ libpcre3-dev libgc-dev python-dev
>> sudo dpkg -i shedskin_0.7_all.deb
>> wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz
>> shedskin pystone.py
>> gokr@quigon:~/python/shedskin-examples-0.7$ make
>> g++  -O2 -march=native -fomit-frame-pointer -Wno-deprecated  -I.
>> -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp
>> /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre
>>  -o pystone
>> gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la
>> pystone*
>> -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone
>> -rw-r--r-- 1 gokr gokr   9193 2011-02-07 23:00 pystone.cpp
>> -rw-r--r-- 1 gokr gokr   1893 2011-02-07 23:00 pystone.hpp
>> -rw-r--r-- 1 gokr gokr   5774 2010-12-11 11:40 pystone.py
>> gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
>> This machine benchmarks at 2500000.000000 pystones/second
>>
>> NOTE: I am wondering a bit about this. It tells the same whatever loops I
>> give it... But ok, perhaps it is all fine.
>>
>>
>> Summary
>> =======
>>
>> - The regular Squeak VM has not been overrun by CPython in these 6 years
>> time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post
>> anymore). Now they are equal more or less, Squeak still a teeny bit faster.
>>
>> - Cog is brutally fast on this one. Compared to CPython and regular Squeak
>> almost 10x faster.
>>
>> - Pypy is about 20% faster than Cog if given enough time to actually start
>> jitting. Cool for the Pypy project! And cool that they aren't that much
>> faster than Cog. :)
>>
>> - Shedskin is the "state of the art" of statically compiling Python via C++
>> using type inferencing etc etc - so I hear. It is said to be faster than
>> Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I
>> say "only" because that seems pretty good to me given that Cog is a JIT and
>> still pretty young and that Shedskin can only run a subset of Python.
>>
>>
>> regards, Göran
>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] Squeak vs Python "smack down"

Göran Krampe
In reply to this post by Göran Krampe
One small correction:

Rerunning Cog and CPython too with 5000000 loops didn't change the
numbers much, but for Shedskin it turns out more proper:

gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
This machine benchmarks at 3546099.290780 pystones/second

50000 loops was just too small. So in fact it is about 4.4x faster than
Cog. And yes, the ported pystone for shedskin had the loops hard coded. :)

regards, Göran

Reply | Threaded
Open this post in threaded view
|

New numbers! (was Re: [Vm-dev] Squeak vs Python "smack down")

Göran Krampe
In reply to this post by Göran Krampe
Hi!

So a few small notes were made by a few people on how Sqystone is
written, so I tweaked it a bit and these are the more correct numbers,
all run with 5000000 loops.

On 02/08/2011 12:01 AM, Göran Krampe wrote:
> Cog!
> ====
> Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary
> download):
> Pystone(1.1) time for 50000 passes = 0.06
> This machine benchmarks at 833333.3 pystones/second

Pystone(1.1) time for 5000000 passes = 5.448
This machine benchmarks at 917768.0 pystones/second

> Regular Squeak
> ==============
> Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357
> (built from src):
> Pystone(1.1) time for 50000 passes = 0.503
> This machine benchmarks at 99403.6 pystones/second

Pystone(1.1) time for 5000000 passes = 46.448
This machine benchmarks at 107647.3 pystones/second

> Regular CPython
> ===============
> CPython 3.1.2 (newest in Ubuntu Meerkat, minimal):
> gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py
> Pystone(1.1) time for 50000 passes = 0.57
> This machine benchmarks at 87719.3 pystones/second
>
> NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.

gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py 5000000
Pystone(1.1) time for 5000000 passes = 58.6
This machine benchmarks at 85324.2 pystones/second


>
> Pypy 1.4
> ========
> wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2
> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy
> ./lib-python/2.5.2/test/pystone.py
> Pystone(1.1) time for 50000 passes = 0.15
> This machine benchmarks at 333333 pystones/second
> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy
> ./lib-python/2.5.2/test/pystone.py 5000000
> Pystone(1.1) time for 5000000 passes = 4.8
> This machine benchmarks at 1.04167e+06 pystones/second
>
> NOTE: Also not built from source. Here we run pystone a second time with
> 100x more loops and get a substantially better number.
>
>
> Shedskin 0.7
> ============
 > sudo apt-get install g++ libpcre3-dev libgc-dev python-dev
 > sudo dpkg -i shedskin_0.7_all.deb
 > wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz
 > shedskin pystone.py
 > gokr@quigon:~/python/shedskin-examples-0.7$ make
 > g++ -O2 -march=native -fomit-frame-pointer -Wno-deprecated -I.
 > -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp
 > pystone.cpp /usr/share/shedskin/lib/time.cpp
 > /usr/share/shedskin/lib/re.cpp -lgc -lpcre -o pystone
 > gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la
 > pystone*
 > -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone
 > -rw-r--r-- 1 gokr gokr 9193 2011-02-07 23:00 pystone.cpp
 > -rw-r--r-- 1 gokr gokr 1893 2011-02-07 23:00 pystone.hpp
 > -rw-r--r-- 1 gokr gokr 5774 2010-12-11 11:40 pystone.py
 > gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
 > This machine benchmarks at 2500000.000000 pystones/second

5000000 loops gave a more precise number:

gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
This machine benchmarks at 3378378.378378 pystones/second

> Summary
> =======
>
> - The regular Squeak VM has not been overrun by CPython in these 6 years
> time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the
> post anymore). Now they are equal more or less, Squeak still a teeny bit
> faster.

Actually a bit more (not latest 3.2 Python used though):

Squeak = 1.28 x CPython

> - Cog is brutally fast on this one. Compared to CPython and regular
> Squeak almost 10x faster.

Cog = 8.4 x Squeak
Cog = 10.75 x CPython

> - Pypy is about 20% faster than Cog if given enough time to actually
> start jitting. Cool for the Pypy project! And cool that they aren't that
> much faster than Cog. :)

Cog = 0.88 x Pypy

> - Shedskin is the "state of the art" of statically compiling Python via
> C++ using type inferencing etc etc - so I hear. It is said to be faster
> than Cython and Psyco. It ends up beating Cog, but "only" by a factor of
> 3x. I say "only" because that seems pretty good to me given that Cog is
> a JIT and still pretty young and that Shedskin can only run a subset of
> Python.

Shedskin = 3.68 x Cog

regards, Göran

Reply | Threaded
Open this post in threaded view
|

Re: New numbers! (was Re: [Vm-dev] Squeak vs Python "smack down")

Göran Krampe
On 02/09/2011 09:01 AM, Bert Freudenberg wrote:

> On 09.02.2011, at 01:50, Göran Krampe<[hidden email]>  wrote:
>
>   Shedskin can only run a subset of
>>> Python.
>>
>> Shedskin = 3.68 x Cog
>
> A Slang version would be the equivalent of this, right?
>
> - Bert -

Mmmm, well, Shedskin takes Python source as input, does type inference
and produces C++ source, then compiled as normal. But yes, it is a
restricted Python, but AFAICT not at all as restricted as Slang.

So I wouldn't say Slang is really fair - being C in disguise :)

BUT... Unladen Swallow and IronPython might be interesting to test too.
Unladen has a goal of being at least 5x faster than CPython, which in
the light of Cog seems like a rather modest goal. AFAIK it is not fast
at all right now though.

Still, as we all know - Python *is* and probably always will be
*CPython* - all other implementations will be "imitations" with limited
auidences. Now I note there is a py3k-jit branch, but someone mentioned
that it hasn't moved in 7 months.

Pypy is probably the only project that possibly could step in and
"become" a next generation platform for Python - or what do you guys
that have more insight into the Python community than I do?

regards, Göran

Reply | Threaded
Open this post in threaded view
|

Re: New numbers! (was Re: [Vm-dev] Squeak vs Python "smack down")

Stefan Marr
In reply to this post by Göran Krampe
Hi Göran:

Those numbers are certainly fun, but wouldn't it be much more interesting to get the SqueakVM and Cog back into the Language Benchmark Game?

http://shootout.alioth.debian.org/

The Dhrystone benchmarks also do not really have the reputation of being, well, interesting, i.e., kind of resemble standard workloads.

Bringing Cog to the Benchmark game would also allow comparison with for instance LuaJIT2.

Best regards
Stefan

On 09 Feb 2011, at 01:50, Göran Krampe wrote:

> Hi!
>
> So a few small notes were made by a few people on how Sqystone is written, so I tweaked it a bit and these are the more correct numbers, all run with 5000000 loops.
>
> On 02/08/2011 12:01 AM, Göran Krampe wrote:
>> Cog!
>> ====
>> Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary
>> download):
>> Pystone(1.1) time for 50000 passes = 0.06
>> This machine benchmarks at 833333.3 pystones/second
>
> Pystone(1.1) time for 5000000 passes = 5.448
> This machine benchmarks at 917768.0 pystones/second
>
>> Regular Squeak
>> ==============
>> Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357
>> (built from src):
>> Pystone(1.1) time for 50000 passes = 0.503
>> This machine benchmarks at 99403.6 pystones/second
>
> Pystone(1.1) time for 5000000 passes = 46.448
> This machine benchmarks at 107647.3 pystones/second
>
>> Regular CPython
>> ===============
>> CPython 3.1.2 (newest in Ubuntu Meerkat, minimal):
>> gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py
>> Pystone(1.1) time for 50000 passes = 0.57
>> This machine benchmarks at 87719.3 pystones/second
>>
>> NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
>
> gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py 5000000
> Pystone(1.1) time for 5000000 passes = 58.6
> This machine benchmarks at 85324.2 pystones/second
>
>
>>
>> Pypy 1.4
>> ========
>> wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2
>> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy
>> ./lib-python/2.5.2/test/pystone.py
>> Pystone(1.1) time for 50000 passes = 0.15
>> This machine benchmarks at 333333 pystones/second
>> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy
>> ./lib-python/2.5.2/test/pystone.py 5000000
>> Pystone(1.1) time for 5000000 passes = 4.8
>> This machine benchmarks at 1.04167e+06 pystones/second
>>
>> NOTE: Also not built from source. Here we run pystone a second time with
>> 100x more loops and get a substantially better number.
>>
>>
>> Shedskin 0.7
>> ============
> > sudo apt-get install g++ libpcre3-dev libgc-dev python-dev
> > sudo dpkg -i shedskin_0.7_all.deb
> > wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz
> > shedskin pystone.py
> > gokr@quigon:~/python/shedskin-examples-0.7$ make
> > g++ -O2 -march=native -fomit-frame-pointer -Wno-deprecated -I.
> > -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp
> > pystone.cpp /usr/share/shedskin/lib/time.cpp
> > /usr/share/shedskin/lib/re.cpp -lgc -lpcre -o pystone
> > gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la
> > pystone*
> > -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone
> > -rw-r--r-- 1 gokr gokr 9193 2011-02-07 23:00 pystone.cpp
> > -rw-r--r-- 1 gokr gokr 1893 2011-02-07 23:00 pystone.hpp
> > -rw-r--r-- 1 gokr gokr 5774 2010-12-11 11:40 pystone.py
> > gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
> > This machine benchmarks at 2500000.000000 pystones/second
>
> 5000000 loops gave a more precise number:
>
> gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
> This machine benchmarks at 3378378.378378 pystones/second
>
>> Summary
>> =======
>>
>> - The regular Squeak VM has not been overrun by CPython in these 6 years
>> time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the
>> post anymore). Now they are equal more or less, Squeak still a teeny bit
>> faster.
>
> Actually a bit more (not latest 3.2 Python used though):
>
> Squeak = 1.28 x CPython
>
>> - Cog is brutally fast on this one. Compared to CPython and regular
>> Squeak almost 10x faster.
>
> Cog = 8.4 x Squeak
> Cog = 10.75 x CPython
>
>> - Pypy is about 20% faster than Cog if given enough time to actually
>> start jitting. Cool for the Pypy project! And cool that they aren't that
>> much faster than Cog. :)
>
> Cog = 0.88 x Pypy
>
>> - Shedskin is the "state of the art" of statically compiling Python via
>> C++ using type inferencing etc etc - so I hear. It is said to be faster
>> than Cython and Psyco. It ends up beating Cog, but "only" by a factor of
>> 3x. I say "only" because that seems pretty good to me given that Cog is
>> a JIT and still pretty young and that Shedskin can only run a subset of
>> Python.
>
> Shedskin = 3.68 x Cog
>
> regards, Göran

--
Stefan Marr
Software Languages Lab
Vrije Universiteit Brussel
Pleinlaan 2 / B-1050 Brussels / Belgium
http://soft.vub.ac.be/~smarr
Phone: +32 2 629 2974
Fax:   +32 2 629 3525


Reply | Threaded
Open this post in threaded view
|

Re: New numbers! (was Re: [Vm-dev] Squeak vs Python "smack down")

Göran Krampe
On 02/09/2011 05:42 PM, Stefan Marr wrote:

> Hi Göran:
>
> Those numbers are certainly fun, but wouldn't it be much more interesting to get the SqueakVM and Cog back into the Language Benchmark Game?
>
> http://shootout.alioth.debian.org/
>
> The Dhrystone benchmarks also do not really have the reputation of being, well, interesting, i.e., kind of resemble standard workloads.
>
> Bringing Cog to the Benchmark game would also allow comparison with for instance LuaJIT2.
>
> Best regards
> Stefan

Yes, I know. I even talked with Eliot about this just the other day.
Now, that would take some effort though - even though Eliot has a few of
those benchmarks I think. Sqystone on the other hand I already had.

So it is always a matter of available time :)

regards, Göran

Reply | Threaded
Open this post in threaded view
|

Re: New numbers! (was Re: [Vm-dev] Squeak vs Python "smack down")

Stefan Marr
Hi Göran:

On 09 Feb 2011, at 17:46, Göran Krampe wrote:

> Yes, I know. I even talked with Eliot about this just the other day. Now, that would take some effort though - even though Eliot has a few of those benchmarks I think. Sqystone on the other hand I already had.
What exactly would need to be done for that?
I saw quite a number of the standard benchmarks in the CVS repo of the benchmark game and am using some for performance regression testing for the RoarVM.

Well, personally I would be more interested in parallel benchmarks, but I could possibly give a hand and extend my set of benchmarks, too.

A while ago Igor also talked about tracking performance, as Step did today. So there might be some time and interest.

And with regard to tracking, PyPy's speed center is really nice.
I modified it to have it up and running for our purposes. It is certainly a nice option.

https://github.com/tobami/codespeed

Best regards
Stefan



>
> So it is always a matter of available time :)
>
> regards, Göran
>

--
Stefan Marr
Software Languages Lab
Vrije Universiteit Brussel
Pleinlaan 2 / B-1050 Brussels / Belgium
http://soft.vub.ac.be/~smarr
Phone: +32 2 629 2974
Fax:   +32 2 629 3525