Squeak vs Python "smack down"

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Squeak vs Python "smack down"

Göran Krampe
 
Hi folks!

Since we are on the verge of 4.2, and we have a brand new Cog VM to play
with I felt like dusting off my old Pystone port to Squeak - Sqystone,
which I wrote back in 2004.

At that time Squeak was around 5 times faster than CPython. How do we
stack up today? Yeah, I know - hardly a good benchmark, they all lie etc
etc. :)

I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.


Cog!
====
Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download):
Pystone(1.1) time for 50000 passes = 0.06
This machine benchmarks at 833333.3 pystones/second

NOTE: AFAICT running with more passes does not improve it. Also, not
sure if I could get more out of this if I built from source on my box.


Regular Squeak
==============
Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357
(built from src):
Pystone(1.1) time for 50000 passes = 0.503
This machine benchmarks at 99403.6 pystones/second


Regular CPython
===============
CPython 3.1.2 (newest in Ubuntu Meerkat, minimal):
gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py
Pystone(1.1) time for 50000 passes = 0.57
This machine benchmarks at 87719.3 pystones/second

NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.


Pypy 1.4
========
wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2
gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy
./lib-python/2.5.2/test/pystone.py
Pystone(1.1) time for 50000 passes = 0.15
This machine benchmarks at 333333 pystones/second
gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy
./lib-python/2.5.2/test/pystone.py 5000000
Pystone(1.1) time for 5000000 passes = 4.8
This machine benchmarks at 1.04167e+06 pystones/second

NOTE: Also not built from source. Here we run pystone a second time with
100x more loops and get a substantially better number.


Shedskin 0.7
============
sudo apt-get install g++ libpcre3-dev libgc-dev python-dev
sudo dpkg -i shedskin_0.7_all.deb
wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz
shedskin pystone.py
gokr@quigon:~/python/shedskin-examples-0.7$ make
g++  -O2 -march=native -fomit-frame-pointer -Wno-deprecated  -I.
-I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp
pystone.cpp /usr/share/shedskin/lib/time.cpp
/usr/share/shedskin/lib/re.cpp -lgc -lpcre  -o pystone
gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la
pystone*
-rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone
-rw-r--r-- 1 gokr gokr   9193 2011-02-07 23:00 pystone.cpp
-rw-r--r-- 1 gokr gokr   1893 2011-02-07 23:00 pystone.hpp
-rw-r--r-- 1 gokr gokr   5774 2010-12-11 11:40 pystone.py
gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
This machine benchmarks at 2500000.000000 pystones/second

NOTE: I am wondering a bit about this. It tells the same whatever loops
I give it... But ok, perhaps it is all fine.


Summary
=======

- The regular Squeak VM has not been overrun by CPython in these 6 years
time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the
post anymore). Now they are equal more or less, Squeak still a teeny bit
faster.

- Cog is brutally fast on this one. Compared to CPython and regular
Squeak almost 10x faster.

- Pypy is about 20% faster than Cog if given enough time to actually
start jitting. Cool for the Pypy project! And cool that they aren't that
much faster than Cog. :)

- Shedskin is the "state of the art" of statically compiling Python via
C++ using type inferencing etc etc - so I hear. It is said to be faster
than Cython and Psyco. It ends up beating Cog, but "only" by a factor of
3x. I say "only" because that seems pretty good to me given that Cog is
a JIT and still pretty young and that Shedskin can only run a subset of
Python.


regards, Göran
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Squeak vs Python "smack down"

Eliot Miranda-2
 
Hi Göran,

    is it possible to repeat the runs with a much higher iteration count so that we get less temporal quantization?  The numbers below are extremely round simply because the number of iterations are so low (on modern hardware).  Also, is the source available?  URL?  Ta!

thanks; this has brightened my day!!

best
Eliot

2011/2/7 Göran Krampe <[hidden email]>
Hi folks!

Since we are on the verge of 4.2, and we have a brand new Cog VM to play with I felt like dusting off my old Pystone port to Squeak - Sqystone, which I wrote back in 2004.

At that time Squeak was around 5 times faster than CPython. How do we stack up today? Yeah, I know - hardly a good benchmark, they all lie etc etc. :)

I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.


Cog!
====
Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download):
Pystone(1.1) time for 50000 passes = 0.06
This machine benchmarks at 833333.3 pystones/second

NOTE: AFAICT running with more passes does not improve it. Also, not sure if I could get more out of this if I built from source on my box.


Regular Squeak
==============
Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357 (built from src):
Pystone(1.1) time for 50000 passes = 0.503
This machine benchmarks at 99403.6 pystones/second


Regular CPython
===============
CPython 3.1.2 (newest in Ubuntu Meerkat, minimal):
gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py
Pystone(1.1) time for 50000 passes = 0.57
This machine benchmarks at 87719.3 pystones/second

NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.


Pypy 1.4
========
wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2
gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py
Pystone(1.1) time for 50000 passes = 0.15
This machine benchmarks at 333333 pystones/second
gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py 5000000
Pystone(1.1) time for 5000000 passes = 4.8
This machine benchmarks at 1.04167e+06 pystones/second

NOTE: Also not built from source. Here we run pystone a second time with 100x more loops and get a substantially better number.


Shedskin 0.7
============
sudo apt-get install g++ libpcre3-dev libgc-dev python-dev
sudo dpkg -i shedskin_0.7_all.deb
wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz
shedskin pystone.py
gokr@quigon:~/python/shedskin-examples-0.7$ make
g++  -O2 -march=native -fomit-frame-pointer -Wno-deprecated  -I. -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre  -o pystone
gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la pystone*
-rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone
-rw-r--r-- 1 gokr gokr   9193 2011-02-07 23:00 pystone.cpp
-rw-r--r-- 1 gokr gokr   1893 2011-02-07 23:00 pystone.hpp
-rw-r--r-- 1 gokr gokr   5774 2010-12-11 11:40 pystone.py
gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
This machine benchmarks at 2500000.000000 pystones/second

NOTE: I am wondering a bit about this. It tells the same whatever loops I give it... But ok, perhaps it is all fine.


Summary
=======

- The regular Squeak VM has not been overrun by CPython in these 6 years time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post anymore). Now they are equal more or less, Squeak still a teeny bit faster.

- Cog is brutally fast on this one. Compared to CPython and regular Squeak almost 10x faster.

- Pypy is about 20% faster than Cog if given enough time to actually start jitting. Cool for the Pypy project! And cool that they aren't that much faster than Cog. :)

- Shedskin is the "state of the art" of statically compiling Python via C++ using type inferencing etc etc - so I hear. It is said to be faster than Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I say "only" because that seems pretty good to me given that Cog is a JIT and still pretty young and that Shedskin can only run a subset of Python.


regards, Göran


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Squeak vs Python "smack down"

Levente Uzonyi-2
 
On Mon, 7 Feb 2011, Eliot Miranda wrote:

> Hi Göran,
>
>    is it possible to repeat the runs with a much higher iteration count so
> that we get less temporal quantization?  The numbers below are extremely
> round simply because the number of iterations are so low (on modern
> hardware).  Also, is the source available?  URL?  Ta!

The code is available on SqueakMap:
http://map.squeak.org/package/81a1617a-228e-4fcf-80fe-fada0ccde940/autoversion/1


Levente

>
> thanks; this has brightened my day!!
>
> best
> Eliot
>
> 2011/2/7 Göran Krampe <[hidden email]>
>
>> Hi folks!
>>
>> Since we are on the verge of 4.2, and we have a brand new Cog VM to play
>> with I felt like dusting off my old Pystone port to Squeak - Sqystone, which
>> I wrote back in 2004.
>>
>> At that time Squeak was around 5 times faster than CPython. How do we stack
>> up today? Yeah, I know - hardly a good benchmark, they all lie etc etc. :)
>>
>> I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.
>>
>>
>> Cog!
>> ====
>> Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary
>> download):
>> Pystone(1.1) time for 50000 passes = 0.06
>> This machine benchmarks at 833333.3 pystones/second
>>
>> NOTE: AFAICT running with more passes does not improve it. Also, not sure
>> if I could get more out of this if I built from source on my box.
>>
>>
>> Regular Squeak
>> ==============
>> Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357
>> (built from src):
>> Pystone(1.1) time for 50000 passes = 0.503
>> This machine benchmarks at 99403.6 pystones/second
>>
>>
>> Regular CPython
>> ===============
>> CPython 3.1.2 (newest in Ubuntu Meerkat, minimal):
>> gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py
>> Pystone(1.1) time for 50000 passes = 0.57
>> This machine benchmarks at 87719.3 pystones/second
>>
>> NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
>>
>>
>> Pypy 1.4
>> ========
>> wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2
>> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy
>> ./lib-python/2.5.2/test/pystone.py
>> Pystone(1.1) time for 50000 passes = 0.15
>> This machine benchmarks at 333333 pystones/second
>> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy
>> ./lib-python/2.5.2/test/pystone.py 5000000
>> Pystone(1.1) time for 5000000 passes = 4.8
>> This machine benchmarks at 1.04167e+06 pystones/second
>>
>> NOTE: Also not built from source. Here we run pystone a second time with
>> 100x more loops and get a substantially better number.
>>
>>
>> Shedskin 0.7
>> ============
>> sudo apt-get install g++ libpcre3-dev libgc-dev python-dev
>> sudo dpkg -i shedskin_0.7_all.deb
>> wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz
>> shedskin pystone.py
>> gokr@quigon:~/python/shedskin-examples-0.7$ make
>> g++  -O2 -march=native -fomit-frame-pointer -Wno-deprecated  -I.
>> -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp
>> /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre
>>  -o pystone
>> gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la
>> pystone*
>> -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone
>> -rw-r--r-- 1 gokr gokr   9193 2011-02-07 23:00 pystone.cpp
>> -rw-r--r-- 1 gokr gokr   1893 2011-02-07 23:00 pystone.hpp
>> -rw-r--r-- 1 gokr gokr   5774 2010-12-11 11:40 pystone.py
>> gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
>> This machine benchmarks at 2500000.000000 pystones/second
>>
>> NOTE: I am wondering a bit about this. It tells the same whatever loops I
>> give it... But ok, perhaps it is all fine.
>>
>>
>> Summary
>> =======
>>
>> - The regular Squeak VM has not been overrun by CPython in these 6 years
>> time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post
>> anymore). Now they are equal more or less, Squeak still a teeny bit faster.
>>
>> - Cog is brutally fast on this one. Compared to CPython and regular Squeak
>> almost 10x faster.
>>
>> - Pypy is about 20% faster than Cog if given enough time to actually start
>> jitting. Cool for the Pypy project! And cool that they aren't that much
>> faster than Cog. :)
>>
>> - Shedskin is the "state of the art" of statically compiling Python via C++
>> using type inferencing etc etc - so I hear. It is said to be faster than
>> Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I
>> say "only" because that seems pretty good to me given that Cog is a JIT and
>> still pretty young and that Shedskin can only run a subset of Python.
>>
>>
>> regards, Göran
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Squeak vs Python "smack down"

Göran Krampe
In reply to this post by Göran Krampe
 
One small correction:

Rerunning Cog and CPython too with 5000000 loops didn't change the
numbers much, but for Shedskin it turns out more proper:

gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
This machine benchmarks at 3546099.290780 pystones/second

50000 loops was just too small. So in fact it is about 4.4x faster than
Cog. And yes, the ported pystone for shedskin had the loops hard coded. :)

regards, Göran
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Squeak vs Python "smack down"

stephane ducasse-2
In reply to this post by Göran Krampe

This is really interesting to see that type inferencing can really help.
This is definitively a topic I would like to explore in the coming years.

Stef


> Hi folks!
>
> Since we are on the verge of 4.2, and we have a brand new Cog VM to play with I felt like dusting off my old Pystone port to Squeak - Sqystone, which I wrote back in 2004.
>
> At that time Squeak was around 5 times faster than CPython. How do we stack up today? Yeah, I know - hardly a good benchmark, they all lie etc etc. :)
>
> I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.
>
>
> Cog!
> ====
> Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download):
> Pystone(1.1) time for 50000 passes = 0.06
> This machine benchmarks at 833333.3 pystones/second
>
> NOTE: AFAICT running with more passes does not improve it. Also, not sure if I could get more out of this if I built from source on my box.
>
>
> Regular Squeak
> ==============
> Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357 (built from src):
> Pystone(1.1) time for 50000 passes = 0.503
> This machine benchmarks at 99403.6 pystones/second
>
>
> Regular CPython
> ===============
> CPython 3.1.2 (newest in Ubuntu Meerkat, minimal):
> gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py
> Pystone(1.1) time for 50000 passes = 0.57
> This machine benchmarks at 87719.3 pystones/second
>
> NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
>
>
> Pypy 1.4
> ========
> wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2
> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py
> Pystone(1.1) time for 50000 passes = 0.15
> This machine benchmarks at 333333 pystones/second
> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py 5000000
> Pystone(1.1) time for 5000000 passes = 4.8
> This machine benchmarks at 1.04167e+06 pystones/second
>
> NOTE: Also not built from source. Here we run pystone a second time with 100x more loops and get a substantially better number.
>
>
> Shedskin 0.7
> ============
> sudo apt-get install g++ libpcre3-dev libgc-dev python-dev
> sudo dpkg -i shedskin_0.7_all.deb
> wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz
> shedskin pystone.py
> gokr@quigon:~/python/shedskin-examples-0.7$ make
> g++  -O2 -march=native -fomit-frame-pointer -Wno-deprecated  -I. -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre  -o pystone
> gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la pystone*
> -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone
> -rw-r--r-- 1 gokr gokr   9193 2011-02-07 23:00 pystone.cpp
> -rw-r--r-- 1 gokr gokr   1893 2011-02-07 23:00 pystone.hpp
> -rw-r--r-- 1 gokr gokr   5774 2010-12-11 11:40 pystone.py
> gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
> This machine benchmarks at 2500000.000000 pystones/second
>
> NOTE: I am wondering a bit about this. It tells the same whatever loops I give it... But ok, perhaps it is all fine.
>
>
> Summary
> =======
>
> - The regular Squeak VM has not been overrun by CPython in these 6 years time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post anymore). Now they are equal more or less, Squeak still a teeny bit faster.
>
> - Cog is brutally fast on this one. Compared to CPython and regular Squeak almost 10x faster.
>
> - Pypy is about 20% faster than Cog if given enough time to actually start jitting. Cool for the Pypy project! And cool that they aren't that much faster than Cog. :)
>
> - Shedskin is the "state of the art" of statically compiling Python via C++ using type inferencing etc etc - so I hear. It is said to be faster than Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I say "only" because that seems pretty good to me given that Cog is a JIT and still pretty young and that Shedskin can only run a subset of Python.
>
>
> regards, Göran
>


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Squeak vs Python "smack down"

Nicolas Cellier

Just a detail, Python range(a,b) are semi open [a,b)
think like C for(i=0;i<n; i++)

In the Smalltalk proc8:withwith:with:with:: the range was interpreted as [a,b].
This Smalltalk (and Cog) unfairly perform too much work.

Nicolas

2011/2/8 stephane ducasse <[hidden email]>:

>
> This is really interesting to see that type inferencing can really help.
> This is definitively a topic I would like to explore in the coming years.
>
> Stef
>
>
>> Hi folks!
>>
>> Since we are on the verge of 4.2, and we have a brand new Cog VM to play with I felt like dusting off my old Pystone port to Squeak - Sqystone, which I wrote back in 2004.
>>
>> At that time Squeak was around 5 times faster than CPython. How do we stack up today? Yeah, I know - hardly a good benchmark, they all lie etc etc. :)
>>
>> I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.
>>
>>
>> Cog!
>> ====
>> Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download):
>> Pystone(1.1) time for 50000 passes = 0.06
>> This machine benchmarks at 833333.3 pystones/second
>>
>> NOTE: AFAICT running with more passes does not improve it. Also, not sure if I could get more out of this if I built from source on my box.
>>
>>
>> Regular Squeak
>> ==============
>> Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357 (built from src):
>> Pystone(1.1) time for 50000 passes = 0.503
>> This machine benchmarks at 99403.6 pystones/second
>>
>>
>> Regular CPython
>> ===============
>> CPython 3.1.2 (newest in Ubuntu Meerkat, minimal):
>> gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py
>> Pystone(1.1) time for 50000 passes = 0.57
>> This machine benchmarks at 87719.3 pystones/second
>>
>> NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
>>
>>
>> Pypy 1.4
>> ========
>> wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2
>> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py
>> Pystone(1.1) time for 50000 passes = 0.15
>> This machine benchmarks at 333333 pystones/second
>> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py 5000000
>> Pystone(1.1) time for 5000000 passes = 4.8
>> This machine benchmarks at 1.04167e+06 pystones/second
>>
>> NOTE: Also not built from source. Here we run pystone a second time with 100x more loops and get a substantially better number.
>>
>>
>> Shedskin 0.7
>> ============
>> sudo apt-get install g++ libpcre3-dev libgc-dev python-dev
>> sudo dpkg -i shedskin_0.7_all.deb
>> wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz
>> shedskin pystone.py
>> gokr@quigon:~/python/shedskin-examples-0.7$ make
>> g++  -O2 -march=native -fomit-frame-pointer -Wno-deprecated  -I. -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre  -o pystone
>> gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la pystone*
>> -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone
>> -rw-r--r-- 1 gokr gokr   9193 2011-02-07 23:00 pystone.cpp
>> -rw-r--r-- 1 gokr gokr   1893 2011-02-07 23:00 pystone.hpp
>> -rw-r--r-- 1 gokr gokr   5774 2010-12-11 11:40 pystone.py
>> gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
>> This machine benchmarks at 2500000.000000 pystones/second
>>
>> NOTE: I am wondering a bit about this. It tells the same whatever loops I give it... But ok, perhaps it is all fine.
>>
>>
>> Summary
>> =======
>>
>> - The regular Squeak VM has not been overrun by CPython in these 6 years time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post anymore). Now they are equal more or less, Squeak still a teeny bit faster.
>>
>> - Cog is brutally fast on this one. Compared to CPython and regular Squeak almost 10x faster.
>>
>> - Pypy is about 20% faster than Cog if given enough time to actually start jitting. Cool for the Pypy project! And cool that they aren't that much faster than Cog. :)
>>
>> - Shedskin is the "state of the art" of statically compiling Python via C++ using type inferencing etc etc - so I hear. It is said to be faster than Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I say "only" because that seems pretty good to me given that Cog is a JIT and still pretty young and that Shedskin can only run a subset of Python.
>>
>>
>> regards, Göran
>>
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Squeak vs Python "smack down"

Levente Uzonyi-2
 
On Tue, 8 Feb 2011, Nicolas Cellier wrote:

>
> Just a detail, Python range(a,b) are semi open [a,b)
> think like C for(i=0;i<n; i++)
>
> In the Smalltalk proc8:withwith:with:with:: the range was interpreted as [a,b].
> This Smalltalk (and Cog) unfairly perform too much work.

And there's another one. In #proc0:block: the loops are (1 to: loops) do:
instead of 1 to: loops do:.


Levente

>
> Nicolas
>
> 2011/2/8 stephane ducasse <[hidden email]>:
>>
>> This is really interesting to see that type inferencing can really help.
>> This is definitively a topic I would like to explore in the coming years.
>>
>> Stef
>>
>>
>>> Hi folks!
>>>
>>> Since we are on the verge of 4.2, and we have a brand new Cog VM to play with I felt like dusting off my old Pystone port to Squeak - Sqystone, which I wrote back in 2004.
>>>
>>> At that time Squeak was around 5 times faster than CPython. How do we stack up today? Yeah, I know - hardly a good benchmark, they all lie etc etc. :)
>>>
>>> I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.
>>>
>>>
>>> Cog!
>>> ====
>>> Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download):
>>> Pystone(1.1) time for 50000 passes = 0.06
>>> This machine benchmarks at 833333.3 pystones/second
>>>
>>> NOTE: AFAICT running with more passes does not improve it. Also, not sure if I could get more out of this if I built from source on my box.
>>>
>>>
>>> Regular Squeak
>>> ==============
>>> Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357 (built from src):
>>> Pystone(1.1) time for 50000 passes = 0.503
>>> This machine benchmarks at 99403.6 pystones/second
>>>
>>>
>>> Regular CPython
>>> ===============
>>> CPython 3.1.2 (newest in Ubuntu Meerkat, minimal):
>>> gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py
>>> Pystone(1.1) time for 50000 passes = 0.57
>>> This machine benchmarks at 87719.3 pystones/second
>>>
>>> NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
>>>
>>>
>>> Pypy 1.4
>>> ========
>>> wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2
>>> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py
>>> Pystone(1.1) time for 50000 passes = 0.15
>>> This machine benchmarks at 333333 pystones/second
>>> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py 5000000
>>> Pystone(1.1) time for 5000000 passes = 4.8
>>> This machine benchmarks at 1.04167e+06 pystones/second
>>>
>>> NOTE: Also not built from source. Here we run pystone a second time with 100x more loops and get a substantially better number.
>>>
>>>
>>> Shedskin 0.7
>>> ============
>>> sudo apt-get install g++ libpcre3-dev libgc-dev python-dev
>>> sudo dpkg -i shedskin_0.7_all.deb
>>> wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz
>>> shedskin pystone.py
>>> gokr@quigon:~/python/shedskin-examples-0.7$ make
>>> g++  -O2 -march=native -fomit-frame-pointer -Wno-deprecated  -I. -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre  -o pystone
>>> gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la pystone*
>>> -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone
>>> -rw-r--r-- 1 gokr gokr   9193 2011-02-07 23:00 pystone.cpp
>>> -rw-r--r-- 1 gokr gokr   1893 2011-02-07 23:00 pystone.hpp
>>> -rw-r--r-- 1 gokr gokr   5774 2010-12-11 11:40 pystone.py
>>> gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
>>> This machine benchmarks at 2500000.000000 pystones/second
>>>
>>> NOTE: I am wondering a bit about this. It tells the same whatever loops I give it... But ok, perhaps it is all fine.
>>>
>>>
>>> Summary
>>> =======
>>>
>>> - The regular Squeak VM has not been overrun by CPython in these 6 years time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post anymore). Now they are equal more or less, Squeak still a teeny bit faster.
>>>
>>> - Cog is brutally fast on this one. Compared to CPython and regular Squeak almost 10x faster.
>>>
>>> - Pypy is about 20% faster than Cog if given enough time to actually start jitting. Cool for the Pypy project! And cool that they aren't that much faster than Cog. :)
>>>
>>> - Shedskin is the "state of the art" of statically compiling Python via C++ using type inferencing etc etc - so I hear. It is said to be faster than Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I say "only" because that seems pretty good to me given that Cog is a JIT and still pretty young and that Shedskin can only run a subset of Python.
>>>
>>>
>>> regards, Göran
>>>
>>
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Squeak vs Python "smack down"

Nicolas Cellier

2011/2/8 Levente Uzonyi <[hidden email]>:

>
> On Tue, 8 Feb 2011, Nicolas Cellier wrote:
>
>>
>> Just a detail, Python range(a,b) are semi open [a,b)
>> think like C for(i=0;i<n; i++)
>>
>> In the Smalltalk proc8:withwith:with:with:: the range was interpreted as [a,b].
>> This Smalltalk (and Cog) unfairly perform too much work.
>
> And there's another one. In #proc0:block: the loops are (1 to: loops) do: instead of 1 to: loops do:.
>
>
> Levente
>

Yes I noticed, but empty loop time is subtracted.
I wonder if it makes a difference.

Nicolas

>>
>> Nicolas
>>
>> 2011/2/8 stephane ducasse <[hidden email]>:
>>>
>>> This is really interesting to see that type inferencing can really help.
>>> This is definitively a topic I would like to explore in the coming years.
>>>
>>> Stef
>>>
>>>
>>>> Hi folks!
>>>>
>>>> Since we are on the verge of 4.2, and we have a brand new Cog VM to play with I felt like dusting off my old Pystone port to Squeak - Sqystone, which I wrote back in 2004.
>>>>
>>>> At that time Squeak was around 5 times faster than CPython. How do we stack up today? Yeah, I know - hardly a good benchmark, they all lie etc etc. :)
>>>>
>>>> I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.
>>>>
>>>>
>>>> Cog!
>>>> ====
>>>> Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download):
>>>> Pystone(1.1) time for 50000 passes = 0.06
>>>> This machine benchmarks at 833333.3 pystones/second
>>>>
>>>> NOTE: AFAICT running with more passes does not improve it. Also, not sure if I could get more out of this if I built from source on my box.
>>>>
>>>>
>>>> Regular Squeak
>>>> ==============
>>>> Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357 (built from src):
>>>> Pystone(1.1) time for 50000 passes = 0.503
>>>> This machine benchmarks at 99403.6 pystones/second
>>>>
>>>>
>>>> Regular CPython
>>>> ===============
>>>> CPython 3.1.2 (newest in Ubuntu Meerkat, minimal):
>>>> gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py
>>>> Pystone(1.1) time for 50000 passes = 0.57
>>>> This machine benchmarks at 87719.3 pystones/second
>>>>
>>>> NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
>>>>
>>>>
>>>> Pypy 1.4
>>>> ========
>>>> wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2
>>>> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py
>>>> Pystone(1.1) time for 50000 passes = 0.15
>>>> This machine benchmarks at 333333 pystones/second
>>>> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py 5000000
>>>> Pystone(1.1) time for 5000000 passes = 4.8
>>>> This machine benchmarks at 1.04167e+06 pystones/second
>>>>
>>>> NOTE: Also not built from source. Here we run pystone a second time with 100x more loops and get a substantially better number.
>>>>
>>>>
>>>> Shedskin 0.7
>>>> ============
>>>> sudo apt-get install g++ libpcre3-dev libgc-dev python-dev
>>>> sudo dpkg -i shedskin_0.7_all.deb
>>>> wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz
>>>> shedskin pystone.py
>>>> gokr@quigon:~/python/shedskin-examples-0.7$ make
>>>> g++  -O2 -march=native -fomit-frame-pointer -Wno-deprecated  -I. -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre  -o pystone
>>>> gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la pystone*
>>>> -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone
>>>> -rw-r--r-- 1 gokr gokr   9193 2011-02-07 23:00 pystone.cpp
>>>> -rw-r--r-- 1 gokr gokr   1893 2011-02-07 23:00 pystone.hpp
>>>> -rw-r--r-- 1 gokr gokr   5774 2010-12-11 11:40 pystone.py
>>>> gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
>>>> This machine benchmarks at 2500000.000000 pystones/second
>>>>
>>>> NOTE: I am wondering a bit about this. It tells the same whatever loops I give it... But ok, perhaps it is all fine.
>>>>
>>>>
>>>> Summary
>>>> =======
>>>>
>>>> - The regular Squeak VM has not been overrun by CPython in these 6 years time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post anymore). Now they are equal more or less, Squeak still a teeny bit faster.
>>>>
>>>> - Cog is brutally fast on this one. Compared to CPython and regular Squeak almost 10x faster.
>>>>
>>>> - Pypy is about 20% faster than Cog if given enough time to actually start jitting. Cool for the Pypy project! And cool that they aren't that much faster than Cog. :)
>>>>
>>>> - Shedskin is the "state of the art" of statically compiling Python via C++ using type inferencing etc etc - so I hear. It is said to be faster than Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I say "only" because that seems pretty good to me given that Cog is a JIT and still pretty young and that Shedskin can only run a subset of Python.
>>>>
>>>>
>>>> regards, Göran
>>>>
>>>
>>>
>>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Squeak vs Python "smack down"

Levente Uzonyi-2
 
On Tue, 8 Feb 2011, Nicolas Cellier wrote:

>
> 2011/2/8 Levente Uzonyi <[hidden email]>:
>>
>> On Tue, 8 Feb 2011, Nicolas Cellier wrote:
>>
>>>
>>> Just a detail, Python range(a,b) are semi open [a,b)
>>> think like C for(i=0;i<n; i++)
>>>
>>> In the Smalltalk proc8:withwith:with:with:: the range was interpreted as [a,b].
>>> This Smalltalk (and Cog) unfairly perform too much work.
>>
>> And there's another one. In #proc0:block: the loops are (1 to: loops) do: instead of 1 to: loops do:.
>>
>>
>> Levente
>>
>
> Yes I noticed, but empty loop time is subtracted.
> I wonder if it makes a difference.
~10% on my pc using Cog r2361. Probably the blocks make the difference.


Levente

>
> Nicolas
>
>>>
>>> Nicolas
>>>
>>> 2011/2/8 stephane ducasse <[hidden email]>:
>>>>
>>>> This is really interesting to see that type inferencing can really help.
>>>> This is definitively a topic I would like to explore in the coming years.
>>>>
>>>> Stef
>>>>
>>>>
>>>>> Hi folks!
>>>>>
>>>>> Since we are on the verge of 4.2, and we have a brand new Cog VM to play with I felt like dusting off my old Pystone port to Squeak - Sqystone, which I wrote back in 2004.
>>>>>
>>>>> At that time Squeak was around 5 times faster than CPython. How do we stack up today? Yeah, I know - hardly a good benchmark, they all lie etc etc. :)
>>>>>
>>>>> I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.
>>>>>
>>>>>
>>>>> Cog!
>>>>> ====
>>>>> Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download):
>>>>> Pystone(1.1) time for 50000 passes = 0.06
>>>>> This machine benchmarks at 833333.3 pystones/second
>>>>>
>>>>> NOTE: AFAICT running with more passes does not improve it. Also, not sure if I could get more out of this if I built from source on my box.
>>>>>
>>>>>
>>>>> Regular Squeak
>>>>> ==============
>>>>> Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357 (built from src):
>>>>> Pystone(1.1) time for 50000 passes = 0.503
>>>>> This machine benchmarks at 99403.6 pystones/second
>>>>>
>>>>>
>>>>> Regular CPython
>>>>> ===============
>>>>> CPython 3.1.2 (newest in Ubuntu Meerkat, minimal):
>>>>> gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py
>>>>> Pystone(1.1) time for 50000 passes = 0.57
>>>>> This machine benchmarks at 87719.3 pystones/second
>>>>>
>>>>> NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
>>>>>
>>>>>
>>>>> Pypy 1.4
>>>>> ========
>>>>> wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2
>>>>> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py
>>>>> Pystone(1.1) time for 50000 passes = 0.15
>>>>> This machine benchmarks at 333333 pystones/second
>>>>> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py 5000000
>>>>> Pystone(1.1) time for 5000000 passes = 4.8
>>>>> This machine benchmarks at 1.04167e+06 pystones/second
>>>>>
>>>>> NOTE: Also not built from source. Here we run pystone a second time with 100x more loops and get a substantially better number.
>>>>>
>>>>>
>>>>> Shedskin 0.7
>>>>> ============
>>>>> sudo apt-get install g++ libpcre3-dev libgc-dev python-dev
>>>>> sudo dpkg -i shedskin_0.7_all.deb
>>>>> wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz
>>>>> shedskin pystone.py
>>>>> gokr@quigon:~/python/shedskin-examples-0.7$ make
>>>>> g++  -O2 -march=native -fomit-frame-pointer -Wno-deprecated  -I. -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre  -o pystone
>>>>> gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la pystone*
>>>>> -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone
>>>>> -rw-r--r-- 1 gokr gokr   9193 2011-02-07 23:00 pystone.cpp
>>>>> -rw-r--r-- 1 gokr gokr   1893 2011-02-07 23:00 pystone.hpp
>>>>> -rw-r--r-- 1 gokr gokr   5774 2010-12-11 11:40 pystone.py
>>>>> gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
>>>>> This machine benchmarks at 2500000.000000 pystones/second
>>>>>
>>>>> NOTE: I am wondering a bit about this. It tells the same whatever loops I give it... But ok, perhaps it is all fine.
>>>>>
>>>>>
>>>>> Summary
>>>>> =======
>>>>>
>>>>> - The regular Squeak VM has not been overrun by CPython in these 6 years time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post anymore). Now they are equal more or less, Squeak still a teeny bit faster.
>>>>>
>>>>> - Cog is brutally fast on this one. Compared to CPython and regular Squeak almost 10x faster.
>>>>>
>>>>> - Pypy is about 20% faster than Cog if given enough time to actually start jitting. Cool for the Pypy project! And cool that they aren't that much faster than Cog. :)
>>>>>
>>>>> - Shedskin is the "state of the art" of statically compiling Python via C++ using type inferencing etc etc - so I hear. It is said to be faster than Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I say "only" because that seems pretty good to me given that Cog is a JIT and still pretty young and that Shedskin can only run a subset of Python.
>>>>>
>>>>>
>>>>> regards, Göran
>>>>>
>>>>
>>>>
>>>>
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Squeak vs Python "smack down"

Eliot Miranda-2
In reply to this post by Nicolas Cellier
 


On Tue, Feb 8, 2011 at 5:01 AM, Nicolas Cellier <[hidden email]> wrote:

2011/2/8 Levente Uzonyi <[hidden email]>:
>
> On Tue, 8 Feb 2011, Nicolas Cellier wrote:
>
>>
>> Just a detail, Python range(a,b) are semi open [a,b)
>> think like C for(i=0;i<n; i++)
>>
>> In the Smalltalk proc8:withwith:with:with:: the range was interpreted as [a,b].
>> This Smalltalk (and Cog) unfairly perform too much work.
>
> And there's another one. In #proc0:block: the loops are (1 to: loops) do: instead of 1 to: loops do:.
>
>
> Levente
>

Yes I noticed, but empty loop time is subtracted.
I wonder if it makes a difference.

There is a huge difference between 1 to: loops do: [:i|] and (1 to: loops) do: [:i|].  IIRC the empty block is written as 1 to: loops do: [:i|].

SimpleStackBasedCogit:
Time millisecondsToRun:
[| loops |
loops := 100000000.
1 to: loops do: [:i|]] 757

Time millisecondsToRun:
[| loops |
loops := 100000000.
(1 to: loops) do: [:i|]] 16953

StackToRegisterMappingCogit
Time millisecondsToRun:
[| loops |
loops := 100000000.
1 to: loops do: [:i|]] 244

Time millisecondsToRun:
[| loops |
loops := 100000000.
(1 to: loops) do: [:i|]] 17149



Nicolas

>>
>> Nicolas
>>
>> 2011/2/8 stephane ducasse <[hidden email]>:
>>>
>>> This is really interesting to see that type inferencing can really help.
>>> This is definitively a topic I would like to explore in the coming years.
>>>
>>> Stef
>>>
>>>
>>>> Hi folks!
>>>>
>>>> Since we are on the verge of 4.2, and we have a brand new Cog VM to play with I felt like dusting off my old Pystone port to Squeak - Sqystone, which I wrote back in 2004.
>>>>
>>>> At that time Squeak was around 5 times faster than CPython. How do we stack up today? Yeah, I know - hardly a good benchmark, they all lie etc etc. :)
>>>>
>>>> I am using Ubuntu 10.10 on a corei7, so this is running on a 64 bit CPU.
>>>>
>>>>
>>>> Cog!
>>>> ====
>>>> Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary download):
>>>> Pystone(1.1) time for 50000 passes = 0.06
>>>> This machine benchmarks at 833333.3 pystones/second
>>>>
>>>> NOTE: AFAICT running with more passes does not improve it. Also, not sure if I could get more out of this if I built from source on my box.
>>>>
>>>>
>>>> Regular Squeak
>>>> ==============
>>>> Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357 (built from src):
>>>> Pystone(1.1) time for 50000 passes = 0.503
>>>> This machine benchmarks at 99403.6 pystones/second
>>>>
>>>>
>>>> Regular CPython
>>>> ===============
>>>> CPython 3.1.2 (newest in Ubuntu Meerkat, minimal):
>>>> gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py
>>>> Pystone(1.1) time for 50000 passes = 0.57
>>>> This machine benchmarks at 87719.3 pystones/second
>>>>
>>>> NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
>>>>
>>>>
>>>> Pypy 1.4
>>>> ========
>>>> wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2
>>>> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py
>>>> Pystone(1.1) time for 50000 passes = 0.15
>>>> This machine benchmarks at 333333 pystones/second
>>>> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy ./lib-python/2.5.2/test/pystone.py 5000000
>>>> Pystone(1.1) time for 5000000 passes = 4.8
>>>> This machine benchmarks at 1.04167e+06 pystones/second
>>>>
>>>> NOTE: Also not built from source. Here we run pystone a second time with 100x more loops and get a substantially better number.
>>>>
>>>>
>>>> Shedskin 0.7
>>>> ============
>>>> sudo apt-get install g++ libpcre3-dev libgc-dev python-dev
>>>> sudo dpkg -i shedskin_0.7_all.deb
>>>> wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz
>>>> shedskin pystone.py
>>>> gokr@quigon:~/python/shedskin-examples-0.7$ make
>>>> g++  -O2 -march=native -fomit-frame-pointer -Wno-deprecated  -I. -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp pystone.cpp /usr/share/shedskin/lib/time.cpp /usr/share/shedskin/lib/re.cpp -lgc -lpcre  -o pystone
>>>> gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la pystone*
>>>> -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone
>>>> -rw-r--r-- 1 gokr gokr   9193 2011-02-07 23:00 pystone.cpp
>>>> -rw-r--r-- 1 gokr gokr   1893 2011-02-07 23:00 pystone.hpp
>>>> -rw-r--r-- 1 gokr gokr   5774 2010-12-11 11:40 pystone.py
>>>> gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
>>>> This machine benchmarks at 2500000.000000 pystones/second
>>>>
>>>> NOTE: I am wondering a bit about this. It tells the same whatever loops I give it... But ok, perhaps it is all fine.
>>>>
>>>>
>>>> Summary
>>>> =======
>>>>
>>>> - The regular Squeak VM has not been overrun by CPython in these 6 years time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the post anymore). Now they are equal more or less, Squeak still a teeny bit faster.
>>>>
>>>> - Cog is brutally fast on this one. Compared to CPython and regular Squeak almost 10x faster.
>>>>
>>>> - Pypy is about 20% faster than Cog if given enough time to actually start jitting. Cool for the Pypy project! And cool that they aren't that much faster than Cog. :)
>>>>
>>>> - Shedskin is the "state of the art" of statically compiling Python via C++ using type inferencing etc etc - so I hear. It is said to be faster than Cython and Psyco. It ends up beating Cog, but "only" by a factor of 3x. I say "only" because that seems pretty good to me given that Cog is a JIT and still pretty young and that Shedskin can only run a subset of Python.
>>>>
>>>>
>>>> regards, Göran
>>>>
>>>
>>>
>>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Squeak vs Python "smack down"

Nicolas Cellier

2011/2/8 Eliot Miranda <[hidden email]>:

>
>
>
> On Tue, Feb 8, 2011 at 5:01 AM, Nicolas Cellier <[hidden email]> wrote:
>>
>> 2011/2/8 Levente Uzonyi <[hidden email]>:
>> >
>> > On Tue, 8 Feb 2011, Nicolas Cellier wrote:
>> >
>> >>
>> >> Just a detail, Python range(a,b) are semi open [a,b)
>> >> think like C for(i=0;i<n; i++)
>> >>
>> >> In the Smalltalk proc8:withwith:with:with:: the range was interpreted as [a,b].
>> >> This Smalltalk (and Cog) unfairly perform too much work.
>> >
>> > And there's another one. In #proc0:block: the loops are (1 to: loops) do: instead of 1 to: loops do:.
>> >
>> >
>> > Levente
>> >
>>
>> Yes I noticed, but empty loop time is subtracted.
>> I wonder if it makes a difference.
>
> There is a huge difference between 1 to: loops do: [:i|] and (1 to: loops) do: [:i|].  IIRC the empty block is written as 1 to: loops do: [:i|].
> SimpleStackBasedCogit:
> Time millisecondsToRun:
> [| loops |
> loops := 100000000.
> 1 to: loops do: [:i|]] 757
> Time millisecondsToRun:
> [| loops |
> loops := 100000000.
> (1 to: loops) do: [:i|]] 16953
> StackToRegisterMappingCogit
> Time millisecondsToRun:
> [| loops |
> loops := 100000000.
> 1 to: loops do: [:i|]] 244
> Time millisecondsToRun:
> [| loops |
> loops := 100000000.
> (1 to: loops) do: [:i|]] 17149
>

I was more after the difference with empty loops, like:

[| loops x |
loops := 10000000.
x := 1.
(1 to: loops) do: [:i| x := x + 1. x squared]]  timeToRun
-
[| loops |
loops := 10000000.
(1 to: loops) do: [:i| ]]  timeToRun
 5405 5397


[| loops x |
loops := 10000000.
x := 1.
1 to: loops do: [:i| x := x + 1. x squared]]  timeToRun
-
[| loops |
loops := 10000000.
1 to: loops do: [:i| ]]  timeToRun
 5272 5205

Nicolas
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Squeak vs Python "smack down"

Göran Krampe
In reply to this post by Eliot Miranda-2
 
On 02/08/2011 05:26 PM, Eliot Miranda wrote:

> On Tue, Feb 8, 2011 at 5:01 AM, Nicolas Cellier
> <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>
>     2011/2/8 Levente Uzonyi <[hidden email] <mailto:[hidden email]>>:
>      >
>      > On Tue, 8 Feb 2011, Nicolas Cellier wrote:
>      >
>      >>
>      >> Just a detail, Python range(a,b) are semi open [a,b)
>      >> think like C for(i=0;i<n; i++)
>      >>
>      >> In the Smalltalk proc8:withwith:with:with:: the range was
>     interpreted as [a,b].
>      >> This Smalltalk (and Cog) unfairly perform too much work.
>      >
>      > And there's another one. In #proc0:block: the loops are (1 to:
>     loops) do: instead of 1 to: loops do:.
>      >
>      >
>      > Levente
>      >
>
>     Yes I noticed, but empty loop time is subtracted.
>     I wonder if it makes a difference.
>
>
> There is a huge difference between 1 to: loops do: [:i|] and (1 to:
> loops) do: [:i|].  IIRC the empty block is written as 1 to: loops do: [:i|].

When I did the port I did not try to rewrite the code to "proper
Smalltalk style" since it would then typically not compare the VMs. At
least that was my thought (it was a while since I did this).

So I wrote it with *explicit* creation of Intervals since I presumed
Python created ranges - but perhaps Python... never mind, google tells us:

http://wiki.python.org/moin/PythonSpeed/PerformanceTips#Usexrangeinsteadofrange


Aha! So yes, it was a fair comparison before, but now in Python3 they
don't instantiate the range but create an iterator instead. Then we
should change it too I guess :)

I will make a new version of it.

regards, Göran
Reply | Threaded
Open this post in threaded view
|

New numbers! (was Re: [Vm-dev] Squeak vs Python "smack down")

Göran Krampe
In reply to this post by Göran Krampe
 
Hi!

So a few small notes were made by a few people on how Sqystone is
written, so I tweaked it a bit and these are the more correct numbers,
all run with 5000000 loops.

On 02/08/2011 12:01 AM, Göran Krampe wrote:
> Cog!
> ====
> Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary
> download):
> Pystone(1.1) time for 50000 passes = 0.06
> This machine benchmarks at 833333.3 pystones/second

Pystone(1.1) time for 5000000 passes = 5.448
This machine benchmarks at 917768.0 pystones/second

> Regular Squeak
> ==============
> Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357
> (built from src):
> Pystone(1.1) time for 50000 passes = 0.503
> This machine benchmarks at 99403.6 pystones/second

Pystone(1.1) time for 5000000 passes = 46.448
This machine benchmarks at 107647.3 pystones/second

> Regular CPython
> ===============
> CPython 3.1.2 (newest in Ubuntu Meerkat, minimal):
> gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py
> Pystone(1.1) time for 50000 passes = 0.57
> This machine benchmarks at 87719.3 pystones/second
>
> NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.

gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py 5000000
Pystone(1.1) time for 5000000 passes = 58.6
This machine benchmarks at 85324.2 pystones/second


>
> Pypy 1.4
> ========
> wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2
> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy
> ./lib-python/2.5.2/test/pystone.py
> Pystone(1.1) time for 50000 passes = 0.15
> This machine benchmarks at 333333 pystones/second
> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy
> ./lib-python/2.5.2/test/pystone.py 5000000
> Pystone(1.1) time for 5000000 passes = 4.8
> This machine benchmarks at 1.04167e+06 pystones/second
>
> NOTE: Also not built from source. Here we run pystone a second time with
> 100x more loops and get a substantially better number.
>
>
> Shedskin 0.7
> ============
 > sudo apt-get install g++ libpcre3-dev libgc-dev python-dev
 > sudo dpkg -i shedskin_0.7_all.deb
 > wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz
 > shedskin pystone.py
 > gokr@quigon:~/python/shedskin-examples-0.7$ make
 > g++ -O2 -march=native -fomit-frame-pointer -Wno-deprecated -I.
 > -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp
 > pystone.cpp /usr/share/shedskin/lib/time.cpp
 > /usr/share/shedskin/lib/re.cpp -lgc -lpcre -o pystone
 > gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la
 > pystone*
 > -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone
 > -rw-r--r-- 1 gokr gokr 9193 2011-02-07 23:00 pystone.cpp
 > -rw-r--r-- 1 gokr gokr 1893 2011-02-07 23:00 pystone.hpp
 > -rw-r--r-- 1 gokr gokr 5774 2010-12-11 11:40 pystone.py
 > gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
 > This machine benchmarks at 2500000.000000 pystones/second

5000000 loops gave a more precise number:

gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
This machine benchmarks at 3378378.378378 pystones/second

> Summary
> =======
>
> - The regular Squeak VM has not been overrun by CPython in these 6 years
> time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the
> post anymore). Now they are equal more or less, Squeak still a teeny bit
> faster.

Actually a bit more (not latest 3.2 Python used though):

Squeak = 1.28 x CPython

> - Cog is brutally fast on this one. Compared to CPython and regular
> Squeak almost 10x faster.

Cog = 8.4 x Squeak
Cog = 10.75 x CPython

> - Pypy is about 20% faster than Cog if given enough time to actually
> start jitting. Cool for the Pypy project! And cool that they aren't that
> much faster than Cog. :)

Cog = 0.88 x Pypy

> - Shedskin is the "state of the art" of statically compiling Python via
> C++ using type inferencing etc etc - so I hear. It is said to be faster
> than Cython and Psyco. It ends up beating Cog, but "only" by a factor of
> 3x. I say "only" because that seems pretty good to me given that Cog is
> a JIT and still pretty young and that Shedskin can only run a subset of
> Python.

Shedskin = 3.68 x Cog

regards, Göran
Reply | Threaded
Open this post in threaded view
|

Re: New numbers! (was Re: [Vm-dev] Squeak vs Python "smack down")

Bert Freudenberg



On 09.02.2011, at 01:50, Göran Krampe <[hidden email]> wrote:

 Shedskin can only run a subset of
>> Python.
>
> Shedskin = 3.68 x Cog

A Slang version would be the equivalent of this, right?

- Bert -

Reply | Threaded
Open this post in threaded view
|

Re: New numbers! (was Re: [Vm-dev] Squeak vs Python "smack down")

Göran Krampe
 
On 02/09/2011 09:01 AM, Bert Freudenberg wrote:

> On 09.02.2011, at 01:50, Göran Krampe<[hidden email]>  wrote:
>
>   Shedskin can only run a subset of
>>> Python.
>>
>> Shedskin = 3.68 x Cog
>
> A Slang version would be the equivalent of this, right?
>
> - Bert -

Mmmm, well, Shedskin takes Python source as input, does type inference
and produces C++ source, then compiled as normal. But yes, it is a
restricted Python, but AFAICT not at all as restricted as Slang.

So I wouldn't say Slang is really fair - being C in disguise :)

BUT... Unladen Swallow and IronPython might be interesting to test too.
Unladen has a goal of being at least 5x faster than CPython, which in
the light of Cog seems like a rather modest goal. AFAIK it is not fast
at all right now though.

Still, as we all know - Python *is* and probably always will be
*CPython* - all other implementations will be "imitations" with limited
auidences. Now I note there is a py3k-jit branch, but someone mentioned
that it hasn't moved in 7 months.

Pypy is probably the only project that possibly could step in and
"become" a next generation platform for Python - or what do you guys
that have more insight into the Python community than I do?

regards, Göran
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Re: New numbers! (was Re: [Vm-dev] Squeak vs Python "smack down")

stephane ducasse-2


On Feb 9, 2011, at 10:06 AM, Göran Krampe wrote:

> On 02/09/2011 09:01 AM, Bert Freudenberg wrote:
>> On 09.02.2011, at 01:50, Göran Krampe<[hidden email]>  wrote:
>>
>>  Shedskin can only run a subset of
>>>> Python.
>>>
>>> Shedskin = 3.68 x Cog
>>
>> A Slang version would be the equivalent of this, right?
>>
>> - Bert -
>
> Mmmm, well, Shedskin takes Python source as input, does type inference and produces C++ source, then compiled as normal. But yes, it is a restricted Python, but AFAICT not at all as restricted as Slang.


restricted python is really a subset of python so that some type analysis can be performed. So this is clearly not like Slang.

> So I wouldn't say Slang is really fair - being C in disguise :)
>
> BUT... Unladen Swallow and IronPython might be interesting to test too. Unladen has a goal of being at least 5x faster than CPython, which in the light of Cog seems like a rather modest goal. AFAIK it is not fast at all right now though.
>
> Still, as we all know - Python *is* and probably always will be *CPython* - all other implementations will be "imitations" with limited auidences. Now I note there is a py3k-jit branch, but someone mentioned that it hasn't moved in 7 months.
>
> Pypy is probably the only project that possibly could step in and "become" a next generation platform for Python - or what do you guys that have more insight into the Python community than I do?

at least we should have a build system that monitor the performance of the vm as pypy are doing.

>
> regards, Göran
>

Reply | Threaded
Open this post in threaded view
|

Re: New numbers! (was Re: [Vm-dev] Squeak vs Python "smack down")

Stefan Marr
In reply to this post by Göran Krampe

Hi Göran:

Those numbers are certainly fun, but wouldn't it be much more interesting to get the SqueakVM and Cog back into the Language Benchmark Game?

http://shootout.alioth.debian.org/

The Dhrystone benchmarks also do not really have the reputation of being, well, interesting, i.e., kind of resemble standard workloads.

Bringing Cog to the Benchmark game would also allow comparison with for instance LuaJIT2.

Best regards
Stefan

On 09 Feb 2011, at 01:50, Göran Krampe wrote:

> Hi!
>
> So a few small notes were made by a few people on how Sqystone is written, so I tweaked it a bit and these are the more correct numbers, all run with 5000000 loops.
>
> On 02/08/2011 12:01 AM, Göran Krampe wrote:
>> Cog!
>> ====
>> Squeak 4.2-10966 (soon to be released) + latest Cog r2361 (binary
>> download):
>> Pystone(1.1) time for 50000 passes = 0.06
>> This machine benchmarks at 833333.3 pystones/second
>
> Pystone(1.1) time for 5000000 passes = 5.448
> This machine benchmarks at 917768.0 pystones/second
>
>> Regular Squeak
>> ==============
>> Squeak 4.2-10966 (soon to be released) + regular Squeak VM 4.4.7-2357
>> (built from src):
>> Pystone(1.1) time for 50000 passes = 0.503
>> This machine benchmarks at 99403.6 pystones/second
>
> Pystone(1.1) time for 5000000 passes = 46.448
> This machine benchmarks at 107647.3 pystones/second
>
>> Regular CPython
>> ===============
>> CPython 3.1.2 (newest in Ubuntu Meerkat, minimal):
>> gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py
>> Pystone(1.1) time for 50000 passes = 0.57
>> This machine benchmarks at 87719.3 pystones/second
>>
>> NOTE: 3.2 is reportedly a teeny bit faster. Also not built from source.
>
> gokr@quigon:/usr/lib/python3.1/test$ python3.1 pystone.py 5000000
> Pystone(1.1) time for 5000000 passes = 58.6
> This machine benchmarks at 85324.2 pystones/second
>
>
>>
>> Pypy 1.4
>> ========
>> wget http://pypy.org/download/pypy-1.4.1-linux64.tar.bz2
>> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy
>> ./lib-python/2.5.2/test/pystone.py
>> Pystone(1.1) time for 50000 passes = 0.15
>> This machine benchmarks at 333333 pystones/second
>> gokr@quigon:~/python/pypy-1.4.1-linux64$ ./bin/pypy
>> ./lib-python/2.5.2/test/pystone.py 5000000
>> Pystone(1.1) time for 5000000 passes = 4.8
>> This machine benchmarks at 1.04167e+06 pystones/second
>>
>> NOTE: Also not built from source. Here we run pystone a second time with
>> 100x more loops and get a substantially better number.
>>
>>
>> Shedskin 0.7
>> ============
> > sudo apt-get install g++ libpcre3-dev libgc-dev python-dev
> > sudo dpkg -i shedskin_0.7_all.deb
> > wget http://shedskin.googlecode.com/files/shedskin-examples-0.7.tgz
> > shedskin pystone.py
> > gokr@quigon:~/python/shedskin-examples-0.7$ make
> > g++ -O2 -march=native -fomit-frame-pointer -Wno-deprecated -I.
> > -I/usr/share/shedskin/lib /usr/share/shedskin/lib/builtin.cpp
> > pystone.cpp /usr/share/shedskin/lib/time.cpp
> > /usr/share/shedskin/lib/re.cpp -lgc -lpcre -o pystone
> > gokr@quigon:~/python/gokr@quigon:~/python/shedskin-examples-0.7$ ls -la
> > pystone*
> > -rwxr-xr-x 1 gokr gokr 297329 2011-02-07 23:01 pystone
> > -rw-r--r-- 1 gokr gokr 9193 2011-02-07 23:00 pystone.cpp
> > -rw-r--r-- 1 gokr gokr 1893 2011-02-07 23:00 pystone.hpp
> > -rw-r--r-- 1 gokr gokr 5774 2010-12-11 11:40 pystone.py
> > gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
> > This machine benchmarks at 2500000.000000 pystones/second
>
> 5000000 loops gave a more precise number:
>
> gokr@quigon:~/python/shedskin-examples-0.7$ ./pystone
> This machine benchmarks at 3378378.378378 pystones/second
>
>> Summary
>> =======
>>
>> - The regular Squeak VM has not been overrun by CPython in these 6 years
>> time. When I wrote Sqystone Squeak was 5x faster IIRC (can't find the
>> post anymore). Now they are equal more or less, Squeak still a teeny bit
>> faster.
>
> Actually a bit more (not latest 3.2 Python used though):
>
> Squeak = 1.28 x CPython
>
>> - Cog is brutally fast on this one. Compared to CPython and regular
>> Squeak almost 10x faster.
>
> Cog = 8.4 x Squeak
> Cog = 10.75 x CPython
>
>> - Pypy is about 20% faster than Cog if given enough time to actually
>> start jitting. Cool for the Pypy project! And cool that they aren't that
>> much faster than Cog. :)
>
> Cog = 0.88 x Pypy
>
>> - Shedskin is the "state of the art" of statically compiling Python via
>> C++ using type inferencing etc etc - so I hear. It is said to be faster
>> than Cython and Psyco. It ends up beating Cog, but "only" by a factor of
>> 3x. I say "only" because that seems pretty good to me given that Cog is
>> a JIT and still pretty young and that Shedskin can only run a subset of
>> Python.
>
> Shedskin = 3.68 x Cog
>
> regards, Göran

--
Stefan Marr
Software Languages Lab
Vrije Universiteit Brussel
Pleinlaan 2 / B-1050 Brussels / Belgium
http://soft.vub.ac.be/~smarr
Phone: +32 2 629 2974
Fax:   +32 2 629 3525