A tinyBenchmark
Locked
 
|
see http://fbanados.wordpress.com/2011/02/10/a-tinybenchmark
Maybe Eliot is able to say something here on the main differences between Cog <-> VW VM. Thx T. -- NEU: FreePhone - kostenlos mobil telefonieren und surfen! Jetzt informieren: http://www.gmx.net/de/go/freephone |
|
|
On Thu, 10 Feb 2011, Torsten Bergmann wrote:
> see http://fbanados.wordpress.com/2011/02/10/a-tinybenchmark > > Maybe Eliot is able to say something here on the main > differences between Cog <-> VW VM. You made several mistakes: - You didn't post the source code for #tinyBenchmarks. Do all those Smalltalks implement it? Are the implementations equal? - You didn't specify the version numbers for the VMs used. Or if you built them yourself, then the sourcecode + compiler version + configuration parameters. Without these your measurements can't be reproduced. - There are two different Cog VMs. You didn't specify which one did you use with Pharo. - You used a CogVM (JIT) with Pharo but SqueakVM (interpreter) with Squeak 4.2. What's even worse is that you based your conclusion on this mistake. - Does #tinyBenchmarks alone give fair performance comparison? Hardly. Levente > > Thx > T. > -- > NEU: FreePhone - kostenlos mobil telefonieren und surfen! > Jetzt informieren: http://www.gmx.net/de/go/freephone > > |
|
|
On Thu, 10 Feb 2011, Levente Uzonyi wrote:
> On Thu, 10 Feb 2011, Torsten Bergmann wrote: > >> see http://fbanados.wordpress.com/2011/02/10/a-tinybenchmark >> >> Maybe Eliot is able to say something here on the main >> differences between Cog <-> VW VM. > > You made several mistakes: I mean Felipe, not you Torsten. Levente > - You didn't post the source code for #tinyBenchmarks. Do all those > Smalltalks implement it? Are the implementations equal? > - You didn't specify the version numbers for the VMs used. Or if you built > them yourself, then the sourcecode + compiler version + configuration > parameters. Without these your measurements can't be reproduced. > - There are two different Cog VMs. You didn't specify which one did you use > with Pharo. > - You used a CogVM (JIT) with Pharo but SqueakVM (interpreter) with Squeak > 4.2. What's even worse is that you based your conclusion on this mistake. > - Does #tinyBenchmarks alone give fair performance comparison? Hardly. > > > Levente > >> >> Thx >> T. >> -- >> NEU: FreePhone - kostenlos mobil telefonieren und surfen! >> Jetzt informieren: http://www.gmx.net/de/go/freephone >> >> > > |
|
|
In reply to this post by Levente Uzonyi-2
Hi Everybody! This is my first post on the list, even though i've been subscribed for more than 6 months.
2011/2/10 Levente Uzonyi <[hidden email]>
No, they don't implement it. Just Pharo and Squeak. I just added the same implementation to Integer in VA, VisualWorks and GNU Smalltalk ( Integer>>tinyBenchmarks, Integer>>benchmark and Integer>>benchFib), so the source code is the same.
- You didn't specify the version numbers for the VMs used. Or if you built them yourself, then the sourcecode + compiler version + configuration parameters. Without these your measurements can't be reproduced. Every System was the one available from their websites. (Got Pharo 1.2 from Hudson about 3 days ago.) However, VisualWorks is not available from its website right now. I installed it from a Cincom CD-ROM I got on Smalltalks 2010 .
- There are two different Cog VMs. You didn't specify which one did you use with Pharo. Smalltalk vmVersion returns 'Croquet Closure Cog VM [CoInterpreter VMMaker-oscog.35]').
- You used a CogVM (JIT) with Pharo but SqueakVM (interpreter) with Squeak 4.2. What's even worse is that you based your conclusion on this mistake. Yes, it is true. Daniel Galdames also pointed this out last night on the blog. But as I told before, it is just the benchmarks you get from the "download the latest version available" approach. I couldn't find a Cog-based Squeak on its ftp site, (perhaps I was too lazy). I won't have the time to repeat this until next week, then I'll try this.
- Does #tinyBenchmarks alone give fair performance comparison? Hardly. Well, it was the benchmark I knew of. Do you know a better performance benchmark available? If so, I could retry the experiment.
-- Felipe Bañados Schwerter Estudiante de Ingeniería Civil en Computación http://www.dcc.uchile.cl/~fbanados Universidad de Chile |
|
|
Squeak 4.2 is cog ready |
|
|
In reply to this post by Felipe Bañados Schwerter
On 10 Feb 2011, at 17:45, Felipe Bañados Schwerter wrote: > Every System was the one available from their websites. (Got Pharo 1.2 from Hudson about 3 days ago.) However, VisualWorks is not available from its website right now. I installed it from a Cincom CD-ROM I got on Smalltalks 2010 . > > - There are two different Cog VMs. You didn't specify which one did you use with Pharo. > > Smalltalk vmVersion returns 'Croquet Closure Cog VM [CoInterpreter VMMaker-oscog.35]'). +1 There does indeed not seem to be a very good way to get at the proper VM version (let alone build) from inside the Smalltalk image. It frustrates me too. > - You used a CogVM (JIT) with Pharo but SqueakVM (interpreter) with Squeak 4.2. What's even worse is that you based your conclusion on this mistake. > > Yes, it is true. Daniel Galdames also pointed this out last night on the blog. But as I told before, it is just the benchmarks you get from the "download the latest version available" approach. I couldn't find a Cog-based Squeak on its ftp site, (perhaps I was too lazy). I won't have the time to repeat this until next week, then I'll try this. Again, you are right: even though Squeak 4.2 is ready for Cog, there is (not yet) a simple one-click installer using it (as far as I can tell from a casual look). So your experience mirrors what some others might find. Benchmarking is always fuzzy, you picked one that was available and has been there for years, so if it would be no good, somebody should have replaced it. Regards, Sven |
|
|
In reply to this post by Felipe Bañados Schwerter
2011/2/10 Felipe Bañados Schwerter <[hidden email]> Hi Everybody! This is my first post on the list, even though i've been subscribed for more than 6 months. But Smalltalk getSystemAttribute: 1008 returns the JIT version, which is much more important for this performance measurement.
As large a benchmark that is as close to your working case as possible. tinyBenchmarks is little more than a smoke test (which is what we use it for at work; if tinyBenchmarks hasn't plummeted ten the build is OK performance-wise).
In the Cog-Benchmarks (in Cog package) that I uploaded to squeaksource recently you'll find ShootoutTests which has 4 tests from the computer language shootouts that are pure Smalltalk.
|
|
|
In reply to this post by Felipe Bañados Schwerter
Welcome Felipe!
stef |
|
|
In reply to this post by Felipe Bañados Schwerter
On Feb 10, 2011, at 10:18 AM, Eliot Miranda wrote: > > In the Cog-Benchmarks (in Cog package) that I uploaded to squeaksource recently you'll find ShootoutTests which has 4 tests from the computer language shootouts that are pure Smalltalk. > Camillo and Toon did a nice benchmarks framework for Pinnoccio... we should look at it. And we need something like this: http://speed.pypy.org/timeline/ Marcus -- Marcus Denker -- http://www.marcusdenker.de INRIA Lille -- Nord Europe. Team RMoD. |
|
|
On 10 Feb 2011, at 19:49, Marcus Denker wrote: > > On Feb 10, 2011, at 10:18 AM, Eliot Miranda wrote: >> >> In the Cog-Benchmarks (in Cog package) that I uploaded to squeaksource recently you'll find ShootoutTests which has 4 tests from the computer language shootouts that are pure Smalltalk. >> > > Camillo and Toon did a nice benchmarks framework for Pinnoccio... we should look at it. > > And we need something like this: http://speed.pypy.org/timeline/ This is impressive. Doru > Marcus > > -- > Marcus Denker -- http://www.marcusdenker.de > INRIA Lille -- Nord Europe. Team RMoD. > > -- www.tudorgirba.com "To lead is not to demand things, it is to make them happen." |
|
|
>>> In the Cog-Benchmarks (in Cog package) that I uploaded to squeaksource recently you'll find ShootoutTests which has 4 tests from the computer language shootouts that are pure Smalltalk. >>> >> >> Camillo and Toon did a nice benchmarks framework for Pinnoccio... we should look at it. >> >> And we need something like this: http://speed.pypy.org/timeline/ > > This is impressive. Yes and it gives a strong feeling. We should do the same. Stef |
|
|
In reply to this post by Felipe Bañados Schwerter
On Thu, 10 Feb 2011, Felipe Ba?ados Schwerter wrote:
> Hi Everybody! This is my first post on the list, even though i've been > subscribed for more than 6 months. > > 2011/2/10 Levente Uzonyi <[hidden email]> > >> On Thu, 10 Feb 2011, Torsten Bergmann wrote: >> >> see http://fbanados.wordpress.com/2011/02/10/a-tinybenchmark >>> >>> Maybe Eliot is able to say something here on the main >>> differences between Cog <-> VW VM. >>> >> >> You made several mistakes: >> - You didn't post the source code for #tinyBenchmarks. Do all those >> Smalltalks implement it? Are the implementations equal? >> > > No, they don't implement it. Just Pharo and Squeak. I just added the same > implementation to Integer in VA, VisualWorks and GNU Smalltalk ( > Integer>>tinyBenchmarks, Integer>>benchmark and Integer>>benchFib), so the > source code is the same. had to rewrite the code to make it run on both of them. > > - You didn't specify the version numbers for the VMs used. Or if you built >> them yourself, then the sourcecode + compiler version + configuration >> parameters. Without these your measurements can't be reproduced. >> > > Every System was the one available from their websites. (Got Pharo 1.2 from > Hudson about 3 days ago.) However, VisualWorks is not available from its > website right now. I installed it from a Cincom CD-ROM I got on Smalltalks > 2010 . > > >> - There are two different Cog VMs. You didn't specify which one did you use >> with Pharo. >> > > Smalltalk vmVersion returns 'Croquet Closure Cog VM [CoInterpreter > VMMaker-oscog.35]'). So it's SimpleStackBasedCogit. If you use StackToRegisterMappingCogit you'll get much better results. > > >> - You used a CogVM (JIT) with Pharo but SqueakVM (interpreter) with Squeak >> 4.2. What's even worse is that you based your conclusion on this mistake. >> > > Yes, it is true. Daniel Galdames also pointed this out last night on the > blog. But as I told before, it is just the benchmarks you get from the > "download the latest version available" approach. I couldn't find a > Cog-based Squeak on its ftp site, (perhaps I was too lazy). I won't have the > time to repeat this until next week, then I'll try this. though you have to dig deep to download it from the Pharo website. Finding Pharo 1.1.1 is obvious from there :). Anyway it doesn't really matter which image you use for #tinyBenchmarks, because it's a VM benchmark. The image can hardly affect it's results. > > >> - Does #tinyBenchmarks alone give fair performance comparison? Hardly. >> >> Well, it was the benchmark I knew of. Do you know a better performance > benchmark available? If so, I could retry the experiment. See what Eliot suggested. Levente > > >> >> Levente >> >> >> >>> Thx >>> T. >>> -- >>> NEU: FreePhone - kostenlos mobil telefonieren und surfen! >>> Jetzt informieren: http://www.gmx.net/de/go/freephone >>> >>> >>> >> > > > -- > Felipe Ba?ados Schwerter > Estudiante de Ingeniería Civil en Computación > http://www.dcc.uchile.cl/~fbanados > Universidad de Chile > |
|
|
In reply to this post by Sven Van Caekenberghe
On Thu, 10 Feb 2011, Sven Van Caekenberghe wrote:
> > On 10 Feb 2011, at 17:45, Felipe Ba?ados Schwerter wrote: > >> Every System was the one available from their websites. (Got Pharo 1.2 from Hudson about 3 days ago.) However, VisualWorks is not available from its website right now. I installed it from a Cincom CD-ROM I got on Smalltalks 2010 . >> >> - There are two different Cog VMs. You didn't specify which one did you use with Pharo. >> >> Smalltalk vmVersion returns 'Croquet Closure Cog VM [CoInterpreter VMMaker-oscog.35]'). > > +1 > > There does indeed not seem to be a very good way to get at the proper VM version (let alone build) from inside the Smalltalk image. It frustrates me too. These methods should do it: Smalltalk buildDate. 'Win32 built on Feb 6 2011 18:06:24 Compiler: 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)'. Smalltalk isRunningCog. true. Smalltalk isRunningCogit. true. Smalltalk interpreterClass. 'CoInterpreter VMMaker-oscog.47 uuid: a4053f20-cb87-415d-aad3-3bb29b03724c Feb 6 2011'. Smalltalk cogitClass. 'StackToRegisterMappingCogit VMMaker-oscog.47 uuid: a4053f20-cb87-415d-aad3-3bb29b03724c Feb 6 2011'. > >> - You used a CogVM (JIT) with Pharo but SqueakVM (interpreter) with Squeak 4.2. What's even worse is that you based your conclusion on this mistake. >> >> Yes, it is true. Daniel Galdames also pointed this out last night on the blog. But as I told before, it is just the benchmarks you get from the "download the latest version available" approach. I couldn't find a Cog-based Squeak on its ftp site, (perhaps I was too lazy). I won't have the time to repeat this until next week, then I'll try this. > > Again, you are right: even though Squeak 4.2 is ready for Cog, there is (not yet) a simple one-click installer using it (as far as I can tell from a casual look). So your experience mirrors what some others might find. There will be when it will be released. ;) > > Benchmarking is always fuzzy, you picked one that was available and has been there for years, so if it would be no good, somebody should have replaced it. It's fine for benchmarking the VM. As the name suggests, it's not a system wide benchmark, just a tiny one. Levente > > Regards, > > Sven > > > > > > |
|
|
In reply to this post by Sven Van Caekenberghe
On Thu, Feb 10, 2011 at 07:12:03PM +0100, Sven Van Caekenberghe wrote:
> > On 10 Feb 2011, at 17:45, Felipe Ba?ados Schwerter wrote: > > > Every System was the one available from their websites. (Got Pharo 1.2 from Hudson about 3 days ago.) However, VisualWorks is not available from its website right now. I installed it from a Cincom CD-ROM I got on Smalltalks 2010 . > > > > - There are two different Cog VMs. You didn't specify which one did you use with Pharo. > > > > Smalltalk vmVersion returns 'Croquet Closure Cog VM [CoInterpreter VMMaker-oscog.35]'). > > +1 > > There does indeed not seem to be a very good way to get at the proper > VM version (let alone build) from inside the Smalltalk image. It frustrates me too. > Check out SystemReporter on SqueakSource. It's being used in Squeak now and AFIK should load in Pharo also. Very convenient, and tells you everything you need to know about VM version and lots of other things. Dave |
|
|
so much noise for so insignificant smoke test.
I'd rather prefer seeing Cog doing copy-down compiled method, at least for most simple cases - self sends :) -- Best regards, Igor Stasenko AKA sig. |
|
|
On Fri, 11 Feb 2011, Igor Stasenko wrote:
> so much noise for so insignificant smoke test. > I'd rather prefer seeing Cog doing copy-down compiled method, at least > for most simple cases - self sends :) Do you mean inlining? Levente > > > -- > Best regards, > Igor Stasenko AKA sig. > > |
|
|
In reply to this post by David T. Lewis
it is on the radar
http://code.google.com/p/pharo/issues/detail?id=3546 Stef On Feb 11, 2011, at 2:03 AM, David T. Lewis wrote: > On Thu, Feb 10, 2011 at 07:12:03PM +0100, Sven Van Caekenberghe wrote: >> >> On 10 Feb 2011, at 17:45, Felipe Ba?ados Schwerter wrote: >> >>> Every System was the one available from their websites. (Got Pharo 1.2 from Hudson about 3 days ago.) However, VisualWorks is not available from its website right now. I installed it from a Cincom CD-ROM I got on Smalltalks 2010 . >>> >>> - There are two different Cog VMs. You didn't specify which one did you use with Pharo. >>> >>> Smalltalk vmVersion returns 'Croquet Closure Cog VM [CoInterpreter VMMaker-oscog.35]'). >> >> +1 >> >> There does indeed not seem to be a very good way to get at the proper >> VM version (let alone build) from inside the Smalltalk image. It frustrates me too. >> > > Check out SystemReporter on SqueakSource. It's being used in Squeak now > and AFIK should load in Pharo also. Very convenient, and tells you everything > you need to know about VM version and lots of other things. > > Dave > > |
|
|
In reply to this post by Levente Uzonyi-2
On 11 February 2011 02:52, Levente Uzonyi <[hidden email]> wrote:
> On Fri, 11 Feb 2011, Igor Stasenko wrote: > >> so much noise for so insignificant smoke test. >> I'd rather prefer seeing Cog doing copy-down compiled method, at least >> for most simple cases - self sends :) > > Do you mean inlining? > > yes > Levente > >> >> >> -- >> Best regards, >> Igor Stasenko AKA sig. >> >> > > -- Best regards, Igor Stasenko AKA sig. |
«
Return to Pharo Smalltalk Developers
|
1 view|%1 views
| Free forum by Nabble | Edit this page |