A small benchmark comparing Pharo with Amber in speed
Locked
A small benchmark comparing Pharo with Amber in speed
 
|
A very small benchmark
-- [(1 to: 1000000) do:[:each|1 + each]] timeToRun Takes for Pharo 9 milliseconds Takes for Amber 7337 milliseconds . Thats 815 times slower than Pharo.
Both run inside Workspace. I assume a fair comparison for Amber would be to turn this into a method , commit it as a package so it compiles to javascript and then call it inside Workspace ?
Or am I doing it completely the wrong way ? You received this message because you are subscribed to the Google Groups "amber-lang" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Re: A small benchmark comparing Pharo with Amber in speed
|
It always compiles to method doIt anyway and run as such.
The main difference here is (1 to: 1000000). In Pharo, it is an Interval object. In Amber, no Interval objects are present, so NUmber >> to: fakes it by creating the actual Array (pretty big one in your example). Try #to:do: for fairer comparision. kilon alios wrote: > A very small benchmark > > [(1 to: 1000000) do:[:each|1 + each]] timeToRun > > Takes for Pharo 9 milliseconds > > Takes for Amber 7337 milliseconds . Thats 815 times slower than Pharo. > > Both run inside Workspace. > > I assume a fair comparison for Amber would be to turn this into a > method , commit it as a package so it compiles to javascript and then > call it inside Workspace ? > > Or am I doing it completely the wrong way ? > > -- > You received this message because you are subscribed to the Google > Groups "amber-lang" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to amber-lang+unsubscribe@goog > <mailto:[hidden email]>. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "amber-lang" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Re: A small benchmark comparing Pharo with Amber in speed
|
|
I don't understand #to: do: , can you show me the actual code ? On Wed, Jul 2, 2014 at 11:54 PM, Herby Vojčík <[hidden email]> wrote: It always compiles to method doIt anyway and run as such. You received this message because you are subscribed to the Google Groups "amber-lang" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Re: A small benchmark comparing Pharo with Amber in speed
|
|
I assume you mean this [1 to: 1000000 do:[:each|1 + each]] timeToRun, it gives me 4219 . Still very slow though around half time than the previous effort. "It always compiles to method doIt anyway and run as such."
you mean it compiles to javascript anyway ? On Thu, Jul 3, 2014 at 12:00 AM, kilon alios <[hidden email]> wrote:
You received this message because you are subscribed to the Google Groups "amber-lang" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Re: A small benchmark comparing Pharo with Amber in speed
|
|
kilon alios <[hidden email]> writes: > I assume you mean this > > [1 to: 1000000 do:[:each|1 + each]] timeToRun, it gives me 4219 . Still > very slow though around half time than the previous effort. > > "It always compiles to method doIt anyway and run as such." > > you mean it compiles to javascript anyway ? Yes, it does. Now with the current compiler and all the debug information it generates, you will get slower results than Pharo. I'm working on a variation of the compiler that will produce deployment-ready code without any context information/source code. It will be significantly faster than the current one. Nico > > > On Thu, Jul 3, 2014 at 12:00 AM, kilon alios <[hidden email]> wrote: > >> I don't understand #to: do: , can you show me the actual code ? >> >> >> On Wed, Jul 2, 2014 at 11:54 PM, Herby Vojčík <[hidden email]> wrote: >> >>> It always compiles to method doIt anyway and run as such. >>> >>> The main difference here is (1 to: 1000000). >>> In Pharo, it is an Interval object. >>> In Amber, no Interval objects are present, so NUmber >> to: fakes it by >>> creating the actual Array >>> (pretty big one in your example). >>> >>> Try #to:do: for fairer comparision. >>> >>> kilon alios wrote: >>> >>>> A very small benchmark >>>> >>>> [(1 to: 1000000) do:[:each|1 + each]] timeToRun >>>> >>>> Takes for Pharo 9 milliseconds >>>> >>>> Takes for Amber 7337 milliseconds . Thats 815 times slower than Pharo. >>>> >>>> Both run inside Workspace. >>>> >>>> I assume a fair comparison for Amber would be to turn this into a method >>>> , commit it as a package so it compiles to javascript and then call it >>>> inside Workspace ? >>>> >>>> Or am I doing it completely the wrong way ? >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "amber-lang" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to amber-lang+unsubscribe@goog >>>> >>> legroups.com >>> >>>> <mailto:[hidden email]>. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "amber-lang" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [hidden email]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- Nicolas Petton http://nicolas-petton.fr -- You received this message because you are subscribed to the Google Groups "amber-lang" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Re: A small benchmark comparing Pharo with Amber in speed
|
|
Thank you Niko for your efforts. On Thu, Jul 3, 2014 at 11:57 AM, Nicolas Petton <[hidden email]> wrote:
You received this message because you are subscribed to the Google Groups "amber-lang" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Re: A small benchmark comparing Pharo with Amber in speed
|
|
In reply to this post by kilon.alios
2014-07-02 23:14 GMT+02:00 kilon alios <[hidden email]>:
1 to: 1000000 do: [:each|1 + each] is also faster than (1 to: 1000000) do: [:each|1 + each]
in Pharo because the block is inlined at compile-time with #to:do: to a loop. On my computer: In Pharo: [1 to: 1000000 do:[:each|1 + each]] timeToRun 2
[(1 to: 1000000) do:[:each|1 + each]] timeToRun 11 In Amber:
[1 to: 1000000 do:[:each|1 + each]] timeToRun 6683 [(1 to: 1000000) do:[:each|1 + each]] timeToRun 10215 In Amber (from http://amber-lang.net/) the block seems never to be inlined at compile time, Amber always relies on BlockClosure>>#whileTrue: which calls the javascript loop. In addition the integer addition is not inlined by default as shown below: Compiler new compile: (Benchfib >> #benchLoop) source forClass: Benchfib 'smalltalk.method({
selector: "benchLoop", source: "benchLoop\x0a ^ [1 to: 1000000 do:[:each|1 + each]] timeToRun ",
fn: function(){ var self=this; return smalltalk.withContext(function($ctx1) {
var $1; $1=_st((function(){ return smalltalk.withContext(function($ctx2) {
return (1)._to_do_((1000000),(function(each){ return smalltalk.withContext(function($ctx3) { return (1).__plus(each);
}, function($ctx3) {$ctx3.fillBlock({each:each},$ctx2,2)})})); }, function($ctx2) {$ctx2.fillBlock({},$ctx1,1)})}))._timeToRun();
return $1; }, function($ctx1) {$ctx1.fill(self,"benchLoop",{},smalltalk.Benchfib)})}, messageSends: ["timeToRun", "to:do:", "+"],
args: [], referencedClasses: []})' V8 from Chrome is supposed to inline the different methods to find out that it can cast the Numbers to int32, and then use faster machine code operations. It seems that inlining does not work in this case, probably because V8 has issue with closures with escaping variables such as the functions to recreate contexts in Amber.
If you try: benchLoopJS < for (var i=0 ; i < 1000000; i++ ) {i + 1} >
which is similar to [1 to: 1000000 do:[:each|1 + each]] timeToRun for Pharo (as #to:do: is statically inlined in Pharo), then the timeToRun gives you 1ms in Chrome, which is (as expected) faster than Pharo. We are currently improving Pharo's JIT for this kind of cases :-).
I guess you would reach this speed in Amber if its compiler would inline at compile-time Number>>#+ and Number>>#to:do:, because then V8 JIT's could easily cast the Numbers to int32.
Now this is is a microbench Kilon. So we all know it does not mean anything for real application. Clement
You received this message because you are subscribed to the Google Groups "amber-lang" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Re: A small benchmark comparing Pharo with Amber in speed
|
|
yes exactly this is why I did not took my benchmark seriously. In a real scenario most likely I would be coding the demanding parts in pure Javascript anyway. Afterall calling Js from Amber is a lot easier than calling C from Pharo ;)
On Thu, Jul 3, 2014 at 12:18 PM, Clément Bera <[hidden email]> wrote:
You received this message because you are subscribed to the Google Groups "amber-lang" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Re: A small benchmark comparing Pharo with Amber in speed
|
|
In reply to this post by Clément Béra
Yes, we could do it (and had it in the past). Amber doesn't inline at
compile time arithmetic operations (nor boolean operations). Nico Clément Bera <[hidden email]> writes: > 2014-07-02 23:14 GMT+02:00 kilon alios <[hidden email]>: > >> I assume you mean this >> >> [1 to: 1000000 do:[:each|1 + each]] timeToRun, it gives me 4219 . Still >> very slow though around half time than the previous effort. >> >> "It always compiles to method doIt anyway and run as such." >> >> you mean it compiles to javascript anyway ? >> > > 1 to: 1000000 do: [:each|1 + each] > is also faster than > (1 to: 1000000) do: [:each|1 + each] > in Pharo because the block is inlined at compile-time with #to:do: to a > loop. > > On my computer: > > In Pharo: > [1 to: 1000000 do:[:each|1 + each]] timeToRun 2 > [(1 to: 1000000) do:[:each|1 + each]] timeToRun 11 > > In Amber: > [1 to: 1000000 do:[:each|1 + each]] timeToRun 6683 > [(1 to: 1000000) do:[:each|1 + each]] timeToRun 10215 > > In Amber (from http://amber-lang.net/) the block seems never to be inlined > at compile time, Amber always relies on BlockClosure>>#whileTrue: which > calls the javascript loop. In addition the integer addition is not inlined > by default as shown below: > > Compiler new compile: (Benchfib >> #benchLoop) source forClass: Benchfib > 'smalltalk.method({ > selector: "benchLoop", > source: "benchLoop\x0a ^ [1 to: 1000000 do:[:each|1 + each]] timeToRun ", > fn: function(){ > var self=this; > return smalltalk.withContext(function($ctx1) { > var $1; > $1=_st((function(){ > return smalltalk.withContext(function($ctx2) { > return (1)._to_do_((1000000),(function(each){ > return smalltalk.withContext(function($ctx3) { > return (1).__plus(each); > }, function($ctx3) {$ctx3.fillBlock({each:each},$ctx2,2)})})); > }, function($ctx2) {$ctx2.fillBlock({},$ctx1,1)})}))._timeToRun(); > return $1; > }, function($ctx1) {$ctx1.fill(self,"benchLoop",{},smalltalk.Benchfib)})}, > messageSends: ["timeToRun", "to:do:", "+"], > args: [], > referencedClasses: []})' > > V8 from Chrome is supposed to inline the different methods to find out that > it can cast the Numbers to int32, and then use faster machine code > operations. It seems that inlining does not work in this case, probably > because V8 has issue with closures with escaping variables such as the > functions to recreate contexts in Amber. > > If you try: > > benchLoopJS > < for (var i=0 ; i < 1000000; i++ ) {i + 1} > > > which is similar to > [1 to: 1000000 do:[:each|1 + each]] timeToRun > for Pharo (as #to:do: is statically inlined in Pharo), then the timeToRun > gives you 1ms in Chrome, which is (as expected) faster than Pharo. We are > currently improving Pharo's JIT for this kind of cases :-). > > I guess you would reach this speed in Amber if its compiler would inline at > compile-time Number>>#+ and Number>>#to:do:, because then V8 JIT's could > easily cast the Numbers to int32. > > Now this is is a microbench Kilon. So we all know it does not mean anything > for real application. > > Clement > >> >> >> On Thu, Jul 3, 2014 at 12:00 AM, kilon alios <[hidden email]> >> wrote: >> >>> I don't understand #to: do: , can you show me the actual code ? >>> >>> >>> On Wed, Jul 2, 2014 at 11:54 PM, Herby Vojčík <[hidden email]> wrote: >>> >>>> It always compiles to method doIt anyway and run as such. >>>> >>>> The main difference here is (1 to: 1000000). >>>> In Pharo, it is an Interval object. >>>> In Amber, no Interval objects are present, so NUmber >> to: fakes it by >>>> creating the actual Array >>>> (pretty big one in your example). >>>> >>>> Try #to:do: for fairer comparision. >>>> >>>> kilon alios wrote: >>>> >>>>> A very small benchmark >>>>> >>>>> [(1 to: 1000000) do:[:each|1 + each]] timeToRun >>>>> >>>>> Takes for Pharo 9 milliseconds >>>>> >>>>> Takes for Amber 7337 milliseconds . Thats 815 times slower than Pharo. >>>>> >>>>> Both run inside Workspace. >>>>> >>>>> I assume a fair comparison for Amber would be to turn this into a >>>>> method , commit it as a package so it compiles to javascript and then call >>>>> it inside Workspace ? >>>>> >>>>> Or am I doing it completely the wrong way ? >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "amber-lang" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to amber-lang+unsubscribe@goog >>>>> >>>> legroups.com >>>> >>>>> <mailto:[hidden email]>. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "amber-lang" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [hidden email]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "amber-lang" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [hidden email]. >> For more options, visit https://groups.google.com/d/optout. >> -- Nicolas Petton http://nicolas-petton.fr -- You received this message because you are subscribed to the Google Groups "amber-lang" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Re: A small benchmark comparing Pharo with Amber in speed
|
|
In reply to this post by kilon.alios
kilon alios <[hidden email]> writes: > yes exactly this is why I did not took my benchmark seriously. In a real > scenario most likely I would be coding the demanding parts in pure > Javascript anyway. Afterall calling Js from Amber is a lot easier than > calling C from Pharo ;) Well, I never needed to do this. Amber has always been fast enough for all the applications I wrote. Sure, arithmetic operations are slower (and there's a penalty for context information as well), but it has never been an issue for me. Now as I said, there will be a deployment wizard in Helios someday which will remove all the context info and source code, migitating this overhead. Nico > > > On Thu, Jul 3, 2014 at 12:18 PM, Clément Bera <[hidden email]> > wrote: > >> >> >> >> 2014-07-02 23:14 GMT+02:00 kilon alios <[hidden email]>: >> >> I assume you mean this >>> >>> [1 to: 1000000 do:[:each|1 + each]] timeToRun, it gives me 4219 . Still >>> very slow though around half time than the previous effort. >>> >>> "It always compiles to method doIt anyway and run as such." >>> >>> you mean it compiles to javascript anyway ? >>> >> >> 1 to: 1000000 do: [:each|1 + each] >> is also faster than >> (1 to: 1000000) do: [:each|1 + each] >> in Pharo because the block is inlined at compile-time with #to:do: to a >> loop. >> >> On my computer: >> >> In Pharo: >> [1 to: 1000000 do:[:each|1 + each]] timeToRun 2 >> [(1 to: 1000000) do:[:each|1 + each]] timeToRun 11 >> >> In Amber: >> [1 to: 1000000 do:[:each|1 + each]] timeToRun 6683 >> [(1 to: 1000000) do:[:each|1 + each]] timeToRun 10215 >> >> In Amber (from http://amber-lang.net/) the block seems never to be >> inlined at compile time, Amber always relies on BlockClosure>>#whileTrue: >> which calls the javascript loop. In addition the integer addition is not >> inlined by default as shown below: >> >> Compiler new compile: (Benchfib >> #benchLoop) source forClass: Benchfib >> 'smalltalk.method({ >> selector: "benchLoop", >> source: "benchLoop\x0a ^ [1 to: 1000000 do:[:each|1 + each]] timeToRun ", >> fn: function(){ >> var self=this; >> return smalltalk.withContext(function($ctx1) { >> var $1; >> $1=_st((function(){ >> return smalltalk.withContext(function($ctx2) { >> return (1)._to_do_((1000000),(function(each){ >> return smalltalk.withContext(function($ctx3) { >> return (1).__plus(each); >> }, function($ctx3) {$ctx3.fillBlock({each:each},$ctx2,2)})})); >> }, function($ctx2) {$ctx2.fillBlock({},$ctx1,1)})}))._timeToRun(); >> return $1; >> }, function($ctx1) {$ctx1.fill(self,"benchLoop",{},smalltalk.Benchfib)})}, >> messageSends: ["timeToRun", "to:do:", "+"], >> args: [], >> referencedClasses: []})' >> >> V8 from Chrome is supposed to inline the different methods to find out >> that it can cast the Numbers to int32, and then use faster machine code >> operations. It seems that inlining does not work in this case, probably >> because V8 has issue with closures with escaping variables such as the >> functions to recreate contexts in Amber. >> >> If you try: >> >> benchLoopJS >> < for (var i=0 ; i < 1000000; i++ ) {i + 1} > >> >> which is similar to >> [1 to: 1000000 do:[:each|1 + each]] timeToRun >> for Pharo (as #to:do: is statically inlined in Pharo), then the timeToRun >> gives you 1ms in Chrome, which is (as expected) faster than Pharo. We are >> currently improving Pharo's JIT for this kind of cases :-). >> >> I guess you would reach this speed in Amber if its compiler would inline >> at compile-time Number>>#+ and Number>>#to:do:, because then V8 JIT's could >> easily cast the Numbers to int32. >> >> Now this is is a microbench Kilon. So we all know it does not mean >> anything for real application. >> >> Clement >> >>> >>> >>> On Thu, Jul 3, 2014 at 12:00 AM, kilon alios <[hidden email]> >>> wrote: >>> >>>> I don't understand #to: do: , can you show me the actual code ? >>>> >>>> >>>> On Wed, Jul 2, 2014 at 11:54 PM, Herby Vojčík <[hidden email]> wrote: >>>> >>>>> It always compiles to method doIt anyway and run as such. >>>>> >>>>> The main difference here is (1 to: 1000000). >>>>> In Pharo, it is an Interval object. >>>>> In Amber, no Interval objects are present, so NUmber >> to: fakes it by >>>>> creating the actual Array >>>>> (pretty big one in your example). >>>>> >>>>> Try #to:do: for fairer comparision. >>>>> >>>>> kilon alios wrote: >>>>> >>>>>> A very small benchmark >>>>>> >>>>>> [(1 to: 1000000) do:[:each|1 + each]] timeToRun >>>>>> >>>>>> Takes for Pharo 9 milliseconds >>>>>> >>>>>> Takes for Amber 7337 milliseconds . Thats 815 times slower than Pharo. >>>>>> >>>>>> Both run inside Workspace. >>>>>> >>>>>> I assume a fair comparison for Amber would be to turn this into a >>>>>> method , commit it as a package so it compiles to javascript and then call >>>>>> it inside Workspace ? >>>>>> >>>>>> Or am I doing it completely the wrong way ? >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "amber-lang" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>> an email to amber-lang+unsubscribe@goog >>>>>> >>>>> legroups.com >>>>> >>>>>> <mailto:[hidden email]>. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "amber-lang" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [hidden email]. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "amber-lang" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [hidden email]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "amber-lang" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [hidden email]. >> For more options, visit https://groups.google.com/d/optout. >> -- Nicolas Petton http://nicolas-petton.fr -- You received this message because you are subscribed to the Google Groups "amber-lang" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
«
Return to Amber Smalltalk
|
1 view|%1 views
| Free forum by Nabble | Edit this page |