RoarVM: The Manycore SqueakVM

> RoarVM is based on the work of David Ungar and Sam S. Adams at IBM Research.
> The port to x86 multicore systems was done by me. They open-sourced their VM,
> formerly know as Renaissance VM (RVM), under the Eclipse Public License [1].
> Official announcement of the IBM source code release:
>    http://soft.vub.ac.be/~smarr/rvm-open-source-release/
>
> The source code of the RoarVM has been released as open source to enable the
> Smalltalk community to evaluate the ideas and possibly integrate them into
> existing systems. So, the RoarVM is meant to experiment with Smalltalk systems
> on multi- and manycore machines.

> On 11/3/2010 10:27 AM, Stefan Marr wrote:
>> On 03 Nov 2010, at 17:57, Andreas Raab wrote:
>>>> The source code of the RoarVM has been released as open source to enable the
>>>> Smalltalk community to evaluate the ideas and possibly integrate them into
>>>> existing systems. So, the RoarVM is meant to experiment with Smalltalk systems
>>>> on multi- and manycore machines.
>>>
>>> Can you give us a high-level overview about what "the trick" is? I.e., which approach did you take to make this possible?
>> Ehm, sorry, I do not really know where to start.
>> Could you be a bit more specific with your question?
>
> Well, I was hoping you could tell us where to start. After all you know more about this stuff than we do :-)

> !Integer methodsFor: 'benchmarks' stamp: 'sm 10/11/2010 22:30'!
> tinyBenchmarksParallel16Processes
> "Report the results of running the two tiny Squeak benchmarks.
> ar 9/10/1999: Adjusted to run at least 1 sec to get more stable results"
> "0 tinyBenchmarks"
> "On a 292 MHz G3 Mac: 22727272 bytecodes/sec; 984169 sends/sec"
> "On a 400 MHz PII/Win98:  18028169 bytecodes/sec; 1081272 sends/sec"
> | t1 t2 r n1 n2 |
> n1 := 1.
> [t1 := Time millisecondsToRun: [n1 benchmark].
> t1 < 1000] whileTrue:[n1 := n1 * 2]. "Note: #benchmark's runtime is about O(n)"
>
> "now n1 is the value for which we do the measurement"
> t1 := Time millisecondsToRun: [self run: #benchmark on: n1 times: 16].
>
> n2 := 28.
> [t2 := Time millisecondsToRun: [r := n2 benchFib].
> t2 < 1000] whileTrue:[n2 := n2 + 1].
> "Note: #benchFib's runtime is about O(k^n),
> where k is the golden number = (1 + 5 sqrt) / 2 = 1.618...."
>
> "now we have our target n2 and r value.
> lets take the time for it"
> t2 := Time millisecondsToRun: [self run: #benchFib on: n2 times: 16].
>
> ^ { ((n1 * 16 * 500000 * 1000) // t1). " printString, ' bytecodes/sec; ',"
>   ((r * 16 * 1000) // t2) " printString, ' sends/sec'"
>   }! !
> !Integer methodsFor: 'benchmarks' stamp: 'sm 10/11/2010 22:29'!
> run: aSymbol on: aReceiver times: nTimes
> | mtx sig n |
>
> mtx := Semaphore forMutualExclusion.
> sig := Semaphore new.
> n := nTimes.
>
> nTimes timesRepeat: [
> [ aReceiver perform: aSymbol.
> mtx critical: [
> n := n - 1.
> (n == 0) ifTrue: [sig signal]]
> ] fork
> ].
> sig wait.! !

>
> On 03.11.2010, at 14:13, Stefan Marr wrote:
>
>> A small teaser:
>>  1 core   66286897 bytecodes/sec;  2910474 sends/sec
>>  8 cores 470588235 bytecodes/sec; 19825677 sends/sec
>
> I tried your precompiled OS X VM and the Sly3 image.
>
> 1 core:  93,910,491 bytecodes/sec; 4,056,440 sends/sec
> 2 cores: 91,559,370 bytecodes/sec; 4,007,927 sends/sec
> 3 cores: can't start
> 4 cores: 90,844,570 bytecodes/sec; 3,935,516 sends/sec
> 5 cores: can't start
> 6 cores: can't start
> 7 cores: can't start
> 8 cores: 89,698,668 bytecodes/sec; 3,910,787 sends/sec
>
> So it looks like you have to use a power-of-two cores?
>
> And the benchmark invocation should be different if you want to actually use multiple cores. What's the magic incantation?
>
> I tried something myself:
>
> n := 16.
> q := SharedQueue new.
> time := Time millisecondsToRun:
>        [n timesRepeat: [[q nextPut: [30 benchFib] timeToRun] fork].
>        n timesRepeat: [Transcript space; show: q next]].
> Transcript space; show: time; cr
>
> 1 core:  664 664 665 666 667 662 664 664 668 665 667 665 666 669 666 10700
> 2 cores: 675 674 672 669 677 669 669 672 678 670 668 669 674 668 668 5425
> 4 cores: 721 726 729 740 713 728 740 734 731 737 721 737 734 756 788 749 3030
> 8 cores: 786 807 837 847 865 872 916 840 800 873 792 880 846 865 829 1820
>
> Now that scales pretty nicely :) The overhead is about 25% at 8 cores, 12% for 4 cores.
>

> On 04 Nov 2010, at 19:07, Bert Freudenberg wrote:
>
>> So it looks like you have to use a power-of-two cores?
> Yes, that is right. At the moment, the system isn't able to handle other numbers of cores.
>
>
>> And the benchmark invocation should be different if you want to actually use multiple cores. What's the magic incantation?
> The code I used to generate the numbers isn't actually in any image yet.
> I pasted it below for reference, its just a quick hack to have a parallel tinyBenchmarks version.
>
>
>> So RoarVM is about 4 times slower in sends, even more so for bytecodes. It needs 8 cores to be faster the regular interpreter on a single core. To the good news is that it can beat the old interpreter :)  But why is it so much slower than the normal interpreter?
> Well, one the one hand, we don't use stuff like the GCC label-as-value extension to have threaded-interpretation, which should help quite a bit.
> Then, the current implementation based on pthreads is quite a bit slower then our version which uses plain Unix processes.
> The GC is really not state of the art.
> And all that adds up rather quickly I suppose...

>>>> Btw, user interrupt didn't work on the Mac.
>>> Cmd+. ? Works for me ;) Well, can you be a bit more specific? In which situation did it not work?
>>
>> I was doing the equivalent of
>>
>> SharedQueue new next
>>
>> and that seems not interruptable. Also, when there are multiple processes, closing the window does not quit all processes, and even ctrl-c did not quit the VM.
>
> Closing the window, how does that relate to processes? You mean a window inside the image?
> Per se, processes are not really owned or managed, so there is also nobody kill processes. However, I am not sure what you are referring to exactly.  

>>>> And in the Squeak-4.1 image, when running on 2 or more cores Morphic gets incredibly sluggish, pretty much unusably so.
>>> Yes, same here. Sorry. Any hints where to start looking to fix such issues are appreciated.
>>
>> Hmm. There are long freezes of many seconds and I would have no idea where to start even ...
> Ok, several seconds, hm. That does not really sound like the GC pauses I see.
> But I haven't used the Squeak image a lot myself on the RoarVM.
> I was more thinking in the direction of what kind of tricks are currently pulled to make things fast.
> Perhaps, the X11 interface is already not the fastest compared to the standard approach, or there are some plugins in the VM which help performance but aren't included in RoarVM yet. I have never looked at the Squeak VM code myself, so I don't know a lot about what is actually done there.
>
> Best regards
> Stefan

> On 11/3/2010 6:13 AM, Stefan Marr wrote:
>> We are happy to announce, now officially, RoarVM: the first single-image
>> manycore virtual machine for Smalltalk.
>
> Congrats! That's a great step forward.
>
>> RoarVM is based on the work of David Ungar and Sam S. Adams at IBM Research.
>> The port to x86 multicore systems was done by me. They open-sourced their VM,
>> formerly know as Renaissance VM (RVM), under the Eclipse Public License [1].
>> Official announcement of the IBM source code release:
>>   http://soft.vub.ac.be/~smarr/rvm-open-source-release/
>>
>> The source code of the RoarVM has been released as open source to enable the
>> Smalltalk community to evaluate the ideas and possibly integrate them into
>> existing systems. So, the RoarVM is meant to experiment with Smalltalk systems
>> on multi- and manycore machines.
>
> Can you give us a high-level overview about what "the trick" is? I.e., which approach did you take to make this possible?
>
>> If the community does not object, we would like to occupy the
>> [hidden email] mailinglist for related discussions. So, if
>> you run into any trouble while experimenting with the RoarVM, do not hesitate
>> to report any problems and ask any questions.
>
> Sounds great. Thanks for sharing.
>
> Cheers,
>  - Andreas

>
> Hi Bert:
>
>
> On 04 Nov 2010, at 19:07, Bert Freudenberg wrote:
>
>> So it looks like you have to use a power-of-two cores?
> Yes, that is right. At the moment, the system isn't able to handle other numbers of cores.
>
>
>> And the benchmark invocation should be different if you want to actually use multiple cores. What's the magic incantation?
> The code I used to generate the numbers isn't actually in any image yet.
> I pasted it below for reference, its just a quick hack to have a parallel tinyBenchmarks version.
>
>
>> So RoarVM is about 4 times slower in sends, even more so for bytecodes. It needs 8 cores to be faster the regular interpreter on a single core. To the good news is that it can beat the old interpreter :)  But why is it so much slower than the normal interpreter?
> Well, one the one hand, we don't use stuff like the GCC label-as-value extension to have threaded-interpretation, which should help quite a bit.
> Then, the current implementation based on pthreads is quite a bit slower then our version which uses plain Unix processes.
> The GC is really not state of the art.
> And all that adds up rather quickly I suppose...
>
>
>> Btw, user interrupt didn't work on the Mac.
> Cmd+. ? Works for me ;) Well, can you be a bit more specific? In which situation did it not work?
>
>
>> And in the Squeak-4.1 image, when running on 2 or more cores Morphic gets incredibly sluggish, pretty much unusably so.
> Yes, same here. Sorry. Any hints where to start looking to fix such issues are appreciated.
>
>
>
> Best regards
> Stefan
>
>
> My tiny Benchmarks:
>> !Integer methodsFor: 'benchmarks' stamp: 'sm 10/11/2010 22:30'!
>> tinyBenchmarksParallel16Processes
>> "Report the results of running the two tiny Squeak benchmarks.
>> ar 9/10/1999: Adjusted to run at least 1 sec to get more stable results"
>> "0 tinyBenchmarks"
>> "On a 292 MHz G3 Mac: 22727272 bytecodes/sec; 984169 sends/sec"
>> "On a 400 MHz PII/Win98:  18028169 bytecodes/sec; 1081272 sends/sec"
>> | t1 t2 r n1 n2 |
>> n1 := 1.
>> [t1 := Time millisecondsToRun: [n1 benchmark].
>> t1 < 1000] whileTrue:[n1 := n1 * 2]. "Note: #benchmark's runtime is about O(n)"
>>
>> "now n1 is the value for which we do the measurement"
>> t1 := Time millisecondsToRun: [self run: #benchmark on: n1 times: 16].
>>
>> n2 := 28.
>> [t2 := Time millisecondsToRun: [r := n2 benchFib].
>> t2 < 1000] whileTrue:[n2 := n2 + 1].
>> "Note: #benchFib's runtime is about O(k^n),
>> where k is the golden number = (1 + 5 sqrt) / 2 = 1.618...."
>>
>> "now we have our target n2 and r value.
>> lets take the time for it"
>> t2 := Time millisecondsToRun: [self run: #benchFib on: n2 times: 16].
>>
>> ^ { ((n1 * 16 * 500000 * 1000) // t1). " printString, ' bytecodes/sec; ',"
>>   ((r * 16 * 1000) // t2) " printString, ' sends/sec'"
>>   }! !
>> !Integer methodsFor: 'benchmarks' stamp: 'sm 10/11/2010 22:29'!
>> run: aSymbol on: aReceiver times: nTimes
>> | mtx sig n |
>>
>> mtx := Semaphore forMutualExclusion.
>> sig := Semaphore new.
>> n := nTimes.
>>
>> nTimes timesRepeat: [
>> [ aReceiver perform: aSymbol.
>> mtx critical: [
>> n := n - 1.
>> (n == 0) ifTrue: [sig signal]]
>> ] fork
>> ].
>> sig wait.! !
>
> --
> Stefan Marr
> Software Languages Lab
> Vrije Universiteit Brussel
> Pleinlaan 2 / B-1050 Brussels / Belgium
> http://soft.vub.ac.be/~smarr
> Phone: +32 2 629 2974
> Fax:   +32 2 629 3525
>

>
> On 04.11.2010, at 21:18, Stefan Marr wrote:
>
>> Hi Bert:
>>
>> On 04 Nov 2010, at 20:20, Bert Freudenberg wrote:
>>
>>>>> So RoarVM is about 4 times slower in sends, even more so for bytecodes. It needs 8 cores to be faster the regular interpreter on a single core. To the good news is that it can beat the old interpreter :)  But why is it so much slower than the normal interpreter?
>>>> Well, one the one hand, we don't use stuff like the GCC label-as-value extension to have threaded-interpretation, which should help quite a bit.
>>>> Then, the current implementation based on pthreads is quite a bit slower then our version which uses plain Unix processes.
>
> Wait. What do you mean by "current version" vs. "our version"?
>
>>>> The GC is really not state of the art.
>>>> And all that adds up rather quickly I suppose...
>>>
>>> Hmm, that doesn't sound like it should make it 4x slower ...
>> Do you know some numbers for the switch/case-based vs. the threaded version on the standard VM?
>> How much do you typically gain by it?
>
> I don't really remember but it was well below 50%, more like 10%-20% I think.
>
>> One thing I forgot to mentioned in this context, it the object table we use.
>> That is also something which is not exactly making the VM faster.
>
> Ah, yes. That could make quite a difference. You wouldn't be calling a function for each object access though I hope?
>
>>>>> Btw, user interrupt didn't work on the Mac.
>>>> Cmd+. ? Works for me ;) Well, can you be a bit more specific? In which situation did it not work?
>>>
>>> I was doing the equivalent of
>>>
>>> SharedQueue new next
>>>
>>> and that seems not interruptable. Also, when there are multiple processes, closing the window does not quit all processes, and even ctrl-c did not quit the VM.
>>
>> Closing the window, how does that relate to processes? You mean a window inside the image?
>> Per se, processes are not really owned or managed, so there is also nobody kill processes. However, I am not sure what you are referring to exactly.  
>
> The X11 window.
>
>> Ctrl-C doesn't work, that's true.
>> However, closing the X11 window usually does the trick ;)
>
> Well, it didn't. At least not immediately. Took several seconds ad ctrl-c's untill I got my prompt back.
>
>>>>> And in the Squeak-4.1 image, when running on 2 or more cores Morphic gets incredibly sluggish, pretty much unusably so.
>>>> Yes, same here. Sorry. Any hints where to start looking to fix such issues are appreciated.
>>>
>>> Hmm. There are long freezes of many seconds and I would have no idea where to start even ...
>> Ok, several seconds, hm. That does not really sound like the GC pauses I see.
>> But I haven't used the Squeak image a lot myself on the RoarVM.
>> I was more thinking in the direction of what kind of tricks are currently pulled to make things fast.
>> Perhaps, the X11 interface is already not the fastest compared to the standard approach, or there are some plugins in the VM which help performance but aren't included in RoarVM yet. I have never looked at the Squeak VM code myself, so I don't know a lot about what is actually done there.
>>
>> Best regards
>> Stefan
>
> I guess some profiling is in order ...
>
> - Bert -
>
>