[squeak-dev] [ANN] A new scheduler + VM changes alpha-release

> i added a new changeset in mantis which deals with callbackEnter: code.
> Unfortunately, a callback handling , to work correctly needs an
> image-side changes as well.
>
> P.S. Having fun to watch how scheduler handling signals, by inspecting
> Process and selecting its 'signals' ivar :)
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>

> A quick comparion for Delays:
>
>        delay := Delay forMilliseconds: 1.
>        bag := Bag new.
>        1000 timesRepeat:[bag add: [delay wait] timeToRun].
>        bag sortedCounts
>
> on my 4-core box it yields:
>
> - with AdvancedProcessorScheduler install
>   a SortedCollection(951->2 49->1)
>
> - with Processor fallbackToOldScheduler
>  a SortedCollection(953->2 47->1)
>
> - with old VM
>   a SortedCollection(952->2 47->1 1->3)
>
> not much overhead huh? :)

> Igor Stasenko wrote:
>>
>> A quick comparion for Delays:
>>
>>       delay := Delay forMilliseconds: 1.
>>       bag := Bag new.
>>       1000 timesRepeat:[bag add: [delay wait] timeToRun].
>>       bag sortedCounts
>>
>> on my 4-core box it yields:
>>
>> - with AdvancedProcessorScheduler install
>>  a SortedCollection(951->2 49->1)
>>
>> - with Processor fallbackToOldScheduler
>>  a SortedCollection(953->2 47->1)
>>
>> - with old VM
>>  a SortedCollection(952->2 47->1 1->3)
>>
>> not much overhead huh? :)
>
> That's not exactly the kind of benchmark I was looking for (if your process
> scheduler takes milliseconds to do a switch I think we're not even close to
> the ballpark ;-) More interesting for comparison is this (requires
> closures):
>
>  semas := Array new: 10000.
>  plist := Array new: 10000.
>  1 to: semas size do:[:i| semas at: i put: Semaphore new].
>  1 to: plist size-1 do:[:i| plist at: i put: [(semas at: i) wait. (semas
> at:i+1) signal] fork].
>  [semas first signal. semas last wait] timeToRun.
>

> 2009/4/30 Andreas Raab <[hidden email]>:
>> Igor Stasenko wrote:
>>>
>>> A quick comparion for Delays:
>>>
>>>       delay := Delay forMilliseconds: 1.
>>>       bag := Bag new.
>>>       1000 timesRepeat:[bag add: [delay wait] timeToRun].
>>>       bag sortedCounts
>>>
>>> on my 4-core box it yields:
>>>
>>> - with AdvancedProcessorScheduler install
>>>  a SortedCollection(951->2 49->1)
>>>
>>> - with Processor fallbackToOldScheduler
>>>  a SortedCollection(953->2 47->1)
>>>
>>> - with old VM
>>>  a SortedCollection(952->2 47->1 1->3)
>>>
>>> not much overhead huh? :)
>>
>> That's not exactly the kind of benchmark I was looking for (if your process
>> scheduler takes milliseconds to do a switch I think we're not even close to
>> the ballpark ;-) More interesting for comparison is this (requires
>> closures):
>>
>>  semas := Array new: 10000.
>>  plist := Array new: 10000.
>>  1 to: semas size do:[:i| semas at: i put: Semaphore new].
>>  1 to: plist size-1 do:[:i| plist at: i put: [(semas at: i) wait. (semas
>> at:i+1) signal] fork].
>>  [semas first signal. semas last wait] timeToRun.
>>
>
> I not yet tried to apply changes to VMMaker with closures.
> Do you have a recipe, which base image could load latest VMMaker w/o
> problems? Is 3.10.2 ok for it?
>
>> Cheers,
>>  - Andreas
>>
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>

> Igor Stasenko wrote:
>>
>> Running this test with #fixTemps.
>> if we not count a spikes which seems just an intrusion of delays, the
>> difference about 3 times.
>> Fantastic results (bytecode interpreted scheduling works 3 times
>> slower than C code) :)
>
> Indeed. That is quite impressive since every single process switch now
> requires two process switches (one to the scheduler process and one away
> from it). That is very encouraging.
>

> This will switch between 100 processes for 5 seconds.
>
> Cheers,
>  - Andreas
>
> Andreas Raab wrote:
>>
>> Igor Stasenko wrote:
>>>
>>> A quick comparion for Delays:
>>>
>>>       delay := Delay forMilliseconds: 1.
>>>       bag := Bag new.
>>>       1000 timesRepeat:[bag add: [delay wait] timeToRun].
>>>       bag sortedCounts
>>>
>>> on my 4-core box it yields:
>>>
>>> - with AdvancedProcessorScheduler install
>>>  a SortedCollection(951->2 49->1)
>>>
>>> - with Processor fallbackToOldScheduler
>>>  a SortedCollection(953->2 47->1)
>>>
>>> - with old VM
>>>  a SortedCollection(952->2 47->1 1->3)
>>>
>>> not much overhead huh? :)
>>
>> That's not exactly the kind of benchmark I was looking for (if your
>> process scheduler takes milliseconds to do a switch I think we're not even
>> close to the ballpark ;-) More interesting for comparison is this (requires
>> closures):
>>
>>  semas := Array new: 10000.
>>  plist := Array new: 10000.
>>  1 to: semas size do:[:i| semas at: i put: Semaphore new].
>>  1 to: plist size-1 do:[:i| plist at: i put: [(semas at: i) wait. (semas
>> at:i+1) signal] fork].
>>  [semas first signal. semas last wait] timeToRun.
>>
>> Cheers,
>>  - Andreas
>>
>>
>
>
>
>
>

> 2009/4/30 Andreas Raab <[hidden email]>:
>> A better benchmark is attached. It allows adjusting the number of  
>> processes
>> and the total time the benchmark should run and answers the number of
>> process switches per second. This is our "standard" process switch
>> benchmark. Run it like here:
>>
>> 100 "processes" benchSwitch: 5. "seconds"
>>
>
> I have built a new VM with closure support (VMMaker.dtl.120).
> A changeset can be loaded to VMMaker w/o changes, except changing the
> Interpreter class redefinition.
> All is needed is to add 3 class vars to Interpreter manually:
> ProcessActionIndex InterruptedProcessIndex InterruptProcessIndex
>
> Running squeak-3.10 (no closure compiler).
>
> 100 "processes" benchSwitch: 5. "seconds"
>
> Processor fallbackToOldScheduler
>
> '1,156,135 switches/sec'
> '1,144,994 switches/sec'
> '1,147,744 switches/sec'
>
> AdvancedProcessorScheduler install
>
> '148,067 switches/sec'
> '141,211 switches/sec'
> '141,692 switches/sec'
> '148,650 switches/sec'
>
> (1156/148) ~~ 7.8 ratio
>
>
>
>> This will switch between 100 processes for 5 seconds.
>>
>> Cheers,
>>  - Andreas
>>
>> Andreas Raab wrote:
>>>
>>> Igor Stasenko wrote:
>>>>
>>>> A quick comparion for Delays:
>>>>
>>>>       delay := Delay forMilliseconds: 1.
>>>>       bag := Bag new.
>>>>       1000 timesRepeat:[bag add: [delay wait] timeToRun].
>>>>       bag sortedCounts
>>>>
>>>> on my 4-core box it yields:
>>>>
>>>> - with AdvancedProcessorScheduler install
>>>>  a SortedCollection(951->2 49->1)
>>>>
>>>> - with Processor fallbackToOldScheduler
>>>>  a SortedCollection(953->2 47->1)
>>>>
>>>> - with old VM
>>>>  a SortedCollection(952->2 47->1 1->3)
>>>>
>>>> not much overhead huh? :)
>>>
>>> That's not exactly the kind of benchmark I was looking for (if your
>>> process scheduler takes milliseconds to do a switch I think we're  
>>> not even
>>> close to the ballpark ;-) More interesting for comparison is this  
>>> (requires
>>> closures):
>>>
>>>  semas := Array new: 10000.
>>>  plist := Array new: 10000.
>>>  1 to: semas size do:[:i| semas at: i put: Semaphore new].
>>>  1 to: plist size-1 do:[:i| plist at: i put: [(semas at: i) wait.  
>>> (semas
>>> at:i+1) signal] fork].
>>>  [semas first signal. semas last wait] timeToRun.
>>>
>>> Cheers,
>>>  - Andreas
>>>
>>>
>>
>>
>>
>>
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
>

>
> Thanks
>
>
> On May 1, 2009, at 5:11 AM, Igor Stasenko wrote:
>
>> 2009/4/30 Andreas Raab <[hidden email]>:
>>> A better benchmark is attached. It allows adjusting the number of
>>> processes
>>> and the total time the benchmark should run and answers the number of
>>> process switches per second. This is our "standard" process switch
>>> benchmark. Run it like here:
>>>
>>> 100 "processes" benchSwitch: 5. "seconds"
>>>
>>
>> I have built a new VM with closure support (VMMaker.dtl.120).
>> A changeset can be loaded to VMMaker w/o changes, except changing the
>> Interpreter class redefinition.
>> All is needed is to add 3 class vars to Interpreter manually:
>> ProcessActionIndex InterruptedProcessIndex InterruptProcessIndex
>>
>> Running squeak-3.10 (no closure compiler).
>>
>> 100 "processes" benchSwitch: 5. "seconds"
>>
>> Processor fallbackToOldScheduler
>>
>> '1,156,135 switches/sec'
>> '1,144,994 switches/sec'
>> '1,147,744 switches/sec'
>>
>> AdvancedProcessorScheduler install
>>
>> '148,067 switches/sec'
>> '141,211 switches/sec'
>> '141,692 switches/sec'
>> '148,650 switches/sec'
>>
>> (1156/148) ~~ 7.8 ratio
>>
>>
>>
>>> This will switch between 100 processes for 5 seconds.
>>>
>>> Cheers,
>>>  - Andreas
>>>
>>> Andreas Raab wrote:
>>>>
>>>> Igor Stasenko wrote:
>>>>>
>>>>> A quick comparion for Delays:
>>>>>
>>>>>       delay := Delay forMilliseconds: 1.
>>>>>       bag := Bag new.
>>>>>       1000 timesRepeat:[bag add: [delay wait] timeToRun].
>>>>>       bag sortedCounts
>>>>>
>>>>> on my 4-core box it yields:
>>>>>
>>>>> - with AdvancedProcessorScheduler install
>>>>>  a SortedCollection(951->2 49->1)
>>>>>
>>>>> - with Processor fallbackToOldScheduler
>>>>>  a SortedCollection(953->2 47->1)
>>>>>
>>>>> - with old VM
>>>>>  a SortedCollection(952->2 47->1 1->3)
>>>>>
>>>>> not much overhead huh? :)
>>>>
>>>> That's not exactly the kind of benchmark I was looking for (if your
>>>> process scheduler takes milliseconds to do a switch I think we're
>>>> not even
>>>> close to the ballpark ;-) More interesting for comparison is this
>>>> (requires
>>>> closures):
>>>>
>>>>  semas := Array new: 10000.
>>>>  plist := Array new: 10000.
>>>>  1 to: semas size do:[:i| semas at: i put: Semaphore new].
>>>>  1 to: plist size-1 do:[:i| plist at: i put: [(semas at: i) wait.
>>>> (semas
>>>> at:i+1) signal] fork].
>>>>  [semas first signal. semas last wait] timeToRun.
>>>>
>>>> Cheers,
>>>>  - Andreas
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko AKA sig.
>>
>>
>
>

> Igor Stasenko wrote:
>>
>> I have built a new VM with closure support (VMMaker.dtl.120).
>> A changeset can be loaded to VMMaker w/o changes, except changing the
>> Interpreter class redefinition.
>> All is needed is to add 3 class vars to Interpreter manually:
>> ProcessActionIndex InterruptedProcessIndex InterruptProcessIndex
>
> Hm ... I compiled a VM with your changes but I can't recreate your results.
> My results are more in the range of 15:1 to 20:1 in a before-after
> comparison. Are you using the code from
> http://bugs.squeak.org/view.php?id=7345 or something more elaborate by now?
>